If they did not fall out there would be no need for a connector holder.
When the spark plug is used as a dummy plug not intended to fire, there is rarely a plug wire attached and connector holders will not work without a connector..
And yes even properly torqued plugs fall out more often than you might expect.
