This type problem is what I referred to earlier about plug not firing under compression and the use of a known good plug. To me known good plug is one that is working in another engine.
I have seen new out of the box plugs to fail. This is why you need a spark gap type tester to make it not the ignition coil. Those little glow (neon) type are not a good indication of the spark strength, they just a present of a spark.
I had one engine on air compressor that would be fine at first start up with empty air tank, fill the tank, and throttle down valve engages. Then when time came for it to throttle back up die. The plug simply could not fire under the extra compression load.
I also have seen engines be knocking at part or full throttle like a rod was loose or carbon on top of the piston to be only a failing plug. I also replaced quite a few NGK CMR6H and CMR7H plugs had failed this way. Note: these are 10mm plugs.
With experience comes wisdom is old adage but true many times that why many of us old techs are so good at repairs. We build on our previous experiences and like you have probably made the same mistakes, we just learn not to repeat things that don't work over and over.
Personally I have learn to do my non-invasive tests first as they can save a lot time and expense. I am firm believer in the KISS system.
