Here is the deal - the factory CDI box allows the plugs to continue to fire after the ignition is turned off to burn any remaining fuel dumped in by the CIS system. (you know that CIS is always spraying fuel into every cylinder when the fuel pumps are running). So, on the OEM setup, when you turn the key off, the pumps shut down but the CDI will deliver spark if the distributor signals it for a few more seconds - that's why these cars seem to 'run on' for a second or 2 when you turn the key off. They're burning the last bit of fuel sprayed in. The Crane unit has no capability to do this (at least the Crane units that I'm familiar with - HI-6, HI-6S, HI-6N), so you're getting the pop when this fuel gets lit by some other hot component (prolly the exhaust as you've surmized).
With a well tuned EFI system, the pulsewidth should be 'right', and there shouldn't be enough fuel 'laying around' to create any kind of a burp. It might be possible to have 1 or 2 cylinders woth of fuel in there if your setup is a batch fire system (the ECU fires all of the injectors at the same time for each revolution and each cylinder's intake stroke pulls the fuel in) and thereby cause a burp. If you have sequential EFI, my only input is that your setup is not tuned correctly or optimally.