Over the last 25 years I have accumulated well over 80, probably closer to 100 HDPE's on my car. I have never felt the need for track day insurance. I feel much safer on track than on the street, with everyone on the same page, going the same direction, no driveways or cross streets, and clearly enforced rules. It just doesn't get any safer than that.
You control your own destiny out there. You go at whatever pace you like. No one can make you go any faster than you are comfortable. The only real danger, which many have already alluded to, is you - if you let your ego take control, if you exercise poor judgement, then you have no one to blame but yourself. And, well, going in, you really, really need to have a brutally honest reckoning of "you", and then decide if you think you might "need" track day insurance. And, well, if the answer to that is an honest "yes", you might be better off staying home.