[notmytarga]

I could repost Red's lovely service panel picture again for this question to him. Why did you route the three phase (large) conductors to the top of the panel/box - could the breaker frame be inverted to make your install Even Cleaner?? Or does the box require the orientation to remain this way??

If the problem is not loose main connections this box needs to be stripped down and cleaned up or better, replaced. You referred to it as a new box - but its pretty old and crowded. Time for a new box. My bet is that the box connections are deteriorated, pulling the breakers will show the corrosion. I replaced DIY a pool subpanel that went south after 35 years - it was creating lots of heat from poor connections behind the breakers.