Downshifting and loading the gearbox on track is NOT 100% about ego. It has its place in high performance driving, but like anything it's all about doing it in the right place at the right time in the right amount.
There's a turn at my local racetrack. It's a wide 180+ degree turn that transitions from entering on a fast downhill to existing climbing a hill.
If one trail brakes into it in 3rd (as my car is geared) and drops it to 2nd just as you apex, it wags the tail barely, locks the LSD, and you launch up the hill with the throttle to the floor.
I know guys who downshift to 2nd later in that turn and complain about how they push out to track out and have to lift because they aren't rotated enough.
But that's at the track. On the street, downshifting and using the tranny to decel is just to feel cool and because the engine sounds neat. That's ego.
I personally prefer avoiding absolutes when we start talking racecraft. There is usually at least one example that can be found to challenge the "all the time" claims.
