I had the same thought at around 35, after a promotion. I'd been racking up the hours and saving/investing 50% of my take-home. I decided that while I was preparing for a comfortable life down the line, I wasn't having much fun right then.
Oddly, for some people, my idea of fun was to start running marathons.
But yeah, I also scaled back on the long hours, live-for-tomorrow funding and started traveling, (a few summers we went away literally every weekend), got a more comfortable place, nicer cars. I'm still living well below my means, so no debt stress, but more relaxed.
I don't regret it. At my job, we have "cohorts" - sort of like your class in high school, people all brought into the company the same year. A couple of my cohort have been promoted ahead of me, but they also complain of the stress and often wonder why I look younger than them, and about 60% are gone. People tell me I must love my job, but the truth is simply like my life more, now that it's more well-rounded.