Another vote for the Loudspeaker Design Cookbook. It's not hard, but it can get as complicated as you make it to be. Once you begin to understand and start such a hobby, like anything, you can end up in the never ending quest of finding the perfect driver, enclosure, and sound. You need at least the three basic Theil-Small parameters of a driver - Fs, Vas, and Qts - to calculate the optimal enclosure volume. There are sealed, ported, and bandpass enclosures and each has its own sound characteristics and tradeoffs. And that would only be the beginning, as you'll also be searching for the ideal midrange/tweeter combination and crossover design.
To simplify things, I'd start by working with the best drivers and designs first. You may also wish to consider preengineered designs from a place such as
zalytron. Google Joe D'Appolito and the MTM arrangement. Twenty years ago, some friends and I listened to a new J D'Appolito MTM design called the Aria 5 using 2 Focal Kevlar inverted dome tweeters and four Focal 5" midbass drivers. Combined with some separately powered Vieta subwoofers, we were absolutely blown away. I still remember that day. The music was so clear, 3 dimensional, I felt like a scrub that had just had his first bite of prime USDA ribeye after having eaten rump roast his whole life.
It also pioneered a new design in the speaker industry that still continues to this day.