More important than output in a subwoofer is response. Small enclosures roll off quickly which can leave the bottom end pretty weak. If bass is important to you, this is a deal breaker as the lowest notes can often have the biggest (musical) impact. For many however, you're right, a small sub in a small enclosure will do a nice job.
But we are talking about real hifi here...
