When you go to someone's house, and they do something rude, you don't go back. Everyone has their own threshold, and these things work themselves out over time.
If the guy really is a chainsmoking pig, wallowing in a filthy, stinky house, then ultimately he'll end up alone there buried in a disgusting pile of ashes and butts.
If you're the one who's oversensitive and everyone else doesn't mind, then you'll stay away and miss some good parties.
But you really can't do whatever you want, even in your own house. Think of all the things you could do a a party that might offend your guests. You don't do them, because you want to be a good host, and not offend your guests.
Bryan, do you enjoy testing your guests' tolerance?
What other offensive things to you do because it's YOUR HOUSE? Maybe
Edited - no need for such a graphic example? Sure, it's rude, but it's your house, right?