Flint, leave it for a couple years (since you are rich
), change your address and use that as your primary res and get the 500k tax break between you and your wife. You primary will become your vacation home.
Hate to tell you this, but there is a reason its called Investment property. Its there to make money, and making money need to be taxed.