My understanding is that early withdrawal from an IRA is ALWAYS due the tax that was deferred at the time it was paid into the IRA. Additionally, if you're withdrawing from the IRA early, there's an extra Federal penalty (think it's 10%) involved when taking out more than $10,000, and this is the penalty that's waived if you're using the funds to purchase your first home.
I'm not sure that the circumstances you describe would meet "first home" criteria. I'd seek professional advice from someone that knows what they're talking about..
