Okay that changes things - good job on getting a written contract. Did you spell out the terms of payment schedule? It's not uncommon to have 25% up front, 25% at 50% completion and the balance upon satisfactory completion. If the work is really shoddy and you can document it, you could successfully withhold the final payment pending resolution of outstanding/unsatisfactory items.
You may want to check with the manufacturer of the flooring you used (Pergo?, who was it?) and see if you can get a copy of their standard specifications and/or manufacturer's installation instructions. If the contractor clearly didn't follow the manufacturer's instructions, you probably have a very good case to show they didn't follow accepted industry standards of care and prove sub-standard work. Just be thankful that it's flooring and not a life-safety item. You wouldn't believe the stuff some of these guys try to skimp on. . . Anyway I digress - try to get the manufacturer's data. In the future, stipulate in the contract that all work "shall be performed in full accordance with accepted industry standards, manufacturer's specifications and installation instructions and applicable regulations or ordinances" (or words to that effect). There's a freebie for ya.
If the contractor included removal of waste debris as part of his bid and didn't, you should be entitled to AT LEAST a credit for that. You're fine there.
Litigation sucks, but it sounds like you might have to drag this one through the courts in order to get resolution. I'd talk to an attorney and as has been stated, make sure there aren't any liens placed on your property in the meantime (or if there are, find out all the details and be prepared for a long fight). Document everything, make a timeline/chronological history of all communications, payments, activity, etc.
If he is a licensed contractor, you can file a grievance with the state regulatory board. The downside is that even if he loses this and loses big, he'll simply be back in business under a different name two weeks later. That probably won't happen on a job of this scope (pretty small), but it's the oldest trick in the book and a lot of the sleazier contractors keep it in their back pocket as "justification" to engage in really distasteful practices.
Best of luck with this - keep us all posted on how it turns out - be curious to hear.