Ok...so to help anyone else who may have this problem:
The heat shield does just pull off, it was surprisingly tight (thanks Big Dav)
My O2 sensor was "welded" in there. I tried penetrating oil, heat, oil, heat, big wrench more oil and heat. Nothing budged it and the wrench was starting to round the corners off so I took an autobody air saw and cut the top off so I could put a 6pt socket on there. It finally came off with a big cheater and the threads are kinda messed up. I hope I can chase the bung in the exhaust with a tap.
