"And I can't believe there has been no "Melbourne is better" spam yet either"
Well it has to be believable to be worth spamming
If you've got a spare day it's worth getting out of the cbd and doing a drive down south (i'm biassed living down this way). You can see some great ocean scenery, valleys, waterfalls etc. Hire something like an mx5 from a local company & if you're interested I'll send you a little map with a fun route or meet up on the day for a drive. You won't see any tourist buses but will see some lovely sights & quite a bit of wildlife.
Around the cbd I'd recommend getting hold of a brochure that's available down at circular quay (and probably in all of the hotels) titled 'walking around sydney' or similar. It's all pretty easy stuff & gets you off the beaten path, while still being in the thick of things. If you don't want to do a guided harbour tour, the ferry to watsons bay is quite nice, with doyles on the water offering a casual seafood lunch. It's a short walk up the hill to a coastal track with some nice harbour & ocean views. If you feel like a lengthy walk (not sure how the little one would go), you can pretty much walk all the way to the eastern suburbs beaches. A shorter stroll along the harbour foreshore from Watsons bay should give you some semi-interesting architecture of the housing variety to look at.
If you've never done a twilight or dawn zoo experience before, Taronga Park Zoo offers (or used to ) a feeding type tour that's a bit different. The zoo's not big by international standards but it is set on the harbour which has its own benefits.
And I second prasits thai - very tasty.
Edit - if you're in sydney for a while it may be worth looking at apartments if you've after a non hotel experience. There are plenty around the harbour with great views & you don't need to worry about the overly 'casual' hotel service that way.
If you're into japanese food I'd recommend Azumas but only for dinner as the lunch feel is a bit 'suity'. Tetsuyas is also worth a look for a different take on japanese influenced french. Sounds strange but it's quite nice.