Was just down there. There's a great Cuban restaurant right off Mallory Square (El Meson de Pepe) with live music. Wife and I spent a great evening there eating, dancing, and drinking Rum Runners.
http://cuban-food-usa.com/keys_meson_pepe.html
If you like to sail, you can rent windsurfer's & Hobie Getaways down near the Casa Marina on the South end of the island. Great snorkling/fishing everywhere.
Had a great island bar-b-que dinner on the Hard Rock 90ft sailing cruise - with a live steel drum band for 3 hours one night - it was $$ but worth it.
Early in our stay, it was tough to find a cab back to our hotel one night (Casa Marina) and so we took a pedal cab. Saw everything up and down Duvall St., and went to a half a dozen other nice restaurants. Good way to get a feel for the place. (If you're not with wife/girlfriend, could be weird, I suppose!) There are bars with live rock/country/creole up and down the street, starting with Sloppy Joe's.
If you're there on work or with buddies, Duvall is the place to be at night. And no matter who you're with, when you get up the next day, get on a boat and do
something! Was on the best beach on the island, and it was pretty lousy (no waves, very little sand, not much sea life). Need a boat to get to the sand bars & snorkling/scuba areas.