I've been twice, and love it.
I enjoy taking a few hours hiking around Videy island - you can walk to the ferry dock from wherever you're staying, and it's about a 5-minute ferry ride. I like modern art, and Richard Serra has a piece there.
https://visitreykjavik.is/videy-island
In Reykjadalur, east of Reykjavik and west of Selfoss, there's a nice hike you can do up to a hot springs river. Bring a swimsuit and lunch and hang out in the water for a while.
I'll certainly go back eventually.