Don't worry, it's great! I don't think anyone is trying to scare you. Saba is awesome. But since you're going that far and you're not a diver you might want to just check out some options.
St. Barths, St. Maarten, Anguilla, St. Kitts, and Nevis are all about +- 30 miles from Saba. 10 minute flight. You can fly easily between every island.
www.winair.com is a good start but there are others.
Take an air shuttle (worth the entire trip all by itself due to the landing) to St. Barths, rent a car, and get a cheeseburger at Le Select in Gustavia. AKA the "Cheeseburger in Paradise." Then drive your car to the roundabout above the airport and watch some planes land. Drive to Le Petit Anse and look off the overlook. Pretty cool.
Take a shuttle to Anguilla and go to the Sunshine Shack in Rendezvous Bay. Also go to Shoal Bay on the north-east side of the island. One of the prettiest beaches.
Go to Phillipsburg, St. Maarten and sit at the Sunset beach bar and watch a few planes land. You can drive over to the Dutch side of the island for some fun but I don't know much about that. Someone here does.
Shoot a round of Golf at the Four Seasons in Nevis. Watch for ball-thief monkeys. Go to "Sunshine's" bar and order the "killer bee" drink. Don't drink more than about two of them. Eat some conch fritters.
Go to St. Kitts and play with some more monkeys. Nevermind, don't touch the monkeys. Go to the Shipwreck Beach bar and "Mongoose" bar in South Friars bay. Mongoose isn't much of a bar it's just an old man in a shack with a cooler full of beer and it's awesome if you're into that kind of stuff.
Last March-April St. Maarten and St. Barths were pretty trashed from the hurricanes. Anguilla not too bad. St. Kitts/Nevis not quite as bad either but still dinged up a little.
One other option is to have an amazing time in Saba! Totally doable.