If you are exploring for a couple of days make sure you do the 99 steps tour (knees permitting). Have have a look on your left, as you ascend, at the remnants of Hotel 1829. Stayed there in the mid-90s when it included a restaurent and again in 2009. A truly unique building and hopefully the current onwers will get it restored and operating again. The amber waterfall in its courtyard was a nice feature. https://stthomassource.com/content/2018/03/04/hurricane-repairs-underway-at-hotel-1829/

a bit farther up the hill is Galleon House....now if your are looking for a workout on the stairs then THAT is the place to walk to and it has a pool as your reward for making it up all of the steps.