If you can swing it, I suggest walking over to Trellis Bay when you land and taking the free water taxi to Scrub Island for the night. And yes, Cane Garden is doable for sure!
Way better waking up on a beach! Enjoy