My suggestion, stay two nights and do the hike one day and the next dive as Scoob suggested. Don't rush the trip, enjoy a couple dinners on island and enjoy the peace that the island offers. Very different experience to SXM.

Also I would fly but not sure how that might adjust your diving schedule or equipment needed that you would take. Not an active diver.