You can pre-ticket for the ferry, but you really can't consider that a reservation, it is only a passing flirtation with reality. Round trip tickets are of the same value--just don't go there. You pay for tickets on the next ferry going where you want to go, leaving the ferry terminal where you are standing with your entire party.
Your best chance at the 8:30 ferry is to put the Fear of Morticia into your party of six to get their luggage and get outside baggage claim to the taxi-line as fast as possible. It's only about 10 minutes drive time to the CA ferry dock, but you need to get your group loaded and your driver convinced to not wait for others ( which makes him more money). Grease Palms 101. Convince the driver you and yours are all he needs to make bank on this run.
If you make it to the ferry landing by 8:15, you are probably good to go on that first ferry back to Tortola ( because it probably won't leave until 8:45 or later).
Now, since things can always go sideways and pear-shaped ( someone's luggage didn't make it on the plane), remember that since there are no reservations, if the 8:30 doesn't work, the 10:00am will. You truly can check your luggage at the taxi-stand and go find breakfast, wander around Frenchtown, breathe some tropical air, slow the heck down, and be ready to slide into your trip with a smile on your face.