There's a ferry that runs out of French Cul de Sac during the day for about $10-15 per person round trip. The road to Cul de Sac splits off at the roundabout between Orient Bay and Grand Case. That road will take you straight to the ferry dock, where there is plenty of parking.

If you are more adventurous, you could rent kayaks or stand up paddleboards from Caribbean Paddling right there at the dock and paddle over. Baie du Cul de Sac is well protected and very calm (and beautiful).