Yeah, to avoid the cruise ship crowd I'd recommend going between 8-10:30 or after 3:30. If you're in Leverick and driving you can see if there's a cruise ship in North Sound or Spanishtown and time your trip accordingly. Cruisers also come over from Tortola, but in smaller numbers in my experience.

Don't forget your national park card/receipt on the boat, or you'll have to pay admisson (again) to the Baths at the top of the hill.