You have been given some great suggestions here Mark.

One place my 2 year old nephew really liked was Spring Bay National Park on Virgin Gorda. There are huge boulders there that a 5 year old and his Dad would have a ball climbing all over. There's also a swing set, a picnic area (out of the sun) and some great little nooks and crannies to discover.

This is a path less traveled than the baths and the little ones love it! Highly recommended. Accessible by land or by sea. The path in by land is a lot easier to negotiate than the one at the baths.