I'm not trying to be nonchalant about hurricanes but personally I don't worry about it too much. Yes, if a level 5 makes a direct hit on the BVI's I'd rather not be there but...

You are only exposing yourself to the risk for the days you are there and people live there year round their entire lives.
A lot of hurricanes are still building that far out. Many of the level 4 or 5 hurricanes that hit the US were level 1 or two when they passed by the BVI's.
The smaller the hurricane is and the further away the less of an issue it is.
There is plenty of warning before it gets there.
The charter outfit will call you and tell you what they want you to do if they deem it serious.
You probably won't be able to evacuate but the charter outfit will help you find a safe place to stay.

We like May the best - no north swell, not crowded, no hurricanes and everything is open but this year our schedule dictated we'll be there June 27 - July 11. Overall, I think we're taking a low risk.

We did experience a level 2 hurricane one September that passed 75 miles south of St. Croix. We used two lines to a mooring in Cane Garden bay. During the worst of it we may have seen 50 knot gusts and lots of rain but it wasn't what I'd call scary. On the second morning I finally got to a weather map and saw a huge tail of clouds that were just dumping rain for three days on the BVI. It almost filled the dingy! On the third day we and other boats were on our cloudy way again. When we turned the boat in they said they had try to call us, I told them we had the phone plugged-in but never got the call. They didn't seem upset so I'm guessing they wouldn't have called us back to the base they were just looking out for us. Anyway, that didn't ruin the vacation for us but it might for some.

Edit to add: None of the above really matters. What really matters is if the Admiral-to-be is up for it and how you feel about that. If you can't compromise on this you will know what a force 5 hurricane is all about! I still can't really sail a monohull to it's capacity because she hates to lean but we've compromised on that.