I think at this point, even if Emerald Beach was a 5 star resort, you would be sleeping with one eye open. It's in your head. I would be the same. Really no reason to stress over it. Emerald checks the convenience box but it's not like you are missing out on some bucket list place. Just enjoy Secret Harbour with a clear mind, drink in hand and hopefully an amazing BVI experience to reflect on!