I recommend that you get in touch with Alex Varlack. He is doing tours by boat. Ask for Alex at Neptune's. You can look him up on Facebook as "Anegada Adventures".

Answering your original question: Yes, you can get there by dinghy. If the wind is light it won't be bad. There are a lot of shallow spots and reefs between Setting Point and the east end. Stay fairly far off the island, where you can see that the water has adequate depth.

If there is the normal late November wind, then don't do it by dinghy. It will be a miserable trip back.