Unfortunately, if there is any S component to the wind, the southern anchorages on Peter are not comfortable. If you don't mind some rolling, White Bay or South Bay would be safe, but again, not comfortable.

The only exception might be Key Bay, but it's a very small anchorage. I've been all settled in there, nice and comfy, all by myself, when some [censored]-clown in a ginormous cat has anchored right on top of me. And it's a lee shore, so a bit daunting.

You might head to Benures, just across the way. Or, if you have a crapload of chain, Great Harbor is always a safe bet. (And there might even be a mooring available.)