There are several points along the eastern shore of Great Harbour on Peter where I have seen yachts anchored though I have never anchored there and can't provide any personal experience. Looking a the charts and google maps it appears there is quite a bit of sand once you are off the reef that is up against the shoreline in what would be 10-20 feet of water. I'm sure someone here has some experience anchoring along that area and could provide some extra reassurance though.

I do love the seclusion and holding along the eastern shore of Little Harbour though as an alternative. I have day anchored in Deadman's Bay but it is likely not ideal for an overnight stay dependent on the prevailing wind at the time.