It’s a well protected anchorage and it’s very rare for a south swell to make it uncomfortable.
And yes to answer your question the swel would affect everyone in the anchorage