Little harbor on Yost is well protected in a N swell. The only place really affected on Yost is White Bay. The nice thing if there is such a thing is the N swell is very accurately forecast. Take a look 2 days out and make new plans if it's in the forecast. I would go counterclockwise if it's in the forecast for the first two days of your trip.
G