Olga's was our go to as well. We are going to stay one night at The Mary Anne in February. It looks nice and has a view of the harbor but I don't think it is walking distance to the ferry in the AM.
Cheers,