The reverse is true as well. If the swimmer is not visible, it is not easy for a boat to avoid a swimmer. While it may be technically correct that a boat must avoid a swimmer, the swimmer always loses the battle if there is contact. I would assume that most boaters make assumptions as to what would be considered a normal swimming/snorkelling area. You don't typically see swimmers 400 yards offshore.