I can't say I think the ferry reliability has improved over the last few years. Given that, it looks like you've given yourself plenty of time. It might be a good idea to see what your alternate would be just in case that ferry doesn't run.

Life involves risks, take some prudent ones (NOT with the BVI ferries)!