St. John Inn is about a 4 minute walk to town and about a 5 minute walk back due to an incline , so it's an ideal spot for doing Cruz Bay.
But I recommend renting a car to see all of what the island offers.