We always try and start mid-week for the same reason above.
1. cheaper flights
2. less crowded anchorages - The charter boat flow is very predictable and pretty much do the same counter clockwise loop. starting mid-week means you're not bunched up with the majority of people starting on the weekend.
3. much better ratio of Charter company dock-hands, service people etc. to patron