As someone who has spent many weeks on a charter sailboat over the years, and who loves sailing smaller boats such as sunfish, lasers, hobies, I'd say that yes this is a pretty unworkable/challenging situation (hauling/towing another smaller sailcraft) unless you're aboard something in the 60'+ range (multihull). The only thing I'd haul that also sails, would be a windsurfer or kiteboard (which I've done many times/in fact always do) but that's not really what you're considering. However, what I've done on many charter vacations is find smaller boats to sail at different stops, though that's dependent upon where you are and what's available and I suspect is more challenging these days in BVI's post Irma.