I wouldn't get overly carried away trying to avoid the popular beaches just because it's the weekend. Sundays tend to be more of a local residents' beach day than Saturday, and this time of year, with fewer tourists on the island, I doubt any of the beaches will be overrun with people.

Jazzgal's recommendation of Anse Marcel beach is spot on, and is exactly what you're asking for. Even though the restaurant there is upscale, all beachside restaurants are casual, and as long as you have a coverup on over your swimsuit, you'll be appropriately dressed.