Personally, I think you are trying to cram too much into one week, but perhaps that's just me.
I think you are missing the most beautiful beach on the island: Anse Marcel. The restaurant there is fabulous. There's another beach bar with simply marvelous food - Dreams in Baie Nettle. The food may be slightly better than at Anse Marcel, but the beach isn't as nice. Both are not-to-be-missed locations IMO.
And sailing on the
Passaat Schooner is one of my most favorite island activities.
Enjoy!