Optometrist chiming in....

I were both plastic and glass sunglasses. While the glass will do much better, it's not scratch proof. That said, wind blown sand will not scratch a lens. Having stood behind the KLM 747 with my plastic lenses on takeoff, not a single scratch was had. Now, some plastic glass lenses are polycarbonate and so while they are impact resistant, they are more likely to pick up fine scratches.

It's more than likely that you may be wiping the glasses when the tiny sand particles are still in them and in essence, grinding the scratches into the lens.

So to minimize that, ALWAYS clean your lenses under running water, mild soap and then dry the lenses. And consider glass lenses