It depends on weather at the time of your visit, the location of the house and your tolerance. Aside from staying at Caneel (where I never visited the pool) we always rented houses with pools.

In late January, the pool at one house was absolutely frigid and I dipped only a toe in. At the same house, one year later, we used the pool every day and took our baby in as well (it was that much warmer). Nothing changed except the weather was consistently warmer for the second week so the water was water and we were warmer.

Location on the island matters as well. We rented a house during the last week of January/first week of February in the Maria Bluff neighborhood that got full sun all day and the pool was delightful.

Our other winter visits were hit and miss pool-wise when it came to water temps but in general, we used the pools more often than not.

My family is more willing to use the pool because it is a novelty. If the water is close to 80 degrees, we get in. My close friend that travels with us needs it to be at least 85 degrees before she considers swimming.