I was tooling around the Inner West of Sydney recently, and decided to take a peek at Cabarita Pool. Fortunately there was a great viewing area on the outside, cos I didn't have my swimming stuff with me (no, I don't ALWAYS carry it around....note to self: point to consider!)
There was one lone lap swimmer, and it's true it was a fairly windy day, but the board at the entrance said the water temp was 26 deg. Not bad at all.
To get to the pool, you need to enter through Cabarita Park, on Hen and Chicken Bay, part of Sydney Harbour/Parramatta River.