I used to wear only upper-knee length swim trunks. Then I bought some that were way shorter just to try. Turns out nobody cares because we all expect to see a lot of you at a pool. Just do whats comfortable. Maybe try something a bit shorter to see if you like it when in the water.