Once you reach a certain level, you just need to add other sporting activities to your swimming sessions for overall fitness.  

Muscle-building exercise (ie weight training) will strengthen and tone your muscles, while endurance training (walking, jogging, cycling) will help build stamina, improve your breathing and strengthen your heart.

Some activities, like racket and ball sports, or dance, will also help develop your co-ordination and balance, helping to harmonise your movements in the pool.