Sounds like your skin is drying out from the chlorine and breaking out because of it. You can always use cocoa butter every night and morning to help moisten your skin and keep it from drying out. It is gentle on sensitive skin, also vitamin E is good to.
As for the break outs, wash with a gentler soap and make sure that you rinse the chlorine off right after you swime. Hand made ones with chamomile should help, also look for something that has Eucalyptus. For really bad break out spots, you can take a wet tea back of chamomile and set on the spots, as its an anti inflammatory and should help, or drinking the tea once a day at least should help.
I also suggest talking to your doctor about something medical as well. When looking at Natural Remedies, its also good to back it with medical care as well. Its all about what combo works good for your body. Your doctor could probably give you an ointment that can calm the rash quicker than any natural Remedy.