Yes Muslim children should be excused from swimming on religious objections. All religious objections should be recognized, assuming they do not cause harm to anyone. We allow people to refuse immunizations for their children based on religious objections, so it seems wrong to even question a simple request from a Muslim family for their children to be excused from swimming.
This country was founded on religious freedom. The state cannot mandate a law or rule infringing on the freedom of religion. This applies across all walks of life. We are a religiously free nation, and so we have to respect that. We are a nation of many religions and the laws apply to them all.
It is incredible what some people pass off as swimsuits today. A person would be embarrassed to wear that outfit to school or shopping, but as soon as you call it a bathing suit, it's okay to run around in next to nothing. It's more important to allow children to make a different choice than to order them to display their bodies.
I believe that Muslim children should be excused from swimming lessons if their parents choose to exclude them due to religious reasons. I think that all people should have the right to exercise their religious freedoms in the way that best suits them. If parents have an issue with immodest dress required for swim instruction, I think that is their right.