Spandex allows people to wear athletic clothes that do not chafe. It allows comfort and stretch in casual clothing. Instead of focusing on the visual appeal for a few, it is more important to think of the comfort of a person who wishes to wear it. This is more important than being scenery for someone else.
Spandex, as a material, is wonderful. It molds my socks to my calves, making droopy socks a thing of the past. Only a bit of spandex in my shirt creates some stretch while allowing the material to retain its shape. On a healthy-weight person, a spandex swimsuit can be a pleasant sight indeed. But moderation and common sense are necessary, too. Spandex shouldn't be the primary material on clothing you wear all day, such as underwear, and people who are overweight shouldn't expect to look good in spandex clothing.
Most eye-googling men and women like spandex for aesthetic reasons, but many like spandex for athletic reasons. Spandex is much better for reducing chafing. As a runner, cotton shorts with loose material can rub one the wrong way. Plus, while exercising clothing with excess material can be a plain nuisance by constantly getting in the way.
Having said that, I'd like to reiterate that spandex is good for exercise. It's not good to wear when going to your kid's school functions, work, or attending dinner parties. Just like a Speedo, there is a time and a place for spandex and meeting in-laws for the first time isn't one of them:)
Being a woman it is important for me to wear breathable type fabric. So I tend to stay away from fabric that doesn't breath, spandex is such a fabric. The irony is that spandex is used in many types exercise clothing I guess because of the fact that its so stretchable. But than so is cotton knit which is also comfortable and breathable.