While there are some sports that are more violent than others and more debatable (hockey, American football) many sports like basketball, baseball, track, swimming, etc are not. There can certainly be random acts of violence in the heat of the moment, but with the exception of hockey, it's not the norm and is usually punished.
No, organized sport is not legalized violence. All sports take a lot of measures to protect the athletes, be it wearing protective gear, or calling foul on certain kinds of hits that could cause too much damage to the players. Even combat sports take measures against things like head kicks or downward elbows because these things are shown to be too harmful.