They say that every action starts with an idea. And ideas are brought to life first in speech. If we consistently talk a certain way and hear that way all the time, then we begin to take our talk for reality.
We also begin to create more of that reality around us. So talking about hating particular groups and doing violence is almost as bad as the activity itself.
No, hate speech alone does not lead to crimes and violent actions, because many people express opinions without acting on them. It is important that people feel they can discuss their opinions. Allowing freedom of expression may even prevent some offenses, because people feel they can talk about their opinions, rather than only acting on them.