Unfortunately, the definition of "freedom of speech" has been stretched and toyed with far too much in the Internet era. People who hold it up as their shield believe that it means "I can say whatever I want and no one can do anything about it," when in point of fact it simply means the government can't stop you from expressing yourself.
False accusations fall under slander and libel, and those are indeed crimes. Sadly, there isn't much that can be done to enforce it, as the entirety of Tumblr will occasionally decide that a TV writer beats his wife simply because they think he wrote a misogynist joke last week. Regardless, slander and libel are not protected under freedom of speech, and the First Amendment exists to give us the ability to critique our government without fear of retribution, not spout off actual traceable lies without getting ourselves checked.
False, negative rumors against celebrities should be considered illegal. The integrity of most of the people who create false rumors is usually very low, and to defame someone's character when done intentionally, should always be considered illegal and carry a penalty to those who commit these acts in order to prevent them and others from doing it any more.
Legally, you can give any opinion you want, as long as you make it clear that it is an opinion. If you want to state something as fact, then you had either be ready to back it up or face criminal charges.
Today, a false claim can cost you your career, land you behind bars or even get you killed. Imagine if it was acceptable to claim someone is a pedophile, or a terrorist. Imagine what it would do to your life if you were falsely accused of being one. Then you would be forced to publicly defend those charges and still some would believe the rumors. Someone would have to pay for your losses and that should/would be the person who made them in the first place.
False rumors and negative claims about celebrities are already illegal if the individual can show damages of some sort. If false claims are made and the effect caused damage to personal relationships, the person's career or other financial matters, there are already laws in place giving recourse for the affected individual.
Negative rumors against celebrities should not be illegal. Normal people have negative rumors the surround them, ruining their reputations, with no way to remove that stigma. Celebrities have thousands and millions of dollars to put towards lawyers who can sue for defamation. This alone is enough to scare away the negative rumors.
Everyone in America and most 1st world countries have the freedom of speech. If negative rumours about normal people aren't illegal, there should be no special exceptions for those who are well-known. Now if they are physical and possible threats against the specific person, that opens up a completely different window. That's already a law. (In Canada at least.)
People have freedom of speech. It is fine for them to say what they think. Even if what they think is not kind or true, it is their choice if they want to say anything about it. It is not the government's place to stop them from expressing what they feel.