Yes, restricting evangelism significantly contravenes freedom of speech laws, because people should be free to say as much as they want to. Evangelism is not bad, simply because a lot of people agree with it. To the contrary, because evangelism is popular is a sign that it is good. People should be allowed to say what they want to, as much as they want to, with whomever they want to.
Although evangelism is generally associated with money-grubbing "megachurches" and preaching, to non-religious and liberal Christians, a doctrine of hate, that does not necessarily make it the case for every evangelical church. It is only those churches that do in fact preach doctrine of explicit hatred - against other religions, races, or genders - that some form of action should be taken, but not their right to preach or practice, simply taking away their status as a tax-exempt religion should suffice.