Freedom to do anything without legal repercussions is a dangerous idea. Things like murder and assault should by all means be illegal and be met with a police force. Without laws people may do whatever they want if they think they can avoid the vengeance of their community. To protect people, freedom must be limited.
Freedoms should only be taken from a man when his personal choices are almost certainly going to lead to the harm of an innocent person. Limiting freedoms is only acceptable if it is the lesser evil, as in needed to protect someone's well-being. Beyond that, the government or people have no right to step on liberty.