Although there are plenty of bad preachers and members of various religions, banning all holy men, preachers or religion would be an invasion of privacy and a restriction on freedom. Just because Asaram Bapu allegedly committed a crime does not mean that all holy men are bad, just that he possibly did something wrong. It is OK to bring charges against him, but to group all holy men with him is unfair.
We can not generalise anything done by a person. If any officer or politician is corrupt, should we remove all officers and politicians from this world. I think we should do hard labour to search out miscreants and avoid them. Also we can not ignore social sayings and good works of any person just because of his personnel activities. The two activities should be seen separately and we should use the good one for ourselves. Whatever Mahatma Gandhi did for us can not be ignored just for a few personnel controversial incidences of his life. After every body is human and may do mistakes. Hence we should never generalise any thing so easily.