Especially if you want to consider the sexual misconduct side of things. In some professions it could be an unnecessary liability when theres an equally suitable (insert gender here) that could fill the role. There are alot of people out there, none are really so unique that they cant be replaced by someone else to fill the role.
It really doesn't matter to me, however I've noticed my female managers have been a little more power-hungry than the males... They tend to be more strict/less lenient, a little more harsh, less thankful of me for completing tasks, etc. Everywhere I've worked has been like that.