The appearance of someone is broken down into two categories. The first is genetic, how a person looks because of their genes, and the second is cosmetic, how a person looks based on how well they take care of themselves. To the first part, while it may seem unfair to judge a person based on their inborn traits, over which they have no control, the reality is that a relationship between two consenting adults is going to involve sex. If you are not sexually attracted to a person, there is a basic building block of what a relationship is missing. It may be unfair to the naturally less attractive among us to miss out on finding a good mate due to circumstances beyond their control, but it is equally unfair to begin a relationship with them that will end in catastrophe because humans require sex and you will inevitably seek out someone else you are attracted to.
As for cosmetic appearance, I think we can all agree that if you don't keep up with basic hygiene and keep to a HEALTHY weight, that is, for the purposes of health and not beauty, then you would not make a suitable mate anyway. Find me a person who never brushes their teeth or washes their face, but is a really nice person on the inside and tell me you'd marry them, and I'll eat my shoes.
When it comes to appearances there are cues that tell you if they will be a good match. For example a person who looks sloppy when you are a beat person would not make a perfect match although you may over look it. And you have the right to choose if a person is attractive to you. If you don't think they are attractive then you have the right not to consider a relationship as getting with someone you aren't attracted to May cause some problems down the line.