Yes, voters should care about a politician's marital and family life, because a marital and family life is probably the most honest window into the true character of the politician. If the politician has children with behavioral issues, they likely have leadership issues themselves. If they are divorced a few times, they probably are not loyal.
The primary concern of voters should be how well (or poorly) a politician is at doing his or her job. Their private lives are just that - private. They are none of our business. If a politician can carry our his or her job duties, their personal life does not matter.
Marital and family matters should not be important to voters because these are only a few constructs that make up who the politician is as a person. A married politician has no more morality than a non-married politician. Additionally, the politician may have family members that do not necessarily have the same opinion on issues. This difference in opinion should not be taken into voter consideration because people do not always agree.
I think that a politician should be judged on his principals and policies not what is going on with his home life or anything like that. His home life usually does not apply to a politicians policies and therefore should not be considered. People have just been wrapped up in these reality tv shows and they believe that if a person does not have the picture perfect home life then they will not be any good in a public office.
It seems that more and more the public is concerned about politicians family and marital life. We should learn to keep these two domains apart. I am more concerned about what a politicians views are on important government policies than what is going on with their marriage and family life.