Although Cantor is a conservative Republican, he lost to a Tea Party challenger in part because he was viewed as being too liberal. Many Republican incumbents are facing similar challenges, as the Tea Party skews to the extreme crazy right. In Tea Party territory, it's unsafe to be seen as even remotely liberal.
I think when looking over all of the evidence I believe he lost because he simply lost sight of his home state. He was in Washington DC on the night of the election and many say he hardly campaigned at all. He wrongfully assumed he had a strong base and home and now he is paying the price. It's a lesson that many politicians will learn from.
No, it was not conservative views that led to Cantor being voted out of office. Cantor was actually trying to project more moderate views in order to appeal to the voters. He had backed away from conservative Tea Party views. Cantor simply became too comfortable with his perceived popularity and was replaced by someone who worked harder go get the votes.
No, Cantor's conservative views did not leave him out of touch with Virginians, rather it was his liberal views. Cantor and Boehner have gone with an extremely liberal agenda. They have increased the debt ceiling, and they took Amash off of his committees when he didn't fall in line. Cantor's views were too out of touch liberal, not out of touch conservative.