If you are just talking about slavery in general. Sometimes you were enslaved just because of your class. If you are refering to the Confederate States example then, however you are absolutly correct. On the other hand you have not listed a specific example so my point stands.
Humans have been enslaving one another almost since time began. The romans were white Europeans and the enslaved many other white Europeans. The Africans enslaved their fellow Africans for many reasons and to this day they still do. The people who say yes are a bunch of race pimps looking to stir up trouble.
If you are talking about the Romans, then anyone could be a slave. There weren't any Roman slaves, at least not very many, but there were slaves of every other race. After someone became a freedman, they could only be distinguished from a slave by their mannerisms, their actions and how they dressed and cut their hair. Nothing about their race, ethnicity, religion or gender would indicate whether they were a slave. Now, the American South before the civil war, yes it did correlate.