Freedom is an abstraction, a concept. Each may interpret what it means to be free, but few would argue that "freedom" in society does not includes equal access to opportunity. "All men are created equal" penned by Thomas Jefferson in the declaration of independence framed the entire concept of what we were declaring. Jefferson knew that equality must be insured for freedom to exist.
Equality is a vital part of how we even measure our "freedom". People rising to whatever level they may obtain is only possible where no man is shackled by constraints that other men do not have.
Let"s take an unusual tact from the obvious. Take it in a direction that a libertarian may like more. Unprovoked shooting a gun at someone is an example of restriction of freedom. I have freedom to a gun, I have freedom to shoot it, but the good men and woman of our country have limited this freedom to do so when it violated the rights of others to remain safe. The equal nature of all citizens to be safe comes before your freedom to shoot people.