The Commission of Presidential Debates, formed by the Democratic and Republican parties in 1987, is obligated to follow rules set by the Federal Election Commission (FEC). The commission is required to be nonpartisan and to employ objective criteria to determine who gets to participate in the debates.
Many voters believe this is the causing result to difficult conflict.
For instance, Hillary Clinton going into the democrat debate and talking about Donald Trump, on the other hand Donald Trump going into the republican debate and talking about Hillary Clinton.
Everyone is happy where they are. Why should we Mix them up. They are on the side they are more comfortable. Why over complicate things. If they are not party affiliated then the debates would just be even more confusing than they already are. There is no point in it.