Yes, I believe that the government should always be able to determine the rights of the prisoners in its custody. Look, when you're in prison it's because you were caught doing a crime. You don't have normal rights, because you couldn't behave like a normal person. Now you have to do what the government tells you.
Simply: yes. There is no better agent to determine these rights, because the system that is set up is designed by the government. Further, if the government is supposed to be a reflection of the people's wishes, any decision made by the government should be reflecting society's wishes and values.
Who else is going to decide and determine prisoner rights? The warden? The locals surrounding the prison. No, you elect the officials into office and they represent you. They make the decisions. If you think their decisions are off, vote them out of office the next time an election comes up.
Unfortunately, it is the government that has to determine prisoners' rights. The government runs the prisons. Therefore only the government can control what rights the prisoners do and do not have. However, the government should not take this responsibility lightly- the government should give prisoners the rights they deserve, and the determination of what rights prisoners deserve should come from studies on how to best prevent re-incarceration of prisoners once they have been released.
The Courts should be the final arbitor of prisoners rights. The government has a vested interest in keeping prisoners under control and not spending very much on them. At times these goals are at odds with a prisoners rights. Courts are best able to balance both the prisoners and governments needs..