Each of us lives dependant and bound by our own individual knowledge and awareness. All that is what we call reality. However, both knowledge and awareness are equivocal, so one's reality might be another's illusion. That being said, we are all living inside our own fantasies, and reality does not exist, only perspective."
There is no functional distinction between a 'universe' in which everybody lives in their own slightly distinct world and one in which everybody lives in the same world but observes it differently. And we do observe it differently, you only have to ask a group of people what a person in a photograph is thinking to get wildly different responses. To reject subjectivity is only a very short step away from rejecting your ability to be wrong (memories quite frequently get things 'wrong' according to an outside observer).
If reality were subjective, this would mean that reality would be at the mercy of any man's thoughts and feelings and imagination in the wink of an eye. Reality would be contradicting. God's face would appear in the sky. There would be objects of thought. There is no such thing as objects of thought. There are only thoughts of objects that we perceive and develop into conceptual knowledge.