Theoretical physicists even state that it is- that if you say you understand theoretical physics, you probably don't at all. Or more succinctly, as the great Richard Feynman said: "If I could explain it to the average person, I wouldn't have been worth the Nobel Prize." and also: "I think I can safely say that nobody understands quantum mechanics."
Theoretical physics is like any other science (or any other academic discipline for that matter). To those who are not well versed in it, I'm sure it is extremely confusing. To those who take the time to understand it, it makes plenty of sense.
It's not terribly confusing.
Gordon Freeman is a PhD holder of Theoretical Physics and demonstrates how that translates to application by crow baring inter-dimensional beings in the face. Einstein: "If you can't explain it simply, you don't understand it well enough."