Yes, the organ is the hardest instrument to play, because it is extremely complicated. A person who plays the organ has to be able to read music extremely well, and read many notes at once. The hands and feet have to do different things, and there are many pedals for the feet. It is extremely complicated and the most complicated instrument.
The organ may indeed be a very complicated and difficult instrument to play, however, one person may find it vastly easier to play compared to some other instrument, based on his or her own individual talents and abilities. Just because the organ is one of the most complex instruments does not mean it is the most difficult to play.