In many causes people break bones, get serious concussions, and also can't pursue the dreams that they may want to pursue. I think football is challenging because you need to learn plays and also learn how to be safe while playing. Football in my opinion is one of the most physical sports.
While Football is indeed challenging, it cannot be described as "most challenging. The gear is extremely lightweight (11-20 pounds), that isn't near enough to effect one's stamina by any great amount. If it is, they probably shouldn't be playing. Take a Sport like, airsoft or paintball, a serious player will have 20-50 pounds of gear attached to their upper body, and they will spend hours, sometimes even full days wearing that gear and running, fighting, and living in it for the entire time.
I personally do not engage in many physical sports, but I have heard many arguments given that there are other sports which pose a much greater physical challenge than (American) football. Rugby is one example. Many of the martial arts sports could also be argued as being extremely physically demanding (and possibly more dangerous), such as the mixed martial arts tournaments.
Football is not the most physically challenging sport. While it does require athletes to be in great physical shape so that they can run, hit, be hit, and throw, it is not constantly demanding throughout he entire game. There are times when the players' bodies are at rest and they are not using any energy.
While there are brief bursts of activity, the athletes are reasonably well protected from injury, have breaks (offensive line, defensive line) depending on if they have the ball or not, not to mention time outs. Sports like basketball are, in my opinion, much more physically demanding as a player can literally be running from one end of the court to the other for the entirety of the game. Even wrestling is more demanding, because of the lack of protective gear. It probably IS one of the most violent sports however.
Football, in the sense of American football, has plenty of breaks and gaps to allow athletes to relax and recuperate. In contrast, events that require continuous aerobic activity such as ultra-marathons or events that require demanding anaerobic activity such as World's Strongest Man competitions, are much more difficult and do not offer athletes as much of a break.