Djokovic is very good, but the argument can't be made yet that he is the best tennis player of all time. Roger Federer, Ivan Lendl, and many others can make the same claim based on their records and dominance in their playing days. It is yet to be seen that Djokovic is in that class.
The game and the players have changed a lot over time. Djokovic is one of the best, but I don't think he's the greatest of all time yet. He's only the 3rd male since Rod Laver to win 4 open titles in a non calendar slam, and that's a huge accomplishment. He's got a "Novak Slam" named after him I think he's on his way.
No, Djokovic is too young to be the best tennis player of all time. While he still may become the greatest, he has yet to win as many grand slams as some of those who came before him. The Williams sisters should not be brushed aside when discussing the greatest athletes in their sport.
Semis is a great talent, there's no doubt, but there are quite a few tennis players of the past that could beat him. McEnroe, Connors, maybe even Agassi, not to mention a prime-aged Pete Sampras. Semis is just another above-average player enjoying his time at the top of the game, he won't be a lasting legend like the truly great.