Yes, I believe it is time for David Letterman to retire. He ceased to be creative, funny, and fresh a few years ago and other late night talk show hosts such as Jimmy Fallon, Jimmy Kimmel, and Conan O'Brien have demonstrated how to conduct a higher energy show while Letterman has gotten stale, sarcastic, and injected too much political views into his shows.
Like Jon Stewart, every great TV personality should lay down the job when they feel it's time. Greats who stay past their time and long after they've burnt out show the wear, and the work and comedy inevitably suffers. Those who leave while they still love it but are ready to branch out preserve the work in its heyday. That can be priceless to the memories.
I think David Letterman should have a right to retire if this is what he wishes to do. He has had a good run as a late night talk show host. So, he might as well go out on top and retire. He wouldn't be the first one to decide to retire from the late night arena, of course Johnny Cason did it years ago and many were shocked on that, but everyone has a time when they decide its time to let go and move on.
While he is old and I do not watch him, it is not like he has a job where being in a certain physical condition is required. If he wants to do his job until he dies, that is his decision to make. If he wants to retire, then more power to him.