Morrowind and Daggerfall are the best though
Although Oblivion does have a better color palette, I'd have to say Skyrim is better. Not only are the controls better (and easier to understand), the side-quests are actually interesting. Oblivion's side-quests were forgettable. Skyrim's graphics are also updated, along with NPC + enemy AI. Skyrim and Oblivion's MQ was equally interesting to me, so I can't really say which one had a better story. Plus, Skyrim has a good meme... "I used to be an adventurer like you, then I took an arrow in the knee."
The Elder Scrolls games get more dumbed-down with each release and Skyrim is the most glaring example. It's a very good game, and a very fun game, but I feel that it watered down much of what made Oblivion good (after Oblivion itself watered down much of what made Morrowind good). Take away Skyrim's graphics, combat and "Radiant" quests and Oblivion is the superior game in every way.