Oblivion for sure. I dislike it so much for being so dumbed down, and technically Skyrim is even more dumbed down so I could have voted for that. But Skyrim does actually improve some things, the shouts are what I think they should have done for a long time. They clearly don't care about RPGs anymore and want to make action games, so they may as well add some actual decent action gameplay. It still doesn't have that... but the shouts are a step towards it. Also the dungeons are more varied which is probably the thing I found most offensive about previous ES games. Also Skyrim is still the best looking game I have ever played, thanks to mods. It looked like shit out of the box, but with the right mods installed it is the prettiest game ever and runs great on my machine. Basically the game has some things going for it, even though it is shit. But Oblivion is like all of the things that are wrong with the Elder Scrolls series, in one game. It removes stuff from Morrowind making it seem more action game than RPG, it has cut and paste dungeons, it has boring quests, lack of interesting gear, lack of interesting locations, lack of variety in the game world, and it has probably the shortest main quest I have ever seen in a big budget game. I think I completed the MQ in about 2 nights. Tards will argue, "YEAH BUT you are supposed to go and do all the side quests too!" but the side quests are terrible. The MQ is the only thing I care about in these games because it is the only thing that has a bit of thought put into it, the side quests are mostly very generic and boring.
I had some fun with Morrowind, the only game in the series I played all the way through and did stuff on the side as well. It was still too shallow for me and lacked in a lot of areas, but at least it had some RPG under the hood, and the main quest and the mage questline were both fun. For me Skyrim was a lot less fun in terms of MQ and side quests, and Oblivion was even worse again. All in all though I don't really care and would gladly see Bethesda go out of business.