Fighters should beat Wizards in direct confrontations 10/10 times. The only reason they didn't in DnD was because the designers were nerds and designed some spells like sequencer just to prevent that. Magical swords should always outdamage magical spells available at the similar levels since they apply magicstuff directly to the target. Reading Conan for a bit should make everyone understand why being a total nerd would not be that useful in a direct combat. DnD made wizards so powerful because it gave them toys they traditionally never had while not giving anything to other classes. A high level DnD wizard (not even an epic level wizard) would wreck Gandalf because Gandalf can't kill things with a single spell, can't maze people, can't cast so many magical defenses, can't use sequencers, can't teleport at wish, can't travel through spheres, can't summon demons at wish etc.
One important aspect of the balance between classes in D&D which tends to get overlooked is that, originally, the "endgame" of D&D started around 10th level. It generally varied between 9th and 11th level, depending on the character class in question.
This was referred to as "Name level", and this was where:
- characters stopped earning a new title each level (for fighters, it stayed at "Lord", for thieves it was "Master Thief", etc)
- characters hit point progression flattened and they gained from 1 to 3 HP per level, instead of rolling a hit die and adding CON bonus
- cumulative XP required to gain the next level stopped doubling and required a set amount for each individual level. This amount per level was generally equal to the cumulative XP required to reach Name level (250,000 for fighters; 220,000 for thieves; 375,000 for magic-users, etc). Given that the gold piece value of treasure acquired was the primary source of XP in all versions of D&D prior to AD&D 2e, it is clear that advancement after this point was assumed to be very slow; a party of 6 fighters would need to acquire treasure worth 1,500,000 gp for each of them to gain a level after 9th. Looking at the Treasure Type tables indicates that it would take a very, very long time to locate this much treasure. You'd probably have to defeat more than two dozen ancient dragons in their lairs to acquire this amount of treasure.
- upon reaching Name level, characters could build a stronghold and attract followers, the details varying significantly from one class to another (fighters would attract their own militia upon building a castle, thieves would attract apprentices when building a hideout, etc). It's not that characters couldn't build a stronghold prior to this point, but this was where the campaign was assumed to shift gears and this shift in focus was built into the rules
- primary spellcasters (magic-users, clerics, etc) were just acquiring the ability to cast 5th or 6th level spells
In addition, non-human characters tended to hit their level limits for their thematically-appropriate classes around Name level (dwarf fighters, elf magic-users), with the level limits for other classes being a bit lower. Thieves were the exception, as every race except the half-orc had unlimited advancement in the thief class. In the AD&D 1e Player's Handbook, druids topped out at 14th level, period (and there could only be a single druid of this level in world, with limited numbers of druids at each level leading up to 14th).
The original (1974) D&D game topped out around this point, with the higher-level stuff (such as 7th to 9th level spells) first appearing in the supplements. AD&D 1st Edition was essentially a compiled and expanded version of OD&D plus the Greyhawk, Blackmoor, and Eldrich Wizardry supplements. The 1981 Basic/Expert D&D game was intended as a revised and streamlined update of OD&D on its own (plus the thief class from the Greyhawk supplement). These versions of D&D implied that PCs would generally settle down and mostly retire from active adventuring as they reached Name levels; the players would create new lower-level characters (as it was the norm for players to have a stable of characters and switch between them from one adventure to the next). When you look at the stats of the toughest monsters in the AD&D 1e monster books, it's clear that they are designed to challenge characters around Name level.
Early versions of D&D were balanced at the campaign level, and Gygax's playtest groups played very frequently. It was expected that PC deaths would be fairly common, that there were likely more players in the campaign than were present for any one session, and that players would create many characters for use in the campaign (either to replace fallen PCs or so as to be able to choose the best PC for a particular adventure). Thus, the randomness inherent in character generation would balance out over time; there wouldn't be many rangers or paladins in a campaign, due to their high stat requirements, but over the course of a campaign, it was likely that a player would be able to qualify for such a class at least once.
This is all rather long, but these inherent assumptions of early D&D are an important piece in understanding the balance between character classes and why the originally squishy thieves evolved into acrobatic ninjas. The design of early D&D has often been accused of being arbitrary and nonsensical, but it was never subject to a comprehensive, top-down process of design. Its design was very organic, with the rules reflecting what worked best in those early playtest groups.
Over time, the way that people played D&D diverged from these assumptions. The modern convention of the same group of players meeting at regular intervals, where each player has one character that he plans to take from 1st level through to 20th level, is very, very different from how it was played in the 80s. The "adventure path" paradigm was already becoming prominent by the time of AD&D 2e (Dragonlance being the prototype of this), but it was 3rd Edition where the rules were changed to reflect these "modern" sensibilities. Unfortunately, the designers of 3e seem to have lacked a deep understanding of the implicit assumptions which informed the design of AD&D, and we got a system that, in hindsight, was far more unbalanced and broken than any early version of D&D. They recognized that very few AD&D games ran long enough for players to get to use all of that high-level content, so they designed 3e such that players could take characters from 1st to 20th level with weekly play in an 18-month campaign (which their research determined was the average amount of time before a campaign fizzled out).
The 3e rules implicitly put a stronger emphasis on combat than did AD&D. By default, combat is more tactical than in AD&D, with 6-second rounds (instead of 1-minute), which makes movement rates much more relevant. On top of that you have attacks of opportunity and the fact that rogues' sneak attack ability depends on flanking, so accurate tracking of position in combat becomes very important. This makes combat take longer, especially when you start using a battle grid. And when combat starts to take up a greater portion of the session's play time, players who don't get to contribute much start to get bored (rightfully).
XP for gold is long gone at this point, with XP being awarded for overcoming "challenges" -- usually combat, but also overcoming traps. But it requires a LOT of traps for rogue to obtain enough XP to go up a level; AD&D thieves were good at stealing valuables, and they advanced in levels quickly, so a well-played thief could advance several levels without engaging in much face-to-face combat. This is virtually impossible for a rogue in 3e if you play by the book. The bulk of XP in 3e comes from defeating opponents in combat, and 4e continued this paradigm -- and also made rogues even more ninja-like. Given the length of a typical 4e combat encounter, a player whose character has little combat utility might as well not even bother showing up.
The Companion (1984) and Master (1985) rules sets for the Mentzer revision of Basic/Expert D&D were designed to focus on 15th through 36th level characters, but they show the artifacts of earlier assumptions of play (halflings stopped at 8th level; elves at 10th; dwarves at 12, with weird "attack ranks" to allow them some measure of progression that still paled next to any human characters). Mentzer himself admits that little to none of this material was ever playtested before release.
TL;DR: character classes in early versions of D&D were actually pretty well balanced when you recognize that 9th level characters were considered "high level" and entering the endgame phase of the campaign. The truly world-shattering magics (7th level spells and up) were never intended to see regular play. Thieves didn't need to be good in combat because they could advance in level by sneaking around, breaking into places, and stealing valuable treasures. Level advancement for everyone was far more dependent upon achieving goals (finding the treasure and getting it out of the dungeon) than it was upon killing monsters standing in the way of those goals, so there was an incentive to avoid unnecessary combat. Engaging every enemy in combat was generally a stupid thing to do, and the XP reward was often not worth the risk or the resources consumed.
It was actually a very, very different game than the D&D of the 2000s (and even the 90s). Early D&D had more of a focus on strategic and operational play; planning and preparation were key, and cautious play was rewarded. Modern D&D has a stronger tactical focus, with much of the "boring" operational planning and preparation glossed over and there are more safeguards built in so that players don't need to be quite so cautious (not entirely a bad thing, as very cautious players can make the game excruciating to play).
