As far as I'm concerned, the problem with Civ games and SMAC is the so-called "infinite city sprawl"(ICS), as the expansion of complexity in your civilization is not properly balanced by existing penalties to "efficiency". From a gameplay perspective this means the expansion of the managerial aspect of the games in a way that is out of proportion to the strategic aspect. I never play larger than Normal maps for this reason, as managing a large empire becomes tedious and extremely time consuming.
My enjoyment of these games usually peaks with the formation of a strategic quagmire in the early to mid parts of the game, when you have 3-4 civilizations standing in relatively equal footing, or in a balanced but tense configuration. Sometimes an interesting status quo is formed and your civilization acquires the aspect of a real nation with "historical" borders. This is what I love most about the games and other 4X's I've played can't replicate it, even my favorite Master of Magic.
As soon as the strategic balance is broken, the strategic aspect is simplified while the managerial aspect becomes predominant. If I feel there's no meaningful challenge ahead, then I consider the game to be already over and there's little point continuing.
Most advanced strats by expert players consist of rushes (tech rushes and early conquest), min-maxing and so on, and although this is the only way to beat the AI at the highest levels of difficulty, it erases all ambiguity from the strategic layer. Therefore I prefer to play the games in short bursts at the cost of remaining a casual gamer.
A major issue is that there's almost never any "reversal of fortune" possible in these games and of course some argue that if there was players would get mad. There's never any tradeoffs like "expand really fast but have internal stability issues vs slow but careful expansion". Actually Imperator has some of the best gameplay in this regard with moving slaves around and various parts of the assimilation vs integration system.
Having strategic tension is somewhat impossible since the games lack any real diplomacy between powers.
Having a system where one power takes a big chomp but then is stalled while they "digest" I think would add a ton to these games. Typically you almost instantly become more powerful after a conflict.
From a practical perspective it is really hard to create a simple and "gamey", aka civ style, strategy game that doesn't devolve into micro hell. There's no system in most games that shifts the focus as your scope increases. Automation is a bit mediocre. In real life the requirement of delegation is what limits expansion. Managing vassals or other forms of delegation could plausible allow you to avoid micro hell. Paradox tries and fails to employ this strategy, in most cases. Very few others even try.
Diplomacy in SMAC involves personality (of the AI, which can be cooperative, belligerent, etc.), the personal relation (built on accepting/refusing demands, trading), ideology (e.g. industrialist vs green, capitalist vs collectivist) and perceived threat (how powerful you are in relation to others). It's a good system but not developed enough (there's no such thing yet). Perhaps new developments in AI will give us something approaching this. IMO, AI doesn't really need to reach human level, only be good enough that it can sustain a credible challenge with minimal inherent advantages(cheating). Of course, playing against human or even superhuman level AI would be fascinating in itself.
Paradox creates lots of artificial barriers to maintain its façade of historical plausibility. This makes for "complex" games that are very simple from a point of view of pure strategy(I still like them for other reasons). The challenge is only in learning how to use every system to your advantage. Take Vic 2, for example:
- Infamy: prevents player from blobbing, but since the player will manage infamy in a smarter way than the AI, he's guaranteed to expand more in relative terms.
- Diplomatic points: allows weaker nations to steal spherelings from stronger nations, but unlike the AI the player can minmax point allocation to steal nations from others while keeping his nations secure.
- Diplomatic crises: allows restive regions to break away from conquering nations. In reality, it causes the self-destruction of stable empires like the UK as they make wrong alliances and get dragged intto useless wars, while the player avoids disasters by making the right alliances.
The advantage of more abstract games like Civ is that it avoids this kind of complexity, which we could call "strategic noise". Ideally, the AI would make dynamic calculations based on a series of factors: is this nation a threat? could it be a threat in the future? could it give an advantage to a powerful adversary if it allied them/fell under their control? how important is this region economically/strategically? if we are hostile/friendly to them, how would others react? Once the AI is competent enough at this, we could introduce "narcissistic" elements like ideology into the mix to create unique and unpredictable gameplay.