Can't go wrong with Valkyria Chronicles, although I'm not sure if it's available on PC. Basically the whole game is about managing your roster, which is enourmos, and fighting. And the fight is always challenging, even on medium difficulty. Plus the story and visuals are great.
Agree with already mentioned JA2, NWN2 and IWD2, but they should be pretty obvious. Almost every jRPG suits your request just fine, but as you already said, they don't provide a challenge. Blackguards don't give you a roster to choose from, and only so much possible builds to play with. On the other hand, the combat is so good, especially the arena. But it gets really easy and boring near the end. Shadowrun Returns is basically the same, but better at every point. Especially Dragonfall DLC. Totally different setting tho.
Banner Saga has a decent roster, but it's insultingly short and easy. Konung series, which I'm enjoying right now, let's you gather up to 10 people in your party, and there is plenty to choose from, but the combat is pretty straight-forward, real-time and Diablo-like. Disciples 2 is a great game, if you don't mind strategy elements, like resources and global map magic, - it has a unique atmosphere to it. And is also all about right roster managment and turn-based combat. If you played and liked Disciples 2, I would also recomend Dominions 4. The last one don't give you direct control over battles, but the pre-battle preparations are incredibly deep and complex. And the ongoing apocalypse atmosphere is there too.
Oh, and Might&Magic series are decently challenging as well. 6 and 7 may be hard, but 8 is my personal favourite.