Putting the 'role' back in role-playing games since 2002.
Donate to Codex
Good Old Games
  • Welcome to rpgcodex.net, a site dedicated to discussing computer based role-playing games in a free and open fashion. We're less strict than other forums, but please refer to the rules.

    "This message is awaiting moderator approval": All new users must pass through our moderation queue before they will be able to post normally. Until your account has "passed" your posts will only be visible to yourself (and moderators) until they are approved. Give us a week to get around to approving / deleting / ignoring your mundane opinion on crap before hassling us about it. Once you have passed the moderation period (think of it as a test), you will be able to post normally, just like all the other retards.

GDC - Witcher 3 dialog animation in cutscenes were mostly based on algorithm

Perkel

Arcane
Joined
Mar 28, 2014
Messages
15,810
Now we know how they were able to make such truckload amount of dialog cutscenes (35hours) and they were mostly well animated.

They can basically generate scene animation and hand tweak it. It takes several minutes to finish one dialog in game.
imo amazing smart design, kind of like face thing from betsheda games (automated lips movement according to voice, don't remember what it was called.

http://www.pcgamer.com/most-of-the-...m_source=facebook&utm_campaign=buffer_pcgamer

Most of The Witcher 3's dialogue scenes were animated by an algorithm

The Witcher 3 is a massive game. It packs in 35 hours of dialogue, each line of which was voice acted and motion captured. If I had been in charge of orchestrating all the moving parts of the game's development, I would've had a breakdown a month in and the dialogue system would've ended up more like Facade. Thankfully, the much more capable Piotr Tomsinski was in charge, and he gave an enlightening talk at GDC on Friday about how much work went into making the characters move and speak so naturally.

The problem going into The Witcher 3 was obvious: they were making a vast, non-linear, fully-voiced RPG. CD Projekt wanted decisions in The Witcher 3 to feel meaningful, and for them to feel meaningful players needed to form emotional attachments with the characters. They wanted to be able to sell drama by showing it, not by telling you up front a scene was supposed to be emotional. Writing 101, essentially.

Doing individual motion capture work for every dialogue scene and then animating them all by hand would've been impossible, or taken up ridiculous resources (Tomsinski showed that a team of only 14 worked on the cinematic dialogue system, including programmers, animators, and QA—other hands likely pitched in, but that seems to be the core team). So CD Projekt built a number of systems, and a huge library of data in the form of reusable and easily modified animations, that could be combined together to create The Witcher 3.

PQc9QQkisSto.878x0.Z-Z96KYq.jpg


With the systems they created, designers could make their own dialogue scenes without needing to pull models into a tool like Maya to do heavy duty animation. When he first showed off their Timeline tool, it looked overwhelmingly complicated—like a more complex version of Logic Pro or Adobe Premiere. But it's actually not so bad: there are different rows for animations, 'lookats' (which is where the characters in the scene are looking), placement (location in 3D space), and a few other elements.

The real magic comes in how they generated the dozens of hours of dialogue scenes using an algorithm, and then went into the timeline to hand-tune each one instead of building it from scratch.

fiJOK9P9P0OD.878x0.Z-Z96KYq.jpg


"It sounds crazy, especially for the artist, but we do generate dialogues by code," Tomsinski said. "The generator's purpose is to fill the timeline with basic units. It creates the first pass of the dialogue loop. We found out it's much faster to fix or modify existing events than to preset every event every time for every character. The generator works so well that some less important dialogues will be untouched by the human hand."

That's right: a bunch of math determined how most of the dialogue in The Witcher 3 was arranged and animated. So how did it work?

"The generator requires three different types of inputs: information about the actors, [some cinematic instructions], and finally the extracted data from voiceovers. We use an algorithm to generate markers, or accents, from the voiceovers, so later we can match the events in animation with the sound. It generates camera movement and placement, facial animation, body animations, and the lookats."

The Witcher 3 has some of the best-looking character interaction in any game, and most of that started with procedural generation. If the animators weren't happy with a scene, they could simply press a button to regenerate it, and the algorithm would conjure up something new with a slightly altered mix of camera movements and animators. Tomsinski showed off some side-by-side examples, and it was easy to see the small distinctions between them; subtle differences between head and body movements, the pauses between movements.

"The generator works so well that some less important dialogues will be untouched by the human hand."
Of course, they didn't let the algorithm run and call it a day. The thing both scenes had in common was that they looked a bit amateurish—really, like awkward actors stumbling over a scene in a film, or the not-quite-natural animation of games that started to really explore cinematic character interactions (i.e. almost everything pre-Mass Effect). Most of the time, the animators would take what the generator had created, then go into the timeline to tweak it by hand, which could deliver a much better scene in just a few minutes. In some cases, they'd add in more elaborate camera movements, reposition characters and facial expressions, and so on, but they already had a great, unpolished base to work from.

The finished example Tomsinski showed adding a lingering camera shot to the end of the scene for a more cinematic transition, and the character Geralt had been talking to made a subtle facial expression as the witcher walked away. It doesn't sound like much, but it's amazing how much more life that gave the scene.

Rxii5KjOnk4v.878x0.Z-Z96KYq.jpg


The building blocks for all those scenes were a set of 2400 dialogue animations, but divided between the various types of characters: men, women, dwarves, elves, children, etc., and different poses (standing, kneeling, and sitting), that number gets significantly smaller. They needed to be reusable.

Tomsinski gave an example: a simple gesture Geralt makes with his hand while standing. What if they wanted Geralt to make that gesture while sitting? They could try adding that animation to the timeline after inserting Geralt in a sitting pose, but that doesn't work—he suddenly appears stood up and waves. So they created a system for additive animations, where only the key part of the body will move—in this case, his arm—allowing animations to be combined. Bam! Geralt is sitting down, but making the same gesture. Other tools, like masking, let them further tweak the movement of specific limbs. In this example, they made sure his legs looked natural as he moved.

There were other key elements to the system, like how they designed the lookat animations with attached poses, so characters would lean on one arm when looking in a certain direction, and how the timeline could dynamically scale for localization to account for longer or shorter dialogue in different languages. But to recap: holy cow, the cinematic dialogue in The Witcher 3 is amazing, and now we know why.


 
Last edited:

Perkel

Arcane
Joined
Mar 28, 2014
Messages
15,810
Damn that's cool as fuck. Specially since the end result is top notch.

i think it is first that kind of animation system used by any studio for cutscenes. Animation morphing isn't new but to use it and actually get it working with cutscenes is great.

explains a lot just how they were able to put so much good stuff in even sidequests.
 

Rahdulan

Omnibus
Patron
Joined
Oct 26, 2012
Messages
5,105
Which is funny if you look at something like Dragon Age: Inquisition. That actually looks like it was fully generated without human hands touching anything because of how clumsily executed everything seems in comparison to Witcher 3. Poles seem to have a struck gold with this technology and actually putting in the effort to develop some proprietary tech or method of implementation was definitely worth it. Not only for Witcher 3 but their future games as well, obviously. Especially lip-sync which was a HUGE jump compared to stilted flop-flopping they employed in Witcher 2.
 

Doktor Best

Arcane
Joined
Feb 2, 2015
Messages
2,849
Damn... Witcher 3 has the best cutscenes in gaming history probably, very surprising that this isnt done by full motion capturing.
 

DosBuster

Arcane
Patron
The Real Fanboy
Joined
Aug 28, 2013
Messages
1,861
Location
God's Dumpster
Codex USB, 2014
Seems very similar to how Valve Software's Faceposer tool works, it's also weird how they're describing animation blend or procedual animation as this new amazing thing.

Regardless, Witcher 3 had some amazing cutscenes and facial animation compared to Fallout 4 which had very inconsistent results.
 

Perkel

Arcane
Joined
Mar 28, 2014
Messages
15,810
Seems very similar to how Valve Software's Faceposer tool works, it's also weird how they're describing animation blend or procedual animation as this new amazing thing.

They are not talking about morphin animation. They are talking about whole scenes being generated on fly in their editor which mostly look ok and then animator just needs to finish it.

So let's say that you want to make a scene.

Character x go to point y, then run, duck and run again. Start talk with character B. In that talk you should be angry and sad.

Dude chooses something like that above. Editor works for a while and shows him up almost complete scene which is based around game reconizing what kind of characters they are, locations, how should they look and for what etc.

Then once scene is done he just edits it which is like 10% of work he would need to do when not using system like this.

I don't think there are any studios who use such setup currently outside of CDPR
 

DosBuster

Arcane
Patron
The Real Fanboy
Joined
Aug 28, 2013
Messages
1,861
Location
God's Dumpster
Codex USB, 2014
CD Projekt RED have clarified some information about this system:

At CD PROJEKT RED we strongly believe in a hands-on, custom approach to content creation. I'm sure you can tell this from the way how the world Geralt traverses was designed - and our interactive cinematic sequences (dialogues as we call them) are no different. It's not true that "a bunch of math determined how most of the dialogue in The Witcher 3 were arranged and animated".

How does creating a dialogue scene like this look like from cinematic designer's point of view? First of all, it's not only animating. In fact, there's very little animating at all. Animations are delivered to the animation library - a huge set of gestures, moves and facial animations. Cinematic designer working on a scene simply uses those libraries, crafting actor performance from pre-made animation blocks. And it's something no algorithm can do. Does the woman who lost everything in the fire do this or that gesture? Should it happen while she speaks or during a pause? In what pose does she stand? Should she look away? For how long? Should she be expressive or hide her feelings?

We would like to emphasize that creating a compelling scene is more than just “animating”, which can be seen as the process of constructing the acting. Creating a compelling scene is in fact editing, preparing cinematography, staging and applying other cinematic means of expression. Algorithm didn't compose our shots so that they have depth and balance. An algorithm didn't decide when to cut the camera to show the NPC's reaction or when to move from a medium shot to a close-up. The algorithm didn't decide when characters moved or changed poses. It didn't tell us if a scene should be fast-paced with wide-angle shots or slow shot with medium lenses.

The “algorithm” or “generator” as we call it, was used only as a solid base for further development of the scene. It was a shortcut, a tool, but never a goal. More of a production-related thing. It created a rough first pass through a scene, which was always tweaked and adjusted by hand - in all 1463 dialogues. In many, the algorithm wasn't used at all, as they demanded custom approach from the very beginning.

Every cinematic dialogue was approached with the same care, attention and goal - to create the most compelling and emotional scene for given quest and story. Only this way, the characters could ring true and players would want to invest in them, to understand them, to help or condemn them. When they act like humans, not voiceover-delivery machines. Achieving this is a deliberate, careful process. Procedural doesn't get you this. A designer with empathy does. Because you have to put your heart into something to move someone else's.
 

the_1990

Educated
Joined
May 23, 2015
Messages
81
Oh how bitter this makes me that they never released the mod tools they promised. Imagine having this technology at your fingertips, being able to create brilliant cinematic cut scenes with such relative ease. Instead we got a crappy substitute that allows you to alter models, textures, stats and not much else. This press release feels like they're now just taunting the modding community that never was.

"Oh, you like this tech? You'd like to be able to make detailed, cinematic, unique dialog scenes for your own story content on the fly? Well fuck you! The technology's ours and your just gonna have to sit there with your mouth watering while we use our technology to create the content we think you want, and you're gonna like it. And the best part is; your content might have been just as good or even better than ours, but now no one will ever know! hahahahahah"

JK: CDPR are bros, but it's still annoying. This tech would be massive incline for modders.

Paradox should license this technology for a Bloodlines sequel.

Fixed
 

Perkel

Arcane
Joined
Mar 28, 2014
Messages
15,810
Oh how bitter this makes me that they never released the mod tools they promised. Imagine having this technology at your fingertips, being able to create brilliant cinematic cut scenes with such relative ease. Instead we got a crappy substitute that allows you to alter models, textures, stats and not much else. This press release feels like they're now just taunting the modding community that never was.

"Oh, you like this tech? You'd like to be able to make detailed, cinematic, unique dialog scenes for your own story content on the fly? Well fuck you! The technology's ours and your just gonna have to sit there with your mouth watering while we use our technology to create the content we think you want, and you're gonna like it. And the best part is; your content might have been just as good or even better than ours, but now no one will ever know! hahahahahah"

JK: CDPR are bros, but it's still annoying. This tech would be massive incline for modders.

Paradox should license this technology for a Bloodlines sequel.

Fixed

TW3 doesn't have proper modding tools because they use too much middleware they simply can't use in modding tools.

UMBRA is the big one as this is i think offline tool that analizes parts of worlds and creates occlusion map or something (idk how this work properly) and bakes it into map or file thus modders wouldn't be able to use LOD system at all.

Modding tools are now way harder to make than it was years ago precisely because of increased amount of middleware games use.
 

the_1990

Educated
Joined
May 23, 2015
Messages
81
TW3 doesn't have proper modding tools because they use too much middleware they simply can't use in modding tools.

UMBRA is the big one as this is i think offline tool that analizes parts of worlds and creates occlusion map or something (idk how this work properly) and bakes it into map or file thus modders wouldn't be able to use LOD system at all.

Modding tools are now way harder to make than it was years ago precisely because of increased amount of middleware games use.

They should have known that before they promised it then...

Witcher 2 used quite a bit of middleware too though, like Speedtree, still released mod tools. Also, couldn't modders just get a copy of UMBRA?
 

Kem0sabe

Arcane
Joined
Mar 7, 2011
Messages
13,076
Location
Azores Islands
Cd projekt are hard working Bros, and have been improving their tech from a simple rework of the nwn Aurora engine, into the best action rpg of the last few years.

Now we need more info on cyberpunk
 

the_1990

Educated
Joined
May 23, 2015
Messages
81
Nevermind, seems Umbra is not just something you buy like that, what a shame, that means CDPR had been promising mod tools while all the while developing their game in a way that made it impossible, that's sad...
 

Perkel

Arcane
Joined
Mar 28, 2014
Messages
15,810
Nevermind, seems Umbra is not just something you buy like that, what a shame, that means CDPR had been promising mod tools while all the while developing their game in a way that made it impossible, that's sad...

They didn't promise anything like modkit from TW2. At least from what i remember they only promised some mod support and they did actually.

And it isn't only Umbra. Speedtree is missed shot though because speedtree is for creating assets (trees duh) which is completely offline.

Generally speaking in large project usually pipeline is waaaaaaay more complicated than any your usually modder would be able to handle. Thus why companies need to release "modkits" which are basically stripped down combined version of all their tools they usually are using.

Bethsheda gets a lot of flak for their age old engine but if they would change it then they would go for a lot of middleware killing modding support
 

the_1990

Educated
Joined
May 23, 2015
Messages
81
They didn't promise anything like modkit from TW2. At least from what i remember they only promised some mod support and they did actually.

They promised exactly that, I remember one of the devs on the Witcher forums telling ppl how awesome the REDkit would be.

And it isn't only Umbra. Speedtree is missed shot though because speedtree is for creating assets (trees duh) which is completely offline.

Like I said, Witcher 2 used speedtree, still released REDkit.

Generally speaking in large project usually pipeline is waaaaaaay more complicated than any your usually modder would be able to handle. Thus why companies need to release "modkits" which are basically stripped down combined version of all their tools they usually are using.

Something tells me they'd make a greater effort to provide mod support if the whole "paid mods" thing took off.

Bethsheda gets a lot of flak for their age old engine but if they would change it then they would go for a lot of middleware killing modding support

Surely they could just develop a new one?
 

Perkel

Arcane
Joined
Mar 28, 2014
Messages
15,810
Like I said, Witcher 2 used speedtree, still released REDkit.

Did you read what i wrote about speedtree ? It is asset creator. It is completely offline tool and doesn't have anything to do with modding availability.

Fallout 4 uses UMBRA and did use speedtree until Skyrim

Source ?

Fallout 4 modding tools are way more late than Skyrim were. So probably they are pulling off middleware from their pipeline and trying to do something different here.

If they will actually release it, that is. (still no release date)
 

DosBuster

Arcane
Patron
The Real Fanboy
Joined
Aug 28, 2013
Messages
1,861
Location
God's Dumpster
Codex USB, 2014

the_1990

Educated
Joined
May 23, 2015
Messages
81
http://umbra3d.com/press-releases/umbra-powers-up-the-massive-open-world-of-fallout-4/

Fallout 4 Creation Kit is coming next month, the reason why they can't do it on release is that they have to prepare documentation and video tutorials, also with Fallout 4's case it's being held back by the Bethesda.net infrastructure where mods will be uploaded, however, they've confirmed it will release in April.

Interesting, so basically that's one less excuse for them going back on their promise?

This means that either there was some other technical issue with the game that prevented them from releasing REDkit, OR CDPR just decided to be like:

:troll:

Makes me rather embarrassed that my first post on this forum was one defending them over the graphical downgrade, ppl seriously thought I worked for CDPR.
 

As an Amazon Associate, rpgcodex.net earns from qualifying purchases.
Back
Top Bottom