Digital Characters Aren’t Fooling Anyone
The ability to create computer generated imagery has progressed so far that you have to wonder when Humphrey Bogart will star in a new movie and Elvis will release a new video.
Answer – it might be a while as it’s harder than it looks. Let’s take a look at some of the problems.
The first is that digital characters are generated by computer algorithms which don’t “think” the same way humans do. I’ve experimented with just getting characters to walk realistically, but they aren’t going to fool anyone no matter how lifelike the character looks. During walking, a computer calculates the motion of each body part based upon the geometry of the joints and upon the weight, momentum, and center of gravity of each body part and the body as a whole. That takes a lot of calculations, but they are in essence nothing but Newton’s Laws of Motion. In particular, the program optimizes all of the factors and produces motion using the very least energy.
People don’t walk like that. Our stride, even in the case of athletes that are coached based on detailed efficiency analysis, still isn’t as optimized as the computer version for digital people. We try to make the digital version more realistic by adding in random, unnecessary motions such as arm swings, head bobs, leg weaves, etc. That makes it better, but it’s still not going to fool anyone for long.
Even when we eventually can get the basic motions nailed down, we’ve still got problems as to how people apply those motions. In short, the computer optimizes everything and we don’t. When I reach from the keyboard to my mouse, I’m not following an optimum path – just one that works. Everything we do is like that. Trying to program in non-optimal motion is really difficult.
A Few Examples
And even then, we use our motions differently. Suppose you are playing center field. You’re at medium depth when a long Texas leaguer is hit into the triangle between you, right field, and second base. A computer would pick up the ball’s trajectory, then have it’s digital human run full speed to the spot where the ball will land, then stand there and wait for the ball to arrive. When a real baseball player sees the ball, he or she begins moving in the general direction of where the ball will land – and ultimately, doesn’t get to that spot until the ball arrives.
People do the same when skeet shooting – tracking the clay along it’s trajectory with their shotgun. A computer would just point at the “shooting spot,” then elevate as needed and wait for the arrival of the clay.
OK, since getting realistic motion is difficult for a computer, how do they make the movies look so good?
The answer is motion capture where a real person is recorded making natural moves. By placing targets on the motion capture actor at joints and other strategic locations, the computer is able to track the movements and apply them to the digital person. If enough targets are used it can track everything in minute detail – enough detail so that all the “human nuances” are recorded and can then be applied to a fully digital character which now does move realistically. With the advent of continually improving A.I. capabilities, I’m sure the motion capture movements are being analyzed to improve the pure computer generated movement algorithms.
However, even when a digital character can be made to move realistically, it still has to “act” – to show appropriate motions as required by the script. This is even harder to do and again motion capture is used to record the real acting of a human being. Instead of just targets on the body, there are also multiple targets placed on the actor’s face to pick up the subtle facial movements of emotion and speech.
One of the true masters of motion capture acting in Andy Serkis. He was the real person behind Gollum in the Lord of the Rings trilogy, Supreme Leader Snoke in the latest Star Wars, and many, many others. One of the most remarkable and difficult scenes he did was Smeagol (Gollum) chasing a jumping fish in a stream. If you look carefully, all the splashing and everything else matches up extremely well with the digital Gollum.
I’ve searched the web, but I couldn’t find a “behind the scenes” of the production of this scene, but it is included on my Blu-Ray. Watching Andy flailing around in the water is remarkable in itself – but even more astonishing and a testament to his “the show must go on” professionalism, is that when they got ready to film him in the water, they first had to chip away all of the ICE before he could crawl around in the frigid water. (Better him than me!) If you do a YouTube search for Andy Serkis you can see many examples of his remarkable work – and how it was done.
Boston Dynamics Dancing Robot
Motion capture and animating a digital character is tough enough, but when you get to animating a physical “character” things get even more difficult. Even just walking across a room, we and our digital doppelgangers are continuously in an unstable position that is only possible because we move to another unstable position before we can fall over from the first position. With a digital character, you can freeze them in a frame without worrying about their lack of stability while you make adjustments. They won’t fall over while you figure out what to do next – gravity has taken a holiday.
With a “physical” character, you can’t stop because they WILL fall over. They must be dynamically stable continuously – their balance must be adjusted many times a second – just as we adjust our balance many times a second. In the following clip from Boston Dynamics, I’m sure that the “dance routine” was derived from motion capture, but executing the choreography required God knows how many adjustments to dynamic stability to be made continuously, autonomously, and in REAL TIME.
It’s pretty amazing and more than a little scary when you get to thinking about it. I can’t begin to describe how many ones and zeros are flying around in silicon to pull this off. I tip my hat to the Boston Dynamics gang.
You can do a YouTube search for Boston Dynamics to see some more of the incredible things their robots can do by themselves without the assistance of motion capture. In those you will see the computer algorithms autonomously “choreographing” their motions – which while they do work well, again wouldn’t fool anyone to thinking that they were human motions.
In short, it’s going to be a while before Andy Serkis is out of a job.
PS. The real problem I have with the Boston Dynamics video: the stinking robots dance a helluva lot better than me!