Malachi - Experimental Treatment

Based on a kid, built from photographs and aged a bit

Youthful Avatar Emotes with Facial Expressions

The Malachi Project started early in 2015 with two goals – to create an avatar that looked like one of our young friends who was helping us test a new system, and to use some shortcuts to develop our avatar from photographs of the face.

Interactive Avatar - Malachi

FROM PICTURES TO 3D SCULPTURE IN SECONDS

Using Blake Senftner’s 3D-Avatar-Store technology, we quickly developed and tweaked the Malachi head to get a close match. It was a little difficult at first to get a good comparison from photos of a boy, not yet in his teens, to the more mature standards of adult avatars in the store’s neural net. Still it was fascinating to see how close the neural net software could come to matching the real person from just a couple of photos.

Malachi Avatar Store

Once the basic 3D sculpture of the head was completed and approved, we then moved on to building the several dozen facial expressions and morph targets in the 3D-Avatar-Store. The store gives us a lot of visualization and customization tools that are very useful and quick. It seemed like we were ready to export the entire file set in a few minutes. (We did take a few iterations to get our process down, of course).

After we exported the project from 3D-Avatar-Store, we had two parallel paths of production – first to work with all the heads and faces we had created in the avatar store, and second, to build our avatar complete with clothes, hands and feet and ready for full-body animation. The heads would be used to generate the facial expressions and lip-sync morph targets, so we went to our friend Isaac Oster to develop a 3D Max script that would do much of the processing automatically.

Malachi Mixamo

The full-body production materials came straight out of Mixamo and Fuse, a technology that many game companies use to generate figures and avatars for their video games. We were able to customize the look and clothing, and add our own modifications to work with the Malachi Project, and were able to see prototype versions of several animations while in Mixamo. We also used Mixamo’s aut0-rigging to prepare the 3D model for animation in Biped (part of Autodesk 3D Max).

Malachi Kinect

Besides downloading the animations out of Mixamo and some purchased mocap files, we also used the Kinect mocap camera by Microsoft and Brekel ProBody 2 to do some in-house mocap. CHOPS’ 3D sculptor and lead animator Tom Knight developed some simple animations that the Malachi character could use in the upcoming demos about autism and how to assist with autism meltdowns.

When all the work on the Malachi avatar was completed, we enjoyed introducing our new avatar to the young man who served as a model and inspiration for this important character.

The Malachi avatar will have many uses, for teen audiences as well as when we are doing demonstrations and trainings in the field of autism and autism treatment. The videos below show the real Malachi meeting his avatar for the first time.