Live2D: Real-time facial animation of 2D illustrations

Illustrations have captured the imagination of millions of people worldwide – from cartoon characters to books and art. Now, it’s possible to unleash that imagination and bring your favorite 2D character to life with nizima LIVE.

About Live2D

Animating 2D illustrations has been the focus of Live2D Inc., a Tokyo-based computer graphics software company, since 2006. Their software – Live2D Cubism – enables users to create animations and interactive content while staying true to the original 2D art.

So far, the technology has powered more than 100 titles in games and apps and continues to grow rapidly. To make animating 2D illustrations easier and more intuitive, Live2D turned to AI.

Animating 2D illustrations with facial expressions

Live2D Inc. wanted to let users animate 2D models using their facial expressions and movements. The first step in getting there was to track users’ faces in real time.

However, developing such a technology requires a lot of research, development, and testing, which is often a challenge to achieve in-house.

Actively developed and perfected for more than two decades, visage|SDK provides the tracking accuracy and reliability crucial for any vision-based application. Its powerful yet flexible algorithms have been recognized by more than 300 clients across industries.

FaceTrack detects and tracks as many as 99 facial landmarks. This makes it possible to animate avatars in real time and fine-tune their movements in great detail.

The tracker is quick and lightweight, delivering great results with a broad range of cameras and devices, including mobile.

Finally, visage|SDK is available for all major platforms and embedded systems. It comes with detailed documentation, samples, and lots of customization options, which makes the integration quick and easy.

Because of its impressive performance, FaceTrack became a crucial part of nizima LIVE – a PC application that allows anyone to easily move a Live2D model by recognizing and tracking facial expressions.

“What made visage|SDK stand out is the ability to acquire detailed facial expressions and recognize large motions. This was very important for making the movements of Live2D avatars look attractive.”

– nizima LIVE Development Team

nizima LIVE – As you move, the avatar moves with you

With Live2D Cubism, users can turn any 2D illustration into an animated avatar. Users can then move their avatars by using nizima LIVE. The user’s face remains hidden, while their facial movements and expressions get transferred to the avatar in real time. In other words, as the user’s face moves, so does their avatar/the avatar follows.

By directly animating original 2D illustrations, it’s easy to create unique content the audience can connect and resonate with. Many of the works created using Live2D have their origins in traditional 2D, whether visual novels or famous illustrations. The authenticity and rich graphics of the original 2D art are preserved while being rendered in modern applications.

Live2D supports a wide range of applications and environments, such as apps for mobile platforms (iOS, Android), game consoles, game engines (Unity, Cocos2d-x), and other communication tools for events and live streaming video. At the same time, the visual consistency and quality of 2D art is maintained across different media platforms.

Nizima LIVE was officially developed to become a VTuber app. Beginners can easily use nizima LIVE with any standard webcam, while the more savvy users can fine-tune the model movements in detail.

Simple operability, high-quality tracking, and enjoyable collaboration functions make nizima LIVE the perfect solution for not only VTubers, but also anyone who wants to breathe a new life into 2D art.