Pika, one of the main pillars of video AI, has introduced a new feature that allows you to create other people speaking in AI-generated videos. Si Chiama Lip Sync is the result of a collaboration with ElevenLabs, which specializes in AI speech generation.
Lip Sync allows you to synchronize characters’ lip movements with your own, if they perform a switched test to audio in the same way as if they process a user-drawn audio file. In this way, it is conceivable to dare to watch an AI-generated video, without going back to real agents or voice-overs.
Lip Sync represents an important step forward in the field of video AI, which remains the goal. If it works well and fixes the teething issues, it can be an ideal touch on the launch of Sora, OpenAI’s AI video platform.
Pro users can now get early access to Lip Sync on https://t. co/nqzjGy82Lx. pic. twitter. com/vCJ88pUWLL
– Pika (@pika_labs) February 27, 2024
Finely speaking, most of the AI-generated videos were undeniable videos about the scene, person, or visual situation at its peak, without any interactivity. I didn’t communicate with the camera or work, restricting the narrative and artistic possibilities.
With Lip Sync, on the other hand, characters can talk to the audience or other characters, making the video more realistic and interesting. The partner’s person, control, or audio and platform will suffice if they are busy generating the lip-sync video. All users who have typed Piano Pro or higher can check out the new feature.
That’s why Lip Sync is the only feature of its kind on the market. There are other instruments, such as Synthesia, that offer the option to create videos with talking characters, but concentrate more on the horizontal setting and generate a single speech test. No internal characters.
Pika Labs and Runway point to the ideal leader for AI video generations in the last period. Runway already brought its voiceover service earlier this year, but it wasn’t in sync with video. Today, the festival is more closed, with several other players entering the video generation box and OpenAI showcasing its impressive Sora AI video platform.
Still, they’re updating their features, such as StabilityAI with its new edition of Stable Video Diffusion and Leonardo with its motion that provides any AI-generated image. Google Light and Meta have Emu put pressure on early adopters to innovate before anything else.
We ended up witnessing the composition of generative AI. These are tools for creating images, tools for creating videos, writing scenes, and others for adding your content. In the long term it may bring greater integration, with the emergence of a platform that would be offering an end-to-end production from an undeniable wealth of checks.
ElevenLabs has also been working with a library of sound effects, and on Suno, you may soon witness the birth of an exclusive platform if you can say “take this scene written through ChatGPT and turn it into shortmetric. “If you can have a timeline with a series of videos, dialogue between characters using ElevenLabs’ artificial voices, and sound effects and music good enough to make the production dynamic.