YouTube’s Next Big AI Upgrade: Lip-Syncing For Auto-Dubbed Videos Coming Soon

YouTube is making it easier for you to watch videos in languages other than your own. No, we are not talking about the existing caption feature. Instead, YouTube is planning to bring lip-sync technology to make the auto-dubbed videos look more natural. Also Read: After Phone And iPad, Instagram Could Soon Be On Your TV: Here’s What We Know

According to a report by Digital Trends, YouTube is developing a feature that utilises AI to ensure the speaker’s lips move naturally in sync with the dubbed audio in the video. It is kind of an addition to the auto-dubbing feature, which was launched last year. But the lip-syncing feature is a whole new challenge for YouTube. Also Read: Meta’s AI Can Now Dub And Translate Instagram And Facebook Reels In Hindi, Portuguese, And More

YouTube Lip-Sync Feature: How Will It Work

YouTube Product Lead, Autodubbing, Buddhika Kottahachchi, told Digital Trends that the lip sync feature will modify the pixels on the screen to match the translated speech in a dubbed video. For this, YouTube needs to develop a 3D understanding of the world, lip shapes, teeth, face, and postures. He further explained that the tech is so far suited for full-HD videos, that is, 1080p. However, the 4K videos still need to wait. Also Read: WhatsApp Status Sharing On Facebook And Instagram Coming Soon: Here Is How It Will Work

The lip-sync feature will support several languages, such as English, Spanish, German, Portuguese, and French. 

When Can We Expect It?

While there is no official word on it. The report suggested that the lip-sync feature is not broadly available, and there are no plans to make it available. Kottahachchi said that the lip-sync feature will be available to more creators, which will help YouTube understand the quality and constraints. It must be noted that the feature is still in the testing phase. 

Instagram Translate And Lip-Sync Feature

The social media giant Instagram has recently expanded its real-time translation feature to India. Instagram creators can translate, dub and lip-sync their content without any other tool. It is available in English and Hindi.