AI-Powered Video Translation

Did you know I speak Spanish fluently?

Unfortunately, I don’t. But with the help of AI, it’s a virtual reality.

Using a tool called HeyGen, I uploaded a video that features at least 30 seconds of me speaking. It then regenerated the video from English to Spanish.

It didn’t just generate captions and translate them (it didn’t do that at all), it cloned my voice and regenerated the audio. It also visually edited my mouth. The final product was that I look and sound like I’m speaking Spanish — in my voice.

It’s not perfect, but it’s pretty darn good. I could see YouTube, TikTok, and Meta building something like this into apps to make videos more accessible, regardless of the language you speak.

I’d love to hear from native Spanish speakers on the accuracy. What do you think?

Category: