Day 10 – Translate Text, Audio and Video – 30 Day AI Challenge 2024

Welcome to the 2024 Edition of the #30DayAIChallenge. This year, we’ll be focusing on making Generative AI more accessible for all. So follow along and give each challenge a try yourself, plus share your own thoughts and experiments with a community of like-minded learners and tinkerers!*

Week 2‘s theme is “The Fast and the Curious: AI Drift“, focused on productivity and designed to help speed up tasks, plus reduce time spent on the mundane, both at work and home.

Today’s Challenge – Translate Anything, Anywhere

Ever since I was a child, I’ve been absolutely enchanted by all things Star Trek. Who wouldn’t be? The series offered a captivating, utopian vision of the future where humanity is united in its quest for scientific advancement and exploration. Among the futuristic innovations, the Universal Translator stood out—its ability to bridge divides across languages, cultures, and even species seemed nothing short of magical. Today, with AI capabilities advancing rapidly, it feels like we are on the brink of turning this science fiction dream into reality.

On Day 10 of our challenge, we harness the power of Generative AI to translate text, audio, and video. But this challenge goes beyond mere word-for-word translation; it’s about capturing context, culture, and the subtle nuances that infuse communications with meaning. As Generative AI continues to evolve, we find ourselves at a turning point where content becomes more accessible, workflows are streamlined, and previously insurmountable barriers between us begin to crumble.

For Beginners: Getting Started with AI Translations

1. Text Translation:

• Tools to Try: Google Translate, DeepL, and Microsoft Translator are great (older) starting points. These platforms use AI to improve translation accuracy over time. The great thing about most cutting-edge Large Language Model (LLM) powered AI chatbots is that they can handle lots of different languages, plus are great at understanding context, so you can now just use your favourite chatbot!
• Typically, for the best results, you need to provide clear, context-rich sentences, avoiding slang or idioms that may not translate well. With the latest chatbots though, they’re getting pretty good to understanding those too (see example below, where I asked Google Gemini to explain what a German saying meant).

2. Audio Translation:

• Tools to Explore: Voice Translator apps and services like iTranslate Voice or Google’s Transcribe feature in their Translate app. Similar to text above, you’re also in good hands with LLM powered chatbots, especially those like ChatGPT and Pi (see below) that do a great job of voice conversations on their mobile apps…pretty handy for when you’re on the go and need to translate something by voice. For critical, professional uses, make sure you test accuracy (perhaps ask a native language speaker) before relying on the translation.

You can also tap into tools like ElevenLabs to create authentic spoken audio in different languages.

…or simply using their Dubbing Studio to translate content across 29 languages (currently) with voice translation, speaker detection, and audio dubbing.

3. Video Translation:

• Starting Point: YouTube offers automatic captions and translations for many videos, which is a good example of AI in action…plus the service aims to add AI powered dubbing very soon. There are also a lot of tools that now do all the heavy lifting in terms of video translations and dubbing, including lip-syncing to the target language! You’ll find a great example of this in the video below, created using HeyGen – I used an English video and converted parts of it to Spanish and Hindi.

Keep in Mind:

• AI translations are getting increasingly sophisticated, but based on training data or model proficiency, can sometimes miss the mark on cultural nuances or idiomatic expressions.
• Always review translations for accuracy when possible, especially for important communications.

Our Test Challenge

To test out these capabilities, here’s a quick challenge:

Write a short story, translate it into a different language and create an AI powered voice narration.

Step 1: Writing the story in English using an AI chatbot…I used Claude.

Step 2: Translating the story into a different language, say Spanish.

Step 3: Using a tool like ElevenLabs read out the story using a preferred voice.

The final result…what do you think? (I created a supporting image using Dall E 3)

For Advanced Users: Enhancing AI Translation Workflows

1. Custom AI Models:

• Train custom models on your specific domain or industry jargon to improve accuracy. Tools like Google Cloud Translation and AWS Translate allow for such customization.

2. Integrating APIs:

• Automate translations by integrating translation APIs into your content management systems, apps, or websites. This can streamline content production and make your services more accessible globally.

3. Advanced Audio and Video:

• Explore tools like Descript for editing translated audio transcriptions or Adobe Premiere’s auto-caption feature for video editing. These can save time and enhance accuracy in multimedia translations.

Innovative Uses:

• Develop multilingual support systems using AI translation to serve a global customer base.
• Create dynamic, translated subtitles for live events or streaming using AI-powered services.

Conclusion: The Future of AI-Powered Translations

The potential of Generative AI in breaking down language barriers is immense, offering a glimpse into a future where language differences no longer impede understanding. As technology advances, we can expect even more accurate, nuanced translations, further enhancing global communication. However, embracing this technology also requires a mindful approach, especially regarding cultural nuances and accuracy.

I’m still holding out hope for that Star Trek inspired Universal Translator…though perhaps a “Global Translator” may be a good place to start! 😄

If you give any of these tools a try, leave a comment and share your experience or creations! If you’re using other platforms, please tag your Facebook, Twitter / X or LinkedIn post with #30DayAIChallenge so others can find it too.

Till tomorrow…

*Please note: Participation in the 30 Day AI Challenge is at your own discretion and responsibility. Always ensure that no sensitive personal information, confidential, or proprietary company data is shared. Adhere to all applicable local laws and company policies. Enjoy exploring AI responsibly!