LipDub AI

31 days in October - that extra day makes it a busy month in marketing. Check out this months Top of the Marketing Charts. Leave comments with thoughts, what you disagree with, have a different perspective on, what I missed.

I got access to LipDub AI beta, its able to translate into different languages, I think it can do more, but I've not had time to look at it all yet. What do you non english speakers?

Newest AI Mystery 👇 Tried a more "topical" subject 🤣 Results, longest watch time and engagement so far. As always, building AI entertainment in public. Comments welcome. Jeffrey Dates Elliot Wolf Leo Kadieff Jennifer Marrero Tech: Images: FLUX (realism LORA) Video: KLING AI Narrator Video: Runway LipDub AI Voice: Cartesia Edit: Adobe

Spent a few hours this weekend tinkering with AI tools to scale video content, and the results were quite promising! Check out the three different vlog variants generated from a single stock (audio-less) video. Process: a. Started with a stock clip of a person walking down the street, talking on a mic - thanks, Pexels! b. Used Runway's video-to-video to transform that clip for different cities and weather. Imagine him strolling through Tokyo during cherry blossom season or navigating the snowy streets of Milan or Munich. c. Got destination specific scripts using OpenAI i.e chatgpt. d. Generated voice-overs using ElevenLabs standard voice characters. e. Lip-synced the transformed videos using LipDub AI. Super impressed with how well Lipdub tracks the mouth & face - even while the subject's walking and the lighting keeps changing! Far superior to the outputs I got from Runway’s lip-sync feature. Why It's Exciting: Thanks to AI, we can now produce multiple variants of the same source video for A/B testing, hyper-personalization, and more, all at a fraction of the cost and time. And we're not just lip-syncing a few variables on the same visuals anymore. We can create entirely new visuals with completely different scripts.  All existing branded content and footage become reusable assets! The videos aren't perfect yet, but imagine how slick they'll be a year from now. Lot's of potential benefits for brands and marketers. Kulfi Collective Akshat Gupt Kunal Prabhu

📲 𝐄𝐧𝐝-𝐭𝐨-𝐞𝐧𝐝 𝐀𝐈 𝐠𝐞𝐧𝐞𝐫𝐚𝐭𝐞𝐝 𝐜𝐥𝐢𝐩𝐬 𝐰𝐢𝐭𝐡 𝐩𝐫𝐞𝐜𝐢𝐬𝐞 𝐥𝐢𝐩-𝐬𝐲𝐧𝐜 👄 You will no longer read subtitles in the films when seeing this. 😯 This clip started as an image in Ideogram, upscaled with Magnific AI, out of which the video was created in Runway Gen-3 Turbo. 🚀 But on top, ElevenLabs made the voice and LipDub AI the awesome precise lip-sync. 💪 Isn't reading the subtitles just too annoying? 💬 Credit: Ryan Phillips 👉 𝐋𝐢𝐤𝐞𝐝 𝐭𝐡𝐢𝐬 𝐩𝐨𝐬𝐭? 𝐅𝐨𝐥𝐥𝐨𝐰 𝐦𝐞 (Jiri) for more AI and digital tech 🔔 #ai #genai #generativeai #lipsync #deepfake #translation

🎥✨ Comparative Overview of Lipsync Tools: LipDub AI, Runway, KLING AI ✨🎥 Finally found the time to explore and compare different lipsync tools. I’m excited to share this video where I put Lipdub, Runway, and Kling to the test (in this order in the video). Workflow used: • The initial image was created with Midjourney. • Enhanced via Magnific and animated with Runway (Gen3 Turbo). Prompt: “Stationary camera slowly zoom out, emotional speaking, natural expressions, natural light”. • Audio, same for all, generated using ElevenLabs. For Kling, the entire video was created in-platform. My thoughts on each software: 🔹 LipDub AI: The result is remarkable: lipsync is highly accurate, and the overall quality is superior to its competitors. However, the processing time is very long, requiring 3 hours to generate 10 seconds of video, making it challenging for rapid iterations and edits during the preview phase. My suggestion? A “quick generation” mode would be extremely useful for previewing results before committing to a final render, especially considering the high cost (24 training credits and 8 for generation). Also, note that the final video presents a slight color shift compared to the original image. 🔹 Runway: Offers one of the fastest and most cost-effective solutions in terms of credits consumed. The final result can be a bit blurry, but with some regenerations, you can achieve an acceptable lipsync, especially given the balance of quality, cost, and time. 🔹 KLING AI: Kling has recently released its lipsync feature, which comes with some impressive characteristics. I used the same prompt and audio file as with Runway but created everything within the platform. The final video appeared darker than the original image, but the rendering time was around 10 minutes, which is reasonable. What stood out to me was the realism, particularly the muscle movements in the neck, which add a convincing depth. The only thing off was the “speed” of the speech, which, while consistent with the audio, sometimes felt sped up, giving an artificial effect. Despite this, the outcome places Kling between Runway and Lipdub in terms of quality. Every tool has its pros and cons and should be chosen based on the specific project requirements. There’s no “right or wrong” choice—it’s all about understanding each tool’s strengths and limitations. I hope this overview helps you navigate the lipsync world! I intentionally left out ComfyUI from this comparison, as I wanted to focus on ready-to-go and easily accessible tools. 🔔 If you’re interested in more content about AI and innovation, follow me to stay updated! 🚀 I’m also curious to hear from you—what lipsync tools have you tried, and which ones work best for you? Let me know in the comments below! 👇 #Lipsync #ToolComparison #AI #Lipdub #Runway #Kling #Innovation #Midjourney #Magnific #Elevenlabs #Animation #Creativity

Lip syncing without reshooting, using LipDub AI I uploaded a video to LipDub AI of myself talking to camera for about a minute. I then uploaded a separate audio clip. The tool merges the new audio to the actor. What this means for film makers and content creators is you don't need to reshoot any video if you want the actor to change what they are saying. Let's say you have an ad campaign and the actor says the wrong line, but you only to realise this afterwards. All you need to do is change the audio and LipDub AI will seamlessly blend the video to the audio. You can change the language or voice and dialog to whatever you need it to be. What I really like is how it keeps the expressions sync'd to the audio. Game changer for many use cases. Can you spot the movies I used for the audio? Andrew More - Thanks for sharing this!

LipDub AI is the best 3rd party lip-syncher out at the moment. The video below was generated with Runway Gen-3 Turbo from a Ideogram image upscaled in Magnific AI and eleven labs voice changer. I was given some credits by LipDub to run some tests. Sound on please! Youtube in 2k: https://lnkd.in/ee2wrcJZ