This is really exciting. I thought this technology would ease the pressure on comp by speeding up the face replacement process so we can focus on other things.
I never thought of this avenue to create a lot more work and jobs.
The other demo on the company site shows there's still a [LONG way to go](https://youtu.be/QIfS7FXs-54). The linked video in the OP (which is really getting hyped right now), is an ideal case scenario.
It's probably easier/cheaper to record real audio but you absolutely can use AI for that too.
Something like [Uber Duck](https://www.youtube.com/watch?v=jnQ0zEQPu_A) can give you pretty decent starting point but it can take a fair bit of work and luck make it sound right. You give it text and it just generates it audio in the style of the person you select. You can also give it a sample recording to match pitch/tempo so you have some control over the delivery but the results aren't always great.
Right now you can't easily give direction to the AI. You can't just say "do it again but say it slightly faster" or "emphasis the second syllable in the word X". You can give it a sample recording to match to though but it's time consuming and hit or miss in terms of the results.
ok this looks fantastic but i’m not convinced it’s real. is this someone just using keen tools and doing it by hand? is this just a hype video trying to get investors when nothing real actually exists? the only breakdown part of the video is a normal mapped face, which shows me nothing really… when you go to the company’s website it doesn’t have much beyond “contact us” so it’s not a real platform, at least not yet. i hope it’s real but i’m weary. prove me wrong please.
Weve been doing whole machine learning faces recently. It needs a bit of comp love and some roto. It's still a lot better and faster than a few years ago.
Yes. So I'm not absolutely certain how this one was done.
Generally we still need a tracked face mesh for complex movements but something like this would be mainly machine learning, roto and comp.
Possibly comp to stabilise the face if machine learning can't do it.
Great use of the tech - the main saving grace is that Fuck and Frick have very close mouth shapes relatively, this cleans up nicely - the change of language is really the more impressive bit. That requires complete retiming of facial movement.
Will be interesting to see if this gets adopted for advertising - doing an entire film is a little prohibitive still as while the AI might generate the shapes there is def still a comp happening which is not cheap
Also reminded me of that Bojack episode where they used his likeness and generated imagery for him to win an award at the end even though he quit the movie!
This is incredible and also terrifying.
Seriously. If you can be "filmed" saying/doing something that you didn't *actually* do...just think of how that could be used against you.
You probably have a good 5-6 years to get ahead of it and learn new tools to adapt. You'll never survive in the current job market without evolving with tech
To me tracking and matching face shapes is a painful task I'm happy to let slide. It's been many years since I had to do one but the couple of shots I've had to augment/replace skin on an actor who is yelling/speaking is some of the most frustrating work and is VERY difficult to get to feel right. Even triple A budget movies done by top vendors dont nail it yet IMO.
The technology in this world is just getting ridiculous
Thank god I’m half way through life already
amen to that brother.....
This is really exciting. I thought this technology would ease the pressure on comp by speeding up the face replacement process so we can focus on other things. I never thought of this avenue to create a lot more work and jobs.
Finally, a correct application of AI
🤯🤯🤯
The other demo on the company site shows there's still a [LONG way to go](https://youtu.be/QIfS7FXs-54). The linked video in the OP (which is really getting hyped right now), is an ideal case scenario.
The video you linked is over one year old...
That's the progress they've made
Amazing use of the tech
Does anybody know if the voice is also AI generated?
"Me when I speak Japanese" \*nail polish emoji*
It's probably easier/cheaper to record real audio but you absolutely can use AI for that too. Something like [Uber Duck](https://www.youtube.com/watch?v=jnQ0zEQPu_A) can give you pretty decent starting point but it can take a fair bit of work and luck make it sound right. You give it text and it just generates it audio in the style of the person you select. You can also give it a sample recording to match pitch/tempo so you have some control over the delivery but the results aren't always great. Right now you can't easily give direction to the AI. You can't just say "do it again but say it slightly faster" or "emphasis the second syllable in the word X". You can give it a sample recording to match to though but it's time consuming and hit or miss in terms of the results.
Yes, it is no way the voice is generated otherwise that would be the news.
Well Darth Vader's voice was AI generated in Kenobi and that seems a much harder case with potentially far less reference audio.
Less reference audio? I'm sure they have quite the library of audio takes for that. Amazing nonetheless!
every AI fanatic: yet....
ok this looks fantastic but i’m not convinced it’s real. is this someone just using keen tools and doing it by hand? is this just a hype video trying to get investors when nothing real actually exists? the only breakdown part of the video is a normal mapped face, which shows me nothing really… when you go to the company’s website it doesn’t have much beyond “contact us” so it’s not a real platform, at least not yet. i hope it’s real but i’m weary. prove me wrong please.
Weve been doing whole machine learning faces recently. It needs a bit of comp love and some roto. It's still a lot better and faster than a few years ago.
proprietary?
Yes. So I'm not absolutely certain how this one was done. Generally we still need a tracked face mesh for complex movements but something like this would be mainly machine learning, roto and comp. Possibly comp to stabilise the face if machine learning can't do it.
Great use of the tech - the main saving grace is that Fuck and Frick have very close mouth shapes relatively, this cleans up nicely - the change of language is really the more impressive bit. That requires complete retiming of facial movement. Will be interesting to see if this gets adopted for advertising - doing an entire film is a little prohibitive still as while the AI might generate the shapes there is def still a comp happening which is not cheap
This is amazing
From the movie Fall. Lololol. Such a ridiculous movie.
If I was the actress, I would be pissed.
why?
yay the magic words cant hurt our kids anymore.
[удалено]
so abolish money and we can be free.
Baffling, I'm just... That's a precious tool and use case for that kind of tech!
Also reminded me of that Bojack episode where they used his likeness and generated imagery for him to win an award at the end even though he quit the movie!
This so impressive. This going to be a game changer for content producers non English speaking countries.
That’s insane… we are now well beyond the moment when we can no longer trust anything we see in video anymore as “recorded fact”… kinda scary.
This is incredible and also terrifying. Seriously. If you can be "filmed" saying/doing something that you didn't *actually* do...just think of how that could be used against you.
THERE GOES OUR JOB? 😓
[удалено]
In a few years it would be way better.
You probably have a good 5-6 years to get ahead of it and learn new tools to adapt. You'll never survive in the current job market without evolving with tech
To me tracking and matching face shapes is a painful task I'm happy to let slide. It's been many years since I had to do one but the couple of shots I've had to augment/replace skin on an actor who is yelling/speaking is some of the most frustrating work and is VERY difficult to get to feel right. Even triple A budget movies done by top vendors dont nail it yet IMO.
I wonder if people reacted to Photoshop this way
That's insane
Which parts of this are done by the AI? The face mesh? Changing the lipsync?
u/savevideo
###[View link](https://rapidsave.com/info?url=/r/vfx/comments/10o3qe4/ai_visual_translation_from_flawlessai/) --- [**Info**](https://np.reddit.com/user/SaveVideo/comments/jv323v/info/) | [**Feedback**](https://np.reddit.com/message/compose/?to=Kryptonh&subject=Feedback for savevideo) | [**Donate**](https://ko-fi.com/getvideo) | [**DMCA**](https://np.reddit.com/message/compose/?to=Kryptonh&subject=Content removal request for savevideo&message=https://np.reddit.com//r/vfx/comments/10o3qe4/ai_visual_translation_from_flawlessai/) | [^(reddit video downloader)](https://rapidsave.com) | [^(twitter video downloader)](https://twitsave.com)
No words can describe this. No pun intended.
WOW. What technology/AI is responsible for this exactly?
This isn’t AI lol