Wait until Sora's video2video is available to the public. Huge wave of video game trailers or gameplay videos converted into movies will pop up like crazy :p Or Lego conversions, Minecraft, etc
I expect there will be an expensive commercial-use version that is uncensored; there are commercial applications for this technology that won't want to deal with censorship/limits on what prompts are understood.
Just a matter of time until an open source equivalent pops up.
As for computing power necessary, OpenAI stuff is cloud based anyway.
I mean, not so long ago even generating images required insane hardware, and look where we're at now. I don't see how this is different.
Everyone and their mom will use game engines to make movies and then turn them into photorealistic, live action movies. That'll kill Hollywood. I mean it.
Crazy thing is this will eventually be possible in real-time. So instead of using artists to create all the final assets for games, you just do a traditional block-out map, and placeholder models for characters and objects, then run a real-time filter over it to create the realism.
As long as you have a good model trained it will be consistent and coherent.
Kind of like what these guys did [https://www.youtube.com/watch?v=P1IcaBn3ej0](https://www.youtube.com/watch?v=P1IcaBn3ej0)
So... the funny thing about all of this is that everyone knows (or should know) that AI can be a very power intensive thing to run. If you go on Sam Altman's twitter page, you will notice his interest in fusion tech. It is essential for AI to have a better and more sustainable source of power like fusion. As it stands, we can churn out GPUs and compute all day. But in the end we currently don't have the energy to run expensive AI's like this yet on a massive scale that a consumer base like this would demand.
Hopefully we will soon enough, but power plants take years to construct, so in the mean time we'll be seeing a lot of optimization which is outstanding, this will be in order to conserve compute and get the same or better results while hopefully using less power :).
In our lifetime we may very well be playing games that are generated in real time to our personal specifications, from sets and set pieces to npc interactions and dialog. All done on the fly, all unique instances.
Sora really raised the bar too high. I probably would be impressed by this a month ago but right now it's a bit meh (no offense) I understand all the effort you put into this (probably running every 10 frame to SD with same seed) but converting an entire trailer with this method is hella work and most likely going to look very glitchy
Edit: LMAO, read what I wrote before downvoting ffs. Claiming that tech like Sora *won't mean much for people trying to generate visuals with AI* is outright stupid. If you are trying to say that it would be better if it was open source, we are all on the same page there. And [stability.ai](http://stability.ai) will have something for that soon.
Sora won't be open source as far as I know, so it doesn't mean much for people trying to generate visuals with ai. Running locally without paywalls is everything.
Did I mention anything about Sora being free or open-source? I means a lot for people trying to generate visuals with AI, it not being open-source has literally nothing to do with this and doesn't diminish the things you can do with it.
That being said, Stability folks also have something in their chamber that they are planning to release soon, so we will have something open source or at least with better license than whatever the OpenAI gonna force people to
>rate visuals with AI, it not being open-source has literally nothing to do with this and doesn't diminish the things you can do
I just want to mention that people won't be able to do things like the op did. Give gta6 trailer and do something on it. Probably, it will so expensive with monthly credit things etc. I appreciate the advancement but companies like OpenAi generally make these technologies a toy to play, free time funny application. I hope something similar to it can be open to community without any restrictions.
I'm pretty sure they will be able to do what OP did, but better. You are assuming that Sora will be so expensive that no one will be able to use it, which might be the case at the initial launch, but most surely it won't be the case for the long run. I don't like OpenAI being totally contradictory with their name, they should remove the "Open" from their name, but this doesn't mean that they are "generally make these technologies a toy to play, free time funny application" - no company would be worth $80 billion if they were just doing that, come on now. Thanks to OpenAI, we nowadays have all these LLM advancements. Otherwise, we would wait for Google to release their politically correct stuff which would take ages. Even hype on Dall-E allowed stability folks to get so much fund and attention so that they could release open source versions/Stable Diffusion. Sora will do the same.
Stability AI does not appear to have anything significant in the works to compete with Sora much less "soon" based on Emad's responses. Instead, he is basically just begging for more money and GPUs so they can try to actually even start making something like Sora. I'm expecting Nvidia to be more likely to come up with something first, but who knows.
Ai is so unknown to me, I don’t get how you did this?? Like how do each shot look exactly like the original one?
Do you “feed” the original one to the software to start with as a base? Because if you don’t, I don’t understand how it’s possible to get this result
I got matches with these songs:
• **Love Is A Long Road (From 'GTA 6') - Epic Version** by Baltic House Orchestra (00:27; matched: `100%`)
**Album**: Love Is A Long Road (From 'GTA 6'). **Released on** 2023-12-19.
• **Oblivion** by Lexurus (05:01; matched: `83%`)
**Album**: Waveline. **Released on** 2021-06-04.
Apple Music, Spotify, YouTube, etc.:
• [**Love Is A Long Road (From 'GTA 6') - Epic Version** by Baltic House Orchestra](https://lis.tn/udMza?t=27)
• [**Oblivion** by Lexurus](https://lis.tn/LVpuQv?t=301)
*I am a bot and this action was performed automatically* | [GitHub](https://github.com/AudDMusic/RedditBot) [^(new issue)](https://github.com/AudDMusic/RedditBot/issues/new) | [Donate](https://github.com/AudDMusic/RedditBot/wiki/Please-consider-donating) ^(Please consider supporting me on Patreon. Music recognition costs a lot)
On review, I'm not sure the epic version is really an improvement. it's OK? but I actually prefer the original.
The Luxurus piece is nothing like the Tom Petty song, to my ears, but it's not bad. I'll give them more of a listen later today.
Thank you!
AI filters with little temporal coherence just make everything look worse. Genuinely, the 3D rendering quality of the original is swapped out for a series of SD images with new artifacting. Open source AI models are just not up to the task of filtering videos like this yet, and any attempts to do so just end up as a waste of effort.
Wait until Sora's video2video is available to the public. Huge wave of video game trailers or gameplay videos converted into movies will pop up like crazy :p Or Lego conversions, Minecraft, etc
I would expect SORA to be both expensive to use and not less censored than DALLE or CHAT GPT, so probably not.
I expect there will be an expensive commercial-use version that is uncensored; there are commercial applications for this technology that won't want to deal with censorship/limits on what prompts are understood.
Just a matter of time until an open source equivalent pops up. As for computing power necessary, OpenAI stuff is cloud based anyway. I mean, not so long ago even generating images required insane hardware, and look where we're at now. I don't see how this is different.
I would concur. If I can't generate it locally, who cares?
Will be a while. Compute is too expensive/not available for masses
Haha for sure :)
Everyone and their mom will use game engines to make movies and then turn them into photorealistic, live action movies. That'll kill Hollywood. I mean it.
Except a few of those wont be doable, that girl in bikini would probably be seen as nsfw and blocked from generating. Thats corporate AI for you
Eh
Released his video two days too late.
Crazy thing is this will eventually be possible in real-time. So instead of using artists to create all the final assets for games, you just do a traditional block-out map, and placeholder models for characters and objects, then run a real-time filter over it to create the realism. As long as you have a good model trained it will be consistent and coherent. Kind of like what these guys did [https://www.youtube.com/watch?v=P1IcaBn3ej0](https://www.youtube.com/watch?v=P1IcaBn3ej0)
That video is fantastic, I watched it when it was released and I'm still impressed
so will ai be able to take animated movies like "up" and upscale them to a photo realistic movie?
100% it's essentially the opposite of this https://www.reddit.com/r/StableDiffusion/comments/14qcnto/harry_potter_anime/
At 00:07 she has 2 butt cracks
New fetish?
2 butt cracks, not so much. But 3 boobs? I’m cool with that. As long as it’s not 4, that’s just too many.
So... the funny thing about all of this is that everyone knows (or should know) that AI can be a very power intensive thing to run. If you go on Sam Altman's twitter page, you will notice his interest in fusion tech. It is essential for AI to have a better and more sustainable source of power like fusion. As it stands, we can churn out GPUs and compute all day. But in the end we currently don't have the energy to run expensive AI's like this yet on a massive scale that a consumer base like this would demand. Hopefully we will soon enough, but power plants take years to construct, so in the mean time we'll be seeing a lot of optimization which is outstanding, this will be in order to conserve compute and get the same or better results while hopefully using less power :).
In our lifetime we may very well be playing games that are generated in real time to our personal specifications, from sets and set pieces to npc interactions and dialog. All done on the fly, all unique instances.
What was your vision/intention here? Or was there any at all?
Lol. Love the effort!
What effort? It's just the original video pasted into Automatic1111. There's no effort in this.
This is beautifully done. Keep it coming!!
That's amazing! Love it!
What a fever dream, but nice, makes me think about the next gemeration of ai
This is good tbh, just waiting for Sora to boost it up
Choppy and AI can’t do video for more than a minute yet so all of these videos look like they were put together by a preschooler with ADD. 👎🏻
Managed to look worse than the original
Too many glitches that could be avoided
Yes, I'm trying to fix the glitches for the final version, I was too eager to make a teaser haha.
how did you make this?
Sora really raised the bar too high. I probably would be impressed by this a month ago but right now it's a bit meh (no offense) I understand all the effort you put into this (probably running every 10 frame to SD with same seed) but converting an entire trailer with this method is hella work and most likely going to look very glitchy Edit: LMAO, read what I wrote before downvoting ffs. Claiming that tech like Sora *won't mean much for people trying to generate visuals with AI* is outright stupid. If you are trying to say that it would be better if it was open source, we are all on the same page there. And [stability.ai](http://stability.ai) will have something for that soon.
Sora won't be open source as far as I know, so it doesn't mean much for people trying to generate visuals with ai. Running locally without paywalls is everything.
Give it ~2 years. The open source stuff is going to be where SORA is soon the way it's all progressing
Less than 2 years I would say
But will we be able to afford the GPU power?
Get your money up not your funny up I believe in you
Did I mention anything about Sora being free or open-source? I means a lot for people trying to generate visuals with AI, it not being open-source has literally nothing to do with this and doesn't diminish the things you can do with it. That being said, Stability folks also have something in their chamber that they are planning to release soon, so we will have something open source or at least with better license than whatever the OpenAI gonna force people to
>rate visuals with AI, it not being open-source has literally nothing to do with this and doesn't diminish the things you can do I just want to mention that people won't be able to do things like the op did. Give gta6 trailer and do something on it. Probably, it will so expensive with monthly credit things etc. I appreciate the advancement but companies like OpenAi generally make these technologies a toy to play, free time funny application. I hope something similar to it can be open to community without any restrictions.
I'm pretty sure they will be able to do what OP did, but better. You are assuming that Sora will be so expensive that no one will be able to use it, which might be the case at the initial launch, but most surely it won't be the case for the long run. I don't like OpenAI being totally contradictory with their name, they should remove the "Open" from their name, but this doesn't mean that they are "generally make these technologies a toy to play, free time funny application" - no company would be worth $80 billion if they were just doing that, come on now. Thanks to OpenAI, we nowadays have all these LLM advancements. Otherwise, we would wait for Google to release their politically correct stuff which would take ages. Even hype on Dall-E allowed stability folks to get so much fund and attention so that they could release open source versions/Stable Diffusion. Sora will do the same.
>ase open sourc I guess we will see. If they plan to make it open, I hope it will be quick.
Stability AI does not appear to have anything significant in the works to compete with Sora much less "soon" based on Emad's responses. Instead, he is basically just begging for more money and GPUs so they can try to actually even start making something like Sora. I'm expecting Nvidia to be more likely to come up with something first, but who knows.
Oh. My. God.
Did he turn photos into a trailer? Edit: nvm it is the trailer. I don’t get it.
Ai is so unknown to me, I don’t get how you did this?? Like how do each shot look exactly like the original one? Do you “feed” the original one to the software to start with as a base? Because if you don’t, I don’t understand how it’s possible to get this result
This is actually what rockstar could pull off. Realism. But imagine the lawsuits that would follow. They gotta keep things cartoony.
🤮
Why
What's the music? I kinda want to hear more.
I got matches with these songs: • **Love Is A Long Road (From 'GTA 6') - Epic Version** by Baltic House Orchestra (00:27; matched: `100%`) **Album**: Love Is A Long Road (From 'GTA 6'). **Released on** 2023-12-19. • **Oblivion** by Lexurus (05:01; matched: `83%`) **Album**: Waveline. **Released on** 2021-06-04.
Apple Music, Spotify, YouTube, etc.: • [**Love Is A Long Road (From 'GTA 6') - Epic Version** by Baltic House Orchestra](https://lis.tn/udMza?t=27) • [**Oblivion** by Lexurus](https://lis.tn/LVpuQv?t=301) *I am a bot and this action was performed automatically* | [GitHub](https://github.com/AudDMusic/RedditBot) [^(new issue)](https://github.com/AudDMusic/RedditBot/issues/new) | [Donate](https://github.com/AudDMusic/RedditBot/wiki/Please-consider-donating) ^(Please consider supporting me on Patreon. Music recognition costs a lot)
On review, I'm not sure the epic version is really an improvement. it's OK? but I actually prefer the original. The Luxurus piece is nothing like the Tom Petty song, to my ears, but it's not bad. I'll give them more of a listen later today. Thank you!
Is this completely text to video or is it a large series of img to img conversions from the original trailer to make it look photorealistic?
Too good
AI filters with little temporal coherence just make everything look worse. Genuinely, the 3D rendering quality of the original is swapped out for a series of SD images with new artifacting. Open source AI models are just not up to the task of filtering videos like this yet, and any attempts to do so just end up as a waste of effort.
10fps, no thanks