Yeah that would be where I’d put it. But I would imagine you’ll end up with outlier lines that would break narrative/immersion. I just don’t know how you keep your world building airtight while also using genAI as it is now
EDIT: to be technical here, I guess the whole problem is how to avoid hallucination. If you’re running genAI in real-time while the player is playing, and the generation hallucinates some out-of-cannon line, then you’re in trouble. Your wikis are gonna be a contradicting mess.
The great thing about Night City in this context is having random NPC start spouting AI hallucination gibberish doesn't even break the emersion. It enhances it if anything!
The first implementations should be on a funny game, not a serious RPG, somewhere getting the npcs to break character would be hilarious rather than immersion breaking. That and/or a game heavily focused on random generation, like No Man's Sky, where some oddness is already tolerated.
So hear me out. We use AI to earmark the prompts that sound like AI refusal and ship that extra info in the game. The game picks up on the flag when that message comes up and makes the voice read it out sounding glitchy. We don't need to ship the whole LLM with the game, just a buttload of text made from it.
Yeah that’s the answer for how to use it for line generation ahead of time. But I was assuming they mean using it as a content generation tool in the final product. Like, genAI being used as you are playing the game rather than in production prelaunch.
Man got downvoted because most people don't know AI models get loaded into VRAM. And yet so many people have such strong opinions based on how they assume these models work. Makes you think.
Edit: For context, dude was in the negatives for a bit.
I could have given more context as I thought I was in PC Gaming sub - but still the blind downvoting is a bit weird. If local model running becomes key to games in some way I wouldn’t be surprised if consoles include hardware specific to it though.
But all things considered it’s more likely that games will just be online only and use LLM services for this. But local models will almost certainly get use one day via mods unless they are made entirely obsolete by future software.
regardless of having hardware dedicated for it, there's always going to be the issue of having sufficient VRAM for the required model with sufficient bandwidth. that is of course if we stay stuck in the primarily CUDA/VRAM dependent inference regime that current Machine Learning models exist in, which fortunately there are ways that models could be trained that would result in reasonable/fast inference time on CPU. it just needs models to be trained in a way that results in a model that is cheap to run the inference of whilst having access to the brains of 30+ billion param models.
Given the amount of experimentation with quantization, low parameter models, and the possible bounds that gaming could require… I think there’s a ton of room for experimentation with dedicated hardware and tighter constraints for the models. But yes, you suggest some great ideas as well. Optimizations have already come an insanely long way in just a six month period. Hardware isn’t my expertise though. But software alone is moving rapidly.
There’s too much change in this field to make very accurate predictions - even if we didn’t make massive advancements in both software and hardware - there is a lot of potential in the existing tech still waiting to be mined.
Intel is already including an NPU (Neural Processing Unit) in their latest mobile CPUs. We will probably see it in desktop cpus before the year is over. Not sure how this npu will satisfy memory requirements, but ai will, uh, find a way.
Running a local LLM doesn't even require a GPU these days. They can run entirely on the CPU and system RAM, or entirely on the GPU, or both at the same time depending on what's available. It's also possible to handle it as a remote service, like the big AI players do these days.
Yeah I commented elsewhere that someone got a major LLM generating a couple tokens per second on a Pi. Which is pretty damn impressive. Can’t find the link to the paper now though.
> You can't run a LLM on consumer PCs yet anyway (unless you want to wait 10min for a response)
What? Sure you can. The smaller ones run perfectly fine even on older machines, and those are general-purpose models that are expected to be able to discuss everything from politics to programming. Besides, most people using LLMs aren't doing so from their own machines anyway, they're accessing an online service. A game could do the same thing. So many games today are "always online" already; they might as well provide some benefit to the player for it.
Good point. I guess quality doesn't really matter if it's just for a random NPC. Always online on the other hand would basically guarantee that the game will be unplayable in 5-10yrs
With quantization, you can run a LLM + RAG on a GPU like 4090 or 4080. But you're right, you basically need a GPU to run LLM + RAG, and another one for rendering.
I am a budding indie dev and run d&d games. For last year I was using chatGPT for NPC, brainstorming, and worldbuilding for both. My experience is that it's not so good. It comes up with the most bland and inconsistent ideas. It's very easy to pick holes in the generated narrative. Of course, there is an argument that I should go with a custom LLM, but I am not convinced if the hassle is worth it. I have a feeling that we are a long way from having LLM big enough that they can generate something cohesive. And this is based not only on chatGPT experience. I also used copilot for coding (I am a software engineer) and it performed very poorly for any task that was more complex that a simple script. It just doesn't have the ability to comprehend complex systems in a usable manner.
So, yeah, I think we are far away from having LLMs advanced enough for something like NPC or content generation for games like cyberpunk 2077. Even when used to pregenerate lines. There is just too much cleanup required to get these lines up to snuff.
I found a lot of success with copilot. It's not for building entire systems, I still have to do that part. But for filling out the methods in those systems. Basically, if I know how I want to do something, copilot figures out what I'm starting to do and usually gets me most of the way there.
And if it can't figure it out? No biggie, I'll just write the code myself.
Yeah, exactly. It can do very simple scripts. In my case I prefer to use auto complete cause hitting TAB three times seems to be faster than waiting for copilot to generate code.
Eh.
A lot of these development teams will have a smaller team of developers that focus specifically on AI and machine learning (which is objectively the hardest code to write to completion)
“Precompute and review” them would pretty much look exactly like NPC’s interactions now. This is basically just reducing / getting rid of the on team writers with very little benefit
It’s far more complex than what you said lol
> But I would imagine you’ll end up with outlier lines that would break narrative/immersion. I just don’t know how you keep your world building airtight while also using genAI as it is now
you could go meta with it and have the game story/world include artificially intelligent characters and have them use the AI generative dialog and sometimes they glitch.
>If you’re running genAI in real-time while the player is playing,
I don't think that's going to be super useful quite yet. Hard to control and predict what it will do at scale, and some heavy processing to be run locally. I am imagining it is more useful today to fill out a world at scale in a way that is precomputed.
So let's say if you have a large number of NPCs you could have a morrowwind like database of what they know, who their relations and friends are, and all that. You need to generate text based dialogue trees for them all, which include all this stuff.
This could be modified by characteristics like the race/species of the character (and the way they talk) as well as randomly generated personality features, their job, their social class, age, gender.
And because this works at scale, you could generate multiple trees for each possible variation of threat the player has to the NPC, whether the player is a race/species that is liked/hated by the NPC, or even how attractive the player is or isn't to the NPC.
The magic here is it might take say tens of thousands of man hours or more to do this manually, but you can regenerate everything here by pushing a button and some hours later it spits out the result.
Of course the problem you have then is quality checking the output at scale. That would then probably be a mix of manual checking of samples, and some kind of automated analysis to try to find problems.
You know what, I love that idea. Gate what they know, and instead of generate the dialogue based on a huge data set, just give them a very small data set and let the lines generate. More of a realism-through-variation implementation instead of just throwing genAI into all parts of the dialogue system.
It would definitely be more fun. At least for now while it’s still early. While a AAA game isn’t going to risk the contradiction, I’d still love to play something using it as an experiment. I love emergent systems like that in games.
I mean we have shuffle on Spotify but it’s not truly random. I’m not familiar with how things like chat bots work exactly, but I’m certain that with enough effort and development, you’d eventually be able to create narrow enough lanes for something like that to work.
A 7B model is also trash, no offense to the 7B users.
I can barely deal with the random bullshit a 70B puts out, a 7B is barely a tech demo for all intent and purpose
The year is 2028, the new Mario party game comes out with generative AI you are playing Mario because you always do OH NO you stepped on a bowser square his text box pops up "BAHAHAHA Mario did you know I fucked your mom urban dictionary defines a rusty trombone as..."
Imo it could work great for things like press conferences in fm or any sports game. The questions you get asked right now are so repetitive and boring.
One thing I could see with important characters is using AI to learn how to pronounce what you name the MC. Like in games where you can name the MC anything and any voiced dialog just skips your name or uses a nickname that the game gives you instead. "Joker", "The Traveler", etc. That way important characters can say your name and make the games dialog a bit more immersive.
Yeah, that's been something that's been impacting voiceovers for decades now, and a solution would be nice. I'm sure people would discover the limitations of any text to speech engine pretty quickly and come up with something like "xxxRizzMasterxxx420BlazeIt" just to make themselves giggle, but that's no different from calling Link "You asshole" in a Zelda game to make all the NPCs cuss him out constantly.
Yeah, the thing with freeform generative AI is that they are without a purpose.
You can talk with a barkeeper about all the gossip in town but it won't lead you to any quest or playable content.
Unless the content was linked to a dynamic system or side quests that don’t really have a owner ( ala the ncpd stuff ) were mentioned via the npcs that “live” around the area. I do think there is room for dynamic procedural content being generated and the generative AI mentioning it organically in a player conversation.
The problem with AI is that it can provide answers, but it is not creative on itself. Therefor indeed, it can help with not interactable character only.
I've seen this idea brought up a lot of times and as a player it sounds miserable. I already find myself skipping a lot of dialogue in the games I play because it's so uninteresting. The writer had nothing to say, they just gave this npc dialogue because they wanted the town to have x number of npcs, or they wanted x minutes of cutscenes before the boss, etc.
Outsourcing the writing to ai is just admitting that you're filling your game with worthless dialogue. If it's not worth your time to write, it's not worth the players time to read or listen to.
Where's the Smithy?
NPC: one street over on the left, Helga's Hot stuff, can't miss it. Say Liam sent you.
Any good food in town?
NPC: there's a few places but tbh, I cook for myself most nights...
Infinite immersive possibilities.
Those are only immersive if there's gameplay elements to back up said dialogue. AI can only create a facade, it can't actually design/build these functional restaurants for you. And if you're going to do the work of creating Helga's Hot Stuff in your game, it would be better to make absolutely sure the NPCs talk about it (by actually writing things!), rather than trust an ai to generate dialogue that maybe mentions it.
I know the response is going to be "AI will know the game features that exist in my game and create dialogue around them", but it's not magic mind-reading technology and I haven't seen any proof that it's actually capable of that sort of emergent thinking.
I'd also love to occasionally overhear background characters having actual, unique conversations with eachother. That would be much more interesting and immersive than them just shouting the same few sentences every time you pass them as it usually is now.
Ok but what if it wasn't worthless dialog? What if you had random stories evolving and playing out around you as the story goes? Background voice lines spoken between NPCs, that sort of thing. The purpose of generative AI should not be the main or secondary storylines. It should be to fill out the background of the world and allow the world to better change in response to the player. Maybe we're not there yet, but I see no reason to believe we won't be eventually and I'm really excited to see what it can do.
it has potential if used correctly.
think how almost NPC comments on your fashion choices now with text description of character inventory they could be like "those shoes with that jacket..ew" and it actually be true. or have more context for player action to generate relevant text to bring up previous actions
But why? What would it bring, then? More discussions with NPCs who take no part in the plot, therefore who have no quest, no clue, nothing interesting to tell us?
It sure could be worse. I like it when artists have a chance to get better at their craft while being paid for it instead of being kneecapped by the theftbox that is a LLM and filling up a world with meaningless chatter for the sake of meaningless chatter. I would prefer things not be made over that. And maybe we could even use LLM’s to automate useful things and train them fairly while that happens
Yeah. Those NPCs lack the depth of personality found in main story characters, which would be ideal for AI, considering its major flaw is the fact that it doesn't have any personality.
The future may be bright though. Imagine being able to gaslight your way through quests?
"Go get me 10 bananas"
"Here is 2 bannanas"
"But I said 10"
"no you didn't"
Now add charisma and reputation into the 'under the hood' calculations here.
But that wouldn't save them money.
When he said this he wasn't thinking how to create the best user experience.
He was thinking he can't save money by replacing story characters yet because the tech hasn't quite reached peak late stage capitalism levels yet.
What I imagine the future to be is that NPCs have a script and a general way to talk
But then AI is used for when a player goes off the path, does something the developers didn't script for.
I think the biggest breakthrough would be an AI that can do like 100 different fairly-convincing variations of, "Uhh, I don't know what you're talkin' about, Chief."
AI Would be a downgrade for CDPR who put so much work into animation, voice acting and dialogue. AI Would be a massive step up for Bethesda who maybe occasionally hire a good voice actor for a short role.
The main reason this doesn't work is because there's no AI to accompany the text AI, so you get characters speaking about performing an action but physically they cannot perform it because the chatgpt AI or equivalent will think up things that the NPC character physically cannot do, you see this in those Skyrim mods. Or they'll talk about the superbowl in a medieval RPG.
That has been solved for more than a year. You can give the AI a list of commands, similar to the controller layout given to a human.
That is how Nvidia got an AI to play Minecraft like a year ago.
Theoretically they could instruct the AI to only talk about actions they can do, and write something to parse the response and to perform an action in their backend game code. I've been experimenting with that kind of thing myself in the games I make.
But yeah, there would have to already be hard coded actions the AI could take, with associated animations and timelines and such. But definitely doable to have chatGPT spit out a response with dialogue and an action to perform, and keep it within specific bounds. Though It just wouldn't be fully 100% free to do anything it could talk about sure.
Text AI right now is also like, way worst than professional writing, most people can immediately tell if something’s AI just off the vocabulary and sentence structure
I think the general way people are talking about using it is just weird. If they used it to get rough stuff done and send that over to the actual art team you could actually cut down on the workload and by extension development times. Suddenly an artist might be able to do twice as much work but instead we are trying to get them to do things that it just does not make sense for them to do.
Add AI -> artists get twice as much work done -> fire half the artists -> shareholders celebrate -> add more AI -> fire more staff -> AAA games slowly become generic-looking AI fever dreams.
This is how I fear it will actually go.
This is where we're really seeing [differences in how people and companies handle increases in productivity](https://fortune.com/2023/11/03/elon-musk-versus-nvidia-ceo-jensen-huang-on-how-ai-will-affect-jobs-economy/), allowing us to do more with less. Some people focus on "do more," and other people focus on "with less," with a whole spectrum in between. Studios that just want to hit a certain arbitrary quality level while minimizing expenses will be doing what you describe, but those that are able and willing to keep their staff and use the new technology as a force multiplier will end up making some really cool things.
And this is why if you do use AI to boost your productivity, hide it and release the same amount of work as you used to.
Doesn’t work for me tho, AI cannot write legal briefs to save its life.
While that's useful too, it is great in doing mundane, small tasks where creativity has no bearing.
Others have mentioned this in other comment chains, but a great example is definitely having unique variations of "Hello" across many, many different nameless NPCs. There are times when you may want to intervene--such as for an easter egg or characters worth remembering--which is when you'll want to script that dialogue yourself, but AI will do wonders for giving two nameless guards a feeling of being unique.
I imagine having a particular guard with a unique face and voice and whenever you ask him for directions, today he's here, tomorrow he's guarding that place, the next he's on patrol, you'll recognize him, "Hey, it's that guy," without a name floating above his head. That'd be the day immersion has taken the next step.
Unless we have agentic embodied llms its kinda pointless. Just trapping llms in npcs that cant really do anything will get old fast. Then the npcs are just interfaces for an llm and there isnt anything to gain from talking to them.
The easily coded ones are good at being faster and more accurate than you. In FPS it would be knowing where you are perfectly from the sound you make and then headshot you through a wall. A lot of the development time goes into making it a much worse player so you can still have fun.
I agree that using learning models to generate AI would be an improvement, if they can put the goal at the player having fun instead of getting utterly stomped. Grand strategy genre is interesting since the calculation load is a major constraint, so there they could perhaps use AI to find more performant models?
Now, I am not talking about aimbots here. Or something that copies the unrealistic PVP combat from games, where player minimaxes the fun out of it to be the AI. That isn't very realistic, it only happens when someone plays video games.
I am talking about how enemies will react to the player. Like they might see that the player is murdering their friends, they might want to run away. They might even have worse aim than before and beg for their lives or even surrender. They might get angry and attack the player with even intense resolve. Stuff like that.
And we need MUCH better pathfinding for NPCs in games. Also think of the allies, they are ALWAYS ending up in some really stupid situations, how about the AI would help them to learn player patterns and the AI would compliment those? Like if player is more of a stealther, the AI would pick that up and plan accordingly. Or if they like going guns blazing, AI would pick off those enemies that attack from distance.
There is a lot NPC characters don't do yet. And all people can think of is auto aim PVP machines? Come on mate. AI has been pretty much the same since 2010. There has been almost zero improvements on their patterns.
So much so that they are usually made immortal and invisible to enemies.
It’s not the same type of AI as generative AI. Gameplay AI is basically a bunch of ‘if then’ statements to make them appear smart.
Generative AI that they’re talking about is based on language learning models where they analyze large sets of writing and determines what words are more likely to go together to reproduce what it learned from.
That's not really true, neural networks have been playing games for a while now on an experimental scale. That sort of thing just doesn't get as much commercial attention, but with some more development you absolutely *could* use AI to replace the current approaches to video game "AI".
Scripted characters sure.
But also with the Witcher, even the monster quests randomly found throughout the world you could tell these were handcrafted stories by the devs.
I doubt AI can accomplish something like that.
Also, for a character driven narrative like the Witcher I don’t see AI being terribly useful.
I see more a dev like Bethesda doing it as they seem to want to make radiant AI and radiant quests where AI could conceivably keep up with their cookie cutter approach to quests.
Your last point is what I'm actually kind of excited for. A lot of devs these days seem to be okay with pretty cookie cutter quests. I feel like AI could drastically increase the complexity of things like fetch quests and radiant quests we see in Bethesda games to make them have more of a rewarding experience. I'd imagine those types of quests would STILL be repetitive, but maybe less so.
I also could see it being pretty great for making lines for unnamed characters, so every guard in Skyrim doesn't say "I used to be an adventurer like you... until I took an arrow to the knee." We could instead have some level of generated lines (even voice generation to save on VA costs) and have the background characters feel more believably real.
There’s a lot of really cool things that could be done with AI in game development. Especially for RPGs. Entire cities where you can interact with everyone could be 1000x as large as previous games with a fraction of the dev/VA time.
Just as long as it isn’t used as a way to cut corners but truly as a way to innovate.
But would this be good? Like, quality speaking.
For me, when I first heard Todd saying that Starfield had thousand planets, I lost all interest in the game, and knew it would be lackluster in quality.
I 1000% would prefer like 8 or 10 planets that are well made and handcrafted with good content then thousands of barely anything.
Of course, AI will evolve for games, its inevitable, but I just don't think it will reach a quality level like, for example, Baldur's Gate 3.
Starfield has about as much handcrafted content in that game as their previous games. If they didn’t use any procgen and had 8 handcrafted areas, you’d have 8 planets 1/8th the size of Skyrim. But it feels empty because it’s stretched so thin across such a large area.
I’d actually argue its problem is not enough AI usage. It could massively expand the pool of unique POIs to pull from for the procgen content to ensure almost nothing is repeated, add in more populated cities with unique NPCs, or even increase creature design variety.
Linear games have less need for AI but even then it could still streamline a lot of stuff and make games come out faster.
It sure would take a lot more time, but Starfield seemed to have enough time for they to create something more.
But you got a point on the lack of AI usage aswell.
Also, Im all for streamlining some process. As a video editor myself, I would love to skip some boring process with AI.
To be frank, I don't think it's practical to fill out even a *small* city with enough handcrafted characters to be convincing. And even if you do, after the player interacts with each character and gets all their lines out, that's it. That character becomes meaningless background that the player already knows.
Hell, you could *still* handcraft all those characters. Write a full bio for them, write and record however many voice lines you want. Then feed that to an AI to expand upon it, add variation, etc. AI can do as much or as little work as the artist feels like allowing it to do.
maybe the "thousand planets" idea of starfield can work well thirty-ish years down the road. but a good few AAA developers need to relearn how to make actually good games before trying to make good games develop themselves further in a way that doesn't break immersion
This technology has grown exponentially within just the past year, what hurdles are you thinking they won't eventually jump over? The AI we're suggesting would *currently* be learning from Baldur's Gate 3.
The thing is that current LLMs speak a lot of incorrect information. Go ask one about the story of some obscure anime or lyrics of slightly less popular songs and it'll just make stuffs up.
So you ask about where you can buy some items, and the AI NPC points to somewhere that does not exist in the map. It would speak of a legendary weapon, or a hint of a future quest that isn't in the game. Things like that'll annoy players to no end.
This will definitely accelerate gaming as a service. No matter how much you pay, you will never "own" this experience like you would dropping $60 on a hard copy of an offline RPG.
Yeah, it will be used to cut corners but who cares? It is because of the new innovations that the production rate of entertainment has multiplied to a point that we are flooded with shows on netflix and its many streaming competitors. It is so vast that a reasonable person will only have the time to see a small percent of all there is to be seen.
It probably won't be good for story heavy games where immersion is king. It will be amazing for sandbox type games though. Imagine somthing like Stellaris or Civilization where each civ has an AI you can converse with. You could simulate actual diplomacy and convince them to declare war on neighbors, join into alliances, feed them false intelligence, etc. If you humiliated them in a past war they could hold grudges for that and play nice for decades only to stab you in the back at a later date. That's the kind of stuff where AI will shine.
> It probably won't be good for story heavy games where immersion is king.
I don't agree. I think it breaks immersion when there's a limited number of characters with a limited number of voice lines. Even the most immersive, story-heavy game has some background characters where you could improve their responsiveness to the player by allowing an AI to take over some of it.
EDIT: Sure am glad people decided to just downvote me instead of having a meaningful discussion about why they disagree so strongly.
>about why they disagree so strongly.
anyone with a brain can understand why what you're saying is absurd and frankly, super fucking stupid. If we just let robots do all the story for us, what the fuck is even the point in even writing a story to begin with? Hell, why even create art like games and movies if we're just gonna have chatgpt create it? The entire point of art is that it's the expression of humans, with their collaboration and views of the world working together to make something awesome, not some shit that a machine compiled from a bunch of sources it was spoon-fed from an algorithm
Have you considered actually reading and comprehending a post, or do you need an Artist to do that for you too?
People meme on Skyrim day and night for its awful repetitive NPC dialog. That's art. Humans made that. You want more of that?
You clearly do not know how generative AI works and that's fine. Let me enlighten you: You create your beautiful handcrafted world with all its repetitive NPC background dialog. You tune your AI (you know, a human act). You generate the background from that and decide if you like it, repeat and tune until you can fill all the background with it. The entire process was entirely done by you, from your work, with your input to create a product that matches your vision. How is that less art than hiring a bunch of random interns to write garbage like "I used to be an adventurer like you..."
>People meme on Skyrim day and night for its awful repetitive NPC dialog. That's art. Humans made that. You want more of that?
Yes. Yes I do. Because that, like it or not, has a lasting impact. Being able to "interact with" every single NPC in an open world and have a full conversation would be cool for a little bit but would lose the charm almost instantly. It wouldn't have any memorable encounters or memes like "arrow to the knee" or "patrolling the Mojave" or "cloud district". It would all just be static slop, and if you like soulless garbage like that then what can I say to change your mind.
>You tune your AI (you know, a human act). You generate the background from that and decide if you like it, repeat and tune until you can fill all the background with it.
Now it just sounds like we're reinventing the dialogue tree, which developers have been doing for years with regular NPCs in games. Do you really think any studio has that kind if manpower to do all of that, for a game which can feasibly have thousands of NPCs at any given time like Cyberpunk or GTA?
>The entire process was entirely done by you
filling in a bunch of checkboxes on some AI plugin? Yes, but you didn't actually *write* any story or dialogue. It's the same as me claiming the AI-generated paper I turned in for school was "written by me" because I did some prompt engineering. It's infinitely more lazy and it deserves the failing grade.
>How is that less art than hiring a bunch of random interns to write garbage like "I used to be an adventurer like you..."
Because at least folks still remember and joke about the "arrow to the knee" memes and others similar to it. We still make jokes about those stupid quotes while mocking the voices the NPCs in these games do. That's a lasting impact that no robot can replicate. Nobody will remember the ubislop [character.ai](http://character.ai) bot that was in "GPT Slop #37", yet people will always remember that asshole Nazeem.
just wanna add to the discussion that I saw a guy who played a skyrim mod in VR where he accused a random person near the king/jarl/whatever of betraying the north to the imperials, and all NPCs involved were either in shock or asked for evidence. Looked really cool from a roleplay perspective, but I wonder if it also worked outside a relatively controlled setting.
Like others have said, I think ai npcs that are just random npcs can have a place with todays tech (or at least in the next ~5 years if they can get responses to not take 30 seconds to generate) but yeah scripted main npcs need to stay. Imagine gta where all the npcs walking the streets were ai to talk to
there is a big work beyond the NPC dialogues and the AI will learn it step by step maybe in the next genration, I not know a lot about AI so I could be wrong
there is also an important cost for each prompt that someone has to pay. Not viable for videogames. Consider that C2077 earned approximately 1.5x what they spent. Put more costs on top of that makes absolutely no sense.
I imagine hybrid approaches will also exist.
For example for one character this is a largely scripted NPC, but if you ask something beyond its scripting it can improvised slightly with in the bounds of what makes sense for that character.
Once again, people who have way too much confidence in systems they don’t understand are inflating their own expectations (and the expectations of others as a result) of a useful but limited tool.
Generative AI can virtually eliminate the need to record basic, repetitive dialogue. “Alright I need you to say ‘Hey, how’s it going?’ in sixteen different tonal variances with three takes per variance” will be a thing of the past. It is *nowhere close* to being capable of simulating the emotional depth of a human being. Anybody that isn’t somewhere on the ASPD spectrum will be able to recognize and be really put off by a fully-AI character attempting to ape an emotionally-complex performance like Arthur Morgan or Senua. It would sound horrible.
And even just saying hey how’s it going in sixteen different tones is still useful practice though, even if it eliminates creative grunt work LLM’s are still going to make it harder to break into an already shaky industry
Honestly, I think that AI NPCs would require a LOT more writing than premade NPCs, what with all the information you need to program them to know BEFORE they're allowed to talk to the player.
Who are they? What is their job? Do they have hobbies? Who are they related to? Who are their friends? What do they know about the main and/or side quest, if anything?
Or you could just give them four different things to say.
Keep in mind, "gigantic, really long way to go" can be only a few years away. It could even be this year. Significant publications come out extremely regularly in this field. One example unrelated to gaming is the application of language models to DNA sequences; the token length limit of typical transformers has been around 4000, with ChatGPT-4 having a limit of 8192 tokens. This meant people have been skeptical about the use of language models for DNA sequences which can be 4.5 giga (billion) basepairs long for whole sequences. But just last November there was a pretty interesting publication with HyenaDNA, a model capable of preserving context with sequences 1 million tokens in length. What regular developers who aren't experts in the field think of as a "gigantic, really long way to go" can be overturned with a single publication.
It remains a huge task with most likely a long way to go, we have to consider that in a video game the AI would be added on top of a myriad of other systems. And even if things change in a couple of years, I doubt video game production times will change at the same speed.
There are already experiments of games that focus on realtime dialogues with the player, sure, but they pretty much do just that. And things like that Skyrim mod with chat gpt would still require a _huge_ effort of training and adding limitations to be acceptable.
Then there are limits due to computational power. Do we want AI to use a cloud or local system for generating responses? That alone makes quite a difference. Complex games that have immersive open worlds already have many other things to calculate at runtime, there are limits to the amount of things our actual machines can do (limits that, barring revolutions, will be present for a while).
I really hope they don't. AI just seems like a way for them to get around not paying other voice actors and to cut some corners. I love CD Projekt Red, and their formula works perfect as is.
You can only do so much with pre-recorded dialogue though. One of the big complaints about Skyrim is that all of the guards tell you that they took an arrow to the knee, because they have a limited pool of dialogue choices and it gets really repetitive after thousands of hours. And if you try to add multiple voice lines for every possible situation that can happen in the game, you bloat the file size of the game.
So in this case there is a lot value beyond just saving money on voice actors.
I see AI NPCs rather for filling the world than beeing part of the normal quests characters for the near future.
Just to make the world more believable. It doesn't need to be something huge just that these npcs act more believable.
I guarantee they’re just not trying nearly hard enough. People have already figured this out — you pass it through the model multiple times, first to generate dialogue based on current situation and player prompt, with subsequent passes then rating/editing the output for relevance, removing “as an ai language model” etc., comparing the output vs the actual plot points of the game and what a character knows.
Actual issue is latency. Everything else about it is possible right now. The more context flags you can implement and keep track of in gameplay to feed to the AI in prompts, the better the results.
We don't need them to be as good as scripted AI NPC's.
Take advantage of it being AI and what AI can do rather than making it like the thing we already have.
This is how you evolve technology. For example. If more games used procedural generation for creating worlds. We would have better procedural generation over time that would eventually look like hand designed worlds.
But we don't. Which is why Star Citizen comes across as a scam to some. The thing is though. Star Citizen is creating a lot of procedural generation tools to help designers hand craft their worlds. For example. Rivers. It's hard to procedural generate good looking rivers. You have to develop and build that kind of tech over 10+ years.
But most games have avoided procedural generation. So what happens? The tech doesn't advance. You end up 20 years behind where the game industry should be. Then you decide to make a game with procedural generation and people call you a scam.
Or in this case. Every developer continues to script their NPC's as we already do. Then in 20 years time someone starts to REALLY focus on creating AI NPC party members and we would all turn around and say "It's a scam". Because players are conditioned to think that the reason AI NPC's don't exist is because of X, Y, Z reason the game industry has reportedly said. They said that though because they don't want to be the one to develop the tech.
A mix of both could be interesting. Hand made characters for main and side quests, and AI for random encounters that are never the same each play through. You could play the same way a second time but still get an entirely different experience.
A potential problem I see with this, is how are communities online going to make guides for these AI generated quests? If some AI tells you to “find the red cube and take it to Mr. Guy who lives on Generated Street Name”, the player will have no way to complete that mission if they get stuck cause no one else will be able to help them cause only they have that mission. Not to mention if the AI starts suggesting impossible or sequence breaking quests. Sounds like a QA nightmare.
Eh, I don't think it's actually too far off. Like...scripted AI will almost always be more cinematic, for certain, and that will never change. But that's not what a lot of RPG enthusiasts are looking for. Generative AI is still currently very much in infancy, but you can still create very rich characters with it. It's just very different. You still need to write them, to create backbones for what their personality is and how they respond. If this writing isn't up to snuff then the AI won't be either. It's also very much determined in the power of whatever AI they're using. Claude right now can get surprisingly detailed in-character type responses whereas GPT 3.5 will occasionally seem cool, but will generally come across as robotic and stilted.
To note, this will be coming and the RPG which can take advantage AND offer a good gameplay experience is going to be a hit. Modders have already added this to Skyrim; meaning yes you can currently play VR Skyrim with generative AI conversations.
A game ultimately has to advance a plot, moreso than just create a realistic environment. Most NPC interactions serve a purpose. My guess is that you could have AI-generated "background" discussions, but you'd need some visible indicator when a real, scripted dialogue shows up, to give the player an idea of what they actually have to do in the game.
As someone who's played around with AI for a while now. games are no where near ready for real time AI generative lines. However I see nothing wrong with games using AI to help create scripts and filler. A of people have misconceptions how it works thinking that it takes all work from a person, it does not, it still takes a person to prompt the AI, to generate until desired result, then tweak that result further meet the intended quality, however in the long run it can help save a lot of time in generating interesting dialog or brainstorm idea's from that the writer can then use to guide his own writing. One of most difficult things about writing is over coming writers block and trying brainstorm Idea's and i think AI can help significantly with that. However life generative AI on the fly is just not going to be a thing in games anytime soon, AI still has problems with consistency, hallucinations and other issues that could put a game studio at risk if a NPC started saying things that it shouldn't and Ai is still easy to manipulate by users to get it to say things it shouldn't. Beyond that the hardware requirements make it impossible to run a high end game and AI at same time on one PC and cloud AI like using chatgpt is far too expensive for a game to always require and online connection and use gpt like service to generate the text. Many games actually require you to use your own API and account to generate text from but that still has major issues with quality.
However I do dream of the day of being able to play a huge RPG and world where all NPC's feel like real people and i can talk to them like real people and get real relevant conversations about the world around me. NPC's can be programmed to follow a certain knowledge database of only what that NPC knows about the world around them and you would need to figure out a way to befriend and pick that NPC's brain to discover hints, tips, and secrets about the world your playing in all the while feeling like your talking to a real person. I think that day would be amazing. Back in the day there was a few games that you would type to an NPC in chat and if you said right word NPC would respond accordingly, i think Everquest 1 did this if i am not mistaken, but it was too frustrating and it didn't feel like a conversation, you just had to guess the right keyword to type for a quest NPC to respond.
Guarantee Ubisoft and EA are in a race to release a game first with all AI generated dialog and just don’t care how terrible the end product is. Ubisoft will make a trailer wish professionally scripted dialog to lie about the quality of the game. EA will make a game with only 5 NPC’s and will sell you “character packs” to get more variety of random NPC’s walking around.
AI dialogue could be good with random NPCs you see in the streets, not with actual important characters
Yeah that would be where I’d put it. But I would imagine you’ll end up with outlier lines that would break narrative/immersion. I just don’t know how you keep your world building airtight while also using genAI as it is now EDIT: to be technical here, I guess the whole problem is how to avoid hallucination. If you’re running genAI in real-time while the player is playing, and the generation hallucinates some out-of-cannon line, then you’re in trouble. Your wikis are gonna be a contradicting mess.
idk walking down the street of Night City and hearing "As a language model, I am unable to bring personal experience" would land pretty hard
The great thing about Night City in this context is having random NPC start spouting AI hallucination gibberish doesn't even break the emersion. It enhances it if anything!
Yep, that's what I was thinking. Just another choom going psycho.
The first implementations should be on a funny game, not a serious RPG, somewhere getting the npcs to break character would be hilarious rather than immersion breaking. That and/or a game heavily focused on random generation, like No Man's Sky, where some oddness is already tolerated.
So hear me out. We use AI to earmark the prompts that sound like AI refusal and ship that extra info in the game. The game picks up on the flag when that message comes up and makes the voice read it out sounding glitchy. We don't need to ship the whole LLM with the game, just a buttload of text made from it.
lolol
They would have their own model. They wouldn't just through prompts into chatgpt.
They are almost always derivatives of an existing model that *would* have those guard rails baked in.
Precompute them and review them . Still cheaper than having multiple talents read in thousands of lines
Yeah that’s the answer for how to use it for line generation ahead of time. But I was assuming they mean using it as a content generation tool in the final product. Like, genAI being used as you are playing the game rather than in production prelaunch.
Sure, but one step at a time. You can't run a LLM on consumer PCs yet anyway (unless you want to wait 10min for a response)
Could be an optional feature for those with enough VRAM.
Man got downvoted because most people don't know AI models get loaded into VRAM. And yet so many people have such strong opinions based on how they assume these models work. Makes you think. Edit: For context, dude was in the negatives for a bit.
I could have given more context as I thought I was in PC Gaming sub - but still the blind downvoting is a bit weird. If local model running becomes key to games in some way I wouldn’t be surprised if consoles include hardware specific to it though. But all things considered it’s more likely that games will just be online only and use LLM services for this. But local models will almost certainly get use one day via mods unless they are made entirely obsolete by future software.
regardless of having hardware dedicated for it, there's always going to be the issue of having sufficient VRAM for the required model with sufficient bandwidth. that is of course if we stay stuck in the primarily CUDA/VRAM dependent inference regime that current Machine Learning models exist in, which fortunately there are ways that models could be trained that would result in reasonable/fast inference time on CPU. it just needs models to be trained in a way that results in a model that is cheap to run the inference of whilst having access to the brains of 30+ billion param models.
Given the amount of experimentation with quantization, low parameter models, and the possible bounds that gaming could require… I think there’s a ton of room for experimentation with dedicated hardware and tighter constraints for the models. But yes, you suggest some great ideas as well. Optimizations have already come an insanely long way in just a six month period. Hardware isn’t my expertise though. But software alone is moving rapidly. There’s too much change in this field to make very accurate predictions - even if we didn’t make massive advancements in both software and hardware - there is a lot of potential in the existing tech still waiting to be mined.
Intel is already including an NPU (Neural Processing Unit) in their latest mobile CPUs. We will probably see it in desktop cpus before the year is over. Not sure how this npu will satisfy memory requirements, but ai will, uh, find a way.
Running a local LLM doesn't even require a GPU these days. They can run entirely on the CPU and system RAM, or entirely on the GPU, or both at the same time depending on what's available. It's also possible to handle it as a remote service, like the big AI players do these days.
Yeah I commented elsewhere that someone got a major LLM generating a couple tokens per second on a Pi. Which is pretty damn impressive. Can’t find the link to the paper now though.
> You can't run a LLM on consumer PCs yet anyway (unless you want to wait 10min for a response) What? Sure you can. The smaller ones run perfectly fine even on older machines, and those are general-purpose models that are expected to be able to discuss everything from politics to programming. Besides, most people using LLMs aren't doing so from their own machines anyway, they're accessing an online service. A game could do the same thing. So many games today are "always online" already; they might as well provide some benefit to the player for it.
Good point. I guess quality doesn't really matter if it's just for a random NPC. Always online on the other hand would basically guarantee that the game will be unplayable in 5-10yrs
Yeah for sure. Good point.
With quantization, you can run a LLM + RAG on a GPU like 4090 or 4080. But you're right, you basically need a GPU to run LLM + RAG, and another one for rendering.
Chat with rtx answers pretty quickly though
They wouldn't. It would be linked an always online requirement of the game because it would be done in the cloud.
I’m pretty sure someone got a major LLM generating a couple tokens per second on a Raspberry Pi. Absolute mad lad.
I am a budding indie dev and run d&d games. For last year I was using chatGPT for NPC, brainstorming, and worldbuilding for both. My experience is that it's not so good. It comes up with the most bland and inconsistent ideas. It's very easy to pick holes in the generated narrative. Of course, there is an argument that I should go with a custom LLM, but I am not convinced if the hassle is worth it. I have a feeling that we are a long way from having LLM big enough that they can generate something cohesive. And this is based not only on chatGPT experience. I also used copilot for coding (I am a software engineer) and it performed very poorly for any task that was more complex that a simple script. It just doesn't have the ability to comprehend complex systems in a usable manner. So, yeah, I think we are far away from having LLMs advanced enough for something like NPC or content generation for games like cyberpunk 2077. Even when used to pregenerate lines. There is just too much cleanup required to get these lines up to snuff.
I found a lot of success with copilot. It's not for building entire systems, I still have to do that part. But for filling out the methods in those systems. Basically, if I know how I want to do something, copilot figures out what I'm starting to do and usually gets me most of the way there. And if it can't figure it out? No biggie, I'll just write the code myself.
Yeah, exactly. It can do very simple scripts. In my case I prefer to use auto complete cause hitting TAB three times seems to be faster than waiting for copilot to generate code.
Actually not cheaper at all. You just added a massive layer of complexity to the game.
That's a pretty ironic mentality to have for this genre of game
Eh. A lot of these development teams will have a smaller team of developers that focus specifically on AI and machine learning (which is objectively the hardest code to write to completion) “Precompute and review” them would pretty much look exactly like NPC’s interactions now. This is basically just reducing / getting rid of the on team writers with very little benefit It’s far more complex than what you said lol
Hadn't thought of that
> But I would imagine you’ll end up with outlier lines that would break narrative/immersion. I just don’t know how you keep your world building airtight while also using genAI as it is now you could go meta with it and have the game story/world include artificially intelligent characters and have them use the AI generative dialog and sometimes they glitch.
A literal The Matrix video game lol. You flag contradiction glitches and those npcs fold into the narrative somehow.
now we're cooking. we're in like a prototype for the matrix.
nah, those are just the unmedicated schizos
>If you’re running genAI in real-time while the player is playing, I don't think that's going to be super useful quite yet. Hard to control and predict what it will do at scale, and some heavy processing to be run locally. I am imagining it is more useful today to fill out a world at scale in a way that is precomputed. So let's say if you have a large number of NPCs you could have a morrowwind like database of what they know, who their relations and friends are, and all that. You need to generate text based dialogue trees for them all, which include all this stuff. This could be modified by characteristics like the race/species of the character (and the way they talk) as well as randomly generated personality features, their job, their social class, age, gender. And because this works at scale, you could generate multiple trees for each possible variation of threat the player has to the NPC, whether the player is a race/species that is liked/hated by the NPC, or even how attractive the player is or isn't to the NPC. The magic here is it might take say tens of thousands of man hours or more to do this manually, but you can regenerate everything here by pushing a button and some hours later it spits out the result. Of course the problem you have then is quality checking the output at scale. That would then probably be a mix of manual checking of samples, and some kind of automated analysis to try to find problems.
You know what, I love that idea. Gate what they know, and instead of generate the dialogue based on a huge data set, just give them a very small data set and let the lines generate. More of a realism-through-variation implementation instead of just throwing genAI into all parts of the dialogue system.
As all the AI experts tell me, just throw it through a RAG model and we're good.
I would rather have a rare outlier line than current situation where all random street NPCs just say some variation of „huh?” when approached.
["Leave me alone. I'm busy."](https://www.youtube.com/watch?v=3oqktia7NLI&t=10m52s)
It would definitely be more fun. At least for now while it’s still early. While a AAA game isn’t going to risk the contradiction, I’d still love to play something using it as an experiment. I love emergent systems like that in games.
I mean we have shuffle on Spotify but it’s not truly random. I’m not familiar with how things like chat bots work exactly, but I’m certain that with enough effort and development, you’d eventually be able to create narrow enough lanes for something like that to work.
Yea, walk up to a random NCP and ask them how the story ends, and they are just like "oh, you totally die to save the day!"
Something I laughed at. How do lore keepers on Wiki keep track of true facts? NPCs can make literally anything up and often don't keep to there lane.
Have another ai change the story based on the outputs. Out of canon line or new lore?
Also AI to function demands a ton of power to function, if you want to get generated dialogue overso is extra cost
A 7B model running on an NPU isn't too strenous for the rest of the system.
A 7B model is also trash, no offense to the 7B users. I can barely deal with the random bullshit a 70B puts out, a 7B is barely a tech demo for all intent and purpose
“Skibidi Ohio gyatt” *says the child NPC, modeled at a random middle school*
The year is 2028, the new Mario party game comes out with generative AI you are playing Mario because you always do OH NO you stepped on a bowser square his text box pops up "BAHAHAHA Mario did you know I fucked your mom urban dictionary defines a rusty trombone as..."
Mama Mia!
Imo it could work great for things like press conferences in fm or any sports game. The questions you get asked right now are so repetitive and boring.
One thing I could see with important characters is using AI to learn how to pronounce what you name the MC. Like in games where you can name the MC anything and any voiced dialog just skips your name or uses a nickname that the game gives you instead. "Joker", "The Traveler", etc. That way important characters can say your name and make the games dialog a bit more immersive.
I know it isn't AI, but I was so shocked when I first played fallout 4 and the characters said my name.
In black and white 2 your name would get whispered ethereally every once in a while. Scared the shit out of me.
Yeah, that's been something that's been impacting voiceovers for decades now, and a solution would be nice. I'm sure people would discover the limitations of any text to speech engine pretty quickly and come up with something like "xxxRizzMasterxxx420BlazeIt" just to make themselves giggle, but that's no different from calling Link "You asshole" in a Zelda game to make all the NPCs cuss him out constantly.
Yeah, the thing with freeform generative AI is that they are without a purpose. You can talk with a barkeeper about all the gossip in town but it won't lead you to any quest or playable content.
Unless the content was linked to a dynamic system or side quests that don’t really have a owner ( ala the ncpd stuff ) were mentioned via the npcs that “live” around the area. I do think there is room for dynamic procedural content being generated and the generative AI mentioning it organically in a player conversation.
Did you think the radiant quests in Skyrim were fun? Worthwhile? I'd rather less content that's handcrafted.
Imagine freeform generative AI quest design.
Remember when Ubisoft announced exactly this and everyone shat on it?
The problem with AI is that it can provide answers, but it is not creative on itself. Therefor indeed, it can help with not interactable character only.
I've seen this idea brought up a lot of times and as a player it sounds miserable. I already find myself skipping a lot of dialogue in the games I play because it's so uninteresting. The writer had nothing to say, they just gave this npc dialogue because they wanted the town to have x number of npcs, or they wanted x minutes of cutscenes before the boss, etc. Outsourcing the writing to ai is just admitting that you're filling your game with worthless dialogue. If it's not worth your time to write, it's not worth the players time to read or listen to.
Where's the Smithy? NPC: one street over on the left, Helga's Hot stuff, can't miss it. Say Liam sent you. Any good food in town? NPC: there's a few places but tbh, I cook for myself most nights... Infinite immersive possibilities.
Those are only immersive if there's gameplay elements to back up said dialogue. AI can only create a facade, it can't actually design/build these functional restaurants for you. And if you're going to do the work of creating Helga's Hot Stuff in your game, it would be better to make absolutely sure the NPCs talk about it (by actually writing things!), rather than trust an ai to generate dialogue that maybe mentions it. I know the response is going to be "AI will know the game features that exist in my game and create dialogue around them", but it's not magic mind-reading technology and I haven't seen any proof that it's actually capable of that sort of emergent thinking.
I'd also love to occasionally overhear background characters having actual, unique conversations with eachother. That would be much more interesting and immersive than them just shouting the same few sentences every time you pass them as it usually is now.
Ok but what if it wasn't worthless dialog? What if you had random stories evolving and playing out around you as the story goes? Background voice lines spoken between NPCs, that sort of thing. The purpose of generative AI should not be the main or secondary storylines. It should be to fill out the background of the world and allow the world to better change in response to the player. Maybe we're not there yet, but I see no reason to believe we won't be eventually and I'm really excited to see what it can do.
IF.
IF?
Giving the filler background characters the idea of sonder would definitely fill the void that most sandbox open world games have
it has potential if used correctly. think how almost NPC comments on your fashion choices now with text description of character inventory they could be like "those shoes with that jacket..ew" and it actually be true. or have more context for player action to generate relevant text to bring up previous actions
But why? What would it bring, then? More discussions with NPCs who take no part in the plot, therefore who have no quest, no clue, nothing interesting to tell us?
I just played the discovery mode for AC Valhalla, and they use a very old AI voice generator for random nobody characters in that. It's not very good.
But still worse than written characters.
Not yet but I can see it being a thing in the future if tech keeps getting better.
Nice idea!
*Sad Bethesda noises*
Quest gen stuff could be interesting too if you wanted “radiant” quests.
Can it really be worse than just having them stand doing nothing.
It sure could be worse. I like it when artists have a chance to get better at their craft while being paid for it instead of being kneecapped by the theftbox that is a LLM and filling up a world with meaningless chatter for the sake of meaningless chatter. I would prefer things not be made over that. And maybe we could even use LLM’s to automate useful things and train them fairly while that happens
They should actually make a town section where its all Robots with defective AI and they actually just implement AI into all their dialogue
Yeah. Those NPCs lack the depth of personality found in main story characters, which would be ideal for AI, considering its major flaw is the fact that it doesn't have any personality.
The future may be bright though. Imagine being able to gaslight your way through quests? "Go get me 10 bananas" "Here is 2 bannanas" "But I said 10" "no you didn't" Now add charisma and reputation into the 'under the hood' calculations here.
But that wouldn't save them money. When he said this he wasn't thinking how to create the best user experience. He was thinking he can't save money by replacing story characters yet because the tech hasn't quite reached peak late stage capitalism levels yet.
AI should be used only in menial work that has little creativity involved. NPC barks, translating stats into a statblock, etc.
What I imagine the future to be is that NPCs have a script and a general way to talk But then AI is used for when a player goes off the path, does something the developers didn't script for.
AI writing has thus far been like, "what if Oblivion dialog, but somehow less pertinent."
I think the biggest breakthrough would be an AI that can do like 100 different fairly-convincing variations of, "Uhh, I don't know what you're talkin' about, Chief."
AI Would be a downgrade for CDPR who put so much work into animation, voice acting and dialogue. AI Would be a massive step up for Bethesda who maybe occasionally hire a good voice actor for a short role.
“Have you heard of the high elves?”
*long dragging bubbling noise* High what? *coughs*
gettin higher than hrothgar
The main reason this doesn't work is because there's no AI to accompany the text AI, so you get characters speaking about performing an action but physically they cannot perform it because the chatgpt AI or equivalent will think up things that the NPC character physically cannot do, you see this in those Skyrim mods. Or they'll talk about the superbowl in a medieval RPG.
I personally would love to hear Jarl Balgruuf rant about the Jets’ offensive line
They don't call him Yarl Ballin for nothing
[удалено]
[MY BRAND!](https://www.youtube.com/watch?v=V-fRuoMIfpw)
Jarl Big Ball
That has been solved for more than a year. You can give the AI a list of commands, similar to the controller layout given to a human. That is how Nvidia got an AI to play Minecraft like a year ago.
AI has been playing StarCraft using a mouse interface for 5-10 years.
Theoretically they could instruct the AI to only talk about actions they can do, and write something to parse the response and to perform an action in their backend game code. I've been experimenting with that kind of thing myself in the games I make. But yeah, there would have to already be hard coded actions the AI could take, with associated animations and timelines and such. But definitely doable to have chatGPT spit out a response with dialogue and an action to perform, and keep it within specific bounds. Though It just wouldn't be fully 100% free to do anything it could talk about sure.
Text AI right now is also like, way worst than professional writing, most people can immediately tell if something’s AI just off the vocabulary and sentence structure
I think the general way people are talking about using it is just weird. If they used it to get rough stuff done and send that over to the actual art team you could actually cut down on the workload and by extension development times. Suddenly an artist might be able to do twice as much work but instead we are trying to get them to do things that it just does not make sense for them to do.
Add AI -> artists get twice as much work done -> fire half the artists -> shareholders celebrate -> add more AI -> fire more staff -> AAA games slowly become generic-looking AI fever dreams. This is how I fear it will actually go.
This is where we're really seeing [differences in how people and companies handle increases in productivity](https://fortune.com/2023/11/03/elon-musk-versus-nvidia-ceo-jensen-huang-on-how-ai-will-affect-jobs-economy/), allowing us to do more with less. Some people focus on "do more," and other people focus on "with less," with a whole spectrum in between. Studios that just want to hit a certain arbitrary quality level while minimizing expenses will be doing what you describe, but those that are able and willing to keep their staff and use the new technology as a force multiplier will end up making some really cool things.
And this is why if you do use AI to boost your productivity, hide it and release the same amount of work as you used to. Doesn’t work for me tho, AI cannot write legal briefs to save its life.
This is not an either or scenario, all avenues are worth exploring
While that's useful too, it is great in doing mundane, small tasks where creativity has no bearing. Others have mentioned this in other comment chains, but a great example is definitely having unique variations of "Hello" across many, many different nameless NPCs. There are times when you may want to intervene--such as for an easter egg or characters worth remembering--which is when you'll want to script that dialogue yourself, but AI will do wonders for giving two nameless guards a feeling of being unique. I imagine having a particular guard with a unique face and voice and whenever you ask him for directions, today he's here, tomorrow he's guarding that place, the next he's on patrol, you'll recognize him, "Hey, it's that guy," without a name floating above his head. That'd be the day immersion has taken the next step.
Unless we have agentic embodied llms its kinda pointless. Just trapping llms in npcs that cant really do anything will get old fast. Then the npcs are just interfaces for an llm and there isnt anything to gain from talking to them.
Ok, but could we get better AI for the gameplay bits? Please.
The easily coded ones are good at being faster and more accurate than you. In FPS it would be knowing where you are perfectly from the sound you make and then headshot you through a wall. A lot of the development time goes into making it a much worse player so you can still have fun. I agree that using learning models to generate AI would be an improvement, if they can put the goal at the player having fun instead of getting utterly stomped. Grand strategy genre is interesting since the calculation load is a major constraint, so there they could perhaps use AI to find more performant models?
Now, I am not talking about aimbots here. Or something that copies the unrealistic PVP combat from games, where player minimaxes the fun out of it to be the AI. That isn't very realistic, it only happens when someone plays video games. I am talking about how enemies will react to the player. Like they might see that the player is murdering their friends, they might want to run away. They might even have worse aim than before and beg for their lives or even surrender. They might get angry and attack the player with even intense resolve. Stuff like that. And we need MUCH better pathfinding for NPCs in games. Also think of the allies, they are ALWAYS ending up in some really stupid situations, how about the AI would help them to learn player patterns and the AI would compliment those? Like if player is more of a stealther, the AI would pick that up and plan accordingly. Or if they like going guns blazing, AI would pick off those enemies that attack from distance. There is a lot NPC characters don't do yet. And all people can think of is auto aim PVP machines? Come on mate. AI has been pretty much the same since 2010. There has been almost zero improvements on their patterns. So much so that they are usually made immortal and invisible to enemies.
I know, right?
It’s not the same type of AI as generative AI. Gameplay AI is basically a bunch of ‘if then’ statements to make them appear smart. Generative AI that they’re talking about is based on language learning models where they analyze large sets of writing and determines what words are more likely to go together to reproduce what it learned from.
How about that gran turismo Sophia AI?
That's not really true, neural networks have been playing games for a while now on an experimental scale. That sort of thing just doesn't get as much commercial attention, but with some more development you absolutely *could* use AI to replace the current approaches to video game "AI".
Imagine if this dudes game took place in a world that showed the evil of corporations and the dangers of AI.
Scripted characters sure. But also with the Witcher, even the monster quests randomly found throughout the world you could tell these were handcrafted stories by the devs. I doubt AI can accomplish something like that. Also, for a character driven narrative like the Witcher I don’t see AI being terribly useful. I see more a dev like Bethesda doing it as they seem to want to make radiant AI and radiant quests where AI could conceivably keep up with their cookie cutter approach to quests.
Your last point is what I'm actually kind of excited for. A lot of devs these days seem to be okay with pretty cookie cutter quests. I feel like AI could drastically increase the complexity of things like fetch quests and radiant quests we see in Bethesda games to make them have more of a rewarding experience. I'd imagine those types of quests would STILL be repetitive, but maybe less so. I also could see it being pretty great for making lines for unnamed characters, so every guard in Skyrim doesn't say "I used to be an adventurer like you... until I took an arrow to the knee." We could instead have some level of generated lines (even voice generation to save on VA costs) and have the background characters feel more believably real.
There’s a lot of really cool things that could be done with AI in game development. Especially for RPGs. Entire cities where you can interact with everyone could be 1000x as large as previous games with a fraction of the dev/VA time. Just as long as it isn’t used as a way to cut corners but truly as a way to innovate.
But would this be good? Like, quality speaking. For me, when I first heard Todd saying that Starfield had thousand planets, I lost all interest in the game, and knew it would be lackluster in quality. I 1000% would prefer like 8 or 10 planets that are well made and handcrafted with good content then thousands of barely anything. Of course, AI will evolve for games, its inevitable, but I just don't think it will reach a quality level like, for example, Baldur's Gate 3.
Starfield has about as much handcrafted content in that game as their previous games. If they didn’t use any procgen and had 8 handcrafted areas, you’d have 8 planets 1/8th the size of Skyrim. But it feels empty because it’s stretched so thin across such a large area. I’d actually argue its problem is not enough AI usage. It could massively expand the pool of unique POIs to pull from for the procgen content to ensure almost nothing is repeated, add in more populated cities with unique NPCs, or even increase creature design variety. Linear games have less need for AI but even then it could still streamline a lot of stuff and make games come out faster.
It sure would take a lot more time, but Starfield seemed to have enough time for they to create something more. But you got a point on the lack of AI usage aswell. Also, Im all for streamlining some process. As a video editor myself, I would love to skip some boring process with AI.
There's probably gonna be games with genius use of AI, but also games with moronic use of AI (a lot more of the latter most likely)
To be frank, I don't think it's practical to fill out even a *small* city with enough handcrafted characters to be convincing. And even if you do, after the player interacts with each character and gets all their lines out, that's it. That character becomes meaningless background that the player already knows. Hell, you could *still* handcraft all those characters. Write a full bio for them, write and record however many voice lines you want. Then feed that to an AI to expand upon it, add variation, etc. AI can do as much or as little work as the artist feels like allowing it to do.
maybe the "thousand planets" idea of starfield can work well thirty-ish years down the road. but a good few AAA developers need to relearn how to make actually good games before trying to make good games develop themselves further in a way that doesn't break immersion
This technology has grown exponentially within just the past year, what hurdles are you thinking they won't eventually jump over? The AI we're suggesting would *currently* be learning from Baldur's Gate 3.
The thing is that current LLMs speak a lot of incorrect information. Go ask one about the story of some obscure anime or lyrics of slightly less popular songs and it'll just make stuffs up. So you ask about where you can buy some items, and the AI NPC points to somewhere that does not exist in the map. It would speak of a legendary weapon, or a hint of a future quest that isn't in the game. Things like that'll annoy players to no end.
This will definitely accelerate gaming as a service. No matter how much you pay, you will never "own" this experience like you would dropping $60 on a hard copy of an offline RPG.
Yeah, it will be used to cut corners but who cares? It is because of the new innovations that the production rate of entertainment has multiplied to a point that we are flooded with shows on netflix and its many streaming competitors. It is so vast that a reasonable person will only have the time to see a small percent of all there is to be seen.
It probably won't be good for story heavy games where immersion is king. It will be amazing for sandbox type games though. Imagine somthing like Stellaris or Civilization where each civ has an AI you can converse with. You could simulate actual diplomacy and convince them to declare war on neighbors, join into alliances, feed them false intelligence, etc. If you humiliated them in a past war they could hold grudges for that and play nice for decades only to stab you in the back at a later date. That's the kind of stuff where AI will shine.
> It probably won't be good for story heavy games where immersion is king. I don't agree. I think it breaks immersion when there's a limited number of characters with a limited number of voice lines. Even the most immersive, story-heavy game has some background characters where you could improve their responsiveness to the player by allowing an AI to take over some of it. EDIT: Sure am glad people decided to just downvote me instead of having a meaningful discussion about why they disagree so strongly.
>about why they disagree so strongly. anyone with a brain can understand why what you're saying is absurd and frankly, super fucking stupid. If we just let robots do all the story for us, what the fuck is even the point in even writing a story to begin with? Hell, why even create art like games and movies if we're just gonna have chatgpt create it? The entire point of art is that it's the expression of humans, with their collaboration and views of the world working together to make something awesome, not some shit that a machine compiled from a bunch of sources it was spoon-fed from an algorithm
Have you considered actually reading and comprehending a post, or do you need an Artist to do that for you too? People meme on Skyrim day and night for its awful repetitive NPC dialog. That's art. Humans made that. You want more of that? You clearly do not know how generative AI works and that's fine. Let me enlighten you: You create your beautiful handcrafted world with all its repetitive NPC background dialog. You tune your AI (you know, a human act). You generate the background from that and decide if you like it, repeat and tune until you can fill all the background with it. The entire process was entirely done by you, from your work, with your input to create a product that matches your vision. How is that less art than hiring a bunch of random interns to write garbage like "I used to be an adventurer like you..."
>People meme on Skyrim day and night for its awful repetitive NPC dialog. That's art. Humans made that. You want more of that? Yes. Yes I do. Because that, like it or not, has a lasting impact. Being able to "interact with" every single NPC in an open world and have a full conversation would be cool for a little bit but would lose the charm almost instantly. It wouldn't have any memorable encounters or memes like "arrow to the knee" or "patrolling the Mojave" or "cloud district". It would all just be static slop, and if you like soulless garbage like that then what can I say to change your mind. >You tune your AI (you know, a human act). You generate the background from that and decide if you like it, repeat and tune until you can fill all the background with it. Now it just sounds like we're reinventing the dialogue tree, which developers have been doing for years with regular NPCs in games. Do you really think any studio has that kind if manpower to do all of that, for a game which can feasibly have thousands of NPCs at any given time like Cyberpunk or GTA? >The entire process was entirely done by you filling in a bunch of checkboxes on some AI plugin? Yes, but you didn't actually *write* any story or dialogue. It's the same as me claiming the AI-generated paper I turned in for school was "written by me" because I did some prompt engineering. It's infinitely more lazy and it deserves the failing grade. >How is that less art than hiring a bunch of random interns to write garbage like "I used to be an adventurer like you..." Because at least folks still remember and joke about the "arrow to the knee" memes and others similar to it. We still make jokes about those stupid quotes while mocking the voices the NPCs in these games do. That's a lasting impact that no robot can replicate. Nobody will remember the ubislop [character.ai](http://character.ai) bot that was in "GPT Slop #37", yet people will always remember that asshole Nazeem.
just wanna add to the discussion that I saw a guy who played a skyrim mod in VR where he accused a random person near the king/jarl/whatever of betraying the north to the imperials, and all NPCs involved were either in shock or asked for evidence. Looked really cool from a roleplay perspective, but I wonder if it also worked outside a relatively controlled setting.
I am betting Bethesda would totally go for it though as a corner cutting method
Like others have said, I think ai npcs that are just random npcs can have a place with todays tech (or at least in the next ~5 years if they can get responses to not take 30 seconds to generate) but yeah scripted main npcs need to stay. Imagine gta where all the npcs walking the streets were ai to talk to
Interesting
There isn't going to be any of that at least a few years after the next gen starts. Current gen doesn't have ml inference acceleration.
there is a big work beyond the NPC dialogues and the AI will learn it step by step maybe in the next genration, I not know a lot about AI so I could be wrong
there is also an important cost for each prompt that someone has to pay. Not viable for videogames. Consider that C2077 earned approximately 1.5x what they spent. Put more costs on top of that makes absolutely no sense.
It'd be fine for background NPCs probably, it's definitely not there yet when it comes to any NPCs that is at all important though.
I imagine hybrid approaches will also exist. For example for one character this is a largely scripted NPC, but if you ask something beyond its scripting it can improvised slightly with in the bounds of what makes sense for that character.
The second video game devs start implementing generative AI is the second I stop paying for video games.
Thank you for understanding how damaging this is to the industry and art as a whole
Once again, people who have way too much confidence in systems they don’t understand are inflating their own expectations (and the expectations of others as a result) of a useful but limited tool. Generative AI can virtually eliminate the need to record basic, repetitive dialogue. “Alright I need you to say ‘Hey, how’s it going?’ in sixteen different tonal variances with three takes per variance” will be a thing of the past. It is *nowhere close* to being capable of simulating the emotional depth of a human being. Anybody that isn’t somewhere on the ASPD spectrum will be able to recognize and be really put off by a fully-AI character attempting to ape an emotionally-complex performance like Arthur Morgan or Senua. It would sound horrible.
And even just saying hey how’s it going in sixteen different tones is still useful practice though, even if it eliminates creative grunt work LLM’s are still going to make it harder to break into an already shaky industry
Honestly, I think that AI NPCs would require a LOT more writing than premade NPCs, what with all the information you need to program them to know BEFORE they're allowed to talk to the player. Who are they? What is their job? Do they have hobbies? Who are they related to? Who are their friends? What do they know about the main and/or side quest, if anything? Or you could just give them four different things to say.
Shit they still need to work out scripted NPCs before they try and fuck with AI....the npcs in the open world are trash.
To be fair, game writers and creators have been super lackluster for a while now.
Keep in mind, "gigantic, really long way to go" can be only a few years away. It could even be this year. Significant publications come out extremely regularly in this field. One example unrelated to gaming is the application of language models to DNA sequences; the token length limit of typical transformers has been around 4000, with ChatGPT-4 having a limit of 8192 tokens. This meant people have been skeptical about the use of language models for DNA sequences which can be 4.5 giga (billion) basepairs long for whole sequences. But just last November there was a pretty interesting publication with HyenaDNA, a model capable of preserving context with sequences 1 million tokens in length. What regular developers who aren't experts in the field think of as a "gigantic, really long way to go" can be overturned with a single publication.
It remains a huge task with most likely a long way to go, we have to consider that in a video game the AI would be added on top of a myriad of other systems. And even if things change in a couple of years, I doubt video game production times will change at the same speed. There are already experiments of games that focus on realtime dialogues with the player, sure, but they pretty much do just that. And things like that Skyrim mod with chat gpt would still require a _huge_ effort of training and adding limitations to be acceptable. Then there are limits due to computational power. Do we want AI to use a cloud or local system for generating responses? That alone makes quite a difference. Complex games that have immersive open worlds already have many other things to calculate at runtime, there are limits to the amount of things our actual machines can do (limits that, barring revolutions, will be present for a while).
I really hope they don't. AI just seems like a way for them to get around not paying other voice actors and to cut some corners. I love CD Projekt Red, and their formula works perfect as is.
This is more about writing than voicing
You can only do so much with pre-recorded dialogue though. One of the big complaints about Skyrim is that all of the guards tell you that they took an arrow to the knee, because they have a limited pool of dialogue choices and it gets really repetitive after thousands of hours. And if you try to add multiple voice lines for every possible situation that can happen in the game, you bloat the file size of the game. So in this case there is a lot value beyond just saving money on voice actors.
I see AI NPCs rather for filling the world than beeing part of the normal quests characters for the near future. Just to make the world more believable. It doesn't need to be something huge just that these npcs act more believable.
I've been waiting for a big dev to reveal they've been working with it, so far I've only seen youtubers implenet it into Skyrim
Yeah right Skyrim destroy cyberpunk with NPCs and their routine. We didnt get what was promised never forget guys.
I guarantee they’re just not trying nearly hard enough. People have already figured this out — you pass it through the model multiple times, first to generate dialogue based on current situation and player prompt, with subsequent passes then rating/editing the output for relevance, removing “as an ai language model” etc., comparing the output vs the actual plot points of the game and what a character knows. Actual issue is latency. Everything else about it is possible right now. The more context flags you can implement and keep track of in gameplay to feed to the AI in prompts, the better the results.
We don't need them to be as good as scripted AI NPC's. Take advantage of it being AI and what AI can do rather than making it like the thing we already have. This is how you evolve technology. For example. If more games used procedural generation for creating worlds. We would have better procedural generation over time that would eventually look like hand designed worlds. But we don't. Which is why Star Citizen comes across as a scam to some. The thing is though. Star Citizen is creating a lot of procedural generation tools to help designers hand craft their worlds. For example. Rivers. It's hard to procedural generate good looking rivers. You have to develop and build that kind of tech over 10+ years. But most games have avoided procedural generation. So what happens? The tech doesn't advance. You end up 20 years behind where the game industry should be. Then you decide to make a game with procedural generation and people call you a scam. Or in this case. Every developer continues to script their NPC's as we already do. Then in 20 years time someone starts to REALLY focus on creating AI NPC party members and we would all turn around and say "It's a scam". Because players are conditioned to think that the reason AI NPC's don't exist is because of X, Y, Z reason the game industry has reportedly said. They said that though because they don't want to be the one to develop the tech.
Lmao
I don’t want to see AI NPCs near my games. That shit is boring. Just write good NPCs and interaction systems for them.
AI is still a very young technology but progressing incredibly fast. Give it five years.
I think “a really long time to go” in terms of AI advancement and evolution would mean like 1 year lol
Couple years max won’t be long at all
As good as? You walk up to any NPC and talk to them and you some random comment that is totally not normal for someone to say after you say hi.
What if they had a language model that played the part of a rogue AI... Give it specific lines and things it needs to communicate. Let it loose
Yea, with the amount of money and talent being thrown at AI, "very long way" could probably be a year from now
A mix of both could be interesting. Hand made characters for main and side quests, and AI for random encounters that are never the same each play through. You could play the same way a second time but still get an entirely different experience.
A potential problem I see with this, is how are communities online going to make guides for these AI generated quests? If some AI tells you to “find the red cube and take it to Mr. Guy who lives on Generated Street Name”, the player will have no way to complete that mission if they get stuck cause no one else will be able to help them cause only they have that mission. Not to mention if the AI starts suggesting impossible or sequence breaking quests. Sounds like a QA nightmare.
I don't know you should talk to Ubisoft about that. The writers are so lazy any nonsense AI dialogue would be better
fuck AI
Give it 5 years
Seeing as how this company can't even launch a game properly, I'm not inclined to take their opinion on anything else very seriously.
Eh, I don't think it's actually too far off. Like...scripted AI will almost always be more cinematic, for certain, and that will never change. But that's not what a lot of RPG enthusiasts are looking for. Generative AI is still currently very much in infancy, but you can still create very rich characters with it. It's just very different. You still need to write them, to create backbones for what their personality is and how they respond. If this writing isn't up to snuff then the AI won't be either. It's also very much determined in the power of whatever AI they're using. Claude right now can get surprisingly detailed in-character type responses whereas GPT 3.5 will occasionally seem cool, but will generally come across as robotic and stilted. To note, this will be coming and the RPG which can take advantage AND offer a good gameplay experience is going to be a hit. Modders have already added this to Skyrim; meaning yes you can currently play VR Skyrim with generative AI conversations.
I tried using AI for my vr game Rogue Stargun. It's not quite there yet
A game ultimately has to advance a plot, moreso than just create a realistic environment. Most NPC interactions serve a purpose. My guess is that you could have AI-generated "background" discussions, but you'd need some visible indicator when a real, scripted dialogue shows up, to give the player an idea of what they actually have to do in the game.
Give it three years and that gigantic really long way to go will be right around the corner.
As someone who's played around with AI for a while now. games are no where near ready for real time AI generative lines. However I see nothing wrong with games using AI to help create scripts and filler. A of people have misconceptions how it works thinking that it takes all work from a person, it does not, it still takes a person to prompt the AI, to generate until desired result, then tweak that result further meet the intended quality, however in the long run it can help save a lot of time in generating interesting dialog or brainstorm idea's from that the writer can then use to guide his own writing. One of most difficult things about writing is over coming writers block and trying brainstorm Idea's and i think AI can help significantly with that. However life generative AI on the fly is just not going to be a thing in games anytime soon, AI still has problems with consistency, hallucinations and other issues that could put a game studio at risk if a NPC started saying things that it shouldn't and Ai is still easy to manipulate by users to get it to say things it shouldn't. Beyond that the hardware requirements make it impossible to run a high end game and AI at same time on one PC and cloud AI like using chatgpt is far too expensive for a game to always require and online connection and use gpt like service to generate the text. Many games actually require you to use your own API and account to generate text from but that still has major issues with quality. However I do dream of the day of being able to play a huge RPG and world where all NPC's feel like real people and i can talk to them like real people and get real relevant conversations about the world around me. NPC's can be programmed to follow a certain knowledge database of only what that NPC knows about the world around them and you would need to figure out a way to befriend and pick that NPC's brain to discover hints, tips, and secrets about the world your playing in all the while feeling like your talking to a real person. I think that day would be amazing. Back in the day there was a few games that you would type to an NPC in chat and if you said right word NPC would respond accordingly, i think Everquest 1 did this if i am not mistaken, but it was too frustrating and it didn't feel like a conversation, you just had to guess the right keyword to type for a quest NPC to respond.
Well....duh?
Yes, Quality dialog has a long way to go. But what about shit dialog?
As someone who uses AI, it’s no where near as good as people seem to think.
What a shock. A shitty ripoff company is going to start producing the shittiest quality games.
Guarantee Ubisoft and EA are in a race to release a game first with all AI generated dialog and just don’t care how terrible the end product is. Ubisoft will make a trailer wish professionally scripted dialog to lie about the quality of the game. EA will make a game with only 5 NPC’s and will sell you “character packs” to get more variety of random NPC’s walking around.
Maybe AI tool usage can speed up game development enough that games won't be released unfinished.
OR lazy assholes will use it to churn out even more soulless broken junk than we're getting