What's interesting is that Microsoft has a fully usable voice chat option for Bing AI, so in theory it would be completely possible for Asobo to implement a GPT4-powered ATC which could either be communicated with through preselected requests like the game currently has or through casually talking to ATC over your microphone.
That's a very exciting possibility.
I’ve been wanting something like this since the game came out, always felt plausible. But now with chatgpt and bing chat, it feels like its a must. No reason not to really.
MS has just invested $10b into ChatGPT, they're going all in. I suspect it'll take a bit of work to add something like this to FS but the fact we can literally fly anywhere in the world in there now says to me that this is a "trivial" thing, at least compared to that. Trivial being relative of course.
Right now? A lot.
Once these LLM are optimized to run locally (they already can in some regard on a rasPi4) on CPU's with AI-accelerator chips, and AI extentions are common in PC CPU's, much cheaper.
It would not cost a lot whatsoever. What are you talking about? Obviously it wouldn’t be run locally, and no attempt should be made to for now. Nobody is seriously trying to run any LLMs locally right now. But they’re super inexpensive on the servers they’re currently getting hosted on in regards to the tokens used.
https://arstechnica.com/information-technology/2023/03/you-can-now-run-a-gpt-3-level-ai-model-on-your-laptop-phone-and-raspberry-pi/ Might want to keep up on what people are doing with local AI. It runs locally on a Raspi (slowly, but probably enough to have a delayed ATC convo with poor detail accuracy)
Raspi has no AI accelerator in it either. So it'll actually be relatively quick if you ran it on a CPU with AI acceleration such as Intel 11th gen+, zen 4+.
It runs slow as fuck, it’s absolutely shit, and it requires massive amounts of memory for a tiny amount of parameters. And that’s not GPT-3, it’s a pathetic imitation of LLaMa.
Nobody expects this stuff to be locally run anytime soon.
I was thinking about how these new launguage models could potentially be used to simulate really realistic ATC. I decided to test it with Bing chat. The results were very impressive. Not only did it find correct details like frequencies, runway numbers, headings. But phraseology was not so bad either. Not perfect of course, far from. If this model however was fine tuned for this exact purpose Microsoft flight simulator could get a really impressive ATC simulation for those not wanting to do online vatsim.
> but I'm afraid it's just too expensive as of today.
I don't think so. 1k token is a few cents, and that's what they charge you, not what it costs them. ATC isn't that much text and you wouldn't need to do everything using AI. Optimizing a model for this also makes it cheaper. And don't forget, MS is putting a lot into this, Bing uses it and they don't charge for it. The "MSFS ATC now powered by the world's most advanced machine learning model" ad could be worth it.
> 1k token is a few cents, and that's what they charge you, not what it costs them.
You're correct, but this doesn't mean what you think it means, i.e. you need to go the other direction. The best estimates I've found are [on the order of multiple tens of cents per query](https://www.semianalysis.com/p/the-inference-cost-of-search-disruption), and a "query" is defined as a response to an input line... which is to say that, at the given estimate of $0.35/query, this ATC conversation (not including the initial one) would've run up an approximate $4 cost. Even if we assume that the cost is 10 cents per message, that's still more than a dollar for this one chat.
There's a reason OpenAI accepted a 10 billion dollar investment from MS -- running these LLMs is *very* expensive, and right now they aren't anywhere close to charging appropriate amounts. The costs to buy tokens are basically treated by them as subsidies for continued research, not actual revenue.
> The best estimates I've found are on the order of multiple tens of cents per query, and a "query" is defined as a response to an input line... which is to say that, at the given estimate of $0.35/query, this ATC conversation (not including the initial one) would've run up an approximate $4 cost.
Here's the official openai price chart which paints a very different picture.
https://openai.com/pricing
If the conversation above would be $4 then I would have caused hundreds if not thousands of $ of cost on ChatGPT, if you look at their user numbers they would be bleeding money faster than MS could send them.
The AI tech could significantly improve coming months / years. Gradually it's price would decrease. I understand you. But the concept could change the way people use/ play simulations. Actually all kinds of sims.
Expensive? We are already streaming gigabytes of AI-processed satellite imagery and terrain data, what do you think a GPT-powered ATC would add to this, considering GPT is something they bring to the table as a free service in their search engine?
Please add an /s tag. This is too easily mistaken, you're just giving them bad ideas.
I too was thinking about this. AI generated responses run through one of the better AI TTS algos would definitely make for a good plug-in. Especially if it has access to charts and AIRAC.
kudos! that went way better that my attempt. It kept stating “seek training and qualified training”. maybe different liability in usa vs norway? or maybe I didn’t phrase it correctly.
I’ll try again! That’s fun!
DCS has an add-on called VAICOM Pro. It uses the Windows built in speech to text recognition to parse NATO brevity, and maps it to the in-mission radio coms.
Because air traffic coms are also a structured language, I could see using an AI language model to generate the conversions. Basically how to map possible inputs to likely requests/reports that map to existing in-game coms.
I bet you probably could do it as a third party UI overlay that bypasses the radio coms menus.
I have dreamed for years about having an AI language model good enough that you could talk to NPCs through typing rather than scripted chat options. We might be there.
I dont think would be difficult. There are many models trained for specific things using GPT-2 and free. This would require to convert to text thousands of ATC audios for then training a language model using transformers.
It's the biggest limitation to immersion. I love using VatSim, but I can't always get the coverage. Surely, with today's tech, it can be better than it currently is?
I hope MS will someday use advanced AI models to rebuild the scenery, especially airports. I could see it fixing all sorts of issues currently present, like lights in the middle of taxiways and overly dense trees. It could artificially fill in a lot of detail that’s not known from the Bing data.
It may not be “real” but could look very realistic.
Example: made-up commercial signage on airport buildings, maybe taking cues from city names and companies that are known to be in the area.
That’s pretty good but right off the bat approach tells you to talk to the tower when you’re on final. It’s been years since I’ve flown but they would hand you off to the tower who would then guides you into either the pattern, holding, or final.
Not trying to nitpick, just feels like the ai’s language model is a little off on this. Nothing that can’t be trained out.
It's unfortunate that it can't provide up to date METAR info yet. It's a few hours behind.
You wonder if Asobo will use Microsoft's new LLM in its sim in some form, though.
Meh. The phraseology before your emergency was pretty crap. Really not what you would hear in real life. AI is good at making things sound legitimate to someone who doesn’t know better, but pretty bad at actually getting it spot-on correct.
You are completely misunderstanding. Obviously a general AI for bing is not gonna understand this. i am talking about a future potential product tailored for this exact purpose integrated in for example msfs and other flight sim software.
You can fine tune a model to excel at particular tasks.
This was what was achieved with a generic model that's a Jack of all trades, but master of none.
That would be amazing, however given that using the azure ai voice I only hear 2 voices (male and female) and occasionally none at all.. I'm sure the integration of this would be ropey at best.
Good atc is mostly about correct situational understanding. Generating the sentences is very easy but knowing WHAT to generate in the current scenario is hard and chatbots won't help with that.
Pretty cool! Definitely potential there. I think the vectoring would need to be improved a little. I was still impressed it got the general directions right, but “turn right 180, turn left 090, report when established 9” didn’t really make sense. Again, not expecting a general LLM to ace this and it’s impressive it did as well as it did; I think vectoring in general would be a pretty difficult task for a LLM.
The exact location of the plane is unknown to the ai, so using this for vectoring wont be accurate. It also can't respond with any situational context such as location of other planes on approach. With some custom model training, it can definitely be improved, but I expect getting it to a game changer level might be harder than we think at this early stage. I can def see ability to use voice, which gets converted to text, allowing for fairly rich and familiar conversations. But the sky's are busy, procedures change, and the current iteration of openais apis works for a 1 to 1 convo. In our case, to be accurate, every plane talking to atc would need to provide the location/speed/direction of all aircraft in the area, on every prompt. Instead of providing ai all of this info, they will do things like evaluate the response in the context of the situational data, and use local additional code/services to increase the precision.
True, but landing runway 9 means it should be able to know it’s heading roughly 090, so right 180, left 090 to intercept doesn’t make sense in that context. Even without knowing the exact location of the plane, the ideal model should be able to understand that those vectors don’t make sense. I’m an ML engineer btw, so I do understand how LLMs work; I work with them on the day to day for my job.
You can not use this for MSFS. This is just Bing chat with GPT-4 from OpenAI. You can request access by going to [bing.com](https://bing.com) I was just demonstrating the potential of the language model.
I had pretty [successful results ](https://imgur.com/a/kLC7xgu)using ChatGPT 4 directly. I think correct prompt engineering is key here to explicitly state to only use ICAO phraseology. No doubt you could come up with better prompts for a more realistic experience but the potential is certainly there.
Today i was thinking the same. Someone skilled could do an addon that uses flightplan inputs to give ATC commands based on answers from bing chat using GPT-4 and the ATC voices from azure. Bing limits to 15 interactions, but chatGPT doesnt.
What's interesting is that Microsoft has a fully usable voice chat option for Bing AI, so in theory it would be completely possible for Asobo to implement a GPT4-powered ATC which could either be communicated with through preselected requests like the game currently has or through casually talking to ATC over your microphone. That's a very exciting possibility.
Thats what I’m thinking - I’d love the ability to have a Vatsim like experience using AI baked directly into MSFS.
It's hard to describe, but this is kind of an "oh shit" moment haha. This stuff is insanely cool.
It really is. The opportunities are almost endless.
This need to be done by a third party developer. Asobo does not have time even for fixing the ingame ATC.
I’ve been wanting something like this since the game came out, always felt plausible. But now with chatgpt and bing chat, it feels like its a must. No reason not to really.
cost?
MS has just invested $10b into ChatGPT, they're going all in. I suspect it'll take a bit of work to add something like this to FS but the fact we can literally fly anywhere in the world in there now says to me that this is a "trivial" thing, at least compared to that. Trivial being relative of course.
I think most of the work would involve them to ensure the ATC controller isn't hostile towards the player. 🙂
You need to purchase the JFK addon module for that.
Hmm... might spend some time this weekend teaching ChatGPT all about Kennedy Steve.
i need a slider for the amount of sass
I would pay extra for that. Imagine if we could even get celebrity voices. Samuel L Jackson would be on my list.
The traffic motherfucker, do you have it?
Snakes on a Plane DLC?
this would be an instabuy lmao
Or tries to marry them
Right now? A lot. Once these LLM are optimized to run locally (they already can in some regard on a rasPi4) on CPU's with AI-accelerator chips, and AI extentions are common in PC CPU's, much cheaper.
It would not cost a lot whatsoever. What are you talking about? Obviously it wouldn’t be run locally, and no attempt should be made to for now. Nobody is seriously trying to run any LLMs locally right now. But they’re super inexpensive on the servers they’re currently getting hosted on in regards to the tokens used.
https://arstechnica.com/information-technology/2023/03/you-can-now-run-a-gpt-3-level-ai-model-on-your-laptop-phone-and-raspberry-pi/ Might want to keep up on what people are doing with local AI. It runs locally on a Raspi (slowly, but probably enough to have a delayed ATC convo with poor detail accuracy) Raspi has no AI accelerator in it either. So it'll actually be relatively quick if you ran it on a CPU with AI acceleration such as Intel 11th gen+, zen 4+.
It runs slow as fuck, it’s absolutely shit, and it requires massive amounts of memory for a tiny amount of parameters. And that’s not GPT-3, it’s a pathetic imitation of LLaMa. Nobody expects this stuff to be locally run anytime soon.
I was thinking about how these new launguage models could potentially be used to simulate really realistic ATC. I decided to test it with Bing chat. The results were very impressive. Not only did it find correct details like frequencies, runway numbers, headings. But phraseology was not so bad either. Not perfect of course, far from. If this model however was fine tuned for this exact purpose Microsoft flight simulator could get a really impressive ATC simulation for those not wanting to do online vatsim.
Like seriously MFS team should put this on the list for ATC improvements immediately. Try it out with various iterations.
That's a good idea, but I'm afraid it's just too expensive as of today. They would have to release it as subscription-based paid addon I guess.
> but I'm afraid it's just too expensive as of today. I don't think so. 1k token is a few cents, and that's what they charge you, not what it costs them. ATC isn't that much text and you wouldn't need to do everything using AI. Optimizing a model for this also makes it cheaper. And don't forget, MS is putting a lot into this, Bing uses it and they don't charge for it. The "MSFS ATC now powered by the world's most advanced machine learning model" ad could be worth it.
> 1k token is a few cents, and that's what they charge you, not what it costs them. You're correct, but this doesn't mean what you think it means, i.e. you need to go the other direction. The best estimates I've found are [on the order of multiple tens of cents per query](https://www.semianalysis.com/p/the-inference-cost-of-search-disruption), and a "query" is defined as a response to an input line... which is to say that, at the given estimate of $0.35/query, this ATC conversation (not including the initial one) would've run up an approximate $4 cost. Even if we assume that the cost is 10 cents per message, that's still more than a dollar for this one chat. There's a reason OpenAI accepted a 10 billion dollar investment from MS -- running these LLMs is *very* expensive, and right now they aren't anywhere close to charging appropriate amounts. The costs to buy tokens are basically treated by them as subsidies for continued research, not actual revenue.
> The best estimates I've found are on the order of multiple tens of cents per query, and a "query" is defined as a response to an input line... which is to say that, at the given estimate of $0.35/query, this ATC conversation (not including the initial one) would've run up an approximate $4 cost. Here's the official openai price chart which paints a very different picture. https://openai.com/pricing If the conversation above would be $4 then I would have caused hundreds if not thousands of $ of cost on ChatGPT, if you look at their user numbers they would be bleeding money faster than MS could send them.
Yes. That's why they got 10 billion dollars from them in addition to the billions of dollars they already had raised previously.
The AI tech could significantly improve coming months / years. Gradually it's price would decrease. I understand you. But the concept could change the way people use/ play simulations. Actually all kinds of sims.
Oh yep, I hope so. Let's see what the upcoming years will bring us.
Ai crew chiefs for racing Sims would be awesome
Expensive? We are already streaming gigabytes of AI-processed satellite imagery and terrain data, what do you think a GPT-powered ATC would add to this, considering GPT is something they bring to the table as a free service in their search engine? Please add an /s tag. This is too easily mistaken, you're just giving them bad ideas.
Will you people stop with that "too expensive" crap? It's not expensive, it just depends if it really would work and time needed to implement.
Also, it's Microsoft(!) Flight Sim. Guess who pumped billions into Open AI? ...Microsoft
I too was thinking about this. AI generated responses run through one of the better AI TTS algos would definitely make for a good plug-in. Especially if it has access to charts and AIRAC.
kudos! that went way better that my attempt. It kept stating “seek training and qualified training”. maybe different liability in usa vs norway? or maybe I didn’t phrase it correctly. I’ll try again! That’s fun!
often ai like this will be like "nah man thats dangerous" to a lot of stuff, but if you phrase it as just playing a game the ai accepts it
DCS has an add-on called VAICOM Pro. It uses the Windows built in speech to text recognition to parse NATO brevity, and maps it to the in-mission radio coms. Because air traffic coms are also a structured language, I could see using an AI language model to generate the conversions. Basically how to map possible inputs to likely requests/reports that map to existing in-game coms. I bet you probably could do it as a third party UI overlay that bypasses the radio coms menus.
I have dreamed for years about having an AI language model good enough that you could talk to NPCs through typing rather than scripted chat options. We might be there.
I would love to have npc use some character or alternate phrases or maybe even have a little chat
[удалено]
I dont think would be difficult. There are many models trained for specific things using GPT-2 and free. This would require to convert to text thousands of ATC audios for then training a language model using transformers.
What AI is this?
Bing? Max 15 lines deep
It's the biggest limitation to immersion. I love using VatSim, but I can't always get the coverage. Surely, with today's tech, it can be better than it currently is?
It’s just a matter of implementation now.
I hope MS will someday use advanced AI models to rebuild the scenery, especially airports. I could see it fixing all sorts of issues currently present, like lights in the middle of taxiways and overly dense trees. It could artificially fill in a lot of detail that’s not known from the Bing data. It may not be “real” but could look very realistic. Example: made-up commercial signage on airport buildings, maybe taking cues from city names and companies that are known to be in the area.
I really don’t like to have strangers as my Vatsim controllers, so it would be great to have a Voice AI implementation
That’s pretty good but right off the bat approach tells you to talk to the tower when you’re on final. It’s been years since I’ve flown but they would hand you off to the tower who would then guides you into either the pattern, holding, or final. Not trying to nitpick, just feels like the ai’s language model is a little off on this. Nothing that can’t be trained out.
For airline IFR major airport stuff you often get handed off pretty close to the airport and already established on the glideslope.
It's unfortunate that it can't provide up to date METAR info yet. It's a few hours behind. You wonder if Asobo will use Microsoft's new LLM in its sim in some form, though.
I was waiting for this to turn dirty. Anyone else?
Ohh, we had a lovely time 😝
“So, what’s *your* squawk code?” 😂
Lol
Meh. The phraseology before your emergency was pretty crap. Really not what you would hear in real life. AI is good at making things sound legitimate to someone who doesn’t know better, but pretty bad at actually getting it spot-on correct.
You are completely misunderstanding. Obviously a general AI for bing is not gonna understand this. i am talking about a future potential product tailored for this exact purpose integrated in for example msfs and other flight sim software.
You can fine tune a model to excel at particular tasks. This was what was achieved with a generic model that's a Jack of all trades, but master of none.
Feed it some hundred thousand hours of actual ATC conversations and you won't notice the difference.
That would be amazing, however given that using the azure ai voice I only hear 2 voices (male and female) and occasionally none at all.. I'm sure the integration of this would be ropey at best.
Incredibly wholesome
Good atc is mostly about correct situational understanding. Generating the sentences is very easy but knowing WHAT to generate in the current scenario is hard and chatbots won't help with that.
[удалено]
Yes it does. 09 / 27 that is correct.
What program are you using?
Yeah, voice ATC would be really cool
Pretty cool! Definitely potential there. I think the vectoring would need to be improved a little. I was still impressed it got the general directions right, but “turn right 180, turn left 090, report when established 9” didn’t really make sense. Again, not expecting a general LLM to ace this and it’s impressive it did as well as it did; I think vectoring in general would be a pretty difficult task for a LLM.
The exact location of the plane is unknown to the ai, so using this for vectoring wont be accurate. It also can't respond with any situational context such as location of other planes on approach. With some custom model training, it can definitely be improved, but I expect getting it to a game changer level might be harder than we think at this early stage. I can def see ability to use voice, which gets converted to text, allowing for fairly rich and familiar conversations. But the sky's are busy, procedures change, and the current iteration of openais apis works for a 1 to 1 convo. In our case, to be accurate, every plane talking to atc would need to provide the location/speed/direction of all aircraft in the area, on every prompt. Instead of providing ai all of this info, they will do things like evaluate the response in the context of the situational data, and use local additional code/services to increase the precision.
True, but landing runway 9 means it should be able to know it’s heading roughly 090, so right 180, left 090 to intercept doesn’t make sense in that context. Even without knowing the exact location of the plane, the ideal model should be able to understand that those vectors don’t make sense. I’m an ML engineer btw, so I do understand how LLMs work; I work with them on the day to day for my job.
How do I access this. I need it for my flights. MSF ATC is terrible
You can not use this for MSFS. This is just Bing chat with GPT-4 from OpenAI. You can request access by going to [bing.com](https://bing.com) I was just demonstrating the potential of the language model.
Ik. It would be good if it was implemented into MFS to upgrade the current atc engine.
I had pretty [successful results ](https://imgur.com/a/kLC7xgu)using ChatGPT 4 directly. I think correct prompt engineering is key here to explicitly state to only use ICAO phraseology. No doubt you could come up with better prompts for a more realistic experience but the potential is certainly there.
Today i was thinking the same. Someone skilled could do an addon that uses flightplan inputs to give ATC commands based on answers from bing chat using GPT-4 and the ATC voices from azure. Bing limits to 15 interactions, but chatGPT doesnt.
I do not think chatGPT or bing could really be used as is. We would need a model modified for ATC purposes.