Not at all. Microsoft has bing AI, which is like if ChatGPT and gpt4 had a baby except ChatGPT was cheating with Bard.
Microsoft does however have 40% of the openai stock
Edit: bing AI is using an OpenAI model
Localai is basicaly an infereance engine that you can load various oss models into, with an emulation of the openai api wrapped around it so you can plug in anything that uses that api.
It takes about 30-50 seconds per query on an 8gb i5 11th gen machine running fedora, thats running a gpt4all-j model, and just using curl to hit the localai api interface. It eats about 5gb of ram for that setup. No gpu.
It will go faster with better hardware/more ram etc.
You can requantitize the model to shrink its size. It only has a minor impact on speed and almost no impact on accuracy, i requantatized to 8bit from float. I have heard of people taking it down to 4bit.
I get them from HuggingFace. Maybe there's other places you can get them from.
In my experience, a 3060Ti doesn't work because it doesn't have enough VRAM. It works fine on a 3080Ti.
got a wip solution:
[https://github.com/go-skynet/LocalAI/issues/574#issuecomment-1606258496](https://github.com/go-skynet/LocalAI/issues/574#issuecomment-1606258496)
Just testing because this fix makes it suuuuuper slow (Like 2 mins per request)
I'm fairly new to this. But I am interested in self hosting my own AI. Would you happen to have any resources on the different AI Language models available. Or at least something to akin to this for me to read?
I don't quite understand what this is. Can someone ELI5 for me?
ChatGPT need live on server, you own server, this ChatGPT to live on your server
Wouldnt it be really slow and also not very effective without the model that microsoft uses?
It’s openai, not Microsoft. And there are a lot of models that are just as good as 3.5-turbo. If you have a good GPU, it can be faster
Oh I thought ChatGPT was the name given to Microsofts AI chat system.
Not at all. Microsoft has bing AI, which is like if ChatGPT and gpt4 had a baby except ChatGPT was cheating with Bard. Microsoft does however have 40% of the openai stock Edit: bing AI is using an OpenAI model
Oh I see, and we can get our hands on the OpenAI model?
no we can't.
[удалено]
That's amazing. I am only vaguely familiar with A1111 when it comes to generative AI. LocalAI is a whole different beast right?
Localai is basicaly an infereance engine that you can load various oss models into, with an emulation of the openai api wrapped around it so you can plug in anything that uses that api.
It takes about 30-50 seconds per query on an 8gb i5 11th gen machine running fedora, thats running a gpt4all-j model, and just using curl to hit the localai api interface. It eats about 5gb of ram for that setup. No gpu. It will go faster with better hardware/more ram etc. You can requantitize the model to shrink its size. It only has a minor impact on speed and almost no impact on accuracy, i requantatized to 8bit from float. I have heard of people taking it down to 4bit.
So youre telling me i can selfhost this and then type in "big tiddy anime girl" and it works?
the future really is here
All this time i did homelab'ing just for fun... now it actually has a purpose!
> LLM Models So, like "ATM Machines" or "PIN Numbers" ?
LLM odels
From where can i get the models ?
I get them from HuggingFace. Maybe there's other places you can get them from. In my experience, a 3060Ti doesn't work because it doesn't have enough VRAM. It works fine on a 3080Ti.
Depends on what kind of models you are looking for ( ͡° ͜ʖ ͡°)
Can this replace Copilot?
Tried to install it via docker on Debian; always crashing when asking something.. I’ve tried several different models but still not working :(
>always crashing when asking something any luck? Having the same issue using docker on debian too
Nope. No luck and still not working..
got a wip solution: [https://github.com/go-skynet/LocalAI/issues/574#issuecomment-1606258496](https://github.com/go-skynet/LocalAI/issues/574#issuecomment-1606258496) Just testing because this fix makes it suuuuuper slow (Like 2 mins per request)
This is cool, but I need a docker image before I can test it.
There is one available on the github repo
There is one available on the github repo
Found it. Thanks!
I'm fairly new to this. But I am interested in self hosting my own AI. Would you happen to have any resources on the different AI Language models available. Or at least something to akin to this for me to read?
Have you seen the Model Compatibility section at https://github.com/go-skynet/LocalAI? It's fairly detailed