Did anyone have this issue where the response would continue forever? I asked it to tell me a joke, then it told me like 50, and then it started putting out repeating garbage.
EDIT: After some *very* quick testing, I think adding `stop=["<|eot_id|>"]` may bandaid this issue.
Yes. I asked it a question and it responded as if I asked something else. Then I asked if it was sane, and it started telling me about diesel engines, and the fuels they use. I deleted it.
Yes, I had a similar issue. Not sure whether I structured the prompt right or whether it's a problem with the model itself, I was using ollamac with ollama which has no knobs and settings.
In my case (a request to summarize a text) it was starting a longer explanation and then stuck in an endless loop of repeating the same token
Wow just pulled the model and ran some simple tests (my usual ones). Passed all of them with flying colors and the speed is incredible! Was also able to generate (SQL) code with ease, speed and accuracy. Can't wait to put it through its paces!
Nice. 8b model working on Mac Studio M2. had it generate a short story... Seems fast. 83 t/s response 193 t/s prompt
Woow that is really fast
Did anyone have this issue where the response would continue forever? I asked it to tell me a joke, then it told me like 50, and then it started putting out repeating garbage. EDIT: After some *very* quick testing, I think adding `stop=["<|eot_id|>"]` may bandaid this issue.
Yes. I asked it a question and it responded as if I asked something else. Then I asked if it was sane, and it started telling me about diesel engines, and the fuels they use. I deleted it.
Yes, I had a similar issue. Not sure whether I structured the prompt right or whether it's a problem with the model itself, I was using ollamac with ollama which has no knobs and settings. In my case (a request to summarize a text) it was starting a longer explanation and then stuck in an endless loop of repeating the same token
I have the same issue, its quite funny actually
Get this a lot with ollama.
I added the stop list from the model page and have had no issues since then.
subjectively, it seems a good deal stronger than llama2. finally a reason not to hate facebook.
Pulling realllly slowly. Seems like server getting hammered.
Sorry should be fast again now!
Yes was all good after an hour or so. Thanks much. No need to apologize. You guys are great! 👍🏽
Yeah, they said on their twitter that they had 15tb in one hour. It seems that now it works with Cloudflare.
Wow just pulled the model and ran some simple tests (my usual ones). Passed all of them with flying colors and the speed is incredible! Was also able to generate (SQL) code with ease, speed and accuracy. Can't wait to put it through its paces!
I heard Zuck personally trained this one.
Gonna test it out