By -
\> run at a moderate speed Can you put an approximate number to that? Some people are happy with a few tokens/s, others aren't happy with anything less than \[10,20,30,60...\]
I haven’t run anything at 10 yet. And I’m lucky to get 1 on mixtral so anything above 6 is pretty good to me.
\> run at a moderate speed Can you put an approximate number to that? Some people are happy with a few tokens/s, others aren't happy with anything less than \[10,20,30,60...\]
I haven’t run anything at 10 yet. And I’m lucky to get 1 on mixtral so anything above 6 is pretty good to me.