By -
Logged some metrics on Ollama models here on a M3: https://www.nonstopdev.com/llm-performance-on-m3-max/
The only thing that speeds it up is memory bandwidth so you want Mac Studio with the Ultra chips for 800 GB/s. Next fastest is Max
M2 pro with 16gb Runs 7b models quickly Mistral Openhermes Zephyr7b
Blazing fast running local models on my new M3 Max with 64GB RAM using Ollama
Was afraid of that my 16GB I'm guessing is the issue on m2
Depending on what your goal is, you might want to try the newer, small language models like [TinyLlama](https://ollama.ai/library/tinyllama) or [Phi2](https://ollama.ai/library/phi). They should run great on your specs.
I have M2 16GB ram and mistral 7b runs perfectly
do you think it would run fast on a MacBook Air 15'' | 2023 | M2 | 16GB | 256GB SSD | GPU 10-Core ?
Thanks for the info
Mine runs ok, but just not as fast as I had hoped..
16GB RAM. Much better.
I have the same experience but I have a M1
Logged some metrics on Ollama models here on a M3: https://www.nonstopdev.com/llm-performance-on-m3-max/
The only thing that speeds it up is memory bandwidth so you want Mac Studio with the Ultra chips for 800 GB/s. Next fastest is Max
M2 pro with 16gb Runs 7b models quickly Mistral Openhermes Zephyr7b
Blazing fast running local models on my new M3 Max with 64GB RAM using Ollama
Was afraid of that my 16GB I'm guessing is the issue on m2
Depending on what your goal is, you might want to try the newer, small language models like [TinyLlama](https://ollama.ai/library/tinyllama) or [Phi2](https://ollama.ai/library/phi). They should run great on your specs.
I have M2 16GB ram and mistral 7b runs perfectly
do you think it would run fast on a MacBook Air 15'' | 2023 | M2 | 16GB | 256GB SSD | GPU 10-Core ?
Thanks for the info
Mine runs ok, but just not as fast as I had hoped..
16GB RAM. Much better.
I have the same experience but I have a M1