I felt sorry for it and stopped it, 14min and the poor thing was just going around in circles.
I’m just looking at running some local llm on my m2 16GB mac and this was with the default settings for the qwen3.5 9b in LM Studio. Any pointers to get the best out of a M2?
i guess my advice is misplaced, given the community name, but in case the goal is to get LLM inference for free, not necessarily a local one, you could try opencode (their free plan gives you access to some decent models), or nvidia’s NIM if you don’t want a vendor lock-in to any specific agent/chat app.
I assume NIM only works with nvidia cards not AMD?
it’s a cloud service. It gives you access to an api. It couldn’t care less about your GPU. Can be used on anything that is capable of internet connection. Specifically NIM has a max requests per minute restriction tho. Due to this, running ai agents on it would be a pain in the ass, but technically possible (still not as painful as local inference tho). And it still works fine for plain chats.