I totally get your point but it really depends on what scope you define. For me, coming from Gnome and not feeling like constructing the whole shell aspect of my desktop myself, these tools do that job very well.

Also Noctalia v5, it’s a joy to use!

There is the Swiss project Apertus AI, and I think I’ve read about others but don’t remember the names.

That being said, if the benchmarks translate to real utility, it is the first competitive open source model, I’d say.

The service starts as it should, right? The errors appear only on request, right? Any other sections in the compose logs that seem relevant?

You use podman-compose with the Podman backend?

And there are no volumes mounted, just to be sure?

No expert but don’t the logs look like the requests are coming through?

I guess some people consider it cool. I’m not one of them.

Yeah right? At least a layer of VPN on top.

Hugging Face used the incident to promote running open-weights LLMs on-prem, which a large part of their business focusses on.

Their blog post

Pixelfed

I’ve used KISS before, but moved to Blue Line Console some months ago. I’d say it’s a bit more opinionated and less customizable than KISS but I guess I like the opinions.

Well compared to the strix, 400GB/s is not that bad, I think with fast system RAM and expert offloading you could squeeze quite something out of it when running stuff in the 100b-a10b regions.

Your bigger problem is going to be future software support.

In case you missed the Ornith 1.0 release (Qwen and Gemma RL finetunes for agentic / coding workloads), they look interesting to bridge the gap until we see larger 3.6 models or a 3.7 release. I didn’t test them yet but according to benchmarks, the 35b MoE seems to be more or less on par with Qwen3.6 27b dense, while ofc a lot faster.

You can control how much context should be fitted with --fit-ctx and how much space the algorithm should leave unallocated (even on a per-GPU basis) with --fit-target.

I currently run Qwen3.6-27b on llama.cpp and use it via openwebui. Mostly, I use it for web research via tavily, to a lesser extent for coding and interactively learning about things that are new to me but common in training data (such as basic math or ML concepts).

Given the 27b is a dense model, I think the numbers are quite ok. Curious about the quant tho.

The cool thing about the strix is its large unified memory, but it lacks memory bandwith for compute intensive workloads. Something like Qwen3.5-122b MoE with only like 12b active parameters might run at twice the speed if it fits the configuration.

Since implementation of the --fit parameter and its relatives, and --fit on becoming the default, llama.cpp intelligently decides what to offload. For me, it made --n-cpu-moe obsolete.

You might want to check out heretic or similar tools. I did not try it but there are a lot of heretic finetunes available ond HF.

A bunch

Late to the party, but this was just released: LiquidAI/LFM2.5-8B-A1B-GGUF

I guess you could fully fit it at Q4 with a little context if you need all the speed you can get, or offload the experts to RAM if you prefer higher quality and/or more context.