FYI
If you use Gemma 4 E4B with llama.cpp, do yourself a favour and grab Google’s latest official chat template:
https://huggingface.co/google/gemma-4-E4B-it
https://huggingface.co/google/gemma-4-E4B-it/blob/main/chat_template.jinja
Use it with --jinja --chat-template-file chat_template.jinja.
The improvement in tool use is remarkable but it also seems to improve conversation continuity:
“Here is XYZ. Now add ABC. Now DEF. Actually, what were Y, B and C?”
That makes a big difference is multi step reasoning IME
Sadly, still no fixes for the other llama.cpp quirks.
- llama.cpp still reserves separate compute buffers for the target and MTP contexts, despite never using them concurrently.
That wastes about 1 GB of VRAM / causes OOM
PR #27489 fixes it, but remains not merged.
https://github.com/ggml-org/llama.cpp/pull/27489
- Audio also still fails through the llama.cpp WebUI.
The model itself can “hear”. just fine - a direct curl request confirms that.
(“In this sound clip, the man says XYZ.”).
See #PR21825
https://github.com/ggml-org/llama.cpp/issues/21825
Dunno how these are resolved in other back ends / front ends, so just an FYI
Do I still need this chat template if I’m using the Unsloth version?
Not sure, sorry.
I moved to lemonade, it works better apparently but I haven’t used it much yet. Not sure if this would be able to be incorporated but I’ll try remember to check
Seems more like ollama as approach. More easy but less optimized than llamacpp?
