MTP support? Why no Vision?
base qwen models had vision already for you to reuse, also MTP, did you retrain the MTP layers?
If not, why not include them, vision and MTP are free lunch
Thanks for your thoughtful questions!
Our optimization for KAT-Coder-V2.5-Dev is focused purely on agentic coding scenarios. This checkpoint targets text-only code agent tasks, and we have not conducted any training for vision capabilities. Therefore, we do not recommend using this model for multimodal vision-related workloads. If you require vision support, the original Qwen3.6-35BA3B base model would be the better choice.
Thanks for your thoughtful questions!
Our optimization for KAT-Coder-V2.5-Dev is focused purely on agentic coding scenarios. This checkpoint targets text-only code agent tasks, and we have not conducted any training for vision capabilities. Therefore, we do not recommend using this model for multimodal vision-related workloads. If you require vision support, the original Qwen3.6-35BA3B base model would be the better choice.
But agentic coders benefit when they can see what they have actually build :p
(I think?) ornith also did not particulary train on vision, yet the model retains quite alot from the base.
That being said, just for testing purposes, would it be possible to wire e.g. qwen3.6's vision tower into this model to see if it retained any capabilities?
Just having vision additionally is just a nice thing to have, even if its not perfect, it can help with development
And just loading it with the model wouldnt hurt the text part either right?
I'm pretty sure you can just cut and paste the MM weights from base 35B into your model and gain the same vision. Same with MTP except you'd have to retain MTP layers on the dataset.
"Our optimization for KAT-Coder-V2.5-Dev is focused purely on agentic coding scenarios."
Which may require browser screenshots, etc. to see renders, shaders, whatevers.
Coder models must see for web development.
Also thanks for this great model, I've tested on a real pipeline yesterday, it successfully completed with high quality.
Skills, /goal and /todo from the harness also helps a lot for best results.
No vision is dealbreaker here, for GUI coding with opencode LLM constantly makes screenshots to check the UI bugs. How do you suppose one can tell in words to LLM what is wrong in UI???
A picture is worth a thousand words for those that do not know. Nice model but unusable for WEB Dev.
just use an mmproj from the same architecture- tried this before on qwen 3.5 fine tunes
just use an mmproj from the same architecture- tried this before on qwen 3.5 fine tunes
Tried, not working. The model keeps saying it can't process the image files. It makes no difference whether you attach them or read them from the disk.
just use an mmproj from the same architecture- tried this before on qwen 3.5 fine tunes
Tried, not working. The model keeps saying it can't process the image files. It makes no difference whether you attach them or read them from the disk.
Then the vision experts are probably dead
just use an mmproj from the same architecture- tried this before on qwen 3.5 fine tunes
Tried, not working. The model keeps saying it can't process the image files. It makes no difference whether you attach them or read them from the disk.
If you check the jinja template, there seems to be a section there that checks of the content is image or video and bailing out if it is. Removal of this section might get the mmproj going...
just use an mmproj from the same architecture- tried this before on qwen 3.5 fine tunes
Tried, not working. The model keeps saying it can't process the image files. It makes no difference whether you attach them or read them from the disk.
If you check the jinja template, there seems to be a section there that checks of the content is image or video and bailing out if it is. Removal of this section might get the mmproj going...
I'm using the same template I'm already using for Qwen 3.6 35B-A3B MTP Vision-ON APEX I-Balance; It has eyes.
This one doesn't. :<
Then it's unusable for Web Dev, no vision no go.
I'm glad there is no MTP crap that degrades code quality. The model is working very good after finding the better parameters then the model card. The vision is working despite the "the vision/multimodal components are not included and are unavailable." Why no vision was released, is this by stupidity or by accident? Web Dev needs vision to debug with chrome dev tools take screenshot actions.
I'm glad there is no MTP crap that degrades code quality.
MTP doesn't degrade the code quality,
actually vice versa;
MTP is protecting from the generation degradation.
Without MTP/Ngram-Mod
35 ts
...
33 ts
...
25 ts
...
and so on
...
12 ts
With MTP/Ngram-Mod
35 ts
...
...
35 ts
...
...
35 ts
Terrific agentic model. 100% delivers on solving the structural repetition alucinations of qwen 3.6.
35 t/s on a 9070XT is very good (no think on ollama, temp 6). I'm very curious about how it would perform with MTP/DFlash.
Terrific agentic model. 100% delivers on solving the structural repetition alucinations of qwen 3.6.
35 t/s on a 9070XT is very good (no think on ollama, temp 6). I'm very curious about how it would perform with MTP/DFlash.
it works very good with me TBH, I'm enjoying it...
llama-server --model Kwaipilot_KAT-Coder-V2.5-Dev-MTP-UD-Q5_K_XL.gguf --jinja --ctx-size 65536 --n-gpu-layers 99 --parallel 1 -lv 3 --threads 6 --threads-batch 10 --ubatch-size 2048 --cache-type-k q8_0 --cache-type-v turbo2 --flash-attn on --temp 1.0 --top-k 20 --top-p 0.95 --host 127.0.0.1 --port 8080 --presence-penalty 1.5 --cache-ram 512 --chat-template-file chat-template-mudler-apex.jinja --fit off --n-cpu-moe 35 --no-mmap --spec-draft-n-max 1 --spec-draft-n-min 0 --spec-draft-p-min 0.9 --spec-ngram-mod-n-min 8 --spec-ngram-mod-n-max 64 --spec-ngram-mod-n-match 48 --spec-type draft-mtp,ngram-mod --reasoning-format auto
LLAMA_ARG_CHAT_TEMPLATE_KWARGS='{"preserve_thinking":true}'
LlamaCpp with TurboQuant:
RTX 3070 8GB
32GB RAM 4800Mhz
i7-12700H CPU
SAMSUNG MZVL21T0HCLR NVMe M.2
I prefer model without vision than model with vision and quality downgrade. I am also a web developer. Some models still have no vision like Deepseek4, and the way to prompt is explain what "unexpected" and what "expected" UI. UI bugs live in CSS and not that hard to describe or debug manually (if you are a web developer)
Its super easy to add vision if you are using llama.cpp and the corresponding gguf/mmproj.
I just tested it now using the unsloth mmproj and adding this flag to llama-server: --mmproj /home/models/mmproj-BF16.gguf
OCR worked and picture descriptions worked as well.
It didn't work with me, I'll try again with your mmproj.
Its super easy to add vision if you are using llama.cpp and the corresponding gguf/mmproj.
I just tested it now using the unsloth mmproj and adding this flag to llama-server:
--mmproj /home/models/mmproj-BF16.ggufOCR worked and picture descriptions worked as well.
how about vllm vision tower? model has preprocessor config and model.visual.*
"Its super easy to add vision if you are using llama.cpp and the corresponding gguf/mmproj.
I just tested it now using the unsloth mmproj and adding this flag to llama-server: --mmproj /home/models/mmproj-BF16.gguf
OCR worked and picture descriptions worked as well." I have tried use this mmproj, and it works. Thanks!