KoboldCpp version
Works:
1.115
Broken:
1.118.1
Environment
OS:
Windows 10 x64 (19045)
GPU:
RTX 2060 6 GB
CPU:
Intel i5 (AVX only)
Backend:
CUDA
Models tested
Works in both versions
Impish_LLAMA_V2-Q8_0.gguf
Broken in 1.118.1
Lumimaid-v0.2-8B-Humanlike.Q4_K_M.gguf
Impish_Bloodmoon-Q4_K_M.gguf
Partially broken
Rocinante-X-12B-v1b-Q4_K_M.gguf
Problem
After updating from KoboldCpp 1.115 to 1.118.1, several ChatML-based GGUF models started producing corrupted or unrelated outputs.
The exact same models and prompts work correctly in 1.115.
No launcher settings were intentionally changed except context size.
Reproduction
Using llama-ui or OpenAI API.
Prompt:
Repeat exactly: 123456789
Expected:
123456789
Actual (Lumimaid):
355670000
That's 0-9 repeated...
...
Next prompt:
Hello
Expected:
Hello!
Actual:
eraldi: Would you like to know about the American Civil War...
Next prompt:
What is 2+2?
Expected:
4
Actual:
Large block of repeated American Civil War text.
Another example (Rocinante-X)
Conversation begins with unrelated garbage:
YoYoüngi) a 2:
not Yoüngi...
updateStatefulset...
\end{table}
...
After that, normal prompts work.
Comparison
Exactly the same Lumimaid model under KoboldCpp 1.115:
Prompt
Hello
Output
Hey there! How's your day going so far? 😊
Prompt
2+2=
Output
2+2 = 4
Prompt
The capital of France is
Output
The capital of France is Paris.
No corruption.
Additional observations
The issue is model-dependent.
Models that appear to use plain completion templates still work.
Models that rely on ChatML formatting seem affected much more often.
Impish_LLAMA_V2-Q8_0 works correctly under 1.118.1.
Lumimaid and Impish_Bloodmoon do not.
Possible regression
This appears to have started after upgrading from 1.115 to 1.118.x.
It may be related to one of:
ChatML prompt formatting
AutoGuess chat template detection
tokenizer handling
EOS/EOT token processing
recent llama.cpp merge
Attachments
KoboldCpp 1.118.1 startup log
KoboldCpp 1.115 startup log
failing request logs
working request logs
I can consistently reproduce this issue by simply replacing KoboldCpp 1.118.1 with 1.115 while keeping the same GGUF model, same prompts, and same settings. The problem disappears immediately when reverting to 1.115.
KoboldCpp version
Works:
1.115
Broken:
1.118.1
Environment
OS:
Windows 10 x64 (19045)
GPU:
RTX 2060 6 GB
CPU:
Intel i5 (AVX only)
Backend:
CUDA
Models tested
Works in both versions
Impish_LLAMA_V2-Q8_0.gguf
Broken in 1.118.1
Lumimaid-v0.2-8B-Humanlike.Q4_K_M.gguf
Impish_Bloodmoon-Q4_K_M.gguf
Partially broken
Rocinante-X-12B-v1b-Q4_K_M.gguf
Problem
After updating from KoboldCpp 1.115 to 1.118.1, several ChatML-based GGUF models started producing corrupted or unrelated outputs.
The exact same models and prompts work correctly in 1.115.
No launcher settings were intentionally changed except context size.
Reproduction
Using llama-ui or OpenAI API.
Prompt:
Repeat exactly: 123456789
Expected:
123456789
Actual (Lumimaid):
355670000
That's 0-9 repeated...
...
Next prompt:
Hello
Expected:
Hello!
Actual:
eraldi: Would you like to know about the American Civil War...
Next prompt:
What is 2+2?
Expected:
4
Actual:
Large block of repeated American Civil War text.
Another example (Rocinante-X)
Conversation begins with unrelated garbage:
YoYoüngi) a 2:
not Yoüngi...
updateStatefulset...
\end{table}
...
After that, normal prompts work.
Comparison
Exactly the same Lumimaid model under KoboldCpp 1.115:
Prompt
Hello
Output
Hey there! How's your day going so far? 😊
Prompt
2+2=
Output
2+2 = 4
Prompt
The capital of France is
Output
The capital of France is Paris.
No corruption.
Additional observations
The issue is model-dependent.
Models that appear to use plain completion templates still work.
Models that rely on ChatML formatting seem affected much more often.
Impish_LLAMA_V2-Q8_0 works correctly under 1.118.1.
Lumimaid and Impish_Bloodmoon do not.
Possible regression
This appears to have started after upgrading from 1.115 to 1.118.x.
It may be related to one of:
ChatML prompt formatting
AutoGuess chat template detection
tokenizer handling
EOS/EOT token processing
recent llama.cpp merge
Attachments
KoboldCpp 1.118.1 startup log
KoboldCpp 1.115 startup log
failing request logs
working request logs
I can consistently reproduce this issue by simply replacing KoboldCpp 1.118.1 with 1.115 while keeping the same GGUF model, same prompts, and same settings. The problem disappears immediately when reverting to 1.115.