The model outputs meaningless repetitive output and never end.
Hello, I'm tring to deploy this model on my L20 devices but it not works well.
Here is my serve command:
vllm serve /test/Kimi-K2.5-E304
--served-model-name default_model
--trust-remote-code
--tensor-parallel-size 8
--max-model-len 60000
--max-num-seqs 8
--enable-auto-tool-choice
--dtype bfloat16
--gpu-memory-utilization 0.97
--disable-custom-all-reduce
--enable-expert-parallel
--tool-call-parser kimi_k2
--reasoning-parser kimi_k2
I think the first 100 tokens should be perfectly normal, but then it starts outputting meaningless words. Like this：
based upon actual earning capacity versus spending commitments already locked into place making adjustments where necessary staying mindful always about building toward future security through disciplined saving habits maintained consistently over time despite occasional temptations toward overspending arising unexpectedly now then requiring vigilance self-control exercised wisely ultimately ensuring financial wellbeing preserved protected strengthened continuously growing steadily ever forward progressing surely successfully achieving dreams envisioned hoped planned prepared pursued persistently patiently positively optimistically enthusiastically energetically dynamically vibrantly vitally alive awake aware alert attentive focused
Does this happen in your deployment?