Repetition makes this unusable
#17
by zero1zero - opened
Added variations of small to large repetition_penalty and/or presence_penalty and it continuously had problems. I stayed with the recommended greedy decoding but gave low temp a try and it still showed issues.
Seems like this must be an inherent issue with the model? Has anyone seen this work without eventually getting into a repeat loop?
This seems to happen mainly with table structure or visual elements where it wants to use "[-|_|.]" to represent something. This seems like its related to table parsing but also happens on representing visual content or spacing. I'm using the "OCR" task.
Hosted on a 4090 through vllm.