Titre original (traduit automatiquement) : Fixing Tool-Call Accuracy in vLLM
Tool call accuracy is an important part of serving open-weight AI models. When your tool calls aren’t accurate, the model builds up on nonsensical context. We’ve been carrying our own fork of vLLM to hit 100% in tool call schema accuracy with Kimi and DeepSeek. Fortunately, vLLM 0. 31 added support for constrained decoding, a technique to guarantee schema-adherence.
By enabling it, you can get to 100% schema accuracy with the newer Kimi and DeepSeek models.
Source originale
ubicloud.com