Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generation
CoRR(2024)
摘要
To ensure that text generated by large language models (LLMs) is in an
expected format, constrained decoding proposes to enforce strict formal
language constraints during generation. However, as we show in this work, not
only do such methods incur performance overhead during generation, but many of
them also significantly impair task accuracy, if they do not correctly align
the underlying LLM sub-word vocabularies with external constraints. To address
this, we present a novel decoding algorithm, DOMINO, that can enforce
constraints in a fully subword-aligned fashion, while leveraging
pre-computation and speculative decoding to achieve virtually no overhead and
in some cases even almost 2× speedup over unconstrained decoding –
thereby outperforming existing approaches by a wide margin.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要