Imagination Augmented Generation: Learning to Imagine Richer Context for Question Answering over Large Language Models
CoRR(2024)
摘要
Retrieval-Augmented-Generation and Gener-ation-Augmented-Generation have been
proposed to enhance the knowledge required for question answering over Large
Language Models (LLMs). However, the former depends on external resources, and
both require incorporating the explicit documents into the context, which
results in longer contexts that lead to more resource consumption. Recent works
indicate that LLMs have modeled rich knowledge, albeit not effectively
triggered or activated. Inspired by this, we propose a novel
knowledge-augmented framework, Imagination-Augmented-Generation (IAG), which
simulates the human capacity to compensate for knowledge deficits while
answering questions solely through imagination, without relying on external
resources. Guided by IAG, we propose an imagine richer context method for
question answering (IMcQA), which obtains richer context through the following
two modules: explicit imagination by generating a short dummy document with
long context compress and implicit imagination with HyperNetwork for generating
adapter weights. Experimental results on three datasets demonstrate that IMcQA
exhibits significant advantages in both open-domain and closed-book settings,
as well as in both in-distribution performance and out-of-distribution
generalizations. Our code will be available at
https://github.com/Xnhyacinth/IAG.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要