See the Unseen: Better Context-Consistent Knowledge-Editing by Noises
CoRR(2024)
摘要
Knowledge-editing updates knowledge of large language models (LLMs) and
contributes to the interpretability and application of LLMs. However, knowledge
applying is context-consistent: LLMs can recall the same knowledge in different
contexts. Existing works ignore this property and the editing lacks
generalization. In this paper, we empirically find that the effects of
different contexts upon LLMs in recalling the same knowledge follow a
Gaussian-like distribution. We then sample Gaussian noises to simulate the
effects of different contexts when updating LLMs. By such, we can make LLMs see
the unseen contexts where the edited knowledge will be applied, therefore
improving the editing generalization. Experimental results on three LLMs
demonstrate the effectiveness of our methods and also distinguish our methods
from the others of fine-tuning LLMs by noises.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要