ParallelPARC: A Scalable Pipeline for Generating Natural-Language Analogies
CoRR(2024)
摘要
Analogy-making is central to human cognition, allowing us to adapt to novel
situations – an ability that current AI systems still lack. Most analogy
datasets today focus on simple analogies (e.g., word analogies); datasets
including complex types of analogies are typically manually curated and very
small. We believe that this holds back progress in computational analogy. In
this work, we design a data generation pipeline, ParallelPARC (Parallel
Paragraph Creator) leveraging state-of-the-art Large Language Models (LLMs) to
create complex, paragraph-based analogies, as well as distractors, both simple
and challenging. We demonstrate our pipeline and create ProPara-Logy, a dataset
of analogies between scientific processes. We publish a gold-set, validated by
humans, and a silver-set, generated automatically. We test LLMs' and humans'
analogy recognition in binary and multiple-choice settings, and found that
humans outperform the best models ( 13
demonstrate that our silver-set is useful for training models. Lastly, we show
challenging distractors confuse LLMs, but not humans. We hope our pipeline will
encourage research in this emerging field.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要