MAFALDA: A Benchmark and Comprehensive Study of Fallacy Detection and Classification
arxiv(2023)
摘要
We introduce MAFALDA, a benchmark for fallacy classification that merges and
unites previous fallacy datasets. It comes with a taxonomy that aligns,
refines, and unifies existing classifications of fallacies. We further provide
a manual annotation of a part of the dataset together with manual explanations
for each annotation. We propose a new annotation scheme tailored for subjective
NLP tasks, and a new evaluation method designed to handle subjectivity. We then
evaluate several language models under a zero-shot learning setting and human
performances on MAFALDA to assess their capability to detect and classify
fallacies.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要