Human Expertise in Algorithmic Prediction
arxiv(2024)
摘要
We introduce a novel framework for incorporating human expertise into
algorithmic predictions. Our approach focuses on the use of human judgment to
distinguish inputs which `look the same' to any feasible predictive algorithm.
We argue that this framing clarifies the problem of human/AI collaboration in
prediction tasks, as experts often have access to information – particularly
subjective information – which is not encoded in the algorithm's training
data. We use this insight to develop a set of principled algorithms for
selectively incorporating human feedback only when it improves the performance
of any feasible predictor. We find empirically that although algorithms often
outperform their human counterparts on average, human judgment can
significantly improve algorithmic predictions on specific instances (which can
be identified ex-ante). In an X-ray classification task, we find that this
subset constitutes nearly 30
a natural way of uncovering this heterogeneity and thus enabling effective
human-AI collaboration.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要