RankSHAP: a Gold Standard Feature Attribution Method for the Ranking Task
arxiv(2024)
摘要
Several works propose various post-hoc, model-agnostic explanations for the
task of ranking, i.e. the task of ordering a set of documents, via feature
attribution methods. However, these attributions are seen to weakly correlate
and sometimes contradict each other. In classification/regression, several
works focus on axiomatic characterization of feature attribution
methods, showing that a certain method uniquely satisfies a set of desirable
properties. However, no such efforts have been taken in the space of feature
attributions for the task of ranking. We take an axiomatic game-theoretic
approach, popular in the feature attribution community, to identify candidate
attribution methods for ranking tasks. We first define desirable axioms:
Rank-Efficiency, Rank-Missingness, Rank-Symmetry and Rank-Monotonicity, all
variants of the classical Shapley axioms. Next, we introduce Rank-SHAP, a
feature attribution algorithm for the general ranking task, which is an
extension to classical Shapley values. We identify a polynomial-time algorithm
for computing approximate Rank-SHAP values and evaluate the computational
efficiency and accuracy of our algorithm under various scenarios. We also
evaluate its alignment with human intuition with a user study. Lastly, we
theoretically examine popular rank attribution algorithms, EXS and Rank-LIME,
and evaluate their capacity to satisfy the classical Shapley axioms.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要