Probabilistic Routing for Graph-Based Approximate Nearest Neighbor Search
CoRR(2024)
摘要
Approximate nearest neighbor search (ANNS) in high-dimensional spaces is a
pivotal challenge in the field of machine learning. In recent years,
graph-based methods have emerged as the superior approach to ANNS, establishing
a new state of the art. Although various optimizations for graph-based ANNS
have been introduced, they predominantly rely on heuristic methods that lack
formal theoretical backing. This paper aims to enhance routing within
graph-based ANNS by introducing a method that offers a probabilistic guarantee
when exploring a node's neighbors in the graph. We formulate the problem as
probabilistic routing and develop two baseline strategies by incorporating
locality-sensitive techniques. Subsequently, we introduce PEOs, a novel
approach that efficiently identifies which neighbors in the graph should be
considered for exact distance computation, thus significantly improving
efficiency in practice. Our experiments demonstrate that equipping PEOs can
increase throughput on a commonly utilized graph index (HNSW) by a factor of
1.6 to 2.5, and its efficiency consistently outperforms the leading-edge
routing technique by 1.1 to 1.4 times.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要