Efficiently Estimating Mutual Information Between Attributes Across Tables
CoRR(2024)
摘要
Relational data augmentation is a powerful technique for enhancing data
analytics and improving machine learning models by incorporating columns from
external datasets. However, it is challenging to efficiently discover relevant
external tables to join with a given input table. Existing approaches rely on
data discovery systems to identify joinable tables from external sources,
typically based on overlap or containment. However, the sheer number of tables
obtained from these systems results in irrelevant joins that need to be
performed; this can be computationally expensive or even infeasible in
practice. We address this limitation by proposing the use of efficient mutual
information (MI) estimation for finding relevant joinable tables. We introduce
a new sketching method that enables efficient evaluation of relationship
discovery queries by estimating MI without materializing the joins and
returning a smaller set of tables that are more likely to be relevant. We also
demonstrate the effectiveness of our approach at approximating MI in extensive
experiments using synthetic and real-world datasets.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要