Tempered Calculus for ML: Application to Hyperbolic Model Embedding
CoRR(2024)
摘要
Most mathematical distortions used in ML are fundamentally integral in
nature: f-divergences, Bregman divergences, (regularized) optimal transport
distances, integral probability metrics, geodesic distances, etc. In this
paper, we unveil a grounded theory and tools which can help improve these
distortions to better cope with ML requirements. We start with a generalization
of Riemann integration that also encapsulates functions that are not strictly
additive but are, more generally, t-additive, as in nonextensive statistical
mechanics. Notably, this recovers Volterra's product integral as a special
case. We then generalize the Fundamental Theorem of calculus using an extension
of the (Euclidean) derivative. This, along with a series of more specific
Theorems, serves as a basis for results showing how one can specifically
design, alter, or change fundamental properties of distortion measures in a
simple way, with a special emphasis on geometric- and ML-related properties
that are the metricity, hyperbolicity, and encoding. We show how to apply it to
a problem that has recently gained traction in ML: hyperbolic embeddings with a
"cheap" and accurate encoding along the hyperbolic vs Euclidean scale. We
unveil a new application for which the Poincaré disk model has very appealing
features, and our theory comes in handy: model embeddings for boosted
combinations of decision trees, trained using the log-loss (trees) and logistic
loss (combinations).
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要