DISA

When? August 21, 14.00-15.30
Where? D2272 or online https://lnu-se.zoom.us/j/68245999256?pwd=diVc4g6cCuanYHH5aN1lrnGMUZzkuW.1&from=addon

Abstract

A Geometric Perspective on Resolving the Likelihood Out-of-Distribution Paradox – Sebastian Hönel

Deep generative models can assign higher likelihood to supposedly out-of-distribution (OOD) samples than to in-distribution data. This apparent likelihood-OOD paradox is often interpreted as a failure of density estimation or falsely attributed to model pathologies. A geometric decomposition instead reveals several distinct mechanisms: a failure to distinguish between likelihood and typicality, the effect and dominance of high codimension that gives an increasingly transverse answer to the intrinsic likelihood question, and the non-identifiability of ambient likelihood; the latter of which is predominantly caused by ignorance.

The seminar includes a primer in differential geometry; introducing, for example, tangent and normal spaces, projectors, immersions and embeddings, together with strong and weak forms of the manifold hypothesis, to distinguish intrinsic manifold densities from ambient densities. We then introduce the tubular likelihood decomposition of the log ambient likelihood as the sum of an intrinsic-density term, a normal-space term, and a volume-change correction. This decomposition serves as an exhaustive oracle likelihood. It shows that ambient likelihood may be dominated by extrinsic variation, even when the intended OOD notion concerns intrinsic (semantic) variation, causing a (likely unexpected) likelihood reversal.

A central consequence exposed by the tubular decomposition is likelihood non-identifiability. When any of the intrinsic law, decoder (representation), or normal-space model is unknown, observations alone do not identify a unique ambient likelihood. Distinct representations and normal-space completions can agree on observed or near-manifold data while inducing substantially different likelihood values and rankings.

These effects are examined in controlled settings based on injective procedural decoders and differentiable image renderers. The generators provide known intrinsic variables, deterministic decoder Jacobians, projected normal displacements, and analytically specified ambient likelihoods. This enables decomposition of ground-truth likelihood into intrinsic, normal, and geometric components, construction of controlled likelihood reversals, and direct comparison between ground-truth ambient likelihoods and likelihoods learned by deep generative models, such as normalizing flows. The resulting framework clarifies what likelihood measures, when ambient likelihood is identifiable, and why likelihood-based OOD scores can fail even when ambient density estimation is formally well-defined.