Dimensionality reduction is often described by the output of an algorithm: a collection of points in fewer coordinates. There is another way to look at the problem. One can ask how the map producing those coordinates is assembled, and how that map changes when the input data change.
This note develops two complementary descriptions. At the geometric level, it separates a nonlinear change of coordinates from a linear projection. At the data-dependent level, it treats a dimension-reduction procedure as a map-valued rule and approximates that rule through local finite-dimensional models. The distinction matters: a factorization of one map is not, by itself, a model for a family of maps, and neither description automatically guarantees preservation of geometry or topology.
1. Deformation, projection, and coordinates
Let \(n>k\) and consider a differentiable map
\[ f:\Omega\subseteq\mathbb R^n\longrightarrow\mathbb R^k. \]
Fix a \(k\)-dimensional linear subspace \(H\subseteq\mathbb R^n\), let \(P_H:\mathbb R^n\to H\) be orthogonal projection, and let \(T:H\to\mathbb R^k\) be a linear isomorphism. The factorization of interest is
\[ \boxed{f=T\circ P_H\circ N}, \]
where \(N\) is a nonlinear change of the ambient coordinates. The three factors perform different operations:
- \(N\) reorganizes the geometry before information is discarded.
- \(P_H\) removes the components orthogonal to the chosen \(k\)-plane.
- \(T\) expresses the projected result in \(\mathbb R^k\).
Without a restriction on \(N\), this notation is too permissive to say much. A useful geometric interpretation is obtained when \(N\) is a local diffeomorphism: a differentiable change of coordinates with a differentiable inverse. In that case all local information loss takes place at the projection stage.
2. What the factorization actually guarantees
The following is a local statement. It follows from the inverse function theorem, but its proof exhibits the coordinate change explicitly.
Proposition (local projection form). Let \(f:\Omega\to\mathbb R^k\) be \(C^1\), and suppose \(\operatorname{rank}Df(x_0)=k\) at some \(x_0\in\Omega\). There are a neighborhood \(U\) of \(x_0\), a \(C^1\) diffeomorphism \(N:U\to N(U)\subseteq\mathbb R^n\), and a projection \(P:\mathbb R^n\to\mathbb R^k\) onto the first \(k\) coordinates such that
\[ f|_U=P\circ N. \]
Proof. Because \(Df(x_0)\) has rank \(k\), one can choose a linear map \(L:\mathbb R^n\to\mathbb R^{n-k}\) for which
\[ v\longmapsto\bigl(Df(x_0)v,Lv\bigr) \]
is invertible. Define
\[ N(x)=\bigl(f(x),L(x-x_0)\bigr). \]
Its derivative at \(x_0\) is exactly that invertible map. The inverse function theorem makes \(N\) a diffeomorphism after restricting to a sufficiently small neighborhood \(U\). Projecting \(N(x)\) onto its first \(k\) coordinates recovers \(f(x)\). \(\square\)
Identifying the first \(k\) coordinates with a chosen plane \(H\), and inserting an isomorphism \(T:H\to\mathbb R^k\), gives the form \(f=T\circ P_H\circ N\). Thus the decomposition is locally available at every regular point of \(f\).
There is a converse obstruction worth keeping in sight. If \(N\) is a local diffeomorphism and \(T\) is a linear isomorphism, then
\[ \operatorname{rank}D(T\circ P_H\circ N)(x)=k. \]
Consequently a map with a critical point cannot admit this factorization with a local diffeomorphism \(N\) across that point. For example, \(f(x,y)=x^2+y^2\) has zero derivative at the origin, so it cannot be written there as a rank-one projection after a local diffeomorphism.
The geometric picture is also local: where \(f\) has full rank, the sets \(f^{-1}(y)\) are locally smooth \((n-k)\)-dimensional fibers. The map \(N\) straightens those fibers into sets on which the first \(k\) coordinates are constant. This is a description of the structure of a smooth submersion, not a claim that arbitrary algorithms have globally well-behaved fibers.
3. When the coordinate change is a flow
One way to construct \(N\) is to evolve points under a vector field. For a sufficiently regular, locally defined time-dependent field \(V_t\), write
\[ \frac{d}{dt}x(t)=V_t(x(t)),\qquad x(0)=x. \]
Where the solutions exist, let \(\Phi_t(x)\) denote the resulting flow map. A deformation-based reduction takes \(N=\Phi_1\) and considers
\[ f=T\circ P_H\circ\Phi_1. \]
This construction puts a dynamical model behind the nonlinear stage. The choice of vector field specifies how points move before projection; it is additional information not determined by the factorization alone.
For a simple example, consider
\[ f(x,y)=x+y^2. \]
The autonomous vector field \(V(x,y)=(y^2,0)\) has flow
\[ \Phi_t(x,y)=(x+ty^2,y). \]
At time one, projection onto the first coordinate yields
\[ P\bigl(\Phi_1(x,y)\bigr)=x+y^2=f(x,y). \]
Here the nonlinear coordinate change is a global diffeomorphism with inverse \((u,v)\mapsto(u-v^2,v)\). This example makes the three-stage description concrete without requiring the flow to be a gradient flow.
A qualification. The local projection proposition does not imply that its coordinate change \(N\) is the time-one map of a specified vector field, much less of a gradient field \(-\nabla E\). Flow realizability and restrictions such as energy descent impose further conditions. Likewise, smoothness and invertibility of \(\Phi_1\) do not establish that the projected output preserves distances, homology, or other topological features.
4. A dimension-reduction algorithm is a map-valued rule
The preceding construction concerns a fixed map \(f\). A data-driven algorithm instead produces a potentially different map for each input dataset or distribution. A useful abstract model is
\[ \mathcal F:\mathcal P(\mathbb R^n)\longrightarrow \mathcal C(\mathbb R^n,\mathbb R^k), \qquad \mu\longmapsto f_\mu, \]
where \(\mathcal P(\mathbb R^n)\) denotes probability measures and \(\mathcal C\) a chosen class of continuous maps. This is initially only notation: to speak of continuity, derivatives, or approximation errors for \(\mathcal F\), one must specify how measures and output maps are parameterized and compared.
For a finite-dimensional surrogate, choose probability measures \(\mu_1,\ldots,\mu_M\) and let
\[ \mu_p=\sum_{j=1}^M p_j\mu_j, \qquad p_j\ge0,\quad \sum_{j=1}^M p_j=1. \]
The mixture coefficients belong to an \((M-1)\)-dimensional simplex. Work in a local coordinate chart \(z\in U\subseteq\mathbb R^{M-1}\) of its interior, with corresponding mixture \(\mu_z\). Suppose the output maps are approximated using fixed feature maps \(\phi_1,\ldots,\phi_N:\mathbb R^n\to\mathbb R^k\):
\[ f_{\mu_z}(\xi)\approx\sum_{i=1}^N B_i(z)\phi_i(\xi). \]
Then \(B:U\to\mathbb R^N\) is a finite-dimensional coefficient rule. Its definition depends on a specified fitting or projection procedure for the features. Treating \(B\) as differentiable is an assumption to be checked, not a consequence of writing \(\mathcal F\).
5. Local second-order models and interpolation
Assume \(B\) is \(C^3\) on an open set \(U\subseteq\mathbb R^{M-1}\). Around an anchor \(a\in U\), its second-order model is
\[ \mathcal T_a(z)=B(a)+DB(a)(z-a) +\tfrac12 D^2B(a)[z-a,z-a]. \]
The linear and bilinear terms describe, respectively, first- and second-order sensitivity of the output coefficients to changes in the input mixture. In a computational setting they can be estimated from evaluations of the underlying algorithm at nearby parameter values, with accuracy depending on the sampling design and conditioning.
One local model need not approximate \(B\) over all of \(U\). Given anchors \(a_1,\ldots,a_s\) and nonnegative weights \(w_i(z)\) with \(\sum_i w_i(z)=1\), form
\[ \widehat B(z)=\sum_{i=1}^s w_i(z)\mathcal T_{a_i}(z). \]
This makes the dependence on nearby input distributions explicit. It also admits a simple conditional error estimate.
Proposition (weighted Taylor error). Suppose, for a query \(z\), every segment joining \(z\) to an anchor with nonzero weight lies in \(U\), and \(\|D^3B\|\le M_3\) along those segments. If every such anchor satisfies \(\|z-a_i\|\le r\), then
\[ \bigl\|B(z)-\widehat B(z)\bigr\| \le \frac{M_3}{6}r^3. \]
Proof. Taylor’s theorem along the segment from \(a_i\) to \(z\) gives \(\|B(z)-\mathcal T_{a_i}(z)\|\le M_3\|z-a_i\|^3/6\). Multiply each inequality by \(w_i(z)\), sum, and use nonnegativity and \(\sum_iw_i(z)=1\). \(\square\)
The estimate is about the approximation of \(B\), not about whether the dimension-reduction algorithm correctly reconstructs a manifold. In particular, it assumes the coefficient rule is smooth and that its derivatives, or their estimates, are reliable.
6. Two levels that should not be confused
The geometric factorization and the local surrogate answer different questions:
- For one \(f_\mu\), a factorization \(T\circ P_H\circ N\) asks how a nonlinear coordinate change and a projection combine to produce lower-dimensional coordinates.
- For a family \(\mu\mapsto f_\mu\), the finite-dimensional model \(B(z)\) asks how an algorithm’s output map responds to a change in the input measure.
Even if every \(f_{\mu_z}\) has a local projection form, it does not follow that the associated coordinate changes \(N_z\) can be chosen globally, uniquely, or smoothly in \(z\). Those are separate questions. A proposed flow-based algorithm must additionally specify its vector field or energy, its domain of existence, its stopping rule, and the properties it is meant to preserve.
The useful principle is therefore modest but structural: separate the nonlinear reorganization of geometry from the dimension-losing projection, and separate the analysis of an individual reduction map from the sensitivity of the procedure that generates it. Each separation makes its own assumptions visible.
Background reference
- John M. Lee, Introduction to Smooth Manifolds, 2nd ed., Springer, 2013, especially the treatments of submersions, the inverse function theorem, and flows. Publisher page.