Greedy‑DiM: Greedy Algorithms for Unreasonably Effective Face Morphs

Existing DiM attacks leave the diffusion model as a black box

A face morphing attack creates a single image which contains the biometric information of multiple identities, such that a face recognition (FR) system registers a false acceptance with any of the contributing identities. Diffusion Morphs (DiM) are the current state-of-the-art for representation‑based morphing attacks: the two bona fide images are encoded into the twin latents of a Diffusion Autoencoder, blended, and decoded by solving the probability-flow ODE. Whilst DiM attains state-of-the-art performance, none of the existing work on DiM has leveraged the iterative nature of the diffusion model — the DiM model is treated as a black box, no differently than one would a GAN or a VAE.

Specifically, vanilla DiM fixes a blend of $w=0.5$, runs the reverse ODE once, and accepts whatever falls out. Morph‑PIPE proposes an extension: a batch of 21 morphs is generated at different blend values, each run to completion, and the candidate which minimizes an ArcFace identity loss is selected. In both cases the $N$ intermediate noise predictions along the diffusion trajectory go entirely unused.

However, a diffusion model is iterative. The probability-flow ODE is solved over $N$ timesteps, and at each timestep $t_n$ the noise prediction $\boldsymbol\epsilon_\theta(\boldsymbol x_t, \boldsymbol z, t_n)$ determines the next state — giving us $N$ knobs to turn, rather than one. We propose to use a greedy strategy to solve this optimization problem by locally choosing the optimal solution at each timestep in the sampling process. We call this family of algorithms Greedy-DiM.

We propose two variants in this family: Greedy-DiM-S, a discrete search over noise blends at each timestep, and Greedy-DiM*, continuous gradient descent on the noise prediction itself. Both run at fewer NFEs than vanilla DiM, and Greedy-DiM* attains a 100% MMPMR against ArcFace, AdaFace, and ElasticFace on SYN-MAD 2022. To the best of our knowledge, Greedy-DiM is the first representation-based morphing attack to consistently outperform landmark-based morphing algorithms.

A greedy strategy on the iterative sampling process

Diffusion models sample by solving the probability-flow ODE from $t=T$ down to $t=0$ via a numerical solver $\Phi$. At every timestep $t_n$, the noise prediction model $\boldsymbol\epsilon_\theta(\boldsymbol x_t,\boldsymbol z,t)$ admits an immediate clean-image estimate via Tweedie's formula — i.e., the noise predictor can be restructured into an image predictor, well before the solver has finished. The key idea of Greedy-DiM is to exploit this fact.

Intuitively, the noise predictor $\boldsymbol\epsilon_\theta$ can be restructured into an image predictor $\hat{\boldsymbol x}_0$, and so an identity heuristic $\mathcal{H}(\hat{\boldsymbol x}_0,\boldsymbol x_0^{(a)},\boldsymbol x_0^{(b)})$ can be evaluated at any timestep. The recipe is then straightforward: at each $t_n$, evaluate $\mathcal H$ on the current clean estimate, take a gradient step on $\boldsymbol\epsilon$ to lower it, feed the optimized noise back into the solver, and proceed. We propose to repeat this at every step, steering the morph trajectory toward an image that fools the FR system rather than passively decoding it from a fixed average.

The identity heuristic

For our heuristic function $\mathcal H$ we adopt the identity loss popularized by MIPGAN, which drives the morph close to both sources in the embedding space of an FR system $F$, whilst penalizing any asymmetry in the similarity to $a$ and $b$ to prevent the optimizer from collapsing onto a single identity.

Here $F$ is an ArcFace network used solely to compute the heuristic during synthesis — it never sees the held-out evaluation systems.

Greedy search and greedy optimization

We propose two algorithms in this family. Greedy-DiM-S, the search variant, runs the noise predictor twice per step — once conditioned on $\boldsymbol z_a$, once on $\boldsymbol z_b$ — and searches over $B=21$ slerp blends of the two predictions for the $w$ which minimizes $\mathcal H$.

Greedy-DiM* drops the two-trajectory framing entirely. Rather than searching for an optimal blend of two noise predictions, we propose to directly optimize the noise prediction itself: a single $\boldsymbol\epsilon$ per step, updated by gradient descent in $\mathbb R^d$. The search space at each step is the full noise space rather than a 1D blend manifold. The greedy optimization strategy provides a significant uplift in performance over the greedy search strategy whilst retaining the same number of NFE; we posit that this is in part due to the enhanced size of the search space, allowing for a more optimal solution to be found. Section 5 makes this precise.

Greedy-DiM* is unreasonably effective

A morphing attack is multi-objective: the attack must fool every recognizer whilst keeping NFE low. The radar below plots all nine attacks against three FR systems (MMPMR at $\mathrm{FMR}=0.1\%$ against AdaFace, ArcFace, ElasticFace) and a compute-efficiency axis ($1-\mathrm{NFE}/2350$). The MMPMR axes are scaled from 65% to 100% in order to make the separation between attacks legible.

We observe that Greedy-DiM* (deep blue fill) is the only attack which saturates all three FR axes, i.e., achieves 100% MMPMR on every tested FR system, whilst remaining cheaper on compute than vanilla DiM-A. This is achieved at only 270 NFEs — fewer than DiM-A, and an order of magnitude below Morph-PIPE's 2350.

Morphing Attack Potential

Whilst MMPMR averages over identities, the Morphing Attack Potential metric (MAP) reports the fraction of morphs which simultaneously fool $c$ FR systems — the appropriate metric for deployments in which a single morph might be presented to multiple recognizers. We observe that Greedy-DiM* attains 100% in every column.

How does Greedy-DiM fare against morph attack detectors?

Fooling every recognizer is necessary but not sufficient. A deployed morph must also survive a morph attack detector (MAD) — a separately trained classifier deployed at the point of capture — whose job is to flag the morph as a forgery. Optimizing for MMPMR alone solves an easier problem than the one defenders actually face. Following the protocol of prior work, we train three SE-ResNeXt101 detectors, each on a different attack family — OpenCV (landmark), MIPGAN-II (GAN), DiM-A (diffusion) — and evaluate all eleven attacks against each.

Greed is, in fact, good

A greedy algorithm chooses a locally optimal solution at each timestep, but it is not automatically guaranteed that this locally optimal solution is globally optimal — famously, the greedy solution to the traveling salesman problem can yield the worst possible route. Motivated by our strong empirical results, we present two theorems which justify the use of a greedy strategy for the iterative sampling process of diffusion models. Under the standard assumptions of DDIM and the probability-flow ODE, the locally optimal $\boldsymbol\epsilon^*_{t_n}$ chosen by Greedy-DiM* at every step is globally optimal; moreover, the parameter space we optimize over has full Lebesgue measure, whilst Morph-PIPE's and Greedy-DiM-S's discrete subspaces have measure zero. Both statements admit short proofs in the paper; we sketch the geometry here.

Theorem 1 — the locally optimal step is globally optimal

Let $\boldsymbol\epsilon^*_{t_n}$ denote the noise prediction that minimizes the heuristic $\mathcal{H}$ at timestep $t_n$ — locally, looking only one step ahead. We claim that the DDIM update with $\boldsymbol\epsilon^*_{t_n}$ produces an $\boldsymbol x_{t_{n-1}}$ that, after the remaining solver steps, decodes to the morph $\hat{\boldsymbol x}_0^{*}$ which also globally minimizes $\mathcal H$ over all sampling trajectories starting from $\boldsymbol x_{t_n}$. The argument is short:

$\hat{\boldsymbol x}_0(\boldsymbol\epsilon_{t_n})$ is the model's clean-image estimate — what one would obtain by “jumping to the end” from the current noise level. Conditioned on $\boldsymbol x_{t_n}$, the entire trajectory from $t_n$ down to $0$ is determined by the choice of $\boldsymbol\epsilon_{t_n}$ — the rest of the schedule is fixed coefficients. Consequently, if $\boldsymbol\epsilon^*_{t_n}$ minimizes $\mathcal H(\hat{\boldsymbol x}_0)$, then the sampler's final output inherits that minimum. Intuitively, if the prediction is optimal at time $t$, then the same prediction at time $s < t$ remains optimal.

Theorem 2 — the prior search spaces have measure zero

The natural next question is where Greedy-DiM* searches. The optimization variable is the noise prediction $\boldsymbol\epsilon \in \mathbb R^d$ at the active timestep, with $d$ the latent dimension. Compare this to Morph-PIPE, which searches over a 21-element discrete set of fixed blends $\{w_i\} \subset [0,1]$, and to Greedy-DiM-S, which searches over a continuous 1D curve in $\mathbb R^d$ parameterized by a single blend coefficient. Whilst these earlier methods enumerate finely, they enumerate along a one-dimensional skeleton; Greedy-DiM* optimizes over the ambient space itself.

Let $\mathcal S^*$, $\mathcal S_S$, and $\mathcal S_P$ denote the three search spaces and let $\mu$ be the data distribution from which the diffusion model implicitly samples. The paper establishes:

That is, under any continuous data distribution, the optimal noise $\boldsymbol\epsilon^*$ lies in Greedy-DiM*'s search space almost surely, and lies in Morph-PIPE's or Greedy-DiM-S's search space with probability zero. Consequently, Morph-PIPE plateaus around 92–96% MMPMR regardless of how many candidates it enumerates; the optimum simply does not lie in its candidate set.

For the full step-by-step pictorial of the algorithm, see the animated overview at the top of the page.

Conclusions

Prior representation‑based diffusion morphs leave the reverse ODE as a black box; the latents are blended at $t=T$ and the solver is run to completion. Morph‑PIPE partially addresses this by searching a discrete grid of 21 blend weights, however, its parameter space has Lebesgue measure zero in the noise manifold and its compute budget runs an order of magnitude above vanilla DiM. We propose to exploit the iterative structure of the probability‑flow ODE directly, performing a greedy optimisation of an identity heuristic at every step. The resulting algorithm, Greedy‑DiM, is unreasonably effective; it consistently outperforms landmark‑based morphs against modern FR systems whilst running at a fraction of the compute of the previous representation‑based state of the art.

Contributions

• A novel family of face morphing algorithms. We propose Greedy‑DiM, which exploits the iterative structure of the probability‑flow ODE as a sequence of $N$ optimisation problems, running an inner search on $\boldsymbol\epsilon$ at each timestep and using the Tweedie $\hat{\boldsymbol x}_0$ identity to evaluate the heuristic before committing to the step. The inner loop touches only $\boldsymbol\epsilon$; no additional U‑Net evaluations are required.
• Thorough experimental and theoretical analysis. We evaluate Greedy‑DiM on SYN‑MAD 2022 against ArcFace, AdaFace, and ElasticFace at $\mathrm{FMR}=0.1\%$, reporting MMPMR and multi‑recogniser MAP; we further study detectability against three MAD scenarios. Theoretically, we prove that the locally optimal $\boldsymbol\epsilon^*_{t_n}$ at every step is the global optimum under standard DDIM assumptions, and that the Greedy‑DiM* search space has full Lebesgue measure whilst Morph‑PIPE's discrete subspace has measure zero.
• First representation morph to consistently beat landmarks. To the best of our knowledge, Greedy‑DiM* is the first representation‑based morphing algorithm that consistently outperforms landmark‑based morphing algorithms on modern margin‑based recognisers. Greedy‑DiM* attains 100% MMPMR across all three FR systems and saturates MAP in every column.
• Reduced computational cost. Greedy‑DiM* uses 270 NFEs in total; fewer than vanilla DiM (350) and less than one‑eighth of Morph‑PIPE's 2350. The added strength comes from the heuristic, not from extra network evaluations.