When Does High-CFG Diffusion Inversion Fail? A Controlled Study of Prompt--Latent Interactions

Abstract

Text-guided diffusion inversion is central to image editing, where an image is mapped to an initial latent and then edited by replaying the denoising process under a modified prompt. In practice, however, inversion is often performed with a lower classifier-free guidance(CFG) scale than the one used for generation or editing. This mismatch is empirically useful but leaves a basic question unresolved: when a target image is generated by a high-CFG trajectory, when can that trajectory actually be inverted? We study this question in a controlled generation--inversion--reconstruction setting, where the true initial latent and denoising trajectory are known. Using prompts taken from an existing diffusion-editing benchmark, we generate images under high CFG and reconstruct them with fixed-point inversion using the same prompt and guidance setting. The results reveal three types of prompt-level reconstruction behavior: easy prompts that reconstruct for most initial latents, hard prompts that fail for most initial latents, and intermediate prompts whose success depends on the prompt--latent pairing. To analyze the generation side, we define prompt pressure, a step-wise measure of how strongly CFG moves the denoising update away from the unconditional trajectory. Total pressure correlates with reconstruction quality and separates easy from hard prompts, but it does not explain the success or failure of intermediate prompt--latent pairs. Text-side analyses further show that the main visual subject and wording can change inversion difficulty. Finally, we evaluate a compact trajectory-consistency intervention that relaxes guidance only at locally unstable inverse steps. This diagnostic check improves reconstruction and Prompt-to-Prompt editing in our controlled setting, supporting the view that high-CFG inversion failure requires local, trajectory-aware analysis.