The Art of Deception: Color Visual Illusions and Diffusion Models

Alex Gomez-Villa1,2, Kai Wang1,2, Alejandro C. Parraga1,2, Bartlomiej Twardowski1,2,4, Jesus Malo3, Javier Vazquez-Corral1,2, Joost van de Weijer1,2
1Computer Vision Center, Barcelona, Spain
2Universitat Autonoma de Barcelona, Barcelona, Spain
3Universitat de Valencia, Valencia, Spain
4IDEAS NCBR, Warsaw, Poland
Paper Supplementary Code (coming soon) arXiv

TL;DR: We demonstrate that diffusion models exhibit human-like perceptual shifts in brightness and color within their latent space. Building on this discovery, we demonstrate how to generate novel visual illusions in realistic images using text-to-image diffusion models.

Description of your image
Observation: Denoising Diffusion Implicit Models (DDIM) have human-like visual illusions. Application: text-to-image generated visual illusions. Examples (a) and (b) show how the responses to physically equal patches shift differently in the path to the latent space. In contrast, (c) shows how physically different stimuli get more similar along the path, achieving color constancy. The observation of this human-like behavior allows us to propose the use of text-to-image models to generate images in which physically identical patches are perceived differently (examples d, e, f).

Abstract

Visual illusions in humans arise when interpreting out-of-distribution stimuli: if the observer is adapted to certain statistics, perception of outliers deviates from reality. Recent studies have shown that artificial neural networks (ANNs) can also be deceived by visual illusions. This revelation raises profound questions about the nature of visual information. Why are two independent systems, both human brains and ANNs, susceptible to the same illusions? Should any ANN be capable of perceiving visual illusions? Are these perceptions a feature or a flaw? In this work, we study how visual illusions are encoded in diffusion models. Remarkably, we show that they present human-like brightness/color shifts in their latent space. We use this fact to demonstrate that diffusion models can predict visual illusions. Furthermore, we also show how to generate new unseen visual illusions in realistic images using text-to-image diffusion models. We validate this ability through psychophysical experiments that show how our model-generated illusions also fool humans.

BibTeX

@article{gomez2024art,
    title={The Art of Deception: Color Visual Illusions and Diffusion Models},
    author={Gomez-Villa, Alex and Wang, Kai and Parraga, Alejandro C. and Twardowski, Bartlomiej and Malo, Jesus and Vazquez-Corral, Javier and van de Weijer, Joost},
    journal={CVPR},
    year={2025},
    url={https://arxiv.org/abs/2412.10122}
}