Publications by Giuseppe Cartella

L’attenzione visiva umana è un processo cognitivo che consente agli individui di interpretare in modo efficiente ambienti visivi complessi. Attraverso la prioritarizzazione di regioni salienti e rilevanti per il problema, gli esseri umani estraggono informazioni critiche sopprimendo al contempo gli stimoli irrilevanti. Questo meccanismo di selezione supporta la percezione, il processo decisionale e l’interazione con un mondo dinamico. Studi nelle neuroscienze hanno stabilito principi chiave dell’attenzione spaziale, della selezione basata sugli attributi visivi e dell’interazione tra controllo top-down e bottom-up. I progressi nell’Intelligenza Artificiale e nella Computer Vision hanno ispirato lo sviluppo di sistemi autonomi in grado di emulare la prioritarizzazione umana delle informazioni visive. A differenza dei modelli di salienza tradizionali, che predicono regioni di interesse statiche, i modelli di scanpath catturano sia gli aspetti spaziali che quelli temporali dell’esplorazione visiva, inclusi la sequenza, la tempistica e la traiettoria dello sguardo. Questa tesi introduce framework probabilistici e generativi per la generazione degli scanpath, contribuendo a una comprensione più approfondita delle dinamiche temporali e della variabilità tra individui. Proponiamo TPP-Gaze, un framework ispirato alla teoria dei Neural Temporal Point Processes, che concettualizza gli scanpath come sequenze di eventi stocastici che accadono in modo irregolare nel tempo. Tale formulazione affronta un limite chiave degli approcci esistenti: mentre molti modelli rappresentano accuratamente dove guardare, spesso mancano di una rappresentazione rigorosa di quando avvengono gli spostamenti dello sguardo. Modellando esplicitamente l’evoluzione temporale, TPP-Gaze fornisce una descrizione più completa delle dinamiche dell’attenzione visiva. L’attenzione umana è intrinsecamente variabile tra individui, influenzata da bias cognitivi, preferenze e contesto. Per catturare tale variabilità, introduciamo ScanDiff, un modello stocastico basato su diffusione, capace di generare traiettorie di sguardo diverse e realistiche. A differenza dei metodi che producono un singolo scanpath medio, ScanDiff riflette la naturale eterogeneità del comportamento visivo. Oltre ai contributi metodologici, questa ricerca fornisce una panoramica completa dei modelli e dei dataset attuali, evidenziando trend, sfide e opportunità per integrare i meccanismi attentivi umani nei sistemi artificiali. La rilevanza pratica dei modelli proposti è dimostrata attraverso esperimenti di eye-tracking e raccolta dati in diversi domini. Questi includono studi sul comportamento visivo in immagini autentiche rispetto a immagini manipolate, l’integrazione di segnali percettivi per il rilevamento dei deepfake e l’analisi dell’attenzione in condizioni neuroevolutive. Inoltre, la tesi introduce un dataset egocentrico multimodale che cattura sguardo, movimento e contesto ambientale in ambiti culturalmente diversificati. Questi dati rivelano pattern attentivi distintivi, contribuendo alla progettazione di modelli centrati sull’uomo per applicazioni quali la valutazione dell’autenticità dei media, l’analisi comportamentale e la modellazione multimodale dell’attenzione. In conclusione, questa tesi avanza lo studio dell’attenzione visiva umana attraverso (i) lo sviluppo di modelli computazionali che catturano la struttura spazio-temporale degli scanpath, (ii) la definizione di evidenze empiriche sulle dinamiche dello sguardo in diverse condizioni e (iii) l’esplorazione di applicazioni reali della modellazione dell’attenzione. Collegando scienze cognitive e intelligenza artificiale, questo lavoro approfondisce la comprensione teorica e migliora la modellazione computazionale del comportamento visivo umano.

Predicting human gaze scanpaths is crucial for understanding visual attention, with applications in human-computer interaction, autonomous systems, and cognitive robotics. While deep learning models have advanced scanpath prediction, most existing approaches generate averaged behaviors, failing to capture the variability of human visual exploration. In this work, we present ScanDiff, a novel architecture that combines diffusion models with Vision Transformers to generate diverse and realistic scanpaths. Our method explicitly models scanpath variability by leveraging the stochastic nature of diffusion models, producing a wide range of plausible gaze trajectories. Additionally, we introduce textual conditioning to enable task-driven scanpath generation, allowing the model to adapt to different visual search objectives. Experiments on benchmark datasets show that ScanDiff surpasses state-of-the-art methods in both free-viewing and task-driven scenarios, producing more diverse and accurate scanpaths. These results highlight its ability to better capture the complexity of human visual behavior, pushing forward gaze prediction research.

We introduce Sanctuaria-Gaze, a multimodal dataset featuring egocentric recordings from 40 visits to four architecturally and culturally significant sanctuaries in Northern Italy. Collected using wearable devices with integrated eye trackers, the dataset offers RGB videos synchronized with streams of gaze coordinates, head motion, and environmental point cloud, resulting in over four hours of recordings. Along with the dataset, we provide a framework for automatic detection and analysis of Areas of Interest (AOIs). This framework fills a critical gap by offering an open-source, flexible tool for gaze-based research that adapts to dynamic settings without requiring manual intervention. Our study analyzes human visual attention to sacred, architectural, and cultural objects, providing insights into how visitors engage with these elements and how their background influences their interactions. By releasing both the dataset and the analysis framework, Sanctuaria-Gaze aims to advance interdisciplinary research on gaze behavior, human-computer interaction, and visual attention in real-world environments. Code and dataset are available at https://github.com/aimagelab/Sanctuaria-Gaze.

Attention guides our gaze to fixate the proper location of the scene and holds it in that location for the deserved amount of time given current processing demands, before shifting to the next one. As such, gaze deployment crucially is a temporal process. Existing computational models have made significant strides in predicting spatial aspects of observer's visual scanpaths (where to look), while often putting on the background the temporal facet of attention dynamics (when). In this paper we present TPP-Gaze, a novel and principled approach to model scanpath dynamics based on Neural Temporal Point Process (TPP), that jointly learns the temporal dynamics of fixations position and duration, integrating deep learning methodologies with point process theory. We conduct extensive experiments across five publicly available datasets. Our results show the overall superior performance of the proposed model compared to state-of-the-art approaches.

Can the very fabric of how we visually explore the world hold the key to distinguishing individuals with Autism Spectrum Disorder (ASD)? While eye tracking has long promised quantifiable insights into neurodevelopmental conditions, the causal underpinnings of gaze behaviour remain largely uncharted territory. Moving beyond traditional descriptive metrics of gaze, this study employs cutting-edge causal discovery methods to reconstruct the directed networks that govern the flow of attention across natural scenes. Given the well-documented atypical patterns of visual attention in ASD, particularly regarding socially relevant cues, our central hypothesis is that individuals with ASD exhibit distinct causal signatures in their gaze patterns, significantly different from those of typically developing controls. To our knowledge, this is the first study to explore the diagnostic potential of causal modeling of eye movements in uncovering the cognitive phenotypes of ASD and offers a novel window into the neurocognitive alterations characteristic of the disorder.

Human attention modelling has proven, in recent years, to be particularly useful not only for understanding the cognitive processes underlying visual exploration, but also for providing support to artificial intelligence models that aim to solve problems in various domains, including image and video processing, vision-and-language applications, and language modelling. This survey offers a reasoned overview of recent efforts to integrate human attention mechanisms into contemporary deep learning models and discusses future research directions and challenges.

Creating high-quality and realistic images is now possible thanks to the impressive advancements in image generation. A description in natural language of your desired output is all you need to obtain breathtaking results. However, as the use of generative models grows, so do concerns about the propagation of malicious content and misinformation. Consequently, the research community is actively working on the development of novel fake detection techniques, primarily focusing on low-level features and possible fingerprints left by generative models during the image generation process. In a different vein, in our work, we leverage human semantic knowledge to investigate the possibility of being included in frameworks of fake image detection. To achieve this, we collect a novel dataset of partially manipulated images using diffusion models and conduct an eye-tracking experiment to record the eye movements of different observers while viewing real and fake stimuli. A preliminary statistical analysis is conducted to explore the distinctive patterns in how humans perceive genuine and altered images. Statistical findings reveal that, when perceiving counterfeit samples, humans tend to focus on more confined regions of the image, in contrast to the more dispersed observational pattern observed when viewing genuine images. Our dataset is publicly available at: https://github.com/aimagelab/unveiling-the-truth.

The rapidly evolving fields of e-commerce and metaverse continue to seek innovative approaches to enhance the consumer experience. At the same time, recent advancements in the development of diffusion models have enabled generative networks to create remarkably realistic images. In this context, image-based virtual try-on, which consists in generating a novel image of a target model wearing a given in-shop garment, has yet to capitalize on the potential of these powerful generative solutions. This work introduces LaDI-VTON, the first Latent Diffusion textual Inversion-enhanced model for the Virtual Try-ON task. The proposed architecture relies on a latent diffusion model extended with a novel additional autoencoder module that exploits learnable skip connections to enhance the generation process preserving the model's characteristics. To effectively maintain the texture and details of the in-shop garment, we propose a textual inversion component that can map the visual features of the garment to the CLIP token embedding space and thus generate a set of pseudo-word token embeddings capable of conditioning the generation process. Experimental results on Dress Code and VITON-HD datasets demonstrate that our approach outperforms the competitors by a consistent margin, achieving a significant milestone for the task.

Fashion illustration is used by designers to communicate their vision and to bring the design idea from conceptualization to realization, showing how clothes interact with the human body. In this context, computer vision can thus be used to improve the fashion design process. Differently from previous works that mainly focused on the virtual try-on of garments, we propose the task of multimodal-conditioned fashion image editing, guiding the generation of human-centric fashion images by following multimodal prompts, such as text, human body poses, and garment sketches. We tackle this problem by proposing a new architecture based on latent diffusion models, an approach that has not been used before in the fashion domain. Given the lack of existing datasets suitable for the task, we also extend two existing fashion datasets, namely Dress Code and VITON-HD, with multimodal annotations collected in a semi-automatic manner. Experimental results on these new datasets demonstrate the effectiveness of our proposal, both in terms of realism and coherence with the given multimodal inputs. Source code and collected multimodal annotations are publicly available at: https://github.com/aimagelab/multimodal-garment-designer.