Publications

Facial age estimation has shown notable progress under controlled conditions. However, in unconstrained real-world environments, accurate age estimation remains challenging. This difficulty becomes more severe when facial images contain partial occlusions, as these obstructions hide important age-related information. Moreover, there is no publicly available occluded age estimation dataset to improve performance in real-world scenarios. To overcome this issue, we propose and publicly release three new datasets, FG-NET-O8, APPA-REAL-O8, and MORPH-O8, derived from existing benchmarks. These datasets contain eight types of realistic occlusions, providing a comprehensive testbed for age estimation under occlusions. These occlusions are generated using multiple diffusion-based methods, including Stable Diffusion Realistic Vision, Blended Latent Diffusion, and Fooocus, while preserving the facial identity of each subject. We also design and conduct a human survey to evaluate the quality of the generated occlusions. Furthermore, we test five state-of-the-art age estimation approaches to analyze the impact of real-world occlusions on age estimation performance. Experimental results demonstrate that all approaches exhibit severe performance degradation for nearly all occlusion types across all three datasets.

Answer grounding in visual question answering over documents remains an open challenge: most existing benchmarks lack grounding annotations or provide annotations of limited quality, and the development of answer grounding datasets is still bounded to expensive manual effort. We introduce DocAttriBench (DAB), an improved large-scale benchmark for fine-grained, element-level source attribution in Document VQA, grounding each answer to specific layout elements such as text blocks, tables, and images. To build DAB, we propose a Mask-based Perplexity-Derived Attribution method (MAPPET) that leverages document layout and language modeling to assign each answer to its most informative document element. MAPPET measures the increase in perplexity when masking candidate elements, attributing the answer to the element that contributes most to model confidence. Applying MAPPET to multiple existing Document VQA datasets yields DAB, which contains 237k documents and 296k question-answer pairs, where each answer is grounded to the corresponding layout elements. We benchmark a range of Multimodal Large Language Models (MLLMs) with grounding capabilities on DAB, evaluating answer accuracy, attribution accuracy, and overall answer quality. Our results show that while larger models tend to achieve higher answer accuracy, even the most capable models often fail to correctly localize the supporting elements. DAB thus provides a scalable benchmark for developing grounded, verifiable, and trustworthy Document VQA models.

Representation alignment has emerged as an effective approach to improve Multimodal Large Language Models (MLLMs) by regularizing their internal representations toward those of an external vision encoder. However, existing methods typically align a fixed layer of the language backbone, overlooking the fine-grained structure of Transformer models. In this work, we propose Head-Wise Representation Alignment (HeRA), a method that enforces cross-modal alignment at the level of individual attention heads. Our approach is grounded in the Platonic Representation Hypothesis, focusing on preserving the topological structure of representations (i.e., their local neighborhood relationships) across modalities. Following the Mutual K-Nearest Neighbor (MKNN) alignment metric, we introduce a contrastive objective that acts as a differentiable proxy for matching local structures. HeRA applies this objective during multimodal training to specific attention heads in the LLM, selected by their alignment score according to the MKNN metric. Counterintuitively, we find that aligning the least aligned heads yields the largest gains. Extensive evaluations across multiple MLLMs and 18 benchmarks demonstrate that HeRA consistently improves performance on challenging vision-centric tasks and serves as an effective regularizer against visual hallucinations by naturally curbing the over-reliance on linguistic priors.

Large Language Models (LLMs) are traditionally evaluated on multiple-choice question answering (MCQA) tasks using First-Token Probability (FTP), which selects the answer option whose initial token has the highest likelihood. While efficient, FTP can be fragile: models may assign high probability to unrelated tokens (misalignment) or use a valid token merely as part of a generic preamble rather than as a clear answer choice (misinterpretation), undermining the reliability of symbolic evaluation. We propose a simple solution: output prefilling, a structured natural-language prefix (e.g., 'The correct option is:') prepended to the model output. Originally explored in AI safety as an attack strategy, we repurpose prefilling to steer the model to respond with a clean, valid option, without modifying its parameters. Through extensive evaluation, we find that the FTP with prefilling strategy substantially improves accuracy, calibration, and output consistency across a broad set of LLMs and MCQA benchmarks. It outperforms standard FTP and often matches the performance of open-ended generation approaches that require full decoding and external classifiers, while being significantly more efficient. Our analysis suggests that prefilling is a simple, robust, and zero-cost method to enhance the reliability of FTP-based evaluation in multiple-choice settings.

Perceptual decisions unfold in time: noisy evidence is integrated until a commitment threshold is reached, jointly producing a choice and a reaction time. Classical race models capture this mechanism through simple parametric drift and diffusion, but cannot represent how rich, high-dimensional stimuli shape accumulation on individual trials. We introduce the Neural Race Model (NRM), a stimulus-conditional neural stochastic differential equation in which one accumulator per alternative races towards a learnable, stimulus-dependent threshold. Drift, diffusion, and threshold are parameterised by neural networks conditioned on a stimulus embedding, allowing both accumulation dynamics and decision urgency to vary across trials. The model is trained end-to-end on the joint reaction-time-and-choice likelihood via a smooth surrogate for the discontinuous first-passage event, combined with Monte Carlo trajectory averaging and gradient propagation through the stochastic adjoint. The surrogate recovers the classical hard threshold in the appropriate limit and is removed at test time. Each component admits a natural correspondence with elements of the primate decision-making circuit, preserving the mechanistic interpretability of classical models. We evaluate the NRM on multiple perceptual decision-making benchmarks against classical sequential-sampling and image-computable reaction-time models. Across tasks and metrics, the NRM tracks the human noise ceiling more closely than all baselines, demonstrating that stimulus-conditional stochastic accumulation and end-to-end differentiability can be achieved within a single, principled framework

Vision-and-Language Navigation (VLN) requires embodied agents to follow natural language instructions while grounding their decisions in complex visual environments. Despite recent progress, most VLN agents remain difficult to interpret, providing limited insight into why a navigation action is selected. In this work, we introduce ExplaiNavility, an interpretable VLN framework that augments navigation with explicit reasoning supervision. Our model combines a visual scene encoder with a multimodal LLM, using structured prompts to jointly condition on the instruction, the navigation history, and the candidate viewpoints available at each step. Beyond predicting the next navigable location, the model is trained to produce human-readable explanations of its decisions and descriptive low-level action labels, such as turning left, turning right, moving forward, or stopping. To supervise these outputs, we augment R2R trajectories with speaker-style action explanations generated from consecutive observations, together with action labels obtained from the simulator. By fine-tuning the embodied multimodal agent with these auxiliary reasoning objectives, ExplaiNavility encourages more transparent decision-making while preserving strong navigation ability. Experiments on the R2R benchmark show that our approach improves over prior explainable VLN methods on unseen environments, while producing interpretable rationales that make the agent's behavior easier to inspect. These results suggest that explicit reasoning supervision can benefit both navigation performance and interpretability in embodied instruction-following agents.

Generating faithful and readable styled text images (especially for Styled Handwritten Text generation-HTG) is an open problem with several possible applications across graphic design, document understanding, and image editing. A lot of research effort in this task is dedicated to developing strategies that reproduce the stylistic characteristics of a given writer, with promising results in terms of style fidelity and generalization achieved by the recently proposed Autoregressive Transformer paradigm for HTG. However, this method requires additional inputs, lacks a proper stop mechanism, and might end up in repetition loops, generating visual artifacts. In this work, we rethink the autoregressive formulation by framing HTG as a multimodal prompt-conditioned generation task, and tackle the content controllability issues by introducing special textual input tokens for better alignment with the visual ones. Moreover, we devise a Classifier-Free-Guidance-based strategy for our autoregressive model. Through extensive experimental validation, we demonstrate that our approach, dubbed Eruku, compared to previous solutions requires fewer inputs, generalizes better to unseen styles, and follows more faithfully the textual prompt, improving content adherence.

Cone-beam computed tomography (CBCT) is widely used for dento-maxillofacial diagnostics and treatment planning, and comprehensive multi-structure segmentation remains time-consuming, limiting large-scale, reproducible research. In this article, we present ToothFairy2, a MICCAI 2024 challenge on multi-structure segmentation in maxillofacial CBCT. The accompanying dataset comprises 530 CBCT volumes (480 public training, 50 hidden test) with expert 3D annotations of 42 classes, including maxilla, mandible, crowns, bridges, implants, inferior alveolar canals, maxillary sinuses, pharynx, and teeth using the International Tooth Numbering System (FDI). 26 international teams participated in ToothFairy2, and their methods were run and evaluated for voxel-wise multi-class segmentation using a standardized protocol. This report extends the evaluation of teeth to also investigate the current capabilities of tooth detection and FDI numbering. Furthermore, ranking stability was analyzed to assess the robustness of the final challenge outcome. Overall, challenge participants achieved consistently high performance for large, high-contrast structures such as jawbones, pharynx, and most teeth, while maxillary sinuses, dental restorations, and fine structures remain challenging due to class imbalance and metal artifacts. Analysis of tooth-related metrics further revealed that assigning correct FDI numbers was more challenging than delineating individual teeth. By releasing CBCT data, 3D annotations, baseline models, and evaluation code, ToothFairy2 establishes a long-term benchmark to drive the development of automated methods for robust, clinically meaningful multi-structure segmentation in maxillofacial CBCT.