Publications by William Raccagni

Answer grounding in visual question answering over documents remains an open challenge: most existing benchmarks lack grounding annotations or provide annotations of limited quality, and the development of answer grounding datasets is still bounded to expensive manual effort. We introduce DocAttriBench (DAB), an improved large-scale benchmark for fine-grained, element-level source attribution in Document VQA, grounding each answer to specific layout elements such as text blocks, tables, and images. To build DAB, we propose a Mask-based Perplexity-Derived Attribution method (MAPPET) that leverages document layout and language modeling to assign each answer to its most informative document element. MAPPET measures the increase in perplexity when masking candidate elements, attributing the answer to the element that contributes most to model confidence. Applying MAPPET to multiple existing Document VQA datasets yields DAB, which contains 237k documents and 296k question-answer pairs, where each answer is grounded to the corresponding layout elements. We benchmark a range of Multimodal Large Language Models (MLLMs) with grounding capabilities on DAB, evaluating answer accuracy, attribution accuracy, and overall answer quality. Our results show that while larger models tend to achieve higher answer accuracy, even the most capable models often fail to correctly localize the supporting elements. DAB thus provides a scalable benchmark for developing grounded, verifiable, and trustworthy Document VQA models.

Vision-and-Language Navigation (VLN) requires embodied agents to follow natural language instructions while grounding their decisions in complex visual environments. Despite recent progress, most VLN agents remain difficult to interpret, providing limited insight into why a navigation action is selected. In this work, we introduce ExplaiNavility, an interpretable VLN framework that augments navigation with explicit reasoning supervision. Our model combines a visual scene encoder with a multimodal LLM, using structured prompts to jointly condition on the instruction, the navigation history, and the candidate viewpoints available at each step. Beyond predicting the next navigable location, the model is trained to produce human-readable explanations of its decisions and descriptive low-level action labels, such as turning left, turning right, moving forward, or stopping. To supervise these outputs, we augment R2R trajectories with speaker-style action explanations generated from consecutive observations, together with action labels obtained from the simulator. By fine-tuning the embodied multimodal agent with these auxiliary reasoning objectives, ExplaiNavility encourages more transparent decision-making while preserving strong navigation ability. Experiments on the R2R benchmark show that our approach improves over prior explainable VLN methods on unseen environments, while producing interpretable rationales that make the agent's behavior easier to inspect. These results suggest that explicit reasoning supervision can benefit both navigation performance and interpretability in embodied instruction-following agents.