Publications by Elisa Ficarra

Multimodal Large Language Models (MLLMs) have achieved impressive performance across a variety of vision–language tasks. However, their internal working mechanisms remain largely underexplored. In his work, we introduce FG-TRACER, a framework designed to analyze the information flow between visual and textual modalities in MLLMs in free-form generation. Notably, our numerically stabilized computational method enables the first systematic analysis of multimodal information flow in underexplored domains such as image captioning and chain-of-thought (CoT) reasoning. We apply FG-TRACER to two state-of-the-art MLLMs—LLaMA 3.2-Vision and LLaVA 1.5—across three vision–language benchmarks—TextVQA, COCO 2014, and ChartQA—and we conduct a word-level analysis of multimodal integration. Our findings uncover distinct patterns of multimodal fusion across models and tasks, demonstrating that fusion dynamics are both model- and task-dependent. Overall, FG-TRACER offers a robust methodology for probing the internal mechanisms of MLLMs in free-form settings, providing new insights into their multimodal reasoning strategies. Our source code is publicly available at https://anonymous.4open.science/r/FG-TRACER-CB5A/.

Medulloblastoma, the most common malignant brain tumor of childhood, exhibits significant biological complexity that demands deeper exploration. Here, we present a large multiomics dataset integrating data from 384 primary medulloblastoma patient samples across five omic layers: CpG methylome, transcriptome, proteome, phosphoproteome, and metabolome, paired with associated clinical metadata. Data integration revealed intertumoral heterogeneity of lipid metabolism across proteomic subtypes. Notably, while the MYC-FASN-SCD axis drives lipid biosynthesis, pathway inhibition elicits a compensatory escape mechanism in vivo through exogenous fatty acid uptake. Unexpectedly, we demonstrated that MYC triggers lipid storage, creating a unique dependency on lipid droplet-mitochondria communications to sustain tumor maintenance in vivo. Together, this comprehensive analysis reveals a targetable vulnerability downstream of MYC that constitutes a promising therapeutic approach to treat currently untreatable medulloblastoma subtypes.

Ovarian high-grade serous carcinoma (ovarian HGSC) is a clinically challenging disease with a poor prognosis, particularly for patients receiving neoadjuvant chemotherapy (NACT) before debulking surgery. In this study, we evaluate the progression-free interval (PFI) after NACT based on hematoxylin and eosin-stained whole-slide images (WSIs) of omental tumor tissue. Digital pathology tools are emerging, aiming at assisting pathologists in diagnosis and analysis; however, distinguishing features associated with response to NACT remain elusive. Multiple instance learning (MIL) coupled with attention mechanisms has shown promise in predicting treatment response from WSIs. Additionally, segmentation tools can identify and delineate regions in WSIs. Whereas some efforts have been made to develop explainable models for clinical outcome, there remains a need for genuinely interpretable models for pathologists. This article introduces the PATHOS framework, a novel approach to explaining crucial features of treatment response based on the PFI time in NACT treated patients from WSIs. PATHOS is composed of three blocks: (1) MIL block to identify informative regions, (2) panoptic segmentation and downstream analysis block for feature computation, and (3) classification block to predict the PFI. The results demonstrate that PATHOS enhances the interpretability of response to NACT in ovarian HGSC patients by highlighting pathologically significant features relevant to PFI prediction, such as tumor cell morphology, stromal abundance, and the spatial distribution of stromal regions. Furthermore, PATHOS identifies approximately 10% of the total WSI area as an informative region for clinical outcome.

A gene fusion is a chromosomal aberration from juxtaposing separate genes. Since some gene fusions are involved in tumorigenesis, proper gene fusion investigation and analysis are crucial in the literature. After DNA/RNA sample extraction, detecting gene fusions requires first gene fusion detection tools, which usually provide many false positives. Given the high experimental costs in wet lab validation of a single fusion, gene fusion prioritization tools were made available over the years to significantly narrow down candidate gene fusions for validation (e.g., Oncofuse, Pegasus, DEEPrior, ChimerDriver). Although a few reviews about gene fusion detection tools are available, a benchmark on prioritization tools is not available yet in the literature. The aim of this paper is twofold: 1. to provide a curated dataset for a fair gene fusion prioritization tool evaluation. 2. to develop a proper comparison based on time, resources, and tool confidence on selected gene fusions. Based on this benchmark, it can be stated that ChimerDriver is the most reliable tool for prioritizing oncogenic fusions.

CNNs and Transformer-based architectures are recently dominating the field of 3D medical segmentation. While CNNs face limitations in the local receptive field, Transformers require significant memory and data, making them less suitable for analyzing large 3D medical volumes. Consequently, fully convolutional network models like U-Net are still leading the 3D segmentation scenario. Although efforts have been made to reduce the Transformers computational complexity, such optimized models still struggle with content-based reasoning. This paper examines Mamba, a Recurrent Neural Network (RNN) based on State Space Models (SSMs), which achieves linear complexity and has outperformed Transformers in long-sequence tasks. Specifically, we assess Mamba’s performance in 3D medical segmentation using three widely recognized and commonly employed datasets and propose architectural enhancements to improve its segmentation effectiveness by mitigating the primary shortcomings of existing Mamba-based solutions.

Whole Slide Images (WSIs) are crucial in histological diagnostics, providing high-resolution insights into cellular structures. In addition to challenges like the gigapixel scale of WSIs and the lack of pixel-level annotations, privacy restrictions further complicate their analysis. For instance, in a hospital network, different facilities need to collaborate on WSI analysis without the possibility of sharing sensitive patient data. A more practical and secure approach involves sharing models capable of continual adaptation to new data. However, without proper measures, catastrophic forgetting can occur. Traditional continual learning techniques rely on storing previous data, which violates privacy restrictions. To address this issue, this paper introduces Context Optimization Multiple Instance Learning (CooMIL), a rehearsal-free continual learning framework explicitly designed for WSI analysis. It employs a WSI-specific prompt learning procedure to adapt classification models across tasks, efficiently preventing catastrophic forgetting. Evaluated on four public WSI datasets from TCGA projects, our model significantly outperforms state-of-the-art methods within the WSI-based continual learning framework. The source code is available at https://github.com/FrancescaMiccolis/CooMIL.

Shigellosis is an acute small intestine infection caused by different species of Shigella. Worldwide, the emergence of antibiotic-resistant strains aggravates the impact of Shigella infections. In this context, human monoclonal antibodies (mAbs) offer an alternative to traditional antimicrobials. However, identifying a potent candidate mAb requires intense and meticulous efforts. Here, we show the potential of Deep Learning to screen mAbs rapidly. We measured the phagocytosis-promoting activity of mAbs by analyzing images collected with a high-throughput and high-content confocal fluorescence microscope. We acquired images of S. sonnei and S. flexneri infecting THP-1-derived macrophages and evaluated the effect of different mAbs and of a wide selection of Deep Learning tools. We found that our model can generalize on strains and mAbs not encountered in training. Importantly, our approach enables the screening and characterization of multiple anti-Shigella mAbs at the same time, facilitating the identification of potent antibacterial candidates. Our code is available on the GitHub repository vOPA_Shigella.

Brain tumor segmentation is a crucial task in medical imaging that involves the integrated modeling of four distinct imaging modalities to identify tumor regions accurately. Unfortunately, in real-life scenarios, the full availability of such four modalities is often violated due to scanning cost, time, and patient condition. Consequently, several deep learning models have been developed to address the challenge of brain tumor segmentation under conditions of missing imaging modalities. However, the majority of these models have been evaluated using the 2018 version of the BraTS dataset, which comprises only $285$ volumes. In this study, we reproduce and extensively analyze the most relevant models using BraTS2023, which includes 1,250 volumes, thereby providing a more comprehensive and reliable comparison of their performance. Furthermore, we propose and evaluate the adoption of Mamba as an alternative fusion mechanism for brain tumor segmentation in the presence of missing modalities. Experimental results demonstrate that transformer-based architectures achieve leading performance on BraTS2023, outperforming purely convolutional models that were instead superior in BraTS2018. Meanwhile, the proposed Mamba-based architecture exhibits promising performance in comparison to state-of-the-art models, competing and even outperforming transformers. The source code of the proposed approach is publicly released alongside the benchmark developed for the evaluation: https://github.com/AImageLab-zip/IM-Fuse.