For decades, histopathological interpretation has depended on the pathologist’s ability to recognize patterns in tissue architecture, cellular morphology and staining characteristics. With the development of whole-slide imaging (WSI), these observations can now be digitized, analyzed and incorporated into increasingly sophisticated computational workflows.
Early artificial intelligence applications in pathology primarily focused on recognizing specific visual patterns. Models were developed to identify particular tissue structures, classify regions of interest, detect abnormalities or estimate selected biomarkers.
However, pathology diagnosis rarely depends on an image in isolation.
A pathologist may interpret a tissue section alongside clinical history, specimen details, previous pathology reports, immunohistochemistry findings, molecular results and other diagnostic information. Understanding how these different sources relate to one another is an important part of the diagnostic process.
This is where multimodal AI in digital pathology is attracting significant research attention.
Instead of analyzing histology images alone, multimodal AI systems aim to integrate visual information with text, clinical data and, in some research settings, molecular information. The objective is to develop models that can capture more of the context involved in pathology interpretation.
Recent research into pathology foundation models suggests that this approach could expand the capabilities of computational pathology beyond conventional image classification.
1. What Is Multimodal AI in Pathology?
Multimodal AI refers to artificial intelligence systems designed to process and relate information from multiple data types, or modalities.
In computational pathology, these modalities may include:
- Histopathology images: H&E-stained whole-slide images, immunohistochemistry slides and selected regions of interest.
- Pathology reports: Microscopic descriptions, diagnostic interpretations and structured reporting information.
- Clinical information: Relevant patient history, specimen details and other available clinical variables.
- Molecular information: Gene expression profiles and other molecular data, where available.
- Structured diagnostic data: Biomarker results, grading information and other standardized observations.
A conventional image-analysis model might identify a particular morphological pattern within a tissue region.
A multimodal model, in contrast, attempts to learn relationships between the image and associated information. For example, it may learn how certain visual patterns correspond to descriptions in pathology reports or how image representations relate to molecular profiles.
The distinction is important: the model is not simply processing more information. It is learning relationships across different forms of information.
2. Why Whole-Slide Images Alone May Not Be Enough
Whole-slide images contain enormous amounts of visual information. A single digitized slide can contain thousands of tissue regions, cellular structures and architectural patterns.
However, image-level information does not always capture the complete diagnostic context.
Consider a tissue specimen containing areas of inflammation, fibrosis, cellular atypia and tissue degeneration. These features may have different diagnostic significance depending on the specimen type, clinical history and other investigation findings.
An image-only model may recognize these morphological features, but interpreting their relevance within a broader clinical context is a more complex task.
This creates several challenges:
Limited diagnostic context
Visual patterns may have different implications depending on the specimen and clinical setting. Models trained only on image labels may not learn all the contextual relationships required for more complex tasks.
Incomplete representation of the case
A pathology case may contain multiple slides, different stains and additional diagnostic information. Analyzing one image or a limited number of regions may not adequately represent the entire case.
Dependence on annotated datasets
Developing task-specific models often requires carefully annotated data. Expert annotation is valuable but time-consuming, and some diagnostic categories have limited available training examples.
Difficulty connecting morphology with other findings
Histological appearances, pathology descriptions and molecular findings represent different types of information. Learning meaningful relationships between them requires appropriate paired datasets and model architectures.
Multimodal learning is being investigated as one approach to addressing these limitations.
3. Recent Research: Pathology Foundation Models Are Moving Beyond Image-Only Analysis
The development of pathology foundation models has accelerated research into more generalizable computational pathology systems.
Unlike models designed for one narrowly defined task, foundation models are pretrained on large datasets to learn reusable representations. These representations can subsequently be adapted or evaluated across different downstream applications.
Several recent studies demonstrate how this research is evolving.
A. TITAN: Connecting Whole-Slide Images With Pathology Reports
A significant study published in Nature Medicine on 5 November 2025 introduced TITAN, a multimodal whole-slide foundation model for pathology.
The researchers pretrained the model using approximately 335,645 whole-slide images. Its multimodal training also incorporated pathology reports and more than 423,000 synthetic descriptions of tissue regions.
The research explored how visual information could be aligned with textual descriptions at both region-of-interest and whole-slide levels.
The model was evaluated across several tasks, including slide classification, rare disease retrieval, cross-modal image-to-report retrieval and pathology report generation.
An important contribution of this work is its emphasis on whole-slide representations rather than relying exclusively on isolated image patches.
Why it matters: Connecting visual tissue representations with pathology language could support more flexible retrieval, classification and research workflows.
Research paper: A Multimodal Whole-Slide Foundation Model for Pathology — Nature Medicine (2025)
B. PRISM2: Clinical Dialogue Meets Whole-Slide Understanding
A more recent study published in Nature Medicine on 31 July 2026 introduced PRISM2, an end-to-end multimodal pathology foundation model.
The model was trained using approximately 2.3 million whole-slide images and 14 million question–answer pairs derived from 700,000 pathology reports.
The researchers used clinical dialogue supervision to align histomorphological information with diagnostic reasoning expressed in pathology reports.
The study evaluated the model across diagnostic, biomarker and outcome-related tasks. It also reported results from prompt-based evaluations involving selected clinical-grade products.
This research is particularly relevant because it explores how pathology AI systems can learn from more than image-level labels. It investigates the relationship between tissue morphology, pathology documentation and clinical questions.
Why it matters: Clinical dialogue-based supervision could provide a way to develop more context-aware pathology AI systems, although further independent validation is necessary before assuming performance will translate into routine clinical practice.
Research paper: End-to-End Multimodal Pathology Foundation Model With Clinical Dialogue — Nature Medicine (2026)
C. PathChat: Exploring Vision-Language Interaction in Pathology
Another important research direction is the development of vision-language systems designed specifically for pathology.
A 2024 study published in Nature introduced PathChat, a multimodal generative AI copilot for human pathology.
The system combined a pathology-focused visual encoder with a large language model and was trained using a large collection of visual-language instructions.
The research explored how an AI system could interact with histopathology images through natural-language questions and responses.
This represents a different direction from conventional image classifiers. Instead of producing only a predefined classification, vision-language systems aim to support richer interactions with image-based information.
Why it matters: Natural-language interaction may make pathology AI systems more flexible for image exploration, educational applications and research assistance. However, generated responses must be assessed carefully for factual accuracy and diagnostic reliability.
Research paper: A Multimodal Generative AI Copilot for Human Pathology — Nature (2024)
D. Prov-GigaPath: Learning From Whole-Slide Images at Scale
Multimodal AI is built on advances in visual foundation models.
A major study published in Nature in May 2024 introduced Prov-GigaPath, a whole-slide foundation model developed using approximately 1.3 billion pathology image tiles from 171,189 whole-slide images.
The research addressed a fundamental computational challenge: whole-slide images are extremely large, making it difficult for conventional models to capture meaningful information across an entire slide.
Prov-GigaPath demonstrated an approach to learning representations at both the image-tile and whole-slide levels.
Although this is not itself a multimodal vision-language model, it illustrates the architectural progress that makes more advanced slide-level and multimodal systems possible.
Why it matters: Understanding tissue at the whole-slide level is essential when developing AI systems intended to support complex pathology workflows.
Research paper: A Whole-Slide Foundation Model for Digital Pathology From Real-World Data — Nature (2024)
4. What Could Multimodal AI Mean for Pathology Workflows?
The potential value of multimodal AI extends beyond improving the performance of an individual image-analysis model.
Its longer-term relevance may lie in how different types of pathology information can be connected across a diagnostic workflow.
Image and report association
AI systems may help retrieve slides associated with particular diagnostic descriptions, morphological patterns or reporting terminology.
This could support case review, research and the organization of large digital slide archives.
More context-aware image analysis
Combining image representations with relevant structured information may help models account for factors that are not apparent from morphology alone.
The usefulness of this approach will depend on the quality, completeness and relevance of the associated information.
Multislide and case-level interpretation
Many diagnostic cases involve several slides, different tissue sections and multiple stains.
Research into slide-level and multislide models aims to move beyond analyzing isolated regions and toward representing information across a larger diagnostic case.
Pathology reporting support
Vision-language models are being investigated for their ability to connect image findings with pathology terminology and generate draft descriptions or reports.
Such capabilities require careful review because an AI-generated statement can be incomplete, inaccurate or inconsistent with the full diagnostic context.
Research and education
Multimodal systems may also help organize image collections, retrieve relevant cases, connect morphological patterns with descriptions and support case-based learning.
These applications could be particularly useful in large digital pathology repositories and medical education environments.
5. The Challenges: Why Research Performance Does Not Automatically Mean Clinical Readiness
Despite the progress, multimodal AI in pathology remains an evolving research area.
A model demonstrating promising results in a research dataset should not automatically be considered suitable for routine diagnostic use.
Several challenges require attention.
1. Data quality and alignment
Images, reports and clinical records must be correctly associated. Incorrect pairings, incomplete documentation or inconsistent terminology can introduce noise into model training.
2. Differences between institutions
Laboratories use different scanners, staining protocols, tissue-processing procedures and reporting practices. These variations can affect model performance when systems are transferred between institutions.
3. Dataset representation
Training datasets may not adequately represent all specimen types, patient populations, diagnostic categories or uncommon morphological patterns.
4. Hallucination and unsupported conclusions
Generative AI systems can produce convincing but incorrect descriptions. In pathology, an unsupported statement can create serious downstream risks if it is accepted without review.
5. Clinical validation
Models require evaluation using independent data and clinically relevant endpoints. Performance should be assessed across institutions, scanners, specimen types and intended use cases.
6. Privacy and data governance
Pathology images and associated clinical information may contain sensitive patient data. Their collection, storage, processing and use for AI training must follow applicable privacy, security and institutional requirements.
7. Human oversight and accountability
AI should not be treated as an independent replacement for professional diagnostic judgment. Pathologists must remain able to review the underlying evidence, recognize limitations and make the final diagnostic decisions within their professional responsibilities.
The next stage of computational pathology will depend not only on developing more capable models, but also on establishing reliable ways to validate, monitor and integrate them into clinical environments.
6. What Digital Pathology Platforms Need to Support
As pathology AI research advances, the surrounding digital infrastructure becomes increasingly important.
A capable digital pathology environment must do more than display a whole-slide image or connect an AI model to a slide.
It needs to support the broader operational requirements of pathology practice.
Important capabilities include:
- Whole-slide image management: Secure storage, organization, retrieval and visualization of high-resolution digital slides.
- AI model integration: The ability to connect suitable AI tools to relevant image-analysis workflows.
- Case and workflow management: Organized handling of cases, review stages, reporting activities and workflow history.
- Structured reporting: Consistent documentation of diagnostic findings and relevant analysis results.
- Data integration: Appropriate connections with laboratory and hospital information systems.
- Access control: Role-based permissions and secure access to patient and diagnostic information.
- Traceability: Records of workflow activities, review actions and relevant changes.
- Scalability: Infrastructure capable of supporting growing image volumes and computational requirements.
These capabilities are important because advanced AI models cannot deliver practical value in isolation. Their usefulness depends on how effectively they connect with the people, information and processes involved in pathology practice.
7. The Road Ahead: From Image Analysis to Connected Pathology Intelligence
The development of multimodal foundation models represents an important direction in computational pathology.
Research is moving from isolated image classification toward systems that can learn relationships between tissue morphology, pathology language, clinical information and other diagnostic data.
This does not mean that every pathology workflow will immediately adopt multimodal AI. Nor does it mean that a larger model will necessarily provide better diagnostic performance in every clinical setting.
The real opportunity lies in developing systems that are clinically relevant, appropriately validated, transparent about their limitations and integrated into established professional workflows.
For digital pathology platforms, this creates an important responsibility: providing the infrastructure through which images, cases, analysis tools and reporting processes can work together in a controlled and traceable environment.
At Onward Assist, we see connected digital pathology workflows and AI-assisted analysis as important components of this evolving landscape. Our focus is on supporting pathology practices through an integrated technology environment that brings together image management, workflow coordination and AI-enabled capabilities.
The future of digital pathology will not be defined by image recognition alone. It will depend on how effectively visual evidence, clinical context and intelligent technology work together to support pathologists.
References and Further Reading
- Ding T, Wagner SJ, Song AH, et al. A multimodal whole-slide foundation model for pathology. Nature Medicine. 2025.
https://www.nature.com/articles/s41591-025-03982-3 - Vorontsov E, Shaikovski G, Casson A, et al. End-to-end multimodal pathology foundation model with clinical dialogue. Nature Medicine. 2026.
https://www.nature.com/articles/s41591-026-04521-4 - Lu MY, Chen RJ, Williamson DFK, et al. A multimodal generative AI copilot for human pathology. Nature. 2024.
https://www.nature.com/articles/s41586-024-07618-3 - Vorontsov E, Bozkurt A, Casson A, et al. A whole-slide foundation model for digital pathology from real-world data. Nature. 2024.
https://www.nature.com/articles/s41586-024-07441-w
Editorial note: This article discusses emerging research and potential applications of AI in computational pathology. Research findings should not be interpreted as evidence of clinical approval or suitability for a specific diagnostic use. Clinical implementation requires appropriate independent validation, regulatory assessment where applicable, institutional governance and pathologist oversight.

