# Pre-trained multimodal large language model enhances dermatological diagnosis using SkinGPT-4

> SkinGPT-4 is an interactive dermatology diagnostic system that uses multimodal large language models and aligns a vision transformer with Llama-2-13b-chat and offers autonomous diagnosis and treatment recommendations.

## Metadata
- Authors: Juexiao Zhou, Xiaonan He, Liyuan Sun, Jiannan Xu, Xiuying Chen, Yuetan Chu, Longxi Zhou, Xingyu Liao, Bin Zhang, Shawn Afvari, Xin Gao
- Journal: Nature Communications
- Published: 2024
- DOI: https://doi.org/10.1038/s41467-024-50043-3
- Citations: 176
- Source: Semantic Scholar
- Access: Open Access

## Technology Hub
- Hub: Large Language Models
- Discipline: Computer Science / AI
- Hub URL: https://science-database.com/technology/large-language-models
- Hub llms.txt: https://science-database.com/technology/large-language-models/llms.txt

## Abstract
Large language models (LLMs) are seen to have tremendous potential in advancing medical diagnosis recently, particularly in dermatological diagnosis, which is a very important task as skin and subcutaneous diseases rank high among the leading contributors to the global burden of nonfatal diseases. Here we present SkinGPT-4, which is an interactive dermatology diagnostic system based on multimodal large language models. We have aligned a pre-trained vision transformer with an LLM named Llama-2-13b-chat by collecting an extensive collection of skin disease images (comprising 52,929 publicly available and proprietary images) along with clinical concepts and doctors’ notes, and designing a two-step training strategy. We have quantitatively evaluated SkinGPT-4 on 150 real-life cases with board-certified dermatologists. With SkinGPT-4, users could upload their own skin photos for diagnosis, and the system could autonomously evaluate the images, identify the characteristics and categories of the skin conditions, perform in-depth analysis, and provide interactive treatment recommendations. Here, authors develop SkinGPT-4, an interactive dermatology diagnostic system that uses multimodal large language models and aligns a vision transformer with Llama-2-13b-chat. Evaluated by dermatologists, it offers autonomous diagnosis and treatment recommendations.

## Links
- DOI: https://doi.org/10.1038/s41467-024-50043-3
- Semantic Scholar: https://www.semanticscholar.org/paper/b283f9bcb85124593ad4fd20039eccb24d5d23c9
- PDF: https://www.nature.com/articles/s41467-024-50043-3.pdf
- JSON API: https://science-database.com/api/v1/technology/large-language-models

---
Generated by science-database.com — The Knowledge Interface
Paper ID: s2-b283f9bcb85124593ad4fd20039eccb24d5d23c9 | Hub: large-language-models