# BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models

> BLIP-2 achieves state-of-the-art performance on various vision-language tasks, despite having significantly fewer trainable parameters than existing methods, and is demonstrated's emerging capabilities of zero-shot image-to-text generation that can follow natural language instructions.

## Metadata
- Authors: Junnan Li, Dongxu Li, S. Savarese, Steven C. H. Hoi
- Published: 2023
- DOI: https://doi.org/10.48550/arXiv.2301.12597
- Citations: 8,021
- Source: Semantic Scholar
- Access: Open Access

## Technology Hub
- Hub: Large Language Models
- Discipline: Computer Science / AI
- Hub URL: https://science-database.com/technology/large-language-models
- Hub llms.txt: https://science-database.com/technology/large-language-models/llms.txt

## Abstract
The cost of vision-and-language pre-training has become increasingly prohibitive due to end-to-end training of large-scale models. This paper proposes BLIP-2, a generic and efficient pre-training strategy that bootstraps vision-language pre-training from off-the-shelf frozen pre-trained image encoders and frozen large language models. BLIP-2 bridges the modality gap with a lightweight Querying Transformer, which is pre-trained in two stages. The first stage bootstraps vision-language representation learning from a frozen image encoder. The second stage bootstraps vision-to-language generative learning from a frozen language model. BLIP-2 achieves state-of-the-art performance on various vision-language tasks, despite having significantly fewer trainable parameters than existing methods. For example, our model outperforms Flamingo80B by 8.7% on zero-shot VQAv2 with 54x fewer trainable parameters. We also demonstrate the model's emerging capabilities of zero-shot image-to-text generation that can follow natural language instructions.

## Links
- DOI: https://doi.org/10.48550/arXiv.2301.12597
- Semantic Scholar: https://www.semanticscholar.org/paper/3f5b31c4f7350dc88002c121aecbdc82f86eb5bb
- PDF: http://arxiv.org/pdf/2301.12597
- JSON API: https://science-database.com/api/v1/technology/large-language-models

---
Generated by science-database.com — The Knowledge Interface
Paper ID: s2-3f5b31c4f7350dc88002c121aecbdc82f86eb5bb | Hub: large-language-models