---
title: "Large Language Models"
slug: "large-language-models"
discipline: "Computer Science / AI"
description: "LLM research and development. Transformer architectures, training methods, alignment, reasoning capabilities, multimodal models, and AI safety."
icon: "🤖"
url: "https://science-database.com/technology/large-language-models"
api: "https://science-database.com/api/v1/technology/large-language-models"
llms_txt: "https://science-database.com/technology/large-language-models/llms.txt"
articles_indexed: 15
last_updated: "2026-09-08T05:36:38.035Z"
search_terms:
  - "large language model transformer"
  - "LLM alignment safety RLHF"
  - "multimodal AI foundation model"
source: "science-database.com"
license: "metadata CC0, abstracts belong to respective publishers"
---

# Large Language Models

LLM research and development. Transformer architectures, training methods, alignment, reasoning capabilities, multimodal models, and AI safety.

**Discipline:** Computer Science / AI  
**Indexed Papers:** 15  
**Last Updated:** 2026-09-08

## Top Publications

Ranked by citation impact across Semantic Scholar, OpenAlex & arXiv.

### Gradient-based learning applied to document recognition

- **Authors:** Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner
- **Journal:** Proceedings of the IEEE
- **Published:** 1998-01-01
- **DOI:** [10.1109/5.726791](https://doi.org/10.1109/5.726791)
- **Citations:** 59,536
- **Source:** OpenAlex
- **Access:** Open Access
- **PDF:** [Download](https://hal.science/hal-03926082/document)
- **llms.txt:** [View](https://science-database.com/technology/large-language-models/paper/oa-W2112796928/llms.txt)

> Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper rev...

### Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

- **Authors:** Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, Baining Guo
- **Journal:** 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- **Published:** 2021-10-01
- **DOI:** [10.1109/iccv48922.2021.00986](https://doi.org/10.1109/iccv48922.2021.00986)
- **Citations:** 32,421
- **Source:** OpenAlex
- **llms.txt:** [View](https://science-database.com/technology/large-language-models/paper/oa-W3138516171/llms.txt)

> This paper presents a new vision Transformer, called Swin Transformer, that capably serves as a general-purpose backbone for computer vision. Challenges in adapting Transformer from language to vision arise from differences between the two domains, such as large variations in the scale of visual entities and the high resolution of pixels in images compared to words in text. To address these differ...

### Exploring the Limits of Transfer Learning with a Unified Text-to-Text\n Transformer

- **Authors:** Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu
- **Journal:** arXiv (Cornell University)
- **Published:** 2019-10-23
- **DOI:** [10.48550/arxiv.1910.10683](https://doi.org/10.48550/arxiv.1910.10683)
- **Citations:** 8,346
- **Source:** OpenAlex
- **Access:** Open Access
- **PDF:** [Download](https://arxiv.org/pdf/1910.10683)
- **llms.txt:** [View](https://science-database.com/technology/large-language-models/paper/oa-W4288089799/llms.txt)

> Transfer learning, where a model is first pre-trained on a data-rich task\nbefore being fine-tuned on a downstream task, has emerged as a powerful\ntechnique in natural language processing (NLP). The effectiveness of transfer\nlearning has given rise to a diversity of approaches, methodology, and\npractice. In this paper, we explore the landscape of transfer learning\ntechniques for NLP by introdu...

### BioBERT: a pre-trained biomedical language representation model for biomedical text mining

- **Authors:** Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, Jaewoo Kang
- **Journal:** Bioinformatics
- **Published:** 2019-09-05
- **DOI:** [10.1093/bioinformatics/btz682](https://doi.org/10.1093/bioinformatics/btz682)
- **Citations:** 7,375
- **Source:** OpenAlex
- **Access:** Open Access
- **PDF:** [Download](https://academic.oup.com/bioinformatics/article-pdf/36/4/1234/48983216/bioinformatics_36_4_1234.pdf)
- **llms.txt:** [View](https://science-database.com/technology/large-language-models/paper/oa-W2911489562/llms.txt)

> MOTIVATION: Biomedical text mining is becoming increasingly important as the number of biomedical documents rapidly grows. With the progress in natural language processing (NLP), extracting valuable information from biomedical literature has gained popularity among researchers, and deep learning has boosted the development of effective biomedical text mining models. However, directly applying the ...

### Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting

- **Authors:** Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, Wancai Zhang
- **Journal:** Proceedings of the AAAI Conference on Artificial Intelligence
- **Published:** 2021-05-18
- **DOI:** [10.1609/aaai.v35i12.17325](https://doi.org/10.1609/aaai.v35i12.17325)
- **Citations:** 6,866
- **Source:** OpenAlex
- **Access:** Open Access
- **PDF:** [Download](https://doi.org/10.1609/aaai.v35i12.17325)
- **llms.txt:** [View](https://science-database.com/technology/large-language-models/paper/oa-W3177318507/llms.txt)

> Many real-world applications require the prediction of long sequence time-series, such as electricity consumption planning. Long sequence time-series forecasting (LSTF) demands a high prediction capacity of the model, which is the ability to capture precise long-range dependency coupling between output and input efficiently. Recent studies have shown the potential of Transformer to increase the pr...

### ChatGPT for good? On opportunities and challenges of large language models for education

- **Authors:** Enkelejda Kasneci, Kathrin Seßler, Stefan Küchemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan Günnemann, Eyke Hüllermeier, Stephan Krusche, Gitta Kutyniok, Tilman Michaeli, Claudia Nerdel, Jürgen Pfeffer, Oleksandra Poquet, Michael Sailer, Albrecht Schmidt, Tina Seidel, Matthias Stadler, J. Weller, Jochen Kühn, Gjergji Kasneci
- **Journal:** Learning and Individual Differences
- **Published:** 2023-03-09
- **DOI:** [10.1016/j.lindif.2023.102274](https://doi.org/10.1016/j.lindif.2023.102274)
- **Citations:** 6,362
- **Source:** OpenAlex
- **Access:** Open Access
- **PDF:** [Download](https://epub.ub.uni-muenchen.de/125071/1/ChatGPT_for_Good_v3.pdf)
- **llms.txt:** [View](https://science-database.com/technology/large-language-models/paper/oa-W4323655724/llms.txt)

### Restormer: Efficient Transformer for High-Resolution Image Restoration

- **Authors:** Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming–Hsuan Yang
- **Journal:** 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- **Published:** 2022-06-01
- **DOI:** [10.1109/cvpr52688.2022.00564](https://doi.org/10.1109/cvpr52688.2022.00564)
- **Citations:** 3,934
- **Source:** OpenAlex
- **llms.txt:** [View](https://science-database.com/technology/large-language-models/paper/oa-W4225672218/llms.txt)

> Since convolutional neural networks (CNNs) perform well at learning generalizable image priors from large-scale data, these models have been extensively applied to image restoration and related tasks. Recently, another class of neural architectures, Transformers, have shown significant performance gains on natural language and high-level vision tasks. While the Transformer model mitigates the shor...

### Large language models encode clinical knowledge

- **Authors:** Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Lee, Hyung Won Chung, Nathan Scales, Ajay Kumar Tanwani, Heather Cole-Lewis, Stephen Pfohl, Perry W. Payne, Martin Seneviratne, Paul Gamble, Christopher Kelly, Abubakr Babiker, Nathanael Schärli, Aakanksha Chowdhery, P. Mansfield, Dina Demner‐Fushman, Blaise Agüera y Arcas, Dale R. Webster, Greg S. Corrado, Yossi Matias, Katherine Chou, Juraj Gottweis, Nenad Tomašev, Yun Liu, Alvin Rajkomar, Joëlle Barral, Christopher Semturs, Alan Karthikesalingam, Vivek Natarajan
- **Journal:** Nature
- **Published:** 2023-07-12
- **DOI:** [10.1038/s41586-023-06291-2](https://doi.org/10.1038/s41586-023-06291-2)
- **Citations:** 3,744
- **Source:** OpenAlex
- **Access:** Open Access
- **PDF:** [Download](https://www.nature.com/articles/s41586-023-06291-2.pdf)
- **llms.txt:** [View](https://science-database.com/technology/large-language-models/paper/oa-W4384071683/llms.txt)

> Abstract Large language models (LLMs) have demonstrated impressive capabilities, but the bar for clinical applications is high. Attempts to assess the clinical knowledge of models typically rely on automated evaluations based on limited benchmarks. Here, to address these limitations, we present MultiMedQA, a benchmark combining six existing medical question answering datasets spanning professional...

### Transformer-XL: Attentive Language Models beyond a Fixed-Length Context

- **Authors:** Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V. Le, Ruslan Salakhutdinov
- **Published:** 2019-01-01
- **DOI:** [10.18653/v1/p19-1285](https://doi.org/10.18653/v1/p19-1285)
- **Citations:** 3,204
- **Source:** OpenAlex
- **Access:** Open Access
- **PDF:** [Download](https://www.aclweb.org/anthology/P19-1285.pdf)
- **llms.txt:** [View](https://science-database.com/technology/large-language-models/paper/oa-W2964110616/llms.txt)

> Transformers have a potential of learning longer-term dependency, but are limited by a fixed-length context in the setting of language modeling.We propose a novel neural architecture Transformer-XL that enables learning dependency beyond a fixed length without disrupting temporal coherence.It consists of a segment-level recurrence mechanism and a novel positional encoding scheme.Our method not onl...

### Point Transformer

- **Authors:** Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip H. S. Torr, Vladlen Koltun
- **Journal:** 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- **Published:** 2021-10-01
- **DOI:** [10.1109/iccv48922.2021.01595](https://doi.org/10.1109/iccv48922.2021.01595)
- **Citations:** 2,359
- **Source:** OpenAlex
- **llms.txt:** [View](https://science-database.com/technology/large-language-models/paper/oa-W4214755140/llms.txt)

> Self-attention networks have revolutionized natural language processing and are making impressive strides in image analysis tasks such as image classification and object detection. Inspired by this success, we investigate the application of self-attention networks to 3D point cloud processing. We design self-attention layers for point clouds and use these to construct self-attention networks for t...

### PaLM: Scaling Language Modeling with Pathways

- **Authors:** Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek S. Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James T. Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier García, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, D. Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Érica Rodrigues Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Díaz, Orhan Fırat, Michele Catasta, Wei, Jason, Meier-Hellstern, Kathy, Douglas Eck, Jeff Dean, Slav Petrov, Noah Fiedel
- **Journal:** arXiv (Cornell University)
- **Published:** 2022-04-05
- **DOI:** [10.48550/arxiv.2204.02311](https://doi.org/10.48550/arxiv.2204.02311)
- **Citations:** 2,136
- **Source:** OpenAlex
- **Access:** Open Access
- **PDF:** [Download](https://arxiv.org/pdf/2204.02311)
- **llms.txt:** [View](https://science-database.com/technology/large-language-models/paper/oa-W4224308101/llms.txt)

> Large language models have been shown to achieve remarkable performance across a variety of natural language tasks using few-shot learning, which drastically reduces the number of task-specific training examples needed to adapt the model to a particular application. To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Tran...

### Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding

- **Authors:** Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J. Fleet, Mohammad Norouzi
- **Journal:** arXiv (Cornell University)
- **Published:** 2022-05-23
- **DOI:** [10.48550/arxiv.2205.11487](https://doi.org/10.48550/arxiv.2205.11487)
- **Citations:** 2,113
- **Source:** OpenAlex
- **Access:** Open Access
- **PDF:** [Download](https://arxiv.org/pdf/2205.11487)
- **llms.txt:** [View](https://science-database.com/technology/large-language-models/paper/oa-W4281485151/llms.txt)

> We present Imagen, a text-to-image diffusion model with an unprecedented degree of photorealism and a deep level of language understanding. Imagen builds on the power of large transformer language models in understanding text and hinges on the strength of diffusion models in high-fidelity image generation. Our key discovery is that generic large language models (e.g. T5), pretrained on text-only c...

### TinyBERT: Distilling BERT for Natural Language Understanding

- **Authors:** Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Dong Chen, Linlin Li, Fang Wang, Qun Liu
- **Published:** 2020-01-01
- **DOI:** [10.18653/v1/2020.findings-emnlp.372](https://doi.org/10.18653/v1/2020.findings-emnlp.372)
- **Citations:** 1,707
- **Source:** OpenAlex
- **Access:** Open Access
- **PDF:** [Download](https://www.aclweb.org/anthology/2020.findings-emnlp.372.pdf)
- **llms.txt:** [View](https://science-database.com/technology/large-language-models/paper/oa-W3105966348/llms.txt)

> Language model pre-training, such as BERT, has significantly improved the performances of many natural language processing tasks. However, pre-trained language models are usually computationally expensive, so it is difficult to efficiently execute them on resourcerestricted devices. To accelerate inference and reduce model size while maintaining accuracy, we first propose a novel Transformer disti...

### The great Transformer: Examining the role of large language models in the political economy of AI

- **Authors:** Dieuwertje Luitse, Wiebke Denkena
- **Journal:** Big Data & Society
- **Published:** 2021-07-01
- **DOI:** [10.1177/20539517211047734](https://doi.org/10.1177/20539517211047734)
- **Citations:** 189
- **Source:** OpenAlex
- **Access:** Open Access
- **PDF:** [Download](https://journals.sagepub.com/doi/pdf/10.1177/20539517211047734)
- **llms.txt:** [View](https://science-database.com/technology/large-language-models/paper/oa-W3202773593/llms.txt)

> In recent years, AI research has become more and more computationally demanding. In natural language processing (NLP), this tendency is reflected in the emergence of large language models (LLMs) like GPT-3. These powerful neural network-based models can be used for a range of NLP tasks and their language generation capacities have become so sophisticated that it can be very difficult to distinguis...

### Transformers and large language models in healthcare: A review

- **Authors:** Subhash Nerella, Sabyasachi Bandyopadhyay, Jiaqing Zhang, Miguel Á. Contreras, Scott Siegel, Aysegül Bumin, Brandon Silva, Jessica Sena, Benjamin Shickel, Azra Bihorac, Kia Khezeli, Parisa Rashidi
- **Journal:** Artificial Intelligence in Medicine
- **Published:** 2024-06-05
- **DOI:** [10.1016/j.artmed.2024.102900](https://doi.org/10.1016/j.artmed.2024.102900)
- **Citations:** 169
- **Source:** OpenAlex
- **Access:** Open Access
- **PDF:** [Download](https://www.ncbi.nlm.nih.gov/pmc/articles/11638972)
- **llms.txt:** [View](https://science-database.com/technology/large-language-models/paper/oa-W4399367209/llms.txt)

---

*Generated by [science-database.com](https://science-database.com) — The Knowledge Interface*  
*Full data available via [JSON API](https://science-database.com/api/v1/technology/large-language-models)*