🤖
Technology Hub

Large Language Models

LLM research and development. Transformer architectures, training methods, alignment, reasoning capabilities, multimodal models, and AI safety.

Computer Science / AI
View curated hub

Results for "large language model transformer"

2,066,219 total results — showing 20 from PubMed + NASA ADS + arXiv + OpenAlex
PubMed 2025 May

Impact of large language model (ChatGPT) in healthcare: an umbrella review and evidence synthesis.

Iqbal Usman, Tanweer Afifa, Rahmanti Annisa Ristya, Greenfield David, Lee Leon Tsung-Ju, Li Yu-Chuan Jack

Journal of biomedical science

Show Abstract

BACKGROUND: The emergence of Artificial Intelligence (AI), particularly Chat Generative Pre-Trained Transformer (ChatGPT), a Large Language Model (LLM), in healthcare promises to reshape patient care, clinical decision-making, and medical education. This review aims to synthesise research findings to consolidate the implications of ChatGPT integration in healthcare and identify research gaps.

MAIN BODY: The umbrella review was conducted following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The Cochrane Library, PubMed, Scopus, Web of Science, and Google Scholar were searched from inception until February 2024. Due to the heterogeneity of the included studies, no quantitative analysis was performed. Instead, information was extracted, summarised, synthesised, and presented in a narrative form. Two reviewers undertook title, abstract, and full text screening independently. The methodological quality and overall rating of the included reviews were assessed using the A Measurement Tool to Assess systematic Reviews (AMSTAR-2) checklist. The review examined 17 studies, comprising 15 systematic reviews and 2 meta-analyses, on ChatGPT in healthcare, revealing diverse focuses. The AMSTAR-2 assessment identified 5 moderate and 12 low-quality reviews, with deficiencies like study design justification and funding source reporting. The most reported theme that emerged was ChatGPT's use in disease diagnosis or clinical decision-making. While 82.4% of studies focused on its general usage, 17.6% explored unique topics like its role in medical examinations and conducting systematic reviews. Among these, 52.9% targeted general healthcare, with 41.2% focusing on specific domains like radiology, neurosurgery, gastroenterology, public health dentistry, and ophthalmology. ChatGPT's use for manuscript review or writing was mentioned in 17.6% of reviews. Promising applications include enhancing patient care and clinical decision-making, though ethical, legal, and accuracy concerns require cautious integration.

CONCLUSION: We summarise the identified areas in reviews regarding ChatGPT's transformative impact in healthcare, highlighting patient care, decision-making, and medical education. Emphasising the importance of ethical regulations and the involvement of policymakers, we urge further investigation to ensure the reliability of ChatGPT and to promote trust in healthcare and research.

PubMed 2025 Mar

A systematic review of large language model (LLM) evaluations in clinical medicine.

Shool Sina, Adimi Sara, Saboori Amleshi Reza, Bitaraf Ehsan, Golpira Reza, Tara Mahmood

BMC medical informatics and decision making

Show Abstract

BACKGROUND: Large Language Models (LLMs), advanced AI tools based on transformer architectures, demonstrate significant potential in clinical medicine by enhancing decision support, diagnostics, and medical education. However, their integration into clinical workflows requires rigorous evaluation to ensure reliability, safety, and ethical alignment.

OBJECTIVE: This systematic review examines the evaluation parameters and methodologies applied to LLMs in clinical medicine, highlighting their capabilities, limitations, and application trends.

METHODS: A comprehensive review of the literature was conducted across PubMed, Scopus, Web of Science, IEEE Xplore, and arXiv databases, encompassing both peer-reviewed and preprint studies. Studies were screened against predefined inclusion and exclusion criteria to identify original research evaluating LLM performance in medical contexts.

RESULTS: The results reveal a growing interest in leveraging LLM tools in clinical settings, with 761 studies meeting the inclusion criteria. While general-domain LLMs, particularly ChatGPT and GPT-4, dominated evaluations (93.55%), medical-domain LLMs accounted for only 6.45%. Accuracy emerged as the most commonly assessed parameter (21.78%). Despite these advancements, the evidence base highlights certain limitations and biases across the included studies, emphasizing the need for careful interpretation and robust evaluation frameworks.

CONCLUSIONS: The exponential growth in LLM research underscores their transformative potential in healthcare. However, addressing challenges such as ethical risks, evaluation variability, and underrepresentation of critical specialties will be essential. Future efforts should prioritize standardized frameworks to ensure safe, effective, and equitable LLM integration in clinical practice.

PubMed 2025 May

Large language models for conducting systematic reviews: on the rise, but not yet ready for use-a scoping review.

Lieberum Judith-Lisa, Toews Markus, Metzendorf Maria-Inti, Heilmeyer Felix, Siemens Waldemar, Haverkamp Christian, Böhringer Daniel, Meerpohl Joerg J, Eisele-Metzger Angelika

Journal of clinical epidemiology

Show Abstract

BACKGROUND AND OBJECTIVES: Machine learning promises versatile help in the creation of systematic reviews (SRs). Recently, further developments in the form of large language models (LLMs) and their application in SR conduct attracted attention. We aimed at providing an overview of LLM applications in SR conduct in health research.

METHODS: We systematically searched MEDLINE, Web of Science, IEEEXplore, ACM Digital Library, Europe PMC (preprints), Google Scholar, and conducted an additional hand search (last search: February 26, 2024). We included scientific articles in English or German, published from April 2021 onwards, building upon the results of a mapping review that has not yet identified LLM applications to support SRs. Two reviewers independently screened studies for eligibility; after piloting, 1 reviewer extracted data, checked by another.

RESULTS: Our database search yielded 8054 hits, and we identified 33 articles from our hand search. We finally included 37 articles on LLM support. LLM approaches covered 10 of 13 defined SR steps, most frequently literature search (n = 15, 41%), study selection (n = 14, 38%), and data extraction (n = 11, 30%). The mostly recurring LLM was Generative Pretrained Transformer (GPT) (n = 33, 89%). Validation studies were predominant (n = 21, 57%). In half of the studies, authors evaluated LLM use as promising (n = 20, 54%), one-quarter as neutral (n = 9, 24%) and one-fifth as nonpromising (n = 8, 22%).

CONCLUSION: Although LLMs show promise in supporting SR creation, fully established or validated applications are often lacking. The rapid increase in research on LLMs for evidence synthesis production highlights their growing relevance.

PLAIN LANGUAGE SUMMARY: Systematic reviews are a crucial tool in health research where experts carefully collect and analyze all available evidence on a specific research question. Creating these reviews is typically time- and resource-intensive, often taking months or even years to complete, as researchers must thoroughly search, evaluate, and synthesize an immense number of scientific studies. For the present article, we conducted a review to understand how new artificial intelligence (AI) tools, specifically large language models (LLMs) like Generative Pretrained Transformer (GPT), can be used to help create systematic reviews in health research. We searched multiple scientific databases and finally found 37 relevant articles. We found that LLMs have been tested to help with various parts of the systematic review process, particularly in 3 main areas: searching scientific literature (41% of studies), selecting relevant studies (38%), and extracting important information from these studies (30%). GPT was the most commonly used LLM, appearing in 89% of the studies. Most of the research (57%) focused on testing whether these AI tools actually work as intended in this context of systematic review production. The results were mixed: about half of the studies found LLMs promising, a quarter were neutral, and one-fifth found them not promising. While LLMs show potential for making the systematic review process more efficient, there is still a lack of fully tested and validated applications. However, the increasing number of studies in this field suggests that these AI tools are becoming increasingly important in creating systematic reviews.

PubMed 2025 Oct

Large Language Model Assistant for Emergency Department Discharge Documentation.

Song Ji Woo, Park Junseong, Kim Ji Hoon, You Seng Chan

JAMA network open

Show Abstract

IMPORTANCE: Emergency department (ED) discharge documentation is time-consuming and often incomplete.

OBJECTIVE: To develop a large language model (LLM) assistant that generates ED discharge notes and to evaluate its effectiveness on documentation quality and workflow efficiency.

DESIGN, SETTING, AND PARTICIPANTS: This comparative effectiveness study, which was conducted at a 2400-bed tertiary care hospital in South Korea, consisted of 2 primary phases: a development phase and sequential validation of the LLM assistant. In the randomized sequential prospective validation, 6 emergency physicians first wrote discharge notes manually (session 1), then edited LLM-generated drafts after a 1-hour washout period (session 2). Three independent physicians evaluated 300 note sets (each containing a manual note, an LLM draft, and an LLM-assisted note). For model development and validation, patient records from ED visits between September 1, 2022, and August 31, 2023, were used. The inclusion criteria encompassed adult patients (aged ≥17 years) and pediatric patients with nondisease conditions (eg, trauma, poisoning, or burns). Emergency physicians selected 592 representative cases for training and 50 for validation.

EXPOSURE: A commercially available text generation transformer model was used as a core LLM, fine-tuned using the 592 training cases. Two distinct processing pipelines were implemented within the LLM assistant due to different input data: (1) for patients managed solely by emergency physicians, using the ED initial record and prescription list, and (2) for those requiring specialty consultations, using the ED initial record and consultation request form.

MAIN OUTCOMES AND MEASURES: Quality of notes using 4C metrics (completeness, correctness, conciseness, and clinical utility) on a Likert scale ranging from 1 to 5 and time taken to complete the notes manually and with the LLM assistant.

RESULTS: Of the 50 test cases, the mean (SD) patient age was 57.7 (23.1) years, and 28 patients (56%) were female. LLM-assisted notes achieved higher scores than manual notes in completeness (4.23 [95% CI, 4.17-4.28] vs 4.03 [95% CI, 3.96-4.09]), correctness (4.38 [95% CI, 4.33-4.42] vs 4.20 [95% CI, 4.14-4.26]), conciseness (4.23 [95% CI, 4.18-4.28] vs 4.11 [95% CI, 4.05-4.17]), and clinical utility (4.17 [95% CI, 4.11-4.23] vs 3.85 [95% CI, 3.78-3.91]) (all P&#x2009;<&#x2009;.001). When compared with LLM drafts, LLM-assisted notes excelled in conciseness (4.23 vs 3.98 [95% CI, 3.91-4.04]; P&#x2009;<&#x2009;.001) and maintained equivalent clinical utility (4.17 vs 4.16 [95% CI, 4.11-4.21]; P&#x2009;>&#x2009;.99), but scored lower in completeness (4.23 vs 4.34 [95% CI, 4.29-4.39]; P&#x2009;=&#x2009;.001) and correctness (4.38 vs 4.45 [95% CI, 4.41-4.49]; P&#x2009;<&#x2009;.001). The median documentation time per note dropped from 69.5 (95% CI, 65.5-78.0) seconds for manual notes to 32.0 (95% CI, 29.5-36.0) seconds for LLM-assisted notes (P&#x2009;<&#x2009;.001).

CONCLUSION: In this comparative effectiveness study, use of an on-site LLM assistant was associated with reduced writing time for ED discharge notes compared with manual note-taking, without compromising documentation quality, representing a critical advancement in the use of artificial intelligence for clinical practice.

PubMed 2024 Jul

Large Language Model-Augmented Auto-Delineation of Treatment Target Volume in Radiation Therapy.

Rajendran Praveenbalaji, Yang Yong, Niedermayr Thomas R, Gensheimer Michael, Beadle Beth, Le Quynh-Thu, Xing Lei, Dai Xianjin

ArXiv

Show Abstract

Radiation therapy (RT) is one of the most effective treatments for cancer, and its success relies on the accurate delineation of targets. However, target delineation is a comprehensive medical decision that currently relies purely on manual processes by human experts. Manual delineation is time-consuming, laborious, and subject to interobserver variations. Although the advancements in artificial intelligence (AI) techniques have significantly enhanced the auto-contouring of normal tissues, accurate delineation of RT target volumes remains a challenge. In this study, we propose a visual language model-based RT target volume auto-delineation network termed Radformer. The Radformer utilizes a hierarchical vision transformer as the backbone and incorporates large language models to extract text-rich features from clinical data. We introduce a visual language attention module (VLAM) for integrating visual and linguistic features for language-aware visual encoding (LAVE). The Radformer has been evaluated on a dataset comprising 2985 patients with head-and-neck cancer who underwent RT. Metrics, including the Dice similarity coefficient (DSC), intersection over union (IOU), and 95th percentile Hausdorff distance (HD95), were used to evaluate the performance of the model quantitatively. Our results demonstrate that the Radformer has superior segmentation performance compared to other state-of-the-art models, validating its potential for adoption in RT practice.

PubMed 2026 Feb

Current concerns and future directions of large language model ChatGPT in medicine: a machine-learning-driven global-scale bibliometric analysis.

Guo Song-Bin, Liu Deng-Yao, Fang Xiao-Jie, Meng Yuan, Zhou Zhen-Zhong, Li Jing, Li Mei, Luo Li-Ling, Li Hai-Long, Cai Xiu-Yu, Huang Wei-Juan, Tian Xiao-Peng

International journal of surgery (London, England)

Show Abstract

BACKGROUND: In its infancy, large language model (LLM) Chat Generative Pre-trained Transformer (ChatGPT) has delivered significant transformational opportunities across an entire healthcare field and is bound to generate an even more impressive impact in the foreseeable future. Nevertheless, it remains fraught with numerous drawbacks and challenges. Therefore, this study aims to determine the current global concerns and future directions of ChatGPT in medicine to inform subsequent research and policymaking.

METHODS: This study retrospectively analyzed the global attention and development patterns of ChatGPT in different medical disciplines and geographical regions. Furthermore, based on machine learning algorithms, it revealed the current global concerns and future directions of ChatGPT in the medical field.

RESULTS: ChatGPT enjoyed a favorable development trend (Growth Rate Per Month: 26.97%) and global cooperation (International Co-authorships: 25.09%) in medicine. Internal Medicine was the best developed, while Surgery [odds ratio (OR), 0.761; 95% confidence interval (CI), 0.608-0.954; P =&#xa0;0.018], Health Care (OR, 0.744; 95% CI, 0.583-0.950; P =&#xa0;0.018), Medical Informatics (OR, 0.622; 95% CI, 0.433-0.893; P =&#xa0;0.010), Radiology (OR, 0.625; 95% CI, 0.433-0.901; P =&#xa0;0.012), Public Health (OR, 0.611; 95% CI, 0.416-0.896; P =&#xa0;0.012), and Oncology (OR, 0.571; 95% CI, 0.346-0.943; P =&#xa0;0.029) needed further development. The unsupervised hierarchical clustering algorithm divided the global concerns of ChatGPT in medicine into six clusters, among which Cluster 2 (The Applications of ChatGPT in Oncology Patient Management and Decision-Making) is the emerging research cluster, and Cluster 4 (The Accuracy and Safety of ChatGPT in Health Information Recommendation) achieves maximal impact. With the Walktrap algorithm, we found that ethics [relevance percentage (RP)&#xa0;=&#xa0;82.1%, development percentage (DP)&#xa0;=&#xa0;92.9%] is well developed but still leaves numerous pending issues, and medical education (RP&#xa0;=&#xa0;100%, DP&#xa0;=&#xa0;32.1%) and clinical decision support (RP&#xa0;=&#xa0;89.3%, DP&#xa0;=&#xa0;35.7%) are highly relevant but under-developed with ChatGPT, highlighting their impressive future research prospects. More importantly, through a comprehensive analysis of ChatGPT's attention, application, and impact in different regions, we found that underdeveloped and resource-poor regions have little, which will exacerbate global health inequalities, emphasizing the urgency of the relevant policy formulation and international assistance.

CONCLUSIONS: This study revealed ChatGPT's global attention and development patterns in different medical disciplines and geographical regions, its current global concerns, and future directions. This information will provide a critical reference for subsequent research and policymaking on LLMs throughout the entire field of medicine.

PubMed Review 2025 07

Road of Large Language Model: Source, Challenge, and Future Perspectives.

Zhao Wei, Yang Xin, Lyu Zhihan, Xu Cai, Guan Ziyu

Research (Washington, D.C.)

Show Abstract

Language model (LM), a foundational algorithm in the development of capable artificial intelligence, has been widely explored, achieving remarkable attainment. As research advances, large language models (LLMs) have emerged by pretraining transformer-based models on large-scale corpora. These models showed great zero-shot and few-shot learning capabilities across a variety of tasks, attracting widespread attention from both academia and industry. Despite impressive performance, LLMs still tackle challenges in addressing complex real-world scenarios. Recently, the advent of DeepSeek has reignited intense interest among researchers. In this paper, we provide a concise development history of LLM and discuss current challenges and future perspective. In practice, we focus on 4 crucial aspects of LLMs, including emergent abilities, human alignment, retrieval augmented generation, and applications in specific domains.

PubMed Review 2023 Oct

ChatGPT and large language model (LLM) chatbots: The current state of acceptability and a proposal for guidelines on utilization in academic medicine.

Kim Jin K, Chua Michael, Rickard Mandy, Lorenzo Armando

Journal of pediatric urology

Show Abstract

INTRODUCTION: There is currently no clear consensus on the standards for using large language models such as ChatGPT in academic medicine. Hence, we performed a scoping review of available literature to understand the current state of LLM use in medicine and to provide a guideline for future utilization in academia.

MATERIALS AND METHODS: A scoping review of the literature was performed through a Medline search on February 16, 2023 using a combination of keywords including artificial intelligence, machine learning, natural language processing, generative pre-trained transformer, ChatGPT, and large language model. There were no restrictions to language or date of publication. Records not pertaining to LLMs were excluded. Records pertaining to LLM ChatBots and ChatGPT were identified and evaluated separately. Among the records pertaining to LLM ChatBots and ChatGPT, those that suggest recommendations for ChatGPT use in academia were utilized to create guideline statements for ChatGPT and LLM use in academic medicine.

RESULTS: A total of 87 records were identified. 30 records were not pertaining to large language models and were excluded. 54 records underwent a full-text review for evaluation. There were 33 records related to LLM ChatBots or ChatGPT.

DISCUSSION: From assessing these texts, five guideline statements for LLM use was developed: (1) ChatGPT/LLM cannot be cited as an author in scientific manuscripts; (2) If use of ChatGPT/LLM are considered for use in academic work, author(s) should have at least a basic understanding of what ChatGPT/LLM is; (3) Do not use ChatGPT/LLM to produce entirety of text in manuscripts; humans must be held accountable for use of ChatGPT/LLM and contents created by ChatGPT/LLM should be meticulously verified by humans; (4) ChatGPT/LLMs may be used for editing and refining of text; (5) Any use of ChatGPT/LLM should be transparent and should be clearly outlined in scientific manuscripts and acknowledged.

CONCLUSION: Future authors should remain mindful of the potential impact their academic work may have on healthcare and continue to uphold the highest ethical standards and integrity when utilizing ChatGPT/LLM.

NASA ADS 2024-08-00
4 citations

Transformer Explainer: Learning LLM Transformers with Interactive Visual Explanation and Experimentation

Cho, Aeree, Kim, Grace C., Karpekov, Alexander, Lee, Seongmin, Helbling, Alec, Hoover, Benjamin, Wang, Zijie J., Kahng, Minsuk, Chau, Duen Horng

arXiv e-prints

Show Abstract

The Transformer architecture underpins modern large language models powering state-of-the-art text generation and AI applications. However, its complexity makes it difficult for non-experts to learn. Existing resources often lack interactivity, rely on static descriptions of simplified architectures, or fail to reflect models' behavior with real data. To address this gap, we introduce Transformer Explainer, an interactive visualization tool for non-experts to learn Transformers. The tool integrates an overview illustrating the Transformer's data flow with on-demand explanations that gradually reveal mathematical details. Smooth transitions across abstraction levels highlight the interplay between high-level structures and low-level operations. Running a live GPT-2 instance directly in the browser, Transformer Explainer empowers learners to experiment with custom input and hyperparameters without setup, observing next-token predictions in real time. A 90-participant user study showed that our tool offered significant advantages in improving user understanding and engagement. Transformer Explainer has attracted over 490,000 users.

NASA ADS 2023-10-00
17 citations

HowToCaption: Prompting LLMs to Transform Video Annotations at Scale

Shvetsova, Nina, Kukleva, Anna, Hong, Xudong, Rupprecht, Christian, Schiele, Bernt, Kuehne, Hilde

arXiv e-prints

Show Abstract

Instructional videos are a common source for learning text-video or even multimodal representations by leveraging subtitles extracted with automatic speech recognition systems (ASR) from the audio signal in the videos. However, in contrast to human-annotated captions, both speech and subtitles naturally differ from the visual content of the videos and thus provide only noisy supervision. As a result, large-scale annotation-free web video training data remains sub-optimal for training text-video models. In this work, we propose to leverage the capabilities of large language models (LLMs) to obtain high-quality video descriptions aligned with videos at scale. Specifically, we prompt an LLM to create plausible video captions based on ASR subtitles of instructional videos. To this end, we introduce a prompting method that is able to take into account a longer text of subtitles, allowing us to capture the contextual information beyond one single sentence. We further prompt the LLM to generate timestamps for each produced caption based on the timestamps of the subtitles and finally align the generated captions to the video temporally. In this way, we obtain human-style video captions at scale without human supervision. We apply our method to the subtitles of the HowTo100M dataset, creating a new large-scale dataset, HowToCaption. Our evaluation shows that the resulting captions not only significantly improve the performance over many different benchmark datasets for zero-shot text-video retrieval and video captioning, but also lead to a disentangling of textual narration from the audio, boosting the performance in text-video-audio tasks.

NASA ADS 2024-12-00
8 citations

PsychAdapter: Adapting LLM Transformers to Reflect Traits, Personality and Mental Health

Vu, Huy, Nguyen, Huy Anh, Ganesan, Adithya V, Juhng, Swanie, Kjell, Oscar N. E., Sedoc, Joao, Kern, Margaret L., Boyd, Ryan L., Ungar, Lyle, Schwartz, H. Andrew, Eichstaedt, Johannes C.

arXiv e-prints

Show Abstract

Artificial intelligence-based language generators are now a part of most people's lives. However, by default, they tend to generate "average" language without reflecting the ways in which people differ. Here, we propose a lightweight modification to the standard language model transformer architecture - "PsychAdapter" - that uses empirically derived trait-language patterns to generate natural language for specified personality, demographic, and mental health characteristics (with or without prompting). We applied PsychAdapters to modify OpenAI's GPT-2, Google's Gemma, and Meta's Llama 3 and found generated text to reflect the desired traits. For example, expert raters evaluated PsychAdapter's generated text output and found it matched intended trait levels with 87.3% average accuracy for Big Five personalities, and 96.7% for depression and life satisfaction. PsychAdapter is a novel method to introduce psychological behavior patterns into language models at the foundation level, independent of prompting, by influencing every transformer layer. This approach can create chatbots with specific personality profiles, clinical training tools that mirror language associated with psychological conditionals, and machine translations that match an authors reading or education level without taking up LLM context windows. PsychAdapter also allows for the exploration psychological constructs through natural language expression, extending the natural language processing toolkit to study human psychology.

arXiv 2024-02-05

A Survey on Transformer Compression

Yehui Tang, Yunhe Wang, Jianyuan Guo, Zhijun Tu, Kai Han, Hailin Hu, Dacheng Tao

arXiv:2402.05964v2 [cs.LG]

Show Abstract

Transformer plays a vital role in the realms of natural language processing (NLP) and computer vision (CV), specially for constructing large language models (LLM) and large vision models (LVM). Model compression methods reduce the memory and computational cost of Transformer, which is a necessary step to implement large language/vision models on practical devices. Given the unique architecture of Transformer, featuring alternative attention and feedforward neural network (FFN) modules, specific compression techniques are usually required. The efficiency of these compression methods is also paramount, as retraining large models on the entire training dataset is usually impractical. This survey provides a comprehensive review of recent compression methods, with a specific focus on their application to Transformer-based models. The compression methods are primarily categorized into pruning, quantization, knowledge distillation, and efficient architecture design (Mamba, RetNet, RWKV, etc.). In each category, we discuss compression methods for both language and vision tasks, highlighting common underlying principles. Finally, we delve into the relation between various compression methods, and discuss further directions in this domain.

arXiv 2024-02-18

Learning From Failure: Integrating Negative Examples when Fine-tuning Large Language Models as Agents

Renxi Wang, Haonan Li, Xudong Han, Yixuan Zhang, Timothy Baldwin

arXiv:2402.11651v2 [cs.CL]

Show Abstract

Large language models (LLMs) have achieved success in acting as agents, which interact with environments through tools such as search engines. However, LLMs are optimized for language generation instead of tool use during training or alignment, limiting their effectiveness as agents. To resolve this problem, previous work has first collected interaction trajectories between LLMs and environments, using only trajectories that successfully finished the task to fine-tune smaller models, making fine-tuning data scarce and acquiring it both difficult and costly. Discarding failed trajectories also leads to significant wastage of data and resources and limits the possible optimization paths during fine-tuning. In this paper, we argue that unsuccessful trajectories offer valuable insights, and LLMs can learn from these trajectories through appropriate quality control and fine-tuning strategies. By simply adding a prefix or suffix that tells the model whether to generate a successful trajectory during training, we improve model performance by a large margin on mathematical reasoning, multi-hop question answering, and strategic question answering tasks. We further analyze the inference results and find that our method provides a better trade-off between valuable information and errors in unsuccessful trajectories. To our knowledge, we are the first to demonstrate the value of negative trajectories and their application in agent-tunning scenarios. Our findings offer guidance for developing better agent-tuning methods and low-resource data usage techniques.

arXiv 2023-12-17

Demystifying Instruction Mixing for Fine-tuning Large Language Models

Renxi Wang, Haonan Li, Minghao Wu, Yuxia Wang, Xudong Han, Chiyu Zhang, Timothy Baldwin

arXiv:2312.10793v3 [cs.CL]

Show Abstract

Instruction tuning significantly enhances the performance of large language models (LLMs) across various tasks. However, the procedure to optimizing the mixing of instruction datasets for LLM fine-tuning is still poorly understood. This study categorizes instructions into three primary types: NLP downstream tasks, coding, and general chat. We explore the effects of instruction tuning on different combinations of datasets on LLM performance, and find that certain instruction types are more advantageous for specific applications but can negatively impact other areas. This work provides insights into instruction mixtures, laying the foundations for future research.

arXiv 2023-04-24

WizardLM: Empowering large pre-trained language models to follow complex instructions

Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, Daxin Jiang

The Twelfth International Conference on Learning Representations (ICLR 2024)

Show Abstract

Training large language models (LLMs) with open-domain instruction following data brings colossal success. However, manually creating such instruction data is very time-consuming and labor-intensive. Moreover, humans may struggle to produce high-complexity instructions. In this paper, we show an avenue for creating large amounts of instruction data with varying levels of complexity using LLM instead of humans. Starting with an initial set of instructions, we use our proposed Evol-Instruct to rewrite them step by step into more complex instructions. Then, we mix all generated instruction data to fine-tune LLaMA. We call the resulting model WizardLM. Human evaluations on a complexity-balanced test bed and Vicuna's testset show that instructions from Evol-Instruct are superior to human-created ones. By analyzing the human evaluation results of the high complexity part, we demonstrate that outputs from our WizardLM are preferred to outputs from OpenAI ChatGPT. In GPT-4 automatic evaluation, WizardLM achieves more than 90\% capacity of ChatGPT on 17 out of 29 skills. Even though WizardLM still lags behind ChatGPT in some aspects, our findings suggest that fine-tuning with AI-evolved instructions is a promising direction for enhancing LLMs. Our code and data are public at https://github.com/nlpxucan/WizardLM

OpenAlex 2025-02-26

Basics of Large Language Models - transformers to LLMs

Andrew Green

Show Abstract

We will introduce Large Language Models (LLMs), focusing on their core architecture, training methodologies, practical applications, and some important considerations. We begin with a brief exploration of the transformer architecture, examining how architectural choices and scaling laws have shaped modern language models. We then investigate key training paradigms, with particular emphasis on Reinforcement Learning from Human Feedback (RLHF) and instruction tuning, while briefly touching on emerging approaches like Direct Preference Optimisation (DPO). The discussion progresses through the evolution of knowledge integration techniques, from basic context stuffing to Retrieval-Augmented Generation (RAG), highlighting their practical implications. We will discuss the critical issue of hallucinations in LLM outputs, providing concrete examples and verification strategies. To conclude, a quick overview of the current software landscape and immediate future developments in the field, and a look ahead to the rest of the webinar series.

OpenAlex 2026-10-11

Generative AI and Large Language Models: Transformers, Prompt Engineering, and AI Agents

Dinesh Bhagwan Hanchate, Prajakta A. Satarakar, Manoj D. Shelar

Show Abstract

Generative AI has transitioned from the lab to the classroom, hospitals, software teams, and boardrooms. Large language models (LLMs), transformer-based foundation models, are at the heart of this paradigm change, with their ability to write texts, reason, summarize, translate, and code. However, these developments have left many students and researchers, as well as practitioners, with disjointed bits of knowledge. This book was developed with that knowledge in mind in order to streamline all that information in a single comprehensive and hands-on natural language processing (NLP) and generative AI guide. The book follows the path of the technology itself. Chapter 1 covers transformer architectures ranging from the self-attention mechanism to encoder-decoder and decoder-only architectures. The course also includes lectures on mixture of experts, scaling laws, pretraining, fine-tuning, alignment and multimodal models. Targeting prompt engineering and in-context learning is the focus of Chapter 2. It uses zero shot, few shot, chain of thought prompting, system prompts and context engineering as skills to learn and test. Chapter 3 focuses on LLM reasoning and test-time compute, presenting various ways in which deep learning systems can be extended to form reasoning models, self-consistency, and search over reasoning paths. As it is not possible to know everything, Chapters 4 and 5 discuss retrieval-augmented generation (RAG). They start with embeddings and vector databases, chunking strategies and semantic search. They then transition to advanced RAG – hybrid search, reranking, knowledge graphs, GraphRAG, and agentic RAG, all while also focusing on evaluation and LLMOps for production. The strands are woven together in Chapter 6 under the umbrella of agentic AI and AI agents. It examines the use of tools, the calling of functions, the memory of an agent, as well as planning and multi-agent systems and the way autonomous agents are transforming work. Chapter 7 ends with trustworthy and responsible AI. It speaks about hallucination, bias/fairness, explainability, data privacy, AI safety, AI ethics and AI governance, as without trust, the capability cannot be sustained. The chapters are designed with learning and reference in mind for each. It introduces the foundations and outlines the key techniques side-by-side in summary tables, covers present open challenges and future directions, and concludes with practical advice. Benchmarking and evaluation is handled very transparently with regard to the deficiencies in existing approaches – and I do hope readers will take the shortcomings as a challenge and a starting point for their own work. The working levels that the book is aimed at are graduate and undergraduate students, academic researchers, machine learning engineers, data scientists, and technology leaders. Can also be used as a textbook in the course where it was used or as a builder's handbook or as a literature map for those preparing an indexed study in a Scopus journal. I'm focusing more on enduring principles than fleeting tools because no one can stay current in a science that is evolving monthly. We hope this book empowers you to grasp generative AI, create responsibly, and shape the future of AI together.