A large language model (LLM) is a neural network trained on massive text datasets to perform language tasks such as generation, summarization, translation, analysis, and question answering.1
Architecture
Most modern LLMs use the transformer architecture because it is efficient and parallelizable compared with earlier recurrent approaches.1 Decoder-only GPT-style models are especially common because they work well with prompting and instruction tuning.1
Capabilities and limitations
LLMs can be highly flexible, but their outputs may be unreliable when the training data is biased, incomplete, or inaccurate.1 As a result, their usefulness depends on data quality, safety, and alignment with human intent.1
Historical development
Early language models in the 1990s and 2000s relied on statistical methods and large text corpora.1 The 2017 transformer breakthrough marked a major turning point, followed by wider adoption of models such as BERT, GPT-2, and GPT-3.1
Evaluation
Benchmarks for LLMs often measure reasoning ability, factual accuracy, alignment, and safety.1 These tests help compare models and track progress over time.1
Related Concepts
Retrieval-augmented generation (RAG) is a common extension pattern for LLMs, where external documents are retrieved at inference time and fed into the prompt2. RAG can reduce hallucinations, improve access to fresh or domain-specific knowledge, and increase answer transparency through source grounding2.
Limitations
RAG does not remove all error modes, because the model can still misread retrieved passages or generate misleading outputs from correct source material2.
RAG does not eliminate errors, because the model can still misread or misunderstand retrieved content2.
Retrieval-Augmented Generation
Retrieval-augmented generation (RAG) extends large language models by retrieving relevant external information before generating answers2. RAG can draw from documents, databases, uploaded files, or web sources to produce more current or domain-specific responses2.
Benefits
RAG helps reduce hallucinations by grounding answers in retrieved material and can improve transparency through cited sources2. It also lets systems use new information without full retraining, which can save time and compute2.