OpenAI’s ChatGPT reached 800 million weekly active users by the September to October 2025 window, the clearest sign yet that large language models have moved from research curiosity to everyday infrastructure. Behind that one chat box sits a system that does something deceptively simple: it predicts the next word, over and over, at massive scale. The mechanics from IBM’s technical playbook, the training pipeline, and the handful of companies driving it, from Anthropic to Meta, are worth understanding before the next model launch reframes the whole conversation again.
Key Takeaways
- A large language model is a deep learning model trained on immense amounts of data, built on a transformer architecture, that works as a giant statistical prediction machine repeatedly predicting the next word.
- The transformer architecture that powers every modern LLM was introduced in 2017, and its self-attention mechanism is the prime innovation that lets models weigh different tokens against each other.
- ChatGPT climbed from approximately 400 million weekly active users in February 2025 to 800 million by the September to October timeframe, with over 4 million developers now building on it.
- Reported parameter counts have stayed near 1 trillion for three years, even as frontier labs stopped reporting them.
- Global AI compute capacity grew 3.3x per year since 2022, reaching 17.1 million H100-equivalents, with Nvidia accounting for over 60% of total compute.
- Meta’s 405-billion-parameter Llama 3.1 is the first frontier-level open source AI model, trained on over 15 trillion tokens and competitive with GPT-4, GPT-4o, and Claude 3.5 Sonnet.
- The transparency trade-off is real: training code, parameter counts, dataset sizes, and training duration are no longer disclosed for the most resource-intensive systems from OpenAI, Anthropic, and Google.
What Is a Large Language Model?
A large language model is, according to IBM, a category of deep learning model trained on immense amounts of data, capable of understanding and generating natural language and other content across a wide range of tasks. Prediction is the defining behaviour. As described by IBM, LLMs work as giant statistical prediction machines that repeatedly predict the next word in a sequence, learning patterns in their training text and generating language that follows those patterns.
That simple loop scales into something powerful. The same model that drafts an email can summarise a research paper, debug code, or draft a legal clause, because all of those tasks reduce to predicting plausible text. The reach is now enormous: according to OpenAI, ChatGPT alone processes over 8 billion tokens per minute through its API. For a sense of how fast adoption has spread across consumer and enterprise use, the Claude vs ChatGPT benchmark data tracks the same models in head-to-head usage and capability terms.
By the numbers: OpenAI reported ChatGPT reaching 800 million weekly active users by late 2025, up from roughly 400 million in February, with over 4 million developers building applications on top of it. The scale shift in under a year is what turned LLMs from a demo into infrastructure that companies now plan around.
What is a large language model in simple terms?
A large language model is software that has read a huge slice of the internet and learned to continue text in a way that matches the patterns it saw. It does not look up answers; it predicts one token at a time, using the statistical relationships it learned in training to make its best guess at every step. Useful, but a guess.
How Do Large Language Models Work?
LLMs turn text into numbers, then turn numbers back into text, and according to IBM, the first step works precisely: input text is broken into smaller machine-readable units called tokens during tokenization, where tokens are words, subwords, or characters, and each token is then mapped to a vector of numbers called an embedding. Those embeddings, not the words themselves, are what the model actually computes on.
Attention sits at the heart of the system. To compute attention, each embedding is projected into three distinct vectors using learned weight matrices: a query, a key, and a value. This is how a model keeps track of what “it” refers to forty words earlier.
Those connections are stored as weights. Per IBM, LLMs can have billions or trillions of these weights, which are one type of parameter, the internal configuration variables that control how the model processes data and makes predictions.
Learning those weights is brute-force statistics. During training, the model makes predictions across millions of examples, a loss function quantifies the error of each prediction, and weights are updated through backpropagation and gradient descent in an iterative cycle. Repeat that enough, and the model learns grammar, facts, reasoning structures, and writing styles.
The count of those weights is where the headline Parameters figures come from, and the trend has surprised observers. That count, according to Stanford HAI data detailed in the parameters section below, shows Parameters plateaued near a trillion for three years while training compute kept climbing, a result that reframes the whole scaling story.
The Transformer: The 2017 Breakthrough Behind Every LLM
The entire modern LLM era traces back to a single architecture paper. Researchers proposed the Transformer, a network architecture based solely on attention mechanisms, dispensing with recurrence and convolutions entirely, and showed it was superior in quality while being more parallelizable and requiring significantly less time to train. That paper, “Attention Is All You Need,” replaced the step-by-step processing of earlier networks with something that reads a whole sequence at once.
Those original results were modest by today’s standards, which makes them a useful anchor. The model achieved 28.4 BLEU on the WMT 2014 English-to-German translation task and a new single-model state-of-the-art 41.8 BLEU on English-to-French after training for 3.5 days on eight GPUs.
Eight GPUs and three and a half days produced a translation model. The architecture stayed; the scale exploded. IBM notes that transformer models, introduced in 2017, are useful due to their self-attention mechanism, which allows them to pay attention to different tokens at different moments.
Key finding: The 2017 transformer that started everything trained in 3.5 days on eight GPUs to hit 41.8 BLEU on English-to-French translation. Less than a decade later, Meta trained a single model on over 16 thousand H100 GPUs. The architecture barely changed; the compute behind it grew by orders of magnitude.
The lesson worth holding onto is that the breakthrough was a design choice, not a hardware win. Attention solved a problem that more compute alone never would have.
How LLMs Are Trained: Pretraining, Fine-Tuning, and RLHF
Training happens in three stages, and IBM’s documentation walks through each: LLMs are initially trained with self-supervised learning, a technique that uses unlabeled data, starting from massive text corpora spanning billions or trillions of words from books, articles, websites, and code. This pretraining is where the model absorbs broad knowledge of language and the world, without anyone hand-labelling the data.
A raw pretrained model knows a lot but follows instructions poorly. That is what fine-tuning fixes. After training, LLMs can be fine-tuned to make them more useful in certain contexts, most often in a supervised context with a much smaller, labelled dataset. A general model can be fine-tuned on legal question-and-answer pairs to build a legal assistant, for example.
The final polish is human feedback. Reinforcement learning from human feedback (RLHF) is a form of fine-tuning where humans rank model outputs and the model is trained to prefer outputs that humans rank higher, and it is often used in alignment, the process of making outputs useful, safe, and consistent with human values. When the model finally answers a prompt, it tokenizes the prompt, converts it into embeddings, and uses its transformer to generate text one token at a time in a process called inference.
| Stage | What it does | Data used |
|---|---|---|
| Pretraining | Builds broad language and world knowledge | Billions to trillions of unlabeled words |
| Supervised fine-tuning | Adapts the model to specific tasks and instructions | Smaller, labelled datasets |
| RLHF | Aligns outputs to be useful, safe, and human-preferred | Human rankings of model outputs |
| Inference | Generates the actual response, one token at a time | The user’s prompt plus context |
Source: IBM Think
How are large language models trained?
LLMs are trained in three passes: pretraining on huge unlabeled text to learn language, supervised fine-tuning on smaller labelled sets to follow instructions, and RLHF, where humans rank outputs and the model learns to prefer higher-ranked ones for alignment. Only after all three does a model behave like the assistants people use.
What Are Parameters, and Why Did They Stop Growing?
Parameters are the dials a model tunes during training, and for years more parameters meant a better model. IBM describes them as the internal configuration variables of a machine learning model that control how it processes data and makes predictions, with LLMs containing billions or trillions of them. The headline parameter numbers grew quickly through the early 2020s, then flattened.
That plateau is striking. According to Stanford HAI’s 2026 AI Index, reported parameter counts have stayed near 1 trillion for three years, though reporting from frontier labs has stopped, while training compute, which can be estimated independently, has continued to rise. Growth moved from parameter count to compute and data quality. Stanford’s researchers found that OLMo 3.1 Think, a 32-billion-parameter model with nearly 90 times fewer parameters than Grok 4, achieves comparable results on several benchmarks through pruning, deduplication, and curation alone.
Worth noting: Stanford HAI reports global AI compute capacity grew 3.3x per year since 2022, reaching 17.1 million H100-equivalents, with Nvidia supplying over 60% of it. The scaling story quietly shifted from “add parameters” to “add compute and cleaner data,” which is why a 32-billion-parameter model can now rival one many times larger.
How many parameters does a large language model have?
Frontier LLMs sit in the range of roughly 1 trillion parameters, a figure that has held for about three years. Open models disclose more: Meta’s largest Llama 3.1 model is the first frontier-level open source AI model, and at 405 billion parameters it puts the figure in the name. Smaller models with a few billion parameters can still perform well on focused tasks.
The Key Players: OpenAI, Anthropic, Google, Meta, and the Open-Source Wave
A handful of companies build the models everyone else uses. IBM frames what those systems share: LLMs are built on a transformer architecture that excels at handling sequences of words and can understand and generate natural language across a wide range of tasks. Industry, not academia, now drives the field. Stanford HAI found that industry produced over 90% of notable AI models in 2025.
Nationally, the picture is concentrated too. The U.S. produced 59 notable models in 2025 to China’s 35, with the U.S. leading in notable model development, per the AI Index. Adoption tracks that lead: ChatGPT’s 800 million weekly active users by late 2025 make OpenAI the most visible name, though the gap at the capability frontier is narrow.
The competition reaches well beyond the chat box. For how that race shows up in the labour market, AI’s impact on the job market tracks the employment side.
| Builder | Flagship interface | Model family | Access model |
|---|---|---|---|
| OpenAI | ChatGPT | GPT series | Closed / API |
| Anthropic | Claude | Claude series | Closed / API |
| Gemini | Gemini, BERT, PaLM | Closed / API | |
| Meta | Llama | Llama series | Open weights |
| Microsoft | Copilot | Built on partner models | Closed / API |
Source: IBM Think, Stanford HAI AI Index 2026
The open-source side has scaled into a force of its own. Stanford HAI counted 5.6 million open-source AI projects on GitHub, with Hugging Face uploads tripling since 2023. That ecosystem is increasingly relevant to anyone choosing what to build on, and it intersects with newer categories like the AI agent adoption data tracking systems that act, not just answer.
The application layer built on top of these models is now its own ecosystem, from chat assistants to the AI tools woven into social platforms that most people already use without realising an LLM sits underneath.
Open vs Closed Models: Llama 3.1 and the Frontier Fork
Open-weight models are no longer a follower tier, and Meta’s Llama 3.1 is the clearest proof. Meta calls its 405-billion-parameter Llama 3.1 model the first frontier-level open-source AI model, expanding context length to 128,000 tokens and adding support across eight languages. The training run matched closed-lab scale: Meta trained the 405-billion-parameter model on over 15 trillion tokens and pushed the run to over 16 thousand H100 GPUs.
Parity is the part that reframes the market. Meta’s own evaluation states that its experimental evaluation suggests its flagship model is competitive with leading foundation models across a range of tasks, including GPT-4, GPT-4o, and Claude 3.5 Sonnet. An openly available model rivalling the best closed systems gives developers a genuine choice between control and convenience.
That choice carries security weight too, since self-hosted models change the threat surface. The AI-generated code vulnerability data shows why the deployment decision is not only about capability.
The Transparency Paradox: Bigger Models, Less Disclosure
The most capable models are now the hardest to inspect. Stanford HAI’s 2026 AI Index put it bluntly: industry produced over 90% of notable AI models in 2025, but the most capable models are now the least transparent. Detail that used to be standard has gone dark. Training code, parameter counts, dataset sizes, and training duration are no longer disclosed for several of the most resource-intensive systems, including those from OpenAI, Anthropic, and Google.
The irony is sharp. As models become more powerful and more embedded in daily decisions, the public has less ability to see how they were built. Open-weight releases push the other way, which is part of what makes the open versus closed split more than a licensing footnote. That tension over who can see inside these systems mirrors what happens inside companies, where employees increasingly adopt tools their own organisations cannot fully monitor.
What LLMs Can and Cannot Do: Limitations and Risks
LLMs are powerful and unreliable in specific, predictable ways. Hallucination is the most cited failure. With hallucinations, the model generates information that is false or misleading while sounding plausible, because it is predicting plausible text, not retrieving verified facts. They also inherit the flaws of their data: LLMs can reflect and amplify biases present in their training data.
Costs are physical as well as ethical. IBM notes that training and running LLMs requires large amounts of computational power and energy, raising both cost and environmental concerns. The numbers are stark: Stanford HAI estimated that Grok 4’s training emissions reached 72,816 tons of CO2 equivalent in 2025. None of this makes LLMs unusable, but it does mean human review stays essential for anything that matters.
Why it matters: Hallucination is structural, not a bug to be patched away. Because an LLM predicts the next token rather than looking up a fact, it can state something false with complete fluency. For high-stakes work, that makes verification a requirement rather than a nicety, which is why naming a source in writing like this still beats trusting a model’s unsourced confidence.
Are large language models the same as AI?
Large language models are one branch of artificial intelligence, not the whole field. AI covers everything from image recognition to recommendation systems, while an LLM specifically is a deep learning model built on a transformer that predicts the next word in a sequence. LLMs feel like the face of AI right now because they handle human language, but they sit inside a much larger discipline.
Who builds the biggest large language models?
Four names dominate the biggest models: OpenAI, Anthropic, Google, and Meta, with the industry overall producing over 90% of notable AI models in 2025. The U.S. leads on output with 59 notable models in 2025 against China’s 35. Among openly disclosed builders, Meta’s 405-billion-parameter Llama 3.1 is the first frontier-level open source AI model, the largest openly available option.
Conclusion
Large language models earned their 800 million weekly users by doing one thing relentlessly well: predicting the next token through a transformer trained on more text than any person could read in a thousand lifetimes. The mechanics are now stable, but the field is moving its pressure points. Parameter counts have flattened near 1 trillion while compute climbs 3.3x a year, open-weight models like Meta’s 405-billion-parameter Llama have closed the gap with closed labs, and the most capable systems have grown less transparent even as they grow more capable.
For readers, the practical takeaway is to treat these systems as fluent, fast, and fallible. They will keep improving on a roughly six-month cadence, and the open versus closed split will shape what developers and enterprises can actually build and inspect. Understanding tokens, attention, and training is no longer a specialist skill; it is the baseline literacy for working alongside the tools that now sit between people and most of their software.
