The 2017 Transformer paper provided the blueprint, but BERT and GPT were the two different “philosophies” that realized its true potential. Their rivalry started the Scale War, moving AI from small, specialized tools to massive, general-purpose foundation models.
1. The Two Philosophies
After the Transformer was released, researchers split into two camps based on how they used the architecture:
The Encoder Camp (BERT: The Reader)
Google released BERT (Bidirectional Encoder Representations from Transformers) in 2018.
- The Strategy: Use only the Encoder half of the Transformer. It looks at the whole sentence at once (bidirectional).
- The Goal: Perfection in understanding. It was trained to fill in the blanks (Masked Language Modeling), like a student taking a multiple-choice test.
- The Result: It crushed benchmarks for search, sentiment analysis, and question-answering.
The Decoder Camp (GPT: The Writer)
OpenAI released GPT-1 (Generative Pre-trained Transformer) also in 2018.
- The Strategy: Use only the Decoder half. It reads text left-to-right, one word at a time (unidirectional).
- The Goal: Mastery of generation. It was trained to predict the next word in a sequence.
- The Result: While GPT-1 was initially seen as less “smart” than BERT at understanding, it had a superpower: it could write coherent text.
2. GPT-2 and the “Weaponization” of Scale (2019)
OpenAI realized that the Decoder architecture had a unique property: it scaled predictably. If you gave it 10x more data and 10x more parameters, it didn’t just get better at predicting the next word; it started “learning” tasks it wasn’t trained for.
- The Jump: GPT-1 had 117 million parameters. GPT-2 had 1.5 billion.
- The Controversy: OpenAI initially refused to release the full GPT-2 model, famously calling it “too dangerous” because it could generate fake news that looked human-written. This was the moment the world realized scale could create emergent intelligence.
3. GPT-3 and the Birth of the “LLM” (2020)
While Google focused on making BERT-style models more efficient (RoBERTa, ALBERT), OpenAI went for “Brute Force.”
- The Scale: GPT-3 was a massive leap to 175 billion parameters.
- Few-Shot Learning: This model was so big that you didn’t need to “fine-tune” it for specific tasks anymore. You could just talk to it. By giving it two examples of a translation, it “figured out” it should be a translator.
- The Impact: This effectively ended the era of “Task-Specific AI” and began the era of “General Purpose Foundation Models.”
4. The Modern Arms Race (2023–2026)
The war moved from “Who has the most parameters?” to “Who has the best data and reasoning?”
| Milestone | Model | Significance |
| 2023 | GPT-4 | Estimated at 1.7 trillion parameters (using MoE). It added vision and advanced logic. |
| 2024 | Llama 3 / Gemini 1.5 | Proved that smaller models trained on better data could beat older, larger ones. |
| 2025-26 | GPT-5 / Gemini 3 | The focus shifted to Reasoning Models (like OpenAI o1/o3) that “think” before they speak, solving the “Scale War” with “Compute-at-Inference.” |
Summary: The Scaling Law
The Scale War was driven by the discovery of Scaling Laws: the mathematical proof that model performance improves linearly with more compute, more data, and more parameters.
Current State: As of 2026, the industry is moving away from just “bigger is better” and toward Mixture of Experts (MoE)—where a model is massive but only uses a fraction of its “brain” for any specific query—and Agentic AI, where the model doesn’t just talk, but executes tasks.