Parameters
Parameters are the internal values a model learns during training that determine its behavior.
What Are Parameters?
Parameters — often called weights — are the internal numbers a model adjusts during training. There's nothing more to a trained model than these numbers arranged in a specific structure: change them and you change everything about how the model behaves, without changing a single line of the code that runs it.
Key Idea
When you hear a model described by its parameter count — a "7 billion parameter model" — that number is a rough proxy for how much capacity the model has to represent complex patterns.
Why Parameter Count Matters
- More parameters generally mean more capacity to represent complex patterns — but also more compute, memory, and cost to train and run.
- Parameter count is one factor in choosing a model, alongside latency, cost, and how well it fits the actual task.
- Quantization reduces the memory a model's parameters take up by storing them with less numeric precision, trading a small amount of accuracy for a large efficiency gain — this is why the same parameter count can run on very different hardware depending on how it's quantized.
- Fine-tuning updates some or all of a model's existing parameters on new data — it doesn't add new ones, it adjusts what's already there.
Common Mistakes
Assuming more parameters always means a better model
Parameter count is one factor among training data quality, task fit, latency, and cost — a smaller, well-suited model can outperform a larger, poorly-matched one for a given task.
Treating parameter count as the only thing that matters for cost
Context length, output length, and how a model is served (including quantization) also significantly affect cost and speed.
Confusing parameters with hyperparameters
Parameters are learned automatically during training; hyperparameters (like learning rate) are settings chosen before training begins.
Interview Question
What are parameters, and why does parameter count matter when choosing a model?
Parameters, often called weights, are the internal numbers a model adjusts during training — they're the entire substance of a trained model, and changing them changes the model's behavior without changing any code. Parameter count is a rough proxy for how much capacity a model has to represent complex patterns, so it matters when choosing between models, but it's one factor among several — training data quality, task fit, latency, and cost all matter too. A larger model isn't automatically the right choice; a smaller, well-suited model can be a better engineering decision for a simple task.
What an interviewer may ask next
- What's the difference between parameters and hyperparameters?
- Why might a model with fewer parameters be the better choice for a task?
- How does quantization relate to a model's parameters?
Explain It in 30 Seconds
Parameters, or weights, are the internal numbers a model learns during training — they're the entire substance of a trained model, determining its behavior without any change to code. Parameter count is a rough proxy for a model's capacity to represent complex patterns, but it's just one factor in choosing a model alongside cost, latency, and task fit. Fine-tuning adjusts existing parameters on new data; it doesn't add new ones.