Neural Network
A neural network is a layered system of connected nodes that learns to transform inputs into outputs.
What Is a Neural Network?
A neural network is a model made of layers of simple connected units, loosely inspired by neurons in a brain. Each unit takes in numbers, applies a simple mathematical transformation, and passes the result to the next layer. No single unit does anything complex — the network's ability to represent complex patterns comes from stacking many of these simple transformations together.
Input Layer
Raw NumbersThe starting values fed into the network.
Hidden Layer
Simple TransformNo single unit does anything complex on its own.
Hidden Layer
Stacked FurtherComplexity comes from stacking these together.
Output Layer
Final ResultThe transformed numbers, ready to be used.
Key Idea
Deep learning gets its name from networks with many stacked layers — "deep" refers to the number of layers, not the sophistication of any single one.
Why Layers Matter
- Each layer learns to represent the input at a different level of abstraction — earlier layers might capture simple patterns, later layers combine those into more complex ones.
- The connections between units are exactly the parameters a neural network learns during training.
- Activation functions between layers introduce non-linearity — without them, stacking layers wouldn't add any representational power beyond a single layer.
- Modern LLMs are built from a specific kind of neural network — the transformer — which stacks attention and feed-forward layers in a particular arrangement.
Common Mistakes
Assuming "neural network" means the model reasons like a brain
The biological inspiration is loose — a neural network is a mathematical function made of layered transformations, not a simulation of biological cognition.
Assuming more layers is always better
Very deep networks can be harder to train effectively and don't automatically outperform a well-designed shallower one for a given task.
Treating "neural network" and "deep learning" as unrelated terms
Deep learning specifically refers to neural networks with many layers — the terms are closely connected, not separate topics.
Interview Question
What is a neural network, and how does stacking layers give it the ability to represent complex patterns?
A neural network is a model made of layers of simple connected units — each one takes in numbers, applies a simple transformation, and passes the result forward. No single unit is complex; the network's power comes from stacking many of these simple transformations, with each layer able to represent the input at a different level of abstraction, and activation functions between layers adding the non-linearity that makes stacking actually useful rather than mathematically equivalent to one layer. Deep learning refers specifically to neural networks with many stacked layers, and modern LLMs are built from a specific architecture — the transformer — which arranges attention and feed-forward layers in a particular way.
What an interviewer may ask next
- Why do activation functions matter between layers?
- Is a neural network actually a simulation of how the brain works?
- How does "deep learning" relate to neural networks specifically?
Explain It in 30 Seconds
A neural network is a model made of layers of simple connected units, each applying a basic transformation and passing the result forward — its ability to represent complex patterns comes from stacking many of these simple layers together, not from any single unit being sophisticated. The connections between units are the parameters learned during training. Deep learning refers to neural networks with many stacked layers, and transformers — the architecture behind modern LLMs — are a specific kind of neural network.