Generative AI
Foundation models, LLMs, transformers, tokens, prompting, and fine-tuning.
0 of 24 available lessons completed
Learning sequence
Generative AI refers to models that create new content such as text, images, audio, or code rather than just classifying existing data.
Foundation models are large models pretrained on broad data that can be adapted to many downstream tasks.
The attention-based architecture behind nearly every modern LLM.
Attention lets a model weigh how relevant each part of the input is when producing each part of the output.
Self-attention lets each position in a sequence weigh every other position in the same sequence to build context-aware representations.
An encoder reads an input sequence and produces a contextual representation of it.
A decoder generates an output sequence, typically one token at a time, using the encoded context and what it has generated so far.
Positional encoding injects information about token order into a transformer, which otherwise has no built-in sense of sequence.
A token is the small unit of text — a word piece, character, or symbol — that a language model reads and generates.
The context window is the maximum amount of text a model can consider at once when generating a response.
Temperature controls how random or deterministic a model's output is during generation.
Hallucination is when a model generates confident, fluent output that is factually incorrect or unsupported.
A prompt is the input text given to a language model to guide what it generates.
A system prompt sets persistent instructions and behavior for a model before the user's messages begin.
A user prompt is the specific request or question a user sends to the model within a conversation.
Zero-shot prompting asks a model to perform a task with instructions alone and no examples.
Few-shot prompting includes a small number of examples in the prompt to show the model the desired pattern.
Chain-of-thought prompting asks a model to reason step by step before giving a final answer.
Prompt templates are reusable prompt structures with variable placeholders filled in at runtime.
Structured output constrains a model to return data in a defined format, such as JSON, instead of free-form text.
Prompt injection is an attack where untrusted text manipulates a model into ignoring its original instructions.
Fine-tuning further trains a pretrained model on a smaller, specific dataset to adapt its behavior for a particular task.
Multimodal AI models process and generate more than one type of data, such as text, images, and audio together.