What is machine learning and how is it different from traditional programming?

What is deep learning, and how does it relate to machine learning?

How would you explain the difference between AI, machine learning, and deep learning?

What is training, and why is it computationally expensive?

What is inference, and why does it matter as an engineering concern rather than just a machine learning concept?

What is a model, and how is it different from training or inference?

What are parameters, and why does parameter count matter when choosing a model?

What is a dataset, and why is it split into training, validation, and test sets?

What is a neural network, and how does stacking layers give it the ability to represent complex patterns?

What is a foundation model, and why did it change how AI applications get built?

What is an LLM, and how does it generate text?

What is fine-tuning, and when would you use it instead of prompting or RAG?

What is a token, and why does tokenization matter?

What is a context window, and why does it matter when building with LLMs?

What does temperature control in an LLM, and when would you change it?

What is hallucination in the context of LLMs, and how would you reduce it in an application?

What is multimodal AI, and how does a model handle different data types like text and images together?

How does attention work, conceptually?

What is self-attention, and how does it help a model resolve something like pronoun ambiguity?

What does an encoder do in a transformer, and how does it differ from a decoder?

How does a decoder generate a response, and why does that make streaming possible?

Why do transformers need positional encoding, and what problem does it solve?

What is a system prompt, and what should and shouldn't go in one?

What is a user prompt, and why does it occupy a different trust level than the system prompt?

What is zero-shot prompting, and when does it tend to fall short?

What is few-shot prompting, and how would you choose good examples for it?

What is chain-of-thought prompting, and why does it sometimes improve results?

What is a prompt template, and why use one instead of building prompts ad hoc in code?

What is structured output, and why would you use it over free-form text?

What is prompt injection, and how would you defend a RAG or agent system against it?

What is a vector, and how does it relate to an embedding?

What is semantic similarity, and how is cosine similarity used to measure it?

What is vector search, and how is it different from keyword search?

What is a vector database, and how is it different from a traditional database?

How would you choose chunk size for a RAG system?

What is retrieval, and how is it different from generation?

What is query transformation, and why might a raw user query retrieve poorly?

Why would you add a reranking step to a RAG system?

What is metadata filtering in a RAG system, and why is it important for access control?

What is hybrid search, and why would you combine vector and keyword search instead of using just one?

How would you evaluate whether a RAG system is actually working well?

What is tool calling, and how does it relate to function calling?

What is a tool schema, and why does its quality directly affect how reliably a model uses tools?

How would you design an API tool for an LLM, and what does the wrapper need to handle?

What is agentic AI, and how does it differ from a simple LLM application?

What is the agent loop, and why does an agent need one instead of a single model call?

What is planning in an AI agent, and how does it differ from the agent loop?

What is tool use, and how does an agent decide which tool to select?

What is the difference between a context window and memory?

What is reflection in an agent system, and what are its limits?

What is human-in-the-loop, and when would you require it for an agent?

When would you use a multi-agent system instead of a single agent, and what does it cost you?

What is agent orchestration, and when do you actually need it?

What does an MCP client do, and how does it change how an agent connects to new tool providers?

What is an MCP server, and what is it responsible for?

What are MCP tools, and how does their discovery differ from tools defined directly in an agent's own configuration?

What is an MCP resource, and how is it different from an MCP tool?

What are the core responsibilities of an LLM gateway, and how do you implement retries and fallback?

What is streaming, and what does it actually improve compared to a standard request/response call?

What can you cache in an LLM application, and what's the risk with semantic caching specifically?

How would you design rate limiting for an AI application that calls an external model provider?

How would you design observability for a production RAG or agent system?

How would you build an evaluation process for an AI system, and why is it necessary?

What are guardrails, and why can't you rely on the model alone to stay within acceptable bounds?

How would you reduce the cost of running an AI application without significantly hurting quality?

What engineering controls would you actually implement to secure an AI application that uses tools and retrieval?

How would you design a production RAG system for an enterprise knowledge base?

How would you architect a production agent system, and what would you put in place to keep it safe and bounded?

How would you design an AI Gateway, and when would you actually introduce one?

How would you design tenant isolation in a multi-tenant RAG or AI system?

How would you scale an AI application as request volume grows?

How would you design the security architecture for a system where an LLM can read retrieved documents and call tools?

How would you design a data architecture that supports RAG and other AI features across an organization?

Why shouldn’t a model provider’s API key ever be called directly from the browser?

Why is Node.js often used for the backend layer of an AI application?

If your main application is built in Java or Node, why might you still use Python for the AI layer?

How is designing an API for an LLM endpoint different from a typical CRUD REST API?

Why do most AI chat interfaces show a “searching” or “thinking” indicator instead of a plain loading spinner?

How would you handle a conversation that grows longer than the model’s context window?

Why does authentication matter more for an AI feature than for a typical CRUD feature?

Why would you use Spark instead of just calling an embedding API in a loop?

How does preparing data for RAG differ from preparing data for a traditional data warehouse?

When would you have a model query Snowflake directly instead of relying on RAG over documents?

What role does a platform like Databricks play in an AI system, compared to the model provider itself?

When would you use pgvector instead of a dedicated vector database like Pinecone?

Why can’t you just chunk a PDF directly without a separate extraction step?

What factors would actually drive your choice of vector database for a new RAG project?

What’s the difference between a RAG demo and a production RAG pipeline?

Why isn’t similarity score alone enough to decide what a RAG system retrieves?

Why can’t you just average a vector similarity score and a keyword relevance score to combine them?

Why might an e-commerce search need more than plain vector similarity to work well?

What happens to a RAG system’s answers if ingestion silently stops running?

Why does an agent need explicit, persisted state rather than just passing messages along?

How would you decide what an agent should actually remember across sessions?

What real problem does a multi-agent orchestration tool solve that a single, larger agent doesn’t?

What happens if no one responds to an agent’s approval request?

Why is evaluating an agent harder than evaluating a single prompt?

An agent produces a wrong answer in production. How would you debug it?

Why is retrying a failed step in an AI workflow riskier than retrying a normal application step?

Why might an enterprise choose Azure OpenAI Service over calling OpenAI’s API directly?

What advantage does Vertex AI offer beyond just calling the Gemini API directly?

Why doesn’t standard CPU-based autoscaling work well for a self-hosted model on Kubernetes?

How would you deploy a scalable AI service on EKS?

Why does request batching matter for GPU inference specifically?

What would you actually lose by writing your own provider-switching logic instead of using a tool like LiteLLM?

You’ve autoscaled your API layer to handle 10x traffic, but response times are still degrading. What’s the likely bottleneck?

Why isn’t trying a new prompt a few times and reading the outputs a sufficient way to validate it?

A model that passed evaluation at launch is producing worse answers three months later. What could explain that?

Why do teams use a dedicated LLM observability tool instead of their existing application logging?

Your model provider bill doubled this month. How would you figure out why?

How would you design an AI feature to stay available during a model provider outage?

A prompt change caused a regression in production. How does versioning help you respond?

Why can’t you use a standard unit-testing approach — exact output matching — for a GenAI CI pipeline?

How is a jailbreak different from a prompt injection attack?

Why not simply store embeddings in PostgreSQL without any additional access controls?

What’s the single biggest security risk specific to giving an LLM agent tool access?

Where in an AI pipeline can personal data leak, beyond the obvious risk of sending it to a model provider?

Why does an internal service calling your AI gateway need its own identity, separate from the end user’s?

What’s different about supply chain security for an AI system compared to a typical software supply chain?

Why does red teaming need to be an ongoing practice rather than a one-time pre-launch exercise?

What does “AI governance” actually mean in an enterprise setting, beyond general security practices?

How is responsible AI different from AI security?

Why is it better to design around a "model family and tier" rather than a specific model version string?

When would you choose pgvector over a purpose-built vector database like Pinecone?

What problem does an AI gateway solve that a direct API integration with one provider doesn’t?

Why do LLM applications often need AI-specific observability tools instead of relying only on general application monitoring?

How would you describe the layers of a real-world AI system to someone who only knows "call the model API"?