API Tools
API tools wrap external APIs so a model can call real services, such as search, weather, or internal systems, as tools.
Prerequisites
What Are API Tools?
Tool calling is the general mechanism; API tools are the most common practical application of it — wrapping a real, existing API (a weather service, an internal database, a search engine) behind a tool schema so a model can call it like any other tool. The tool's implementation is often a thin layer that translates the model's structured call into an actual HTTP request, and translates the API's response back into something the model can read.
Model Tool Call
Structured RequestThe model calls the tool like any other tool.
API Tool Wrapper
Translation LayerTurns the model's call into a real HTTP request.
External API
Real ServiceA weather service, database, or search engine.
Response
Raw API ReplyThe external service's own response format.
Tool Result to Model
Readable ResultTranslated back into something the model can read.
What the Wrapper Is Responsible For
- Translating arguments — converting the model's structured tool call into whatever format the underlying API actually expects.
- Handling failures — a real external API can time out, rate-limit, or return an error; the wrapper needs to translate that into something the model can reasonably react to, rather than letting the whole request crash.
- Authentication — API keys and credentials for the underlying service live in the wrapper, never passed through the model or exposed in a prompt.
- Shaping the response — condensing a verbose or complex API response into something concise enough for the model to use effectively within its context window.
Warning
Never construct an API key or credential into a prompt — the wrapper should hold the credential and inject it directly into the real API call, entirely outside what the model sees.
Common Mistakes
Putting API credentials in a prompt or tool schema
Secrets should live in the wrapper implementation, never in anything the model reads or generates.
Returning the raw, full API response to the model
A verbose response can waste context window and bury the information that actually matters — the wrapper should shape it down to what's useful.
No timeout or error handling around the external call
A slow or failing external API directly affects the model's ability to respond — the wrapper needs its own reliability handling, the same as any other external dependency.
Exposing more of an API's capability than the task requires
Following least privilege, an API tool should only expose the specific operations an agent actually needs, not the API's full capability.
Interview Question
How would you design an API tool for an LLM, and what does the wrapper need to handle?
An API tool wraps a real external API behind a tool schema so a model can call it like any other tool — the wrapper translates the model's structured call into an actual API request, and shapes the response back into something the model can use. It needs to handle authentication itself, keeping credentials out of anything the model sees; handle failures like timeouts or rate limits gracefully, since a real external API is just another dependency that can fail; and condense the response down to what's actually useful, since returning a verbose raw response wastes context window. Following least privilege, I'd also only expose the specific operations a given agent actually needs, not the full capability of the underlying API.
What an interviewer may ask next
- Why should API credentials never appear in a prompt or tool schema?
- What happens if the underlying API is slow or fails, and how should the wrapper handle it?
- Why might you shape or condense an API response before returning it to the model?
Explain It in 30 Seconds
API tools wrap a real external API behind a tool schema so a model can call it like any other tool, with a thin wrapper translating the model's call into a real API request and shaping the response back for the model to use. The wrapper owns authentication, keeping credentials out of anything the model sees, handles failures from the external API gracefully, and condenses responses to avoid wasting context window — and should expose only the specific operations a task actually needs.