How does tool calling (function calling) actually work under the hood?

Step by step

1. Definition. You describe each tool with a name, a natural-language description, and a JSON Schema for its parameters. These are injected into the model's context, often in a special format the model was trained on.
2. Decision. The model was fine-tuned to emit a special structured block when it decides a tool is useful. Many APIs return this as a separate tool_calls field with a stop_reason such as "tool_use".
3. Execution. Your application validates the arguments, runs the real function, and handles errors.
4. Result. You append the tool result to the conversation, linked to the call by an ID, and call the model again.
5. Loop. The model either calls more tools or produces a final answer.

Key points interviewers look for

• Security boundary. Since your code executes the tool, you enforce permissions, validation and rate limits. Never trust model-generated arguments blindly.
• The description is a prompt. Tool descriptions strongly affect which tool gets chosen and when.
• Parallel calls. Most APIs let the model request several tools in one turn (see Q30).
• Forcing. tool_choice can force a specific tool, require any tool, or disable tools.

Common mistakes

• Saying "the LLM calls the API." It only requests the call.
• Forgetting to validate arguments against the schema before execution.

Follow-ups to expect

• What happens if the model returns invalid JSON arguments?
• How do you handle a tool that takes 2 minutes to run?

Related

• Previous: Q2. Explain the ReAct pattern. Why did it matter?
• Next: Q4. Workflow vs agent: when would you NOT use an agent?
• LangChain
• LLM Fundamentals