Your agent has 40+ tools and often picks the wrong one. What do you do?

Why it happens: each extra tool adds tokens and decision complexity. Similar tools (search_docs, search_kb, find_article) confuse the model. Accuracy tends to drop as the tool count grows.

Fixes, roughly in order of effort

1. Audit and consolidate: many tools are thin API wrappers. Merge get_user, get_user_orders and get_user_address into one get_customer_profile(include=[...]).
2. Better descriptions: explicitly distinguish similar tools ("Use X for ...; use Y for ..."). Add namespacing prefixes such as jira_search and github_search.
3. Dynamic tool loading / tool retrieval: embed tool descriptions and retrieve the top-k relevant tools per query, or route by intent first ("billing" intent → 6 billing tools).
4. Hierarchical / sub-agents: a supervisor chooses a domain agent; each domain agent has 5–10 tools.
5. Few-shot examples of correct tool choice for known confusing cases.
6. Model choice: some models are significantly better at tool use, so benchmark them.

Measure it

• Build a labelled set of queries mapped to the expected tool (and arguments).
• Track tool-selection accuracy, argument accuracy, and the number of wasted calls.
• Analyse the confusion matrix: which pairs of tools get mixed up shows where to fix descriptions.

Follow-ups to expect

• Risks of dynamic tool loading? If retrieval misses the right tool, the agent can't recover. Mitigate with an "ask for more tools" escape route or by always including core tools.

Related

• Previous: Q18. Your agent keeps calling the same tool with the same arguments in a loop. How do you debug and fix it?
• Next: Q20. How should an agent handle tool failures and retries?
• LangGraph
• DeepEval