Introduction
Coding agents excel at writing code using popular libraries on which large language models have been extensively trained. However, when faced with custom libraries, new framework versions, internal APIs, or niche technologies, their performance drops significantly. This poses a substantial challenge for development teams working with specialized libraries or enterprise code.
The Challenge of Custom Libraries
Claude Code and other programming agents show evident limitations when interfacing with domain-specific documentation or proprietary frameworks. The lack of adequate context leads to poor code quality, incorrect implementations, and violations of domain-specific best practices.
Tested Claude Code Configurations
The LangGraph team experimented with four different configurations to improve Claude Code performance:
- Claude Vanilla: Standard configuration without modifications
- Claude + MCP: Integration with MCPDoc server for documentation access
- Claude + Claude.md: Detailed file with LangGraph-specific guidelines
- Claude + MCP + Claude.md: Combination of both approaches
MCP Server for Documentation
The MCPDoc server is an open-source tool that exposes two main functions: list_doc_sources and fetch_docs. This system allows agents to dynamically access library documentation, providing specific context when needed.
The Claude.md file implementation includes detailed instructions for common LangGraph project structural requirements, proper export patterns, and deployment best practices. It contains code examples for essential primitives like create_react_agent, supervisor patterns, and swarm patterns for dynamic handoffs.
Evaluation Framework
The evaluation is based on three main categories to measure both functionality and code quality:
Smoke Tests
Verify basic functionality: code compilation, .invoke() method exposure, input state handling, and correct output structure return.
Task-Specific Requirement Tests
Validate task-specific functionality, including deployment configuration, HTTP requests to external APIs, and unit tests specific to each programming task.
Code Quality & Implementation Evaluation
Uses LLM-as-a-Judge to capture aspects that binary tests miss. Includes objective checks and subjective assessments based on expert-written code as reference.
Results and Performance
Results show that Claude + Claude.md outperformed Claude + MCP, despite Claude.md containing only a subset of information available through the MCP server. Trace analysis reveals that Claude didn't invoke MCP tools as much as expected, often stopping at the main page without exploring necessary details.
"High quality, condensed information combined with tools to access more details as needed produced the best results."
LangGraph Team
The Claude + Claude.md + MCP configuration used documentation more effectively, calling MCP tools more frequently and even triggering web search when required.
Key Takeaways and Best Practices
The experiment highlights the importance of a balanced approach in configuring programming agents. Context overload can reduce performance, while well-structured and condensed instructions significantly improve results.
Context Overload Prevention
Dumping large llms.txt files from documentation can crowd the context window, leading to poor performance and higher costs. Smart retrieval tooling that pulls only relevant snippets is worth implementing for extensive documentation.
Claude.md Optimization
A well-crafted Claude.md file should highlight core concepts, unique functionality, and common primitives in your library. Review failed runs manually to find recurring pitfalls and add guidance for them.
Conclusion
For teams developing with custom libraries, the combination of a detailed guide (Claude.md) with selective documentation access represents the most effective approach to achieve superior code quality. While Claude.md provides the highest return on investment, the strongest results come from pairing it with an MCP server for comprehensive coverage.
FAQ
How to configure Claude Code for custom libraries?
Start by creating a Claude.md file with specific guidelines for your library, including common patterns and anti-patterns to avoid.
What's the difference between Claude.md and llms.txt?
Claude.md contains condensed information and specific instructions, while llms.txt is a plain text file with all page content.
Does MCP server improve Claude Code performance?
MCP server alone improves performance by about 10 percentage points, but it's most effective when combined with Claude.md.
What are the implementation costs of custom Claude Code?
Claude + Claude.md is approximately 2.5 times cheaper than MCP configurations, offering the best cost-performance ratio.
How to evaluate AI-generated code quality?
Use a three-tier framework: smoke tests for basic functionality, specific tests for task requirements, and LLM-as-a-Judge for code quality.
What are common errors in programming agents?
Incorrect interrupt() usage, wrong state update patterns, type assumption errors, and overly complex implementations.