DevDocs MCP Server (CyberAGI)
Author: cyberagiinc
Description: DevDocs is a free, self-hosted documentation crawling + indexing platform that exposes crawled docs through an MCP server for LLM/agent tooling (Claude Desktop, Cursor, Windsurf, Cline/Roo Code). It includes a UI frontend, a backend API, an MCP service, and a Crawl4AI-based crawler.
Stars: 2.0k
Forks: 184
License: Apache License 2.0
Category: Open Source
Overview
Installation
# Clone the repository
git clone https://github.com/cyberagiinc/DevDocs.git
# Navigate to the project directory
cd DevDocs
# Configure environment variables
cp .env.template .env
# Ensure NEXT_PUBLIC_BACKEND_URL in .env is set correctly (e.g., http://localhost:24125)
# Start all services using Docker
./docker-start.sh# Clone the repository
git clone https://github.com/cyberagiinc/DevDocs.git
# Navigate to the project directory
cd DevDocs
# Configure environment variables
copy .env.template .env
# Ensure NEXT_PUBLIC_BACKEND_URL in .env is set correctly (e.g., http://localhost:24125)
# Start all services using Docker
docker-start.batdocker logs devdocs-frontend
docker logs devdocs-backend
docker logs devdocs-mcp
docker logs devdocs-crawl4ai
docker logs -f devdocs-backendTable Of Contents Tool
Returns a full or filtered list of documentation topics (documentation index/TOC discovery).
Section Access Tool
Retrieves the detailed content of specific documentation sections (targeted section drill-down).
FAQs
How do I configure DevDocs MCP Server to crawl internal documentation behind a firewall or VPN?
DevDocs runs as a local Docker Compose stack, so it inherits your host machine's network access. If your host can reach internal documentation URLs through VPN or firewall rules, DevDocs accesses those same URLs without additional configuration. Ensure your VPN connection is active before starting the Docker containers. For firewall-restricted environments, allow outbound traffic from the Docker bridge network to your internal documentation hosts. If documentation lives on localhost, use `host.docker.internal` as the target URL in the DevDocs UI to route requests from containers back to the host machine. Test connectivity by curling your internal documentation URL from inside the running devdocs-mcp container using `docker exec` before configuring crawls.
What are the system requirements and resource usage for running DevDocs MCP Server via Docker Compose?
Allocate at least **4GB RAM** and **2 CPU cores** for the three-container stack (Next.js frontend, FastAPI backend, Crawl4AI crawler). Actual usage scales with crawl depth and documentation size. Windows users should expect an additional 2GB RAM baseline from WSL 2 and Docker Desktop. Storage depends on crawl volume; depth-5 crawls with multi-format exports can consume several gigabytes per project. Performance bottlenecks typically appear during active crawls rather than idle retrieval. For production deployments with concurrent crawls, use at least **8GB RAM** and **SSD storage** to avoid I/O constraints.
What are the limitations of DevDocs MCP Server's crawl depth settings and how do I choose the right depth level?
DevDocs MCP Server offers configurable crawl depth from **1–5 levels**: - **Depth 1–2:** Best for flat API reference sites. Risks missing nested guides. - **Depth 3:** Practical default for most documentation sites. - **Depth 4–5:** Necessary for sprawling enterprise docs, but introduces longer crawl times and potential noise from irrelevant pages. DevDocs crawls uniformly rather than adaptively stopping at natural content boundaries, so deeper crawls may retrieve changelog entries or footer links alongside core documentation. Start with depth 2, inspect the crawled Table of Contents output, then increase depth incrementally if critical sections are missing.
What are the most common issues faced when setting up an MCP server?
The most common MCP server setup issues stem from configuration errors, dependency conflicts, and environment mismatches: - **Configuration file syntax errors:** JSON formatting mistakes in the `mcpServers` block (missing commas, incorrect quotes, malformed arrays). - **Dependency resolution failures:** Node.js, Python, or Docker version conflicts causing silent initialization failures. - **Path resolution issues:** Relative paths in `command` or `cwd` fields breaking because the MCP client executes from a different working directory. - **Environment variable propagation failures:** API keys or config values that work in terminal sessions but remain undefined when the MCP client spawns the server process. - **Permission and port conflicts:** Restricted user accounts lacking Docker socket access, or multiple services claiming the same local port. These issues persist because MCP clients often suppress detailed stderr output and lack unified debugging interfaces across Claude Desktop, Cursor, and VS Code.
How can I optimize the performance of an MCP server?
- **Prefer Streamable HTTP over Server-Sent Events** for more reliable communication with reduced latency. - **Implement JWT or Client Credentials authentication** to prevent performance degradation from unauthorized traffic. - **Apply caching hierarchies and connection pooling** to reduce repeated work, and validate tool connections early to catch configuration failures before deployment. - **Build servers in performant languages** like C# to minimize request-handling overhead. - **Profile custom servers under load** and scale horizontally based on observed metrics to handle high-volume agent traffic. - **Connect directly to real data sources** rather than relying on general knowledge lookups to lower computational demands.
What are the best practices for securing an MCP server?
Securing an MCP server requires layered controls across authentication, network isolation, and runtime safety: - **Authentication:** Enforce API keys or short-lived OAuth tokens with role-based access control. Use mutual TLS for HTTP/SSE transports. - **Network isolation:** Bind servers to localhost for local-only access. Deploy HTTPS with TLS 1.3 for remote connections. Apply rate limiting. - **Input validation:** Enforce strict JSON schema checks and sanitize tool parameters. Sandbox tool executions using Firejail, gVisor, or Docker with security options. - **Secrets management:** Never hardcode secrets; use vaults like **AWS Secrets Manager** or **HashiCorp Vault**. - **Monitoring:** Implement structured logging of all requests and enable real-time anomaly detection through Prometheus or Datadog. - **Supply chain security:** Pin dependencies with lockfiles, automate vulnerability scanning, and sign container images.