GitHub user Hui-of-limin created a discussion: Dolphin MCP Pilot: Operate DolphinScheduler via AI Agents
# Dolphin MCP Pilot: Operate DolphinScheduler via AI Agents\n\nHi DolphinScheduler community ๐\n\nWe've built **Dolphin MCP Pilot**, an MCP (Model Context Protocol) server that lets AI agents directly operate DolphinScheduler through natural language โ from creating workflows and managing schedules to troubleshooting failures and performing backfills.\n\n## What is it?\n\nAn integration layer that exposes **58 DolphinScheduler operations** as standardized MCP tools, enabling AI agents (Claude Desktop, Cursor, Cline, custom agents) to:\n\n- Create SQL/DAG workflows with a single sentence\n- Manage schedule lifecycle (create โ online โ offline โ delete)\n- Control process instances (pause / resume / rerun / rerun-from-failure)\n- Intervene at task level (force success / skip failed nodes)\n- Query logs and troubleshoot failures with guided `next_action` prompts\n- Perform serial backfills with ordering guarantees (`complementStartDate`/`complementEndDate`)\n- Manage resource s and roll back workflow versions\n\n## Why this matters\n\nTraditional DolphinScheduler interaction requires:\n1. Opening the web UI\n2. Navigating multiple pages\n3. Manually clicking through operations\n4. Checking logs and states separately\n\nWith Dolphin MCP Pilot, you can tell your AI agent:\n\n> \"The ETL job failed yesterday. Why?\"\n\nThe agent automatically:\n1. Calls `ds_list_process_instances` โ finds the failed instance\n2. Calls `ds_list_task_instances` โ locates the failed task node\n3. Calls `ds_get_task_log` โ pulls the error log\n4. Reports the root cause\n\nNo manual clicking, no page-hopping.\n\n## Architecture\n\n- **Protocol**: MCP over HTTP + SSE\n- **Authentication**: Dual-mode (API Token or Username+Password)\n- **Compatibility**: DolphinScheduler 3.x (tested on 3.2.1)\n- **Deployment**: Docker / bare Python\n\n## Tool Coverage\n\n| Category | Tools | Examples |\n|---|---|---|\n| Project | 5 | `ds_create_project`, `ds_list_projects` |\n| Workflow | 14 | `ds_create_workflow`, `ds_update_task_param`, `ds_clone_workflow` |\n| Schedule | 6 | `ds_set_schedule`, `ds_online_schedule`, `ds_complement_data` |\n| Instance | 13 | `ds_rerun_process_instance`, `ds_force_task_success` |\n| Resource | 10 | `ds_list_resources`, `ds_update_resource_content` |\n| Monitoring | 6 | `ds_get_task_log`, `ds_list_task_instances` |\n| Raw API | 4 | `ds_raw_get`, `ds_raw_post`, `ds_raw_put`, `ds_raw_delete` |\n\n## Quick Start\n\n```bash\ngit clone https://github.com/iflytek/dolphin-mcp-pilot.git\ncd dolphin-mcp-pilot\n\n# Configure\ncp .env.example .env\n# Edit .env: set DS_URL, DS_TOKEN (or DS_USER/DS_PASSWORD)\n\n# Run\ndocker compose --profile dev up -d\n```\n\nConnect your AI agent (example: Claude Desktop):\n\n```json\n{\n \"mcpServers\": {\n \"dolphinscheduler\": {\n \"url\": \"http://localhost:8001/mcp/\",\n \"headers\": {\n \"X-DS-Token\": \"your-token-here\"\n }\n }\n }\n}\n```\n\nNow you can talk to your agent:\n\ n> \"Create a daily SQL workflow in the data-team project that runs at 2 AM, querying `SELECT * FROM user_behavior WHERE dt = '${bizdate}'`, with 3 retries on failure.\"\n\nThe agent orchestrates `ds_create_workflow` โ `ds_set_schedule` โ `ds_online_schedule` automatically.\n\n## Real-World Use Cases\n\n**Scenario 1**: Automated troubleshooting\n- Agent detects a failed workflow, traces the specific task node, pulls logs, and suggests fixes\n\n**Scenario 2**: Bulk backfills\n- \"Rerun all daily reports from July 1 to July 31\" โ agent calls `ds_complement_data` with serial ordering\n\n**Scenario 3**: Version control\n- \"Roll back user_analysis workflow to the previous version\" โ agent calls `ds_rollback_workflow_version`\n\n## Roadmap\n\n- DolphinScheduler 2.x compatibility\n- Pre-built DAG templates (CDC, feature engineering, report distribution)\n- Multi-agent orchestration (scheduling agent โ data quality agent โ alerting agent)\n\n## Links\n\n- **GitHub**: https:// github.com/iflytek/dolphin-mcp-pilot\n- **License**: Apache-2.0\n- **Docs**: [Installation](https://github.com/iflytek/dolphin-mcp-pilot/blob/main/docs/INSTALLATION.md) ยท [Configuration](https://github.com/iflytek/dolphin-mcp-pilot/blob/main/docs/CONFIGURATION.md) ยท [Features](https://github.com/iflytek/dolphin-mcp-pilot/blob/main/docs/FEATURES.md) ยท [API Reference](https://github.com/iflytek/dolphin-mcp-pilot/blob/main/docs/API.md)\n\n---\n\nWe'd love to hear feedback from the community, especially on:\n- API compatibility issues with different DS versions\n- Authentication patterns in production environments\n- Feature requests for operations not yet covered\n\nThis is an open-source project under Apache-2.0. Contributions, issues, and discussions are welcome!\n GitHub link: https://github.com/apache/dolphinscheduler/discussions/18547 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
