norrishuang opened a new pull request, #18069: URL: https://github.com/apache/dolphinscheduler/pull/18069
## Was this PR generated or assisted by AI? YES. The implementation was assisted by AI (Claude) for code generation, with human review, testing and verification on a real AWS EMR Serverless environment. ## Purpose of the pull request Add a new task plugin for Amazon EMR Serverless, enabling users to submit, monitor, and cancel Spark/Hive jobs on EMR Serverless applications directly from DolphinScheduler workflows. Unlike the existing EMR on EC2 task plugin which manages EC2-based clusters, EMR Serverless is a serverless runtime that requires no cluster infrastructure management and automatically scales compute resources on demand. ## Brief change log **Backend (new module: `dolphinscheduler-task-emr-serverless`)** - `EmrServerlessTask` — extends `AbstractRemoteTask`, implements submit/track/cancel lifecycle via AWS SDK v1 (`StartJobRun`, `GetJobRun`, `CancelJobRun`) - `EmrServerlessParameters` — task parameter model (applicationId, executionRoleArn, jobName, startJobRunRequestJson) - `EmrServerlessTaskChannel` / `EmrServerlessTaskChannelFactory` — SPI registration via `@AutoService`, registered as `EMR_SERVERLESS` - `EmrServerlessTaskException` — dedicated exception class - Authentication: reuses `aws.emr.*` config from `aws.yaml`, falls back to `DefaultAWSCredentialsProviderChain` - Supports failover recovery via `appIds` (jobRunId) **Frontend** - `use-emr-serverless.ts` (fields) — form fields for Application Id, Execution Role Arn, Job Name, StartJobRunRequest JSON editor - `use-emr-serverless.ts` (tasks) — task model definition - Registered in task type constants, store, format-data, i18n (en_US/zh_CN) - Task icon (reuses EMR icon) **Documentation** - Chinese doc: `docs/docs/zh/guide/task/emr-serverless.md` - English doc: `docs/docs/en/guide/task/emr-serverless.md` - Includes: overview, task parameters, Spark/Hive JSON examples, AWS auth config, job state transitions, screenshots ## Verify this pull request This change added tests and can be verified as follows: - Added `EmrServerlessTaskTest` with 11 unit tests covering: success/failed/cancelled lifecycle, full state chain, submit error handling, null GetJobRun response, cancel with/without jobRunId, failover recovery, parameter validation, and invalid JSON handling. - Manually verified by deploying to an EC2 instance in Standalone mode and successfully submitting a Spark job to a real AWS EMR Serverless application. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
