jerryshao commented on code in PR #13950: URL: https://github.com/apache/gravitino/pull/13950#discussion_r4409509602
########## docs/k8s-job-executor.md: ########## @@ -0,0 +1,825 @@ +--- +title: "Kubernetes Job Executor" +slug: "/k8s-job-executor" +keyword: "job executor, Kubernetes, k8s, Spark operator, SparkApplication, YuniKorn, Gravitino" +license: "This software is licensed under the Apache License version 2." +--- + +## Introduction + +The `k8s` job executor runs Spark jobs on Kubernetes. Each job becomes a `SparkApplication` resource +of the [Apache Spark Kubernetes Operator](https://github.com/apache/spark-kubernetes-operator), which +starts the driver and executor pods of the job. + +All the job state lives in Kubernetes, not in the Gravitino server. Jobs keep running while Gravitino +restarts, and every Gravitino server of a deployment can query and cancel any job. + +The `k8s` job executor only runs Spark templates. A run of a shell template is rejected. + +For job templates and the job API, see [Manage Jobs](./manage-jobs-in-gravitino.md). To run jobs as +processes on the Gravitino server instead, see [Local Job Executor](./local-job-executor.md). + +## Requirements + +- Apache Spark Kubernetes Operator 0.9.x in the Kubernetes cluster that runs the jobs, watching Review Comment: Added a version table under Requirements in caf9b440d: supported and tested versions of the Spark operator (0.9.x / 0.9.0), Spark (3.5 / 3.5.9), the Iceberg Spark runtime (1.11 / 1.11.0), Kubernetes (what operator 0.9.x runs on / 1.35.6) and YuniKorn (optional / 1.10.0). _🤖 Addressed by [Claude Code](https://claude.com/claude-code)_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
