jerryshao opened a new pull request, #13250:
URL: https://github.com/apache/gravitino/pull/13250
### What changes were proposed in this pull request?
Adds the ability to retrieve a job's captured stdout/stderr output, end to
end:
- `JobExecutor#getJobStdout`/`getJobStderr(jobId, maxLines)` — new default
methods (empty list by
default), implemented by `LocalJobExecutor` via a bounded tail-read of the
job's
`output.log`/`error.log` (no full-file load).
- Core: `JobOperationDispatcher`/`JobManager`'s `getJob` gains an
`includeOutput` parameter;
`JobEntity` carries non-persisted `stdout`/`stderr` fields when requested.
- REST: the existing `GET .../jobs/runs/{jobId}` endpoint gains an
`includeOutput` query
parameter (no new endpoint), so `listJobs`/plain `getJob` never carry the
extra payload.
- Java/Python clients: `SupportsJobs#getJob(jobId, includeOutput)` (Java:
default method
delegating to the new overload; Python: `get_job(job_id,
include_output=False)`), and
`JobHandle#stdout()`/`stderr()`.
- New global config `gravitino.job.outputMaxLines` (default 1000), resolved
by `JobManager` and
passed to the executor per call.
- Also fixes `ExceptionHandlers` not mapping `UnsupportedOperationException`
for job and
job-template operations, and makes `getJob(..., includeOutput=true)`
degrade gracefully to
empty output (instead of a false "job not found") when the job entity
still exists but the
executor's own output bookkeeping has expired.
- Docs: `gravitino.job.outputMaxLines` added to both config reference
tables, plus a new
"Get a Job's Output" section with REST/Java/Python examples.
### Why are the changes needed?
`JobExecutor` only exposed `submitJob`/`getJobStatus`/`cancelJob` — there
was no way to see why a
job failed without directly inspecting the local runner's staging directory.
This closes that
gap for the built-in local executor and lays the API surface for other
executors to do the same.
Fix: #12716 (subtask of epic #12667)
### Does this PR introduce _any_ user-facing change?
Yes:
- New optional `includeOutput` query parameter on `GET
.../jobs/runs/{jobId}`.
- New Java `SupportsJobs#getJob(String jobId, boolean includeOutput)` and
`JobHandle#stdout()`/
`stderr()`; new Python `get_job(job_id, include_output=False)` and
`JobHandle.stdout()`/
`stderr()`.
- New server config property `gravitino.job.outputMaxLines` (default `1000`).
### How was this patch tested?
Added unit tests across `TestLocalJobExecutor`, `TestJobManager`,
`TestJobOperations`,
`TestJobDTO`, `TestSupportsJobs` (Java), and Python
`test_job_dto_serde`/`test_supports_jobs`.
Added an end-to-end integration test (`JobIT.testRunAndGetJobOutput`) that
runs a real local job,
polls to completion, and asserts the actual captured output. Full
`:core:test`/`:server:test`
suites re-verified green (2140+ tests) after a full automated code review
and fixes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
