[
https://issues.apache.org/jira/browse/HDDS-17059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devesh Kumar Singh updated HDDS-17059:
--------------------------------------
Description:
Introduce a Jobworker service that allows OM to dispatch background commands to
registered workers and receive execution results. This provides reusable
infrastructure for operations such as key migration without requiring OM to
perform the underlying data processing.
The framework should be independent of any feature related dependencies,
lifecycle rules etc.
*Scope:*
- Add Jobworker service startup, shutdown, configuration, packaging, and OM
connectivity.
- Introduce the shared registration, heartbeat, command, and result-reporting
protocols.
- Track registered workers, their health, and their availability.
- Deliver queued commands through heartbeat responses.
- Provide extensible command-handler registration and asynchronous execution.
- Track command status, execution results, and reporting timeouts.
- Integrate with OM leadership changes and worker re-registration.
- Provide metrics and diagnostics for worker health and command processing.
Migration-specific commands, task tables, scheduling, and storage-policy
behavior belong to the dependent JIRA.
*Acceptance criteria:*
- A worker can register with OM and maintain its health through heartbeats.
- OM can dispatch a test command, and the worker can execute it and report its
result.
- Worker restart, heartbeat loss, execution failure, duplicate reports, and
command timeouts have defined and tested behavior.
- Leadership changes prevent stale dispatch and allow workers to reconnect to
the active OM.
- Shutdown releases connections, threads, and other resources.
- Tests exercise the complete registration → dispatch → execution → result flow
without storage-policy dependencies.
was:
Introduce a Jobworker service that allows OM to dispatch background commands to
registered workers and receive execution results. This provides reusable
infrastructure for operations such as key migration without requiring OM to
perform the underlying data processing.
The framework should be independent of storage-policy definitions, lifecycle
rules, and policy-aware key rewriting, allowing it to be developed against
master.
*Scope:*
- Add Jobworker service startup, shutdown, configuration, packaging, and OM
connectivity.
- Introduce the shared registration, heartbeat, command, and result-reporting
protocols.
- Track registered workers, their health, and their availability.
- Deliver queued commands through heartbeat responses.
- Provide extensible command-handler registration and asynchronous execution.
- Track command status, execution results, and reporting timeouts.
- Integrate with OM leadership changes and worker re-registration.
- Provide metrics and diagnostics for worker health and command processing.
Migration-specific commands, task tables, scheduling, and storage-policy
behavior belong to the dependent JIRA.
*Acceptance criteria:*
- A worker can register with OM and maintain its health through heartbeats.
- OM can dispatch a test command, and the worker can execute it and report its
result.
- Worker restart, heartbeat loss, execution failure, duplicate reports, and
command timeouts have defined and tested behavior.
- Leadership changes prevent stale dispatch and allow workers to reconnect to
the active OM.
- Shutdown releases connections, threads, and other resources.
- Tests exercise the complete registration → dispatch → execution → result flow
without storage-policy dependencies.
> Introduce a generic Jobworker framework for OM-managed background jobs.
> -----------------------------------------------------------------------
>
> Key: HDDS-17059
> URL: https://issues.apache.org/jira/browse/HDDS-17059
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: Ozone Datanode, Ozone Filesystem, Ozone Manager
> Reporter: Devesh Kumar Singh
> Assignee: Devesh Kumar Singh
> Priority: Major
>
> Introduce a Jobworker service that allows OM to dispatch background commands
> to registered workers and receive execution results. This provides reusable
> infrastructure for operations such as key migration without requiring OM to
> perform the underlying data processing.
> The framework should be independent of any feature related dependencies,
> lifecycle rules etc.
> *Scope:*
> - Add Jobworker service startup, shutdown, configuration, packaging, and OM
> connectivity.
> - Introduce the shared registration, heartbeat, command, and result-reporting
> protocols.
> - Track registered workers, their health, and their availability.
> - Deliver queued commands through heartbeat responses.
> - Provide extensible command-handler registration and asynchronous execution.
> - Track command status, execution results, and reporting timeouts.
> - Integrate with OM leadership changes and worker re-registration.
> - Provide metrics and diagnostics for worker health and command processing.
> Migration-specific commands, task tables, scheduling, and storage-policy
> behavior belong to the dependent JIRA.
> *Acceptance criteria:*
> - A worker can register with OM and maintain its health through heartbeats.
> - OM can dispatch a test command, and the worker can execute it and report
> its result.
> - Worker restart, heartbeat loss, execution failure, duplicate reports, and
> command timeouts have defined and tested behavior.
> - Leadership changes prevent stale dispatch and allow workers to reconnect to
> the active OM.
> - Shutdown releases connections, threads, and other resources.
> - Tests exercise the complete registration → dispatch → execution → result
> flow without storage-policy dependencies.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
