Serge Huber created UNOMI-878:
---------------------------------
Summary: Enhanced Cluster-Aware Task Scheduling Service with
Improved Developer Experience and Persistence Integration
Key: UNOMI-878
URL: https://issues.apache.org/jira/browse/UNOMI-878
Project: Apache Unomi
Issue Type: Bug
Components: unomi(-core)
Affects Versions: unomi-3.0.0
Reporter: Serge Huber
Fix For: unomi-3.0.0
The SchedulerService provides a robust, cluster-aware task scheduling system
with several valuable features that improve reliability, scalability, and
developer experience:
Key Features:
1. Integrated Persistence Layer:
- Seamless integration with Unomi's PersistenceService for task storage
- Automatic task state persistence and recovery
- Support for both persistent and in-memory task storage
- Efficient task querying and filtering capabilities
2. Advanced Cluster Support:
- Node-specific task execution control with `executorNode` configuration
- Ability for nodes to opt-out of task execution
- Automatic task distribution across cluster nodes
- Task execution isolation through node-specific locking
- Built-in crash recovery for failed nodes
- Support for running tasks on all nodes or specific nodes
3. Enhanced Developer Experience:
- Fluent Builder API for intuitive task creation
- Simple recurring task creation for common use cases
- Comprehensive task lifecycle management
- Rich task monitoring capabilities
4. Robust Task Management:
- Automatic task recovery from node crashes
- Configurable retry policies
- Task resumption from checkpoints
- Automatic task purging with configurable TTL
- Support for both one-shot and recurring tasks
Example Usage (New Builder Pattern):
{code:java}
// Create a cluster-wide persistent
taskschedulerService.newTask("dataSync").withPeriod(1,
TimeUnit.HOURS).withSimpleExecutor(() -> syncData()).schedule();// Create a
node-specific memory
taskschedulerService.newTask("localCache").nonPersistent().withPeriod(5,
TimeUnit.MINUTES).withSimpleExecutor(() -> cleanCache()).schedule(); {code}
Technical Benefits:
1. Improved System Reliability:
- Automatic crash recovery prevents task loss
- Persistent storage ensures task survival across restarts
- Cluster-aware execution prevents task conflicts
2. Enhanced Scalability:
- Flexible node participation in task execution
- Efficient task distribution across cluster
- Support for both cluster-wide and node-specific tasks
3. Better Resource Management:
- Option to run memory-only tasks for better performance
- Automatic task cleanup to prevent storage bloat
- Configurable thread pools for task execution
4. Operational Excellence:
- Comprehensive task status tracking
- Built-in monitoring capabilities
- Automatic task recovery and cleanup
Migration Impact:
- Fully backward compatible
- New features are opt-in
- Existing task definitions continue to work as before
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
