dybyte opened a new pull request, #10256: URL: https://github.com/apache/seatunnel/pull/10256
Fixes: https://github.com/apache/seatunnel/issues/10181 ### Purpose of this pull request ## Background Currently, SeaTunnel relies directly on Hazelcast IMap for internal state management. This tight coupling makes it difficult to: - Replace the underlying storage engine (e.g., switching from Hazelcast IMap to RocksDB) - Clearly define the responsibilities of state management components ## Proposal Introduce a **`StateStore` abstraction** to encapsulate all operations currently performed on Hazelcast IMap, along with a **`StateStoreFactory`** to create instances of `StateStore`. ### Components - **StateStore**: Abstracts all methods of Hazelcast IMap used internally in SeaTunnel. - **StateStoreFactory**: Responsible for creating `StateStore` instances. - **DistributedStoreManager**: Creates and provides the appropriate `StateStore` via the `NodeEngine`. - As a result, each service no longer interacts with IMap directly. Instead, it passes the `NodeEngine` to the `DistributedStoreManager` to obtain a `StateStore`. This design allows future replacement of Hazelcast IMap with RocksDB or any other storage engine by simply implementing `StateStore` and `StateStoreFactory` for the new engine. Additionally, creating a storage implementation using RocksDB (or others) becomes clear and straightforward. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Covered by existing tests. ### Check list * [ ] If any new Jar binary package adding in your PR, please add License Notice according [New License Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/contribution/new-license.md) * [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs * [ ] If necessary, please update `incompatible-changes.md` to describe the incompatibility caused by this PR. * [ ] If you are contributing the connector code, please check that the following files are updated: 1. Update [plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties) and add new connector information in it 2. Update the pom file of [seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml) 3. Add ci label in [label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml) 4. Add e2e testcase in [seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/) 5. Update connector [plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
