[
https://issues.apache.org/jira/browse/NIFI-16582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Albert Wong updated NIFI-16582:
-------------------------------
External issue URL: https://github.com/apache/nifi/pull/11919
Description:
h2. Problem
RedisDistributedMapCacheClientService rejects Cluster connection pools and uses
WATCH/MULTI/EXEC for conditional cache operations. Redis Enterprise Software
and Redis Cloud proxy endpoints can already use Standalone mode and the
existing ACL/TLS properties, but the component documentation does not explain
that topology distinction.
h2. Proposed improvement
* Replace conditional get-and-insert and compare-and-set with single-key Lua
scripts, declaring the key explicitly for cluster routing.
* Use SET NX with expiration for conditional inserts, keeping the write and TTL
atomic.
* Accept Cluster connection pools.
* Preserve serialized keys/values, raw-value revisions, create-if-absent
behavior, and the distinction between empty and absent values.
* Preserve TTL on failed comparisons and existing-key inserts; apply TTL on
successful writes; zero TTL creates persistent entries.
* Document Enterprise Software/Cloud database proxy endpoints, Standalone
versus Cluster mode, database index 0, ACL credentials, TLS/mTLS configuration,
and EVAL permissions.
* Document that single-endpoint atomicity does not provide globally serialized
compare-and-set across independent writable Active-Active regions.
The SimpleRedisDistributedMapCacheClientService implementation remains
unchanged. Bulk putAll retains its existing MSET behavior and adds no
cross-slot transaction guarantee.
h2. Implementation and validation
A six-file implementation is prepared against main commit
2d651dc1646ac3a38f1bfc00d60a9727077a558e.
* JDK 21 reactor build and Redis NAR packaging passed.
* Six unit cases and 24 integration cases passed without failures or skips.
* The 12-case integration contract runs against an authenticated Redis 8.8
standalone endpoint and a one-node Redis Cluster owning all 16,384 slots. Both
disable the default user and deny WATCH, MULTI, and EXEC.
* Coverage includes binary/empty values, stale/missing revisions, positive/zero
TTL, expiration on success/conflict, and 32 competing insert/replacement
writers with exactly one winner.
* Checkstyle and patch applicability checks passed.
Live Redis Enterprise Software/Redis Cloud endpoint acceptance, network
TLS/mTLS handshakes, multi-node failover/resharding, JDK 25, and the full
contribution-profile build remain untested. Pull request:
https://github.com/apache/nifi/pull/11919
was:
h2. Problem
RedisDistributedMapCacheClientService rejects Cluster connection pools and uses
WATCH/MULTI/EXEC for conditional cache operations. Redis Enterprise Software
and Redis Cloud proxy endpoints can already use Standalone mode and the
existing ACL/TLS properties, but the component documentation does not explain
that topology distinction.
h2. Proposed improvement
* Replace conditional get-and-insert and compare-and-set with single-key Lua
scripts, declaring the key explicitly for cluster routing.
* Use SET NX with expiration for conditional inserts, keeping the write and TTL
atomic.
* Accept Cluster connection pools.
* Preserve serialized keys/values, raw-value revisions, create-if-absent
behavior, and the distinction between empty and absent values.
* Preserve TTL on failed comparisons and existing-key inserts; apply TTL on
successful writes; zero TTL creates persistent entries.
* Document Enterprise Software/Cloud database proxy endpoints, Standalone
versus Cluster mode, database index 0, ACL credentials, TLS/mTLS configuration,
and EVAL permissions.
* Document that single-endpoint atomicity does not provide globally serialized
compare-and-set across independent writable Active-Active regions.
The SimpleRedisDistributedMapCacheClientService implementation remains
unchanged. Bulk putAll retains its existing MSET behavior and adds no
cross-slot transaction guarantee.
h2. Implementation and validation
A six-file implementation is prepared against main commit
2d651dc1646ac3a38f1bfc00d60a9727077a558e.
* JDK 21 reactor build and Redis NAR packaging passed.
* Six unit cases and 24 integration cases passed without failures or skips.
* The 12-case integration contract runs against an authenticated Redis 8.8
standalone endpoint and a one-node Redis Cluster owning all 16,384 slots. Both
disable the default user and deny WATCH, MULTI, and EXEC.
* Coverage includes binary/empty values, stale/missing revisions, positive/zero
TTL, expiration on success/conflict, and 32 competing insert/replacement
writers with exactly one winner.
* Checkstyle and patch applicability checks passed.
Live Redis Enterprise Software/Redis Cloud endpoint acceptance, network
TLS/mTLS handshakes, multi-node failover/resharding, JDK 25, and the full
contribution-profile build remain untested. A pull request will be linked after
creation.
> Support atomic Redis map cache operations on clustered and managed Redis
> deployments
> ------------------------------------------------------------------------------------
>
> Key: NIFI-16582
> URL: https://issues.apache.org/jira/browse/NIFI-16582
> Project: Apache NiFi
> Issue Type: Improvement
> Environment: Apache NiFi current main (2.13.0-SNAPSHOT); JDK 21;
> Redis 8.8 standalone and Redis Cluster API integration fixtures.
> Reporter: Albert Wong
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h2. Problem
> RedisDistributedMapCacheClientService rejects Cluster connection pools and
> uses WATCH/MULTI/EXEC for conditional cache operations. Redis Enterprise
> Software and Redis Cloud proxy endpoints can already use Standalone mode and
> the existing ACL/TLS properties, but the component documentation does not
> explain that topology distinction.
> h2. Proposed improvement
> * Replace conditional get-and-insert and compare-and-set with single-key Lua
> scripts, declaring the key explicitly for cluster routing.
> * Use SET NX with expiration for conditional inserts, keeping the write and
> TTL atomic.
> * Accept Cluster connection pools.
> * Preserve serialized keys/values, raw-value revisions, create-if-absent
> behavior, and the distinction between empty and absent values.
> * Preserve TTL on failed comparisons and existing-key inserts; apply TTL on
> successful writes; zero TTL creates persistent entries.
> * Document Enterprise Software/Cloud database proxy endpoints, Standalone
> versus Cluster mode, database index 0, ACL credentials, TLS/mTLS
> configuration, and EVAL permissions.
> * Document that single-endpoint atomicity does not provide globally
> serialized compare-and-set across independent writable Active-Active regions.
> The SimpleRedisDistributedMapCacheClientService implementation remains
> unchanged. Bulk putAll retains its existing MSET behavior and adds no
> cross-slot transaction guarantee.
> h2. Implementation and validation
> A six-file implementation is prepared against main commit
> 2d651dc1646ac3a38f1bfc00d60a9727077a558e.
> * JDK 21 reactor build and Redis NAR packaging passed.
> * Six unit cases and 24 integration cases passed without failures or skips.
> * The 12-case integration contract runs against an authenticated Redis 8.8
> standalone endpoint and a one-node Redis Cluster owning all 16,384 slots.
> Both disable the default user and deny WATCH, MULTI, and EXEC.
> * Coverage includes binary/empty values, stale/missing revisions,
> positive/zero TTL, expiration on success/conflict, and 32 competing
> insert/replacement writers with exactly one winner.
> * Checkstyle and patch applicability checks passed.
> Live Redis Enterprise Software/Redis Cloud endpoint acceptance, network
> TLS/mTLS handshakes, multi-node failover/resharding, JDK 25, and the full
> contribution-profile build remain untested. Pull request:
> https://github.com/apache/nifi/pull/11919
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
