[ https://issues.apache.org/jira/browse/NIFI-11794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17742534#comment-17742534 ]
Pierre Villard commented on NIFI-11794: --------------------------------------- The issue only happens when clearing a local state for a component in a NiFi cluster. Even though the state is local, each node of the cluster will receive the request to clear the state for this component at cluster scope, and this causes the Redis State Provider to be called. When all nodes are trying to clear the state at the same time, only one node succeeds and the transaction is aborted for the other nodes, causing the NPE. The fix, prevents the NPE and makes the maxAttempts configurable. The reason being that one node will need as many attempts as there are nodes in the cluster to successfully clear the state. First node: 2023-07-12 17:12:42,428 DEBUG org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1: org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1@5c4008ef Attempt # 1 to clear state for component 46025cdf-0189-1000-ffff-ffffc11ae372 was successful Second node: 2023-07-12 17:12:42,430 DEBUG org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1: org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1@15ede30f Attempt # 1 to clear state for component 46025cdf-0189-1000-ffff-ffffc11ae372 was unsuccessful 2023-07-12 17:12:42,433 DEBUG org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1: org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1@15ede30f Attempt # 2 to clear state for component 46025cdf-0189-1000-ffff-ffffc11ae372 was successful Third node: 2023-07-12 17:12:42,429 DEBUG org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1: org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1@6ed23ad8 Attempt # 1 to clear state for component 46025cdf-0189-1000-ffff-ffffc11ae372 was unsuccessful 2023-07-12 17:12:42,434 DEBUG org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1: org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1@6ed23ad8 Attempt # 2 to clear state for component 46025cdf-0189-1000-ffff-ffffc11ae372 was unsuccessful 2023-07-12 17:12:42,437 DEBUG org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1: org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1@6ed23ad8 Attempt # 3 to clear state for component 46025cdf-0189-1000-ffff-ffffc11ae372 was successful It means that the number should be at least the number of nodes in the NiFi cluster. The default value is 20. > RedisStateProvider failing when used with GCP Memorystore > --------------------------------------------------------- > > Key: NIFI-11794 > URL: https://issues.apache.org/jira/browse/NIFI-11794 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions > Reporter: Pierre Villard > Assignee: Pierre Villard > Priority: Major > Fix For: 1.latest, 2.latest > > Time Spent: 10m > Remaining Estimate: 0h > > Trying to configure the NiFi cluster state provider with the Redis > implementation against a Google Cloud Platform Memorystore instance. > I can see that there is no state saved in the redis instance. > When trying to clear state of a processor: > {code:java} > 2023-07-11 17:57:39,007 ERROR [NiFi Web Server-22] > o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: > java.lang.NullPointerException. Returning Internal Server Error response. > java.lang.NullPointerException: null > at > org.apache.nifi.redis.state.RedisStateProvider.lambda$replace$1(RedisStateProvider.java:248) > at > org.apache.nifi.redis.state.RedisStateProvider.withConnection(RedisStateProvider.java:313) > at > org.apache.nifi.redis.state.RedisStateProvider.replace(RedisStateProvider.java:207) > at > org.apache.nifi.redis.state.RedisStateProvider.clear(RedisStateProvider.java:263) > at > org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1.clear(StandardStateManagerProvider.java:395) > at > org.apache.nifi.controller.state.StandardStateManager.clear(StandardStateManager.java:85) > at > org.apache.nifi.web.dao.impl.StandardComponentStateDAO.clearState(StandardComponentStateDAO.java:58) > at > org.apache.nifi.web.dao.impl.StandardComponentStateDAO.clearState(StandardComponentStateDAO.java:72) > at > org.apache.nifi.web.dao.impl.StandardComponentStateDAO$$FastClassBySpringCGLIB$$51589743.invoke(<generated>) > {code} > However this action results in creating the key in the Redis instance: > {code:java} > # redis-cli -h ... -p ... -a ... --tls --cacert /tmp/server-ca.pem > ...:6378> KEYS * > 1) "nifi/components/46025cdf-0189-1000-ffff-ffffc11ae372" > ...:6378> GET "nifi/components/46025cdf-0189-1000-ffff-ffffc11ae372" > "{\"version\":0,\"encodingVersion\":1,\"stateValues\":{}}" > {code} > The configuration in state-management.xml is looking like: > {code:java} > <cluster-provider> > <id>redis-provider</id> > <class>org.apache.nifi.redis.state.RedisStateProvider</class> > <property name="Redis Mode">Standalone</property> > <property name="Connection String">...:6378</property> > <property name="Password">...</property> > <property name="Enable TLS">true</property> > </cluster-provider> {code} > CA cert has been added in the NiFi truststore. > Still debugging the code to figure out the issue and will also add additional > logs. > -- This message was sent by Atlassian Jira (v8.20.10#820010)