Can you put a forward proxy in front of the cluster and determine what differs about the requests that result in a 400? Either this truly is the client's fault, as the code suggests, or whatever oauth implementation you're hitting is not a fan of issuing as many concurrent tokens as you have nodes in the cluster. Ruling out the former makes your job a bit easier.
Cannon On Tue, Sep 6, 2022, 6:54 PM Mike Thomsen <[email protected]> wrote: > Are you by any chance running Keycloak? > > On Mon, Aug 29, 2022 at 4:03 AM Jens M. Kofoed > <[email protected]> wrote: > > > > Hi community > > > > I'm using the StandardOauth2AccessTokenProvider to get and refresh a > token, which works great. But almost at every refresh, one of the nodes in > the cluster gets this error. It's not the same node which gets the error > every time, all nodes gets it but only one node at a time. > > > > 2022-08-29 06:14:28,081 ERROR [Timer-Driven Process Thread-4] > org.apache.nifi.oauth2.StandardOauth2AccessTokenProvider > StandardOauth2AccessTokenProvider[id=861dbfea-0181-1000-ffff-ffffd19b4cf0] > OAuth2 access token request failed [HTTP 400], response: > > {"error":"invalid_grant","error_description":"Token is not active"} > > 2022-08-29 06:14:28,082 INFO [Timer-Driven Process Thread-4] > org.apache.nifi.oauth2.StandardOauth2AccessTokenProvider > StandardOauth2AccessTokenProvider[id=861dbfea-0181-1000-ffff-ffffd19b4cf0] > Refresh Access Token request failed [ > https://foo.bar/auth/realms/myrealm/protocol/openid-connect/token] > > org.apache.nifi.processor.exception.ProcessException: OAuth2 access > token request failed [HTTP 400] > > at > org.apache.nifi.oauth2.StandardOauth2AccessTokenProvider.getAccessDetails(StandardOauth2AccessTokenProvider.java:327) > > at > org.apache.nifi.oauth2.StandardOauth2AccessTokenProvider.refreshAccessDetails(StandardOauth2AccessTokenProvider.java:315) > > at > org.apache.nifi.oauth2.StandardOauth2AccessTokenProvider.getAccessDetails(StandardOauth2AccessTokenProvider.java:249) > > at sun.reflect.GeneratedMethodAccessor408.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at > org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254) > > at > org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:105) > > at com.sun.proxy.$Proxy183.getAccessDetails(Unknown Source) > > at > org.apache.nifi.processors.standard.InvokeHTTP.lambda$configureRequest$3(InvokeHTTP.java:1108) > > at java.util.Optional.ifPresent(Optional.java:159) > > at > org.apache.nifi.processors.standard.InvokeHTTP.configureRequest(InvokeHTTP.java:1107) > > at > org.apache.nifi.processors.standard.InvokeHTTP.onTrigger(InvokeHTTP.java:927) > > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1283) > > at > org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214) > > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:103) > > at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) > > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > at > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > at java.lang.Thread.run(Thread.java:750) > > > > I can't find any information in the log, when the process successful > refresh the token. So I can't see if all nodes in the cluster is refreshing > the token at the same time, or if it's only the primary nodes which > refresh. If all nodes are refreshing could it be that one nodes is slower > than the others to refresh, and that the old tokens gets invalid after the > first node has refreshed it? > > > > Kind regards > > Jens M. Kofoed >
