Hello The key output is
java.lang.OutOfMemoryError: Java heap space Review batch property options to limit response sizes in the database calls. Thanks On Tue, Mar 19, 2024 at 6:15 AM <e-soci...@gmx.fr> wrote: > Hello > > I got the executeSQL processor does the sql command "select * from > public.table1" > It is a postgresql database. > > Here the end of properties of processor. > > Max Wait Time 0 seconds > Normalize Table/Column Names false > Use Avro Logical Types false > Compression Format NONE > Default Decimal Precision 10 > Default Decimal Scale 0 > Max Rows Per Flow File 0 > Output Batch Size 0 > Fetch Size: 200000 > Set Auto Commit :false > > > The same sql command works in nifi 1.16.3 with the same configuration > > I don't know why it is failed. Thanks > > Need you help .. there is a strange error : > > 2024-03-19 12:58:37,683 ERROR [Load-Balanced Client Thread-6] > org.apache.nifi.engine.FlowEngine Uncaught Exception in Runnable task > java.lang.OutOfMemoryError: Java heap space > 2024-03-19 12:58:37,684 ERROR [Load-Balanced Client Thread-2] > org.apache.nifi.engine.FlowEngine Uncaught Exception in Runnable task > java.lang.OutOfMemoryError: Java heap space > at > java.base/java.util.concurrent.CopyOnWriteArrayList.iterator(CopyOnWriteArrayList.java:1024) > at > java.base/java.util.concurrent.CopyOnWriteArraySet.iterator(CopyOnWriteArraySet.java:389) > at > org.apache.nifi.controller.queue.clustered.client.async.nio.NioAsyncLoadBalanceClientTask.run(NioAsyncLoadBalanceClientTask.java:54) > at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) > at > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) > at > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > at > java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at java.base/java.lang.Thread.run(Thread.java:834) > 2024-03-19 12:58:48,940 INFO [main-EventThread] > o.a.c.f.state.ConnectionStateManager State change: SUSPENDED > 2024-03-19 12:58:49,347 ERROR [Timer-Driven Process Thread-9] > org.apache.nifi.engine.FlowEngine Uncaught Exception in Runnable task > java.lang.OutOfMemoryError: Java heap space > 2024-03-19 12:58:49,351 INFO [NiFi Web Server-5835] > org.apache.nifi.web.server.RequestLog 138.21.169.37 - CN=admin.plants, > OU=NIFI [19/Mar/2024:12:58:48 +0000] "GET /nifi-api/flow/cluster/summary > HTTP/1.1" 200 104 " > https://nifi-01:9091/nifi/?processGroupId=0f426c92-018e-1000-0000-00003fca1e11&componentIds="; > "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, > like Gecko) Chrome/105.0.0.0 Safari/537.36" > 2024-03-19 12:58:48,559 ERROR [Load-Balanced Client Thread-3] > org.apache.nifi.engine.FlowEngine Uncaught Exception in Runnable task > java.lang.OutOfMemoryError: Java heap space > 2024-03-19 12:58:49,745 INFO [NiFi Web Server-5419] > org.apache.nifi.web.server.RequestLog 138.21.169.37 - - > [19/Mar/2024:12:58:49 +0000] "POST > /rb_bf28073qyu?type=js3&sn=v_4_srv_8_sn_B1E58A3741D2949DD454A88FF8A4BAF3_perc_100000_ol_0_mul_1_app-3A4e195de4d0714591_1_app-3A44074a8878754fd3_1_app-3Ad1603d0792f56d4b_1_app-3A8092cfc902bb1761_1_rcs-3Acss_1&svrid=8&flavor=post&vi=QKHPLOMPDHAQNPHMHHTUOHHKWJHFRNCG-0&modifiedSince=1710851904127&rf=https%3A%2F%2Fnifi-01%3A9091%2Fnifi%2F%3FprocessGroupId%3D0f426c92-018e-1000-0000-00003fca1e11%26componentIds%3D&bp=3&app=d1603d0792f56d4b&crc=1268897524&en=7xpdnw1j&end=1 > HTTP/1.1" 200 109 " > https://nifi-01:9091/nifi/?processGroupId=0f426c92-018e-1000-0000-00003fca1e11&componentIds="; > "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, > like Gecko) Chrome/105.0.0.0 Safari/537.36" > 2024-03-19 12:58:57,419 ERROR [Timer-Driven Process Thread-5] > org.apache.nifi.engine.FlowEngine Uncaught Exception in Runnable task > java.lang.OutOfMemoryError: Java heap space > 2024-03-19 12:58:50,209 WARN [NiFi Web Server-5689] > o.a.n.c.l.e.CuratorLeaderElectionManager Unable to determine leader for > role 'Cluster Coordinator'; returning null > org.apache.zookeeper.KeeperException$ConnectionLossException: > KeeperErrorCode = ConnectionLoss for /nifi/clu_quality_2/leaders/Cluster > Coordinator > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:102) > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:54) > at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:2480) > at > org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:243) > at > org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:232) > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:94) > at > org.apache.curator.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:229) > at > org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:220) > at > org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:42) > at > org.apache.curator.framework.recipes.locks.LockInternals.getSortedChildren(LockInternals.java:155) > at > org.apache.curator.framework.recipes.locks.LockInternals.getParticipantNodes(LockInternals.java:135) > at > org.apache.curator.framework.recipes.locks.InterProcessMutex.getParticipantNodes(InterProcessMutex.java:170) > at > org.apache.curator.framework.recipes.leader.LeaderSelector.getLeader(LeaderSelector.java:336) > at > org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.getLeader(CuratorLeaderElectionManager.java:281) > at > org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener.verifyLeader(CuratorLeaderElectionManager.java:572) > at > org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener.isLeader(CuratorLeaderElectionManager.java:526) > at > org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$LeaderRole.isLeader(CuratorLeaderElectionManager.java:467) > at > org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.isLeader(CuratorLeaderElectionManager.java:262) > at > org.apache.nifi.cluster.coordination.node.NodeClusterCoordinator.isActiveClusterCoordinator(NodeClusterCoordinator.java:824) > at > org.apache.nifi.web.api.ApplicationResource.getReplicationTarget(ApplicationResource.java:929) > at > org.apache.nifi.web.api.ApplicationResource.replicateNodeResponse(ApplicationResource.java:1051) > at > org.apache.nifi.web.api.ApplicationResource.replicateNodeResponse(ApplicationResource.java:1025) > at > org.apache.nifi.web.api.ApplicationResource.replicate(ApplicationResource.java:1016) > at > org.apache.nifi.web.api.ApplicationResource.replicate(ApplicationResource.java:1000) > at > org.apache.nifi.web.api.ApplicationResource.replicate(ApplicationResource.java:975) > at > org.apache.nifi.web.api.FlowResource.getControllerStatus(FlowResource.java:1083) > at jdk.internal.reflect.GeneratedMethodAccessor775.invoke(Unknown > Source) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > at > org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:134) > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:177) > at > org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:81) > at > org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478) > at > org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400) > at > org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) > at > org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256) > at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) > at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) > at org.glassfish.jersey.internal.Errors.process(Errors.java:292) > at org.glassfish.jersey.internal.Errors.process(Errors.java:274) > at org.glassfish.jersey.internal.Errors.process(Errors.java:244) > at > org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) > at > org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235) > at > org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684) > at > org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) > at > org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) > at > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:358) > at > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:311) > at > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) > at > org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) > at > org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) > at > org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:352) > at > org.springframework.security.web.access.intercept.AuthorizationFilter.doFilter(AuthorizationFilter.java:100) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:126) > at > org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:120) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.apache.nifi.web.security.log.AuthenticationUserFilter.doFilterInternal(AuthenticationUserFilter.java:57) > at > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.security.oauth2.server.resource.web.authentication.BearerTokenAuthenticationFilter.doFilterInternal(BearerTokenAuthenticationFilter.java:132) > at > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.authenticate(NiFiAuthenticationFilter.java:94) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:56) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.security.web.csrf.CsrfFilter.doFilterInternal(CsrfFilter.java:118) > at > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.apache.nifi.web.security.csrf.SkipReplicatedCsrfFilter.doFilterInternal(SkipReplicatedCsrfFilter.java:59) > at > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:62) > at > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:225) > at > org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:190) > at > org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) > at > org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) > at > org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) > at > org.apache.nifi.web.filter.ExceptionFilter.doFilter(ExceptionFilter.java:46) > at > org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) > at > org.eclipse.jetty.servlets.DoSFilter.doFilterChain(DoSFilter.java:487) > at > org.apache.nifi.web.server.filter.DataTransferExcludedDoSFilter.doFilterChain(DataTransferExcludedDoSFilter.java:51) > at > org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:336) > at > org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:301) > at > org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) > at > org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) > at > org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) > at > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) > at > org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) > at > org.apache.nifi.web.server.log.RequestAuthenticationFilter.doFilterInternal(RequestAuthenticationFilter.java:59) > at > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) > at > org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) > at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) > at > org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:772) > at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) > at > org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:59) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at org.eclipse.jetty.server.Server.handle(Server.java:516) > at > org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) > at > org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) > at > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) > at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) > at > org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) > at > org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) > at > org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) > at > org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) > at > org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) > at > org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) > at > org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) > at > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) > at > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) > at java.base/java.lang.Thread.run(Thread.java:834) > 2024-03-19 12:58:57,856 INFO [NiFi Web Server-5835] > org.apache.nifi.web.server.RequestLog 138.21.169.37 - CN=admin.plants, > OU=NIFI [19/Mar/2024:12:58:57 +0000] "GET /nifi-api/flow/cluster/summary > HTTP/1.1" 200 104 " > https://nifi-01:9091/nifi/?processGroupId=0f426c92-018e-1000-0000-00003fca1e11&componentIds="; > "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, > like Gecko) Chrome/105.0.0.0 Safari/537.36" > 2024-03-19 12:58:58,225 INFO [Curator-ConnectionStateManager-0] > o.a.n.c.l.e.CuratorLeaderElectionManager > org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@2e294ff > Connection State changed to SUSPENDED > 2024-03-19 12:58:58,226 INFO [Curator-ConnectionStateManager-0] > o.a.n.c.l.e.CuratorLeaderElectionManager > org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@43df9ec4 > Connection State changed to SUSPENDED > 2024-03-19 12:58:58,227 ERROR [Timer-Driven Process Thread-2] > org.apache.nifi.engine.FlowEngine Uncaught Exception in Runnable task > java.lang.OutOfMemoryError: Java heap space > 2024-03-19 12:58:58,609 ERROR [Load-Balanced Client Thread-8] > org.apache.nifi.engine.FlowEngine Uncaught Exception in Runnable task > java.lang.OutOfMemoryError: Java heap space > 2024-03-19 12:58:59,370 WARN [Curator-ConnectionStateManager-0] > o.a.c.f.state.ConnectionStateManager Session timeout has elapsed while > SUSPENDED. Injecting a session expiration. Elapsed ms: 10046. Adjusted > session timeout ms: 10000 > 2024-03-19 12:58:59,738 INFO [Curator-ConnectionStateManager-0] > org.apache.zookeeper.ZooKeeperTestable injectSessionExpiration() called > 2024-03-19 12:59:00,921 INFO [main-EventThread] > org.apache.zookeeper.ClientCnxnSocket jute.maxbuffer value is 1048575 Bytes > 2024-03-19 12:59:00,922 INFO [main-EventThread] > o.a.c.f.state.ConnectionStateManager State change: LOST > > >
