ppawel commented on PR #37164: URL: https://github.com/apache/beam/pull/37164#issuecomment-3728717153
Just an update that I made a mistake when preparing a build of our pipelines with the patched Beam so basically patched SolaceIO was not included - feedback above is not based on this PR but rather on the 2.70.0 so not that interesting (it's the same as my original bug report). Sorry for the noise... After properly building the patched SolaceIO, I observe several things already: I see around 40-50% increase in the build time - this seems to be due to integration tests now taking much longer. I will have to take a closer look but we are using containerized Solace (so we don't use the TestStream/TestPipeline approach) with some aggressive cleanup during test shutdown. In the past we had issues with tests interfereing with each other and unfortunately Solace does not provide any nice ways to manage current JVM's consumers/sessions so I was forced to implement deleting the test queue and subscriptions from the broker. Then the consumers get UNBIND instruction with error 503 and next test is cleanly reconnecting to the fresh setup. However, with the patched version, we also get frequent errors in tests about failed NACKs, e.g.: ``` SolaceIO.Read: failed to nack the message with applicationMessageId=null, ackMessageId=1126. 2026-01-09T11:01:55.6533628Z java.lang.IllegalStateException: Attempted an operation on a closed message consumer. 2026-01-09T11:01:55.6534245Z at com.solacesystems.jcsmp.impl.flow.FlowHandleImpl.settle(FlowHandleImpl.java:2384) 2026-01-09T11:01:55.6538259Z at com.solacesystems.jcsmp.impl.JCSMPXMLMessage.settle(JCSMPXMLMessage.java:585) 2026-01-09T11:01:55.6538879Z at com.solacesystems.jcsmp.impl.MessageImpl.settle(MessageImpl.java:43) 2026-01-09T11:01:55.6539539Z at org.apache.beam.sdk.io.solace.read.UnboundedSolaceReader.nackMessages(UnboundedSolaceReader.java:175) 2026-01-09T11:01:55.6540530Z at org.apache.beam.sdk.io.solace.read.UnboundedSolaceReader.lambda$getCheckpointMark$2(UnboundedSolaceReader.java:208) 2026-01-09T11:01:55.6541203Z at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) 2026-01-09T11:01:55.6542730Z at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) 2026-01-09T11:01:55.6543382Z at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) 2026-01-09T11:01:55.6544039Z at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) 2026-01-09T11:01:55.6544678Z at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) 2026-01-09T11:01:55.6545228Z at java.base/java.lang.Thread.run(Thread.java:1583) 2026-01-09T11:01:55.6553547Z 2026-01-09 11:01:55,553 [32mERROR[0;39m [pool-78-thread-4] [36mo.a.b.s.i.s.r.UnboundedSolaceReader[0;39m : SolaceIO.Read: failed to nack the message with applicationMessageId=null, ackMessageId=1127. 2026-01-09T11:01:55.6554257Z java.lang.IllegalStateException: Attempted an operation on a closed message consumer. 2026-01-09T11:01:55.6554830Z at com.solacesystems.jcsmp.impl.flow.FlowHandleImpl.settle(FlowHandleImpl.java:2384) 2026-01-09T11:01:55.6555462Z at com.solacesystems.jcsmp.impl.JCSMPXMLMessage.settle(JCSMPXMLMessage.java:585) 2026-01-09T11:01:55.6563745Z at com.solacesystems.jcsmp.impl.MessageImpl.settle(MessageImpl.java:43) 2026-01-09T11:01:55.6564361Z at org.apache.beam.sdk.io.solace.read.UnboundedSolaceReader.nackMessages(UnboundedSolaceReader.java:175) 2026-01-09T11:01:55.6565037Z at org.apache.beam.sdk.io.solace.read.UnboundedSolaceReader.lambda$getCheckpointMark$2(UnboundedSolaceReader.java:208) 2026-01-09T11:01:55.6572290Z at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) 2026-01-09T11:01:55.6572905Z at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) 2026-01-09T11:01:55.6576578Z at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) 2026-01-09T11:01:55.6577487Z at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) 2026-01-09T11:01:55.6578234Z at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) 2026-01-09T11:01:55.6578904Z at java.base/java.lang.Thread.run(Thread.java:1583) ``` Overall, there are test failures with this patched version so I will need to check deeper why are they failing. When I have a more specific reason, I will post another comment. I might also try to build a test version with tests skipped just to see how the patched version behaves inside Dataflow, but I would rather first fix the failing tests.. Would it make sense to add a way to deactivate this new NACK deadline feature, e.g. if it is set to 0? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
