Re: [I] [Bug]: Kafka Consumer step fails when run inside async web service (hop)
hansva closed issue #7067: [Bug]: Kafka Consumer step fails when run inside async web service URL: https://github.com/apache/hop/issues/7067 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] [Bug]: Kafka Consumer step fails when run inside async web service (hop)
mattcasters commented on issue #7067: URL: https://github.com/apache/hop/issues/7067#issuecomment-4970640251 https://github.com/mattcasters/hop/tree/issue-7067 to be turned into a PR when hop main catches up with the Kafka bug fixes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] [Bug]: Kafka Consumer step fails when run inside async web service (hop)
mattcasters commented on issue #7067: URL: https://github.com/apache/hop/issues/7067#issuecomment-4962089491 So, aside from the bug... The plan is to just read new messages on a topic and then stop the Kafka pipeline somehow? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] [Bug]: Kafka Consumer step fails when run inside async web service (hop)
petertsneg commented on issue #7067: URL: https://github.com/apache/hop/issues/7067#issuecomment-4436622435 @justin-tomlinson I encountered the same issue before. I found that starting from version 2.16, a new 'request header content variable' field was added to the asynchronous web service. It must be configured for the service to function correctly. Hope this helps! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] [Bug]: Kafka Consumer step fails when run inside async web service (hop)
justin-tomlinson commented on issue #7067: URL: https://github.com/apache/hop/issues/7067#issuecomment-4402731387 Current work around for this is to trigger the workflow via the sync call `executeWorkflow` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
[I] [Bug]: Kafka Consumer step fails when run inside async web service (hop)
justin-tomlinson opened a new issue, #7067: URL: https://github.com/apache/hop/issues/7067 ### Apache Hop version? 2.17 ### Java version? 17.0.16 ### Operating system Linux ### What happened? I have an async web service which starts a workflow. The workflow has a pipeline step that consumes from kafka using the kafka consumer (standard, not beam engine) the kafka consumer step has the child step defined and created. Works ok in v2.15 Confirmed that this runs on Hop 2.17 & 2.16 with hop-run cmd and also with hop server call using executeWorkflow endpoint. When running using `asyncRun?service=` in 2.17 or 2.16 it fails with errors as below: ``` 2026/04/30 10:22:42 - touchpoint_sms_unsubscribe_members - Executing this pipeline using the Local Pipeline Engine with run configuration 'local' 2026/04/30 10:22:42 - touchpoint_sms_unsubscribe_members - Execution started for pipeline [touchpoint_sms_unsubscribe_members] 2026/04/30 10:22:42 - touchpoint_sms_unsubscribe_members_child - Executing this pipeline using the Local Pipeline Engine with run configuration 'Kafka' 2026/04/30 10:22:42 - touchpoint_sms_unsubscribe_members_child - Execution started for pipeline [touchpoint_sms_unsubscribe_members_child] 2026/04/30 10:22:42 - Kafka Consumer 2.0 - ERROR: Error initializing sub-transformation 2026/04/30 10:22:42 - Kafka Consumer 2.0 - ERROR: org.apache.hop.core.exception.HopException: 2026/04/30 10:22:42 - Kafka Consumer 2.0 - Unable to load and initialize sub pipeline 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at java.lang.Thread.run (Thread.java:840) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at org.apache.hop.pipeline.transform.TransformInitThread.run (TransformInitThread.java:66) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at org.apache.hop.pipeline.transforms.kafka.consumer.KafkaConsumerInput.init (KafkaConsumerInput.java:98) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at org.apache.hop.pipeline.transforms.kafka.consumer.KafkaConsumerInput.initSubPipeline (KafkaConsumerInput.java:135) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at org.apache.hop.pipeline.TransformWithMappingMeta.replaceVariableValues (TransformWithMappingMeta.java:316) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at org.apache.hop.pipeline.TransformWithMappingMeta.replaceVariableValues (TransformWithMappingMeta.java:309) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at org.apache.hop.pipeline.TransformWithMappingMeta.isInternalVariable (TransformWithMappingMeta.java:324) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at org.apache.hop.pipeline.TransformWithMappingMeta.isWorkflowInternalVariable (TransformWithMappingMeta.java:333) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at java.util.ImmutableCollections$SetN.contains (ImmutableCollections.java:937) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at java.util.Objects.requireNonNull (Objects.java:209) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at org.apache.hop.pipeline.transforms.kafka.consumer.KafkaConsumerInput.initSubPipeline(KafkaConsumerInput.java:204) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at org.apache.hop.pipeline.transforms.kafka.consumer.KafkaConsumerInput.init(KafkaConsumerInput.java:98) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at org.apache.hop.pipeline.transform.TransformInitThread.run(TransformInitThread.java:66) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at java.base/java.lang.Thread.run(Thread.java:840) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - Caused by: java.lang.NullPointerException 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at java.base/java.util.Objects.requireNonNull(Objects.java:209) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at java.base/java.util.ImmutableCollections$SetN.contains(ImmutableCollections.java:937) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at org.apache.hop.pipeline.TransformWithMappingMeta.isWorkflowInternalVariable(TransformWithMappingMeta.java:333) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at org.apache.hop.pipeline.TransformWithMappingMeta.isInternalVariable(TransformWithMappingMeta.java:324) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at org.apache.hop.pipeline.TransformWithMappingMeta.replaceVariableValues(TransformWithMappingMeta.java:309) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at org.apache.hop.pipeline.TransformWithMappingMeta.replaceVariableValues(TransformWithMappingMeta.java:316) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - at org.apache.hop.pipeline.transforms.kafka.consumer.KafkaConsumerInput.initSubPipeline(KafkaConsumerInput.java:135) 2026/04/30 10:22:42 - Kafka Consumer 2.0 - ... 3 more 2026/04/30 10:22:42 - Kafka Consumer 2.0 - ERROR: Error initializing transform [Kafka Consumer 2] 2026/04/30 10:22:42 - touchpoint_sm
