Hi Hans, Thanks for your response.
Just to clarify—my question is not about the merging itself, but rather what happens *after* the merge. The issue arises from how duplicated field names are handled in the downstream transforms. Specifically, the merged output shows duplicated fields with a _1 suffix (as expected) when you use *"Show output fields"* or *"Get fields"* in the downstream transform. However, when the pipeline runs, it throws an error saying the field with the _1 suffix cannot be found. This suggests a mismatch between the metadata inferred during pipeline design and what is actually available during execution. Thanks, Maria On Thu, Jun 26, 2025 at 5:23 PM Hans Van Akelyen <[email protected]> wrote: > Hi Maria, > > It’s actually something that should not have worked in PDI either. It’s by > accident that it does. You get giant warnings when merging 2 streams that > do not line up. So this will be an opportunity to clean up those errors. > > But you have me a bit confused about the duplicate field name, the error > in the ticket might be valid that there is something wrong in a newly > created field resulting in _1. But that is not related to merging 2 streams. > > > Kr, > Hans > > On Thu, 26 Jun 2025 at 09:01, Maria Lim <[email protected]> wrote: > >> Thanks Hans. >> >> Do you mean, a redesign of the pipeline is inevitable for this case? Do >> you know if this is a known issue (duplicate field name not correctly >> passed) for migrated PDI job only, or for all HOP pipelines? >> >> Kind regards, >> Maria >> >> On Thu, Jun 26, 2025 at 4:27 PM Hans Van Akelyen < >> [email protected]> wrote: >> >>> Hi Maria, >>> >>> By default when merging streams the layout should be the same on both >>> sides. Alternatively when merging 2 streams with different lay-outs you can >>> use the stream schema merge transform >>> >>> https://hop.apache.org//manual/latest/pipeline/transforms/streamschemamerge.html >>> >>> This transform will make sure the result is valid. >>> >>> Kr, >>> Hans >>> >>> >>> On Thu, 26 Jun 2025 at 07:39, Maria Lim <[email protected]> wrote: >>> >>>> Hi Apache HOP Support Team, >>>> >>>> I’m currently exploring the migration of ETLs from PDI to Apache HOP. >>>> >>>> While testing a pipeline, I encountered a failure at the "Select >>>> values" transform following the merging of two streams. The error >>>> indicates that a field with the suffix "_1" could not be found—this >>>> corresponds to a duplicated fieldname resulting from the merge. >>>> >>>> The issue appears to align with the one reported on GitHub last year: >>>> https://github.com/apache/hop/issues/3849 >>>> >>>> After reviewing that discussion, I attempted to rebuild the pipeline >>>> entirely in HOP, hoping for a different result. Unfortunately, the >>>> issue persists. >>>> >>>> Aside from known workarounds like renaming fields prior to the merge >>>> to avoid duplication, is there a more direct or quicker fix available >>>> for this behaviour? >>>> >>>> Appreciate your insights and support. >>>> >>>> Kind regards, >>>> Maria >>>> >>>
