GitHub user ggrohmann edited a comment on the discussion: Help with metadata
injection for incremental Load
I'm not quite sure, what the issue is, but sounds like you get too many fields
like for the meta injection of the csv files (thus the step/ resulting pipeline
is searching for fields in the file, that don't exist)?
A good way to remove a defined set of columns, but to keep all other columns in
the stream, which you don't want to specify (as they change for every injected,
created pipeline) is the "select values" step.
The injection would somewhat look like this (one row for the file name, n rows
for the fields)
Also, the csv metadata step creates new lines in the stream, in particular one
row for each input column creates one row with meta information of this row
(column type...)
Note, that the fields part contains the selected values step (the important
part ist to tick the "Include unspecified fields"):
Hope that helps
GitHub link:
https://github.com/apache/hop/discussions/5208#discussioncomment-12907363
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
