[ https://issues.apache.org/jira/browse/NIFI-1077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14985574#comment-14985574 ]
ASF subversion and git services commented on NIFI-1077: ------------------------------------------------------- Commit a549621267c6517c4774579c3d95f0d7a0c1f68b in nifi's branch refs/heads/master from [~JPercivall] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=a549621 ] NIFI-1077 enabling expression language on ConvertCharacterSet input and output Signed-off-by: Mark Payne <marka...@hotmail.com> > Allow ConvertCharacterSet to accept expression language > ------------------------------------------------------- > > Key: NIFI-1077 > URL: https://issues.apache.org/jira/browse/NIFI-1077 > Project: Apache NiFi > Issue Type: Improvement > Reporter: Joseph Percivall > Priority: Minor > Fix For: 0.4.0 > > Attachments: > NIFI-1077_added_expression_validation_to_charset_validator.patch > > > This issue arose from a user on the mailing list. It demonstrates the need to > be able to use expression language to set the incoming (and potentially > outgoing) character sets: > I'm looking to process many files into common formats. The source files are > coming in various character sets, mime types, and new line terminators. > My thinking for a data flow was along these lines: > GetFile (from many sub directories) -> > ExecuteStreamCommand (file -i) -> > ConvertCharacterSet (from previous command to utf8) -> > ReplaceText (to change any \r\n into \n) -> > PutFile (into a directory structure based on values found in the original > file path and filename) > Additional steps would be added for archiving a copy of the original, > converting xml files, etc. > Attempting to process these with Nifi leaves me confused as to how to process > within the tool. If I want to ConvertCharacterSet, I have to know the input > type. I setup a ExecuteStreamCommand to file -i > ${absolute.path:append(${filename})} which returned the expected values. I > don't see a way to turn these results into input for the processor, which > doesn't accept expression language for that field. > I also considered ConvertCSVToAvro as an interim step but notice the same > issue. Any suggestions what this dataflow should look like? -- This message was sent by Atlassian JIRA (v6.3.4#6332)