[ https://issues.apache.org/jira/browse/NIFI-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14991882#comment-14991882 ]
Mark Payne commented on NIFI-1081: ---------------------------------- [~JPercivall] - Reviewing code for this patch. I have a few comments/feedback: * Property names should use Title Casing rather than only capitalizing the first word. * I would avoid using two properties to indicate whether or not to use an attribute and the attribute name. I would instead just use a property to indicate which attribute to use and if none is given write to FlowFIle content (the default behavior). * For the "Max size to put in attribute" - would recommend changing name to "Max Attribute Size" or something along those lines. I feel "Max size to put in attribute" is a bit wordy. Also, it has a Positive Integer validator, but we're talking about a data size here, not a number. Should be using the DATA_SIZE_VALIDATOR with a default value of "256 B" instead of "256". Pretty minor changes suggested, I think. Otherwise it looks good! Thanks -Mark > Add option to ExecuteStreamCommand to put value of execution to an attribute > ---------------------------------------------------------------------------- > > Key: NIFI-1081 > URL: https://issues.apache.org/jira/browse/NIFI-1081 > Project: Apache NiFi > Issue Type: Test > Reporter: Joseph Percivall > Assignee: Joseph Percivall > Priority: Minor > Fix For: 0.4.0 > > Attachments: ExecuteStreamCommandTester.xml, > NIFI-1081_fix_content_claim.patch > > > This issue arose from a user on the mailing list. It demonstrates the need to > be able to put the output of ExecuteStreamCommand to an attribute: > I'm looking to process many files into common formats. The source files are > coming in various character sets, mime types, and new line terminators. > My thinking for a data flow was along these lines: > GetFile (from many sub directories) -> > ExecuteStreamCommand (file -i) -> > ConvertCharacterSet (from previous command to utf8) -> > ReplaceText (to change any \r\n into \n) -> > PutFile (into a directory structure based on values found in the original > file path and filename) > Additional steps would be added for archiving a copy of the original, > converting xml files, etc. > Attempting to process these with Nifi leaves me confused as to how to process > within the tool. If I want to ConvertCharacterSet, I have to know the input > type. I setup a ExecuteStreamCommand to file -i > ${absolute.path:append(${filename})} which returned the expected values. I > don't see a way to turn these results into input for the processor, which > doesn't accept expression language for that field. > I also considered ConvertCSVToAvro as an interim step but notice the same > issue. Any suggestions what this dataflow should look like? > Bryan Bende's response: > One problem with the above flow is that ExecuteStreamCommand will replace the > contents of the FlowFile with the results of the command, so the FlowFIle > will have the encoding value and no longer have the original content. -- This message was sent by Atlassian JIRA (v6.3.4#6332)