[jira] [Commented] (NIFI-642) Eliminate hardcoded HDFS compression codec classnames.
[ https://issues.apache.org/jira/browse/NIFI-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14595776#comment-14595776 ] Tim Reardon commented on NIFI-642: -- Agreed, a single property makes sense. Similarly, I'm not sure that the property I added on PutHDFS to append the file extension makes sense, as I can't envision a use case where one would compress yet not want the file extension added. The extension should be added whenever compression is used. Eliminate hardcoded HDFS compression codec classnames. -- Key: NIFI-642 URL: https://issues.apache.org/jira/browse/NIFI-642 Project: Apache NiFi Issue Type: Improvement Reporter: Tim Reardon Priority: Minor Fix For: 0.2.0 Available HDFS compression codec classes were hardcoded as part of NIFI-600. This ticket will allow PutHDFS to discover the available codecs via CompressionCodecFactory, and allow GetHDFS to choose the codec to use based on file extension. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-642) Eliminate hardcoded HDFS compression codec classnames.
[ https://issues.apache.org/jira/browse/NIFI-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14596354#comment-14596354 ] Mark Payne commented on NIFI-642: - Thanks, Tim. Will close this ticket out. Eliminate hardcoded HDFS compression codec classnames. -- Key: NIFI-642 URL: https://issues.apache.org/jira/browse/NIFI-642 Project: Apache NiFi Issue Type: Improvement Reporter: Tim Reardon Priority: Minor Fix For: 0.2.0 Available HDFS compression codec classes were hardcoded as part of NIFI-600. This ticket will allow PutHDFS to discover the available codecs via CompressionCodecFactory, and allow GetHDFS to choose the codec to use based on file extension. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-642) Eliminate hardcoded HDFS compression codec classnames.
[ https://issues.apache.org/jira/browse/NIFI-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14595898#comment-14595898 ] Mark Payne commented on NIFI-642: - I can't argue there. It's important to remember, too, when implementing this that the associated Provenance SEND event needs to also indicate the filename that was used to send the data to HDFS. Eliminate hardcoded HDFS compression codec classnames. -- Key: NIFI-642 URL: https://issues.apache.org/jira/browse/NIFI-642 Project: Apache NiFi Issue Type: Improvement Reporter: Tim Reardon Priority: Minor Fix For: 0.2.0 Available HDFS compression codec classes were hardcoded as part of NIFI-600. This ticket will allow PutHDFS to discover the available codecs via CompressionCodecFactory, and allow GetHDFS to choose the codec to use based on file extension. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-642) Eliminate hardcoded HDFS compression codec classnames.
[ https://issues.apache.org/jira/browse/NIFI-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591697#comment-14591697 ] Mark Payne commented on NIFI-642: - [~treardon] I agree, I like the addition to the GetHDFS processor and the idea of moving them to a separate ticket. I wonder if we need two properties though. I can't envision a use case where we would say Yes, decompress, but leave the file extension or No, leave the file compressed, but remove the extension. I feel like those two should be combined into a single property that just specifies whether or not the content should automatically be decompressed. Thoughts on that? Eliminate hardcoded HDFS compression codec classnames. -- Key: NIFI-642 URL: https://issues.apache.org/jira/browse/NIFI-642 Project: Apache NiFi Issue Type: Improvement Reporter: Tim Reardon Priority: Minor Fix For: 0.2.0 Available HDFS compression codec classes were hardcoded as part of NIFI-600. This ticket will allow PutHDFS to discover the available codecs via CompressionCodecFactory, and allow GetHDFS to choose the codec to use based on file extension. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-642) Eliminate hardcoded HDFS compression codec classnames.
[ https://issues.apache.org/jira/browse/NIFI-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580857#comment-14580857 ] Mark Payne commented on NIFI-642: - Hi Tim, Thanks for contributing this. Can you explain the motivation here? My guess is that the intent is to allow arbitrary compression mechanisms that were not originally part of the Allowable Values? The problem that I have with this approach is that the user now has to know the fully qualified classname of the desired compression codec. This might be okay for a hadoop developer, but many of the users of NiFi are not developers at all but have more sys admin/operations types of roles. I don't know if requiring the user to come up with the appropriate classname is worth the tradeoff. It is easy enough to simply update the Allowable Values to include new compression codecs as needed. Do others have thoughts on this? Thanks -Mark Eliminate hardcoded HDFS compression codec classnames. -- Key: NIFI-642 URL: https://issues.apache.org/jira/browse/NIFI-642 Project: Apache NiFi Issue Type: Improvement Reporter: Tim Reardon Priority: Minor Fix For: 0.2.0 Available HDFS compression codec classes were hardcoded as part of NIFI-600. This ticket will allow PutHDFS to discover the available codecs via CompressionCodecFactory, and allow GetHDFS to choose the codec to use based on file extension. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
