[GitHub] nifi issue #2711: NIFI-1705 - Adding AttributesToCSV processor
Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/2711 +1 LGTM, thanks all for the reviews and @joetrite for the new feature! I ran unit tests and tried various combinations of options on a live NiFi, everything looks good, merging to master. ---
[GitHub] nifi issue #2711: NIFI-1705 - Adding AttributesToCSV processor
Github user bdesert commented on the issue: https://github.com/apache/nifi/pull/2711 @joetrite , no more comments from my side, thanks for addressing code review comments! @MikeThomsen , LGTM. Could you please give a second look on it now? I believe it's ready, but there are issues with build, which seems not related to this PR. ---
[GitHub] nifi issue #2711: NIFI-1705 - Adding AttributesToCSV processor
Github user joetrite commented on the issue: https://github.com/apache/nifi/pull/2711 @bdesert good catch, we should add the option to push the header to attribute or content based on the selection. The header should also be csv format so that when the header and data are in the output file they will have the same schema. ---
[GitHub] nifi issue #2711: NIFI-1705 - Adding AttributesToCSV processor
Github user bdesert commented on the issue: https://github.com/apache/nifi/pull/2711 @joetrite , @MikeThomsen , So, I tested the processor. It works OK. But I've got a question. In JSON it's easy - json structure will include both attribute name and attribute value. in CSV - there will be only values on the output. Don't you think it would be useful to add a header, or attribute with avro-like generated schema? I agree that specified attributes will be in provided order, but if regex is used - then user won't be able to relate a value to a name. The same is about core attributes - since we add only existing not empty. If we decide to add avro-like schema, we gonna have problems - attribute names could be non Avro-safe. ---
[GitHub] nifi issue #2711: NIFI-1705 - Adding AttributesToCSV processor
Github user joetrite commented on the issue: https://github.com/apache/nifi/pull/2711 @bdesert it looks like the regex support was added to AttributesToJSON after copied the code. I'll add regex support to this processor and check if anything else has changed. ---
[GitHub] nifi issue #2711: NIFI-1705 - Adding AttributesToCSV processor
Github user bdesert commented on the issue: https://github.com/apache/nifi/pull/2711 @joetrite , while AttributesToJSON has regex support for attributes, new AttributesToCSV doesn't have it - is there a reason for not keeping consistency? ---
[GitHub] nifi issue #2711: NIFI-1705 - Adding AttributesToCSV processor
Github user bdesert commented on the issue: https://github.com/apache/nifi/pull/2711 @joetrite , I would like to mention it under the current PR - there was a discussion about having ["record-aware" output](https://github.com/apache/nifi/pull/1589#issuecomment-383639761). This will need to be addressed, but I believe is not part of this PR, as there are use cases when having schema will limit capabilities of this processor. I.e. when list of attributes is empty, it is expected to get all flow file attributes on output, but the list is unknown, so a schema cannot be pre-defined. @mattyb149 , as alternative, we can issue enhancement jira for AttributeToRecord to address schema-specific extracts. ---
