[jira] [Commented] (MINIFICPP-403) Enable tagging of flowfiles with flow metadata information in C++
[ https://issues.apache.org/jira/browse/MINIFICPP-403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481144#comment-16481144 ] ASF GitHub Bot commented on MINIFICPP-403: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/313 > Enable tagging of flowfiles with flow metadata information in C++ > - > > Key: MINIFICPP-403 > URL: https://issues.apache.org/jira/browse/MINIFICPP-403 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: bqiu >Assignee: bqiu >Priority: Minor > Fix For: 1.0.0 > > > Provide framework level support to tag flowfiles with metadata about the flow > that created them. > Design proposal > Right now, MiNiFi support core attributes like > // FlowFile Attribute > enum FlowAttribute { > // The flowfile's path indicates the relative directory to which a FlowFile > belongs and does not contain the filename > PATH = 0, > // The flowfile's absolute path indicates the absolute directory to which a > FlowFile belongs and does not contain the filename > ABSOLUTE_PATH, > // The filename of the FlowFile. The filename should not contain any > directory structure. > FILENAME, > // A unique UUID assigned to this FlowFile. > UUID, > // A numeric value indicating the FlowFile priority > priority, > // The MIME Type of this FlowFile > MIME_TYPE, > // Specifies the reason that a FlowFile is being discarded > DISCARD_REASON, > // Indicates an identifier other than the FlowFile's UUID that is known to > refer to this FlowFile. > ALTERNATE_IDENTIFIER, > MAX_FLOW_ATTRIBUTES > }; > So one approach is in the flow YAML file, specific the list of core flow > attributes along with the processors that inject/import/create the flow files. > When flow was created/imported/injected by this processor, we can apply these > core attributes to the new flow. > Also user can define their own core attributes template and EL for populate > value for these core attributes, for example protocol, TTL, record route > (expected route), key, version, etc. > In current implementation, FILENAME, PATH and UUID are required attributes > when flow was created, others are optional > // Populate the default attributes > addKeyedAttribute(FILENAME, > std::to_string(getTimeNano())); > addKeyedAttribute(PATH, DEFAULT_FLOWFILE_PATH); > addKeyedAttribute(UUID, > getUUIDStr()) > So if user specify the optional meta flow info section for the processor with > the key/value pairs as above, MiNiFI will add these key attributes to the > flow when flow was created by this processor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-403) Enable tagging of flowfiles with flow metadata information in C++
[ https://issues.apache.org/jira/browse/MINIFICPP-403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481143#comment-16481143 ] ASF GitHub Bot commented on MINIFICPP-403: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/331 > Enable tagging of flowfiles with flow metadata information in C++ > - > > Key: MINIFICPP-403 > URL: https://issues.apache.org/jira/browse/MINIFICPP-403 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: bqiu >Assignee: bqiu >Priority: Minor > Fix For: 1.0.0 > > > Provide framework level support to tag flowfiles with metadata about the flow > that created them. > Design proposal > Right now, MiNiFi support core attributes like > // FlowFile Attribute > enum FlowAttribute { > // The flowfile's path indicates the relative directory to which a FlowFile > belongs and does not contain the filename > PATH = 0, > // The flowfile's absolute path indicates the absolute directory to which a > FlowFile belongs and does not contain the filename > ABSOLUTE_PATH, > // The filename of the FlowFile. The filename should not contain any > directory structure. > FILENAME, > // A unique UUID assigned to this FlowFile. > UUID, > // A numeric value indicating the FlowFile priority > priority, > // The MIME Type of this FlowFile > MIME_TYPE, > // Specifies the reason that a FlowFile is being discarded > DISCARD_REASON, > // Indicates an identifier other than the FlowFile's UUID that is known to > refer to this FlowFile. > ALTERNATE_IDENTIFIER, > MAX_FLOW_ATTRIBUTES > }; > So one approach is in the flow YAML file, specific the list of core flow > attributes along with the processors that inject/import/create the flow files. > When flow was created/imported/injected by this processor, we can apply these > core attributes to the new flow. > Also user can define their own core attributes template and EL for populate > value for these core attributes, for example protocol, TTL, record route > (expected route), key, version, etc. > In current implementation, FILENAME, PATH and UUID are required attributes > when flow was created, others are optional > // Populate the default attributes > addKeyedAttribute(FILENAME, > std::to_string(getTimeNano())); > addKeyedAttribute(PATH, DEFAULT_FLOWFILE_PATH); > addKeyedAttribute(UUID, > getUUIDStr()) > So if user specify the optional meta flow info section for the processor with > the key/value pairs as above, MiNiFI will add these key attributes to the > flow when flow was created by this processor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-403) Enable tagging of flowfiles with flow metadata information in C++
[ https://issues.apache.org/jira/browse/MINIFICPP-403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476239#comment-16476239 ] ASF GitHub Bot commented on MINIFICPP-403: -- GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/331 MINIFICPP-403: Update connectables so that they contain a reference t… …o the flow identifier. With this approach the flow identifier will be updated with C2 and automatically apply to any processors applied as a result of that update Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-403 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/331.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #331 commit ba075e0e7594ac2855321646dde5ec04d60077b5 Author: Marc ParisiDate: 2018-05-15T17:45:02Z MINIFICPP-403: Update connectables so that they contain a reference to the flow identifier. With this approach the flow identifier will be updated with C2 and automatically apply to any processors applied as a result of that update > Enable tagging of flowfiles with flow metadata information in C++ > - > > Key: MINIFICPP-403 > URL: https://issues.apache.org/jira/browse/MINIFICPP-403 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: bqiu >Assignee: bqiu >Priority: Minor > Fix For: 1.0.0 > > > Provide framework level support to tag flowfiles with metadata about the flow > that created them. > Design proposal > Right now, MiNiFi support core attributes like > // FlowFile Attribute > enum FlowAttribute { > // The flowfile's path indicates the relative directory to which a FlowFile > belongs and does not contain the filename > PATH = 0, > // The flowfile's absolute path indicates the absolute directory to which a > FlowFile belongs and does not contain the filename > ABSOLUTE_PATH, > // The filename of the FlowFile. The filename should not contain any > directory structure. > FILENAME, > // A unique UUID assigned to this FlowFile. > UUID, > // A numeric value indicating the FlowFile priority > priority, > // The MIME Type of this FlowFile > MIME_TYPE, > // Specifies the reason that a FlowFile is being discarded > DISCARD_REASON, > // Indicates an identifier other than the FlowFile's UUID that is known to > refer to this FlowFile. > ALTERNATE_IDENTIFIER, > MAX_FLOW_ATTRIBUTES > }; > So one approach is in the flow YAML file, specific the list of core flow > attributes along with the processors that inject/import/create the flow files. > When flow was created/imported/injected by this processor, we can apply these > core attributes to the new flow. > Also user can define their own core attributes template and EL for populate > value for these core attributes, for example protocol, TTL, record route > (expected route), key, version, etc. > In current implementation, FILENAME, PATH and UUID are required attributes > when flow was created, others are optional > // Populate the default attributes > addKeyedAttribute(FILENAME, > std::to_string(getTimeNano())); > addKeyedAttribute(PATH, DEFAULT_FLOWFILE_PATH); >
[jira] [Commented] (MINIFICPP-403) Enable tagging of flowfiles with flow metadata information in C++
[ https://issues.apache.org/jira/browse/MINIFICPP-403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459801#comment-16459801 ] ASF GitHub Bot commented on MINIFICPP-403: -- Github user minifirocks commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/313#discussion_r185257853 --- Diff: libminifi/test/resources/TestHTTPGet.yml --- @@ -19,6 +19,7 @@ Flow Controller: name: MiNiFi Flow id: 2438e3c8-015a-1000-79ca-83af40ec1990 +version: 1 --- End diff -- @phrocker that's flow config yml version, i thought it refer to the the flow version. when you update the YML file, flow version change also > Enable tagging of flowfiles with flow metadata information in C++ > - > > Key: MINIFICPP-403 > URL: https://issues.apache.org/jira/browse/MINIFICPP-403 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: bqiu >Assignee: bqiu >Priority: Minor > Fix For: 1.0.0 > > > Provide framework level support to tag flowfiles with metadata about the flow > that created them. > Design proposal > Right now, MiNiFi support core attributes like > // FlowFile Attribute > enum FlowAttribute { > // The flowfile's path indicates the relative directory to which a FlowFile > belongs and does not contain the filename > PATH = 0, > // The flowfile's absolute path indicates the absolute directory to which a > FlowFile belongs and does not contain the filename > ABSOLUTE_PATH, > // The filename of the FlowFile. The filename should not contain any > directory structure. > FILENAME, > // A unique UUID assigned to this FlowFile. > UUID, > // A numeric value indicating the FlowFile priority > priority, > // The MIME Type of this FlowFile > MIME_TYPE, > // Specifies the reason that a FlowFile is being discarded > DISCARD_REASON, > // Indicates an identifier other than the FlowFile's UUID that is known to > refer to this FlowFile. > ALTERNATE_IDENTIFIER, > MAX_FLOW_ATTRIBUTES > }; > So one approach is in the flow YAML file, specific the list of core flow > attributes along with the processors that inject/import/create the flow files. > When flow was created/imported/injected by this processor, we can apply these > core attributes to the new flow. > Also user can define their own core attributes template and EL for populate > value for these core attributes, for example protocol, TTL, record route > (expected route), key, version, etc. > In current implementation, FILENAME, PATH and UUID are required attributes > when flow was created, others are optional > // Populate the default attributes > addKeyedAttribute(FILENAME, > std::to_string(getTimeNano())); > addKeyedAttribute(PATH, DEFAULT_FLOWFILE_PATH); > addKeyedAttribute(UUID, > getUUIDStr()) > So if user specify the optional meta flow info section for the processor with > the key/value pairs as above, MiNiFI will add these key attributes to the > flow when flow was created by this processor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-403) Enable tagging of flowfiles with flow metadata information in C++
[ https://issues.apache.org/jira/browse/MINIFICPP-403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459797#comment-16459797 ] ASF GitHub Bot commented on MINIFICPP-403: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/313#discussion_r185256992 --- Diff: libminifi/test/resources/TestHTTPGet.yml --- @@ -19,6 +19,7 @@ Flow Controller: name: MiNiFi Flow id: 2438e3c8-015a-1000-79ca-83af40ec1990 +version: 1 --- End diff -- Isn't this the flow config yaml version and not the flow version ? Is the version above intended to be the config yaml version or the flow version? > Enable tagging of flowfiles with flow metadata information in C++ > - > > Key: MINIFICPP-403 > URL: https://issues.apache.org/jira/browse/MINIFICPP-403 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: bqiu >Assignee: bqiu >Priority: Minor > Fix For: 1.0.0 > > > Provide framework level support to tag flowfiles with metadata about the flow > that created them. > Design proposal > Right now, MiNiFi support core attributes like > // FlowFile Attribute > enum FlowAttribute { > // The flowfile's path indicates the relative directory to which a FlowFile > belongs and does not contain the filename > PATH = 0, > // The flowfile's absolute path indicates the absolute directory to which a > FlowFile belongs and does not contain the filename > ABSOLUTE_PATH, > // The filename of the FlowFile. The filename should not contain any > directory structure. > FILENAME, > // A unique UUID assigned to this FlowFile. > UUID, > // A numeric value indicating the FlowFile priority > priority, > // The MIME Type of this FlowFile > MIME_TYPE, > // Specifies the reason that a FlowFile is being discarded > DISCARD_REASON, > // Indicates an identifier other than the FlowFile's UUID that is known to > refer to this FlowFile. > ALTERNATE_IDENTIFIER, > MAX_FLOW_ATTRIBUTES > }; > So one approach is in the flow YAML file, specific the list of core flow > attributes along with the processors that inject/import/create the flow files. > When flow was created/imported/injected by this processor, we can apply these > core attributes to the new flow. > Also user can define their own core attributes template and EL for populate > value for these core attributes, for example protocol, TTL, record route > (expected route), key, version, etc. > In current implementation, FILENAME, PATH and UUID are required attributes > when flow was created, others are optional > // Populate the default attributes > addKeyedAttribute(FILENAME, > std::to_string(getTimeNano())); > addKeyedAttribute(PATH, DEFAULT_FLOWFILE_PATH); > addKeyedAttribute(UUID, > getUUIDStr()) > So if user specify the optional meta flow info section for the processor with > the key/value pairs as above, MiNiFI will add these key attributes to the > flow when flow was created by this processor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-403) Enable tagging of flowfiles with flow metadata information in C++
[ https://issues.apache.org/jira/browse/MINIFICPP-403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455601#comment-16455601 ] ASF GitHub Bot commented on MINIFICPP-403: -- GitHub user minifirocks opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/313 MINIFICPP-403: Add version into flow attributes Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/minifirocks/nifi-minifi-cpp add_version_to_flow Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/313.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #313 > Enable tagging of flowfiles with flow metadata information in C++ > - > > Key: MINIFICPP-403 > URL: https://issues.apache.org/jira/browse/MINIFICPP-403 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: bqiu >Assignee: bqiu >Priority: Minor > Fix For: 1.0.0 > > > Provide framework level support to tag flowfiles with metadata about the flow > that created them. > Design proposal > Right now, MiNiFi support core attributes like > // FlowFile Attribute > enum FlowAttribute { > // The flowfile's path indicates the relative directory to which a FlowFile > belongs and does not contain the filename > PATH = 0, > // The flowfile's absolute path indicates the absolute directory to which a > FlowFile belongs and does not contain the filename > ABSOLUTE_PATH, > // The filename of the FlowFile. The filename should not contain any > directory structure. > FILENAME, > // A unique UUID assigned to this FlowFile. > UUID, > // A numeric value indicating the FlowFile priority > priority, > // The MIME Type of this FlowFile > MIME_TYPE, > // Specifies the reason that a FlowFile is being discarded > DISCARD_REASON, > // Indicates an identifier other than the FlowFile's UUID that is known to > refer to this FlowFile. > ALTERNATE_IDENTIFIER, > MAX_FLOW_ATTRIBUTES > }; > So one approach is in the flow YAML file, specific the list of core flow > attributes along with the processors that inject/import/create the flow files. > When flow was created/imported/injected by this processor, we can apply these > core attributes to the new flow. > Also user can define their own core attributes template and EL for populate > value for these core attributes, for example protocol, TTL, record route > (expected route), key, version, etc. > In current implementation, FILENAME, PATH and UUID are required attributes > when flow was created, others are optional > // Populate the default attributes > addKeyedAttribute(FILENAME, > std::to_string(getTimeNano())); > addKeyedAttribute(PATH, DEFAULT_FLOWFILE_PATH); > addKeyedAttribute(UUID, > getUUIDStr()) > So if user specify the optional meta flow info section for the processor with > the key/value pairs as above, MiNiFI will add these key attributes to the > flow when flow was created by this processor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)