[jira] [Commented] (MINIFICPP-357) Upgrade YAML parsing to support version 3 of the config schema and work with later toolkits
[ https://issues.apache.org/jira/browse/MINIFICPP-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16319157#comment-16319157 ] ASF GitHub Bot commented on MINIFICPP-357: -- Github user achristianson closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/230 > Upgrade YAML parsing to support version 3 of the config schema and work with > later toolkits > --- > > Key: MINIFICPP-357 > URL: https://issues.apache.org/jira/browse/MINIFICPP-357 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Aldrin Piri >Assignee: Andrew Christianson > Fix For: 0.4.0 > > Attachments: 1.4-Template.xml, 1.5-Template.xml > > > Currently minificpp makes use of what is effectively YAML, v1 and requires > the usage of the 0.0.1 toolkit to perform transformations. We should get > these in sync across agent implementations and allow users to make use of one > toolkit for performing transformations. > Scope involves mapping v3 config to what is currently supported. Not all > features in reference config files (e.g. dynamic properties) are yet > supported in MiNiFi. Support of those features is out of scope. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-357) Upgrade YAML parsing to support version 3 of the config schema and work with later toolkits
[ https://issues.apache.org/jira/browse/MINIFICPP-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16319156#comment-16319156 ] ASF GitHub Bot commented on MINIFICPP-357: -- Github user achristianson commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/230 @apiri sure thing. > Upgrade YAML parsing to support version 3 of the config schema and work with > later toolkits > --- > > Key: MINIFICPP-357 > URL: https://issues.apache.org/jira/browse/MINIFICPP-357 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Aldrin Piri >Assignee: Andrew Christianson > Fix For: 0.4.0 > > Attachments: 1.4-Template.xml, 1.5-Template.xml > > > Currently minificpp makes use of what is effectively YAML, v1 and requires > the usage of the 0.0.1 toolkit to perform transformations. We should get > these in sync across agent implementations and allow users to make use of one > toolkit for performing transformations. > Scope involves mapping v3 config to what is currently supported. Not all > features in reference config files (e.g. dynamic properties) are yet > supported in MiNiFi. Support of those features is out of scope. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-357) Upgrade YAML parsing to support version 3 of the config schema and work with later toolkits
[ https://issues.apache.org/jira/browse/MINIFICPP-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16319106#comment-16319106 ] ASF GitHub Bot commented on MINIFICPP-357: -- Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/230 @achristianson just realized I had a misfire on the number pad. could you please close this PR manually? work was merged in c6f9dc628bd28e5788e2235bfe4b7f7c9000c941 > Upgrade YAML parsing to support version 3 of the config schema and work with > later toolkits > --- > > Key: MINIFICPP-357 > URL: https://issues.apache.org/jira/browse/MINIFICPP-357 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Aldrin Piri >Assignee: Andrew Christianson > Fix For: 0.4.0 > > Attachments: 1.4-Template.xml, 1.5-Template.xml > > > Currently minificpp makes use of what is effectively YAML, v1 and requires > the usage of the 0.0.1 toolkit to perform transformations. We should get > these in sync across agent implementations and allow users to make use of one > toolkit for performing transformations. > Scope involves mapping v3 config to what is currently supported. Not all > features in reference config files (e.g. dynamic properties) are yet > supported in MiNiFi. Support of those features is out of scope. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-357) Upgrade YAML parsing to support version 3 of the config schema and work with later toolkits
[ https://issues.apache.org/jira/browse/MINIFICPP-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16318781#comment-16318781 ] ASF GitHub Bot commented on MINIFICPP-357: -- Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/230 @achristianson think we're all good here. will get this merged in today. thanks > Upgrade YAML parsing to support version 3 of the config schema and work with > later toolkits > --- > > Key: MINIFICPP-357 > URL: https://issues.apache.org/jira/browse/MINIFICPP-357 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Aldrin Piri >Assignee: Andrew Christianson > Fix For: 0.4.0 > > Attachments: 1.4-Template.xml, 1.5-Template.xml > > > Currently minificpp makes use of what is effectively YAML, v1 and requires > the usage of the 0.0.1 toolkit to perform transformations. We should get > these in sync across agent implementations and allow users to make use of one > toolkit for performing transformations. > Scope involves mapping v3 config to what is currently supported. Not all > features in reference config files (e.g. dynamic properties) are yet > supported in MiNiFi. Support of those features is out of scope. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-357) Upgrade YAML parsing to support version 3 of the config schema and work with later toolkits
[
https://issues.apache.org/jira/browse/MINIFICPP-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16318575#comment-16318575
]
ASF GitHub Bot commented on MINIFICPP-357:
--
Github user achristianson commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/230#discussion_r160433655
--- Diff: libminifi/include/core/yaml/YamlConfiguration.h ---
@@ -40,13 +40,18 @@ namespace core {
#define CONFIG_YAML_CONNECTIONS_KEY "Connections"
#define CONFIG_YAML_CONTROLLER_SERVICES_KEY "Controller Services"
#define CONFIG_YAML_REMOTE_PROCESS_GROUP_KEY "Remote Processing Groups"
+#define CONFIG_YAML_REMOTE_PROCESS_GROUP_KEY_V3 "Remote Process Groups"
#define CONFIG_YAML_PROVENANCE_REPORT_KEY "Provenance Reporting"
class YamlConfiguration : public FlowConfiguration {
public:
- explicit YamlConfiguration(std::shared_ptr repo,
std::shared_ptr flow_file_repo,
std::shared_ptr content_repo,
- std::shared_ptr
stream_factory, std::shared_ptr configuration, const std::string
path = DEFAULT_FLOW_YAML_FILE_NAME)
+ explicit YamlConfiguration(std::shared_ptr repo,
--- End diff --
Updated my IDE to 200 and the auto-formatter still does it this way. TBH it
is much more readable to have them on separate lines and conforms with the
Google style guide as we follow. I don't think it's worth blocking the PR over.
> Upgrade YAML parsing to support version 3 of the config schema and work with
> later toolkits
> ---
>
> Key: MINIFICPP-357
> URL: https://issues.apache.org/jira/browse/MINIFICPP-357
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Aldrin Piri
>Assignee: Andrew Christianson
> Fix For: 0.4.0
>
> Attachments: 1.4-Template.xml, 1.5-Template.xml
>
>
> Currently minificpp makes use of what is effectively YAML, v1 and requires
> the usage of the 0.0.1 toolkit to perform transformations. We should get
> these in sync across agent implementations and allow users to make use of one
> toolkit for performing transformations.
> Scope involves mapping v3 config to what is currently supported. Not all
> features in reference config files (e.g. dynamic properties) are yet
> supported in MiNiFi. Support of those features is out of scope.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (MINIFICPP-357) Upgrade YAML parsing to support version 3 of the config schema and work with later toolkits
[ https://issues.apache.org/jira/browse/MINIFICPP-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16317210#comment-16317210 ] ASF GitHub Bot commented on MINIFICPP-357: -- Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/230 Overall this looks okay but uncovered a wrong assumption in the MiNiFi Toolkit that does YAML transformations. I am looking into that a bit to verify a couple of things but can likely move this one ahead. Will follow up and thanks! > Upgrade YAML parsing to support version 3 of the config schema and work with > later toolkits > --- > > Key: MINIFICPP-357 > URL: https://issues.apache.org/jira/browse/MINIFICPP-357 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Aldrin Piri >Assignee: Andrew Christianson > Fix For: 0.4.0 > > Attachments: 1.4-Template.xml, 1.5-Template.xml > > > Currently minificpp makes use of what is effectively YAML, v1 and requires > the usage of the 0.0.1 toolkit to perform transformations. We should get > these in sync across agent implementations and allow users to make use of one > toolkit for performing transformations. > Scope involves mapping v3 config to what is currently supported. Not all > features in reference config files (e.g. dynamic properties) are yet > supported in MiNiFi. Support of those features is out of scope. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-357) Upgrade YAML parsing to support version 3 of the config schema and work with later toolkits
[
https://issues.apache.org/jira/browse/MINIFICPP-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16317185#comment-16317185
]
ASF GitHub Bot commented on MINIFICPP-357:
--
Github user achristianson commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/230#discussion_r160269055
--- Diff: libminifi/include/core/yaml/YamlConfiguration.h ---
@@ -40,13 +40,18 @@ namespace core {
#define CONFIG_YAML_CONNECTIONS_KEY "Connections"
#define CONFIG_YAML_CONTROLLER_SERVICES_KEY "Controller Services"
#define CONFIG_YAML_REMOTE_PROCESS_GROUP_KEY "Remote Processing Groups"
+#define CONFIG_YAML_REMOTE_PROCESS_GROUP_KEY_V3 "Remote Process Groups"
#define CONFIG_YAML_PROVENANCE_REPORT_KEY "Provenance Reporting"
class YamlConfiguration : public FlowConfiguration {
public:
- explicit YamlConfiguration(std::shared_ptr repo,
std::shared_ptr flow_file_repo,
std::shared_ptr content_repo,
- std::shared_ptr
stream_factory, std::shared_ptr configuration, const std::string
path = DEFAULT_FLOW_YAML_FILE_NAME)
+ explicit YamlConfiguration(std::shared_ptr repo,
--- End diff --
Will fix.
> Upgrade YAML parsing to support version 3 of the config schema and work with
> later toolkits
> ---
>
> Key: MINIFICPP-357
> URL: https://issues.apache.org/jira/browse/MINIFICPP-357
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Aldrin Piri
>Assignee: Andrew Christianson
> Fix For: 0.4.0
>
> Attachments: 1.4-Template.xml, 1.5-Template.xml
>
>
> Currently minificpp makes use of what is effectively YAML, v1 and requires
> the usage of the 0.0.1 toolkit to perform transformations. We should get
> these in sync across agent implementations and allow users to make use of one
> toolkit for performing transformations.
> Scope involves mapping v3 config to what is currently supported. Not all
> features in reference config files (e.g. dynamic properties) are yet
> supported in MiNiFi. Support of those features is out of scope.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (MINIFICPP-357) Upgrade YAML parsing to support version 3 of the config schema and work with later toolkits
[
https://issues.apache.org/jira/browse/MINIFICPP-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16317183#comment-16317183
]
ASF GitHub Bot commented on MINIFICPP-357:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/230#discussion_r160268671
--- Diff: libminifi/include/core/yaml/YamlConfiguration.h ---
@@ -40,13 +40,18 @@ namespace core {
#define CONFIG_YAML_CONNECTIONS_KEY "Connections"
#define CONFIG_YAML_CONTROLLER_SERVICES_KEY "Controller Services"
#define CONFIG_YAML_REMOTE_PROCESS_GROUP_KEY "Remote Processing Groups"
+#define CONFIG_YAML_REMOTE_PROCESS_GROUP_KEY_V3 "Remote Process Groups"
#define CONFIG_YAML_PROVENANCE_REPORT_KEY "Provenance Reporting"
class YamlConfiguration : public FlowConfiguration {
public:
- explicit YamlConfiguration(std::shared_ptr repo,
std::shared_ptr flow_file_repo,
std::shared_ptr content_repo,
- std::shared_ptr
stream_factory, std::shared_ptr configuration, const std::string
path = DEFAULT_FLOW_YAML_FILE_NAME)
+ explicit YamlConfiguration(std::shared_ptr repo,
--- End diff --
Did you shorten the line length? I think we abide by 200 characters as per
nifi.
> Upgrade YAML parsing to support version 3 of the config schema and work with
> later toolkits
> ---
>
> Key: MINIFICPP-357
> URL: https://issues.apache.org/jira/browse/MINIFICPP-357
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Aldrin Piri
>Assignee: Andrew Christianson
> Fix For: 0.4.0
>
> Attachments: 1.4-Template.xml, 1.5-Template.xml
>
>
> Currently minificpp makes use of what is effectively YAML, v1 and requires
> the usage of the 0.0.1 toolkit to perform transformations. We should get
> these in sync across agent implementations and allow users to make use of one
> toolkit for performing transformations.
> Scope involves mapping v3 config to what is currently supported. Not all
> features in reference config files (e.g. dynamic properties) are yet
> supported in MiNiFi. Support of those features is out of scope.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (MINIFICPP-357) Upgrade YAML parsing to support version 3 of the config schema and work with later toolkits
[ https://issues.apache.org/jira/browse/MINIFICPP-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16316741#comment-16316741 ] ASF GitHub Bot commented on MINIFICPP-357: -- Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/230 reviewing > Upgrade YAML parsing to support version 3 of the config schema and work with > later toolkits > --- > > Key: MINIFICPP-357 > URL: https://issues.apache.org/jira/browse/MINIFICPP-357 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Aldrin Piri >Assignee: Andrew Christianson > Attachments: 1.4-Template.xml, 1.5-Template.xml > > > Currently minificpp makes use of what is effectively YAML, v1 and requires > the usage of the 0.0.1 toolkit to perform transformations. We should get > these in sync across agent implementations and allow users to make use of one > toolkit for performing transformations. > Scope involves mapping v3 config to what is currently supported. Not all > features in reference config files (e.g. dynamic properties) are yet > supported in MiNiFi. Support of those features is out of scope. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-357) Upgrade YAML parsing to support version 3 of the config schema and work with later toolkits
[ https://issues.apache.org/jira/browse/MINIFICPP-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16316615#comment-16316615 ] ASF GitHub Bot commented on MINIFICPP-357: -- GitHub user achristianson opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/230 MINIFICPP-357 Added support for config YAML v3. Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFICPP-357 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/230.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #230 commit badddf33f9b5ce07d0ffdd887b434bd9e0217334 Author: Andy I. Christianson Date: 2018-01-04T18:00:27Z MINIFICPP-357 Added support for config YAML v3. > Upgrade YAML parsing to support version 3 of the config schema and work with > later toolkits > --- > > Key: MINIFICPP-357 > URL: https://issues.apache.org/jira/browse/MINIFICPP-357 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Aldrin Piri >Assignee: Andrew Christianson > Attachments: 1.4-Template.xml, 1.5-Template.xml > > > Currently minificpp makes use of what is effectively YAML, v1 and requires > the usage of the 0.0.1 toolkit to perform transformations. We should get > these in sync across agent implementations and allow users to make use of one > toolkit for performing transformations. > Scope involves mapping v3 config to what is currently supported. Not all > features in reference config files (e.g. dynamic properties) are yet > supported in MiNiFi. Support of those features is out of scope. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-357) Upgrade YAML parsing to support version 3 of the config schema and work with later toolkits
[ https://issues.apache.org/jira/browse/MINIFICPP-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16310107#comment-16310107 ] Aldrin Piri commented on MINIFICPP-357: --- Should also tackle handling the update to targetIdentifer coming out in 1.5 when available. For the Java implementation this is covered in https://issues.apache.org/jira/browse/MINIFI-421. The client dto libraries in Java handle this process but in C++ we will need to provide this logic. https://issues.apache.org/jira/browse/NIFI-4719 provided an encoding version of 1.2 to identify templates that made use of the new targetIdentifer. If that is the encodingVersion, that is the UUID we should attempt connection. > Upgrade YAML parsing to support version 3 of the config schema and work with > later toolkits > --- > > Key: MINIFICPP-357 > URL: https://issues.apache.org/jira/browse/MINIFICPP-357 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Aldrin Piri >Assignee: Andrew Christianson > > Currently minificpp makes use of what is effectively YAML, v1 and requires > the usage of the 0.0.1 toolkit to perform transformations. We should get > these in sync across agent implementations and allow users to make use of one > toolkit for performing transformations. > Scope involves mapping v3 config to what is currently supported. Not all > features in reference config files (e.g. dynamic properties) are yet > supported in MiNiFi. Support of those features is out of scope. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-357) Upgrade YAML parsing to support version 3 of the config schema and work with later toolkits
[ https://issues.apache.org/jira/browse/MINIFICPP-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16300455#comment-16300455 ] Andrew Christianson commented on MINIFICPP-357: --- Example v3: https://github.com/apache/nifi-minifi/blob/master/minifi-toolkit/minifi-toolkit-configuration/src/test/resources/InvokeHttpMiNiFiTemplateTest.yml > Upgrade YAML parsing to support version 3 of the config schema and work with > later toolkits > --- > > Key: MINIFICPP-357 > URL: https://issues.apache.org/jira/browse/MINIFICPP-357 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Aldrin Piri > > Currently minificpp makes use of what is effectively YAML, v1 and requires > the usage of the 0.0.1 toolkit to perform transformations. We should get > these in sync across agent implementations and allow users to make use of one > toolkit for performing transformations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-357) Upgrade YAML parsing to support version 3 of the config schema and work with later toolkits
[ https://issues.apache.org/jira/browse/MINIFICPP-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16300447#comment-16300447 ] Andrew Christianson commented on MINIFICPP-357: --- Associated links: * https://github.com/apache/nifi-minifi/blob/master/minifi-docs/src/main/markdown/System_Admin_Guide.md#config-file * https://github.com/apache/nifi-minifi/tree/master/minifi-toolkit/minifi-toolkit-configuration is where everything lives * https://github.com/apache/nifi-minifi/tree/master/minifi-toolkit/minifi-toolkit-configuration/src/test/resources has several test files template and back > Upgrade YAML parsing to support version 3 of the config schema and work with > later toolkits > --- > > Key: MINIFICPP-357 > URL: https://issues.apache.org/jira/browse/MINIFICPP-357 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Aldrin Piri > > Currently minificpp makes use of what is effectively YAML, v1 and requires > the usage of the 0.0.1 toolkit to perform transformations. We should get > these in sync across agent implementations and allow users to make use of one > toolkit for performing transformations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
