[jira] [Updated] (NIFI-4652) NiFi pom profiles use of activeByDefault does not work as expected and many maven plugin versions are very out of date

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-4652:
--
Attachment: (was: 0004.NIFI-4652.patch)

> NiFi pom profiles use of activeByDefault does not work as expected and many 
> maven plugin versions are very out of date
> --
>
> Key: NIFI-4652
> URL: https://issues.apache.org/jira/browse/NIFI-4652
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Witt
>Assignee: Joseph Witt
> Fix For: 1.5.0
>
>
> In working NIFI-4645 we've found profiles dont' quite work as expected due to 
> the maven behavior for profiles with 'activeByDefault' being automatically 
> disabled anytime other methods of activation activate any other profile 
> within the same pom.  This leads to builds not working as we expect.  Need to 
> audit, improve, and document a new profile management strategy.
> Further, in reviewing this it is found that many many of our maven plugins 
> are very old/out of date and can benefit from being brought up to date 
> including the org.apache:apache version (latest is 18 and released more than 
> a year ago) and maven compiler which we use 3.2 and latest is 3.7.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-4652) NiFi pom profiles use of activeByDefault does not work as expected and many maven plugin versions are very out of date

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-4652:
--
Attachment: (was: 0003-NIFI-4652.patch)

> NiFi pom profiles use of activeByDefault does not work as expected and many 
> maven plugin versions are very out of date
> --
>
> Key: NIFI-4652
> URL: https://issues.apache.org/jira/browse/NIFI-4652
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Witt
>Assignee: Joseph Witt
> Fix For: 1.5.0
>
>
> In working NIFI-4645 we've found profiles dont' quite work as expected due to 
> the maven behavior for profiles with 'activeByDefault' being automatically 
> disabled anytime other methods of activation activate any other profile 
> within the same pom.  This leads to builds not working as we expect.  Need to 
> audit, improve, and document a new profile management strategy.
> Further, in reviewing this it is found that many many of our maven plugins 
> are very old/out of date and can benefit from being brought up to date 
> including the org.apache:apache version (latest is 18 and released more than 
> a year ago) and maven compiler which we use 3.2 and latest is 3.7.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-4652) NiFi pom profiles use of activeByDefault does not work as expected and many maven plugin versions are very out of date

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-4652:
--
Attachment: (was: 
0001-NIFI-4652-updating-to-latest-apache-version-and-upda.patch)

> NiFi pom profiles use of activeByDefault does not work as expected and many 
> maven plugin versions are very out of date
> --
>
> Key: NIFI-4652
> URL: https://issues.apache.org/jira/browse/NIFI-4652
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Witt
>Assignee: Joseph Witt
> Fix For: 1.5.0
>
>
> In working NIFI-4645 we've found profiles dont' quite work as expected due to 
> the maven behavior for profiles with 'activeByDefault' being automatically 
> disabled anytime other methods of activation activate any other profile 
> within the same pom.  This leads to builds not working as we expect.  Need to 
> audit, improve, and document a new profile management strategy.
> Further, in reviewing this it is found that many many of our maven plugins 
> are very old/out of date and can benefit from being brought up to date 
> including the org.apache:apache version (latest is 18 and released more than 
> a year ago) and maven compiler which we use 3.2 and latest is 3.7.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-4652) NiFi pom profiles use of activeByDefault does not work as expected and many maven plugin versions are very out of date

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-4652:
--
Attachment: (was: 0002-NIFI-4652-checkpoint.patch)

> NiFi pom profiles use of activeByDefault does not work as expected and many 
> maven plugin versions are very out of date
> --
>
> Key: NIFI-4652
> URL: https://issues.apache.org/jira/browse/NIFI-4652
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Witt
>Assignee: Joseph Witt
> Fix For: 1.5.0
>
>
> In working NIFI-4645 we've found profiles dont' quite work as expected due to 
> the maven behavior for profiles with 'activeByDefault' being automatically 
> disabled anytime other methods of activation activate any other profile 
> within the same pom.  This leads to builds not working as we expect.  Need to 
> audit, improve, and document a new profile management strategy.
> Further, in reviewing this it is found that many many of our maven plugins 
> are very old/out of date and can benefit from being brought up to date 
> including the org.apache:apache version (latest is 18 and released more than 
> a year ago) and maven compiler which we use 3.2 and latest is 3.7.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4645) NiFi build on powerpc fails because the google/rpc nar cannot be built/used there

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276351#comment-16276351
 ] 

Joseph Witt commented on NIFI-4645:
---

the work in https://github.com/apache/nifi/pull/2313 takes care of this JIRA.  
Once NIFI-4652 is reviewed/closed then this JIRA is resolved as well.

> NiFi build on powerpc fails because the google/rpc nar cannot be built/used 
> there
> -
>
> Key: NIFI-4645
> URL: https://issues.apache.org/jira/browse/NIFI-4645
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
> Environment: ibm/power pc
>Reporter: Joseph Witt
>Assignee: Joseph Witt
> Fix For: 1.5.0
>
>
> Going to update the poms to detect the os family and if not powerpc will 
> activate module builds and assembly usage of nars that are not capable in 
> that environment.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2311: NIFI-4645: Remove activation rules for including the grpc ...

[joewitt]

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2311
  
this should probably be closed in favor of the work under 
https://github.com/apache/nifi/pull/2313


---

[jira] [Commented] (NIFI-4645) NiFi build on powerpc fails because the google/rpc nar cannot be built/used there

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276350#comment-16276350
 ] 

ASF GitHub Bot commented on NIFI-4645:
--

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2311
  
this should probably be closed in favor of the work under 
https://github.com/apache/nifi/pull/2313


> NiFi build on powerpc fails because the google/rpc nar cannot be built/used 
> there
> -
>
> Key: NIFI-4645
> URL: https://issues.apache.org/jira/browse/NIFI-4645
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
> Environment: ibm/power pc
>Reporter: Joseph Witt
>Assignee: Joseph Witt
> Fix For: 1.5.0
>
>
> Going to update the poms to detect the os family and if not powerpc will 
> activate module builds and assembly usage of nars that are not capable in 
> that environment.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-4652) NiFi pom profiles use of activeByDefault does not work as expected and many maven plugin versions are very out of date

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-4652:
--
Attachment: 0004.NIFI-4652.patch

> NiFi pom profiles use of activeByDefault does not work as expected and many 
> maven plugin versions are very out of date
> --
>
> Key: NIFI-4652
> URL: https://issues.apache.org/jira/browse/NIFI-4652
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Witt
>Assignee: Joseph Witt
> Fix For: 1.5.0
>
> Attachments: 
> 0001-NIFI-4652-updating-to-latest-apache-version-and-upda.patch, 
> 0002-NIFI-4652-checkpoint.patch, 0003-NIFI-4652.patch, 0004.NIFI-4652.patch
>
>
> In working NIFI-4645 we've found profiles dont' quite work as expected due to 
> the maven behavior for profiles with 'activeByDefault' being automatically 
> disabled anytime other methods of activation activate any other profile 
> within the same pom.  This leads to builds not working as we expect.  Need to 
> audit, improve, and document a new profile management strategy.
> Further, in reviewing this it is found that many many of our maven plugins 
> are very old/out of date and can benefit from being brought up to date 
> including the org.apache:apache version (latest is 18 and released more than 
> a year ago) and maven compiler which we use 3.2 and latest is 3.7.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-4652) NiFi pom profiles use of activeByDefault does not work as expected and many maven plugin versions are very out of date

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-4652:
--
Attachment: 0003-NIFI-4652.patch

> NiFi pom profiles use of activeByDefault does not work as expected and many 
> maven plugin versions are very out of date
> --
>
> Key: NIFI-4652
> URL: https://issues.apache.org/jira/browse/NIFI-4652
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Witt
>Assignee: Joseph Witt
> Fix For: 1.5.0
>
> Attachments: 
> 0001-NIFI-4652-updating-to-latest-apache-version-and-upda.patch, 
> 0002-NIFI-4652-checkpoint.patch, 0003-NIFI-4652.patch
>
>
> In working NIFI-4645 we've found profiles dont' quite work as expected due to 
> the maven behavior for profiles with 'activeByDefault' being automatically 
> disabled anytime other methods of activation activate any other profile 
> within the same pom.  This leads to builds not working as we expect.  Need to 
> audit, improve, and document a new profile management strategy.
> Further, in reviewing this it is found that many many of our maven plugins 
> are very old/out of date and can benefit from being brought up to date 
> including the org.apache:apache version (latest is 18 and released more than 
> a year ago) and maven compiler which we use 3.2 and latest is 3.7.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4652) NiFi pom profiles use of activeByDefault does not work as expected and many maven plugin versions are very out of date

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276268#comment-16276268
 ] 

Joseph Witt commented on NIFI-4652:
---

Updating to latest apache and checkstyle versions reveals a number of lurking 
checkstyle issues we had that were simply not getting executed. Updating those.



> NiFi pom profiles use of activeByDefault does not work as expected and many 
> maven plugin versions are very out of date
> --
>
> Key: NIFI-4652
> URL: https://issues.apache.org/jira/browse/NIFI-4652
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Witt
>Assignee: Joseph Witt
> Fix For: 1.5.0
>
> Attachments: 
> 0001-NIFI-4652-updating-to-latest-apache-version-and-upda.patch, 
> 0002-NIFI-4652-checkpoint.patch
>
>
> In working NIFI-4645 we've found profiles dont' quite work as expected due to 
> the maven behavior for profiles with 'activeByDefault' being automatically 
> disabled anytime other methods of activation activate any other profile 
> within the same pom.  This leads to builds not working as we expect.  Need to 
> audit, improve, and document a new profile management strategy.
> Further, in reviewing this it is found that many many of our maven plugins 
> are very old/out of date and can benefit from being brought up to date 
> including the org.apache:apache version (latest is 18 and released more than 
> a year ago) and maven compiler which we use 3.2 and latest is 3.7.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3709) Export NiFi flow dataset lineage to Apache Atlas

[Koji Kawamura (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276267#comment-16276267
 ] 

Koji Kawamura commented on NIFI-3709:
-

Until NIFI-4564 is addressed, S2S RAW protocol will not be reported in Atlas 
lineage.

> Export NiFi flow dataset lineage to Apache Atlas
> 
>
> Key: NIFI-3709
> URL: https://issues.apache.org/jira/browse/NIFI-3709
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>
> While Apache NiFi has provenance and event level lineage support within its 
> data flow, Apache Atlas also does manage lineage between dataset and process 
> those interacting with such data. 
> It would be beneficial for users who use both NiFi and Atlas and if they can 
> see end-to-end data lineage on Atlas lineage graph, as some type of dataset 
> are processed by both NiFi and technologies around Atlas such as Storm, 
> Falcon or Sqoop. For example, Kafka topics and Hive tables.
> In order to make this integration happen, I propose a NiFi reporting task 
> that analyzes NiFi flow then creates DataSet and Process entities in Atlas.
> The challenge is how to design NiFi flow dataset level lineage within Atlas 
> lineage graph.
> If we just add a single NiFi process and connect every DataSet from/to it, it 
> would be too ambiguous since it won't be clear which part of a NiFi flow 
> actually interact with certain dataset.
> But if we put every NiFi processor as independent process in Atlas, it would 
> be too granular, too. Also, we already have detailed event level lineage in 
> NiFi, we wouldn't need the same level in Atlas.
> If we can group certain processors in a NiFI flow as a process in Atlas, it 
> would be a nice granularity.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Issue Comment Deleted] (NIFI-4654) RAW S2S transit URI should contain Port ID instead of FlowFile ID

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-4654:
--
Comment: was deleted

(was: Updating to latest apache and checkstyle versions reveals a number of 
lurking checkstyle issues we had that were simply not getting executed.  
Updating those.)

> RAW S2S transit URI should contain Port ID instead of FlowFile ID
> -
>
> Key: NIFI-4654
> URL: https://issues.apache.org/jira/browse/NIFI-4654
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>
> In order to track where a FlowFile is sent to or received from, a provenance 
> event record has to have target RemoteGroupPort ID (if it's S2S client) or 
> RootGroupPort ID (if it's S2S server) in it.
> RemoteGroupPort used to have its component UUID as the same UUID as 
> corresponding RootGroupPort's, but NIFI-3155 changed each RemoteGroupPort 
> instance to have different UUIDs.
> HTTP protocol still contains port uuid in transit URI. But RAW does not 
> contain a port identifier neither in transit URI nor provenance event record 
> any longer. 
> Currently, each S2S protocol generates provenance event with transit URI as 
> follows:
> * RAW
> ** nifi://remote-host:remote-port/flowfile-uuid
> ** E.g. nifi://nifi1.example.com:8081/cb729f05-b2ee-4488-909d-c6696cc34588
> * HTTP
> ** 
> http(s)://homote-host:remote-port/nifi-api/data-transfer/in|output-ports/portId/transactions/txId/flow-files
> ** E.g. 
> http://nifi1.example.coml:8080/nifi-api/data-transfer/input-ports/35dbc0ab-015e-1000-144c-a8d71255027d/transactions/89335043-f105-4de7-a0ef-46f2ef0c7c51/flow-files
> Also provenance events created when a FlowFile is received from remotely have 
> 'Source FlowFile Id' field which contains FlowFile UUID in remote instance.
> This JIRA changes RAW S2S protocol to embed Port ID instead of FlowFile ID 
> (which is available in Source FlowFile Id if needed).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4654) RAW S2S transit URI should contain Port ID instead of FlowFile ID

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276266#comment-16276266
 ] 

Joseph Witt commented on NIFI-4654:
---

Updating to latest apache and checkstyle versions reveals a number of lurking 
checkstyle issues we had that were simply not getting executed.  Updating those.

> RAW S2S transit URI should contain Port ID instead of FlowFile ID
> -
>
> Key: NIFI-4654
> URL: https://issues.apache.org/jira/browse/NIFI-4654
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>
> In order to track where a FlowFile is sent to or received from, a provenance 
> event record has to have target RemoteGroupPort ID (if it's S2S client) or 
> RootGroupPort ID (if it's S2S server) in it.
> RemoteGroupPort used to have its component UUID as the same UUID as 
> corresponding RootGroupPort's, but NIFI-3155 changed each RemoteGroupPort 
> instance to have different UUIDs.
> HTTP protocol still contains port uuid in transit URI. But RAW does not 
> contain a port identifier neither in transit URI nor provenance event record 
> any longer. 
> Currently, each S2S protocol generates provenance event with transit URI as 
> follows:
> * RAW
> ** nifi://remote-host:remote-port/flowfile-uuid
> ** E.g. nifi://nifi1.example.com:8081/cb729f05-b2ee-4488-909d-c6696cc34588
> * HTTP
> ** 
> http(s)://homote-host:remote-port/nifi-api/data-transfer/in|output-ports/portId/transactions/txId/flow-files
> ** E.g. 
> http://nifi1.example.coml:8080/nifi-api/data-transfer/input-ports/35dbc0ab-015e-1000-144c-a8d71255027d/transactions/89335043-f105-4de7-a0ef-46f2ef0c7c51/flow-files
> Also provenance events created when a FlowFile is received from remotely have 
> 'Source FlowFile Id' field which contains FlowFile UUID in remote instance.
> This JIRA changes RAW S2S protocol to embed Port ID instead of FlowFile ID 
> (which is available in Source FlowFile Id if needed).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-49) NiFi Expression Language support

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-49?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276258#comment-16276258
 ] 

ASF GitHub Bot commented on MINIFICPP-49:
-

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/188


> NiFi Expression Language support
> 
>
> Key: MINIFICPP-49
> URL: https://issues.apache.org/jira/browse/MINIFICPP-49
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Jeremy Dyer
>Assignee: Andrew Christianson
>
> An implementation of NiFi's expression language is needed for the cpp version 
> of the project. The feature should support all of the same EL syntax that 
> NiFi supports so that any flow developed in NiFi and then ran through the 
> minifi-toolkit will generate a valid .yml configuration file. Currently if 
> there are any expression language values in the flow.yml file the agent will 
> not act as anticipated or crash.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi-minifi-cpp pull request #188: MINIFICPP-49 Added initial implementation...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/188


---

[jira] [Commented] (NIFI-4654) RAW S2S transit URI should contain Port ID instead of FlowFile ID

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276254#comment-16276254
 ] 

Joseph Witt commented on NIFI-4654:
---

If we need to create a new version of the s2s raw protocol we could explore 
that.

But we cannot just change the behavior/semantics of the values in an existing 
protocol.  I am certain there are users that depend on the old behavior and it 
meaning that it was a flowfile id.

Thanks

> RAW S2S transit URI should contain Port ID instead of FlowFile ID
> -
>
> Key: NIFI-4654
> URL: https://issues.apache.org/jira/browse/NIFI-4654
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>
> In order to track where a FlowFile is sent to or received from, a provenance 
> event record has to have target RemoteGroupPort ID (if it's S2S client) or 
> RootGroupPort ID (if it's S2S server) in it.
> RemoteGroupPort used to have its component UUID as the same UUID as 
> corresponding RootGroupPort's, but NIFI-3155 changed each RemoteGroupPort 
> instance to have different UUIDs.
> HTTP protocol still contains port uuid in transit URI. But RAW does not 
> contain a port identifier neither in transit URI nor provenance event record 
> any longer. 
> Currently, each S2S protocol generates provenance event with transit URI as 
> follows:
> * RAW
> ** nifi://remote-host:remote-port/flowfile-uuid
> ** E.g. nifi://nifi1.example.com:8081/cb729f05-b2ee-4488-909d-c6696cc34588
> * HTTP
> ** 
> http(s)://homote-host:remote-port/nifi-api/data-transfer/in|output-ports/portId/transactions/txId/flow-files
> ** E.g. 
> http://nifi1.example.coml:8080/nifi-api/data-transfer/input-ports/35dbc0ab-015e-1000-144c-a8d71255027d/transactions/89335043-f105-4de7-a0ef-46f2ef0c7c51/flow-files
> Also provenance events created when a FlowFile is received from remotely have 
> 'Source FlowFile Id' field which contains FlowFile UUID in remote instance.
> This JIRA changes RAW S2S protocol to embed Port ID instead of FlowFile ID 
> (which is available in Source FlowFile Id if needed).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (NIFI-4654) RAW S2S transit URI should contain Port ID instead of FlowFile ID

[Koji Kawamura (JIRA)]

Koji Kawamura created NIFI-4654:
---

 Summary: RAW S2S transit URI should contain Port ID instead of 
FlowFile ID
 Key: NIFI-4654
 URL: https://issues.apache.org/jira/browse/NIFI-4654
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Koji Kawamura
Assignee: Koji Kawamura


In order to track where a FlowFile is sent to or received from, a provenance 
event record has to have target RemoteGroupPort ID (if it's S2S client) or 
RootGroupPort ID (if it's S2S server) in it.

RemoteGroupPort used to have its component UUID as the same UUID as 
corresponding RootGroupPort's, but NIFI-3155 changed each RemoteGroupPort 
instance to have different UUIDs.

HTTP protocol still contains port uuid in transit URI. But RAW does not contain 
a port identifier neither in transit URI nor provenance event record any 
longer. 

Currently, each S2S protocol generates provenance event with transit URI as 
follows:

* RAW
** nifi://remote-host:remote-port/flowfile-uuid
** E.g. nifi://nifi1.example.com:8081/cb729f05-b2ee-4488-909d-c6696cc34588
* HTTP
** 
http(s)://homote-host:remote-port/nifi-api/data-transfer/in|output-ports/portId/transactions/txId/flow-files
** E.g. 
http://nifi1.example.coml:8080/nifi-api/data-transfer/input-ports/35dbc0ab-015e-1000-144c-a8d71255027d/transactions/89335043-f105-4de7-a0ef-46f2ef0c7c51/flow-files

Also provenance events created when a FlowFile is received from remotely have 
'Source FlowFile Id' field which contains FlowFile UUID in remote instance.

This JIRA changes RAW S2S protocol to embed Port ID instead of FlowFile ID 
(which is available in Source FlowFile Id if needed).






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-4652) NiFi pom profiles use of activeByDefault does not work as expected and many maven plugin versions are very out of date

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-4652:
--
Attachment: 0001-NIFI-4652-updating-to-latest-apache-version-and-upda.patch
0002-NIFI-4652-checkpoint.patch

> NiFi pom profiles use of activeByDefault does not work as expected and many 
> maven plugin versions are very out of date
> --
>
> Key: NIFI-4652
> URL: https://issues.apache.org/jira/browse/NIFI-4652
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Witt
>Assignee: Joseph Witt
> Fix For: 1.5.0
>
> Attachments: 
> 0001-NIFI-4652-updating-to-latest-apache-version-and-upda.patch, 
> 0002-NIFI-4652-checkpoint.patch
>
>
> In working NIFI-4645 we've found profiles dont' quite work as expected due to 
> the maven behavior for profiles with 'activeByDefault' being automatically 
> disabled anytime other methods of activation activate any other profile 
> within the same pom.  This leads to builds not working as we expect.  Need to 
> audit, improve, and document a new profile management strategy.
> Further, in reviewing this it is found that many many of our maven plugins 
> are very old/out of date and can benefit from being brought up to date 
> including the org.apache:apache version (latest is 18 and released more than 
> a year ago) and maven compiler which we use 3.2 and latest is 3.7.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4652) NiFi pom profiles use of activeByDefault does not work as expected and many maven plugin versions are very out of date

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276143#comment-16276143
 ] 

ASF GitHub Bot commented on NIFI-4652:
--

GitHub user joewitt opened a pull request:

https://github.com/apache/nifi/pull/2313

NIFI-4652 updating to latest apache version and updating key maven pl…

Not for review yet.  Helping currently with validation on various systems.  
Will squash and notify before review.

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/joewitt/incubator-nifi NIFI-4652

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2313.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2313


commit 78a16cf019c2d0c2b4a2293d6fa68e64100a1de5
Author: joewitt 
Date:   2017-12-03T22:45:03Z

NIFI-4652 updating to latest apache version and updating key maven plugins




> NiFi pom profiles use of activeByDefault does not work as expected and many 
> maven plugin versions are very out of date
> --
>
> Key: NIFI-4652
> URL: https://issues.apache.org/jira/browse/NIFI-4652
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Witt
>Assignee: Joseph Witt
> Fix For: 1.5.0
>
>
> In working NIFI-4645 we've found profiles dont' quite work as expected due to 
> the maven behavior for profiles with 'activeByDefault' being automatically 
> disabled anytime other methods of activation activate any other profile 
> within the same pom.  This leads to builds not working as we expect.  Need to 
> audit, improve, and document a new profile management strategy.
> Further, in reviewing this it is found that many many of our maven plugins 
> are very old/out of date and can benefit from being brought up to date 
> including the org.apache:apache version (latest is 18 and released more than 
> a year ago) and maven compiler which we use 3.2 and latest is 3.7.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2313: NIFI-4652 updating to latest apache version and upd...

[joewitt]

GitHub user joewitt opened a pull request:

https://github.com/apache/nifi/pull/2313

NIFI-4652 updating to latest apache version and updating key maven pl…

Not for review yet.  Helping currently with validation on various systems.  
Will squash and notify before review.

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/joewitt/incubator-nifi NIFI-4652

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2313.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2313


commit 78a16cf019c2d0c2b4a2293d6fa68e64100a1de5
Author: joewitt 
Date:   2017-12-03T22:45:03Z

NIFI-4652 updating to latest apache version and updating key maven plugins




---

[jira] [Updated] (NIFI-4600) Upgrade to Kafka 1.0

[Mark Payne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Payne updated NIFI-4600:
-
Fix Version/s: 1.5.0
   Status: Patch Available  (was: Open)

> Upgrade to Kafka 1.0
> 
>
> Key: NIFI-4600
> URL: https://issues.apache.org/jira/browse/NIFI-4600
> Project: Apache NiFi
>  Issue Type: Task
>Reporter: Dustin Rodrigues
>Assignee: Mark Payne
> Fix For: 1.5.0
>
>
> Support Kafka 1.0 for Publish/Consume Kafka processors



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2312: NIFI-4600: Added nifi-kafka-1-0-nar and nifi-kafka-...

[markap14]

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/2312

NIFI-4600: Added nifi-kafka-1-0-nar and nifi-kafka-1-0-processors mod…

…ules

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-4600

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2312.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2312


commit a40a89b1d2bf0fda5db7c283374934b824ab1589
Author: Mark Payne 
Date:   2017-12-03T19:36:10Z

NIFI-4600: Added nifi-kafka-1-0-nar and nifi-kafka-1-0-processors modules




---

[jira] [Created] (NIFI-4653) Variables are not copied when a Process Group is copied

[Mark Payne (JIRA)]

Mark Payne created NIFI-4653:


 Summary: Variables are not copied when a Process Group is copied
 Key: NIFI-4653
 URL: https://issues.apache.org/jira/browse/NIFI-4653
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Mark Payne
Assignee: Mark Payne


I have a Process Group with 3 variables defined. When I copy and paste the 
Process Group, the newly created Process Group doesn't have those variables.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (NIFI-4600) Upgrade to Kafka 1.0

[Mark Payne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Payne reassigned NIFI-4600:


Assignee: Mark Payne

> Upgrade to Kafka 1.0
> 
>
> Key: NIFI-4600
> URL: https://issues.apache.org/jira/browse/NIFI-4600
> Project: Apache NiFi
>  Issue Type: Task
>Reporter: Dustin Rodrigues
>Assignee: Mark Payne
>
> Support Kafka 1.0 for Publish/Consume Kafka processors



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (NIFI-4652) NiFi pom profiles use of activeByDefault does not work as expected and many maven plugin versions are very out of date

[Joseph Witt (JIRA)]

Joseph Witt created NIFI-4652:
-

 Summary: NiFi pom profiles use of activeByDefault does not work as 
expected and many maven plugin versions are very out of date
 Key: NIFI-4652
 URL: https://issues.apache.org/jira/browse/NIFI-4652
 Project: Apache NiFi
  Issue Type: Bug
Reporter: Joseph Witt
Assignee: Joseph Witt
 Fix For: 1.5.0


In working NIFI-4645 we've found profiles dont' quite work as expected due to 
the maven behavior for profiles with 'activeByDefault' being automatically 
disabled anytime other methods of activation activate any other profile within 
the same pom.  This leads to builds not working as we expect.  Need to audit, 
improve, and document a new profile management strategy.

Further, in reviewing this it is found that many many of our maven plugins are 
very old/out of date and can benefit from being brought up to date including 
the org.apache:apache version (latest is 18 and released more than a year ago) 
and maven compiler which we use 3.2 and latest is 3.7.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4583) Restructure package nifi-solr-processors

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16275926#comment-16275926
 ] 

ASF GitHub Bot commented on NIFI-4583:
--

Github user JohannesDaniel commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2285#discussion_r154523361
  
--- Diff: 
nifi-nar-bundles/nifi-solr-bundle/nifi-solr-processors/src/main/java/org/apache/nifi/processors/solr/SolrUtils.java
 ---
@@ -0,0 +1,202 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.nifi.processors.solr;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.client.HttpClient;
+import org.apache.http.conn.scheme.Scheme;
+import org.apache.http.conn.ssl.SSLSocketFactory;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.context.PropertyContext;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.serialization.record.ListRecordSet;
+import org.apache.nifi.serialization.record.MapRecord;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordField;
+import org.apache.nifi.serialization.record.RecordFieldType;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.serialization.record.RecordSet;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.solr.client.solrj.SolrClient;
+import org.apache.solr.client.solrj.impl.CloudSolrClient;
+import org.apache.solr.client.solrj.impl.HttpClientUtil;
+import org.apache.solr.client.solrj.impl.HttpSolrClient;
+import org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer;
+import org.apache.solr.client.solrj.response.QueryResponse;
+import org.apache.solr.client.solrj.util.ClientUtils;
+import org.apache.solr.common.SolrDocument;
+import org.apache.solr.common.SolrInputDocument;
+import org.apache.solr.common.params.ModifiableSolrParams;
+
+import javax.net.ssl.SSLContext;
+import java.io.IOException;
+import java.io.OutputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+public class SolrUtils {
--- End diff --

@bbende any other comments? As soon as this pull request is merged, I can 
create a pull request for NIFI-4516


> Restructure package nifi-solr-processors
> 
>
> Key: NIFI-4583
> URL: https://issues.apache.org/jira/browse/NIFI-4583
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Johannes Peter
>Assignee: Johannes Peter
>Priority: Minor
>
> Several functionalities currently implemented e. g. in GetSolr or 
> SolrProcessor should be made available for other processors or controller 
> services. A class SolrUtils should be created containing several static 
> methods. This includes the methods 
> - getRequestParams (PutSolrContentStream)
> - solrDocumentsToRecordSet (GetSolr) 
> - createSolrClient (SolrProcessor)
> and the inner class QueryResponseOutputStreamCallback (GetSolr)
> Some unit tests might be affected.
> The method declaration  
> protected SolrClient createSolrClient(final ProcessContext context, final 
> String solrLocation)
> should be changed to 
> public static SolrClient createSolrClient(final PropertyContext context, 
> final String solrLocation)
> to be suitable also for controller services.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2285: NIFI-4583 Restructure nifi-solr-processors

[JohannesDaniel]

Github user JohannesDaniel commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2285#discussion_r154523361
  
--- Diff: 
nifi-nar-bundles/nifi-solr-bundle/nifi-solr-processors/src/main/java/org/apache/nifi/processors/solr/SolrUtils.java
 ---
@@ -0,0 +1,202 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.nifi.processors.solr;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.client.HttpClient;
+import org.apache.http.conn.scheme.Scheme;
+import org.apache.http.conn.ssl.SSLSocketFactory;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.context.PropertyContext;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.serialization.record.ListRecordSet;
+import org.apache.nifi.serialization.record.MapRecord;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordField;
+import org.apache.nifi.serialization.record.RecordFieldType;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.serialization.record.RecordSet;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.solr.client.solrj.SolrClient;
+import org.apache.solr.client.solrj.impl.CloudSolrClient;
+import org.apache.solr.client.solrj.impl.HttpClientUtil;
+import org.apache.solr.client.solrj.impl.HttpSolrClient;
+import org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer;
+import org.apache.solr.client.solrj.response.QueryResponse;
+import org.apache.solr.client.solrj.util.ClientUtils;
+import org.apache.solr.common.SolrDocument;
+import org.apache.solr.common.SolrInputDocument;
+import org.apache.solr.common.params.ModifiableSolrParams;
+
+import javax.net.ssl.SSLContext;
+import java.io.IOException;
+import java.io.OutputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+public class SolrUtils {
--- End diff --

@bbende any other comments? As soon as this pull request is merged, I can 
create a pull request for NIFI-4516


---