[GitHub] nifi issue #2701: NIFI-5194: Ensure that even if calling KafkaConsumer.resum...

[bdesert]

Github user bdesert commented on the issue:

https://github.com/apache/nifi/pull/2701
  
+1 LGTM.
For committers:
Related to Kafka 0.9 only. Other implementations (0.10,0.11 and 1.0) don't 
use consumer's "resume" API explicitly.


---

[jira] [Commented] (NIFI-5194) ConsumeKafka processor can occasionally hang

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475302#comment-16475302
 ] 

ASF GitHub Bot commented on NIFI-5194:
--

Github user bdesert commented on the issue:

https://github.com/apache/nifi/pull/2701
  
+1 LGTM.
For committers:
Related to Kafka 0.9 only. Other implementations (0.10,0.11 and 1.0) don't 
use consumer's "resume" API explicitly.


> ConsumeKafka processor can occasionally hang
> 
>
> Key: NIFI-5194
> URL: https://issues.apache.org/jira/browse/NIFI-5194
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.7.0
>
>
> When using ConsumeKafka, we can occasionally get into a situation where the 
> threads will "hang" and stopping the processor won't help. In such a case, we 
> can see the following stack trace in thread dumps:
> {code:java}
> "Timer-Driven Process Thread-42" Id=201 WAITING on 
> java.util.concurrent.locks.ReentrantLock$NonfairSync@502555e6
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> at 
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
> at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
> at 
> org.apache.nifi.processors.kafka.pubsub.ConsumerLease.poll(ConsumerLease.java:147)
> at 
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka.onTrigger(ConsumeKafka.java:367)
> at 
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at 
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at 
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Number of Locked Synchronizers: 1
> - java.util.concurrent.ThreadPoolExecutor$Worker@6759e7f6
> {code}
>  
> Note that this issue affects only ConsumeKafka and not ConsumeKafka_0_10, 
> ConsumeKafka_0_11, ConsumeKafka_1_0, or any ConsumeKafkaRecord* processors. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5041) Add convenient SPNEGO/Kerberos authentication support to LivySessionController

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475184#comment-16475184
 ] 

ASF GitHub Bot commented on NIFI-5041:
--

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2630#discussion_r188147689
  
--- Diff: 
nifi-nar-bundles/nifi-spark-bundle/nifi-livy-controller-service/src/main/java/org/apache/nifi/controller/livy/LivySessionController.java
 ---
@@ -241,12 +241,14 @@ public void onConfigured(final ConfigurationContext 
context) {
 while (enabled) {
 try {
 manageSessions();
+} catch (Exception e) {
+getLogger().error("Livy Session Manager Thread run 
into an error, but continues to run", e);
--- End diff --

This keeps the manageSessions() thread alive, but will there be an 
indication on the UI that the error is not recoverable? I'm thinking 
specifically about the 401 Authorization Required error where the Livy API 
returns HTML rather than JSON when you try to log in without Kerberos when the 
server has been Kerberized. Should we set an AtomicReference or 
something on the LivySessionController and throw a checked exception when any 
API call is made (such as isEmpty() which is called from 
ExecuteSparkInteractive)? I think we need to make it obvious (at least in that 
case) that the processor and/or CS is suffering from a non-recoverable error 
and needs manual intervention.


> Add convenient SPNEGO/Kerberos authentication support to LivySessionController
> --
>
> Key: NIFI-5041
> URL: https://issues.apache.org/jira/browse/NIFI-5041
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Peter Toth
>Priority: Minor
>
> Livy requires SPNEGO/Kerberos authentication on a secured cluster. Initiating 
> such an authentication from NiFi is a viable by providing a 
> java.security.auth.login.config system property 
> (https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/lab/part6.html),
>  but this is a bit cumbersome and needs kinit running outside of NiFi.
> An alternative and more sophisticated solution would be to do the SPNEGO 
> negotiation programmatically.
>  * This solution would add some new properties to the LivySessionController 
> to fetch kerberos principal and password/keytab
>  * Add the required HTTP Negotiate header (with an SPNEGO token) to the 
> HttpURLConnection to do the authentication programmatically 
> (https://tools.ietf.org/html/rfc4559)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2630: NIFI-5041 Adds SPNEGO authentication to LivySession...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2630#discussion_r188147689
  
--- Diff: 
nifi-nar-bundles/nifi-spark-bundle/nifi-livy-controller-service/src/main/java/org/apache/nifi/controller/livy/LivySessionController.java
 ---
@@ -241,12 +241,14 @@ public void onConfigured(final ConfigurationContext 
context) {
 while (enabled) {
 try {
 manageSessions();
+} catch (Exception e) {
+getLogger().error("Livy Session Manager Thread run 
into an error, but continues to run", e);
--- End diff --

This keeps the manageSessions() thread alive, but will there be an 
indication on the UI that the error is not recoverable? I'm thinking 
specifically about the 401 Authorization Required error where the Livy API 
returns HTML rather than JSON when you try to log in without Kerberos when the 
server has been Kerberized. Should we set an AtomicReference or 
something on the LivySessionController and throw a checked exception when any 
API call is made (such as isEmpty() which is called from 
ExecuteSparkInteractive)? I think we need to make it obvious (at least in that 
case) that the processor and/or CS is suffering from a non-recoverable error 
and needs manual intervention.


---

[jira] [Commented] (NIFI-2738) Documentation - Remove references to nifi.cluster.request.replication.claim.timeout

[Mark Bean (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475128#comment-16475128
 ] 

Mark Bean commented on NIFI-2738:
-

In the Admin Guide, is the entire paragraph on "Claim Management" no longer 
relevant? I know the property listed in the table is not used any longer.

> Documentation - Remove references to 
> nifi.cluster.request.replication.claim.timeout
> ---
>
> Key: NIFI-2738
> URL: https://issues.apache.org/jira/browse/NIFI-2738
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Documentation  Website
>Reporter: Matt Gilman
>Priority: Minor
>
> The property for
> {noformat}nifi.cluster.request.replication.claim.timeout{noformat}
> has been removed as of NiFi 1.0.0 however there are still references to it in 
> the administration guide and default defaults in some poms.
> Also consider updating the section on Cluster Claim Management.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MINIFICPP-476) Support true/false literals as function args in EL

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475119#comment-16475119
 ] 

ASF GitHub Bot commented on MINIFICPP-476:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/329


> Support true/false literals as function args in EL
> --
>
> Key: MINIFICPP-476
> URL: https://issues.apache.org/jira/browse/MINIFICPP-476
> Project: NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Andrew Christianson
>Assignee: Andrew Christianson
>Priority: Major
>
> Currently writing "true" as a literal function arg causes an EL parse 
> exception.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4907) Provenance authorization refactoring

[Mark Bean (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475118#comment-16475118
 ] 

Mark Bean commented on NIFI-4907:
-

In addition to basic unit testing, performed the following integration testing 
steps:

 

Remove ‘query provenance’ Global Policy; remove ‘view provenance’ Component 
Policy

Cannot query provenance; Global menu item grayed out; component menu item not 
displayed

Add ‘view provenance’ Component Policy at root Process Group. Still Global menu 
item grayed out; component menu item not displayed

Remove ‘view provenance’ Component Policy at root Process Group.

 

Create ‘query provenance’ Global Policy; add user

Global ‘Data Provenance’ runs query, but does not display any results

Component ‘View data provenance’ runs query, but does not display any results

 

Create ‘view provenance’ Component Policy at root Process Group; add user

Global ‘Data Provenance’ runs query, and displays all event results; Lineage 
graph displays all provenance events. Additional details are not available from 
the query results list nor lineage graph.

 

Create ‘view the data’ Component Policy at root Process Group; add user.

Global ‘Data Provenance’ runs query, and displays all event results; Lineage 
graph displays all provenance events. Additional details are available from the 
query results list and lineage graph.

 

Override ‘view provenance’ Component Policy for one component (e.g. 
GenerateFlowFile). 

Global ‘Data Provenance’ runs query, and displays all event results except 
CREATE event associated with Component whose policy was overridden. Lineage 
graph displays all provenance events with “CREATE” event being replaced by 
“UNKNOWN”. Attempting to double-click on the “UNKNOWN” event results in 
“Insufficient Permissions: Unable to view the provenance data for Processor 
with ID \{UUID}. Contact the system administrator.” Other lineage events allow 
access to additional details when double-clicked.

Component ‘View data provenance’ runs query, but does not display any results

 

Repeat with consistent results for Persistent, Volatile and Write-Ahead 
Provenance repository types

> Provenance authorization refactoring
> 
>
> Key: NIFI-4907
> URL: https://issues.apache.org/jira/browse/NIFI-4907
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.5.0
>Reporter: Mark Bean
>Assignee: Mark Bean
>Priority: Major
>
> Currently, the 'view the data' component policy is too tightly coupled with 
> Provenance queries. The 'query provenance' policy should be the only policy 
> required for viewing Provenance query results. Both 'view the component' and 
> 'view the data' policies should be used to refine the appropriate visibility 
> of event details - but not the event itself.
> 1) Component Visibility
> The authorization of Provenance events is inconsistent with the behavior of 
> the graph. For example, if a user does not have 'view the component' policy, 
> the graph shows this component as a "black box" (no details such as name, 
> UUID, etc.) However, when querying Provenance, this component will show up 
> including the Component Type and the Component Name. This is in effect a 
> violation of the policy. These component details should be obscured in the 
> Provenance event displayed if user does not have the appropriate 'view the 
> component' policy.
> 2) Data Visibility
> For a Provenance query, all events should be visible as long as the user 
> performing the query belongs to the 'query provenance' global policy. As 
> mentioned above, some information about the component may be obscured 
> depending on 'view the component' policy, but the event itself should be 
> visible. Additionally, details of the event (clicking the View Details "i" 
> icon) should only be accessible if the user belongs to the 'view the data' 
> policy for the affected component. If the user is not in the appropriate 
> 'view the data' policy, a popup warning should be displayed indicating the 
> reason details are not visible with more specific detail than the current 
> "Contact the system administrator".
> 3) Lineage Graphs
> As with the Provenance table view recommendation above, the lineage graph 
> should display all events. Currently, if the lineage graph includes an event 
> belonging to a component which the user does not have 'view the data', it is 
> shown on the graph as "UNKNOWN". As with Data Visibility mentioned above, the 
> graph should indicate the event type as long as the user is in the 'view the 
> component'. Subsequent "View Details" on the event should only be visible if 
> the user is in the 'view the data' policy.
> In summary, for Provenance query results and lineage graphs, all events 
> should be shown. Component 

[GitHub] nifi-minifi-cpp pull request #329: MINIFICPP-476 Added support for literal t...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/329


---

[jira] [Commented] (NIFI-4907) Provenance authorization refactoring

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475117#comment-16475117
 ] 

ASF GitHub Bot commented on NIFI-4907:
--

GitHub user markobean opened a pull request:

https://github.com/apache/nifi/pull/2703

NIFI-4907: add 'view provenance' component policy

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [x] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markobean/nifi NIFI-4907

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2703.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2703


commit eed76e9aa2891eeccb7ef4bb4a5890a178aa7a8f
Author: Mark Bean 
Date:   2018-05-12T20:32:29Z

NIFI-4907: add 'view provenance' component policy




> Provenance authorization refactoring
> 
>
> Key: NIFI-4907
> URL: https://issues.apache.org/jira/browse/NIFI-4907
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.5.0
>Reporter: Mark Bean
>Assignee: Mark Bean
>Priority: Major
>
> Currently, the 'view the data' component policy is too tightly coupled with 
> Provenance queries. The 'query provenance' policy should be the only policy 
> required for viewing Provenance query results. Both 'view the component' and 
> 'view the data' policies should be used to refine the appropriate visibility 
> of event details - but not the event itself.
> 1) Component Visibility
> The authorization of Provenance events is inconsistent with the behavior of 
> the graph. For example, if a user does not have 'view the component' policy, 
> the graph shows this component as a "black box" (no details such as name, 
> UUID, etc.) However, when querying Provenance, this component will show up 
> including the Component Type and the Component Name. This is in effect a 
> violation of the policy. These component details should be obscured in the 
> Provenance event displayed if user does not have the appropriate 'view the 
> component' policy.
> 2) Data Visibility
> For a Provenance query, all events should be visible as long as the user 
> performing the query belongs to the 'query provenance' global policy. As 
> mentioned above, some information about the component may be obscured 
> depending on 'view the component' policy, but the event itself should be 
> visible. Additionally, details of the event (clicking the View Details "i" 
> icon) should only be accessible if the user belongs to the 'view the data' 
> policy for the affected component. If the user is not in the appropriate 
> 'view the data' policy, a popup warning should be displayed indicating the 
> reason details are not visible with more specific detail than the current 
> "Contact the system administrator".
> 3) Lineage Graphs
> As with the Provenance table view recommendation above, the lineage graph 
> should display all events. 

[GitHub] nifi pull request #2703: NIFI-4907: add 'view provenance' component policy

[markobean]

GitHub user markobean opened a pull request:

https://github.com/apache/nifi/pull/2703

NIFI-4907: add 'view provenance' component policy

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [x] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markobean/nifi NIFI-4907

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2703.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2703


commit eed76e9aa2891eeccb7ef4bb4a5890a178aa7a8f
Author: Mark Bean 
Date:   2018-05-12T20:32:29Z

NIFI-4907: add 'view provenance' component policy




---

[jira] [Commented] (MINIFICPP-495) bootstrap.sh will exit with error in default install Debian systems

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475106#comment-16475106
 ] 

ASF GitHub Bot commented on MINIFICPP-495:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/330


> bootstrap.sh will exit with error in default install Debian systems
> ---
>
> Key: MINIFICPP-495
> URL: https://issues.apache.org/jira/browse/MINIFICPP-495
> Project: NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Aldrin Piri
>Assignee: Aldrin Piri
>Priority: Major
> Fix For: 0.5.0
>
>
> With the shebang line following comments, the default non-interactive shell 
> will be used on the system.  Because Debian systems default to dash for 
> non-interactive shells the bootstrap will fail starting at Line 35 where bash 
> syntax is utilized. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-minifi-cpp pull request #330: MINIFICPP-495 Move shebang line to first ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/330


---

[jira] [Commented] (NIFI-5145) MockPropertyValue.evaluateExpressionLanguage(FlowFile) cannot handle null inputs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475027#comment-16475027
 ] 

ASF GitHub Bot commented on NIFI-5145:
--

Github user MikeThomsen commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2672#discussion_r188126334
  
--- Diff: 
nifi-mock/src/main/java/org/apache/nifi/util/MockPropertyValue.java ---
@@ -202,6 +203,9 @@ public PropertyValue evaluateAttributeExpressions(final 
AttributeValueDecorator
 
 @Override
 public PropertyValue evaluateAttributeExpressions(final FlowFile 
flowFile) throws ProcessException {
+if (flowFile == null) {
--- End diff --

Ok. I'll comment it up and make it clear what it's doing so anyone jumping 
in there later will understand exactly what's going on there.


> MockPropertyValue.evaluateExpressionLanguage(FlowFile) cannot handle null 
> inputs
> 
>
> Key: NIFI-5145
> URL: https://issues.apache.org/jira/browse/NIFI-5145
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>Priority: Major
>
> The method mentioned in the title line cannot handle null inputs, even though 
> the main NiFi execution classes can handle that scenario. This forces hack to 
> pass testing with nulls that looks like this:
> String val = flowFile != null ? 
> context.getProperty(PROP).evaluateExpressionLanguage(flowfile).getValue() : 
> context.getProperty(PROP).evaluateExpressionLanguage(new 
> HashMap()).getValue();



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2672: NIFI-5145 Made MockPropertyValue.evaluateExpression...

[MikeThomsen]

Github user MikeThomsen commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2672#discussion_r188126334
  
--- Diff: 
nifi-mock/src/main/java/org/apache/nifi/util/MockPropertyValue.java ---
@@ -202,6 +203,9 @@ public PropertyValue evaluateAttributeExpressions(final 
AttributeValueDecorator
 
 @Override
 public PropertyValue evaluateAttributeExpressions(final FlowFile 
flowFile) throws ProcessException {
+if (flowFile == null) {
--- End diff --

Ok. I'll comment it up and make it clear what it's doing so anyone jumping 
in there later will understand exactly what's going on there.


---

[GitHub] nifi issue #2702: Added Apache Pulsar processors

[david-streamlio]

Github user david-streamlio commented on the issue:

https://github.com/apache/nifi/pull/2702
  
Looks like there are error in the Email processors causing the CI build to 
fail.

[ERROR] Tests run: 5, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 
1.996 s <<< FAILURE! - in org.apache.nifi.processors.email.TestConsumeEmail
[ERROR] testConsumePOP3(org.apache.nifi.processors.email.TestConsumeEmail)  
Time elapsed: 1.014 s  <<< ERROR!
java.lang.IllegalStateException: Could not start mail server 
imap:127.0.0.1:3143, try to set server startup timeout > 1000 via 
ServerSetup.setServerStartupTimeout(timeoutInMs)
at 
org.apache.nifi.processors.email.TestConsumeEmail.setUp(TestConsumeEmail.java:55)

[ERROR] testConsumePOP3(org.apache.nifi.processors.email.TestConsumeEmail)  
Time elapsed: 1.017 s  <<< ERROR!
java.lang.NullPointerException
at 
org.apache.nifi.processors.email.TestConsumeEmail.cleanUp(TestConsumeEmail.java:66)

[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Errors: 
[ERROR] 
org.apache.nifi.processors.email.TestConsumeEmail.testConsumePOP3(org.apache.nifi.processors.email.TestConsumeEmail)
[ERROR]   Run 1: TestConsumeEmail.setUp:55 » IllegalState Could not 
start mail server imap:127
[ERROR]   Run 2: TestConsumeEmail.cleanUp:66 NullPointer
[INFO] 
[INFO] 
[ERROR] Tests run: 21, Failures: 0, Errors: 1, Skipped: 0


---

[jira] [Commented] (NIFI-5145) MockPropertyValue.evaluateExpressionLanguage(FlowFile) cannot handle null inputs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475012#comment-16475012
 ] 

ASF GitHub Bot commented on NIFI-5145:
--

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2672#discussion_r188124392
  
--- Diff: 
nifi-mock/src/main/java/org/apache/nifi/util/MockPropertyValue.java ---
@@ -202,6 +203,9 @@ public PropertyValue evaluateAttributeExpressions(final 
AttributeValueDecorator
 
 @Override
 public PropertyValue evaluateAttributeExpressions(final FlowFile 
flowFile) throws ProcessException {
+if (flowFile == null) {
--- End diff --

Sorry - don't think I was clear in what I was trying to communicate here. I 
do understand the intent. But it took me a while to understand how the PR 
differed logically from what already was there. What would seem natural to me 
is to simply pass the null FlowFile along to an override, filling in other 
default values, as it already does. But clearly that is a bug because of how 
this is handled in the actual implementing override. At first look, though, it 
seemed to me to be the same logic because in many places an empty map is 
treated the same as a null map.

So while I now understand what it's doing, I think it's a bit confusing at 
first glance. That's why i suggested passing a boolean down the stack. The PR 
wouldn't cause me much angst if there were just some inline comments, I think, 
about how the implementing override checks if flowfile is null and variables is 
null and if so handles things differently... but at that point it makes it 
really easy to change the implementation later and leave comments in the code 
that are no longer valid. but i'd be okay either way.


> MockPropertyValue.evaluateExpressionLanguage(FlowFile) cannot handle null 
> inputs
> 
>
> Key: NIFI-5145
> URL: https://issues.apache.org/jira/browse/NIFI-5145
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mike Thomsen
>Assignee: Mike Thomsen
>Priority: Major
>
> The method mentioned in the title line cannot handle null inputs, even though 
> the main NiFi execution classes can handle that scenario. This forces hack to 
> pass testing with nulls that looks like this:
> String val = flowFile != null ? 
> context.getProperty(PROP).evaluateExpressionLanguage(flowfile).getValue() : 
> context.getProperty(PROP).evaluateExpressionLanguage(new 
> HashMap()).getValue();



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2672: NIFI-5145 Made MockPropertyValue.evaluateExpression...

[markap14]

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2672#discussion_r188124392
  
--- Diff: 
nifi-mock/src/main/java/org/apache/nifi/util/MockPropertyValue.java ---
@@ -202,6 +203,9 @@ public PropertyValue evaluateAttributeExpressions(final 
AttributeValueDecorator
 
 @Override
 public PropertyValue evaluateAttributeExpressions(final FlowFile 
flowFile) throws ProcessException {
+if (flowFile == null) {
--- End diff --

Sorry - don't think I was clear in what I was trying to communicate here. I 
do understand the intent. But it took me a while to understand how the PR 
differed logically from what already was there. What would seem natural to me 
is to simply pass the null FlowFile along to an override, filling in other 
default values, as it already does. But clearly that is a bug because of how 
this is handled in the actual implementing override. At first look, though, it 
seemed to me to be the same logic because in many places an empty map is 
treated the same as a null map.

So while I now understand what it's doing, I think it's a bit confusing at 
first glance. That's why i suggested passing a boolean down the stack. The PR 
wouldn't cause me much angst if there were just some inline comments, I think, 
about how the implementing override checks if flowfile is null and variables is 
null and if so handles things differently... but at that point it makes it 
really easy to change the implementation later and leave comments in the code 
that are no longer valid. but i'd be okay either way.


---

[GitHub] nifi issue #2542: NIFI-4971: ReportLineageToAtlas complete path can miss one...

[MikeThomsen]

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/2542
  
@ijokarumawak I went back and looked at the Jira ticket again, and my 
interpretation is that what I saw in the screenshots above (with the graph) 
matches your description of the behavior you wanted to implement.


---

[jira] [Commented] (NIFI-4971) ReportLineageToAtlas 'complete path' strategy can miss one-time lineages

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474982#comment-16474982
 ] 

ASF GitHub Bot commented on NIFI-4971:
--

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/2542
  
@ijokarumawak I went back and looked at the Jira ticket again, and my 
interpretation is that what I saw in the screenshots above (with the graph) 
matches your description of the behavior you wanted to implement.


> ReportLineageToAtlas 'complete path' strategy can miss one-time lineages
> 
>
> Key: NIFI-4971
> URL: https://issues.apache.org/jira/browse/NIFI-4971
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.5.0
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>Priority: Major
>
> For the simplest example, with GetFlowFIle (GFF) -> PutFlowFile (PFF), where 
> GFF gets files and PFF saves those files into a different directory, then 
> following provenance events will be generated:
>  # GFF RECEIVE file1
>  # PFF SEND file2
> From above provenance events, following entities and lineages should be 
> created in Atlas, labels in brackets are Atlas type names:
> {code}
> file1 (fs_path) -> GFF, PFF (nifi_flow_path) -> file2 (fs_path)
> {code}
> Entities shown in above graph are created. However, the 'nifi_flow_path' 
> entity do not have inputs/outputs referencing 'fs_path', so lineage can not 
> be seen in Atlas UI.
> This issue was discovered by [~nayakmahesh616]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4136) GrokReader - Add a failure option to unmatch behavior options

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474860#comment-16474860
 ] 

ASF GitHub Bot commented on NIFI-4136:
--

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/1955
  
@ottobackwards can you compare this against your latest changes and see if 
it's still needed?


> GrokReader - Add a failure option to unmatch behavior options
> -
>
> Key: NIFI-4136
> URL: https://issues.apache.org/jira/browse/NIFI-4136
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Major
>
> At the moment, when using the GrokReader, if a line does not match the grok 
> expression (and is not part of a stack trace), the line can be either ignored 
> (the line will be completely skipped) or  appended to the last field from the 
> previous line.
> In the case where appending is not desired and that data should not be 
> ignored/deleted, we should add the option to route the full flow file to the 
> failure relationship. This way the flow file could be treated in a different 
> way (for example with SplitText and ExtractGrok to isolate the incorrect 
> lines and re-route the correct lines back to the Record processors).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #1955: NIFI-4136 Add a failure option to unmatch behavior options...

[MikeThomsen]

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/1955
  
@ottobackwards can you compare this against your latest changes and see if 
it's still needed?


---

[jira] [Commented] (NIFI-5113) Add XML record writer

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474854#comment-16474854
 ] 

ASF GitHub Bot commented on NIFI-5113:
--

Github user JohannesDaniel commented on the issue:

https://github.com/apache/nifi/pull/2675
  
@markap14 implemented changes as discussed


> Add XML record writer
> -
>
> Key: NIFI-5113
> URL: https://issues.apache.org/jira/browse/NIFI-5113
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Johannes Peter
>Assignee: Johannes Peter
>Priority: Major
>
> Corresponding writer for the XML record reader



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2675: NIFI-5113 Add XMLRecordSetWriter

[JohannesDaniel]

Github user JohannesDaniel commented on the issue:

https://github.com/apache/nifi/pull/2675
  
@markap14 implemented changes as discussed


---

[jira] [Updated] (NIFI-5175) NiFi built with Java 1.8 needs to run on Java 9

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck updated NIFI-5175:
--
Description: 
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||Status||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|In Progress|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9|Done|

 

 
||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred
 ..._specific class usage snipped_...
 WARNING: Use --illegal-access=warn to enable warnings of further illegal 
reflective access operations
 WARNING: All illegal access operations will be denied in a future 
release|Reflective invocations are common in the code used in NiFi and its 
dependencies on Java 1.8|Full compliant migration to Java 9 and use 
dependencies that are Java 9 compliant|

  was:
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||Status||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|In Progress|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9|Done|

 

 
||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred
 ..._specific class usage snipped_...
 WARNING: Use --illegal-access=warn to enable warnings of further illegal 
reflective access operations
 WARNING: All illegal access operations will be denied in a future 
release|Reflective invocations are common in the code used in NiFi and its 
dependencies on Java 1.8|Full compliant migration to Java 9 and using 
dependencies that are Java 9 compliant|


> NiFi built with Java 1.8 needs to run on Java 9
> ---
>
> Key: NIFI-5175
> URL: https://issues.apache.org/jira/browse/NIFI-5175
> Project: Apache NiFi
>  Issue Type: Sub-task
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Major
>
> The following issues have been encountered while attempting to run a Java 
> 1.8-built NiFi on Java 9:
> ||Issue||Solution||Status||
> |Java 9 ServiceLoader requires providers have a default no-arg public 
> constructor|Add no-arg public constructors|In Progress|
> |NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
> stop|Detect if NiFi is running on Java 9, and reflectively invoke 
> Process.pid(), which was newly added to the Process API in Java 9|Done|
>  
> 
>  
> ||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
> 9+||Description||Solution||
> |WARNING: An illegal reflective access operation has occurred
>  ..._specific class usage snipped_...
>  WARNING: Use --illegal-access=warn to enable warnings of further illegal 
> reflective access operations
>  WARNING: All illegal access operations will be denied in a future 
> release|Reflective invocations are common in the code used in NiFi and its 
> dependencies on Java 1.8|Full compliant migration to Java 9 and use 
> dependencies that are Java 9 compliant|



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-5175) NiFi built with Java 1.8 needs to run on Java 9

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck updated NIFI-5175:
--
Description: 
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||Status||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|In Progress|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9|Done|

 

 
||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred
 ..._specific class usage snipped_...
 WARNING: Use --illegal-access=warn to enable warnings of further illegal 
reflective access operations
 WARNING: All illegal access operations will be denied in a future 
release|Reflective invocations are common in the code used in NiFi and its 
dependencies on Java 1.8|Full compliant migration to Java 9 and using 
dependencies that are Java 9 compliant|

  was:
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||Status||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|In Progress|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9|Done|

 

 
||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred
 ..._specific class usage snipped_...
 Please consider reporting this to the maintainers of 
org.xerial.snappy.SnappyLoader
 WARNING: Use --illegal-access=warn to enable warnings of further illegal 
reflective access operations
 WARNING: All illegal access operations will be denied in a future 
release|Reflective invocations are common in the code used in NiFi and its 
dependencies on Java 1.8|Full compliant migration to Java 9 and using 
dependencies that are Java 9 compliant|


> NiFi built with Java 1.8 needs to run on Java 9
> ---
>
> Key: NIFI-5175
> URL: https://issues.apache.org/jira/browse/NIFI-5175
> Project: Apache NiFi
>  Issue Type: Sub-task
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Major
>
> The following issues have been encountered while attempting to run a Java 
> 1.8-built NiFi on Java 9:
> ||Issue||Solution||Status||
> |Java 9 ServiceLoader requires providers have a default no-arg public 
> constructor|Add no-arg public constructors|In Progress|
> |NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
> stop|Detect if NiFi is running on Java 9, and reflectively invoke 
> Process.pid(), which was newly added to the Process API in Java 9|Done|
>  
> 
>  
> ||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
> 9+||Description||Solution||
> |WARNING: An illegal reflective access operation has occurred
>  ..._specific class usage snipped_...
>  WARNING: Use --illegal-access=warn to enable warnings of further illegal 
> reflective access operations
>  WARNING: All illegal access operations will be denied in a future 
> release|Reflective invocations are common in the code used in NiFi and its 
> dependencies on Java 1.8|Full compliant migration to Java 9 and using 
> dependencies that are Java 9 compliant|



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-5175) NiFi built with Java 1.8 needs to run on Java 9

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck updated NIFI-5175:
--
Description: 
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||Status||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|In Progress|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9|Done|

 

 
||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred
...
Please consider reporting this to the maintainers of 
org.xerial.snappy.SnappyLoader
WARNING: Use --illegal-access=warn to enable warnings of further illegal 
reflective access operations
WARNING: All illegal access operations will be denied in a future 
release|Reflective invocations are common in the code used in NiFi and its 
dependencies on Java 1.8|Full compliant migration to Java 9 and using 
dependencies that are Java 9 compliant|

  was:
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||Status||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|In Progress|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9|Done|

 

 
||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred|Reflective 
invocations are common in the code used in NiFi and its dependencies on Java 
1.8|Full compliant migration to Java 9 and using dependencies that are Java 9 
compliant|


> NiFi built with Java 1.8 needs to run on Java 9
> ---
>
> Key: NIFI-5175
> URL: https://issues.apache.org/jira/browse/NIFI-5175
> Project: Apache NiFi
>  Issue Type: Sub-task
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Major
>
> The following issues have been encountered while attempting to run a Java 
> 1.8-built NiFi on Java 9:
> ||Issue||Solution||Status||
> |Java 9 ServiceLoader requires providers have a default no-arg public 
> constructor|Add no-arg public constructors|In Progress|
> |NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
> stop|Detect if NiFi is running on Java 9, and reflectively invoke 
> Process.pid(), which was newly added to the Process API in Java 9|Done|
>  
> 
>  
> ||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
> 9+||Description||Solution||
> |WARNING: An illegal reflective access operation has occurred
> ...
> Please consider reporting this to the maintainers of 
> org.xerial.snappy.SnappyLoader
> WARNING: Use --illegal-access=warn to enable warnings of further illegal 
> reflective access operations
> WARNING: All illegal access operations will be denied in a future 
> release|Reflective invocations are common in the code used in NiFi and its 
> dependencies on Java 1.8|Full compliant migration to Java 9 and using 
> dependencies that are Java 9 compliant|



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-5175) NiFi built with Java 1.8 needs to run on Java 9

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck updated NIFI-5175:
--
Description: 
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||Status||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|In Progress|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9|Done|

 

 
||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred
 ..._specific class usage snipped_...
 Please consider reporting this to the maintainers of 
org.xerial.snappy.SnappyLoader
 WARNING: Use --illegal-access=warn to enable warnings of further illegal 
reflective access operations
 WARNING: All illegal access operations will be denied in a future 
release|Reflective invocations are common in the code used in NiFi and its 
dependencies on Java 1.8|Full compliant migration to Java 9 and using 
dependencies that are Java 9 compliant|

  was:
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||Status||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|In Progress|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9|Done|

 

 
||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred
...
Please consider reporting this to the maintainers of 
org.xerial.snappy.SnappyLoader
WARNING: Use --illegal-access=warn to enable warnings of further illegal 
reflective access operations
WARNING: All illegal access operations will be denied in a future 
release|Reflective invocations are common in the code used in NiFi and its 
dependencies on Java 1.8|Full compliant migration to Java 9 and using 
dependencies that are Java 9 compliant|


> NiFi built with Java 1.8 needs to run on Java 9
> ---
>
> Key: NIFI-5175
> URL: https://issues.apache.org/jira/browse/NIFI-5175
> Project: Apache NiFi
>  Issue Type: Sub-task
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Major
>
> The following issues have been encountered while attempting to run a Java 
> 1.8-built NiFi on Java 9:
> ||Issue||Solution||Status||
> |Java 9 ServiceLoader requires providers have a default no-arg public 
> constructor|Add no-arg public constructors|In Progress|
> |NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
> stop|Detect if NiFi is running on Java 9, and reflectively invoke 
> Process.pid(), which was newly added to the Process API in Java 9|Done|
>  
> 
>  
> ||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
> 9+||Description||Solution||
> |WARNING: An illegal reflective access operation has occurred
>  ..._specific class usage snipped_...
>  Please consider reporting this to the maintainers of 
> org.xerial.snappy.SnappyLoader
>  WARNING: Use --illegal-access=warn to enable warnings of further illegal 
> reflective access operations
>  WARNING: All illegal access operations will be denied in a future 
> release|Reflective invocations are common in the code used in NiFi and its 
> dependencies on Java 1.8|Full compliant migration to Java 9 and using 
> dependencies that are Java 9 compliant|



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-5175) NiFi built with Java 1.8 needs to run on Java 9

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck updated NIFI-5175:
--
Description: 
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||Status||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|In Progress|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9|Done|

 

 
||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred|Reflective 
invocations are common in the code used in NiFi and its dependencies on Java 
1.8|Full compliant migration to Java 9 and using dependencies that are Java 9 
compliant|

  was:
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||Status||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|In Progress|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9|Done|

 

 
||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred|Reflective 
invocations are common in the code used in NiFi and its dependencies on Java 
1.8|Full migration to Java 9 and using dependencies that are Java 9 compliant|


> NiFi built with Java 1.8 needs to run on Java 9
> ---
>
> Key: NIFI-5175
> URL: https://issues.apache.org/jira/browse/NIFI-5175
> Project: Apache NiFi
>  Issue Type: Sub-task
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Major
>
> The following issues have been encountered while attempting to run a Java 
> 1.8-built NiFi on Java 9:
> ||Issue||Solution||Status||
> |Java 9 ServiceLoader requires providers have a default no-arg public 
> constructor|Add no-arg public constructors|In Progress|
> |NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
> stop|Detect if NiFi is running on Java 9, and reflectively invoke 
> Process.pid(), which was newly added to the Process API in Java 9|Done|
>  
> 
>  
> ||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
> 9+||Description||Solution||
> |WARNING: An illegal reflective access operation has occurred|Reflective 
> invocations are common in the code used in NiFi and its dependencies on Java 
> 1.8|Full compliant migration to Java 9 and using dependencies that are Java 9 
> compliant|



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2614: Added Apache Pulsar Processors and Controller Service

[david-streamlio]

Github user david-streamlio commented on the issue:

https://github.com/apache/nifi/pull/2614
  
Abandoning this PR in favor of PR #2702

https://github.com/apache/nifi/pull/2702


---

[GitHub] nifi pull request #2614: Added Apache Pulsar Processors and Controller Servi...

[david-streamlio]

Github user david-streamlio closed the pull request at:

https://github.com/apache/nifi/pull/2614


---

[jira] [Updated] (NIFI-5175) NiFi built with Java 1.8 needs to run on Java 9

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck updated NIFI-5175:
--
Description: 
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||Status||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|In Progress|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9|Done|

 

 
||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred|Reflective 
invocations are common in the code used in NiFi and its dependencies on Java 
1.8|Full migration to Java 9 and using dependencies that are Java 9 compliant|

  was:
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||Status||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|In Progress|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9|Done|

 

 
||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred|Reflective 
invocations are common in the code used in NiFi and its dependencies on Java 
1.8|Full migration to Java 9 and using dependencies that are Java 9 compilant|


> NiFi built with Java 1.8 needs to run on Java 9
> ---
>
> Key: NIFI-5175
> URL: https://issues.apache.org/jira/browse/NIFI-5175
> Project: Apache NiFi
>  Issue Type: Sub-task
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Major
>
> The following issues have been encountered while attempting to run a Java 
> 1.8-built NiFi on Java 9:
> ||Issue||Solution||Status||
> |Java 9 ServiceLoader requires providers have a default no-arg public 
> constructor|Add no-arg public constructors|In Progress|
> |NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
> stop|Detect if NiFi is running on Java 9, and reflectively invoke 
> Process.pid(), which was newly added to the Process API in Java 9|Done|
>  
> 
>  
> ||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
> 9+||Description||Solution||
> |WARNING: An illegal reflective access operation has occurred|Reflective 
> invocations are common in the code used in NiFi and its dependencies on Java 
> 1.8|Full migration to Java 9 and using dependencies that are Java 9 compliant|



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2702: Added Apache Pulsar processors

[david-streamlio]

GitHub user david-streamlio opened a pull request:

https://github.com/apache/nifi/pull/2702

Added Apache Pulsar processors

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/david-streamlio/nifi NIFI-4914

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2702.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2702


commit f67fb157b9e6f04c8b913fe30b815b181eebf53a
Author: David Kjerrumgaard 
Date:   2018-05-14T21:30:50Z

Added Apache Pulsar processors




---

[jira] [Updated] (NIFI-5175) NiFi built with Java 1.8 needs to run on Java 9

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck updated NIFI-5175:
--
Description: 
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||Status||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors| In Progress|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9| Done|

 

 
||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred|Reflective 
invocations are common in the code used in NiFi and its dependencies on Java 
1.8|Full migration to Java 9 and using dependencies that are Java 9 compilant|

  was:
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||Solved||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors| |
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9| |

 

 
||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred|Reflective 
invocations are common in the code used in NiFi and its dependencies on Java 
1.8|Full migration to Java 9 and using dependencies that are Java 9 compilant|


> NiFi built with Java 1.8 needs to run on Java 9
> ---
>
> Key: NIFI-5175
> URL: https://issues.apache.org/jira/browse/NIFI-5175
> Project: Apache NiFi
>  Issue Type: Sub-task
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Major
>
> The following issues have been encountered while attempting to run a Java 
> 1.8-built NiFi on Java 9:
> ||Issue||Solution||Status||
> |Java 9 ServiceLoader requires providers have a default no-arg public 
> constructor|Add no-arg public constructors| In Progress|
> |NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
> stop|Detect if NiFi is running on Java 9, and reflectively invoke 
> Process.pid(), which was newly added to the Process API in Java 9| Done|
>  
> 
>  
> ||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
> 9+||Description||Solution||
> |WARNING: An illegal reflective access operation has occurred|Reflective 
> invocations are common in the code used in NiFi and its dependencies on Java 
> 1.8|Full migration to Java 9 and using dependencies that are Java 9 compilant|



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-5175) NiFi built with Java 1.8 needs to run on Java 9

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck updated NIFI-5175:
--
Description: 
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||Status||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|In Progress|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9|Done|

 

 
||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred|Reflective 
invocations are common in the code used in NiFi and its dependencies on Java 
1.8|Full migration to Java 9 and using dependencies that are Java 9 compilant|

  was:
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||Status||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors| In Progress|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9| Done|

 

 
||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred|Reflective 
invocations are common in the code used in NiFi and its dependencies on Java 
1.8|Full migration to Java 9 and using dependencies that are Java 9 compilant|


> NiFi built with Java 1.8 needs to run on Java 9
> ---
>
> Key: NIFI-5175
> URL: https://issues.apache.org/jira/browse/NIFI-5175
> Project: Apache NiFi
>  Issue Type: Sub-task
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Major
>
> The following issues have been encountered while attempting to run a Java 
> 1.8-built NiFi on Java 9:
> ||Issue||Solution||Status||
> |Java 9 ServiceLoader requires providers have a default no-arg public 
> constructor|Add no-arg public constructors|In Progress|
> |NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
> stop|Detect if NiFi is running on Java 9, and reflectively invoke 
> Process.pid(), which was newly added to the Process API in Java 9|Done|
>  
> 
>  
> ||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
> 9+||Description||Solution||
> |WARNING: An illegal reflective access operation has occurred|Reflective 
> invocations are common in the code used in NiFi and its dependencies on Java 
> 1.8|Full migration to Java 9 and using dependencies that are Java 9 compilant|



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-5175) NiFi built with Java 1.8 needs to run on Java 9

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck updated NIFI-5175:
--
Description: 
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||Solved||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors| |
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9| |

 

 
||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred|Reflective 
invocations are common in the code used in NiFi and its dependencies on Java 
1.8|Full migration to Java 9 and using dependencies that are Java 9 compilant|

  was:
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9|

 

 
||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred|Reflective 
invocations are common in the code used in NiFi and its dependencies on Java 
1.8|Full migration to Java 9 and using dependencies that are Java 9 compilant|


> NiFi built with Java 1.8 needs to run on Java 9
> ---
>
> Key: NIFI-5175
> URL: https://issues.apache.org/jira/browse/NIFI-5175
> Project: Apache NiFi
>  Issue Type: Sub-task
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Major
>
> The following issues have been encountered while attempting to run a Java 
> 1.8-built NiFi on Java 9:
> ||Issue||Solution||Solved||
> |Java 9 ServiceLoader requires providers have a default no-arg public 
> constructor|Add no-arg public constructors| |
> |NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
> stop|Detect if NiFi is running on Java 9, and reflectively invoke 
> Process.pid(), which was newly added to the Process API in Java 9| |
>  
> 
>  
> ||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
> 9+||Description||Solution||
> |WARNING: An illegal reflective access operation has occurred|Reflective 
> invocations are common in the code used in NiFi and its dependencies on Java 
> 1.8|Full migration to Java 9 and using dependencies that are Java 9 compilant|



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-5175) NiFi built with Java 1.8 needs to run on Java 9

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck updated NIFI-5175:
--
Description: 
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9|

 

 
||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred|Reflective 
invocations are common in the code used in NiFi and its dependencies on Java 
1.8|Full migration to Java 9 and using dependencies that are Java 9 compilant|

  was:
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9|
\\

\\
||Unaddressed issues of NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred|Reflective 
invocations are common in the code used in NiFi and its dependencies on Java 
1.8|Full migration to Java 9 and using dependencies that are Java 9 compilant|


> NiFi built with Java 1.8 needs to run on Java 9
> ---
>
> Key: NIFI-5175
> URL: https://issues.apache.org/jira/browse/NIFI-5175
> Project: Apache NiFi
>  Issue Type: Sub-task
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Major
>
> The following issues have been encountered while attempting to run a Java 
> 1.8-built NiFi on Java 9:
> ||Issue||Solution||
> |Java 9 ServiceLoader requires providers have a default no-arg public 
> constructor|Add no-arg public constructors|
> |NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
> stop|Detect if NiFi is running on Java 9, and reflectively invoke 
> Process.pid(), which was newly added to the Process API in Java 9|
>  
> 
>  
> ||Unaddressed issues/warnings with NiFi compiled on Java 1.8 running on Java 
> 9+||Description||Solution||
> |WARNING: An illegal reflective access operation has occurred|Reflective 
> invocations are common in the code used in NiFi and its dependencies on Java 
> 1.8|Full migration to Java 9 and using dependencies that are Java 9 compilant|



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-5175) NiFi built with Java 1.8 needs to run on Java 9

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck updated NIFI-5175:
--
Description: 
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9|
\\

\\
||Unaddressed issues of NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred|Reflective 
invocations are common in the code used in NiFi and its dependencies on Java 
1.8|Full migration to Java 9 and using dependencies that are Java 9 compilant|

  was:
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9|

||Unaddressed issues of NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred|Reflective 
invocations are common in the code used in NiFi and its dependencies on Java 
1.8|Full migration to Java 9 and using dependencies that are Java 9 compilant|


> NiFi built with Java 1.8 needs to run on Java 9
> ---
>
> Key: NIFI-5175
> URL: https://issues.apache.org/jira/browse/NIFI-5175
> Project: Apache NiFi
>  Issue Type: Sub-task
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Major
>
> The following issues have been encountered while attempting to run a Java 
> 1.8-built NiFi on Java 9:
> ||Issue||Solution||
> |Java 9 ServiceLoader requires providers have a default no-arg public 
> constructor|Add no-arg public constructors|
> |NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
> stop|Detect if NiFi is running on Java 9, and reflectively invoke 
> Process.pid(), which was newly added to the Process API in Java 9|
> \\
> 
> \\
> ||Unaddressed issues of NiFi compiled on Java 1.8 running on Java 
> 9+||Description||Solution||
> |WARNING: An illegal reflective access operation has occurred|Reflective 
> invocations are common in the code used in NiFi and its dependencies on Java 
> 1.8|Full migration to Java 9 and using dependencies that are Java 9 compilant|



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-5175) NiFi built with Java 1.8 needs to run on Java 9

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck updated NIFI-5175:
--
Description: 
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
stop|Detect if NiFi is running on Java 9, and reflectively invoke 
Process.pid(), which was newly added to the Process API in Java 9|

||Unaddressed issues of NiFi compiled on Java 1.8 running on Java 
9+||Description||Solution||
|WARNING: An illegal reflective access operation has occurred|Reflective 
invocations are common in the code used in NiFi and its dependencies on Java 
1.8|Full migration to Java 9 and using dependencies that are Java 9 compilant|

  was:
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh stop| |


> NiFi built with Java 1.8 needs to run on Java 9
> ---
>
> Key: NIFI-5175
> URL: https://issues.apache.org/jira/browse/NIFI-5175
> Project: Apache NiFi
>  Issue Type: Sub-task
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Major
>
> The following issues have been encountered while attempting to run a Java 
> 1.8-built NiFi on Java 9:
> ||Issue||Solution||
> |Java 9 ServiceLoader requires providers have a default no-arg public 
> constructor|Add no-arg public constructors|
> |NiFI boostrap not able to determine PID, restarts nifi after nifi.sh 
> stop|Detect if NiFi is running on Java 9, and reflectively invoke 
> Process.pid(), which was newly added to the Process API in Java 9|
> 
> ||Unaddressed issues of NiFi compiled on Java 1.8 running on Java 
> 9+||Description||Solution||
> |WARNING: An illegal reflective access operation has occurred|Reflective 
> invocations are common in the code used in NiFi and its dependencies on Java 
> 1.8|Full migration to Java 9 and using dependencies that are Java 9 compilant|



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5191) Revert NIFI-4631 since it causes issues with CIFS

[Otto Fowler (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474837#comment-16474837
 ] 

Otto Fowler commented on NIFI-5191:
---

No, I mean i can't run `tree` in the console.  It is slow without nifi

> Revert NIFI-4631 since it causes issues with CIFS
> -
>
> Key: NIFI-5191
> URL: https://issues.apache.org/jira/browse/NIFI-5191
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Marco Gaido
>Priority: Major
>
> NIFI-4631 was done to improve performances. Unfortunately, it relies on the 
> NIO API which has better performance but it also has problems with CIFS 
> mounted filesystems.
> In the previous configuration, the processor was working properly listing all 
> files, but it was slow. Now it doesn't return the files within the directory.
> So we should revert the change.
> Unfortunately, I cannot attach logs, since the environment in which it 
> happens cannot have log data shared back out.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MINIFICPP-469) Implement base64 encode/decode EL functions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474768#comment-16474768
 ] 

ASF GitHub Bot commented on MINIFICPP-469:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/307


> Implement base64 encode/decode EL functions
> ---
>
> Key: MINIFICPP-469
> URL: https://issues.apache.org/jira/browse/MINIFICPP-469
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Andrew Christianson
>Assignee: Andrew Christianson
>Priority: Major
> Fix For: 0.5.0
>
>
> * 
> [base64Encode|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#base64encode]
>  * 
> [base64Decode|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#base64decode]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (MINIFICPP-469) Implement base64 encode/decode EL functions

[Aldrin Piri (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aldrin Piri resolved MINIFICPP-469.
---
   Resolution: Fixed
Fix Version/s: 0.5.0

> Implement base64 encode/decode EL functions
> ---
>
> Key: MINIFICPP-469
> URL: https://issues.apache.org/jira/browse/MINIFICPP-469
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Andrew Christianson
>Assignee: Andrew Christianson
>Priority: Major
> Fix For: 0.5.0
>
>
> * 
> [base64Encode|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#base64encode]
>  * 
> [base64Decode|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#base64decode]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-minifi-cpp pull request #307: MINIFICPP-469 Added encode/decode base64 ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/307


---

[jira] [Commented] (MINIFICPP-469) Implement base64 encode/decode EL functions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474767#comment-16474767
 ] 

ASF GitHub Bot commented on MINIFICPP-469:
--

Github user apiri commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/307
  
reverified builds on OS X and Debian.  will merge.


> Implement base64 encode/decode EL functions
> ---
>
> Key: MINIFICPP-469
> URL: https://issues.apache.org/jira/browse/MINIFICPP-469
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Andrew Christianson
>Assignee: Andrew Christianson
>Priority: Major
>
> * 
> [base64Encode|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#base64encode]
>  * 
> [base64Decode|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#base64decode]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-minifi-cpp issue #307: MINIFICPP-469 Added encode/decode base64 EL func...

[apiri]

Github user apiri commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/307
  
reverified builds on OS X and Debian.  will merge.


---

[jira] [Updated] (NIFI-5175) NiFi built with Java 1.8 needs to run on Java 9

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck updated NIFI-5175:
--
Summary: NiFi built with Java 1.8 needs to run on Java 9  (was: NiFi built 
in Java 1.8 needs to run on Java 9)

> NiFi built with Java 1.8 needs to run on Java 9
> ---
>
> Key: NIFI-5175
> URL: https://issues.apache.org/jira/browse/NIFI-5175
> Project: Apache NiFi
>  Issue Type: Sub-task
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Major
>
> The following issues have been encountered while attempting to run a Java 
> 1.8-built NiFi on Java 9:
> ||Issue||Solution||
> |Java 9 ServiceLoader requires providers have a default no-arg public 
> constructor|Add no-arg public constructors|
> |NiFI boostrap not able to determine PID, restarts nifi after nifi.sh stop| |



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5191) Revert NIFI-4631 since it causes issues with CIFS

[Marco Gaido (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474739#comment-16474739
 ] 

Marco Gaido commented on NIFI-5191:
---

Yes, exactly. So I think we have to revert the fix which was done for 
performance reason, meanwhile we have to understand the problem and eventually 
come out with a new and better solution which works also in this environment. 
Any help in debugging and finding the best solution would be great.

Thank you [~ottobackwards] for looking at this.

> Revert NIFI-4631 since it causes issues with CIFS
> -
>
> Key: NIFI-5191
> URL: https://issues.apache.org/jira/browse/NIFI-5191
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Marco Gaido
>Priority: Major
>
> NIFI-4631 was done to improve performances. Unfortunately, it relies on the 
> NIO API which has better performance but it also has problems with CIFS 
> mounted filesystems.
> In the previous configuration, the processor was working properly listing all 
> files, but it was slow. Now it doesn't return the files within the directory.
> So we should revert the change.
> Unfortunately, I cannot attach logs, since the environment in which it 
> happens cannot have log data shared back out.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-5194) ConsumeKafka processor can occasionally hang

[Mark Payne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Payne updated NIFI-5194:
-
Status: Patch Available  (was: Open)

> ConsumeKafka processor can occasionally hang
> 
>
> Key: NIFI-5194
> URL: https://issues.apache.org/jira/browse/NIFI-5194
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.7.0
>
>
> When using ConsumeKafka, we can occasionally get into a situation where the 
> threads will "hang" and stopping the processor won't help. In such a case, we 
> can see the following stack trace in thread dumps:
> {code:java}
> "Timer-Driven Process Thread-42" Id=201 WAITING on 
> java.util.concurrent.locks.ReentrantLock$NonfairSync@502555e6
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> at 
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
> at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
> at 
> org.apache.nifi.processors.kafka.pubsub.ConsumerLease.poll(ConsumerLease.java:147)
> at 
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka.onTrigger(ConsumeKafka.java:367)
> at 
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at 
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at 
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Number of Locked Synchronizers: 1
> - java.util.concurrent.ThreadPoolExecutor$Worker@6759e7f6
> {code}
>  
> Note that this issue affects only ConsumeKafka and not ConsumeKafka_0_10, 
> ConsumeKafka_0_11, ConsumeKafka_1_0, or any ConsumeKafkaRecord* processors. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5194) ConsumeKafka processor can occasionally hang

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474723#comment-16474723
 ] 

ASF GitHub Bot commented on NIFI-5194:
--

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/2701

NIFI-5194: Ensure that even if calling KafkaConsumer.resume() throws …

…an Exception, that we still release the lock that we are holding

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-5194

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2701.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2701


commit 4b7bba92b22fb6cdebb4ab5cddd18ebf374b5c51
Author: Mark Payne 
Date:   2018-05-14T19:50:50Z

NIFI-5194: Ensure that even if calling KafkaConsumer.resume() throws an 
Exception, that we still release the lock that we are holding




> ConsumeKafka processor can occasionally hang
> 
>
> Key: NIFI-5194
> URL: https://issues.apache.org/jira/browse/NIFI-5194
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.7.0
>
>
> When using ConsumeKafka, we can occasionally get into a situation where the 
> threads will "hang" and stopping the processor won't help. In such a case, we 
> can see the following stack trace in thread dumps:
> {code:java}
> "Timer-Driven Process Thread-42" Id=201 WAITING on 
> java.util.concurrent.locks.ReentrantLock$NonfairSync@502555e6
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> at 
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
> at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
> at 
> org.apache.nifi.processors.kafka.pubsub.ConsumerLease.poll(ConsumerLease.java:147)
> at 
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka.onTrigger(ConsumeKafka.java:367)
> at 
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at 
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at 
> 

[GitHub] nifi pull request #2701: NIFI-5194: Ensure that even if calling KafkaConsume...

[markap14]

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/2701

NIFI-5194: Ensure that even if calling KafkaConsumer.resume() throws …

…an Exception, that we still release the lock that we are holding

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-5194

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2701.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2701


commit 4b7bba92b22fb6cdebb4ab5cddd18ebf374b5c51
Author: Mark Payne 
Date:   2018-05-14T19:50:50Z

NIFI-5194: Ensure that even if calling KafkaConsumer.resume() throws an 
Exception, that we still release the lock that we are holding




---

[jira] [Commented] (MINIFICPP-495) bootstrap.sh will exit with error in default install Debian systems

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474719#comment-16474719
 ] 

ASF GitHub Bot commented on MINIFICPP-495:
--

GitHub user apiri opened a pull request:

https://github.com/apache/nifi-minifi-cpp/pull/330

MINIFICPP-495 Move shebang line to first in the file.

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced
 in the commit message?

- [X] Does your PR title start with MINIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [X] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [X] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file?
- [ ] If applicable, have you updated the NOTICE file?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apiri/nifi-minifi-cpp MINIFI-495

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi-cpp/pull/330.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #330


commit 58026f926f7f36fd90d8e2802bc2876ef6035051
Author: Aldrin Piri 
Date:   2018-05-14T19:49:09Z

MINIFICPP-495 Move shebang line to first in the file.




> bootstrap.sh will exit with error in default install Debian systems
> ---
>
> Key: MINIFICPP-495
> URL: https://issues.apache.org/jira/browse/MINIFICPP-495
> Project: NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Aldrin Piri
>Assignee: Aldrin Piri
>Priority: Major
> Fix For: 0.5.0
>
>
> With the shebang line following comments, the default non-interactive shell 
> will be used on the system.  Because Debian systems default to dash for 
> non-interactive shells the bootstrap will fail starting at Line 35 where bash 
> syntax is utilized. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (MINIFICPP-495) bootstrap.sh will exit with error in default install Debian systems

[Aldrin Piri (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aldrin Piri updated MINIFICPP-495:
--
Status: Patch Available  (was: Open)

> bootstrap.sh will exit with error in default install Debian systems
> ---
>
> Key: MINIFICPP-495
> URL: https://issues.apache.org/jira/browse/MINIFICPP-495
> Project: NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Aldrin Piri
>Assignee: Aldrin Piri
>Priority: Major
> Fix For: 0.5.0
>
>
> With the shebang line following comments, the default non-interactive shell 
> will be used on the system.  Because Debian systems default to dash for 
> non-interactive shells the bootstrap will fail starting at Line 35 where bash 
> syntax is utilized. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-minifi-cpp pull request #330: MINIFICPP-495 Move shebang line to first ...

[apiri]

GitHub user apiri opened a pull request:

https://github.com/apache/nifi-minifi-cpp/pull/330

MINIFICPP-495 Move shebang line to first in the file.

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced
 in the commit message?

- [X] Does your PR title start with MINIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [X] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [X] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file?
- [ ] If applicable, have you updated the NOTICE file?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apiri/nifi-minifi-cpp MINIFI-495

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi-cpp/pull/330.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #330


commit 58026f926f7f36fd90d8e2802bc2876ef6035051
Author: Aldrin Piri 
Date:   2018-05-14T19:49:09Z

MINIFICPP-495 Move shebang line to first in the file.




---

[jira] [Created] (NIFI-5194) ConsumeKafka processor can occasionally hang

[Mark Payne (JIRA)]

Mark Payne created NIFI-5194:


 Summary: ConsumeKafka processor can occasionally hang
 Key: NIFI-5194
 URL: https://issues.apache.org/jira/browse/NIFI-5194
 Project: Apache NiFi
  Issue Type: Bug
  Components: Extensions
Reporter: Mark Payne
Assignee: Mark Payne
 Fix For: 1.7.0


When using ConsumeKafka, we can occasionally get into a situation where the 
threads will "hang" and stopping the processor won't help. In such a case, we 
can see the following stack trace in thread dumps:
{code:java}
"Timer-Driven Process Thread-42" Id=201 WAITING on 
java.util.concurrent.locks.ReentrantLock$NonfairSync@502555e6
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
at 
java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
at 
org.apache.nifi.processors.kafka.pubsub.ConsumerLease.poll(ConsumerLease.java:147)
at 
org.apache.nifi.processors.kafka.pubsub.ConsumeKafka.onTrigger(ConsumeKafka.java:367)
at 
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at 
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
at 
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at 
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at 
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Number of Locked Synchronizers: 1
- java.util.concurrent.ThreadPoolExecutor$Worker@6759e7f6
{code}
 

Note that this issue affects only ConsumeKafka and not ConsumeKafka_0_10, 
ConsumeKafka_0_11, ConsumeKafka_1_0, or any ConsumeKafkaRecord* processors. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (MINIFICPP-495) bootstrap.sh will exit with error in default install Debian systems

[Aldrin Piri (JIRA)]

Aldrin Piri created MINIFICPP-495:
-

 Summary: bootstrap.sh will exit with error in default install 
Debian systems
 Key: MINIFICPP-495
 URL: https://issues.apache.org/jira/browse/MINIFICPP-495
 Project: NiFi MiNiFi C++
  Issue Type: Bug
Reporter: Aldrin Piri
Assignee: Aldrin Piri
 Fix For: 0.5.0


With the shebang line following comments, the default non-interactive shell 
will be used on the system.  Because Debian systems default to dash for 
non-interactive shells the bootstrap will fail starting at Line 35 where bash 
syntax is utilized. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2671: NiFi-5102 - Adding Processors for MarkLogic DB

[vivekmuniyandi]

Github user vivekmuniyandi commented on the issue:

https://github.com/apache/nifi/pull/2671
  
@joewitt I have addressed all your comments except for the License and 
Notice comments. Can you please let us know what more should we add apart from 
the LICENSE and NOTICE file prepared by our legal team which we have included 
in the root directory of the nar? That constitutes for all the dependecies 
added. What more should be added? Please help. Thanks.


---

[GitHub] nifi issue #2671: NiFi-5102 - Adding Processors for MarkLogic DB

[vivekmuniyandi]

Github user vivekmuniyandi commented on the issue:

https://github.com/apache/nifi/pull/2671
  
@MikeThomsen Sure, will add that to our backlog. Thanks!


---

[jira] [Commented] (NIFI-5192) Allow expression language in 'Schema File' property for ValidateXML processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474655#comment-16474655
 ] 

ASF GitHub Bot commented on NIFI-5192:
--

Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2699#discussion_r188064016
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateXml.java
 ---
@@ -72,6 +72,7 @@
 .name("Schema File")
 .description("The path to the Schema file that is to be used 
for validation")
 .required(true)
+.expressionLanguageSupported(true)
--- End diff --

contrib-check should fail for this no?


> Allow expression language in 'Schema File' property for ValidateXML processor 
> --
>
> Key: NIFI-5192
> URL: https://issues.apache.org/jira/browse/NIFI-5192
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Bartłomiej Tartanus
>Priority: Minor
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> Is there any reason that ValidateXML processor doesn't allow expression 
> language in 'Schema File' property? If no it would be useful to allow this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2699: [NIFI-5192] allow expression language in Schema Fil...

[ottobackwards]

Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2699#discussion_r188064016
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateXml.java
 ---
@@ -72,6 +72,7 @@
 .name("Schema File")
 .description("The path to the Schema file that is to be used 
for validation")
 .required(true)
+.expressionLanguageSupported(true)
--- End diff --

contrib-check should fail for this no?


---

[jira] [Commented] (MINIFICPP-476) Support true/false literals as function args in EL

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474651#comment-16474651
 ] 

ASF GitHub Bot commented on MINIFICPP-476:
--

GitHub user achristianson opened a pull request:

https://github.com/apache/nifi-minifi-cpp/pull/329

MINIFICPP-476 Added support for literal true/false values

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
 in the commit message?

- [x] Does your PR title start with MINIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x ] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] If applicable, have you updated the LICENSE file?
- [x] If applicable, have you updated the NOTICE file?

### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFICPP-476

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi-cpp/pull/329.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #329


commit 1e3ceecb1a3c5ad494bfd5b0fae19ef8cfe58f14
Author: Andrew I. Christianson 
Date:   2018-05-14T19:02:28Z

MINIFICPP-476 Added support for literal true/false values




> Support true/false literals as function args in EL
> --
>
> Key: MINIFICPP-476
> URL: https://issues.apache.org/jira/browse/MINIFICPP-476
> Project: NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Andrew Christianson
>Assignee: Andrew Christianson
>Priority: Major
>
> Currently writing "true" as a literal function arg causes an EL parse 
> exception.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-minifi-cpp pull request #329: MINIFICPP-476 Added support for literal t...

[achristianson]

GitHub user achristianson opened a pull request:

https://github.com/apache/nifi-minifi-cpp/pull/329

MINIFICPP-476 Added support for literal true/false values

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
 in the commit message?

- [x] Does your PR title start with MINIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x ] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] If applicable, have you updated the LICENSE file?
- [x] If applicable, have you updated the NOTICE file?

### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFICPP-476

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi-cpp/pull/329.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #329


commit 1e3ceecb1a3c5ad494bfd5b0fae19ef8cfe58f14
Author: Andrew I. Christianson 
Date:   2018-05-14T19:02:28Z

MINIFICPP-476 Added support for literal true/false values




---

[jira] [Commented] (NIFI-5113) Add XML record writer

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474600#comment-16474600
 ] 

ASF GitHub Bot commented on NIFI-5113:
--

Github user MikeThomsen commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2675#discussion_r188055672
  
--- Diff: 
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/xml/XMLRecordSetWriter.java
 ---
@@ -0,0 +1,196 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.xml;
+
+import org.apache.nifi.NullSuppression;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.serialization.DateTimeTextRecordSetWriter;
+import org.apache.nifi.serialization.RecordSetWriter;
+import org.apache.nifi.serialization.RecordSetWriterFactory;
+import org.apache.nifi.serialization.record.RecordSchema;
+
+import java.io.IOException;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+
+@Tags({"xml", "resultset", "writer", "serialize", "record", "recordset", 
"row"})
+@CapabilityDescription("Writes a RecordSet to XML. The records are wrapped 
by a root tag.")
+public class XMLRecordSetWriter extends DateTimeTextRecordSetWriter 
implements RecordSetWriterFactory {
+
+public static final AllowableValue ALWAYS_SUPPRESS = new 
AllowableValue("always-suppress", "Always Suppress",
+"Fields that are missing (present in the schema but not in the 
record), or that have a value of null, will not be written out");
+public static final AllowableValue NEVER_SUPPRESS = new 
AllowableValue("never-suppress", "Never Suppress",
+"Fields that are missing (present in the schema but not in the 
record), or that have a value of null, will be written out as a null value");
+public static final AllowableValue SUPPRESS_MISSING = new 
AllowableValue("suppress-missing", "Suppress Missing Values",
+"When a field has a value of null, it will be written out. 
However, if a field is defined in the schema and not present in the record, the 
field will not be written out.");
+
+public static final AllowableValue USE_PROPERTY_AS_WRAPPER = new 
AllowableValue("use-property-as-wrapper", "Use Property as Wrapper",
+"The value of the property \"Array Tag Name\" will be used as 
the tag name to wrap elements of an array. The field name of the array field 
will be used for the tag name " +
+"of the elements.");
+public static final AllowableValue USE_PROPERTY_FOR_ELEMENTS = new 
AllowableValue("use-property-for-elements", "Use Property for Elements",
+"The value of the property \"Array Tag Name\" will be used for 
the tag name of the elements of an array. The field name of the array field 
will be used as the tag name " +
+"to wrap elements.");
+public static final AllowableValue NO_WRAPPING = new 
AllowableValue("no-wrapping", "No Wrapping",
+"The elements of an array will not be wrapped");
+
+public static final PropertyDescriptor SUPPRESS_NULLS = new 
PropertyDescriptor.Builder()
+.name("suppress_nulls")
+.displayName("Suppress Null Values")
+.description("Specifies how 

[GitHub] nifi pull request #2675: NIFI-5113 Add XMLRecordSetWriter

[MikeThomsen]

Github user MikeThomsen commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2675#discussion_r188055672
  
--- Diff: 
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/xml/XMLRecordSetWriter.java
 ---
@@ -0,0 +1,196 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.xml;
+
+import org.apache.nifi.NullSuppression;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.serialization.DateTimeTextRecordSetWriter;
+import org.apache.nifi.serialization.RecordSetWriter;
+import org.apache.nifi.serialization.RecordSetWriterFactory;
+import org.apache.nifi.serialization.record.RecordSchema;
+
+import java.io.IOException;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+
+@Tags({"xml", "resultset", "writer", "serialize", "record", "recordset", 
"row"})
+@CapabilityDescription("Writes a RecordSet to XML. The records are wrapped 
by a root tag.")
+public class XMLRecordSetWriter extends DateTimeTextRecordSetWriter 
implements RecordSetWriterFactory {
+
+public static final AllowableValue ALWAYS_SUPPRESS = new 
AllowableValue("always-suppress", "Always Suppress",
+"Fields that are missing (present in the schema but not in the 
record), or that have a value of null, will not be written out");
+public static final AllowableValue NEVER_SUPPRESS = new 
AllowableValue("never-suppress", "Never Suppress",
+"Fields that are missing (present in the schema but not in the 
record), or that have a value of null, will be written out as a null value");
+public static final AllowableValue SUPPRESS_MISSING = new 
AllowableValue("suppress-missing", "Suppress Missing Values",
+"When a field has a value of null, it will be written out. 
However, if a field is defined in the schema and not present in the record, the 
field will not be written out.");
+
+public static final AllowableValue USE_PROPERTY_AS_WRAPPER = new 
AllowableValue("use-property-as-wrapper", "Use Property as Wrapper",
+"The value of the property \"Array Tag Name\" will be used as 
the tag name to wrap elements of an array. The field name of the array field 
will be used for the tag name " +
+"of the elements.");
+public static final AllowableValue USE_PROPERTY_FOR_ELEMENTS = new 
AllowableValue("use-property-for-elements", "Use Property for Elements",
+"The value of the property \"Array Tag Name\" will be used for 
the tag name of the elements of an array. The field name of the array field 
will be used as the tag name " +
+"to wrap elements.");
+public static final AllowableValue NO_WRAPPING = new 
AllowableValue("no-wrapping", "No Wrapping",
+"The elements of an array will not be wrapped");
+
+public static final PropertyDescriptor SUPPRESS_NULLS = new 
PropertyDescriptor.Builder()
+.name("suppress_nulls")
+.displayName("Suppress Null Values")
+.description("Specifies how the writer should handle a null 
field")
+.allowableValues(NEVER_SUPPRESS, ALWAYS_SUPPRESS, 
SUPPRESS_MISSING)
+.defaultValue(NEVER_SUPPRESS.getValue())
+.required(true)
+

[jira] [Commented] (NIFI-5189) If a schema is accessed using 'Use 'Schema Text' Property', the name of the schema is not available

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474596#comment-16474596
 ] 

ASF GitHub Bot commented on NIFI-5189:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2700


> If a schema is accessed using 'Use 'Schema Text' Property', the name of the 
> schema is not available
> ---
>
> Key: NIFI-5189
> URL: https://issues.apache.org/jira/browse/NIFI-5189
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Johannes Peter
>Assignee: Johannes Peter
>Priority: Major
> Fix For: 1.7.0
>
>
> If a schema is accessed using 'Use 'Schema Text' Property', the Avro schema 
> object will be transformed to a RecordSchema using the method 
> AvroTypeUtil.create(Schema avroSchema). This method returns a RecordSchema 
> with an empty SchemaIdentifier. Therefore, the name of the schema cannot be 
> accessed. The method should at least return a RecordSchema with a 
> SchemaIdentifier containing the name of the schema. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (NIFI-5189) If a schema is accessed using 'Use 'Schema Text' Property', the name of the schema is not available

[Mike Thomsen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Thomsen resolved NIFI-5189.

   Resolution: Fixed
Fix Version/s: 1.7.0

Based on the description in the linked issue, and what I saw in the patch it 
looks correct. Merged.

> If a schema is accessed using 'Use 'Schema Text' Property', the name of the 
> schema is not available
> ---
>
> Key: NIFI-5189
> URL: https://issues.apache.org/jira/browse/NIFI-5189
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Johannes Peter
>Assignee: Johannes Peter
>Priority: Major
> Fix For: 1.7.0
>
>
> If a schema is accessed using 'Use 'Schema Text' Property', the Avro schema 
> object will be transformed to a RecordSchema using the method 
> AvroTypeUtil.create(Schema avroSchema). This method returns a RecordSchema 
> with an empty SchemaIdentifier. Therefore, the name of the schema cannot be 
> accessed. The method should at least return a RecordSchema with a 
> SchemaIdentifier containing the name of the schema. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2700: NIFI-5189 Schema name is not available for RecordSc...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2700


---

[jira] [Commented] (NIFI-5189) If a schema is accessed using 'Use 'Schema Text' Property', the name of the schema is not available

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474594#comment-16474594
 ] 

ASF subversion and git services commented on NIFI-5189:
---

Commit 04745a556a3805597948f672292959e684401ddf in nifi's branch 
refs/heads/master from [~jope]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=04745a5 ]

NIFI-5189 Schema name is not available for RecordSchema

This closes #2700

Signed-off-by: Mike Thomsen 


> If a schema is accessed using 'Use 'Schema Text' Property', the name of the 
> schema is not available
> ---
>
> Key: NIFI-5189
> URL: https://issues.apache.org/jira/browse/NIFI-5189
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Johannes Peter
>Assignee: Johannes Peter
>Priority: Major
>
> If a schema is accessed using 'Use 'Schema Text' Property', the Avro schema 
> object will be transformed to a RecordSchema using the method 
> AvroTypeUtil.create(Schema avroSchema). This method returns a RecordSchema 
> with an empty SchemaIdentifier. Therefore, the name of the schema cannot be 
> accessed. The method should at least return a RecordSchema with a 
> SchemaIdentifier containing the name of the schema. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-5114) Support Basic Authentication at WebSocket components

[Mike Thomsen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Thomsen updated NIFI-5114:
---
Fix Version/s: 1.7.0

> Support Basic Authentication at WebSocket components
> 
>
> Key: NIFI-5114
> URL: https://issues.apache.org/jira/browse/NIFI-5114
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>Priority: Major
> Fix For: 1.7.0
>
>
> Some WebSocket server uses Basic Authentication for WebSocket upgrade 
> requests. In order to connect such endpoints, NiFi should allow users to 
> configure user name and password at WebSocket client component. Specifically 
> JettyWebSocketClientService.
> In addition to that, if JettyWebSocketServerService can be configured to use 
> Basic Authentication, it makes NiFi WebSocket server more secure and also 
> useful to test Basic Auth between NiFi WebSocket client components. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5189) If a schema is accessed using 'Use 'Schema Text' Property', the name of the schema is not available

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474590#comment-16474590
 ] 

ASF GitHub Bot commented on NIFI-5189:
--

Github user JohannesDaniel commented on the issue:

https://github.com/apache/nifi/pull/2700
  
Discussion related to this ticket: 
https://github.com/apache/nifi/pull/2675#discussion_r186106245


> If a schema is accessed using 'Use 'Schema Text' Property', the name of the 
> schema is not available
> ---
>
> Key: NIFI-5189
> URL: https://issues.apache.org/jira/browse/NIFI-5189
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Johannes Peter
>Assignee: Johannes Peter
>Priority: Major
>
> If a schema is accessed using 'Use 'Schema Text' Property', the Avro schema 
> object will be transformed to a RecordSchema using the method 
> AvroTypeUtil.create(Schema avroSchema). This method returns a RecordSchema 
> with an empty SchemaIdentifier. Therefore, the name of the schema cannot be 
> accessed. The method should at least return a RecordSchema with a 
> SchemaIdentifier containing the name of the schema. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-5114) Support Basic Authentication at WebSocket components

[Mike Thomsen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Thomsen updated NIFI-5114:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Checked it over, looked good and all tests ran.

> Support Basic Authentication at WebSocket components
> 
>
> Key: NIFI-5114
> URL: https://issues.apache.org/jira/browse/NIFI-5114
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>Priority: Major
>
> Some WebSocket server uses Basic Authentication for WebSocket upgrade 
> requests. In order to connect such endpoints, NiFi should allow users to 
> configure user name and password at WebSocket client component. Specifically 
> JettyWebSocketClientService.
> In addition to that, if JettyWebSocketServerService can be configured to use 
> Basic Authentication, it makes NiFi WebSocket server more secure and also 
> useful to test Basic Auth between NiFi WebSocket client components. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5114) Support Basic Authentication at WebSocket components

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474589#comment-16474589
 ] 

ASF GitHub Bot commented on NIFI-5114:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2652


> Support Basic Authentication at WebSocket components
> 
>
> Key: NIFI-5114
> URL: https://issues.apache.org/jira/browse/NIFI-5114
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>Priority: Major
>
> Some WebSocket server uses Basic Authentication for WebSocket upgrade 
> requests. In order to connect such endpoints, NiFi should allow users to 
> configure user name and password at WebSocket client component. Specifically 
> JettyWebSocketClientService.
> In addition to that, if JettyWebSocketServerService can be configured to use 
> Basic Authentication, it makes NiFi WebSocket server more secure and also 
> useful to test Basic Auth between NiFi WebSocket client components. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2652: NIFI-5114: Added Basic auth support to WebSocket co...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2652


---

[GitHub] nifi issue #2700: NIFI-5189 Schema name is not available for RecordSchema

[JohannesDaniel]

Github user JohannesDaniel commented on the issue:

https://github.com/apache/nifi/pull/2700
  
Discussion related to this ticket: 
https://github.com/apache/nifi/pull/2675#discussion_r186106245


---

[jira] [Commented] (NIFI-5114) Support Basic Authentication at WebSocket components

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474587#comment-16474587
 ] 

ASF subversion and git services commented on NIFI-5114:
---

Commit 03adaeca222e2b040fce5680bbc03572d0ba473e in nifi's branch 
refs/heads/master from [~ijokarumawak]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=03adaec ]

NIFI-5114: Added Basic auth support to WebSocket components

This closes #2652

Signed-off-by: Mike Thomsen 


> Support Basic Authentication at WebSocket components
> 
>
> Key: NIFI-5114
> URL: https://issues.apache.org/jira/browse/NIFI-5114
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>Priority: Major
>
> Some WebSocket server uses Basic Authentication for WebSocket upgrade 
> requests. In order to connect such endpoints, NiFi should allow users to 
> configure user name and password at WebSocket client component. Specifically 
> JettyWebSocketClientService.
> In addition to that, if JettyWebSocketServerService can be configured to use 
> Basic Authentication, it makes NiFi WebSocket server more secure and also 
> useful to test Basic Auth between NiFi WebSocket client components. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5189) If a schema is accessed using 'Use 'Schema Text' Property', the name of the schema is not available

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474584#comment-16474584
 ] 

ASF GitHub Bot commented on NIFI-5189:
--

Github user JohannesDaniel commented on the issue:

https://github.com/apache/nifi/pull/2700
  
@markap14 I wanted to fix this before finishing the XMLRecordSetWriter. I 
especially expect beginners to use the schema access via the text property - 
they should not run into this problem.


> If a schema is accessed using 'Use 'Schema Text' Property', the name of the 
> schema is not available
> ---
>
> Key: NIFI-5189
> URL: https://issues.apache.org/jira/browse/NIFI-5189
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Johannes Peter
>Assignee: Johannes Peter
>Priority: Major
>
> If a schema is accessed using 'Use 'Schema Text' Property', the Avro schema 
> object will be transformed to a RecordSchema using the method 
> AvroTypeUtil.create(Schema avroSchema). This method returns a RecordSchema 
> with an empty SchemaIdentifier. Therefore, the name of the schema cannot be 
> accessed. The method should at least return a RecordSchema with a 
> SchemaIdentifier containing the name of the schema. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (NIFI-5193) Improve ConfigEncryptionTool handling of complex user search mapping values

[Andy LoPresto (JIRA)]

Andy LoPresto created NIFI-5193:
---

 Summary: Improve ConfigEncryptionTool handling of complex user 
search mapping values
 Key: NIFI-5193
 URL: https://issues.apache.org/jira/browse/NIFI-5193
 Project: Apache NiFi
  Issue Type: Bug
  Components: Tools and Build
Affects Versions: 1.6.0
Reporter: Andy LoPresto


The {{ConfigEncryptionTool}} can fail to encrypt 
{{login-identity-providers.xml}} or {{authorizers.xml}} if the XML contains a 
User Search Mapping value which is interpreted as having regular expression 
capture groups. 

{code}
(& 
(objectCategory=Person)(sAMAccountName=*)(!(UserAccountControl:1.2.840.113556.1.4.803:=2))(!(sAMAccountName=$*)))
{code}

Results in:

{code}
2018/05/14 15:05:22 ERROR [main] 
org.apache.nifi.properties.ConfigEncryptionTool: Encountered an error
java.lang.IllegalArgumentException: Illegal group reference
at java.util.regex.Matcher.appendReplacement(Matcher.java:857)
at java.util.regex.Matcher.replaceFirst(Matcher.java:1004)
at java.lang.String.replaceFirst(String.java:2178)
at java_lang_String$replaceFirst$6.call(Unknown Source)
at 
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133)
at 
org.apache.nifi.properties.ConfigEncryptionTool.serializeAuthorizersAndPreserveFormat(ConfigEncryptionTool.groovy:1246)
at 
org.apache.nifi.properties.ConfigEncryptionTool$serializeAuthorizersAndPreserveFormat$6.callStatic(Unknown
 Source)
at 
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:214)
at 
org.apache.nifi.properties.ConfigEncryptionTool.writeAuthorizers(ConfigEncryptionTool.groovy:1118)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)
at 
org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:71)
at 
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
at 
org.apache.nifi.properties.ConfigEncryptionTool.main(ConfigEncryptionTool.groovy:1485)
at 
org.apache.nifi.properties.ConfigEncryptionTool$main.call(Unknown Source)
at 
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at 
org.apache.nifi.toolkit.encryptconfig.LegacyMode.run(LegacyMode.groovy:30)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSite.invoke(PogoMetaMethodSite.java:169)
at 
org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:71)
at 
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at 
org.apache.nifi.toolkit.encryptconfig.EncryptConfigMain.main(EncryptConfigMain.groovy:109)
Encountered an error writing the master key to the bootstrap.conf file and the 
encrypted properties to nifi.properties
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2700: NIFI-5189 Schema name is not available for RecordSchema

[JohannesDaniel]

Github user JohannesDaniel commented on the issue:

https://github.com/apache/nifi/pull/2700
  
@markap14 I wanted to fix this before finishing the XMLRecordSetWriter. I 
especially expect beginners to use the schema access via the text property - 
they should not run into this problem.


---

[jira] [Commented] (NIFI-5189) If a schema is accessed using 'Use 'Schema Text' Property', the name of the schema is not available

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474580#comment-16474580
 ] 

ASF GitHub Bot commented on NIFI-5189:
--

GitHub user JohannesDaniel opened a pull request:

https://github.com/apache/nifi/pull/2700

NIFI-5189 Schema name is not available for RecordSchema

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/JohannesDaniel/nifi NIFI-5189

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2700.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2700


commit 4ff6be45148e6e6d4fa1121fec72dc7c51fbad22
Author: JohannesDaniel 
Date:   2018-05-14T14:28:46Z

NIFI-5189 Schema name is not available for RecordSchema




> If a schema is accessed using 'Use 'Schema Text' Property', the name of the 
> schema is not available
> ---
>
> Key: NIFI-5189
> URL: https://issues.apache.org/jira/browse/NIFI-5189
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Johannes Peter
>Assignee: Johannes Peter
>Priority: Major
>
> If a schema is accessed using 'Use 'Schema Text' Property', the Avro schema 
> object will be transformed to a RecordSchema using the method 
> AvroTypeUtil.create(Schema avroSchema). This method returns a RecordSchema 
> with an empty SchemaIdentifier. Therefore, the name of the schema cannot be 
> accessed. The method should at least return a RecordSchema with a 
> SchemaIdentifier containing the name of the schema. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2700: NIFI-5189 Schema name is not available for RecordSc...

[JohannesDaniel]

GitHub user JohannesDaniel opened a pull request:

https://github.com/apache/nifi/pull/2700

NIFI-5189 Schema name is not available for RecordSchema

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/JohannesDaniel/nifi NIFI-5189

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2700.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2700


commit 4ff6be45148e6e6d4fa1121fec72dc7c51fbad22
Author: JohannesDaniel 
Date:   2018-05-14T14:28:46Z

NIFI-5189 Schema name is not available for RecordSchema




---

[GitHub] nifi issue #2614: Added Apache Pulsar Processors and Controller Service

[david-streamlio]

Github user david-streamlio commented on the issue:

https://github.com/apache/nifi/pull/2614
  
I tend to agree with @joewitt on thisLet's start from a fresh master 
and add in the new nifi-nar-bundleIt shouldn't be this hard to add in a 
completely fresh bundle, as the files are all net new and shouldn't have any 
upstream conflicts..


---

[jira] [Commented] (NIFI-5192) Allow expression language in 'Schema File' property for ValidateXML processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474554#comment-16474554
 ] 

ASF GitHub Bot commented on NIFI-5192:
--

Github user MikeThomsen commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2699#discussion_r188045964
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateXml.java
 ---
@@ -72,6 +72,7 @@
 .name("Schema File")
 .description("The path to the Schema file that is to be used 
for validation")
 .required(true)
+.expressionLanguageSupported(true)
--- End diff --

This should use the new `ExpressionLanguageScope` type instead.


> Allow expression language in 'Schema File' property for ValidateXML processor 
> --
>
> Key: NIFI-5192
> URL: https://issues.apache.org/jira/browse/NIFI-5192
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Bartłomiej Tartanus
>Priority: Minor
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> Is there any reason that ValidateXML processor doesn't allow expression 
> language in 'Schema File' property? If no it would be useful to allow this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2699: [NIFI-5192] allow expression language in Schema Fil...

[MikeThomsen]

Github user MikeThomsen commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2699#discussion_r188045964
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateXml.java
 ---
@@ -72,6 +72,7 @@
 .name("Schema File")
 .description("The path to the Schema file that is to be used 
for validation")
 .required(true)
+.expressionLanguageSupported(true)
--- End diff --

This should use the new `ExpressionLanguageScope` type instead.


---

[jira] [Commented] (NIFI-5191) Revert NIFI-4631 since it causes issues with CIFS

[Otto Fowler (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474540#comment-16474540
 ] 

Otto Fowler commented on NIFI-5191:
---

thanks [~mgaido], So if someone can just setup a mount with some files, the 
files will not list?  

My only CIFS mount is over VPN and is too slow to be usable unfortunately.

 

> Revert NIFI-4631 since it causes issues with CIFS
> -
>
> Key: NIFI-5191
> URL: https://issues.apache.org/jira/browse/NIFI-5191
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Marco Gaido
>Priority: Major
>
> NIFI-4631 was done to improve performances. Unfortunately, it relies on the 
> NIO API which has better performance but it also has problems with CIFS 
> mounted filesystems.
> In the previous configuration, the processor was working properly listing all 
> files, but it was slow. Now it doesn't return the files within the directory.
> So we should revert the change.
> Unfortunately, I cannot attach logs, since the environment in which it 
> happens cannot have log data shared back out.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4907) Provenance authorization refactoring

[Matt Gilman (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474537#comment-16474537
 ] 

Matt Gilman commented on NIFI-4907:
---

An endpoint could be categorized as data plane (downloading or manipulating 
attributes/content, etc) or control plane (configuring a component, etc - 
basically anything not data plane). For data plane endpoints, NiFi requires the 
entire chain to be authorized. This allows the authorizer to verify that an end 
user can have the data and that any proxy in the chain is also authorized to 
have the data (since the data is ultimately routing through those servers). In 
a NiFi cluster, any node can act as the Cluster Coorindator which replicates 
(routes) the requests to any/all nodes.

The existing 'view the data' policies would be considered data plane. I do not 
believe that the provenance event details would be considered data and would 
only require the end user has the necessary permission.

The authorization in this case would be:

1) Ensure the user is allowed to 'query provenance'

For each event

2) Ensure the user is allowed to 'view provenance' for the component

For each authorized event

3) Include component name/type if the user is allowed to 'view component'

4) Include attributes and content details if the user is allowed to 'view the 
data' for the component

Step (1) should already be in place. As part of this PR, you should be 
adding/updating (2), (3), and (4). In order to do (2), you'll need to add 
support for viewing/configuring the 'view provenance' policy for each 
component. Hope this helps.

> Provenance authorization refactoring
> 
>
> Key: NIFI-4907
> URL: https://issues.apache.org/jira/browse/NIFI-4907
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.5.0
>Reporter: Mark Bean
>Assignee: Mark Bean
>Priority: Major
>
> Currently, the 'view the data' component policy is too tightly coupled with 
> Provenance queries. The 'query provenance' policy should be the only policy 
> required for viewing Provenance query results. Both 'view the component' and 
> 'view the data' policies should be used to refine the appropriate visibility 
> of event details - but not the event itself.
> 1) Component Visibility
> The authorization of Provenance events is inconsistent with the behavior of 
> the graph. For example, if a user does not have 'view the component' policy, 
> the graph shows this component as a "black box" (no details such as name, 
> UUID, etc.) However, when querying Provenance, this component will show up 
> including the Component Type and the Component Name. This is in effect a 
> violation of the policy. These component details should be obscured in the 
> Provenance event displayed if user does not have the appropriate 'view the 
> component' policy.
> 2) Data Visibility
> For a Provenance query, all events should be visible as long as the user 
> performing the query belongs to the 'query provenance' global policy. As 
> mentioned above, some information about the component may be obscured 
> depending on 'view the component' policy, but the event itself should be 
> visible. Additionally, details of the event (clicking the View Details "i" 
> icon) should only be accessible if the user belongs to the 'view the data' 
> policy for the affected component. If the user is not in the appropriate 
> 'view the data' policy, a popup warning should be displayed indicating the 
> reason details are not visible with more specific detail than the current 
> "Contact the system administrator".
> 3) Lineage Graphs
> As with the Provenance table view recommendation above, the lineage graph 
> should display all events. Currently, if the lineage graph includes an event 
> belonging to a component which the user does not have 'view the data', it is 
> shown on the graph as "UNKNOWN". As with Data Visibility mentioned above, the 
> graph should indicate the event type as long as the user is in the 'view the 
> component'. Subsequent "View Details" on the event should only be visible if 
> the user is in the 'view the data' policy.
> In summary, for Provenance query results and lineage graphs, all events 
> should be shown. Component Name and Component Type information should be 
> conditionally visible depending on the corresponding component policy 'view 
> the component' policy. Event details including Provenance event type and 
> FlowFile information should be conditionally available depending on the 
> corresponding component policy 'view the data'. Inability to display event 
> details should provide feedback to the user indicating the reason.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-5175) NiFi built in Java 1.8 needs to run on Java 9

[Jeff Storck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck updated NIFI-5175:
--
Description: 
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||
|Java 9 ServiceLoader requires providers have a default no-arg public 
constructor|Add no-arg public constructors|
|NiFI boostrap not able to determine PID, restarts nifi after nifi.sh stop| |

  was:
The following issues have been encountered while attempting to run a Java 
1.8-built NiFi on Java 9:
||Issue||Solution||
|JAXB complains that classes do not have a default no-arg public 
constructor|Add no-arg public constructors|


> NiFi built in Java 1.8 needs to run on Java 9
> -
>
> Key: NIFI-5175
> URL: https://issues.apache.org/jira/browse/NIFI-5175
> Project: Apache NiFi
>  Issue Type: Sub-task
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Major
>
> The following issues have been encountered while attempting to run a Java 
> 1.8-built NiFi on Java 9:
> ||Issue||Solution||
> |Java 9 ServiceLoader requires providers have a default no-arg public 
> constructor|Add no-arg public constructors|
> |NiFI boostrap not able to determine PID, restarts nifi after nifi.sh stop| |



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4731) BigQuery processors

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474502#comment-16474502
 ] 

ASF GitHub Bot commented on NIFI-4731:
--

Github user zenfenan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2682#discussion_r188031876
  
--- Diff: 
nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/src/main/java/org/apache/nifi/processors/gcp/bigquery/BigQueryAttributes.java
 ---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.gcp.bigquery;
+
+/**
+ * Attributes associated with the BigQuery processors
+ */
+public class BigQueryAttributes {
+private BigQueryAttributes() {}
+
+public static final String DATASET_ATTR = "bq.dataset";
--- End diff --

Prefer if we could prefix it with `gcp`. Example: `gcp.bq.dataset`. 
Thoughts?


> BigQuery processors
> ---
>
> Key: NIFI-4731
> URL: https://issues.apache.org/jira/browse/NIFI-4731
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Mikhail Sosonkin
>Priority: Major
>
> NIFI should have processors for putting data into BigQuery (Streaming and 
> Batch).
> Initial working processors can be found this repository: 
> https://github.com/nologic/nifi/tree/NIFI-4731/nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/src/main/java/org/apache/nifi/processors/gcp/bigquery
> I'd like to get them into Nifi proper.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4731) BigQuery processors

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474501#comment-16474501
 ] 

ASF GitHub Bot commented on NIFI-4731:
--

Github user zenfenan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2682#discussion_r188031584
  
--- Diff: nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/pom.xml ---
@@ -80,6 +85,16 @@
 
 
 
+
+com.google.cloud
+google-cloud-storage
+1.25.0
+
+
+com.google.cloud
+google-cloud-bigquery
+1.25.0
--- End diff --

@danieljimenez Please rebase it and make the necessary changes (on 
dependency) that @pvillard31 suggested. 


> BigQuery processors
> ---
>
> Key: NIFI-4731
> URL: https://issues.apache.org/jira/browse/NIFI-4731
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Mikhail Sosonkin
>Priority: Major
>
> NIFI should have processors for putting data into BigQuery (Streaming and 
> Batch).
> Initial working processors can be found this repository: 
> https://github.com/nologic/nifi/tree/NIFI-4731/nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/src/main/java/org/apache/nifi/processors/gcp/bigquery
> I'd like to get them into Nifi proper.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4731) BigQuery processors

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474503#comment-16474503
 ] 

ASF GitHub Bot commented on NIFI-4731:
--

Github user zenfenan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2682#discussion_r188032966
  
--- Diff: 
nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/src/main/java/org/apache/nifi/processors/gcp/bigquery/PutBigQueryStream.java
 ---
@@ -0,0 +1,360 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.gcp.bigquery;
+
+import com.google.cloud.bigquery.BigQuery;
+import com.google.cloud.bigquery.BigQueryError;
+import com.google.cloud.bigquery.InsertAllRequest;
+import com.google.cloud.bigquery.InsertAllResponse;
+import com.google.cloud.bigquery.JobInfo;
+import com.google.cloud.bigquery.Schema;
+import com.google.cloud.bigquery.StandardTableDefinition;
+import com.google.cloud.bigquery.Table;
+import com.google.cloud.bigquery.TableId;
+import com.google.cloud.bigquery.TableInfo;
+import com.google.common.collect.ImmutableList;
+import com.google.gson.Gson;
+import com.google.gson.reflect.TypeToken;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.lang.reflect.Type;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+/**
+ *
+ */
+
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"google", "google cloud", "bq", "bigquery"})
+@CapabilityDescription("Streams flow files to a Google BigQuery table.")
+
+@WritesAttributes({
+@WritesAttribute(attribute = BigQueryAttributes.DATASET_ATTR, 
description = BigQueryAttributes.DATASET_DESC),
+@WritesAttribute(attribute = BigQueryAttributes.TABLE_NAME_ATTR, 
description = BigQueryAttributes.TABLE_NAME_DESC),
+@WritesAttribute(attribute = BigQueryAttributes.TABLE_SCHEMA_ATTR, 
description = BigQueryAttributes.TABLE_SCHEMA_DESC),
+@WritesAttribute(attribute = BigQueryAttributes.BATCH_SIZE_ATTR, 
description = BigQueryAttributes.BATCH_SIZE_DESC),
+@WritesAttribute(attribute = BigQueryAttributes.MAX_ROW_SIZE_ATTR, 
description = BigQueryAttributes.MAX_ROW_SIZE_DESC),
+@WritesAttribute(attribute = 
BigQueryAttributes.CREATE_DISPOSITION_ATTR, description = 
BigQueryAttributes.CREATE_DISPOSITION_DESC),
+@WritesAttribute(attribute = 
BigQueryAttributes.TABLE_CACHE_RESET_ATTR, description = 
BigQueryAttributes.TABLE_CACHE_RESET_DESC),
+@WritesAttribute(attribute = BigQueryAttributes.JOB_ERROR_MSG_ATTR, 
description = BigQueryAttributes.JOB_ERROR_MSG_DESC),
+@WritesAttribute(attribute = BigQueryAttributes.JOB_ERROR_REASON_ATTR, 
description = BigQueryAttributes.JOB_ERROR_REASON_DESC),
+

[GitHub] nifi pull request #2682: NIFI-4731: BQ Processors and GCP library update.

[zenfenan]

Github user zenfenan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2682#discussion_r188031584
  
--- Diff: nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/pom.xml ---
@@ -80,6 +85,16 @@
 
 
 
+
+com.google.cloud
+google-cloud-storage
+1.25.0
+
+
+com.google.cloud
+google-cloud-bigquery
+1.25.0
--- End diff --

@danieljimenez Please rebase it and make the necessary changes (on 
dependency) that @pvillard31 suggested. 


---

[GitHub] nifi pull request #2682: NIFI-4731: BQ Processors and GCP library update.

[zenfenan]

Github user zenfenan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2682#discussion_r188032966
  
--- Diff: 
nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/src/main/java/org/apache/nifi/processors/gcp/bigquery/PutBigQueryStream.java
 ---
@@ -0,0 +1,360 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.gcp.bigquery;
+
+import com.google.cloud.bigquery.BigQuery;
+import com.google.cloud.bigquery.BigQueryError;
+import com.google.cloud.bigquery.InsertAllRequest;
+import com.google.cloud.bigquery.InsertAllResponse;
+import com.google.cloud.bigquery.JobInfo;
+import com.google.cloud.bigquery.Schema;
+import com.google.cloud.bigquery.StandardTableDefinition;
+import com.google.cloud.bigquery.Table;
+import com.google.cloud.bigquery.TableId;
+import com.google.cloud.bigquery.TableInfo;
+import com.google.common.collect.ImmutableList;
+import com.google.gson.Gson;
+import com.google.gson.reflect.TypeToken;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.lang.reflect.Type;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+/**
+ *
+ */
+
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"google", "google cloud", "bq", "bigquery"})
+@CapabilityDescription("Streams flow files to a Google BigQuery table.")
+
+@WritesAttributes({
+@WritesAttribute(attribute = BigQueryAttributes.DATASET_ATTR, 
description = BigQueryAttributes.DATASET_DESC),
+@WritesAttribute(attribute = BigQueryAttributes.TABLE_NAME_ATTR, 
description = BigQueryAttributes.TABLE_NAME_DESC),
+@WritesAttribute(attribute = BigQueryAttributes.TABLE_SCHEMA_ATTR, 
description = BigQueryAttributes.TABLE_SCHEMA_DESC),
+@WritesAttribute(attribute = BigQueryAttributes.BATCH_SIZE_ATTR, 
description = BigQueryAttributes.BATCH_SIZE_DESC),
+@WritesAttribute(attribute = BigQueryAttributes.MAX_ROW_SIZE_ATTR, 
description = BigQueryAttributes.MAX_ROW_SIZE_DESC),
+@WritesAttribute(attribute = 
BigQueryAttributes.CREATE_DISPOSITION_ATTR, description = 
BigQueryAttributes.CREATE_DISPOSITION_DESC),
+@WritesAttribute(attribute = 
BigQueryAttributes.TABLE_CACHE_RESET_ATTR, description = 
BigQueryAttributes.TABLE_CACHE_RESET_DESC),
+@WritesAttribute(attribute = BigQueryAttributes.JOB_ERROR_MSG_ATTR, 
description = BigQueryAttributes.JOB_ERROR_MSG_DESC),
+@WritesAttribute(attribute = BigQueryAttributes.JOB_ERROR_REASON_ATTR, 
description = BigQueryAttributes.JOB_ERROR_REASON_DESC),
+@WritesAttribute(attribute = 
BigQueryAttributes.JOB_ERROR_LOCATION_ATTR, description = 
BigQueryAttributes.JOB_ERROR_LOCATION_DESC)
+})
+
+public class PutBigQueryStream extends AbstractBigQueryProcessor {
+public static 

[GitHub] nifi pull request #2682: NIFI-4731: BQ Processors and GCP library update.

[zenfenan]

Github user zenfenan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2682#discussion_r188031876
  
--- Diff: 
nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/src/main/java/org/apache/nifi/processors/gcp/bigquery/BigQueryAttributes.java
 ---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.gcp.bigquery;
+
+/**
+ * Attributes associated with the BigQuery processors
+ */
+public class BigQueryAttributes {
+private BigQueryAttributes() {}
+
+public static final String DATASET_ATTR = "bq.dataset";
--- End diff --

Prefer if we could prefix it with `gcp`. Example: `gcp.bq.dataset`. 
Thoughts?


---

[GitHub] nifi issue #2614: Added Apache Pulsar Processors and Controller Service

[joewitt]

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2614
  
I would rework this thing personally so that it gets back to just the 
commit(s) meant to be included.  Like start fresh off latest master, apply the 
actual changes intended, then end up with a single commit and either force push 
that OR send a diff PR.  This one is tough to follow as-is but seems like a 
good/cool/useful PR.


---

[GitHub] nifi issue #2614: Added Apache Pulsar Processors and Controller Service

[MikeThomsen]

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/2614
  
@joewitt I think if he were to rebase and drop [this 
commit](https://github.com/apache/nifi/pull/2614/commits/99a980cc1e6bbb45e64fb3431545023eefb8522c)
 it should fix that. What do you think?


---

[jira] [Commented] (MINIFICPP-474) Implement getDelimitedField EL function

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474467#comment-16474467
 ] 

ASF GitHub Bot commented on MINIFICPP-474:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/328


> Implement getDelimitedField EL function
> ---
>
> Key: MINIFICPP-474
> URL: https://issues.apache.org/jira/browse/MINIFICPP-474
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Andrew Christianson
>Assignee: Andrew Christianson
>Priority: Major
>
> *Description*: Parses the Subject as a delimited line of text and returns 
> just a single field from that delimited text.
> *Subject Type*: String
> *Arguments*:
>  * _index_ : The index of the field to return. A value of 1 will return the 
> first field, a value of 2 will return the second field, and so on.
>  * _delimiter_ : Optional argument that provides the character to use as a 
> field separator. If not specified, a comma will be used. This value must be 
> exactly 1 character.
>  * _quoteChar_ : Optional argument that provides the character that can be 
> used to quote values so that the delimiter can be used within a single field. 
> If not specified, a double-quote (") will be used. This value must be exactly 
> 1 character.
>  * _escapeChar_ : Optional argument that provides the character that can be 
> used to escape the Quote Character or the Delimiter within a field. If not 
> specified, a backslash (\) is used. This value must be exactly 1 character.
>  * _stripChars_ : Optional argument that specifies whether or not quote 
> characters and escape characters should be stripped. For example, if we have 
> a field value "1, 2, 3" and this value is true, we will get the value {{1, 2, 
> 3}}, but if this value is false, we will get the value {{"1, 2, 3"}} with the 
> quotes. The default value is false. This value must be either {{true}}or 
> {{false}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-minifi-cpp pull request #328: MINIFICPP-474 Added getDelimitedField EL ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/328


---

[jira] [Commented] (MINIFICPP-474) Implement getDelimitedField EL function

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474461#comment-16474461
 ] 

ASF GitHub Bot commented on MINIFICPP-474:
--

Github user achristianson commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/328
  
@phrocker Also rm -rf thirdparty/flex. It might have some junk from another 
PR that is no longer in git.


> Implement getDelimitedField EL function
> ---
>
> Key: MINIFICPP-474
> URL: https://issues.apache.org/jira/browse/MINIFICPP-474
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Andrew Christianson
>Assignee: Andrew Christianson
>Priority: Major
>
> *Description*: Parses the Subject as a delimited line of text and returns 
> just a single field from that delimited text.
> *Subject Type*: String
> *Arguments*:
>  * _index_ : The index of the field to return. A value of 1 will return the 
> first field, a value of 2 will return the second field, and so on.
>  * _delimiter_ : Optional argument that provides the character to use as a 
> field separator. If not specified, a comma will be used. This value must be 
> exactly 1 character.
>  * _quoteChar_ : Optional argument that provides the character that can be 
> used to quote values so that the delimiter can be used within a single field. 
> If not specified, a double-quote (") will be used. This value must be exactly 
> 1 character.
>  * _escapeChar_ : Optional argument that provides the character that can be 
> used to escape the Quote Character or the Delimiter within a field. If not 
> specified, a backslash (\) is used. This value must be exactly 1 character.
>  * _stripChars_ : Optional argument that specifies whether or not quote 
> characters and escape characters should be stripped. For example, if we have 
> a field value "1, 2, 3" and this value is true, we will get the value {{1, 2, 
> 3}}, but if this value is false, we will get the value {{"1, 2, 3"}} with the 
> quotes. The default value is false. This value must be either {{true}}or 
> {{false}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-minifi-cpp issue #328: MINIFICPP-474 Added getDelimitedField EL functio...

[achristianson]

Github user achristianson commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/328
  
@phrocker Also rm -rf thirdparty/flex. It might have some junk from another 
PR that is no longer in git.


---

[jira] [Commented] (MINIFICPP-474) Implement getDelimitedField EL function

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474453#comment-16474453
 ] 

ASF GitHub Bot commented on MINIFICPP-474:
--

Github user achristianson commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/328
  
@phrocker 

Try deleting the following generated files & trying again:

/extensions/expression-language/Parser.*
/extensions/expression-language/Scanner.cpp
/extensions/expression-language/location.hh
/extensions/expression-language/position.hh
/extensions/expression-language/stack.hh



> Implement getDelimitedField EL function
> ---
>
> Key: MINIFICPP-474
> URL: https://issues.apache.org/jira/browse/MINIFICPP-474
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Andrew Christianson
>Assignee: Andrew Christianson
>Priority: Major
>
> *Description*: Parses the Subject as a delimited line of text and returns 
> just a single field from that delimited text.
> *Subject Type*: String
> *Arguments*:
>  * _index_ : The index of the field to return. A value of 1 will return the 
> first field, a value of 2 will return the second field, and so on.
>  * _delimiter_ : Optional argument that provides the character to use as a 
> field separator. If not specified, a comma will be used. This value must be 
> exactly 1 character.
>  * _quoteChar_ : Optional argument that provides the character that can be 
> used to quote values so that the delimiter can be used within a single field. 
> If not specified, a double-quote (") will be used. This value must be exactly 
> 1 character.
>  * _escapeChar_ : Optional argument that provides the character that can be 
> used to escape the Quote Character or the Delimiter within a field. If not 
> specified, a backslash (\) is used. This value must be exactly 1 character.
>  * _stripChars_ : Optional argument that specifies whether or not quote 
> characters and escape characters should be stripped. For example, if we have 
> a field value "1, 2, 3" and this value is true, we will get the value {{1, 2, 
> 3}}, but if this value is false, we will get the value {{"1, 2, 3"}} with the 
> quotes. The default value is false. This value must be either {{true}}or 
> {{false}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-minifi-cpp issue #328: MINIFICPP-474 Added getDelimitedField EL functio...

[achristianson]

Github user achristianson commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/328
  
@phrocker 

Try deleting the following generated files & trying again:

/extensions/expression-language/Parser.*
/extensions/expression-language/Scanner.cpp
/extensions/expression-language/location.hh
/extensions/expression-language/position.hh
/extensions/expression-language/stack.hh



---

[jira] [Commented] (NIFI-4494) Add a FetchOracleRow processor

[Shawn Weeks (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474450#comment-16474450
 ] 

Shawn Weeks commented on NIFI-4494:
---

We actually wrote a custom Groovy script to do effectively the same thing. 
Being able to fetch a single row via sql and write to attributes would be 
useful in a lot of cases much like the FetchHBaseRow Processor. This should be 
a generic FetchSQLRow or equivalent though and probably should raise an error 
if multiple rows are returned or provide someway to handle it.

> Add a FetchOracleRow processor
> --
>
> Key: NIFI-4494
> URL: https://issues.apache.org/jira/browse/NIFI-4494
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
> Environment: oracle
>Reporter: Fred Liu
>Priority: Major
>
> We encounter a lot of demand, poor data quality, no primary key, no time 
> stamp, and even a lot of duplicate data. But the customer requires a high 
> performance and accuracy.
> Using GenerateTableFetch or QueryDatabaseTable, we can not meet the 
> functional and performance requirements. So we want to add a new processor, 
> it is specifically for the oracle database, able to ingest very poor quality 
> data and have better performance.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2614: Added Apache Pulsar Processors and Controller Service

[joewitt]

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2614
  
@david-streamlio your PR appears to delete the nifi-grpc directory...You 
will want to avoid doing that.


---