[jira] [Assigned] (NIFI-1867) improve ModifyBytes to make it easy to remove all flowfile content

[Joe Skora (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joe Skora reassigned NIFI-1867:
---

Assignee: Joe Skora

> improve ModifyBytes to make it easy to remove all flowfile content
> --
>
> Key: NIFI-1867
> URL: https://issues.apache.org/jira/browse/NIFI-1867
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 0.6.1
>Reporter: Ben Icore
>Assignee: Joe Skora
>
> update ModifyBytes processor to include a "Remove all content" property.  
> this property shouild default to false so existing functionality is not 
> changed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (NIFI-2411) ModifyBytes should use long instead of int for offsets.

[Joe Skora (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joe Skora reassigned NIFI-2411:
---

Assignee: Joe Skora

> ModifyBytes should use long instead of int for offsets.
> ---
>
> Key: NIFI-2411
> URL: https://issues.apache.org/jira/browse/NIFI-2411
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Joe Skora
>Assignee: Joe Skora
>  Labels: easyfix
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> ModifyBytes.onTrigger() uses Java 32 bit {{int}}s for byte offsets limiting 
> it to 2 Gigabytes, switching to {{long}}'s will all it to handle up to 15 
> Exabytes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2078) State management for processors whose states are managed externally

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396881#comment-15396881
 ] 

ASF GitHub Bot commented on NIFI-2078:
--

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/563#discussion_r72562357
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/coordination/heartbeat/ClusterProtocolHeartbeatMonitor.java
 ---
@@ -91,7 +92,7 @@ public ClusterProtocolHeartbeatMonitor(final 
ClusterCoordinator clusterCoordinat
 this.clusterNodesPath = 
zkClientConfig.resolvePath("cluster/nodes");
 
 String hostname = 
properties.getProperty(NiFiProperties.CLUSTER_NODE_ADDRESS);
-if (hostname == null) {
+if (StringUtils.isEmpty(hostname)) {
--- End diff --

Thanks for pointing this out. I will remove these changes from this PR. 
Posted my comments on #688.


> State management for processors whose states are managed externally
> ---
>
> Key: NIFI-2078
> URL: https://issues.apache.org/jira/browse/NIFI-2078
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
> Fix For: 1.0.0
>
>
> Inherently by the nature of a given processor it may involve state managed by 
> itself (using nifi state management), or can be managed by some external 
> service it interacts with (kafka's offset), and theoretically some might have 
> both going on. With the new state management, we're giving users a way to 
> reset state managed by nifi for a given processor. But it doesnt apply to 
> those processors who have external state.
> we should consider offering a way to reset state that allows a processor to 
> call out to whatever external store it impacts



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #563: NIFI-2078, 2363, 2364: External state management. CL...

[ijokarumawak]

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/563#discussion_r72562357
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/coordination/heartbeat/ClusterProtocolHeartbeatMonitor.java
 ---
@@ -91,7 +92,7 @@ public ClusterProtocolHeartbeatMonitor(final 
ClusterCoordinator clusterCoordinat
 this.clusterNodesPath = 
zkClientConfig.resolvePath("cluster/nodes");
 
 String hostname = 
properties.getProperty(NiFiProperties.CLUSTER_NODE_ADDRESS);
-if (hostname == null) {
+if (StringUtils.isEmpty(hostname)) {
--- End diff --

Thanks for pointing this out. I will remove these changes from this PR. 
Posted my comments on #688.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #688: NIFI-2304: If node address is not set, default to lo...

[ijokarumawak]

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/688#discussion_r72561906
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/coordination/heartbeat/ClusterProtocolHeartbeatMonitor.java
 ---
@@ -91,12 +91,8 @@ public ClusterProtocolHeartbeatMonitor(final 
ClusterCoordinator clusterCoordinat
 this.clusterNodesPath = 
zkClientConfig.resolvePath("cluster/nodes");
 
 String hostname = 
properties.getProperty(NiFiProperties.CLUSTER_NODE_ADDRESS);
-if (hostname == null) {
-try {
-hostname = InetAddress.getLocalHost().getHostName();
-} catch (UnknownHostException e) {
-throw new RuntimeException("Unable to determine local 
hostname and the '" + NiFiProperties.CLUSTER_NODE_ADDRESS + "' property is not 
set");
-}
+if (hostname == null || hostname.trim().isEmpty()) {
--- End diff --

There's another similar null check against port `if (port == null) {`. I 
think it should be addressed, too.
We may be able to take advantage of our existing StringUtils.isBlank().


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2304) Cluster Coordinator reported incorrectly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396872#comment-15396872
 ] 

ASF GitHub Bot commented on NIFI-2304:
--

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/688#discussion_r72561906
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/coordination/heartbeat/ClusterProtocolHeartbeatMonitor.java
 ---
@@ -91,12 +91,8 @@ public ClusterProtocolHeartbeatMonitor(final 
ClusterCoordinator clusterCoordinat
 this.clusterNodesPath = 
zkClientConfig.resolvePath("cluster/nodes");
 
 String hostname = 
properties.getProperty(NiFiProperties.CLUSTER_NODE_ADDRESS);
-if (hostname == null) {
-try {
-hostname = InetAddress.getLocalHost().getHostName();
-} catch (UnknownHostException e) {
-throw new RuntimeException("Unable to determine local 
hostname and the '" + NiFiProperties.CLUSTER_NODE_ADDRESS + "' property is not 
set");
-}
+if (hostname == null || hostname.trim().isEmpty()) {
--- End diff --

There's another similar null check against port `if (port == null) {`. I 
think it should be addressed, too.
We may be able to take advantage of our existing StringUtils.isBlank().


> Cluster Coordinator reported incorrectly
> 
>
> Key: NIFI-2304
> URL: https://issues.apache.org/jira/browse/NIFI-2304
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Bryan Bende
>Assignee: Mark Payne
>Priority: Minor
> Fix For: 1.0.0
>
>
> I created a two node cluster locally, running embedded ZK on node1.. both 
> nodes start up, but when trying to access the UI on the second node, it says 
> no coordinator exists. 
> Looking in the logs the coordinator is being reported as ":8889" where 8889 
> is the node protocol port. I left the node host blank assuming it would 
> default to localhost, but looks like it isn't. When I filled in the node host 
> everything worked as expected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2304) Cluster Coordinator reported incorrectly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396867#comment-15396867
 ] 

ASF GitHub Bot commented on NIFI-2304:
--

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/688#discussion_r72561569
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/coordination/heartbeat/ClusterProtocolHeartbeatMonitor.java
 ---
@@ -91,12 +91,8 @@ public ClusterProtocolHeartbeatMonitor(final 
ClusterCoordinator clusterCoordinat
 this.clusterNodesPath = 
zkClientConfig.resolvePath("cluster/nodes");
 
 String hostname = 
properties.getProperty(NiFiProperties.CLUSTER_NODE_ADDRESS);
-if (hostname == null) {
-try {
-hostname = InetAddress.getLocalHost().getHostName();
-} catch (UnknownHostException e) {
-throw new RuntimeException("Unable to determine local 
hostname and the '" + NiFiProperties.CLUSTER_NODE_ADDRESS + "' property is not 
set");
-}
+if (hostname == null || hostname.trim().isEmpty()) {
+hostname = "localhost";
--- End diff --

Is there any specific reason to change from 
`InetAddress.getLocalHost().getHostName()` to `"localhost"`?

I thought if server's hostname is properly configured at OS level, 
administrator doesn't have to set hostname in nifi.properties, which makes 
configuration management easier since they can use the same properties file 
among nodes.


> Cluster Coordinator reported incorrectly
> 
>
> Key: NIFI-2304
> URL: https://issues.apache.org/jira/browse/NIFI-2304
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Bryan Bende
>Assignee: Mark Payne
>Priority: Minor
> Fix For: 1.0.0
>
>
> I created a two node cluster locally, running embedded ZK on node1.. both 
> nodes start up, but when trying to access the UI on the second node, it says 
> no coordinator exists. 
> Looking in the logs the coordinator is being reported as ":8889" where 8889 
> is the node protocol port. I left the node host blank assuming it would 
> default to localhost, but looks like it isn't. When I filled in the node host 
> everything worked as expected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #688: NIFI-2304: If node address is not set, default to lo...

[ijokarumawak]

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/688#discussion_r72561569
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/coordination/heartbeat/ClusterProtocolHeartbeatMonitor.java
 ---
@@ -91,12 +91,8 @@ public ClusterProtocolHeartbeatMonitor(final 
ClusterCoordinator clusterCoordinat
 this.clusterNodesPath = 
zkClientConfig.resolvePath("cluster/nodes");
 
 String hostname = 
properties.getProperty(NiFiProperties.CLUSTER_NODE_ADDRESS);
-if (hostname == null) {
-try {
-hostname = InetAddress.getLocalHost().getHostName();
-} catch (UnknownHostException e) {
-throw new RuntimeException("Unable to determine local 
hostname and the '" + NiFiProperties.CLUSTER_NODE_ADDRESS + "' property is not 
set");
-}
+if (hostname == null || hostname.trim().isEmpty()) {
+hostname = "localhost";
--- End diff --

Is there any specific reason to change from 
`InetAddress.getLocalHost().getHostName()` to `"localhost"`?

I thought if server's hostname is properly configured at OS level, 
administrator doesn't have to set hostname in nifi.properties, which makes 
configuration management easier since they can use the same properties file 
among nodes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-2415) GenerateTableFetch/QueryDatabaseTable cannot store state

[Haimo Liu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haimo Liu updated NIFI-2415:

Description: 
running NIFI in a clustered mode, 7/27 build

Both GenerateTableFetch and QueryDatabaseTable cannot store state in a 
clustered mode (checked ListFile, can store state successfully)

see attached screenshot, trace below:

2016-07-27 23:25:00,014 INFO [ProcessThread(sid:1 cport:-1):] 
o.a.z.server.PrepRequestProcessor Got user-level KeeperException when 
processing sessionid:0x2562f7875130009 type:create cxid:0xe4d zxid:0x73733 
txntype:-1 reqpath:n/a Error 
Path:/nifi/components/2f810bb3-0156-1000--16128f8b 
Error:KeeperErrorCode = InvalidACL for 
/nifi/components/2f810bb3-0156-1000--16128f8b
2016-07-27 23:25:00,017 WARN [Framework Task Thread Thread-1] 
o.a.n.c.s.m.StandardStateManagerProvider Component with ID {} was removed from 
NiFi instance but failed to clear clustered state for the component
java.io.IOException: Failed to set cluster-wide state in ZooKeeper for 
component with ID 2f810bb3-0156-1000--16128f8b
at 
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:343)
 ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at 
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:283)
 ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at 
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:228)
 ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at 
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.clear(ZooKeeperStateProvider.java:453)
 ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at 
org.apache.nifi.controller.state.StandardStateManager.clear(StandardStateManager.java:85)
 ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at 
org.apache.nifi.controller.state.manager.StandardStateManagerProvider.onComponentRemoved(StandardStateManagerProvider.java:276)
 ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at 
org.apache.nifi.groups.StandardProcessGroup$1.run(StandardProcessGroup.java:730)
 [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[na:1.8.0_77]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
[na:1.8.0_77]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
 [na:1.8.0_77]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
 [na:1.8.0_77]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[na:1.8.0_77]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_77]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
Caused by: org.apache.zookeeper.KeeperException$InvalidACLException: 
KeeperErrorCode = InvalidACL for 
/nifi/components/2f810bb3-0156-1000--16128f8b
at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:121) 
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) 
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783) 
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at 
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.createNode(ZooKeeperStateProvider.java:360)
 ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at 
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:321)
 ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
... 13 common frames omitted

  was:
running NIFI in a clustered mode, 7/27 build

Both GenerateTableFetch and QueryDatabaseTable cannot store state in a 
clustered mode (checked ListFile, can store state successfully)

see attached screenshot


> GenerateTableFetch/QueryDatabaseTable cannot store state
> 
>
> Key: NIFI-2415
> URL: https://issues.apache.org/jira/browse/NIFI-2415
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0
>Reporter: Haimo Liu
>Assignee: Matt Burgess
>Priority: Critical
> Fix For: 1.0.0
>
> Attachments: Screen Shot 2016-07-27 at 11.10.45 PM.png
>
>
> running NIFI in a clustered mode, 7/27 build
> Both GenerateTableFetch and QueryDatabaseTable cannot store state in a 
> clustered mode (checked 

[jira] [Updated] (NIFI-2415) GenerateTableFetch/QueryDatabaseTable cannot store state

[Haimo Liu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haimo Liu updated NIFI-2415:

Attachment: Screen Shot 2016-07-27 at 11.10.45 PM.png

> GenerateTableFetch/QueryDatabaseTable cannot store state
> 
>
> Key: NIFI-2415
> URL: https://issues.apache.org/jira/browse/NIFI-2415
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0
>Reporter: Haimo Liu
>Assignee: Matt Burgess
>Priority: Critical
> Fix For: 1.0.0
>
> Attachments: Screen Shot 2016-07-27 at 11.10.45 PM.png
>
>
> running NIFI in a clustered mode, 7/27 build
> Both GenerateTableFetch and QueryDatabaseTable cannot store state in a 
> clustered mode (checked ListFile, can store state successfully)
> see attached screenshot



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-2415) GenerateTableFetch/QueryDatabaseTable cannot store state

[Haimo Liu (JIRA)]

Haimo Liu created NIFI-2415:
---

 Summary: GenerateTableFetch/QueryDatabaseTable cannot store state
 Key: NIFI-2415
 URL: https://issues.apache.org/jira/browse/NIFI-2415
 Project: Apache NiFi
  Issue Type: Bug
  Components: Extensions
Affects Versions: 1.0.0
Reporter: Haimo Liu
Assignee: Matt Burgess
Priority: Critical
 Fix For: 1.0.0
 Attachments: Screen Shot 2016-07-27 at 11.10.45 PM.png

running NIFI in a clustered mode, 7/27 build

Both GenerateTableFetch and QueryDatabaseTable cannot store state in a 
clustered mode (checked ListFile, can store state successfully)

see attached screenshot



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #563: NIFI-2078, 2363, 2364: External state management. CL...

[ijokarumawak]

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/563#discussion_r72559943
  
--- Diff: 
nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-pubsub-processors/pom.xml ---
@@ -35,6 +35,10 @@
 nifi-utils
 
 
+org.apache.nifi
+nifi-expression-language
--- End diff --

BOOTSTRAP_SERVERS, TOPIC, CLIENT_ID are the properties shared among 
Consume/ProduceKafka. If we can separate and let Consume, ProduceKafka have 
those as its own property, ProduceKafka to support EL but not ConsumeKafka, 
then the dependency can be removed.

But existing flow configuration may have to be updated if one uses EL for 
those properties of ConsumeKafka. 

@olegz @JPercivall How do you think?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2078) State management for processors whose states are managed externally

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396843#comment-15396843
 ] 

ASF GitHub Bot commented on NIFI-2078:
--

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/563#discussion_r72559943
  
--- Diff: 
nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-pubsub-processors/pom.xml ---
@@ -35,6 +35,10 @@
 nifi-utils
 
 
+org.apache.nifi
+nifi-expression-language
--- End diff --

BOOTSTRAP_SERVERS, TOPIC, CLIENT_ID are the properties shared among 
Consume/ProduceKafka. If we can separate and let Consume, ProduceKafka have 
those as its own property, ProduceKafka to support EL but not ConsumeKafka, 
then the dependency can be removed.

But existing flow configuration may have to be updated if one uses EL for 
those properties of ConsumeKafka. 

@olegz @JPercivall How do you think?


> State management for processors whose states are managed externally
> ---
>
> Key: NIFI-2078
> URL: https://issues.apache.org/jira/browse/NIFI-2078
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
> Fix For: 1.0.0
>
>
> Inherently by the nature of a given processor it may involve state managed by 
> itself (using nifi state management), or can be managed by some external 
> service it interacts with (kafka's offset), and theoretically some might have 
> both going on. With the new state management, we're giving users a way to 
> reset state managed by nifi for a given processor. But it doesnt apply to 
> those processors who have external state.
> we should consider offering a way to reset state that allows a processor to 
> call out to whatever external store it impacts



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2078) State management for processors whose states are managed externally

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396834#comment-15396834
 ] 

ASF GitHub Bot commented on NIFI-2078:
--

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/563#discussion_r72559265
  
--- Diff: 
nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-pubsub-processors/pom.xml ---
@@ -35,6 +35,10 @@
 nifi-utils
 
 
+org.apache.nifi
+nifi-expression-language
--- End diff --

I added EL dependency to use StandardPropertyValue class at 
ConsumeKafka.onPropertyModified.

Since ConsumeKafka allows EL for properties that needed to get state from 
Kafka, EL has to be evaluated at onPropertyModified, too. But to do so, we need 
PropertyValue instance, which is not available at onPropertyModified because it 
only receives String representation of oldValue and newValue. An implementation 
class of PropertyValue is not included in nifi-api. So, I needed to add the EL 
dependency.

However, I felt it's a little bit strange to support EL for things like 
'topic' or 'bootstrap_servers', since ConsumeKafka doesn't take input flow 
files, and these properties can not be changed after ConsumerKafka connects to 
Kafka. It keep using the same kafkaResource instance. So, those property values 
are more static, rather than dynamically evaluated. It makes sense to support 
EL for those values for PublishKafka, though.


> State management for processors whose states are managed externally
> ---
>
> Key: NIFI-2078
> URL: https://issues.apache.org/jira/browse/NIFI-2078
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
> Fix For: 1.0.0
>
>
> Inherently by the nature of a given processor it may involve state managed by 
> itself (using nifi state management), or can be managed by some external 
> service it interacts with (kafka's offset), and theoretically some might have 
> both going on. With the new state management, we're giving users a way to 
> reset state managed by nifi for a given processor. But it doesnt apply to 
> those processors who have external state.
> we should consider offering a way to reset state that allows a processor to 
> call out to whatever external store it impacts



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #563: NIFI-2078, 2363, 2364: External state management. CL...

[ijokarumawak]

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/563#discussion_r72559265
  
--- Diff: 
nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-pubsub-processors/pom.xml ---
@@ -35,6 +35,10 @@
 nifi-utils
 
 
+org.apache.nifi
+nifi-expression-language
--- End diff --

I added EL dependency to use StandardPropertyValue class at 
ConsumeKafka.onPropertyModified.

Since ConsumeKafka allows EL for properties that needed to get state from 
Kafka, EL has to be evaluated at onPropertyModified, too. But to do so, we need 
PropertyValue instance, which is not available at onPropertyModified because it 
only receives String representation of oldValue and newValue. An implementation 
class of PropertyValue is not included in nifi-api. So, I needed to add the EL 
dependency.

However, I felt it's a little bit strange to support EL for things like 
'topic' or 'bootstrap_servers', since ConsumeKafka doesn't take input flow 
files, and these properties can not be changed after ConsumerKafka connects to 
Kafka. It keep using the same kafkaResource instance. So, those property values 
are more static, rather than dynamically evaluated. It makes sense to support 
EL for those values for PublishKafka, though.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2078) State management for processors whose states are managed externally

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396807#comment-15396807
 ] 

ASF GitHub Bot commented on NIFI-2078:
--

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/563#discussion_r72557366
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/coordination/http/endpoints/ClearComponentStateEndpointMerger.java
 ---
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.cluster.coordination.http.endpoints;
+
+import org.apache.nifi.cluster.manager.NodeResponse;
+import org.apache.nifi.cluster.protocol.NodeIdentifier;
+import org.apache.nifi.web.api.entity.ClearComponentStateResultEntity;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.net.URI;
+import java.util.Map;
+import java.util.Set;
+import java.util.regex.Pattern;
+
+public class ClearComponentStateEndpointMerger extends 
AbstractSingleEntityEndpoint {
+public static final Pattern PROCESSOR_STATE_URI_PATTERN = 
Pattern.compile("/nifi-api/processors/[a-f0-9\\-]{36}/state/clear-requests");
+public static final Pattern CONTROLLER_SERVICE_STATE_URI_PATTERN = 
Pattern.compile("/nifi-api/controller-services/[a-f0-9\\-]{36}/state/clear-requests");
+public static final Pattern REPORTING_TASK_STATE_URI_PATTERN = 
Pattern.compile("/nifi-api/reporting-tasks/[a-f0-9\\-]{36}/state/clear-requests");
--- End diff --

Yes, that's how it's done today. There may be some way to get URI patterns 
align with corresponding Resource class methods, though.


> State management for processors whose states are managed externally
> ---
>
> Key: NIFI-2078
> URL: https://issues.apache.org/jira/browse/NIFI-2078
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
> Fix For: 1.0.0
>
>
> Inherently by the nature of a given processor it may involve state managed by 
> itself (using nifi state management), or can be managed by some external 
> service it interacts with (kafka's offset), and theoretically some might have 
> both going on. With the new state management, we're giving users a way to 
> reset state managed by nifi for a given processor. But it doesnt apply to 
> those processors who have external state.
> we should consider offering a way to reset state that allows a processor to 
> call out to whatever external store it impacts



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #563: NIFI-2078, 2363, 2364: External state management. CL...

[ijokarumawak]

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/563#discussion_r72557366
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/coordination/http/endpoints/ClearComponentStateEndpointMerger.java
 ---
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.cluster.coordination.http.endpoints;
+
+import org.apache.nifi.cluster.manager.NodeResponse;
+import org.apache.nifi.cluster.protocol.NodeIdentifier;
+import org.apache.nifi.web.api.entity.ClearComponentStateResultEntity;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.net.URI;
+import java.util.Map;
+import java.util.Set;
+import java.util.regex.Pattern;
+
+public class ClearComponentStateEndpointMerger extends 
AbstractSingleEntityEndpoint {
+public static final Pattern PROCESSOR_STATE_URI_PATTERN = 
Pattern.compile("/nifi-api/processors/[a-f0-9\\-]{36}/state/clear-requests");
+public static final Pattern CONTROLLER_SERVICE_STATE_URI_PATTERN = 
Pattern.compile("/nifi-api/controller-services/[a-f0-9\\-]{36}/state/clear-requests");
+public static final Pattern REPORTING_TASK_STATE_URI_PATTERN = 
Pattern.compile("/nifi-api/reporting-tasks/[a-f0-9\\-]{36}/state/clear-requests");
--- End diff --

Yes, that's how it's done today. There may be some way to get URI patterns 
align with corresponding Resource class methods, though.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2193) Command Line Keystore and Truststore utility

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396758#comment-15396758
 ] 

ASF GitHub Bot commented on NIFI-2193:
--

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/695
  
@alopresto I agree with your concerns about the different entry points, 
I'll try to unify them and get down to a single shell script with the main 
method and argument parsing determining the action


> Command Line Keystore and Truststore utility
> 
>
> Key: NIFI-2193
> URL: https://issues.apache.org/jira/browse/NIFI-2193
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Bryan Rosander
>Assignee: Bryan Rosander
>
> In order to facilitate secure setup of NiFi, it would be useful to have a 
> command line utility capable of generating the required keystores, 
> truststore, and relevant configuration files.
> It should be able to generate keystores for each NiFi node, a truststore that 
> they all use, and relevant passwords and configuration files for using the 
> keystores and truststore.
> Additionally, in order to support distributed deployment, a web based 
> certificate authority with corresponding client will allow for each NiFi 
> instance to generate its own keypair and then request signing by the CA.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #695: NIFI-2193 - Command line SSL config utility as well ...

[brosander]

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi/pull/695#discussion_r72551830
  
--- Diff: 
nifi-toolkit/nifi-toolkit-tls/src/main/java/org/apache/nifi/toolkit/tls/util/TlsHelper.java
 ---
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.toolkit.tls.util;
+
+import org.apache.nifi.security.util.CertificateUtils;
+import org.apache.nifi.toolkit.tls.commandLine.TlsToolkitCommandLine;
+import org.apache.nifi.toolkit.tls.configuration.TlsHelperConfig;
+import org.bouncycastle.cert.X509CertificateHolder;
+import org.bouncycastle.cert.crmf.CRMFException;
+import org.bouncycastle.cert.jcajce.JcaX509CertificateConverter;
+import org.bouncycastle.cert.jcajce.JcaX509ExtensionUtils;
+import org.bouncycastle.eac.EACException;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+import org.bouncycastle.openssl.PEMParser;
+import org.bouncycastle.openssl.jcajce.JcaMiscPEMGenerator;
+import org.bouncycastle.operator.OperatorCreationException;
+import org.bouncycastle.operator.jcajce.JcaContentSignerBuilder;
+import org.bouncycastle.pkcs.PKCS10CertificationRequest;
+import org.bouncycastle.pkcs.jcajce.JcaPKCS10CertificationRequest;
+import org.bouncycastle.pkcs.jcajce.JcaPKCS10CertificationRequestBuilder;
+import org.bouncycastle.util.io.pem.PemWriter;
+
+import javax.crypto.Mac;
+import javax.crypto.spec.SecretKeySpec;
+import javax.security.auth.x500.X500Principal;
+import java.io.IOException;
+import java.io.StringReader;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.security.GeneralSecurityException;
+import java.security.InvalidKeyException;
+import java.security.KeyPair;
+import java.security.KeyPairGenerator;
+import java.security.KeyStore;
+import java.security.KeyStoreException;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
+import java.security.NoSuchProviderException;
+import java.security.PublicKey;
+import java.security.Security;
+import java.security.SignatureException;
+import java.security.cert.Certificate;
+import java.security.cert.CertificateException;
+import java.security.cert.X509Certificate;
+import java.security.spec.InvalidKeySpecException;
+
+public class TlsHelper {
+public static final String PROVIDER = 
BouncyCastleProvider.PROVIDER_NAME;
+public static final String PKCS12 = "PKCS12";
+private final KeyPairGenerator keyPairGenerator;
+private final int days;
+private final String signingAlgorithm;
+
+public TlsHelper(TlsHelperConfig tlsHelperConfig) throws 
NoSuchAlgorithmException {
+this(tlsHelperConfig.getDays(), tlsHelperConfig.getKeySize(), 
tlsHelperConfig.getKeyPairAlgorithm(), tlsHelperConfig.getSigningAlgorithm());
+}
+
+public TlsHelper(TlsToolkitCommandLine tlsToolkitCommandLine) throws 
NoSuchAlgorithmException {
+this(tlsToolkitCommandLine.getTlsHelperConfig());
+}
+
+public TlsHelper(int days, int keySize, String keyPairAlgorithm, 
String signingAlgorithm) throws NoSuchAlgorithmException {
+this(createKeyPairGenerator(keyPairAlgorithm, keySize), days, 
signingAlgorithm);
+}
+
+protected TlsHelper(KeyPairGenerator keyPairGenerator, int days, 
String signingAlgorithm) {
+this.keyPairGenerator = keyPairGenerator;
+this.days = days;
+this.signingAlgorithm = signingAlgorithm;
+}
+
+public static void addBouncyCastleProvider() {
+Security.addProvider(new BouncyCastleProvider());
+}
+
+private static KeyPairGenerator createKeyPairGenerator(String 
algorithm, int keySize) throws NoSuchAlgorithmException {
+KeyPairGenerator instance = 
KeyPairGenerator.getInstance(algorithm);
+instance.initialize(keySize);
+return instance;
+}
  

[jira] [Commented] (NIFI-2193) Command Line Keystore and Truststore utility

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396754#comment-15396754
 ] 

ASF GitHub Bot commented on NIFI-2193:
--

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi/pull/695#discussion_r72551782
  
--- Diff: 
nifi-toolkit/nifi-toolkit-tls/src/main/java/org/apache/nifi/toolkit/tls/util/TlsHelper.java
 ---
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.toolkit.tls.util;
+
+import org.apache.nifi.security.util.CertificateUtils;
+import org.apache.nifi.toolkit.tls.commandLine.TlsToolkitCommandLine;
+import org.apache.nifi.toolkit.tls.configuration.TlsHelperConfig;
+import org.bouncycastle.cert.X509CertificateHolder;
+import org.bouncycastle.cert.crmf.CRMFException;
+import org.bouncycastle.cert.jcajce.JcaX509CertificateConverter;
+import org.bouncycastle.cert.jcajce.JcaX509ExtensionUtils;
+import org.bouncycastle.eac.EACException;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+import org.bouncycastle.openssl.PEMParser;
+import org.bouncycastle.openssl.jcajce.JcaMiscPEMGenerator;
+import org.bouncycastle.operator.OperatorCreationException;
+import org.bouncycastle.operator.jcajce.JcaContentSignerBuilder;
+import org.bouncycastle.pkcs.PKCS10CertificationRequest;
+import org.bouncycastle.pkcs.jcajce.JcaPKCS10CertificationRequest;
+import org.bouncycastle.pkcs.jcajce.JcaPKCS10CertificationRequestBuilder;
+import org.bouncycastle.util.io.pem.PemWriter;
+
+import javax.crypto.Mac;
+import javax.crypto.spec.SecretKeySpec;
+import javax.security.auth.x500.X500Principal;
+import java.io.IOException;
+import java.io.StringReader;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.security.GeneralSecurityException;
+import java.security.InvalidKeyException;
+import java.security.KeyPair;
+import java.security.KeyPairGenerator;
+import java.security.KeyStore;
+import java.security.KeyStoreException;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
+import java.security.NoSuchProviderException;
+import java.security.PublicKey;
+import java.security.Security;
+import java.security.SignatureException;
+import java.security.cert.Certificate;
+import java.security.cert.CertificateException;
+import java.security.cert.X509Certificate;
+import java.security.spec.InvalidKeySpecException;
+
+public class TlsHelper {
+public static final String PROVIDER = 
BouncyCastleProvider.PROVIDER_NAME;
+public static final String PKCS12 = "PKCS12";
+private final KeyPairGenerator keyPairGenerator;
+private final int days;
+private final String signingAlgorithm;
+
+public TlsHelper(TlsHelperConfig tlsHelperConfig) throws 
NoSuchAlgorithmException {
+this(tlsHelperConfig.getDays(), tlsHelperConfig.getKeySize(), 
tlsHelperConfig.getKeyPairAlgorithm(), tlsHelperConfig.getSigningAlgorithm());
+}
+
+public TlsHelper(TlsToolkitCommandLine tlsToolkitCommandLine) throws 
NoSuchAlgorithmException {
+this(tlsToolkitCommandLine.getTlsHelperConfig());
+}
+
+public TlsHelper(int days, int keySize, String keyPairAlgorithm, 
String signingAlgorithm) throws NoSuchAlgorithmException {
+this(createKeyPairGenerator(keyPairAlgorithm, keySize), days, 
signingAlgorithm);
+}
+
+protected TlsHelper(KeyPairGenerator keyPairGenerator, int days, 
String signingAlgorithm) {
+this.keyPairGenerator = keyPairGenerator;
+this.days = days;
+this.signingAlgorithm = signingAlgorithm;
+}
+
+public static void addBouncyCastleProvider() {
+Security.addProvider(new BouncyCastleProvider());
+}
+
+private static KeyPairGenerator 

[GitHub] nifi pull request #695: NIFI-2193 - Command line SSL config utility as well ...

[brosander]

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi/pull/695#discussion_r72551782
  
--- Diff: 
nifi-toolkit/nifi-toolkit-tls/src/main/java/org/apache/nifi/toolkit/tls/util/TlsHelper.java
 ---
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.toolkit.tls.util;
+
+import org.apache.nifi.security.util.CertificateUtils;
+import org.apache.nifi.toolkit.tls.commandLine.TlsToolkitCommandLine;
+import org.apache.nifi.toolkit.tls.configuration.TlsHelperConfig;
+import org.bouncycastle.cert.X509CertificateHolder;
+import org.bouncycastle.cert.crmf.CRMFException;
+import org.bouncycastle.cert.jcajce.JcaX509CertificateConverter;
+import org.bouncycastle.cert.jcajce.JcaX509ExtensionUtils;
+import org.bouncycastle.eac.EACException;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+import org.bouncycastle.openssl.PEMParser;
+import org.bouncycastle.openssl.jcajce.JcaMiscPEMGenerator;
+import org.bouncycastle.operator.OperatorCreationException;
+import org.bouncycastle.operator.jcajce.JcaContentSignerBuilder;
+import org.bouncycastle.pkcs.PKCS10CertificationRequest;
+import org.bouncycastle.pkcs.jcajce.JcaPKCS10CertificationRequest;
+import org.bouncycastle.pkcs.jcajce.JcaPKCS10CertificationRequestBuilder;
+import org.bouncycastle.util.io.pem.PemWriter;
+
+import javax.crypto.Mac;
+import javax.crypto.spec.SecretKeySpec;
+import javax.security.auth.x500.X500Principal;
+import java.io.IOException;
+import java.io.StringReader;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.security.GeneralSecurityException;
+import java.security.InvalidKeyException;
+import java.security.KeyPair;
+import java.security.KeyPairGenerator;
+import java.security.KeyStore;
+import java.security.KeyStoreException;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
+import java.security.NoSuchProviderException;
+import java.security.PublicKey;
+import java.security.Security;
+import java.security.SignatureException;
+import java.security.cert.Certificate;
+import java.security.cert.CertificateException;
+import java.security.cert.X509Certificate;
+import java.security.spec.InvalidKeySpecException;
+
+public class TlsHelper {
+public static final String PROVIDER = 
BouncyCastleProvider.PROVIDER_NAME;
+public static final String PKCS12 = "PKCS12";
+private final KeyPairGenerator keyPairGenerator;
+private final int days;
+private final String signingAlgorithm;
+
+public TlsHelper(TlsHelperConfig tlsHelperConfig) throws 
NoSuchAlgorithmException {
+this(tlsHelperConfig.getDays(), tlsHelperConfig.getKeySize(), 
tlsHelperConfig.getKeyPairAlgorithm(), tlsHelperConfig.getSigningAlgorithm());
+}
+
+public TlsHelper(TlsToolkitCommandLine tlsToolkitCommandLine) throws 
NoSuchAlgorithmException {
+this(tlsToolkitCommandLine.getTlsHelperConfig());
+}
+
+public TlsHelper(int days, int keySize, String keyPairAlgorithm, 
String signingAlgorithm) throws NoSuchAlgorithmException {
+this(createKeyPairGenerator(keyPairAlgorithm, keySize), days, 
signingAlgorithm);
+}
+
+protected TlsHelper(KeyPairGenerator keyPairGenerator, int days, 
String signingAlgorithm) {
+this.keyPairGenerator = keyPairGenerator;
+this.days = days;
+this.signingAlgorithm = signingAlgorithm;
+}
+
+public static void addBouncyCastleProvider() {
+Security.addProvider(new BouncyCastleProvider());
+}
+
+private static KeyPairGenerator createKeyPairGenerator(String 
algorithm, int keySize) throws NoSuchAlgorithmException {
+KeyPairGenerator instance = 
KeyPairGenerator.getInstance(algorithm);
+instance.initialize(keySize);
+return instance;
+}
  

[GitHub] nifi pull request #695: NIFI-2193 - Command line SSL config utility as well ...

[alopresto]

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/695#discussion_r72549313
  
--- Diff: 
nifi-toolkit/nifi-toolkit-tls/src/main/java/org/apache/nifi/toolkit/tls/util/TlsHelper.java
 ---
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.toolkit.tls.util;
+
+import org.apache.nifi.security.util.CertificateUtils;
+import org.apache.nifi.toolkit.tls.commandLine.TlsToolkitCommandLine;
+import org.apache.nifi.toolkit.tls.configuration.TlsHelperConfig;
+import org.bouncycastle.cert.X509CertificateHolder;
+import org.bouncycastle.cert.crmf.CRMFException;
+import org.bouncycastle.cert.jcajce.JcaX509CertificateConverter;
+import org.bouncycastle.cert.jcajce.JcaX509ExtensionUtils;
+import org.bouncycastle.eac.EACException;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+import org.bouncycastle.openssl.PEMParser;
+import org.bouncycastle.openssl.jcajce.JcaMiscPEMGenerator;
+import org.bouncycastle.operator.OperatorCreationException;
+import org.bouncycastle.operator.jcajce.JcaContentSignerBuilder;
+import org.bouncycastle.pkcs.PKCS10CertificationRequest;
+import org.bouncycastle.pkcs.jcajce.JcaPKCS10CertificationRequest;
+import org.bouncycastle.pkcs.jcajce.JcaPKCS10CertificationRequestBuilder;
+import org.bouncycastle.util.io.pem.PemWriter;
+
+import javax.crypto.Mac;
+import javax.crypto.spec.SecretKeySpec;
+import javax.security.auth.x500.X500Principal;
+import java.io.IOException;
+import java.io.StringReader;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.security.GeneralSecurityException;
+import java.security.InvalidKeyException;
+import java.security.KeyPair;
+import java.security.KeyPairGenerator;
+import java.security.KeyStore;
+import java.security.KeyStoreException;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
+import java.security.NoSuchProviderException;
+import java.security.PublicKey;
+import java.security.Security;
+import java.security.SignatureException;
+import java.security.cert.Certificate;
+import java.security.cert.CertificateException;
+import java.security.cert.X509Certificate;
+import java.security.spec.InvalidKeySpecException;
+
+public class TlsHelper {
+public static final String PROVIDER = 
BouncyCastleProvider.PROVIDER_NAME;
+public static final String PKCS12 = "PKCS12";
+private final KeyPairGenerator keyPairGenerator;
+private final int days;
+private final String signingAlgorithm;
+
+public TlsHelper(TlsHelperConfig tlsHelperConfig) throws 
NoSuchAlgorithmException {
+this(tlsHelperConfig.getDays(), tlsHelperConfig.getKeySize(), 
tlsHelperConfig.getKeyPairAlgorithm(), tlsHelperConfig.getSigningAlgorithm());
+}
+
+public TlsHelper(TlsToolkitCommandLine tlsToolkitCommandLine) throws 
NoSuchAlgorithmException {
+this(tlsToolkitCommandLine.getTlsHelperConfig());
+}
+
+public TlsHelper(int days, int keySize, String keyPairAlgorithm, 
String signingAlgorithm) throws NoSuchAlgorithmException {
+this(createKeyPairGenerator(keyPairAlgorithm, keySize), days, 
signingAlgorithm);
+}
+
+protected TlsHelper(KeyPairGenerator keyPairGenerator, int days, 
String signingAlgorithm) {
+this.keyPairGenerator = keyPairGenerator;
+this.days = days;
+this.signingAlgorithm = signingAlgorithm;
+}
+
+public static void addBouncyCastleProvider() {
+Security.addProvider(new BouncyCastleProvider());
+}
+
+private static KeyPairGenerator createKeyPairGenerator(String 
algorithm, int keySize) throws NoSuchAlgorithmException {
+KeyPairGenerator instance = 
KeyPairGenerator.getInstance(algorithm);
+instance.initialize(keySize);
+return instance;
+}
  

[jira] [Commented] (NIFI-2193) Command Line Keystore and Truststore utility

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396726#comment-15396726
 ] 

ASF GitHub Bot commented on NIFI-2193:
--

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/695#discussion_r72548970
  
--- Diff: 
nifi-toolkit/nifi-toolkit-tls/src/main/java/org/apache/nifi/toolkit/tls/util/TlsHelper.java
 ---
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.toolkit.tls.util;
+
+import org.apache.nifi.security.util.CertificateUtils;
+import org.apache.nifi.toolkit.tls.commandLine.TlsToolkitCommandLine;
+import org.apache.nifi.toolkit.tls.configuration.TlsHelperConfig;
+import org.bouncycastle.cert.X509CertificateHolder;
+import org.bouncycastle.cert.crmf.CRMFException;
+import org.bouncycastle.cert.jcajce.JcaX509CertificateConverter;
+import org.bouncycastle.cert.jcajce.JcaX509ExtensionUtils;
+import org.bouncycastle.eac.EACException;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+import org.bouncycastle.openssl.PEMParser;
+import org.bouncycastle.openssl.jcajce.JcaMiscPEMGenerator;
+import org.bouncycastle.operator.OperatorCreationException;
+import org.bouncycastle.operator.jcajce.JcaContentSignerBuilder;
+import org.bouncycastle.pkcs.PKCS10CertificationRequest;
+import org.bouncycastle.pkcs.jcajce.JcaPKCS10CertificationRequest;
+import org.bouncycastle.pkcs.jcajce.JcaPKCS10CertificationRequestBuilder;
+import org.bouncycastle.util.io.pem.PemWriter;
+
+import javax.crypto.Mac;
+import javax.crypto.spec.SecretKeySpec;
+import javax.security.auth.x500.X500Principal;
+import java.io.IOException;
+import java.io.StringReader;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.security.GeneralSecurityException;
+import java.security.InvalidKeyException;
+import java.security.KeyPair;
+import java.security.KeyPairGenerator;
+import java.security.KeyStore;
+import java.security.KeyStoreException;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
+import java.security.NoSuchProviderException;
+import java.security.PublicKey;
+import java.security.Security;
+import java.security.SignatureException;
+import java.security.cert.Certificate;
+import java.security.cert.CertificateException;
+import java.security.cert.X509Certificate;
+import java.security.spec.InvalidKeySpecException;
+
+public class TlsHelper {
+public static final String PROVIDER = 
BouncyCastleProvider.PROVIDER_NAME;
+public static final String PKCS12 = "PKCS12";
+private final KeyPairGenerator keyPairGenerator;
+private final int days;
+private final String signingAlgorithm;
+
+public TlsHelper(TlsHelperConfig tlsHelperConfig) throws 
NoSuchAlgorithmException {
+this(tlsHelperConfig.getDays(), tlsHelperConfig.getKeySize(), 
tlsHelperConfig.getKeyPairAlgorithm(), tlsHelperConfig.getSigningAlgorithm());
+}
+
+public TlsHelper(TlsToolkitCommandLine tlsToolkitCommandLine) throws 
NoSuchAlgorithmException {
+this(tlsToolkitCommandLine.getTlsHelperConfig());
+}
+
+public TlsHelper(int days, int keySize, String keyPairAlgorithm, 
String signingAlgorithm) throws NoSuchAlgorithmException {
+this(createKeyPairGenerator(keyPairAlgorithm, keySize), days, 
signingAlgorithm);
+}
+
+protected TlsHelper(KeyPairGenerator keyPairGenerator, int days, 
String signingAlgorithm) {
+this.keyPairGenerator = keyPairGenerator;
+this.days = days;
+this.signingAlgorithm = signingAlgorithm;
+}
+
+public static void addBouncyCastleProvider() {
+Security.addProvider(new BouncyCastleProvider());
+}
+
+private static KeyPairGenerator 

[GitHub] nifi pull request #695: NIFI-2193 - Command line SSL config utility as well ...

[alopresto]

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/695#discussion_r72548970
  
--- Diff: 
nifi-toolkit/nifi-toolkit-tls/src/main/java/org/apache/nifi/toolkit/tls/util/TlsHelper.java
 ---
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.toolkit.tls.util;
+
+import org.apache.nifi.security.util.CertificateUtils;
+import org.apache.nifi.toolkit.tls.commandLine.TlsToolkitCommandLine;
+import org.apache.nifi.toolkit.tls.configuration.TlsHelperConfig;
+import org.bouncycastle.cert.X509CertificateHolder;
+import org.bouncycastle.cert.crmf.CRMFException;
+import org.bouncycastle.cert.jcajce.JcaX509CertificateConverter;
+import org.bouncycastle.cert.jcajce.JcaX509ExtensionUtils;
+import org.bouncycastle.eac.EACException;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+import org.bouncycastle.openssl.PEMParser;
+import org.bouncycastle.openssl.jcajce.JcaMiscPEMGenerator;
+import org.bouncycastle.operator.OperatorCreationException;
+import org.bouncycastle.operator.jcajce.JcaContentSignerBuilder;
+import org.bouncycastle.pkcs.PKCS10CertificationRequest;
+import org.bouncycastle.pkcs.jcajce.JcaPKCS10CertificationRequest;
+import org.bouncycastle.pkcs.jcajce.JcaPKCS10CertificationRequestBuilder;
+import org.bouncycastle.util.io.pem.PemWriter;
+
+import javax.crypto.Mac;
+import javax.crypto.spec.SecretKeySpec;
+import javax.security.auth.x500.X500Principal;
+import java.io.IOException;
+import java.io.StringReader;
+import java.io.StringWriter;
+import java.nio.charset.StandardCharsets;
+import java.security.GeneralSecurityException;
+import java.security.InvalidKeyException;
+import java.security.KeyPair;
+import java.security.KeyPairGenerator;
+import java.security.KeyStore;
+import java.security.KeyStoreException;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
+import java.security.NoSuchProviderException;
+import java.security.PublicKey;
+import java.security.Security;
+import java.security.SignatureException;
+import java.security.cert.Certificate;
+import java.security.cert.CertificateException;
+import java.security.cert.X509Certificate;
+import java.security.spec.InvalidKeySpecException;
+
+public class TlsHelper {
+public static final String PROVIDER = 
BouncyCastleProvider.PROVIDER_NAME;
+public static final String PKCS12 = "PKCS12";
+private final KeyPairGenerator keyPairGenerator;
+private final int days;
+private final String signingAlgorithm;
+
+public TlsHelper(TlsHelperConfig tlsHelperConfig) throws 
NoSuchAlgorithmException {
+this(tlsHelperConfig.getDays(), tlsHelperConfig.getKeySize(), 
tlsHelperConfig.getKeyPairAlgorithm(), tlsHelperConfig.getSigningAlgorithm());
+}
+
+public TlsHelper(TlsToolkitCommandLine tlsToolkitCommandLine) throws 
NoSuchAlgorithmException {
+this(tlsToolkitCommandLine.getTlsHelperConfig());
+}
+
+public TlsHelper(int days, int keySize, String keyPairAlgorithm, 
String signingAlgorithm) throws NoSuchAlgorithmException {
+this(createKeyPairGenerator(keyPairAlgorithm, keySize), days, 
signingAlgorithm);
+}
+
+protected TlsHelper(KeyPairGenerator keyPairGenerator, int days, 
String signingAlgorithm) {
+this.keyPairGenerator = keyPairGenerator;
+this.days = days;
+this.signingAlgorithm = signingAlgorithm;
+}
+
+public static void addBouncyCastleProvider() {
+Security.addProvider(new BouncyCastleProvider());
+}
+
+private static KeyPairGenerator createKeyPairGenerator(String 
algorithm, int keySize) throws NoSuchAlgorithmException {
+KeyPairGenerator instance = 
KeyPairGenerator.getInstance(algorithm);
+instance.initialize(keySize);
+return instance;
+}
  

[GitHub] nifi issue #695: NIFI-2193 - Command line SSL config utility as well as cert...

[alopresto]

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/695
  
I don't really understand the structure of the tool here -- it looks like 
`TlsToolkitMain` is invoked from the scripts, and it has an internal instance 
of `TlsToolkitCommandLine` to parse the command line arguments, and an instance 
of `TlsHelper`. I would expect `TlsToolkitMain` to be a standalone class which 
encapsulated the logic (and ideally, to be named something like 
`CertificateGeneratorTool` as it is focusing solely on the key and certificate 
generation), and `TlsToolkitCommandLine` to be a wrapper class which handles 
and parses command line input to then invoke the main class logic. This way, 
the logic could be invoked programmatically from other classes with direct 
parameter provision. It seems to me that the current structure tightly couples 
all certificate generation & keystore population the with command line entry 
point. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #476: NIFI-1942 Processor to validate CSV against user-sup...

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/476#discussion_r72542364
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateCsv.java
 ---
@@ -0,0 +1,613 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.io.OutputStream;
+import java.io.Reader;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.supercsv.cellprocessor.Optional;
+import org.supercsv.cellprocessor.ParseBigDecimal;
+import org.supercsv.cellprocessor.ParseBool;
+import org.supercsv.cellprocessor.ParseChar;
+import org.supercsv.cellprocessor.ParseDate;
+import org.supercsv.cellprocessor.ParseDouble;
+import org.supercsv.cellprocessor.ParseInt;
+import org.supercsv.cellprocessor.ParseLong;
+import org.supercsv.cellprocessor.constraint.DMinMax;
+import org.supercsv.cellprocessor.constraint.Equals;
+import org.supercsv.cellprocessor.constraint.ForbidSubStr;
+import org.supercsv.cellprocessor.constraint.IsIncludedIn;
+import org.supercsv.cellprocessor.constraint.LMinMax;
+import org.supercsv.cellprocessor.constraint.NotNull;
+import org.supercsv.cellprocessor.constraint.RequireHashCode;
+import org.supercsv.cellprocessor.constraint.RequireSubStr;
+import org.supercsv.cellprocessor.constraint.StrMinMax;
+import org.supercsv.cellprocessor.constraint.StrNotNullOrEmpty;
+import org.supercsv.cellprocessor.constraint.StrRegEx;
+import org.supercsv.cellprocessor.constraint.Strlen;
+import org.supercsv.cellprocessor.constraint.Unique;
+import org.supercsv.cellprocessor.constraint.UniqueHashCode;
+import org.supercsv.cellprocessor.ift.CellProcessor;
+import org.supercsv.exception.SuperCsvCellProcessorException;
+import org.supercsv.io.CsvListReader;
+import org.supercsv.prefs.CsvPreference;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"csv", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified CSV schema. " +
+"Take a look at the additional documentation of this processor for 
some schema examples.")
+public class ValidateCsv extends AbstractProcessor {
+
+private final static List 

[GitHub] nifi pull request #476: NIFI-1942 Processor to validate CSV against user-sup...

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/476#discussion_r72541263
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateCsv.java
 ---
@@ -0,0 +1,613 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.io.OutputStream;
+import java.io.Reader;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.supercsv.cellprocessor.Optional;
+import org.supercsv.cellprocessor.ParseBigDecimal;
+import org.supercsv.cellprocessor.ParseBool;
+import org.supercsv.cellprocessor.ParseChar;
+import org.supercsv.cellprocessor.ParseDate;
+import org.supercsv.cellprocessor.ParseDouble;
+import org.supercsv.cellprocessor.ParseInt;
+import org.supercsv.cellprocessor.ParseLong;
+import org.supercsv.cellprocessor.constraint.DMinMax;
+import org.supercsv.cellprocessor.constraint.Equals;
+import org.supercsv.cellprocessor.constraint.ForbidSubStr;
+import org.supercsv.cellprocessor.constraint.IsIncludedIn;
+import org.supercsv.cellprocessor.constraint.LMinMax;
+import org.supercsv.cellprocessor.constraint.NotNull;
+import org.supercsv.cellprocessor.constraint.RequireHashCode;
+import org.supercsv.cellprocessor.constraint.RequireSubStr;
+import org.supercsv.cellprocessor.constraint.StrMinMax;
+import org.supercsv.cellprocessor.constraint.StrNotNullOrEmpty;
+import org.supercsv.cellprocessor.constraint.StrRegEx;
+import org.supercsv.cellprocessor.constraint.Strlen;
+import org.supercsv.cellprocessor.constraint.Unique;
+import org.supercsv.cellprocessor.constraint.UniqueHashCode;
+import org.supercsv.cellprocessor.ift.CellProcessor;
+import org.supercsv.exception.SuperCsvCellProcessorException;
+import org.supercsv.io.CsvListReader;
+import org.supercsv.prefs.CsvPreference;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"csv", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified CSV schema. " +
+"Take a look at the additional documentation of this processor for 
some schema examples.")
+public class ValidateCsv extends AbstractProcessor {
+
+private final static List 

[GitHub] nifi pull request #476: NIFI-1942 Processor to validate CSV against user-sup...

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/476#discussion_r72539247
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateCsv.java
 ---
@@ -0,0 +1,613 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.io.OutputStream;
+import java.io.Reader;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.supercsv.cellprocessor.Optional;
+import org.supercsv.cellprocessor.ParseBigDecimal;
+import org.supercsv.cellprocessor.ParseBool;
+import org.supercsv.cellprocessor.ParseChar;
+import org.supercsv.cellprocessor.ParseDate;
+import org.supercsv.cellprocessor.ParseDouble;
+import org.supercsv.cellprocessor.ParseInt;
+import org.supercsv.cellprocessor.ParseLong;
+import org.supercsv.cellprocessor.constraint.DMinMax;
+import org.supercsv.cellprocessor.constraint.Equals;
+import org.supercsv.cellprocessor.constraint.ForbidSubStr;
+import org.supercsv.cellprocessor.constraint.IsIncludedIn;
+import org.supercsv.cellprocessor.constraint.LMinMax;
+import org.supercsv.cellprocessor.constraint.NotNull;
+import org.supercsv.cellprocessor.constraint.RequireHashCode;
+import org.supercsv.cellprocessor.constraint.RequireSubStr;
+import org.supercsv.cellprocessor.constraint.StrMinMax;
+import org.supercsv.cellprocessor.constraint.StrNotNullOrEmpty;
+import org.supercsv.cellprocessor.constraint.StrRegEx;
+import org.supercsv.cellprocessor.constraint.Strlen;
+import org.supercsv.cellprocessor.constraint.Unique;
+import org.supercsv.cellprocessor.constraint.UniqueHashCode;
+import org.supercsv.cellprocessor.ift.CellProcessor;
+import org.supercsv.exception.SuperCsvCellProcessorException;
+import org.supercsv.io.CsvListReader;
+import org.supercsv.prefs.CsvPreference;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"csv", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified CSV schema. " +
+"Take a look at the additional documentation of this processor for 
some schema examples.")
+public class ValidateCsv extends AbstractProcessor {
+
+private final static List 

[jira] [Commented] (NIFI-2193) Command Line Keystore and Truststore utility

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396558#comment-15396558
 ] 

ASF GitHub Bot commented on NIFI-2193:
--

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/695
  
I am reviewing, focusing on the cryptographic code in here as well as the 
token signing for communication between the nodes and CA over an untrusted 
channel. 

@brosander can you please provide some documentation/admin guide? The only 
thing I have seen so far is the usage in the NiFiToolkitTLS class. 


> Command Line Keystore and Truststore utility
> 
>
> Key: NIFI-2193
> URL: https://issues.apache.org/jira/browse/NIFI-2193
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Bryan Rosander
>Assignee: Bryan Rosander
>
> In order to facilitate secure setup of NiFi, it would be useful to have a 
> command line utility capable of generating the required keystores, 
> truststore, and relevant configuration files.
> It should be able to generate keystores for each NiFi node, a truststore that 
> they all use, and relevant passwords and configuration files for using the 
> keystores and truststore.
> Additionally, in order to support distributed deployment, a web based 
> certificate authority with corresponding client will allow for each NiFi 
> instance to generate its own keypair and then request signing by the CA.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #695: NIFI-2193 - Command line SSL config utility as well as cert...

[alopresto]

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/695
  
I am reviewing, focusing on the cryptographic code in here as well as the 
token signing for communication between the nodes and CA over an untrusted 
channel. 

@brosander can you please provide some documentation/admin guide? The only 
thing I have seen so far is the usage in the NiFiToolkitTLS class. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #476: NIFI-1942 Processor to validate CSV against user-sup...

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/476#discussion_r72538224
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateCsv.java
 ---
@@ -0,0 +1,613 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.io.OutputStream;
+import java.io.Reader;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.supercsv.cellprocessor.Optional;
+import org.supercsv.cellprocessor.ParseBigDecimal;
+import org.supercsv.cellprocessor.ParseBool;
+import org.supercsv.cellprocessor.ParseChar;
+import org.supercsv.cellprocessor.ParseDate;
+import org.supercsv.cellprocessor.ParseDouble;
+import org.supercsv.cellprocessor.ParseInt;
+import org.supercsv.cellprocessor.ParseLong;
+import org.supercsv.cellprocessor.constraint.DMinMax;
+import org.supercsv.cellprocessor.constraint.Equals;
+import org.supercsv.cellprocessor.constraint.ForbidSubStr;
+import org.supercsv.cellprocessor.constraint.IsIncludedIn;
+import org.supercsv.cellprocessor.constraint.LMinMax;
+import org.supercsv.cellprocessor.constraint.NotNull;
+import org.supercsv.cellprocessor.constraint.RequireHashCode;
+import org.supercsv.cellprocessor.constraint.RequireSubStr;
+import org.supercsv.cellprocessor.constraint.StrMinMax;
+import org.supercsv.cellprocessor.constraint.StrNotNullOrEmpty;
+import org.supercsv.cellprocessor.constraint.StrRegEx;
+import org.supercsv.cellprocessor.constraint.Strlen;
+import org.supercsv.cellprocessor.constraint.Unique;
+import org.supercsv.cellprocessor.constraint.UniqueHashCode;
+import org.supercsv.cellprocessor.ift.CellProcessor;
+import org.supercsv.exception.SuperCsvCellProcessorException;
+import org.supercsv.io.CsvListReader;
+import org.supercsv.prefs.CsvPreference;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"csv", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified CSV schema. " +
+"Take a look at the additional documentation of this processor for 
some schema examples.")
+public class ValidateCsv extends AbstractProcessor {
+
+private final static List 

[jira] [Commented] (NIFI-2328) Update UI for JoltTransform processor

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396524#comment-15396524
 ] 

Joseph Witt commented on NIFI-2328:
---

removed fix version for this until someone is able to make progress on it.

> Update UI for JoltTransform processor
> -
>
> Key: NIFI-2328
> URL: https://issues.apache.org/jira/browse/NIFI-2328
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Extensions
>Affects Versions: 1.0.0
>Reporter: Haimo Liu
>
> UI of the JoltTransform processor is un-finished as-is, need to update it



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2328) Update UI for JoltTransform processor

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-2328:
--
Fix Version/s: (was: 1.0.0)

> Update UI for JoltTransform processor
> -
>
> Key: NIFI-2328
> URL: https://issues.apache.org/jira/browse/NIFI-2328
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Extensions
>Affects Versions: 1.0.0
>Reporter: Haimo Liu
>
> UI of the JoltTransform processor is un-finished as-is, need to update it



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-1969) Update Documentation for new Authorization Properties

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-1969:
--
Priority: Critical  (was: Blocker)

> Update Documentation for new Authorization Properties
> -
>
> Key: NIFI-1969
> URL: https://issues.apache.org/jira/browse/NIFI-1969
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Documentation & Website
>Affects Versions: 1.0.0
>Reporter: Mark Payne
>Assignee: Andrew Lim
>Priority: Critical
>  Labels: migration
> Fix For: 1.0.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2395) PersistentProvenanceRepository Deadlocks caused by a blocked journal merge

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-2395:
--
Assignee: Mark Payne  (was: Joseph Witt)

> PersistentProvenanceRepository Deadlocks caused by a blocked journal merge
> --
>
> Key: NIFI-2395
> URL: https://issues.apache.org/jira/browse/NIFI-2395
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 0.6.0, 0.7.0
>Reporter: Brian Davis
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 1.0.0, 0.8.0
>
>
> I have a nifi instance that I have been running for about a week and has 
> deadlocked at least 3 times during this time.  When I say deadlock the whole 
> nifi instance stops doing any progress on flowfiles.  I looked at the stack 
> trace and there are a lot of threads stuck doing tasks in the 
> PersistentProvenanceRepository.  Looking at the code I think this is what is 
> happening:
> There is a ReadWriteLock that all the reads are waiting for a write.  The 
> write is in the loop:
> {code}
> while (journalFileCount > journalCountThreshold || repoSize > 
> sizeThreshold) {
> // if a shutdown happens while we are in this loop, kill 
> the rollover thread and break
> if (this.closed.get()) {
> if (future != null) {
> future.cancel(true);
> }
> break;
> }
> if (repoSize > sizeThreshold) {
> logger.debug("Provenance Repository has exceeded its 
> size threshold; will trigger purging of oldest events");
> purgeOldEvents();
> journalFileCount = getJournalCount();
> repoSize = getSize(getLogFiles(), 0L);
> continue;
> } else {
> // if we are constrained by the number of journal 
> files rather than the size of the repo,
> // then we will just sleep a bit because another 
> thread is already actively merging the journals,
> // due to the runnable that we scheduled above
> try {
> Thread.sleep(100L);
> } catch (final InterruptedException ie) {
> }
> }
> logger.debug("Provenance Repository is still behind. 
> Keeping flow slowed down "
> + "to accommodate. Currently, there are {} 
> journal files ({} bytes) and "
> + "threshold for blocking is {} ({} bytes)", 
> journalFileCount, repoSize, journalCountThreshold, sizeThreshold);
> journalFileCount = getJournalCount();
> repoSize = getSize(getLogFiles(), 0L);
> }
> logger.info("Provenance Repository has now caught up with 
> rolling over journal files. Current number of "
> + "journal files to be rolled over is {}", 
> journalFileCount);
> }
> {code}
> My nifi is at the sleep indefinitely.  The reason my nifi cannot move forward 
> is because of the thread doing the merge is stopped.  The thread doing the 
> merge is at:
> {code}
> accepted = eventQueue.offer(new Tuple<>(record, blockIndex), 10, 
> TimeUnit.MILLISECONDS);
> {code}
> so the queue is full.  
> What I believe happened is that the callables created here:
> {code}
> final Callable callable = new 
> Callable() {
> @Override
> public Object call() throws IOException {
> while (!eventQueue.isEmpty() || 
> !finishedAdding.get()) {
> final 
> Tuple tuple;
> try {
> tuple = eventQueue.poll(10, 
> TimeUnit.MILLISECONDS);
> } catch (final InterruptedException 
> ie) {
> continue;
> }
> if (tuple == null) {
> continue;
> }
> indexingAction.index(tuple.getKey(), 
> indexWriter, tuple.getValue());
> }
> return null;
> }
> {code}
> finish before the offer adds its first event because I 

[jira] [Updated] (NIFI-1969) Update Documentation for new Authorization Properties

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-1969:
--
Labels: migration  (was: )

> Update Documentation for new Authorization Properties
> -
>
> Key: NIFI-1969
> URL: https://issues.apache.org/jira/browse/NIFI-1969
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Documentation & Website
>Affects Versions: 1.0.0
>Reporter: Mark Payne
>Assignee: Andrew Lim
>Priority: Blocker
>  Labels: migration
> Fix For: 1.0.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2175) Uploading a template (via Operator palette) has improper error checking

[Joseph Percivall (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Percivall updated NIFI-2175:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Uploading a template (via Operator palette) has improper error checking
> ---
>
> Key: NIFI-2175
> URL: https://issues.apache.org/jira/browse/NIFI-2175
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Affects Versions: 1.0.0
> Environment: Mac OS X, Chrome
>Reporter: Andrew Lim
>Assignee: Matt Gilman
>  Labels: UI
> Fix For: 1.0.0
>
> Attachments: NIFI-2175_notemplateselected.png
>
>
> Issue 1:
> Select the "Upload Template" button in the Operator palette.  Immediately 
> select "Upload".  The error "The specified template is not in a valid 
> format." is shown even though the user has not selected a template.  See 
> attached screenshot.
> Issue 2:
> This time, select a valid template to upload and select "Upload".  The 
> Success message "Template successfully imported." is displayed.  Select "OK".
> Now select "Upload Template" button from the Operator palette again.  
> Immediately select "Upload".  The Success message "Template successfully 
> imported." is displayed even though no template was selected.
> If you go to the Templates window via the management drop-down, you can see 
> that the template selected earlier was imported again even though it wasn't 
> explicitly chosen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2223) Able to upload same template multiple times

[Joseph Percivall (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Percivall updated NIFI-2223:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Able to upload same template multiple times
> ---
>
> Key: NIFI-2223
> URL: https://issues.apache.org/jira/browse/NIFI-2223
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Affects Versions: 1.0.0
> Environment: Mac OS X; Chrome
>Reporter: Andrew Lim
>Assignee: Matt Gilman
>  Labels: UI
> Fix For: 1.0.0
>
> Attachments: NIFI-2223_ErrorMessage.png, NIFI-2223_sameTemp3X.png
>
>
> UI allows the same template to be uploaded multiple times as shown in the 
> attached screenshot.
> In 0.x, this was not allowed and the error message "A templated named 'XYZ' 
> already exists."  Screenshot attached for this also.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2175) Uploading a template (via Operator palette) has improper error checking

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396467#comment-15396467
 ] 

ASF subversion and git services commented on NIFI-2175:
---

Commit ae344806c0cdbe81104c339d9e810fb3cff1a3c1 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=ae34480 ]

NIFI-2224: - Ensuring the template form is reset when the upload template 
dialog is closed. NIFI-2175: - Not submitting the template form is a template 
isn't choosen. NIFI-2176: - Ensuring a template is specified during creation. 
NIFI-2223: - Ensuring templates with the same name cannot be added. NIFI-2296: 
- Updating the tooltip for the upload template browse button.

- Cleaning up un-used parameters to REST endpoints.

This closes #725

Signed-off-by: jpercivall 


> Uploading a template (via Operator palette) has improper error checking
> ---
>
> Key: NIFI-2175
> URL: https://issues.apache.org/jira/browse/NIFI-2175
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Affects Versions: 1.0.0
> Environment: Mac OS X, Chrome
>Reporter: Andrew Lim
>Assignee: Matt Gilman
>  Labels: UI
> Fix For: 1.0.0
>
> Attachments: NIFI-2175_notemplateselected.png
>
>
> Issue 1:
> Select the "Upload Template" button in the Operator palette.  Immediately 
> select "Upload".  The error "The specified template is not in a valid 
> format." is shown even though the user has not selected a template.  See 
> attached screenshot.
> Issue 2:
> This time, select a valid template to upload and select "Upload".  The 
> Success message "Template successfully imported." is displayed.  Select "OK".
> Now select "Upload Template" button from the Operator palette again.  
> Immediately select "Upload".  The Success message "Template successfully 
> imported." is displayed even though no template was selected.
> If you go to the Templates window via the management drop-down, you can see 
> that the template selected earlier was imported again even though it wasn't 
> explicitly chosen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2224) Upload template dialog does not show template name selected

[Joseph Percivall (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Percivall updated NIFI-2224:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Upload template dialog does not show template name selected
> ---
>
> Key: NIFI-2224
> URL: https://issues.apache.org/jira/browse/NIFI-2224
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Affects Versions: 1.0.0
> Environment: Mac OS X; Chrome
>Reporter: Andrew Lim
>Assignee: Matt Gilman
>  Labels: UI
> Fix For: 1.0.0
>
> Attachments: NIFI-2224.mp4
>
>
> Following STRs are shown in the attached video:
> 1. Select Upload Template from operator palette
> 2. Select magnifying glass icon to browse for template
> 3. Select template '1.xml'
> 4. Select open button
> 5. Upload template dialog in NiFi shows '1.xml' as selected
> 6. Select Upload button
> 7. Template uploaded successfully
> 8. Select Upload Template from operator palette
> 9. Select magnifying glass icon to browse for template
> 10 Select template '1.xml' AGAIN
> 11. Select open button
> 12. Upload template dialog in NiFi does NOT show '1.xml' as selected
> Separate bug (NIFI-2223)
> 13.  Select Upload button
> 14. Template uploaded successfully



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2176) Able to create a template without a name

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396468#comment-15396468
 ] 

ASF subversion and git services commented on NIFI-2176:
---

Commit ae344806c0cdbe81104c339d9e810fb3cff1a3c1 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=ae34480 ]

NIFI-2224: - Ensuring the template form is reset when the upload template 
dialog is closed. NIFI-2175: - Not submitting the template form is a template 
isn't choosen. NIFI-2176: - Ensuring a template is specified during creation. 
NIFI-2223: - Ensuring templates with the same name cannot be added. NIFI-2296: 
- Updating the tooltip for the upload template browse button.

- Cleaning up un-used parameters to REST endpoints.

This closes #725

Signed-off-by: jpercivall 


> Able to create a template without a name
> 
>
> Key: NIFI-2176
> URL: https://issues.apache.org/jira/browse/NIFI-2176
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Andrew Lim
>Assignee: Matt Gilman
>  Labels: UI
> Fix For: 1.0.0
>
>
> Select "Create Template" from the Operator palette.  Immediately select 
> "Create" from the "Create Template" dialog leaving the name field blank.  A 
> success message "Template '' was successfully created." is displayed.
> In 0.x, if a user attempted this they would get an error "Template name 
> cannot be blank."



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2224) Upload template dialog does not show template name selected

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396466#comment-15396466
 ] 

ASF subversion and git services commented on NIFI-2224:
---

Commit ae344806c0cdbe81104c339d9e810fb3cff1a3c1 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=ae34480 ]

NIFI-2224: - Ensuring the template form is reset when the upload template 
dialog is closed. NIFI-2175: - Not submitting the template form is a template 
isn't choosen. NIFI-2176: - Ensuring a template is specified during creation. 
NIFI-2223: - Ensuring templates with the same name cannot be added. NIFI-2296: 
- Updating the tooltip for the upload template browse button.

- Cleaning up un-used parameters to REST endpoints.

This closes #725

Signed-off-by: jpercivall 


> Upload template dialog does not show template name selected
> ---
>
> Key: NIFI-2224
> URL: https://issues.apache.org/jira/browse/NIFI-2224
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Affects Versions: 1.0.0
> Environment: Mac OS X; Chrome
>Reporter: Andrew Lim
>Assignee: Matt Gilman
>  Labels: UI
> Fix For: 1.0.0
>
> Attachments: NIFI-2224.mp4
>
>
> Following STRs are shown in the attached video:
> 1. Select Upload Template from operator palette
> 2. Select magnifying glass icon to browse for template
> 3. Select template '1.xml'
> 4. Select open button
> 5. Upload template dialog in NiFi shows '1.xml' as selected
> 6. Select Upload button
> 7. Template uploaded successfully
> 8. Select Upload Template from operator palette
> 9. Select magnifying glass icon to browse for template
> 10 Select template '1.xml' AGAIN
> 11. Select open button
> 12. Upload template dialog in NiFi does NOT show '1.xml' as selected
> Separate bug (NIFI-2223)
> 13.  Select Upload button
> 14. Template uploaded successfully



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #725: Addressing issues surrounding Template uploading

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/725


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2223) Able to upload same template multiple times

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396469#comment-15396469
 ] 

ASF subversion and git services commented on NIFI-2223:
---

Commit ae344806c0cdbe81104c339d9e810fb3cff1a3c1 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=ae34480 ]

NIFI-2224: - Ensuring the template form is reset when the upload template 
dialog is closed. NIFI-2175: - Not submitting the template form is a template 
isn't choosen. NIFI-2176: - Ensuring a template is specified during creation. 
NIFI-2223: - Ensuring templates with the same name cannot be added. NIFI-2296: 
- Updating the tooltip for the upload template browse button.

- Cleaning up un-used parameters to REST endpoints.

This closes #725

Signed-off-by: jpercivall 


> Able to upload same template multiple times
> ---
>
> Key: NIFI-2223
> URL: https://issues.apache.org/jira/browse/NIFI-2223
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Affects Versions: 1.0.0
> Environment: Mac OS X; Chrome
>Reporter: Andrew Lim
>Assignee: Matt Gilman
>  Labels: UI
> Fix For: 1.0.0
>
> Attachments: NIFI-2223_ErrorMessage.png, NIFI-2223_sameTemp3X.png
>
>
> UI allows the same template to be uploaded multiple times as shown in the 
> attached screenshot.
> In 0.x, this was not allowed and the error message "A templated named 'XYZ' 
> already exists."  Screenshot attached for this also.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #496: NIFI-1965 - Implement QueryDNS Processor

[pvillard31]

Github user pvillard31 commented on the issue:

https://github.com/apache/nifi/pull/496
  
Hey @trixpan, thanks for your work! I've made some comments. Also, could 
you rebase against master? At the moment, I've not been able to test it because 
of the NPE I mentioned in comments. Once comments are addressed, I'll have 
another look!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-1965) Create a QueryDNS processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396461#comment-15396461
 ] 

ASF GitHub Bot commented on NIFI-1965:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/496#discussion_r72530345
  
--- Diff: 
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/test/java/org/apache/nifi/processors/enrich/TestQueryDNS.java
 ---
@@ -0,0 +1,228 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.enrich;
+
+import org.apache.nifi.util.MockFlowFile;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+
+import java.util.HashMap;
+import java.util.Hashtable;
+import java.util.List;
+import java.util.Map;
+
+import org.junit.Before;
+import org.junit.Test;
+import org.mockito.Mockito;
+import org.mockito.invocation.InvocationOnMock;
+import org.mockito.stubbing.Answer;
+
+
+import javax.naming.Context;
+import javax.naming.directory.Attributes;
+import javax.naming.directory.BasicAttribute;
+import javax.naming.directory.BasicAttributes;
+import javax.naming.directory.DirContext;
+
+import static org.junit.Assert.assertTrue;
+
+public class TestQueryDNS  {
+private QueryDNS queryDNS;
+private TestRunner queryDNSTestRunner;
+
+@Before
+public void setupTest() throws Exception {
+this.queryDNS =  new QueryDNS();
+this.queryDNSTestRunner = TestRunners.newTestRunner(queryDNS);
+
+Hashtable env = new Hashtable();
+env.put(Context.INITIAL_CONTEXT_FACTORY, 
FakeDNSInitialDirContextFactory.class.getName());
+
+this.queryDNS.initializeContext(env);
+
+final DirContext mockContext = 
FakeDNSInitialDirContextFactory.getLatestMockContext();
+
+// Capture JNDI's getAttibutes method containing the (String) 
queryValue and (String[]) queryType
+Mockito.when( mockContext.getAttributes(Mockito.anyString(), 
Mockito.any(String[].class)))
+.thenAnswer(new Answer() {
+public Object answer(InvocationOnMock invocation) 
throws Throwable {
+// Craft a false DNS response
+// Note the DNS response will not make use of any 
of the mocked
+// query contents (all input is discarded and 
replies synthetically
+// generated
+return craftResponse(invocation);
+}
+});
+}
+
+@Test
+public void testVanillaQueryWithoutSplit()  {
+queryDNSTestRunner.setProperty(QueryDNS.DNS_QUERY_TYPE, "PTR");
+queryDNSTestRunner.setProperty(QueryDNS.DNS_RETRIES, "1");
+queryDNSTestRunner.setProperty(QueryDNS.DNS_TIMEOUT, "1000");
+queryDNSTestRunner.setProperty(QueryDNS.QUERY_INPUT, 
"${ip_address:getDelimitedField(4, '.'):trim()}" +
+".${ip_address:getDelimitedField(3, '.'):trim()}" +
+".${ip_address:getDelimitedField(2, '.'):trim()}" +
+".${ip_address:getDelimitedField(1, '.'):trim()}" +
+".in-addr.arpa");
+queryDNSTestRunner.setProperty(QueryDNS.QUERY_PARSER, 
QueryDNS.NONE.getValue());
+
+final Map attributeMap = new HashMap<>();
+attributeMap.put("ip_address", "123.123.123.123");
+
+queryDNSTestRunner.enqueue(new byte[0], attributeMap);
+queryDNSTestRunner.enqueue("teste teste teste 
chocolate".getBytes());
+
+queryDNSTestRunner.run(1,true, false);
+
+List results = 
queryDNSTestRunner.getFlowFilesForRelationship(QueryDNS.REL_FOUND);
+assertTrue(results.size() == 1);
+String result = 

[GitHub] nifi pull request #496: NIFI-1965 - Implement QueryDNS Processor

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/496#discussion_r72530345
  
--- Diff: 
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/test/java/org/apache/nifi/processors/enrich/TestQueryDNS.java
 ---
@@ -0,0 +1,228 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.enrich;
+
+import org.apache.nifi.util.MockFlowFile;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+
+import java.util.HashMap;
+import java.util.Hashtable;
+import java.util.List;
+import java.util.Map;
+
+import org.junit.Before;
+import org.junit.Test;
+import org.mockito.Mockito;
+import org.mockito.invocation.InvocationOnMock;
+import org.mockito.stubbing.Answer;
+
+
+import javax.naming.Context;
+import javax.naming.directory.Attributes;
+import javax.naming.directory.BasicAttribute;
+import javax.naming.directory.BasicAttributes;
+import javax.naming.directory.DirContext;
+
+import static org.junit.Assert.assertTrue;
+
+public class TestQueryDNS  {
+private QueryDNS queryDNS;
+private TestRunner queryDNSTestRunner;
+
+@Before
+public void setupTest() throws Exception {
+this.queryDNS =  new QueryDNS();
+this.queryDNSTestRunner = TestRunners.newTestRunner(queryDNS);
+
+Hashtable env = new Hashtable();
+env.put(Context.INITIAL_CONTEXT_FACTORY, 
FakeDNSInitialDirContextFactory.class.getName());
+
+this.queryDNS.initializeContext(env);
+
+final DirContext mockContext = 
FakeDNSInitialDirContextFactory.getLatestMockContext();
+
+// Capture JNDI's getAttibutes method containing the (String) 
queryValue and (String[]) queryType
+Mockito.when( mockContext.getAttributes(Mockito.anyString(), 
Mockito.any(String[].class)))
+.thenAnswer(new Answer() {
+public Object answer(InvocationOnMock invocation) 
throws Throwable {
+// Craft a false DNS response
+// Note the DNS response will not make use of any 
of the mocked
+// query contents (all input is discarded and 
replies synthetically
+// generated
+return craftResponse(invocation);
+}
+});
+}
+
+@Test
+public void testVanillaQueryWithoutSplit()  {
+queryDNSTestRunner.setProperty(QueryDNS.DNS_QUERY_TYPE, "PTR");
+queryDNSTestRunner.setProperty(QueryDNS.DNS_RETRIES, "1");
+queryDNSTestRunner.setProperty(QueryDNS.DNS_TIMEOUT, "1000");
+queryDNSTestRunner.setProperty(QueryDNS.QUERY_INPUT, 
"${ip_address:getDelimitedField(4, '.'):trim()}" +
+".${ip_address:getDelimitedField(3, '.'):trim()}" +
+".${ip_address:getDelimitedField(2, '.'):trim()}" +
+".${ip_address:getDelimitedField(1, '.'):trim()}" +
+".in-addr.arpa");
+queryDNSTestRunner.setProperty(QueryDNS.QUERY_PARSER, 
QueryDNS.NONE.getValue());
+
+final Map attributeMap = new HashMap<>();
+attributeMap.put("ip_address", "123.123.123.123");
+
+queryDNSTestRunner.enqueue(new byte[0], attributeMap);
+queryDNSTestRunner.enqueue("teste teste teste 
chocolate".getBytes());
+
+queryDNSTestRunner.run(1,true, false);
+
+List results = 
queryDNSTestRunner.getFlowFilesForRelationship(QueryDNS.REL_FOUND);
+assertTrue(results.size() == 1);
+String result = 
results.get(0).getAttribute("enrich.dns.record0.group0");
+
+assertTrue(result.contains("apache.nifi.org"));
+
+
+}
+
+@Test
+public void testValidDataWithSplit()  {
+

[jira] [Commented] (NIFI-1965) Create a QueryDNS processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396454#comment-15396454
 ] 

ASF GitHub Bot commented on NIFI-1965:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/496#discussion_r72530181
  
--- Diff: 
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/enrich/QueryDNS.java
 ---
@@ -0,0 +1,269 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.enrich;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Hashtable;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import javax.naming.Context;
+import javax.naming.NameNotFoundException;
+import javax.naming.NamingEnumeration;
+import javax.naming.NamingException;
+import javax.naming.directory.Attributes;
+import javax.naming.directory.BasicAttributes;
+import javax.naming.directory.DirContext;
+import javax.naming.directory.InitialDirContext;
+
+import org.apache.commons.lang3.StringUtils;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"dns", "enrich", "ip"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("A powerful DNS query processor primary designed to 
enrich DataFlows with DNS based APIs " +
+"(e.g. RBLs, ShadowServer's ASN lookup) but that can be also used 
to perform regular DNS lookups.")
+@WritesAttributes({
+@WritesAttribute(attribute = "enrich.dns.record*.group*", 
description = "The captured fields of the DNS query response for each of the 
records received"),
+})
+public class QueryDNS extends AbstractEnrichProcessor {
+
+public static final PropertyDescriptor DNS_QUERY_TYPE = new 
PropertyDescriptor.Builder()
+.name("DNS Query Type")
+.description("The DNS query type to be used by the processor 
(e.g. TXT, A)")
+.required(true)
+.defaultValue("TXT")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_SERVER = new 
PropertyDescriptor.Builder()
+.name("DNS Servers")
+.description("A comma separated list of  DNS servers to be 
used. (Defaults to system wide if none is used)")
+.required(false)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_TIMEOUT = new 
PropertyDescriptor.Builder()
+.name("DNS Query Timeout")
+.description("The amount of milliseconds to wait until 
considering a query as failed")
+.required(true)
+.defaultValue("1500")
+

[GitHub] nifi pull request #496: NIFI-1965 - Implement QueryDNS Processor

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/496#discussion_r72530250
  
--- Diff: 
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/enrich/QueryDNS.java
 ---
@@ -0,0 +1,269 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.enrich;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Hashtable;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import javax.naming.Context;
+import javax.naming.NameNotFoundException;
+import javax.naming.NamingEnumeration;
+import javax.naming.NamingException;
+import javax.naming.directory.Attributes;
+import javax.naming.directory.BasicAttributes;
+import javax.naming.directory.DirContext;
+import javax.naming.directory.InitialDirContext;
+
+import org.apache.commons.lang3.StringUtils;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"dns", "enrich", "ip"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("A powerful DNS query processor primary designed to 
enrich DataFlows with DNS based APIs " +
+"(e.g. RBLs, ShadowServer's ASN lookup) but that can be also used 
to perform regular DNS lookups.")
+@WritesAttributes({
+@WritesAttribute(attribute = "enrich.dns.record*.group*", 
description = "The captured fields of the DNS query response for each of the 
records received"),
+})
+public class QueryDNS extends AbstractEnrichProcessor {
+
+public static final PropertyDescriptor DNS_QUERY_TYPE = new 
PropertyDescriptor.Builder()
+.name("DNS Query Type")
+.description("The DNS query type to be used by the processor 
(e.g. TXT, A)")
+.required(true)
+.defaultValue("TXT")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_SERVER = new 
PropertyDescriptor.Builder()
+.name("DNS Servers")
+.description("A comma separated list of  DNS servers to be 
used. (Defaults to system wide if none is used)")
+.required(false)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_TIMEOUT = new 
PropertyDescriptor.Builder()
+.name("DNS Query Timeout")
+.description("The amount of milliseconds to wait until 
considering a query as failed")
+.required(true)
+.defaultValue("1500")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_RETRIES = new 
PropertyDescriptor.Builder()
+.name("DNS Query Retries")
+

[GitHub] nifi pull request #496: NIFI-1965 - Implement QueryDNS Processor

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/496#discussion_r72530181
  
--- Diff: 
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/enrich/QueryDNS.java
 ---
@@ -0,0 +1,269 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.enrich;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Hashtable;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import javax.naming.Context;
+import javax.naming.NameNotFoundException;
+import javax.naming.NamingEnumeration;
+import javax.naming.NamingException;
+import javax.naming.directory.Attributes;
+import javax.naming.directory.BasicAttributes;
+import javax.naming.directory.DirContext;
+import javax.naming.directory.InitialDirContext;
+
+import org.apache.commons.lang3.StringUtils;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"dns", "enrich", "ip"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("A powerful DNS query processor primary designed to 
enrich DataFlows with DNS based APIs " +
+"(e.g. RBLs, ShadowServer's ASN lookup) but that can be also used 
to perform regular DNS lookups.")
+@WritesAttributes({
+@WritesAttribute(attribute = "enrich.dns.record*.group*", 
description = "The captured fields of the DNS query response for each of the 
records received"),
+})
+public class QueryDNS extends AbstractEnrichProcessor {
+
+public static final PropertyDescriptor DNS_QUERY_TYPE = new 
PropertyDescriptor.Builder()
+.name("DNS Query Type")
+.description("The DNS query type to be used by the processor 
(e.g. TXT, A)")
+.required(true)
+.defaultValue("TXT")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_SERVER = new 
PropertyDescriptor.Builder()
+.name("DNS Servers")
+.description("A comma separated list of  DNS servers to be 
used. (Defaults to system wide if none is used)")
+.required(false)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_TIMEOUT = new 
PropertyDescriptor.Builder()
+.name("DNS Query Timeout")
+.description("The amount of milliseconds to wait until 
considering a query as failed")
+.required(true)
+.defaultValue("1500")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_RETRIES = new 
PropertyDescriptor.Builder()
+.name("DNS Query Retries")
+

[jira] [Commented] (NIFI-1965) Create a QueryDNS processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396447#comment-15396447
 ] 

ASF GitHub Bot commented on NIFI-1965:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/496#discussion_r72529729
  
--- Diff: 
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/enrich/QueryDNS.java
 ---
@@ -0,0 +1,269 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.enrich;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Hashtable;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import javax.naming.Context;
+import javax.naming.NameNotFoundException;
+import javax.naming.NamingEnumeration;
+import javax.naming.NamingException;
+import javax.naming.directory.Attributes;
+import javax.naming.directory.BasicAttributes;
+import javax.naming.directory.DirContext;
+import javax.naming.directory.InitialDirContext;
+
+import org.apache.commons.lang3.StringUtils;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"dns", "enrich", "ip"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("A powerful DNS query processor primary designed to 
enrich DataFlows with DNS based APIs " +
+"(e.g. RBLs, ShadowServer's ASN lookup) but that can be also used 
to perform regular DNS lookups.")
+@WritesAttributes({
+@WritesAttribute(attribute = "enrich.dns.record*.group*", 
description = "The captured fields of the DNS query response for each of the 
records received"),
+})
+public class QueryDNS extends AbstractEnrichProcessor {
+
+public static final PropertyDescriptor DNS_QUERY_TYPE = new 
PropertyDescriptor.Builder()
+.name("DNS Query Type")
+.description("The DNS query type to be used by the processor 
(e.g. TXT, A)")
+.required(true)
+.defaultValue("TXT")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_SERVER = new 
PropertyDescriptor.Builder()
+.name("DNS Servers")
+.description("A comma separated list of  DNS servers to be 
used. (Defaults to system wide if none is used)")
+.required(false)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_TIMEOUT = new 
PropertyDescriptor.Builder()
+.name("DNS Query Timeout")
+.description("The amount of milliseconds to wait until 
considering a query as failed")
+.required(true)
+.defaultValue("1500")
+

[GitHub] nifi pull request #496: NIFI-1965 - Implement QueryDNS Processor

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/496#discussion_r72529865
  
--- Diff: 
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/enrich/QueryDNS.java
 ---
@@ -0,0 +1,269 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.enrich;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Hashtable;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import javax.naming.Context;
+import javax.naming.NameNotFoundException;
+import javax.naming.NamingEnumeration;
+import javax.naming.NamingException;
+import javax.naming.directory.Attributes;
+import javax.naming.directory.BasicAttributes;
+import javax.naming.directory.DirContext;
+import javax.naming.directory.InitialDirContext;
+
+import org.apache.commons.lang3.StringUtils;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"dns", "enrich", "ip"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("A powerful DNS query processor primary designed to 
enrich DataFlows with DNS based APIs " +
+"(e.g. RBLs, ShadowServer's ASN lookup) but that can be also used 
to perform regular DNS lookups.")
+@WritesAttributes({
+@WritesAttribute(attribute = "enrich.dns.record*.group*", 
description = "The captured fields of the DNS query response for each of the 
records received"),
+})
+public class QueryDNS extends AbstractEnrichProcessor {
+
+public static final PropertyDescriptor DNS_QUERY_TYPE = new 
PropertyDescriptor.Builder()
+.name("DNS Query Type")
+.description("The DNS query type to be used by the processor 
(e.g. TXT, A)")
+.required(true)
+.defaultValue("TXT")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_SERVER = new 
PropertyDescriptor.Builder()
+.name("DNS Servers")
+.description("A comma separated list of  DNS servers to be 
used. (Defaults to system wide if none is used)")
+.required(false)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_TIMEOUT = new 
PropertyDescriptor.Builder()
+.name("DNS Query Timeout")
+.description("The amount of milliseconds to wait until 
considering a query as failed")
+.required(true)
+.defaultValue("1500")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_RETRIES = new 
PropertyDescriptor.Builder()
+.name("DNS Query Retries")
+

[jira] [Commented] (NIFI-1965) Create a QueryDNS processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396424#comment-15396424
 ] 

ASF GitHub Bot commented on NIFI-1965:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/496#discussion_r72528893
  
--- Diff: 
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/enrich/QueryDNS.java
 ---
@@ -0,0 +1,269 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.enrich;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Hashtable;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import javax.naming.Context;
+import javax.naming.NameNotFoundException;
+import javax.naming.NamingEnumeration;
+import javax.naming.NamingException;
+import javax.naming.directory.Attributes;
+import javax.naming.directory.BasicAttributes;
+import javax.naming.directory.DirContext;
+import javax.naming.directory.InitialDirContext;
+
+import org.apache.commons.lang3.StringUtils;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"dns", "enrich", "ip"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("A powerful DNS query processor primary designed to 
enrich DataFlows with DNS based APIs " +
+"(e.g. RBLs, ShadowServer's ASN lookup) but that can be also used 
to perform regular DNS lookups.")
+@WritesAttributes({
+@WritesAttribute(attribute = "enrich.dns.record*.group*", 
description = "The captured fields of the DNS query response for each of the 
records received"),
+})
+public class QueryDNS extends AbstractEnrichProcessor {
+
+public static final PropertyDescriptor DNS_QUERY_TYPE = new 
PropertyDescriptor.Builder()
+.name("DNS Query Type")
+.description("The DNS query type to be used by the processor 
(e.g. TXT, A)")
+.required(true)
+.defaultValue("TXT")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_SERVER = new 
PropertyDescriptor.Builder()
+.name("DNS Servers")
+.description("A comma separated list of  DNS servers to be 
used. (Defaults to system wide if none is used)")
+.required(false)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_TIMEOUT = new 
PropertyDescriptor.Builder()
+.name("DNS Query Timeout")
+.description("The amount of milliseconds to wait until 
considering a query as failed")
+.required(true)
+.defaultValue("1500")
+

[GitHub] nifi pull request #496: NIFI-1965 - Implement QueryDNS Processor

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/496#discussion_r72528893
  
--- Diff: 
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/enrich/QueryDNS.java
 ---
@@ -0,0 +1,269 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.enrich;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Hashtable;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import javax.naming.Context;
+import javax.naming.NameNotFoundException;
+import javax.naming.NamingEnumeration;
+import javax.naming.NamingException;
+import javax.naming.directory.Attributes;
+import javax.naming.directory.BasicAttributes;
+import javax.naming.directory.DirContext;
+import javax.naming.directory.InitialDirContext;
+
+import org.apache.commons.lang3.StringUtils;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"dns", "enrich", "ip"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("A powerful DNS query processor primary designed to 
enrich DataFlows with DNS based APIs " +
+"(e.g. RBLs, ShadowServer's ASN lookup) but that can be also used 
to perform regular DNS lookups.")
+@WritesAttributes({
+@WritesAttribute(attribute = "enrich.dns.record*.group*", 
description = "The captured fields of the DNS query response for each of the 
records received"),
+})
+public class QueryDNS extends AbstractEnrichProcessor {
+
+public static final PropertyDescriptor DNS_QUERY_TYPE = new 
PropertyDescriptor.Builder()
+.name("DNS Query Type")
+.description("The DNS query type to be used by the processor 
(e.g. TXT, A)")
+.required(true)
+.defaultValue("TXT")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_SERVER = new 
PropertyDescriptor.Builder()
+.name("DNS Servers")
+.description("A comma separated list of  DNS servers to be 
used. (Defaults to system wide if none is used)")
+.required(false)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_TIMEOUT = new 
PropertyDescriptor.Builder()
+.name("DNS Query Timeout")
+.description("The amount of milliseconds to wait until 
considering a query as failed")
+.required(true)
+.defaultValue("1500")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_RETRIES = new 
PropertyDescriptor.Builder()
+.name("DNS Query Retries")
+

[GitHub] nifi pull request #496: NIFI-1965 - Implement QueryDNS Processor

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/496#discussion_r72527914
  
--- Diff: 
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/enrich/QueryDNS.java
 ---
@@ -0,0 +1,269 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.enrich;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Hashtable;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import javax.naming.Context;
+import javax.naming.NameNotFoundException;
+import javax.naming.NamingEnumeration;
+import javax.naming.NamingException;
+import javax.naming.directory.Attributes;
+import javax.naming.directory.BasicAttributes;
+import javax.naming.directory.DirContext;
+import javax.naming.directory.InitialDirContext;
+
+import org.apache.commons.lang3.StringUtils;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"dns", "enrich", "ip"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("A powerful DNS query processor primary designed to 
enrich DataFlows with DNS based APIs " +
+"(e.g. RBLs, ShadowServer's ASN lookup) but that can be also used 
to perform regular DNS lookups.")
+@WritesAttributes({
+@WritesAttribute(attribute = "enrich.dns.record*.group*", 
description = "The captured fields of the DNS query response for each of the 
records received"),
+})
+public class QueryDNS extends AbstractEnrichProcessor {
+
+public static final PropertyDescriptor DNS_QUERY_TYPE = new 
PropertyDescriptor.Builder()
+.name("DNS Query Type")
+.description("The DNS query type to be used by the processor 
(e.g. TXT, A)")
+.required(true)
+.defaultValue("TXT")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_SERVER = new 
PropertyDescriptor.Builder()
+.name("DNS Servers")
+.description("A comma separated list of  DNS servers to be 
used. (Defaults to system wide if none is used)")
+.required(false)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_TIMEOUT = new 
PropertyDescriptor.Builder()
+.name("DNS Query Timeout")
+.description("The amount of milliseconds to wait until 
considering a query as failed")
+.required(true)
+.defaultValue("1500")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor DNS_RETRIES = new 
PropertyDescriptor.Builder()
+.name("DNS Query Retries")
+

[jira] [Commented] (NIFI-1965) Create a QueryDNS processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396405#comment-15396405
 ] 

ASF GitHub Bot commented on NIFI-1965:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/496#discussion_r72527360
  
--- Diff: 
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/enrich/AbstractEnrichProcessor.java
 ---
@@ -0,0 +1,151 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.enrich;
+
+
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+public abstract class AbstractEnrichProcessor extends AbstractProcessor {
+public static final PropertyDescriptor QUERY_INPUT = new 
PropertyDescriptor.Builder()
+.name("Format the query should be to be executed ")
+.required(true)
+.description("The value that should be used to populate the 
query")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final AllowableValue SPLIT= new AllowableValue("Split", 
"Split",
+"String.splitUse a delimiter character or RegEx  to split the 
results into attributes");
+public static final AllowableValue REGEX = new AllowableValue("RegEx", 
"RegEx",
+"Use a regular expression to split the results into attributes 
");
+public static final AllowableValue NONE = new AllowableValue("None", 
"None",
+"Do not split results");
+
+public static final PropertyDescriptor QUERY_PARSER = new 
PropertyDescriptor.Builder()
+.name("Results Parser")
+.description("The method used to slice the results into 
attribute groups")
+.allowableValues(SPLIT, REGEX, NONE)
+.required(true)
+.defaultValue(NONE.getValue())
+.build();
+
+public static final PropertyDescriptor QUERY_PARSER_INPUT = new 
PropertyDescriptor.Builder()
+.name("Parser RegEx")
+.description("Choice between a splitter and regex matcher used 
to parse the results of the query into attribute groups")
+.expressionLanguageSupported(false)
+.required(false)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+
+public static final Relationship REL_FOUND = new Relationship.Builder()
+.name("found")
+.description("Where to route flow files after successfully 
enriching attributes with data")
+.build();
+
+public static final Relationship REL_NOT_FOUND = new 
Relationship.Builder()
+.name("not found")
+.description("Where to route flow if data enrichment query 
rendered no results")
+.build();
+
+
+@Override
+public List customValidate(ValidationContext 
validationContext) {
+final List results = new 
ArrayList<>(super.customValidate(validationContext));
+
+final String chosenQUERY_PARSER = 
validationContext.getProperty(QUERY_PARSER).getValue();
+final boolean QUERY_PARSER_INPUT_isSet = 
validationContext.getProperty(QUERY_PARSER_INPUT).isSet();
+
+if ((!chosenQUERY_PARSER.equals(NONE.getValue()) 

[GitHub] nifi pull request #496: NIFI-1965 - Implement QueryDNS Processor

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/496#discussion_r72527360
  
--- Diff: 
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/enrich/AbstractEnrichProcessor.java
 ---
@@ -0,0 +1,151 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.enrich;
+
+
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+public abstract class AbstractEnrichProcessor extends AbstractProcessor {
+public static final PropertyDescriptor QUERY_INPUT = new 
PropertyDescriptor.Builder()
+.name("Format the query should be to be executed ")
+.required(true)
+.description("The value that should be used to populate the 
query")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final AllowableValue SPLIT= new AllowableValue("Split", 
"Split",
+"String.splitUse a delimiter character or RegEx  to split the 
results into attributes");
+public static final AllowableValue REGEX = new AllowableValue("RegEx", 
"RegEx",
+"Use a regular expression to split the results into attributes 
");
+public static final AllowableValue NONE = new AllowableValue("None", 
"None",
+"Do not split results");
+
+public static final PropertyDescriptor QUERY_PARSER = new 
PropertyDescriptor.Builder()
+.name("Results Parser")
+.description("The method used to slice the results into 
attribute groups")
+.allowableValues(SPLIT, REGEX, NONE)
+.required(true)
+.defaultValue(NONE.getValue())
+.build();
+
+public static final PropertyDescriptor QUERY_PARSER_INPUT = new 
PropertyDescriptor.Builder()
+.name("Parser RegEx")
+.description("Choice between a splitter and regex matcher used 
to parse the results of the query into attribute groups")
+.expressionLanguageSupported(false)
+.required(false)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+
+public static final Relationship REL_FOUND = new Relationship.Builder()
+.name("found")
+.description("Where to route flow files after successfully 
enriching attributes with data")
+.build();
+
+public static final Relationship REL_NOT_FOUND = new 
Relationship.Builder()
+.name("not found")
+.description("Where to route flow if data enrichment query 
rendered no results")
+.build();
+
+
+@Override
+public List customValidate(ValidationContext 
validationContext) {
+final List results = new 
ArrayList<>(super.customValidate(validationContext));
+
+final String chosenQUERY_PARSER = 
validationContext.getProperty(QUERY_PARSER).getValue();
+final boolean QUERY_PARSER_INPUT_isSet = 
validationContext.getProperty(QUERY_PARSER_INPUT).isSet();
+
+if ((!chosenQUERY_PARSER.equals(NONE.getValue()) ) && ( 
!QUERY_PARSER_INPUT_isSet )) {
+results.add(new ValidationResult.Builder().input("Parser 
RegEx")
+.explanation("Split and Regex parsers require a valid 
Regular Expression")
+

[jira] [Commented] (NIFI-1965) Create a QueryDNS processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396403#comment-15396403
 ] 

ASF GitHub Bot commented on NIFI-1965:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/496#discussion_r72526887
  
--- Diff: 
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/enrich/AbstractEnrichProcessor.java
 ---
@@ -0,0 +1,151 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.enrich;
+
+
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+public abstract class AbstractEnrichProcessor extends AbstractProcessor {
+public static final PropertyDescriptor QUERY_INPUT = new 
PropertyDescriptor.Builder()
+.name("Format the query should be to be executed ")
+.required(true)
+.description("The value that should be used to populate the 
query")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final AllowableValue SPLIT= new AllowableValue("Split", 
"Split",
+"String.splitUse a delimiter character or RegEx  to split the 
results into attributes");
+public static final AllowableValue REGEX = new AllowableValue("RegEx", 
"RegEx",
+"Use a regular expression to split the results into attributes 
");
+public static final AllowableValue NONE = new AllowableValue("None", 
"None",
+"Do not split results");
+
+public static final PropertyDescriptor QUERY_PARSER = new 
PropertyDescriptor.Builder()
+.name("Results Parser")
+.description("The method used to slice the results into 
attribute groups")
+.allowableValues(SPLIT, REGEX, NONE)
+.required(true)
+.defaultValue(NONE.getValue())
+.build();
+
+public static final PropertyDescriptor QUERY_PARSER_INPUT = new 
PropertyDescriptor.Builder()
+.name("Parser RegEx")
+.description("Choice between a splitter and regex matcher used 
to parse the results of the query into attribute groups")
+.expressionLanguageSupported(false)
+.required(false)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+
+public static final Relationship REL_FOUND = new Relationship.Builder()
+.name("found")
+.description("Where to route flow files after successfully 
enriching attributes with data")
+.build();
+
+public static final Relationship REL_NOT_FOUND = new 
Relationship.Builder()
+.name("not found")
+.description("Where to route flow if data enrichment query 
rendered no results")
--- End diff --

'flow files' (to be consistent with 'found' relationship


> Create a QueryDNS processor
> ---
>
> Key: NIFI-1965
> URL: https://issues.apache.org/jira/browse/NIFI-1965
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Andre
>
> As part of a data pipeline security teams frequently must enrich data using 
> DNS enabled APIs such as:
> ShadowServer BGP and ASN lookup via DNS
> 

[GitHub] nifi pull request #496: NIFI-1965 - Implement QueryDNS Processor

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/496#discussion_r72526887
  
--- Diff: 
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/enrich/AbstractEnrichProcessor.java
 ---
@@ -0,0 +1,151 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.enrich;
+
+
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+public abstract class AbstractEnrichProcessor extends AbstractProcessor {
+public static final PropertyDescriptor QUERY_INPUT = new 
PropertyDescriptor.Builder()
+.name("Format the query should be to be executed ")
+.required(true)
+.description("The value that should be used to populate the 
query")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final AllowableValue SPLIT= new AllowableValue("Split", 
"Split",
+"String.splitUse a delimiter character or RegEx  to split the 
results into attributes");
+public static final AllowableValue REGEX = new AllowableValue("RegEx", 
"RegEx",
+"Use a regular expression to split the results into attributes 
");
+public static final AllowableValue NONE = new AllowableValue("None", 
"None",
+"Do not split results");
+
+public static final PropertyDescriptor QUERY_PARSER = new 
PropertyDescriptor.Builder()
+.name("Results Parser")
+.description("The method used to slice the results into 
attribute groups")
+.allowableValues(SPLIT, REGEX, NONE)
+.required(true)
+.defaultValue(NONE.getValue())
+.build();
+
+public static final PropertyDescriptor QUERY_PARSER_INPUT = new 
PropertyDescriptor.Builder()
+.name("Parser RegEx")
+.description("Choice between a splitter and regex matcher used 
to parse the results of the query into attribute groups")
+.expressionLanguageSupported(false)
+.required(false)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+
+public static final Relationship REL_FOUND = new Relationship.Builder()
+.name("found")
+.description("Where to route flow files after successfully 
enriching attributes with data")
+.build();
+
+public static final Relationship REL_NOT_FOUND = new 
Relationship.Builder()
+.name("not found")
+.description("Where to route flow if data enrichment query 
rendered no results")
--- End diff --

'flow files' (to be consistent with 'found' relationship


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #496: NIFI-1965 - Implement QueryDNS Processor

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/496#discussion_r72526705
  
--- Diff: 
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/enrich/AbstractEnrichProcessor.java
 ---
@@ -0,0 +1,151 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.enrich;
+
+
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+public abstract class AbstractEnrichProcessor extends AbstractProcessor {
+public static final PropertyDescriptor QUERY_INPUT = new 
PropertyDescriptor.Builder()
+.name("Format the query should be to be executed ")
+.required(true)
+.description("The value that should be used to populate the 
query")
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final AllowableValue SPLIT= new AllowableValue("Split", 
"Split",
+"String.splitUse a delimiter character or RegEx  to split the 
results into attributes");
--- End diff --

wording?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-1965) Create a QueryDNS processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396398#comment-15396398
 ] 

ASF GitHub Bot commented on NIFI-1965:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/496#discussion_r72526574
  
--- Diff: 
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/enrich/AbstractEnrichProcessor.java
 ---
@@ -0,0 +1,151 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.enrich;
+
+
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+public abstract class AbstractEnrichProcessor extends AbstractProcessor {
+public static final PropertyDescriptor QUERY_INPUT = new 
PropertyDescriptor.Builder()
+.name("Format the query should be to be executed ")
--- End diff --

please use name() and displayName() when defining properties [1]


https://mail-archives.apache.org/mod_mbox/nifi-dev/201605.mbox/%3c7e801f27-09cb-4c02-983d-f34ac4420...@hortonworks.com%3E


> Create a QueryDNS processor
> ---
>
> Key: NIFI-1965
> URL: https://issues.apache.org/jira/browse/NIFI-1965
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Andre
>
> As part of a data pipeline security teams frequently must enrich data using 
> DNS enabled APIs such as:
> ShadowServer BGP and ASN lookup via DNS
> https://www.shadowserver.org/wiki/pmwiki.php/Services/IP-BGP#toc7 
> Team Cymru Malware Hash Registry
> http://www.team-cymru.org/MHR.html
> Spamhaus (SBL, XBL, etc)
> and others
> QueryDNS will use an expression language enabled property to run a query 
> against DNS and add the raw result to an attribute (for later processing if 
> necessary). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #496: NIFI-1965 - Implement QueryDNS Processor

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/496#discussion_r72526574
  
--- Diff: 
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/enrich/AbstractEnrichProcessor.java
 ---
@@ -0,0 +1,151 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.enrich;
+
+
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+public abstract class AbstractEnrichProcessor extends AbstractProcessor {
+public static final PropertyDescriptor QUERY_INPUT = new 
PropertyDescriptor.Builder()
+.name("Format the query should be to be executed ")
--- End diff --

please use name() and displayName() when defining properties [1]


https://mail-archives.apache.org/mod_mbox/nifi-dev/201605.mbox/%3c7e801f27-09cb-4c02-983d-f34ac4420...@hortonworks.com%3E


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #496: NIFI-1965 - Implement QueryDNS Processor

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/496#discussion_r72526422
  
--- Diff: 
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/enrich/AbstractEnrichProcessor.java
 ---
@@ -0,0 +1,151 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.enrich;
+
+
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+public abstract class AbstractEnrichProcessor extends AbstractProcessor {
+public static final PropertyDescriptor QUERY_INPUT = new 
PropertyDescriptor.Builder()
+.name("Format the query should be to be executed ")
--- End diff --

wording?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (NIFI-2348) When processor is selected, cannot copy id from palette

[Scott Aslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan reassigned NIFI-2348:
-

Assignee: Scott Aslan

> When processor is selected, cannot copy id from palette
> ---
>
> Key: NIFI-2348
> URL: https://issues.apache.org/jira/browse/NIFI-2348
> Project: Apache NiFi
>  Issue Type: Sub-task
>Reporter: Joseph Percivall
>Assignee: Scott Aslan
>Priority: Minor
> Fix For: 1.0.0
>
>
> Processor selected
> Processor details show up in palette
> highlight uuid in palette (processor remains "selected" on graph)
> copy then copies the processor instead of the uuid
> As a user, I would like to copy the uuid when I have the text highlighted 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2396) Bulletin Board shell header and filter control not fixed position

[Scott Aslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan updated NIFI-2396:
--
Status: Patch Available  (was: In Progress)

https://github.com/apache/nifi/pull/732

> Bulletin Board shell header and filter control not fixed position
> -
>
> Key: NIFI-2396
> URL: https://issues.apache.org/jira/browse/NIFI-2396
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Rob Moran
>Assignee: Scott Aslan
> Attachments: Screen Shot 2016-07-25 at 3.28.51 PM.png
>
>
> See attached screenshot – the header and filters control should stay in a 
> fixed position while only the bulletin list scrolls.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2141) Hide bulletin icon on components

[Scott Aslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan updated NIFI-2141:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Hide bulletin icon on components 
> -
>
> Key: NIFI-2141
> URL: https://issues.apache.org/jira/browse/NIFI-2141
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Rob Moran
>Assignee: Scott Aslan
>
> When there is no bulletin for a component to call out, recommend hiding them. 
> It is consistent with current behavior. Having them visible all the time only 
> adds clutter and is a bit distracting on processors.
> When a bulletin does come in, everything should remain in place as it is now. 
> The background-color will change to the warning color (#ba554a) and the 
> bulletin icon will be white.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2387) Controller Service bulletin highlighting is "off"

[Scott Aslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan updated NIFI-2387:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Controller Service bulletin highlighting is "off"
> -
>
> Key: NIFI-2387
> URL: https://issues.apache.org/jira/browse/NIFI-2387
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Joseph Percivall
>Assignee: Scott Aslan
> Attachments: Screen Shot 2016-07-25 at 10.22.17 AM.png
>
>
> View root level controller services
> start a DistributedMapCacheServer with the same port as the NiFi instance
> See a red square partially covering the "bulletin" icon
> See attached image



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2025) Birdseye does not update when repositioning an existing item on the canvas

[Scott Aslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan updated NIFI-2025:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Birdseye does not update when repositioning an existing item on the canvas
> --
>
> Key: NIFI-2025
> URL: https://issues.apache.org/jira/browse/NIFI-2025
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Scott Aslan
>Assignee: Scott Aslan
> Fix For: 1.0.0
>
> Attachments: screenshot-1.png, screenshot-2.png
>
>
> Birdseye does not update when repositioning an existing item on the canvas 
> but it does update appropriately when adding/deleting items from the canvas. 
> We need to refresh the birdseye when repositioning items on the canvas.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2367) Overlapping text in Cluster pane after error

[Scott Aslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan updated NIFI-2367:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Overlapping text in Cluster pane after error
> 
>
> Key: NIFI-2367
> URL: https://issues.apache.org/jira/browse/NIFI-2367
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Joseph Percivall
>Assignee: Scott Aslan
> Attachments: Screen Shot 2016-07-22 at 2.42.48 PM.png
>
>
> 3 node secure cluster
> In cluster pane
> hit 500 error when refreshing pane (output in developer tools): 
> https://localhost:8481/nifi-api/controller/cluster Failed to load resource: 
> the server responded with a status of 500 (Internal Server Error)
> See overlapping text of "fix" and "home" in resulting pane (see attached 
> screenshot).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #732: [NIFI-2396] Update Bulletin Board shell header and f...

[scottyaslan]

GitHub user scottyaslan opened a pull request:

https://github.com/apache/nifi/pull/732

[NIFI-2396] Update Bulletin Board shell header and filter control pos…

…ition

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/scottyaslan/nifi responsiveDevBranch

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/732.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #732


commit b0c10b0c6284bac8a03fd52b3030b679ff19fc85
Author: Scott Aslan 
Date:   2016-07-27T20:41:18Z

[NIFI-2396] Update Bulletin Board shell header and filter control position




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2141) Hide bulletin icon on components

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396306#comment-15396306
 ] 

ASF subversion and git services commented on NIFI-2141:
---

Commit f3e49fefa083667fd9638dc38fa23ffa7d4d8ba0 in nifi's branch 
refs/heads/master from [~scottyaslan]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f3e49fe ]

[NIFI-2367] Overlapping links repositioned on error/login pages
[NIFI-2025] update birdseye after dragging/dropping element on the canvas
[NIFI-2367] update width of content viewer combo
[NIFI-2355] update table sorting based on auth efforts
[NIFI-2027] update EL editors
[NIFI-2387] update bulletin alert backgroud color to actually change the color 
of the icon not the div background
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2400] close any open combos contained within a shell when closing the 
shell
[NIFI-2404] remove extra scrollbar from #node-events
[NIFI-2027] account for min widths of EL editors
[NIFI-2025] update birdseye after changing color
[NIFI-2027] Update EL editor checkbox text
[NIFI-2027] update checkbox text
[NIFI-2355] update table sorting
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2027] update EL editors
[NIFI-2367] Overlapping links repositioned on error/login pages
This closes #715


> Hide bulletin icon on components 
> -
>
> Key: NIFI-2141
> URL: https://issues.apache.org/jira/browse/NIFI-2141
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Rob Moran
>Assignee: Scott Aslan
>
> When there is no bulletin for a component to call out, recommend hiding them. 
> It is consistent with current behavior. Having them visible all the time only 
> adds clutter and is a bit distracting on processors.
> When a bulletin does come in, everything should remain in place as it is now. 
> The background-color will change to the warning color (#ba554a) and the 
> bulletin icon will be white.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2337) IllegalMonitorStateException when disconnecting node

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396319#comment-15396319
 ] 

ASF GitHub Bot commented on NIFI-2337:
--

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/731

NIFI-2337: Turn off logging for org.apache.curator.framework.recipes.…

…leader.LeaderSelector since it logs only a single error, and that error is 
a but in Curator

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-2337

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/731.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #731


commit 9e9fce91eef9d3f8aa627407aa5d3677f7691c26
Author: Mark Payne 
Date:   2016-07-27T20:30:44Z

NIFI-2337: Turn off logging for 
org.apache.curator.framework.recipes.leader.LeaderSelector since it logs only a 
single error, and that error is a but in Curator




> IllegalMonitorStateException when disconnecting node
> 
>
> Key: NIFI-2337
> URL: https://issues.apache.org/jira/browse/NIFI-2337
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.0.0
>
>
> When a node is disconnected from the cluster, it often logs a stacktrace 
> similar to:
> 2016-07-20 20:20:47,814 ERROR [Leader Election Notification Thread-1] 
> o.a.c.f.recipes.leader.LeaderSelector The leader threw an exception
> java.lang.IllegalMonitorStateException: You do not own the lock: 
> /leaders/Cluster Coordinator
> at 
> org.apache.curator.framework.recipes.locks.InterProcessMutex.release(InterProcessMutex.java:140)
>  ~[curator-recipes-2.10.0.jar:na]
> at 
> org.apache.curator.framework.recipes.leader.LeaderSelector.doWork(LeaderSelector.java:425)
>  [curator-recipes-2.10.0.jar:na]
> at 
> org.apache.curator.framework.recipes.leader.LeaderSelector.doWorkLoop(LeaderSelector.java:441)
>  [curator-recipes-2.10.0.jar:na]
> at 
> org.apache.curator.framework.recipes.leader.LeaderSelector.access$100(LeaderSelector.java:64)
>  [curator-recipes-2.10.0.jar:na]
> at 
> org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:245)
>  [curator-recipes-2.10.0.jar:na]
> at 
> org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:239)
>  [curator-recipes-2.10.0.jar:na]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [na:1.8.0_74]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_74]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [na:1.8.0_74]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_74]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [na:1.8.0_74]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_74]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>  [na:1.8.0_74]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_74]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_74]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_74]
> This error, I believe, is harmless but is alarming and should be addressed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2025) Birdseye does not update when repositioning an existing item on the canvas

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396301#comment-15396301
 ] 

ASF subversion and git services commented on NIFI-2025:
---

Commit f3e49fefa083667fd9638dc38fa23ffa7d4d8ba0 in nifi's branch 
refs/heads/master from [~scottyaslan]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f3e49fe ]

[NIFI-2367] Overlapping links repositioned on error/login pages
[NIFI-2025] update birdseye after dragging/dropping element on the canvas
[NIFI-2367] update width of content viewer combo
[NIFI-2355] update table sorting based on auth efforts
[NIFI-2027] update EL editors
[NIFI-2387] update bulletin alert backgroud color to actually change the color 
of the icon not the div background
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2400] close any open combos contained within a shell when closing the 
shell
[NIFI-2404] remove extra scrollbar from #node-events
[NIFI-2027] account for min widths of EL editors
[NIFI-2025] update birdseye after changing color
[NIFI-2027] Update EL editor checkbox text
[NIFI-2027] update checkbox text
[NIFI-2355] update table sorting
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2027] update EL editors
[NIFI-2367] Overlapping links repositioned on error/login pages
This closes #715


> Birdseye does not update when repositioning an existing item on the canvas
> --
>
> Key: NIFI-2025
> URL: https://issues.apache.org/jira/browse/NIFI-2025
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Scott Aslan
>Assignee: Scott Aslan
> Fix For: 1.0.0
>
> Attachments: screenshot-1.png, screenshot-2.png
>
>
> Birdseye does not update when repositioning an existing item on the canvas 
> but it does update appropriately when adding/deleting items from the canvas. 
> We need to refresh the birdseye when repositioning items on the canvas.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2337) IllegalMonitorStateException when disconnecting node

[Mark Payne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Payne updated NIFI-2337:
-
Status: Patch Available  (was: Open)

> IllegalMonitorStateException when disconnecting node
> 
>
> Key: NIFI-2337
> URL: https://issues.apache.org/jira/browse/NIFI-2337
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.0.0
>
>
> When a node is disconnected from the cluster, it often logs a stacktrace 
> similar to:
> 2016-07-20 20:20:47,814 ERROR [Leader Election Notification Thread-1] 
> o.a.c.f.recipes.leader.LeaderSelector The leader threw an exception
> java.lang.IllegalMonitorStateException: You do not own the lock: 
> /leaders/Cluster Coordinator
> at 
> org.apache.curator.framework.recipes.locks.InterProcessMutex.release(InterProcessMutex.java:140)
>  ~[curator-recipes-2.10.0.jar:na]
> at 
> org.apache.curator.framework.recipes.leader.LeaderSelector.doWork(LeaderSelector.java:425)
>  [curator-recipes-2.10.0.jar:na]
> at 
> org.apache.curator.framework.recipes.leader.LeaderSelector.doWorkLoop(LeaderSelector.java:441)
>  [curator-recipes-2.10.0.jar:na]
> at 
> org.apache.curator.framework.recipes.leader.LeaderSelector.access$100(LeaderSelector.java:64)
>  [curator-recipes-2.10.0.jar:na]
> at 
> org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:245)
>  [curator-recipes-2.10.0.jar:na]
> at 
> org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:239)
>  [curator-recipes-2.10.0.jar:na]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [na:1.8.0_74]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_74]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [na:1.8.0_74]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_74]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [na:1.8.0_74]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_74]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>  [na:1.8.0_74]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_74]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_74]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_74]
> This error, I believe, is harmless but is alarming and should be addressed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2367) Overlapping text in Cluster pane after error

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396311#comment-15396311
 ] 

ASF GitHub Bot commented on NIFI-2367:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/715


> Overlapping text in Cluster pane after error
> 
>
> Key: NIFI-2367
> URL: https://issues.apache.org/jira/browse/NIFI-2367
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Joseph Percivall
>Assignee: Scott Aslan
> Attachments: Screen Shot 2016-07-22 at 2.42.48 PM.png
>
>
> 3 node secure cluster
> In cluster pane
> hit 500 error when refreshing pane (output in developer tools): 
> https://localhost:8481/nifi-api/controller/cluster Failed to load resource: 
> the server responded with a status of 500 (Internal Server Error)
> See overlapping text of "fix" and "home" in resulting pane (see attached 
> screenshot).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2367) Overlapping text in Cluster pane after error

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396317#comment-15396317
 ] 

ASF subversion and git services commented on NIFI-2367:
---

Commit f3e49fefa083667fd9638dc38fa23ffa7d4d8ba0 in nifi's branch 
refs/heads/master from [~scottyaslan]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f3e49fe ]

[NIFI-2367] Overlapping links repositioned on error/login pages
[NIFI-2025] update birdseye after dragging/dropping element on the canvas
[NIFI-2367] update width of content viewer combo
[NIFI-2355] update table sorting based on auth efforts
[NIFI-2027] update EL editors
[NIFI-2387] update bulletin alert backgroud color to actually change the color 
of the icon not the div background
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2400] close any open combos contained within a shell when closing the 
shell
[NIFI-2404] remove extra scrollbar from #node-events
[NIFI-2027] account for min widths of EL editors
[NIFI-2025] update birdseye after changing color
[NIFI-2027] Update EL editor checkbox text
[NIFI-2027] update checkbox text
[NIFI-2355] update table sorting
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2027] update EL editors
[NIFI-2367] Overlapping links repositioned on error/login pages
This closes #715


> Overlapping text in Cluster pane after error
> 
>
> Key: NIFI-2367
> URL: https://issues.apache.org/jira/browse/NIFI-2367
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Joseph Percivall
>Assignee: Scott Aslan
> Attachments: Screen Shot 2016-07-22 at 2.42.48 PM.png
>
>
> 3 node secure cluster
> In cluster pane
> hit 500 error when refreshing pane (output in developer tools): 
> https://localhost:8481/nifi-api/controller/cluster Failed to load resource: 
> the server responded with a status of 500 (Internal Server Error)
> See overlapping text of "fix" and "home" in resulting pane (see attached 
> screenshot).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2027) Properties Tables

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396304#comment-15396304
 ] 

ASF subversion and git services commented on NIFI-2027:
---

Commit f3e49fefa083667fd9638dc38fa23ffa7d4d8ba0 in nifi's branch 
refs/heads/master from [~scottyaslan]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f3e49fe ]

[NIFI-2367] Overlapping links repositioned on error/login pages
[NIFI-2025] update birdseye after dragging/dropping element on the canvas
[NIFI-2367] update width of content viewer combo
[NIFI-2355] update table sorting based on auth efforts
[NIFI-2027] update EL editors
[NIFI-2387] update bulletin alert backgroud color to actually change the color 
of the icon not the div background
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2400] close any open combos contained within a shell when closing the 
shell
[NIFI-2404] remove extra scrollbar from #node-events
[NIFI-2027] account for min widths of EL editors
[NIFI-2025] update birdseye after changing color
[NIFI-2027] Update EL editor checkbox text
[NIFI-2027] update checkbox text
[NIFI-2355] update table sorting
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2027] update EL editors
[NIFI-2367] Overlapping links repositioned on error/login pages
This closes #715


> Properties Tables
> -
>
> Key: NIFI-2027
> URL: https://issues.apache.org/jira/browse/NIFI-2027
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Scott Aslan
>Assignee: Scott Aslan
> Fix For: 1.0.0
>
> Attachments: screenshot-1.png
>
>
> - Non-EL Property editor, cannot resize horizontally
> - EL Property editor, resize renders too slowly
> - Is empty check box positioning is off
> - window resize event needs to resize any visible properties tables
> - Property table heights are too short
> - GoTo icon is right aligned
> In the property editing dialog, if it is resized very small, the layout of 
> the OK and cancel buttons get misaligned, and the OK button ends up covering 
> the resize widget. If there's no text in the box, a very small part of the 
> widget is still clickable and usable. Once there's text in the box, the user 
> is unable to click on the resize widget.
> I added a ListFile Processor. Clicked Configure, switched to Properties tab. 
> Clicked to change the value of the "Input Directory" property. When I placed 
> the mouse over the "Cancel" button, the mouse cursor changes to the 'Move' 
> cursor instead of the 'pointer' cursor



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2367) Overlapping text in Cluster pane after error

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396318#comment-15396318
 ] 

ASF GitHub Bot commented on NIFI-2367:
--

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/715
  
Looks great @scottyaslan! This has been merged to master. Thanks!


> Overlapping text in Cluster pane after error
> 
>
> Key: NIFI-2367
> URL: https://issues.apache.org/jira/browse/NIFI-2367
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Joseph Percivall
>Assignee: Scott Aslan
> Attachments: Screen Shot 2016-07-22 at 2.42.48 PM.png
>
>
> 3 node secure cluster
> In cluster pane
> hit 500 error when refreshing pane (output in developer tools): 
> https://localhost:8481/nifi-api/controller/cluster Failed to load resource: 
> the server responded with a status of 500 (Internal Server Error)
> See overlapping text of "fix" and "home" in resulting pane (see attached 
> screenshot).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2141) Hide bulletin icon on components

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396315#comment-15396315
 ] 

ASF subversion and git services commented on NIFI-2141:
---

Commit f3e49fefa083667fd9638dc38fa23ffa7d4d8ba0 in nifi's branch 
refs/heads/master from [~scottyaslan]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f3e49fe ]

[NIFI-2367] Overlapping links repositioned on error/login pages
[NIFI-2025] update birdseye after dragging/dropping element on the canvas
[NIFI-2367] update width of content viewer combo
[NIFI-2355] update table sorting based on auth efforts
[NIFI-2027] update EL editors
[NIFI-2387] update bulletin alert backgroud color to actually change the color 
of the icon not the div background
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2400] close any open combos contained within a shell when closing the 
shell
[NIFI-2404] remove extra scrollbar from #node-events
[NIFI-2027] account for min widths of EL editors
[NIFI-2025] update birdseye after changing color
[NIFI-2027] Update EL editor checkbox text
[NIFI-2027] update checkbox text
[NIFI-2355] update table sorting
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2027] update EL editors
[NIFI-2367] Overlapping links repositioned on error/login pages
This closes #715


> Hide bulletin icon on components 
> -
>
> Key: NIFI-2141
> URL: https://issues.apache.org/jira/browse/NIFI-2141
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Rob Moran
>Assignee: Scott Aslan
>
> When there is no bulletin for a component to call out, recommend hiding them. 
> It is consistent with current behavior. Having them visible all the time only 
> adds clutter and is a bit distracting on processors.
> When a bulletin does come in, everything should remain in place as it is now. 
> The background-color will change to the warning color (#ba554a) and the 
> bulletin icon will be white.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2355) Unable to order by any column when viewing Controller Services

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396314#comment-15396314
 ] 

ASF subversion and git services commented on NIFI-2355:
---

Commit f3e49fefa083667fd9638dc38fa23ffa7d4d8ba0 in nifi's branch 
refs/heads/master from [~scottyaslan]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f3e49fe ]

[NIFI-2367] Overlapping links repositioned on error/login pages
[NIFI-2025] update birdseye after dragging/dropping element on the canvas
[NIFI-2367] update width of content viewer combo
[NIFI-2355] update table sorting based on auth efforts
[NIFI-2027] update EL editors
[NIFI-2387] update bulletin alert backgroud color to actually change the color 
of the icon not the div background
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2400] close any open combos contained within a shell when closing the 
shell
[NIFI-2404] remove extra scrollbar from #node-events
[NIFI-2027] account for min widths of EL editors
[NIFI-2025] update birdseye after changing color
[NIFI-2027] Update EL editor checkbox text
[NIFI-2027] update checkbox text
[NIFI-2355] update table sorting
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2027] update EL editors
[NIFI-2367] Overlapping links repositioned on error/login pages
This closes #715


> Unable to order by any column when viewing Controller Services
> --
>
> Key: NIFI-2355
> URL: https://issues.apache.org/jira/browse/NIFI-2355
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Joseph Percivall
>Assignee: Scott Aslan
>
> Have multiple controller services in a process group
> attempt to order by a column by clicking it
> nothing happens (see arrow flip)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2025) Birdseye does not update when repositioning an existing item on the canvas

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396310#comment-15396310
 ] 

ASF subversion and git services commented on NIFI-2025:
---

Commit f3e49fefa083667fd9638dc38fa23ffa7d4d8ba0 in nifi's branch 
refs/heads/master from [~scottyaslan]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f3e49fe ]

[NIFI-2367] Overlapping links repositioned on error/login pages
[NIFI-2025] update birdseye after dragging/dropping element on the canvas
[NIFI-2367] update width of content viewer combo
[NIFI-2355] update table sorting based on auth efforts
[NIFI-2027] update EL editors
[NIFI-2387] update bulletin alert backgroud color to actually change the color 
of the icon not the div background
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2400] close any open combos contained within a shell when closing the 
shell
[NIFI-2404] remove extra scrollbar from #node-events
[NIFI-2027] account for min widths of EL editors
[NIFI-2025] update birdseye after changing color
[NIFI-2027] Update EL editor checkbox text
[NIFI-2027] update checkbox text
[NIFI-2355] update table sorting
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2027] update EL editors
[NIFI-2367] Overlapping links repositioned on error/login pages
This closes #715


> Birdseye does not update when repositioning an existing item on the canvas
> --
>
> Key: NIFI-2025
> URL: https://issues.apache.org/jira/browse/NIFI-2025
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Scott Aslan
>Assignee: Scott Aslan
> Fix For: 1.0.0
>
> Attachments: screenshot-1.png, screenshot-2.png
>
>
> Birdseye does not update when repositioning an existing item on the canvas 
> but it does update appropriately when adding/deleting items from the canvas. 
> We need to refresh the birdseye when repositioning items on the canvas.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2027) Properties Tables

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396312#comment-15396312
 ] 

ASF subversion and git services commented on NIFI-2027:
---

Commit f3e49fefa083667fd9638dc38fa23ffa7d4d8ba0 in nifi's branch 
refs/heads/master from [~scottyaslan]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f3e49fe ]

[NIFI-2367] Overlapping links repositioned on error/login pages
[NIFI-2025] update birdseye after dragging/dropping element on the canvas
[NIFI-2367] update width of content viewer combo
[NIFI-2355] update table sorting based on auth efforts
[NIFI-2027] update EL editors
[NIFI-2387] update bulletin alert backgroud color to actually change the color 
of the icon not the div background
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2400] close any open combos contained within a shell when closing the 
shell
[NIFI-2404] remove extra scrollbar from #node-events
[NIFI-2027] account for min widths of EL editors
[NIFI-2025] update birdseye after changing color
[NIFI-2027] Update EL editor checkbox text
[NIFI-2027] update checkbox text
[NIFI-2355] update table sorting
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2027] update EL editors
[NIFI-2367] Overlapping links repositioned on error/login pages
This closes #715


> Properties Tables
> -
>
> Key: NIFI-2027
> URL: https://issues.apache.org/jira/browse/NIFI-2027
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Scott Aslan
>Assignee: Scott Aslan
> Fix For: 1.0.0
>
> Attachments: screenshot-1.png
>
>
> - Non-EL Property editor, cannot resize horizontally
> - EL Property editor, resize renders too slowly
> - Is empty check box positioning is off
> - window resize event needs to resize any visible properties tables
> - Property table heights are too short
> - GoTo icon is right aligned
> In the property editing dialog, if it is resized very small, the layout of 
> the OK and cancel buttons get misaligned, and the OK button ends up covering 
> the resize widget. If there's no text in the box, a very small part of the 
> widget is still clickable and usable. Once there's text in the box, the user 
> is unable to click on the resize widget.
> I added a ListFile Processor. Clicked Configure, switched to Properties tab. 
> Clicked to change the value of the "Input Directory" property. When I placed 
> the mouse over the "Cancel" button, the mouse cursor changes to the 'Move' 
> cursor instead of the 'pointer' cursor



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2355) Unable to order by any column when viewing Controller Services

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396303#comment-15396303
 ] 

ASF subversion and git services commented on NIFI-2355:
---

Commit f3e49fefa083667fd9638dc38fa23ffa7d4d8ba0 in nifi's branch 
refs/heads/master from [~scottyaslan]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f3e49fe ]

[NIFI-2367] Overlapping links repositioned on error/login pages
[NIFI-2025] update birdseye after dragging/dropping element on the canvas
[NIFI-2367] update width of content viewer combo
[NIFI-2355] update table sorting based on auth efforts
[NIFI-2027] update EL editors
[NIFI-2387] update bulletin alert backgroud color to actually change the color 
of the icon not the div background
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2400] close any open combos contained within a shell when closing the 
shell
[NIFI-2404] remove extra scrollbar from #node-events
[NIFI-2027] account for min widths of EL editors
[NIFI-2025] update birdseye after changing color
[NIFI-2027] Update EL editor checkbox text
[NIFI-2027] update checkbox text
[NIFI-2355] update table sorting
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2027] update EL editors
[NIFI-2367] Overlapping links repositioned on error/login pages
This closes #715


> Unable to order by any column when viewing Controller Services
> --
>
> Key: NIFI-2355
> URL: https://issues.apache.org/jira/browse/NIFI-2355
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Joseph Percivall
>Assignee: Scott Aslan
>
> Have multiple controller services in a process group
> attempt to order by a column by clicking it
> nothing happens (see arrow flip)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2404) In the node details pop-up of the cluster panel, there are multiple scroll bars when node events get too long

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396308#comment-15396308
 ] 

ASF subversion and git services commented on NIFI-2404:
---

Commit f3e49fefa083667fd9638dc38fa23ffa7d4d8ba0 in nifi's branch 
refs/heads/master from [~scottyaslan]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f3e49fe ]

[NIFI-2367] Overlapping links repositioned on error/login pages
[NIFI-2025] update birdseye after dragging/dropping element on the canvas
[NIFI-2367] update width of content viewer combo
[NIFI-2355] update table sorting based on auth efforts
[NIFI-2027] update EL editors
[NIFI-2387] update bulletin alert backgroud color to actually change the color 
of the icon not the div background
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2400] close any open combos contained within a shell when closing the 
shell
[NIFI-2404] remove extra scrollbar from #node-events
[NIFI-2027] account for min widths of EL editors
[NIFI-2025] update birdseye after changing color
[NIFI-2027] Update EL editor checkbox text
[NIFI-2027] update checkbox text
[NIFI-2355] update table sorting
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2027] update EL editors
[NIFI-2367] Overlapping links repositioned on error/login pages
This closes #715


> In the node details pop-up of the cluster panel, there are multiple scroll 
> bars when node events get too long
> -
>
> Key: NIFI-2404
> URL: https://issues.apache.org/jira/browse/NIFI-2404
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Joseph Percivall
>Assignee: Scott Aslan
>Priority: Trivial
> Attachments: Screen Shot 2016-07-26 at 11.28.43 AM.png
>
>
> Depending on the size of the window, there is sometimes a second scroll bar 
> on the node details pop-up scrolling all the contents when there is plenty of 
> room for the whole pop-up.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2367) Overlapping text in Cluster pane after error

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396300#comment-15396300
 ] 

ASF subversion and git services commented on NIFI-2367:
---

Commit f3e49fefa083667fd9638dc38fa23ffa7d4d8ba0 in nifi's branch 
refs/heads/master from [~scottyaslan]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f3e49fe ]

[NIFI-2367] Overlapping links repositioned on error/login pages
[NIFI-2025] update birdseye after dragging/dropping element on the canvas
[NIFI-2367] update width of content viewer combo
[NIFI-2355] update table sorting based on auth efforts
[NIFI-2027] update EL editors
[NIFI-2387] update bulletin alert backgroud color to actually change the color 
of the icon not the div background
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2400] close any open combos contained within a shell when closing the 
shell
[NIFI-2404] remove extra scrollbar from #node-events
[NIFI-2027] account for min widths of EL editors
[NIFI-2025] update birdseye after changing color
[NIFI-2027] Update EL editor checkbox text
[NIFI-2027] update checkbox text
[NIFI-2355] update table sorting
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2027] update EL editors
[NIFI-2367] Overlapping links repositioned on error/login pages
This closes #715


> Overlapping text in Cluster pane after error
> 
>
> Key: NIFI-2367
> URL: https://issues.apache.org/jira/browse/NIFI-2367
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Reporter: Joseph Percivall
>Assignee: Scott Aslan
> Attachments: Screen Shot 2016-07-22 at 2.42.48 PM.png
>
>
> 3 node secure cluster
> In cluster pane
> hit 500 error when refreshing pane (output in developer tools): 
> https://localhost:8481/nifi-api/controller/cluster Failed to load resource: 
> the server responded with a status of 500 (Internal Server Error)
> See overlapping text of "fix" and "home" in resulting pane (see attached 
> screenshot).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #715: [NIFI-2367] [NIFI-2025] [NIFI-2107] [NIFI-2355] [NIFI-2027]...

[mcgilman]

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/715
  
Looks great @scottyaslan! This has been merged to master. Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #731: NIFI-2337: Turn off logging for org.apache.curator.f...

[markap14]

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/731

NIFI-2337: Turn off logging for org.apache.curator.framework.recipes.…

…leader.LeaderSelector since it logs only a single error, and that error 
is a but in Curator

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-2337

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/731.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #731


commit 9e9fce91eef9d3f8aa627407aa5d3677f7691c26
Author: Mark Payne 
Date:   2016-07-27T20:30:44Z

NIFI-2337: Turn off logging for 
org.apache.curator.framework.recipes.leader.LeaderSelector since it logs only a 
single error, and that error is a but in Curator




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #715: [NIFI-2367] [NIFI-2025] [NIFI-2107] [NIFI-2355] [NIF...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/715


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-2401) Unable to hit start/stop with 2 processors and connection on canvas

[Matt Gilman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-2401:
--
Fix Version/s: 1.0.0

> Unable to hit start/stop with 2 processors and connection on canvas
> ---
>
> Key: NIFI-2401
> URL: https://issues.apache.org/jira/browse/NIFI-2401
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Percivall
>Assignee: Matt Gilman
> Fix For: 1.0.0
>
>
> secure 3 node cluster
> have modify/view privileges to PG (and processors inherited)
> have the PG "selected"
> unable to hit start/stop in palette
> click and drag to select all 3 (two processors and connection)
> unable to hit start/stop in palette or context menu
> select only the two processors
> able to hit start/stop in palette and context menu



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-2414) Proxy Headers

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-2414:
-

 Summary: Proxy Headers
 Key: NIFI-2414
 URL: https://issues.apache.org/jira/browse/NIFI-2414
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Matt Gilman
Priority: Blocker
 Fix For: 1.0.0


Proxy headers need to be set by the node that receives the request and not by 
the request replicator. The proxy headers will be used to form URIs for the 
components and must match the address the end user is using to interact with 
NiFi.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #725: Addressing issues surrounding Template uploading

[JPercivall]

Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/725
  
+1

Visually verified code and did a contrib check build. In both standalone 
and 3 node secure cluster, I tested all 5 situations and various other checks 
relating to templates. Thanks @mcgilman, I will merge it in.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2412) Allow user to set serializer and de serializer in publish and consume kafka processors

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396208#comment-15396208
 ] 

Joseph Witt commented on NIFI-2412:
---

[~ozhurakousky] If you have a PR then hopefully someone can review it.  I'd be 
happy to do so, for example.

> Allow user to set serializer and de serializer in publish and consume kafka 
> processors
> --
>
> Key: NIFI-2412
> URL: https://issues.apache.org/jira/browse/NIFI-2412
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 0.7.0
>Reporter: Arpit Gupta
>Assignee: Oleg Zhurakousky
>
> Publish kafka today sets key and value serializer to class 
> org.apache.kafka.common.serialization.ByteArraySerializer
> Even if the user passes different values in the processor config they get 
> overwritten.
> We should allow dynamic values for serializer and de serializer as needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #725: Addressing issues surrounding Template uploading

[mcgilman]

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/725
  
Good catch and great suggestion. I've updated the PR accordingly. Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-2413) Replicated request error messages include node information for both cluster coordinator and responding node.

[Joseph Percivall (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Percivall updated NIFI-2413:
---
Attachment: Screen Shot 2016-07-27 at 2.36.39 PM.png

Screenshot of error message and word wrapping attached.

> Replicated request error messages include node information for both cluster 
> coordinator and responding node.
> 
>
> Key: NIFI-2413
> URL: https://issues.apache.org/jira/browse/NIFI-2413
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Percivall
>Priority: Minor
> Fix For: 1.1.0
>
> Attachments: Screen Shot 2016-07-27 at 2.36.39 PM.png
>
>
> While testing pr/725 I encountered this:
> 3 node secure cluster
> upload template with no name (currently is a server side check)
> Encounter a error pop-up saying:
> "Node localhost:8481 is unable to fulfill this request due to: Node 
> localhost:8482 is unable to fulfill this request due to: Template name cannot 
> be blank."
> To a user this confusing as to which node is actually stopping the request. 
> Also the way that the word wrapping happens it makes it look ambiguous on 
> whether it's both nodes reporting the same message.
> https://github.com/apache/nifi/pull/725



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-2413) Replicated request error messages include node information for both cluster coordinator and responding node.

[Joseph Percivall (JIRA)]

Joseph Percivall created NIFI-2413:
--

 Summary: Replicated request error messages include node 
information for both cluster coordinator and responding node.
 Key: NIFI-2413
 URL: https://issues.apache.org/jira/browse/NIFI-2413
 Project: Apache NiFi
  Issue Type: Bug
Reporter: Joseph Percivall
Priority: Minor
 Fix For: 1.1.0
 Attachments: Screen Shot 2016-07-27 at 2.36.39 PM.png

While testing pr/725 I encountered this:
3 node secure cluster
upload template with no name (currently is a server side check)
Encounter a error pop-up saying:

"Node localhost:8481 is unable to fulfill this request due to: Node 
localhost:8482 is unable to fulfill this request due to: Template name cannot 
be blank."

To a user this confusing as to which node is actually stopping the request. 
Also the way that the word wrapping happens it makes it look ambiguous on 
whether it's both nodes reporting the same message.

https://github.com/apache/nifi/pull/725



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #725: Addressing issues surrounding Template uploading

[JPercivall]

Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/725
  
Empty name template checking appears to only occur as a server side check. 
This would be a simple client side check to add and would be a nice UX addition.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #725: Addressing issues surrounding Template uploading

[JPercivall]

Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/725
  
3 node secure cluster testing NIFI-2223, I see "conflict" and not "A 
template named 'XYZ' already exists."

![screen shot 2016-07-27 at 2 17 59 
pm](https://cloud.githubusercontent.com/assets/11302527/17187001/75c83fdc-5405-11e6-8649-98c4dfac5b69.png)



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2223) Able to upload same template multiple times

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396103#comment-15396103
 ] 

ASF GitHub Bot commented on NIFI-2223:
--

Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/725
  
3 node secure cluster testing NIFI-2223, I see "conflict" and not "A 
template named 'XYZ' already exists."

![screen shot 2016-07-27 at 2 17 59 
pm](https://cloud.githubusercontent.com/assets/11302527/17187001/75c83fdc-5405-11e6-8649-98c4dfac5b69.png)



> Able to upload same template multiple times
> ---
>
> Key: NIFI-2223
> URL: https://issues.apache.org/jira/browse/NIFI-2223
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core UI
>Affects Versions: 1.0.0
> Environment: Mac OS X; Chrome
>Reporter: Andrew Lim
>Assignee: Matt Gilman
>  Labels: UI
> Fix For: 1.0.0
>
> Attachments: NIFI-2223_ErrorMessage.png, NIFI-2223_sameTemp3X.png
>
>
> UI allows the same template to be uploaded multiple times as shown in the 
> attached screenshot.
> In 0.x, this was not allowed and the error message "A templated named 'XYZ' 
> already exists."  Screenshot attached for this also.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2193) Command Line Keystore and Truststore utility

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396057#comment-15396057
 ] 

ASF GitHub Bot commented on NIFI-2193:
--

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/695#discussion_r72488748
  
--- Diff: nifi-toolkit/nifi-toolkit-assembly/NOTICE ---
@@ -0,0 +1,112 @@
+Apache NiFi Toolkit
+Copyright 2014-2016 The Apache Software Foundation
+
+This product includes software developed at
+The Apache Software Foundation (http://www.apache.org/).
+
+===
+Apache Software License v2
+===
+
+The following binary components are provided under the Apache Software 
License v2
+
+  (ASLv2) Apache NiFi
+The following NOTICE information applies:
+  Apache NiFi
+  Copyright 2014-2016 The Apache Software Foundation
+
+  This product includes software developed at
+  The Apache Software Foundation (http://www.apache.org/).
+
+  This product includes the following work from the Apache Hadoop 
project:
+
+  BoundedByteArrayOutputStream.java adapted to 
SoftLimitBoundedByteArrayOutputStream.java
+
+  (ASLv2) Apache Commons CLI
+The following NOTICE information applies:
+  Apache Commons IO
+  Copyright 2001-2015 The Apache Software Foundation
+
+  (ASLv2) Apache Commons Codec
+The following NOTICE information applies:
+  Apache Commons Codec
+  Copyright 2002-2014 The Apache Software Foundation
+
+  src/test/org/apache/commons/codec/language/DoubleMetaphoneTest.java
+  contains test data from http://aspell.net/test/orig/batch0.tab.
+  Copyright (C) 2002 Kevin Atkinson (kev...@gnu.org)
+
+  
===
+
+  The content of package org.apache.commons.codec.language.bm has been 
translated
+  from the original php source code available at 
http://stevemorse.org/phoneticinfo.htm
+  with permission from the original authors.
+  Original source copyright:
+  Copyright (c) 2008 Alexander Beider & Stephen P. Morse.
+
+  (ASLv2) Apache Commons IO
+The following NOTICE information applies:
+  Apache Commons IO
+  Copyright 2002-2012 The Apache Software Foundation
+
+  (ASLv2) Apache Commons Lang
+The following NOTICE information applies:
+  Apache Commons Lang
+  Copyright 2001-2015 The Apache Software Foundation
+
+  (ASLv2) Apache Commons Logging
--- End diff --

Right it contains all the license and notice information for all the direct 
dependencies that make their way into the distribution. It is on those 
dependencies to make sure their license and notice files accurately reflect the 
dependencies they bundle and it is on us to properly reflect those license and 
notice files in our own files.

This way we don't have to drill all the way down into all transitive deps 
of all the direct deps we import, we just have to reflect the license and 
notice information of the direct deps.


> Command Line Keystore and Truststore utility
> 
>
> Key: NIFI-2193
> URL: https://issues.apache.org/jira/browse/NIFI-2193
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Bryan Rosander
>Assignee: Bryan Rosander
>
> In order to facilitate secure setup of NiFi, it would be useful to have a 
> command line utility capable of generating the required keystores, 
> truststore, and relevant configuration files.
> It should be able to generate keystores for each NiFi node, a truststore that 
> they all use, and relevant passwords and configuration files for using the 
> keystores and truststore.
> Additionally, in order to support distributed deployment, a web based 
> certificate authority with corresponding client will allow for each NiFi 
> instance to generate its own keypair and then request signing by the CA.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #695: NIFI-2193 - Command line SSL config utility as well ...

[brosander]

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi/pull/695#discussion_r72488752
  
--- Diff: nifi-assembly/pom.xml ---
@@ -363,147 +363,6 @@ language governing permissions and limitations under 
the License. -->
 
 
 
-
--- End diff --

@JPercivall still seeing a resolved nifi.properties in the nifi-assembly 
output


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---