[GitHub] nifi issue #1233: NIFI-3011: Added Elasticsearch5 processors

2016-11-21 Thread JPercivall
Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/1233
  
+1

Visually verified code and did a contrib check build. Ran a stand-alone 
instance hitting an ES 5 cluster with and without the security x-pack. All 
comments and L&N changes have been addressed. Thanks @mattyb149 I will squash 
and merge.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] nifi issue #1233: NIFI-3011: Added Elasticsearch5 processors

2016-11-21 Thread JPercivall
Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/1233
  
@mattyb149 ran into two weird errors while testing the lastest update. 

First, I ran into this when I attempted to do a request with the wrong 
attribute name (and thus there were no actions added to the "bulk":
```
016-11-21 13:40:02,766 ERROR [Timer-Driven Process Thread-10] 
o.a.n.p.elasticsearch.PutElasticsearch5 
org.elasticsearch.action.ActionRequestValidationException: Validation 
Failed: 1: no requests added;
at 
org.elasticsearch.action.ValidateActions.addValidationError(ValidateActions.java:29)
 ~[elasticsearch-5.0.1.jar:5.0.1]
at 
org.elasticsearch.action.bulk.BulkRequest.validate(BulkRequest.java:530) 
~[elasticsearch-5.0.1.jar:5.0.1]
at 
org.elasticsearch.action.TransportActionNodeProxy.execute(TransportActionNodeProxy.java:46)
 ~[elasticsearch-5.0.1.jar:5.0.1]
at 
org.elasticsearch.client.transport.support.TransportProxyClient.lambda$execute$0(TransportProxyClient.java:63)
 ~[elasticsearch-5.0.1.jar:5.0.1]
at 
org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:233)
 ~[elasticsearch-5.0.1.jar:5.0.1]
at 
org.elasticsearch.client.transport.support.TransportProxyClient.execute(TransportProxyClient.java:63)
 ~[elasticsearch-5.0.1.jar:5.0.1]
at 
org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:326)
 ~[elasticsearch-5.0.1.jar:5.0.1]
at 
org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:403)
 ~[elasticsearch-5.0.1.jar:5.0.1]
at 
org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:67) 
~[elasticsearch-5.0.1.jar:5.0.1]
at 
org.elasticsearch.client.support.AbstractClient$1.doExecute(AbstractClient.java:1754)
 ~[elasticsearch-5.0.1.jar:5.0.1]
at 
org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:403)
 ~[elasticsearch-5.0.1.jar:5.0.1]
at 
org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:80)
 ~[elasticsearch-5.0.1.jar:5.0.1]
at 
org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:54)
 ~[elasticsearch-5.0.1.jar:5.0.1]
at 
org.apache.nifi.processors.elasticsearch.PutElasticsearch5.onTrigger(PutElasticsearch5.java:230)
 ~[nifi-elasticsearch-5-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at 
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
 [nifi-api-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at 
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1089)
 [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at 
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
 [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at 
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
 [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at 
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
 [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[na:1.8.0_74]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
[na:1.8.0_74]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
 [na:1.8.0_74]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
 [na:1.8.0_74]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[na:1.8.0_74]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_74]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_74]
```


Second a bit more concerning, I saw when I tried to "update" a group of 
flowfile. All of which didn't exist already:
```

 PutElasticsearch5[id=b1fc4435-e5c1-153e-b5dc-61b9af32ef03] Failed to 
insert into Elasticsearch due to Index: 50, Size: 50, transferring to failure: 
java.lang.IndexOutOfBoundsException: Index: 50, Size: 50
2016-11-21 13:50:23,985 ERROR [Timer-Driven Process Thread-10] 
o.a.n.p.elasticsearch.PutElasticsearch5 
java.lang.IndexOutOfBoundsException: Index: 50, Size: 50
at java.util.LinkedList.checkElementIndex(LinkedList.java:555) 
~[na:1.8.0_74]
at java.util.LinkedList.get(LinkedList.java:476) ~[na:1.8.0_74]
at 
org.apache.nifi.processors.elasticsearch.PutElasticsearch5.onTrigger(PutElasticsearch5.java:233)
 ~[nifi-elasticsearch-5-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at 
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:2

[GitHub] nifi issue #1233: NIFI-3011: Added Elasticsearch5 processors

2016-11-16 Thread JPercivall
Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/1233
  
I just confirmed functionality with ES 5.0.1 with and without x-pack 
security. Once the issues commented are addressed it should be good to go.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] nifi issue #1233: NIFI-3011: Added Elasticsearch5 processors

2016-11-16 Thread JPercivall
Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/1233
  
Also, both processors do not emit any provenance events. This should be 
added.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---