[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-20 Thread jfrazee
Github user jfrazee commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50314068 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/pom.xml --- @@ -0,0 +1,80 @@ + + +http://maven.apache.org/POM/4.

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-20 Thread mattyb149
Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50314672 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/pom.xml --- @@ -0,0 +1,80 @@ + + +http://maven.apache.org/POM/

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-20 Thread apiri
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50337656 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/pom.xml --- @@ -0,0 +1,80 @@ + + +http://maven.apache.org/POM/4.0.

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-20 Thread mattyb149
Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50339290 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/pom.xml --- @@ -0,0 +1,80 @@ + + +http://maven.apache.org/POM/

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-21 Thread trixpan
Github user trixpan commented on the pull request: https://github.com/apache/nifi/pull/180#issuecomment-173737898 Is it my impression or this controller is embedding elastic search into NiFi in order to use the Transport protocol? Would this cause all sort of issues around ver

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-21 Thread mattyb149
Github user mattyb149 commented on the pull request: https://github.com/apache/nifi/pull/180#issuecomment-173740429 Yes this controller is using the Transport protocol. The explicit docs you mention are logstash docs, not elasticsearch docs. The Elasticsearch Java Client docs (https:

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-21 Thread trixpan
Github user trixpan commented on the pull request: https://github.com/apache/nifi/pull/180#issuecomment-173761891 @mattyb149 very good point... very good point Indeed and according to a blog post on found.no(Elastic's cloud offering) back in January suggested the use of transp

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-21 Thread mattyb149
Github user mattyb149 commented on the pull request: https://github.com/apache/nifi/pull/180#issuecomment-173769666 You are right that more processing would be needed to coax the flow file into the right format for the *Http processors. If the ES processors were to use the HTTP API th

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-21 Thread trixpan
Github user trixpan commented on the pull request: https://github.com/apache/nifi/pull/180#issuecomment-173770694 sounds like a great idea --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-21 Thread sejasp
Github user sejasp commented on the pull request: https://github.com/apache/nifi/pull/180#issuecomment-173806065 PostHTTP has been working like a charm. However, GetElasticsearch would be a great addition to the "Get" processors. Preferably, scrolling. --- If your project is set up

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread JPercivall
Github user JPercivall commented on the pull request: https://github.com/apache/nifi/pull/180#issuecomment-175086276 Could the unit tests be fleshed out a bit? The core method in AbstractElasticsearchProcessor (createClient) doesn't have any coverage and there are only two general tes

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread markap14
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50871641 --- Diff: nifi-assembly/LICENSE --- @@ -1085,29 +1085,3 @@ information can be found here: http://www.adobe.com/devnet/xmp/library/eula-xmp- LIABILITY,

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread markap14
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50872379 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/AbstractElasticsearchProc

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread markap14
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50872442 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/AbstractElasticsearchProc

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread markap14
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50872609 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/AbstractElasticsearchProc

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread markap14
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50872655 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/AbstractElasticsearchProc

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread mattyb149
Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50872712 --- Diff: nifi-assembly/LICENSE --- @@ -1085,29 +1085,3 @@ information can be found here: http://www.adobe.com/devnet/xmp/library/eula-xmp- LIABILITY,

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread markap14
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50873154 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearch.java

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread markap14
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50872992 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearch.java

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread markap14
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50873286 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearch.java

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread markap14
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50873578 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearch.java

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread markap14
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50874007 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearch.java --

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread markap14
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50878568 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearch.java --

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread markap14
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50878801 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearch.java --

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread markap14
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50879053 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearch.java --

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread markap14
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50879307 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearch.java --

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread markap14
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50879789 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearch.java --

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread markap14
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r50879510 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearch.java --

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-26 Thread markap14
Github user markap14 commented on the pull request: https://github.com/apache/nifi/pull/180#issuecomment-175195734 @mattyb149 - one last comment on here - not sure about the licensing of that TweetExample.json file - is that a legitimate JSON that was pulled from Twitter? It may have

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-28 Thread mattyb149
Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r51127270 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearch.java

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-28 Thread mattyb149
Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r51129036 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearch.java

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-28 Thread mattyb149
Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r51129452 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearch.java -

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-28 Thread mattyb149
Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r51130226 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearch.java -

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-28 Thread markap14
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r51130181 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearch.java

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-28 Thread markap14
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r51130380 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearch.java --

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-28 Thread mattyb149
Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r51132629 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearch.java

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-28 Thread JPercivall
Github user JPercivall commented on the pull request: https://github.com/apache/nifi/pull/180#issuecomment-176433804 In the latest commit there is more code coverage but there are still only a couple tests. Ideally I would like to see tests that work with each of the configuration pro

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-30 Thread rpmiskin
Github user rpmiskin commented on the pull request: https://github.com/apache/nifi/pull/180#issuecomment-177303999 @mattyb149 I've just been trying this out and it seems good, but you might need to look at the error reporting as I'm getting error messages like: `2016-01-30 17:

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-30 Thread mattyb149
Github user mattyb149 commented on the pull request: https://github.com/apache/nifi/pull/180#issuecomment-177304480 @rpmiskin Good catch, thanks! I will make the change and update the PR (with more tests per the other comments, etc.) shortly. --- If your project is set up for it, you

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-30 Thread rpmiskin
Github user rpmiskin commented on the pull request: https://github.com/apache/nifi/pull/180#issuecomment-177421946 @mattyb149 No worries - I seem to be quite good at misconfiguring flows to produce errors... ;) I spent a little time yesterday putting together a proof of c

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-31 Thread mattyb149
Github user mattyb149 commented on the pull request: https://github.com/apache/nifi/pull/180#issuecomment-177632224 @rpmiskin That's a good idea for the ES ControllerService and ReportingTask, I hadn't considered the ReportingTask so I didn't think there would be enough need for the C

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-01-31 Thread rpmiskin
Github user rpmiskin commented on the pull request: https://github.com/apache/nifi/pull/180#issuecomment-177781101 @mattyb149 Seems fine to me for a ControllerService refactor to be a subsequent change provided there is a clear upgrade mechanism. Doing that would allow time fo

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-02-01 Thread JPercivall
Github user JPercivall commented on the pull request: https://github.com/apache/nifi/pull/180#issuecomment-178023848 If the "Identifier Attribute" is being left as the attribute containing the identifier then the "ATTRIBUTE_KEY_VALIDATOR" should be used instead of "NON_EMPTY_VALIDATOR

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-02-01 Thread JPercivall
Github user JPercivall commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r51482142 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/AbstractElasticsearchPr

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-02-01 Thread JPercivall
Github user JPercivall commented on the pull request: https://github.com/apache/nifi/pull/180#issuecomment-178198337 There needs to be support added for passing username/password (specifically for shield integration) --- If your project is set up for it, you can reply to this email a

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-02-01 Thread JPercivall
Github user JPercivall commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r51483875 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/AbstractElasticsearchPr

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-02-01 Thread JPercivall
Github user JPercivall commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r51490929 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/AbstractElasticsearchPr

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

2016-02-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/180 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable