Re: Github mails

[Lars Francke]

Joe, thank you for following up on all of this and the others for chiming
in!

Sorry for dumping this idea here and then not responding to the thread for
a week so again thanks for this!

On Sat, Jul 2, 2016 at 12:55 AM, Joe Witt  wrote:

> Ok team I put in a request for 'issues@nifi.a.o' and updated
> https://issues.apache.org/jira/browse/INFRA-12191# to reflect our
> plans.  Will now edit the website to document the purpose of each
> list.
>
> Thanks
> Joe
>
> On Wed, Jun 29, 2016 at 5:06 AM, Joe Skora  wrote:
> > +1
> >
> > On Tue, Jun 28, 2016 at 12:37 PM, Joe Witt  wrote:
> >
> >> Sean,
> >>
> >> Make sense and I believe it aligns with Lars original intent and
> >> suggestion.
> >>
> >> I've captured these items here
> >> https://issues.apache.org/jira/browse/NIFI-2137
> >>
> >> But in summary will carry out the following tasks pending any further
> >> recommendations that might alter that.
> >>
> >> 1) Request INFRA to create issues@nifi.a.o
> >> 2) Request INFRA to send all Github notifications to issues@nifi.a.o
> >> 3) Request INFRA to send all JIRA notifications to issues@nifi.a.o
> >> 4) Document on webpage for mailing lists the purpose of each mailing
> >> list (dev, commits, issues, users)
> >> 5) Document on webpage for mailing lists a link to
> >> https://lists.apache.org/list.html?nifi.apache.org
> >>
> >> Thanks
> >> Joe
> >>
> >> On Tue, Jun 28, 2016 at 8:17 AM, Sean Busbey  wrote:
> >> > My usual concern with sending github notifications off of dev@ is
> that
> >> > not as many folks watch lists that are defined as a dumping ground for
> >> > automated messages (like a commits@ or notifications@).
> >> >
> >> > However, I usually think of github notifications like JIRA
> >> > notifications. AFAICT we already send all of the JIRA traffic to
> >> > commits@, so it makes sense for folks who want that level of detail
> on
> >> > project implementation to have a single place to go.
> >> >
> >> > At the same time, it might be hard for a new person to realize all
> >> > these things are going to commits@. Maybe we could add a brief
> >> > description to our mailing list page[1] of when folks should look to
> >> > the different lists?
> >> >
> >> > There's enough project activity now on jira/github/commits that I'd be
> >> > in favor of splitting out things that aren't actually commits to a
> >> > notifications@ (or issues@ or whatever).
> >> >
> >> > Also, now that lists.apache is live, I think we should link to it as
> >> > the "user friendly" list UI rather than nabble[2]. Folks do need an
> >> > @apache email address to post through lists.apache, but I don't know
> >> > how much we value the "ease of posting" feature of nabble.
> >> >
> >> > [1]: http://nifi.apache.org/mailing_lists.html
> >> > [2]: https://lists.apache.org/list.html?dev@nifi.apache.org
> >> >
> >> > On Mon, Jun 27, 2016 at 2:31 AM, Lars Francke  >
> >> wrote:
> >> >> Hi everyone,
> >> >>
> >> >> there's lots of mails from the Github integration in the dev@
> mailing
> >> list.
> >> >> It's getting pretty overwhelming. Yes I've created a filter but
> everyone
> >> >> would need to do that. Other projects have opted to move these kind
> of
> >> >> things to a "issues" mailing list.
> >> >>
> >> >> What do you think about that?
> >> >>
> >> >> Cheers,
> >> >> Lars
> >>
>

Re: flow files size

[pradeepbill]

thanks Joe and Joe ,  Control rate seems to work fine for me.



--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/flow-files-size-tp12556p12568.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: Update on 0.7.0 RC2

[Oleg Zhurakousky]

Found a bug for NIFI-2032, will be issuing PR shortly.

Oleg

> On Jul 2, 2016, at 1:02 PM, Joe Percivall  
> wrote:
> 
> Hello Team,
> 
> There have been two new critical bugs identified by Brandon DeVries and added 
> to the 0.7.0 release. The status of the remaining 0.7.0 issues are below.
> 
> - "Enabled ControllerServices disabled on restart" NIFI-2160[1] [status] A 
> regression due to NIFI-2032 that breaks backwards compatibility. Discussion 
> has started between Oleg Zhurakousky and Brandon DeVries.
> 
> - "Fingerprint not detecting flow.xml differences" NIFI-2159[2] [status] 
> Reported by Brandon. No one has started work on it. NEEDS AN ASSIGNEE
> 
> - "Secure clustering returning bad request response" NIFI-2119[3] [status] No 
> updates, Andy LoPresto is looking into it.
> 
> Again for clarity, NIFI-1974 and NIFI-2089 are listed as 0.7.0 and unresolved 
> but have already been merged into 0.7.0. Also due to the reasons highlighted 
> in my previous message I will not be tracking NIFI-2120 and it is punted.
> 
> [1] https://issues.apache.org/jira/browse/NIFI-2160
> [2] https://issues.apache.org/jira/browse/NIFI-2159
> [3] https://issues.apache.org/jira/browse/NIFI-2119
> 
> Hope everyone has a fun and safe fourth,
> Joe
> - - - - - - 
> Joseph Percivall
> linkedin.com/in/Percivall
> e: joeperciv...@yahoo.com
> 
> 
> 
> 
> On Wednesday, June 29, 2016 9:12 AM, Joe Percivall 
>  wrote:
> Hello Team,
> 
> As most know, the closing of RC1 was due to three issues discovered by Joe 
> Witt. A status update for each is below. 
> 
> 
> - "Secure clustering returning bad request response" NIFI-2119[1] [status] A 
> probable source of the problem was discovered by Matt Gilman, NIFI-1753 a 
> security upgrade. Andy LoPresto is looking into a fix.
> 
> - "source release produces an extraneous file" NIFI-2120[2] [status] I looked 
> into how it would be done and I couldn't find any options in the maven 
> release plugin that support excluding specific files from the release. I will 
> be punting this, unless if someone with more experience with the maven 
> release plugin can fix it.
> 
> - "Correct license/notice issues" NIFI-2118[3] [status] I addressed the 
> issues identified by Joe Witt and the fix was merged by Matt Burgess.
> 
> Again I apologize for not finding this issues earlier and will keep you all 
> up to date on any changes. Also since RC1 a commit addressing NIFI-1920[4] 
> has been merged into the 0.x branch. I will be including it in RC2. It is a 
> simple fix that addresses an inadvertent rollback in UnpackContent.
> 
> [1] https://issues.apache.org/jira/browse/NIFI-2119
> [2] https://issues.apache.org/jira/browse/NIFI-2120
> [3] https://issues.apache.org/jira/browse/NIFI-2118
> [4] https://issues.apache.org/jira/browse/NIFI-920
> 
> Thank you,
> Joe
> - - - - - - 
> Joseph Percivall
> linkedin.com/in/Percivall
> e: joeperciv...@yahoo.com
> 

[GitHub] nifi issue #602: NIFI-2165: fix support for inserting timestamps into cassan...

[mattyb149]

Github user mattyb149 commented on the issue:

https://github.com/apache/nifi/pull/602
  
You're right, we'd need some integration tests (possibly using 
CassandraUnit) to show this. I will verify your change then merge as-is, we can 
add an improvement Jira or something to add integration tests and exercise more 
of the type conversion logic.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Update on 0.7.0 RC2

[Oleg Zhurakousky]

For a second there I thought I did but never mind that, I've just commented on 
https://issues.apache.org/jira/browse/NIFI-2160, I can’t reproduce the issue no 
mater what I do. Will keep on digging, but what worried me is that Brandon’s 
original explanation seem to point to some jiggery-trickery with CS lifecycle 
that doesn’t seem to be reliant on public API. So I’ve asked him for more 
details. . .
Oleg

On Jul 5, 2016, at 9:27 AM, Oleg Zhurakousky 
mailto:ozhurakou...@hortonworks.com>> wrote:

Found a bug for NIFI-2032, will be issuing PR shortly.

Oleg

On Jul 2, 2016, at 1:02 PM, Joe Percivall 
mailto:joeperciv...@yahoo.com.invalid>> wrote:

Hello Team,

There have been two new critical bugs identified by Brandon DeVries and added 
to the 0.7.0 release. The status of the remaining 0.7.0 issues are below.

- "Enabled ControllerServices disabled on restart" NIFI-2160[1] [status] A 
regression due to NIFI-2032 that breaks backwards compatibility. Discussion has 
started between Oleg Zhurakousky and Brandon DeVries.

- "Fingerprint not detecting flow.xml differences" NIFI-2159[2] [status] 
Reported by Brandon. No one has started work on it. NEEDS AN ASSIGNEE

- "Secure clustering returning bad request response" NIFI-2119[3] [status] No 
updates, Andy LoPresto is looking into it.

Again for clarity, NIFI-1974 and NIFI-2089 are listed as 0.7.0 and unresolved 
but have already been merged into 0.7.0. Also due to the reasons highlighted in 
my previous message I will not be tracking NIFI-2120 and it is punted.

[1] https://issues.apache.org/jira/browse/NIFI-2160
[2] https://issues.apache.org/jira/browse/NIFI-2159
[3] https://issues.apache.org/jira/browse/NIFI-2119

Hope everyone has a fun and safe fourth,
Joe
- - - - - -
Joseph Percivall
linkedin.com/in/Percivall
e: joeperciv...@yahoo.com




On Wednesday, June 29, 2016 9:12 AM, Joe Percivall 
 wrote:
Hello Team,

As most know, the closing of RC1 was due to three issues discovered by Joe 
Witt. A status update for each is below.


- "Secure clustering returning bad request response" NIFI-2119[1] [status] A 
probable source of the problem was discovered by Matt Gilman, NIFI-1753 a 
security upgrade. Andy LoPresto is looking into a fix.

- "source release produces an extraneous file" NIFI-2120[2] [status] I looked 
into how it would be done and I couldn't find any options in the maven release 
plugin that support excluding specific files from the release. I will be 
punting this, unless if someone with more experience with the maven release 
plugin can fix it.

- "Correct license/notice issues" NIFI-2118[3] [status] I addressed the issues 
identified by Joe Witt and the fix was merged by Matt Burgess.

Again I apologize for not finding this issues earlier and will keep you all up 
to date on any changes. Also since RC1 a commit addressing NIFI-1920[4] has 
been merged into the 0.x branch. I will be including it in RC2. It is a simple 
fix that addresses an inadvertent rollback in UnpackContent.

[1] https://issues.apache.org/jira/browse/NIFI-2119
[2] https://issues.apache.org/jira/browse/NIFI-2120
[3] https://issues.apache.org/jira/browse/NIFI-2118
[4] https://issues.apache.org/jira/browse/NIFI-920

Thank you,
Joe
- - - - - -
Joseph Percivall
linkedin.com/in/Percivall
e: joeperciv...@yahoo.com

[GitHub] nifi issue #602: NIFI-2165: fix support for inserting timestamps into cassan...

[mattyb149]

Github user mattyb149 commented on the issue:

https://github.com/apache/nifi/pull/602
  
@jeffoxenberg mind squashing your commits into one? Please and thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #602: NIFI-2165: fix support for inserting timestamps into cassan...

[mattyb149]

Github user mattyb149 commented on the issue:

https://github.com/apache/nifi/pull/602
  
@jeffoxenberg After your fix is applied, I tried a QueryCassandra to get 
the values out of the table, and I'm getting a similar codec error then. Should 
we edit the Jira to fix both cases, or do you want to handle just the 
PutCassandraQL issue here, and do the QueryCassandra one separately?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #576: NIFI-2068: Add Elasticsearch HTTP processors

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/576#discussion_r69582549
  
--- Diff: 
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearchHttp.java
 ---
@@ -194,65 +209,61 @@ public void onTrigger(final ProcessContext context, 
final ProcessSession session
 final byte[] bodyBytes = body.bytes();
 JsonNode responseJson = parseJsonResponse(new 
ByteArrayInputStream(bodyBytes));
 boolean found = responseJson.get("found").asBoolean(false);
+String retrievedIndex = 
responseJson.get("_index").asText();
+String retrievedType = responseJson.get("_type").asText();
+String retrievedId = responseJson.get("_id").asText();
 
 if (found) {
-flowFile = session.putAttribute(flowFile, "filename", 
docId);
-flowFile = session.putAttribute(flowFile, "es.index", 
index);
-flowFile = session.putAttribute(flowFile, "es.type", 
docType);
+String source = responseJson.get("_source").toString();
+flowFile = session.putAttribute(flowFile, "filename", 
retrievedId);
+flowFile = session.putAttribute(flowFile, "es.index", 
retrievedIndex);
+flowFile = session.putAttribute(flowFile, "es.type", 
retrievedType);
 flowFile = session.write(flowFile, out -> {
-out.write(bodyBytes);
+out.write(source.getBytes());
 });
-logger.debug("Elasticsearch document " + docId + " 
fetched, routing to success");
+logger.debug("Elasticsearch document " + retrievedId + 
" fetched, routing to success");
 
 // emit provenance event
 final long millis = 
TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - startNanos);
-session.getProvenanceReporter().fetch(flowFile, 
url.toExternalForm(), millis);
+if (context.hasIncomingConnection()) {
--- End diff --

This should probably be "hasNonLoopConnection" instead of 
"hasIncomingConnection". That way if the processor is acting as a source but 
the Rel_Retry is routed back it will properly still emit "receive". That or 
create a flag at the beginning of the ontrigger for if the flowfile is is 
created here or is from an incoming relationship.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #576: NIFI-2068: Add Elasticsearch HTTP processors

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/576#discussion_r69582751
  
--- Diff: 
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearchHttp.java
 ---
@@ -194,65 +209,61 @@ public void onTrigger(final ProcessContext context, 
final ProcessSession session
 final byte[] bodyBytes = body.bytes();
 JsonNode responseJson = parseJsonResponse(new 
ByteArrayInputStream(bodyBytes));
 boolean found = responseJson.get("found").asBoolean(false);
+String retrievedIndex = 
responseJson.get("_index").asText();
+String retrievedType = responseJson.get("_type").asText();
+String retrievedId = responseJson.get("_id").asText();
 
 if (found) {
-flowFile = session.putAttribute(flowFile, "filename", 
docId);
-flowFile = session.putAttribute(flowFile, "es.index", 
index);
-flowFile = session.putAttribute(flowFile, "es.type", 
docType);
+String source = responseJson.get("_source").toString();
+flowFile = session.putAttribute(flowFile, "filename", 
retrievedId);
+flowFile = session.putAttribute(flowFile, "es.index", 
retrievedIndex);
+flowFile = session.putAttribute(flowFile, "es.type", 
retrievedType);
 flowFile = session.write(flowFile, out -> {
-out.write(bodyBytes);
+out.write(source.getBytes());
 });
-logger.debug("Elasticsearch document " + docId + " 
fetched, routing to success");
+logger.debug("Elasticsearch document " + retrievedId + 
" fetched, routing to success");
 
 // emit provenance event
 final long millis = 
TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - startNanos);
-session.getProvenanceReporter().fetch(flowFile, 
url.toExternalForm(), millis);
+if (context.hasIncomingConnection()) {
+session.getProvenanceReporter().fetch(flowFile, 
url.toExternalForm(), millis);
+} else {
+session.getProvenanceReporter().receive(flowFile, 
url.toExternalForm(), millis);
+}
 session.transfer(flowFile, REL_SUCCESS);
 } else {
 logger.warn("Failed to read {}/{}/{} from 
Elasticsearch: Document not found",
 new Object[]{index, docType, docId});
 
-// We couldn't find the document, so penalize it and 
send it to "not found"
-// If there was an incoming flow file, penalize and 
transfer. Otherwise just remove it
-if (context.hasIncomingConnection()) {
-flowFile = session.penalize(flowFile);
-session.transfer(flowFile, REL_NOT_FOUND);
-} else {
-session.remove(flowFile);
-}
+// We couldn't find the document, so send it to "not 
found"
+session.transfer(flowFile, REL_NOT_FOUND);
 }
-
 } else {
 if (statusCode == 404) {
 logger.warn("Failed to read {}/{}/{} from 
Elasticsearch: Document not found",
 new Object[]{index, docType, docId});
 
 // We couldn't find the document, so penalize it and 
send it to "not found"
-// If there was an incoming flow file, penalize and 
transfer. Otherwise just remove it
-if (context.hasIncomingConnection()) {
-flowFile = session.penalize(flowFile);
-session.transfer(flowFile, REL_NOT_FOUND);
-} else {
-session.remove(flowFile);
-}
-} else if (context.hasIncomingConnection()) {
-// 5xx -> RETRY
+session.transfer(flowFile, REL_NOT_FOUND);
+} else {
+// 5xx -> RETRY, but a server error might last a 
while, so yield
 if (statusCode / 100 == 5) {
-logger.info("Elasticsearch returned code {} with 
message {}, transferring flow file to retry", new Object[]{statusCode, 
getResponse.message()});
-flowFile = session.penalize(flowFile);
-

[GitHub] nifi issue #602: NIFI-2165: fix support for inserting timestamps into cassan...

[jeffoxenberg]

Github user jeffoxenberg commented on the issue:

https://github.com/apache/nifi/pull/602
  
Squashed my commits.  I'll look into integration testing as well.  I'll try 
to handle the QueryCassandra one separately as I'll be busy for the next fewof 
days with other things.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #576: NIFI-2068: Add Elasticsearch HTTP processors

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/576#discussion_r69584216
  
--- Diff: 
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearchHttp.java
 ---
@@ -194,65 +209,61 @@ public void onTrigger(final ProcessContext context, 
final ProcessSession session
 final byte[] bodyBytes = body.bytes();
 JsonNode responseJson = parseJsonResponse(new 
ByteArrayInputStream(bodyBytes));
 boolean found = responseJson.get("found").asBoolean(false);
+String retrievedIndex = 
responseJson.get("_index").asText();
+String retrievedType = responseJson.get("_type").asText();
+String retrievedId = responseJson.get("_id").asText();
 
 if (found) {
-flowFile = session.putAttribute(flowFile, "filename", 
docId);
-flowFile = session.putAttribute(flowFile, "es.index", 
index);
-flowFile = session.putAttribute(flowFile, "es.type", 
docType);
+String source = responseJson.get("_source").toString();
+flowFile = session.putAttribute(flowFile, "filename", 
retrievedId);
+flowFile = session.putAttribute(flowFile, "es.index", 
retrievedIndex);
+flowFile = session.putAttribute(flowFile, "es.type", 
retrievedType);
 flowFile = session.write(flowFile, out -> {
-out.write(bodyBytes);
+out.write(source.getBytes());
 });
-logger.debug("Elasticsearch document " + docId + " 
fetched, routing to success");
+logger.debug("Elasticsearch document " + retrievedId + 
" fetched, routing to success");
 
 // emit provenance event
 final long millis = 
TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - startNanos);
-session.getProvenanceReporter().fetch(flowFile, 
url.toExternalForm(), millis);
+if (context.hasIncomingConnection()) {
+session.getProvenanceReporter().fetch(flowFile, 
url.toExternalForm(), millis);
+} else {
+session.getProvenanceReporter().receive(flowFile, 
url.toExternalForm(), millis);
+}
 session.transfer(flowFile, REL_SUCCESS);
 } else {
 logger.warn("Failed to read {}/{}/{} from 
Elasticsearch: Document not found",
 new Object[]{index, docType, docId});
 
-// We couldn't find the document, so penalize it and 
send it to "not found"
-// If there was an incoming flow file, penalize and 
transfer. Otherwise just remove it
-if (context.hasIncomingConnection()) {
-flowFile = session.penalize(flowFile);
-session.transfer(flowFile, REL_NOT_FOUND);
-} else {
-session.remove(flowFile);
-}
+// We couldn't find the document, so send it to "not 
found"
+session.transfer(flowFile, REL_NOT_FOUND);
 }
-
 } else {
 if (statusCode == 404) {
 logger.warn("Failed to read {}/{}/{} from 
Elasticsearch: Document not found",
 new Object[]{index, docType, docId});
 
 // We couldn't find the document, so penalize it and 
send it to "not found"
-// If there was an incoming flow file, penalize and 
transfer. Otherwise just remove it
-if (context.hasIncomingConnection()) {
-flowFile = session.penalize(flowFile);
-session.transfer(flowFile, REL_NOT_FOUND);
-} else {
-session.remove(flowFile);
-}
-} else if (context.hasIncomingConnection()) {
-// 5xx -> RETRY
+session.transfer(flowFile, REL_NOT_FOUND);
+} else {
+// 5xx -> RETRY, but a server error might last a 
while, so yield
 if (statusCode / 100 == 5) {
-logger.info("Elasticsearch returned code {} with 
message {}, transferring flow file to retry", new Object[]{statusCode, 
getResponse.message()});
-flowFile = session.penalize(flowFile);
-

[GitHub] nifi pull request #576: NIFI-2068: Add Elasticsearch HTTP processors

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/576#discussion_r69586209
  
--- Diff: 
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearchHttp.java
 ---
@@ -324,7 +351,6 @@ public void onTrigger(final ProcessContext context, 
final ProcessSession session
 logger.error("Failed to insert into Elasticsearch due to {}.", 
new Object[]{ioe.getLocalizedMessage()}, ioe);
--- End diff --

I really don't like this huge try/catch. The only two places that this 
catches are lines 208 and 305. For 208, that is failing to create the URL, you 
should probably yield the context/throw an error because something is weirdly 
wrong. For 305, it is failing to read the bytes of the response, just move it a 
line lower into the try/catch block below it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #602: NIFI-2165: fix support for inserting timestamps into cassan...

[mattyb149]

Github user mattyb149 commented on the issue:

https://github.com/apache/nifi/pull/602
  
Ok sounds good. I gave the Integration Test thing (and the QueryCassandra 
-- actually in AbstractCassandraProcessor) fix a try, the ITs don't work (yet) 
but feel free to take a look: 
https://github.com/mattyb149/nifi/tree/cassandra_time

I'll write up the Jira for QueryCassandra, and if I get a chance to test 
the fix I will submit the PR myself and you can review if you like :)

Will do a final look-around then merge, thanks again!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #563: NIFI-2078: External state management.

[JPercivall]

Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/563
  
Starting to review.

Unless if I'm missing something, why is external state only added for 
GetKafka and ConsumeKafka? Aren't there many other processors that manage state 
externally?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request #21: MINIFI-41 - CLI utility for template.xml -> YA...

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/21#discussion_r69602801
  
--- Diff: minifi-toolkit/minifi-toolkit-assembly/NOTICE ---
@@ -0,0 +1,16 @@
+Apache NiFi
+Copyright 2014-2016 The Apache Software Foundation
+
+This product includes software developed at
+The Apache Software Foundation (http://www.apache.org/).
+
+===
+Apache Software License v2
+===
+
+The following binary components are provided under the Apache Software 
License v2
+
+  (ASLv2) Apache NiFi
--- End diff --

Apache NiFi's notice includes more than just this. It should be this[1]:
> 
> Apache NiFi
> Copyright 2014-2016 The Apache Software Foundation
> 
> This product includes software developed at
> The Apache Software Foundation (http://www.apache.org/).
> 
> This product includes the following work from the Apache Hadoop project:
> 
> BoundedByteArrayOutputStream.java adapted to 
SoftLimitBoundedByteArrayOutputStream.java

[1] https://github.com/apache/nifi/blob/nifi-0.6.0/NOTICE


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request #21: MINIFI-41 - CLI utility for template.xml -> YA...

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/21#discussion_r69603609
  
--- Diff: 
minifi-toolkit/minifi-toolkit-configuration/src/main/java/org/apache/nifi/minifi/toolkit/configuration/ConfigMain.java
 ---
@@ -0,0 +1,342 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.toolkit.configuration;
+
+import org.apache.nifi.controller.Template;
+import org.apache.nifi.minifi.commons.schema.ConfigSchema;
+import org.apache.nifi.minifi.commons.schema.SecurityPropertiesSchema;
+import org.apache.nifi.minifi.commons.schema.common.BaseSchema;
+import org.apache.nifi.minifi.commons.schema.serialization.SchemaLoader;
+import org.apache.nifi.minifi.commons.schema.serialization.SchemaSaver;
+import 
org.apache.nifi.minifi.commons.schema.exception.SchemaLoaderException;
+import org.apache.nifi.ssl.StandardSSLContextService;
+import org.apache.nifi.web.api.dto.ConnectableDTO;
+import org.apache.nifi.web.api.dto.ConnectionDTO;
+import org.apache.nifi.web.api.dto.ControllerServiceDTO;
+import org.apache.nifi.web.api.dto.FlowSnippetDTO;
+import org.apache.nifi.web.api.dto.NiFiComponentDTO;
+import org.apache.nifi.web.api.dto.PortDTO;
+import org.apache.nifi.web.api.dto.ProcessorDTO;
+import org.apache.nifi.web.api.dto.RemoteProcessGroupContentsDTO;
+import org.apache.nifi.web.api.dto.RemoteProcessGroupDTO;
+import org.apache.nifi.web.api.dto.RemoteProcessGroupPortDTO;
+import org.apache.nifi.web.api.dto.TemplateDTO;
+
+import javax.xml.bind.JAXBContext;
+import javax.xml.bind.JAXBException;
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.TreeMap;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+public class ConfigMain {
+public static final int ERR_INVALID_ARGS = 1;
+public static final int ERR_UNABLE_TO_OPEN_OUTPUT = 2;
+public static final int ERR_UNABLE_TO_OPEN_INPUT = 3;
+public static final int ERR_UNABLE_TO_READ_TEMPLATE = 4;
+public static final int ERR_UNABLE_TO_TRANFORM_TEMPLATE = 5;
+public static final int ERR_UNABLE_TO_PARSE_CONFIG = 6;
+public static final int ERR_INVALID_CONFIG = 7;
+
+public static final int SUCCESS = 0;
+
+public static final String TRANSFORM = "transform";
+public static final String VALIDATE = "validate";
+
+private final Map commandMap;
+private final PathInputStreamFactory pathInputStreamFactory;
+private final PathOutputStreamFactory pathOutputStreamFactory;
+
+public ConfigMain() {
+this(FileInputStream::new, FileOutputStream::new);
+}
+
+public ConfigMain(PathInputStreamFactory pathInputStreamFactory, 
PathOutputStreamFactory pathOutputStreamFactory) {
+this.pathInputStreamFactory = pathInputStreamFactory;
+this.pathOutputStreamFactory = pathOutputStreamFactory;
+this.commandMap = createCommandMap();
+}
+
+public static void main(String[] args) {
+System.exit(new ConfigMain().execute(args));
+}
+
+public static void printValidateUsage() {
+System.out.println("Validate Usage:");
+System.out.println();
+System.out.print("java ");
+System.out.print(ConfigMain.class.getCanonicalName());
+System.out.println(" validate INPUT_FILE");
+System.out.println();
+}
+
+public int validate(String[] args) {
+if (args.length != 2) {
+printValidateUsage();
+return ERR_INVALID_ARGS;
+

[GitHub] nifi-minifi pull request #21: MINIFI-41 - CLI utility for template.xml -> YA...

[brosander]

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/21#discussion_r69603602
  
--- Diff: minifi-toolkit/minifi-toolkit-assembly/NOTICE ---
@@ -0,0 +1,16 @@
+Apache NiFi
+Copyright 2014-2016 The Apache Software Foundation
+
+This product includes software developed at
+The Apache Software Foundation (http://www.apache.org/).
+
+===
+Apache Software License v2
+===
+
+The following binary components are provided under the Apache Software 
License v2
+
+  (ASLv2) Apache NiFi
--- End diff --

I took that from [the minifi assembly 
notice](https://github.com/apache/nifi-minifi/blob/a1d2fd3fe6b0b2cdd50394e2451aa10d5c9c81e9/minifi-assembly/NOTICE#L13).
  Should I update that one too or address in separate jira?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request #21: MINIFI-41 - CLI utility for template.xml -> YA...

[brosander]

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/21#discussion_r69603745
  
--- Diff: 
minifi-toolkit/minifi-toolkit-configuration/src/main/java/org/apache/nifi/minifi/toolkit/configuration/ConfigMain.java
 ---
@@ -0,0 +1,342 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.toolkit.configuration;
+
+import org.apache.nifi.controller.Template;
+import org.apache.nifi.minifi.commons.schema.ConfigSchema;
+import org.apache.nifi.minifi.commons.schema.SecurityPropertiesSchema;
+import org.apache.nifi.minifi.commons.schema.common.BaseSchema;
+import org.apache.nifi.minifi.commons.schema.serialization.SchemaLoader;
+import org.apache.nifi.minifi.commons.schema.serialization.SchemaSaver;
+import 
org.apache.nifi.minifi.commons.schema.exception.SchemaLoaderException;
+import org.apache.nifi.ssl.StandardSSLContextService;
+import org.apache.nifi.web.api.dto.ConnectableDTO;
+import org.apache.nifi.web.api.dto.ConnectionDTO;
+import org.apache.nifi.web.api.dto.ControllerServiceDTO;
+import org.apache.nifi.web.api.dto.FlowSnippetDTO;
+import org.apache.nifi.web.api.dto.NiFiComponentDTO;
+import org.apache.nifi.web.api.dto.PortDTO;
+import org.apache.nifi.web.api.dto.ProcessorDTO;
+import org.apache.nifi.web.api.dto.RemoteProcessGroupContentsDTO;
+import org.apache.nifi.web.api.dto.RemoteProcessGroupDTO;
+import org.apache.nifi.web.api.dto.RemoteProcessGroupPortDTO;
+import org.apache.nifi.web.api.dto.TemplateDTO;
+
+import javax.xml.bind.JAXBContext;
+import javax.xml.bind.JAXBException;
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.TreeMap;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+public class ConfigMain {
+public static final int ERR_INVALID_ARGS = 1;
+public static final int ERR_UNABLE_TO_OPEN_OUTPUT = 2;
+public static final int ERR_UNABLE_TO_OPEN_INPUT = 3;
+public static final int ERR_UNABLE_TO_READ_TEMPLATE = 4;
+public static final int ERR_UNABLE_TO_TRANFORM_TEMPLATE = 5;
+public static final int ERR_UNABLE_TO_PARSE_CONFIG = 6;
+public static final int ERR_INVALID_CONFIG = 7;
+
+public static final int SUCCESS = 0;
+
+public static final String TRANSFORM = "transform";
+public static final String VALIDATE = "validate";
+
+private final Map commandMap;
+private final PathInputStreamFactory pathInputStreamFactory;
+private final PathOutputStreamFactory pathOutputStreamFactory;
+
+public ConfigMain() {
+this(FileInputStream::new, FileOutputStream::new);
+}
+
+public ConfigMain(PathInputStreamFactory pathInputStreamFactory, 
PathOutputStreamFactory pathOutputStreamFactory) {
+this.pathInputStreamFactory = pathInputStreamFactory;
+this.pathOutputStreamFactory = pathOutputStreamFactory;
+this.commandMap = createCommandMap();
+}
+
+public static void main(String[] args) {
+System.exit(new ConfigMain().execute(args));
+}
+
+public static void printValidateUsage() {
+System.out.println("Validate Usage:");
+System.out.println();
+System.out.print("java ");
+System.out.print(ConfigMain.class.getCanonicalName());
+System.out.println(" validate INPUT_FILE");
+System.out.println();
+}
+
+public int validate(String[] args) {
+if (args.length != 2) {
+printValidateUsage();
+return ERR_INVALID_ARGS;
+}

[GitHub] nifi-minifi pull request #21: MINIFI-41 - CLI utility for template.xml -> YA...

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/21#discussion_r69603973
  
--- Diff: minifi-bootstrap/src/test/resources/Working_with_Logs.yml ---
@@ -0,0 +1,253 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the \"License\"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an \"AS IS\" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+Flow Controller:
+  name: Working with Logs
+  comment: ''
+Core Properties:
+  flow controller graceful shutdown period: 10 sec
+  flow service write delay interval: 500 ms
+  administrative yield duration: 30 sec
+  bored yield duration: 10 millis
+  max concurrent threads: 1
+FlowFile Repository:
+  partitions: 256
+  checkpoint interval: 2 mins
+  always sync: false
+  Swap:
+threshold: 2
+in period: 5 sec
+in threads: 1
+out period: 5 sec
+out threads: 4
+Content Repository:
+  content claim max appendable size: 10 MB
+  content claim max flow files: 100
+  always sync: false
+Provenance Repository:
+  provenance rollover time: 1 min
+Component Status Repository:
+  buffer size: 1440
+  snapshot frequency: 1 min
+Security Properties:
+  keystore:
+  keystore type:
+  key password:
+  truststore:
+  truststore type:
+  truststore password:
+  ssl protocol:
+  Sensitive Props:
+key:
+algorithm: PBEWITHMD5AND256BITAES-CBC-OPENSSL
+provider: BC
+Processors:
+- name: MergeContent
+  class: org.apache.nifi.processors.standard.MergeContent
+  max concurrent tasks: 1
+  scheduling strategy: TIMER_DRIVEN
+  scheduling period: 0 sec
+  penalization period: 30 sec
+  yield period: 1 sec
+  run duration nanos: 0
+  auto-terminated relationships list:
+  - failure
+  - original
+  Properties:
+Attribute Strategy: Keep Only Common Attributes
+Compression Level: '1'
+Correlation Attribute Name: log.type
+Delimiter Strategy: Text
+Demarcator File: |2+
+
+Footer File:
+Header File:
+Keep Path: 'false'
+Max Bin Age: 5 mins
+Maximum Group Size: 128 MB
+Maximum Number of Entries:
+Maximum number of Bins: '100'
+Merge Format: Binary Concatenation
+Merge Strategy: Bin-Packing Algorithm
+Minimum Group Size: 64 MB
+Minimum Number of Entries: '1'
+- name: PutHDFS
+  class: org.apache.nifi.processors.hadoop.PutHDFS
+  max concurrent tasks: 1
+  scheduling strategy: TIMER_DRIVEN
+  scheduling period: 0 sec
+  penalization period: 30 sec
+  yield period: 1 sec
+  run duration nanos: 0
+  auto-terminated relationships list:
+  - success
+  Properties:
+Block Size:
+Compression codec: NONE
+Conflict Resolution Strategy: fail
+Directory: /logs/${log.type}/${now():format('/MM/dd')}
+Hadoop Configuration Resources: conf/core-site.xml
+IO Buffer Size:
+Kerberos Keytab:
+Kerberos Principal:
+Kerberos Relogin Period: 4 hours
+Permissions umask:
+Remote Group:
+Remote Owner:
+Replication:
+- name: Tail App Log
+  class: org.apache.nifi.processors.standard.TailFile
+  max concurrent tasks: 1
+  scheduling strategy: TIMER_DRIVEN
+  scheduling period: 0 sec
+  penalization period: 30 sec
+  yield period: 1 sec
+  run duration nanos: 0
+  auto-terminated relationships list: []
+  Properties:
+File to Tail: logs/nifi-app.log
+Initial Start Position: Current Time
+Rolling Filename Pattern: nifi-app*
+State File: ./conf/state/737672eb-3f71-4ef9-bc86-dbc3645833d9
+- name: Tail User Log
+  class: org.apache.nifi.processors.standard.TailFile
+  max concurrent tasks: 1
+  scheduling strategy: TIMER_DRIVEN
+  scheduling period: 0 sec
+  penalization period: 30 sec
+  yield period: 1 sec
+  run duration nanos: 0
+  auto-terminated relationships list: []
+  Properties:
+File to Tail: logs/nifi-user.log
+Initial Start Position:

[GitHub] nifi issue #602: NIFI-2165: fix support for inserting timestamps into cassan...

[mattyb149]

Github user mattyb149 commented on the issue:

https://github.com/apache/nifi/pull/602
  
+1 LGTM, merging to master


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #602: NIFI-2165: fix support for inserting timestamps into...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/602


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request #21: MINIFI-41 - CLI utility for template.xml -> YA...

[brosander]

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/21#discussion_r69606845
  
--- Diff: minifi-bootstrap/src/test/resources/Working_with_Logs.yml ---
@@ -0,0 +1,253 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the \"License\"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an \"AS IS\" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+Flow Controller:
+  name: Working with Logs
+  comment: ''
+Core Properties:
+  flow controller graceful shutdown period: 10 sec
+  flow service write delay interval: 500 ms
+  administrative yield duration: 30 sec
+  bored yield duration: 10 millis
+  max concurrent threads: 1
+FlowFile Repository:
+  partitions: 256
+  checkpoint interval: 2 mins
+  always sync: false
+  Swap:
+threshold: 2
+in period: 5 sec
+in threads: 1
+out period: 5 sec
+out threads: 4
+Content Repository:
+  content claim max appendable size: 10 MB
+  content claim max flow files: 100
+  always sync: false
+Provenance Repository:
+  provenance rollover time: 1 min
+Component Status Repository:
+  buffer size: 1440
+  snapshot frequency: 1 min
+Security Properties:
+  keystore:
+  keystore type:
+  key password:
+  truststore:
+  truststore type:
+  truststore password:
+  ssl protocol:
+  Sensitive Props:
+key:
+algorithm: PBEWITHMD5AND256BITAES-CBC-OPENSSL
+provider: BC
+Processors:
+- name: MergeContent
+  class: org.apache.nifi.processors.standard.MergeContent
+  max concurrent tasks: 1
+  scheduling strategy: TIMER_DRIVEN
+  scheduling period: 0 sec
+  penalization period: 30 sec
+  yield period: 1 sec
+  run duration nanos: 0
+  auto-terminated relationships list:
+  - failure
+  - original
+  Properties:
+Attribute Strategy: Keep Only Common Attributes
+Compression Level: '1'
+Correlation Attribute Name: log.type
+Delimiter Strategy: Text
+Demarcator File: |2+
+
+Footer File:
+Header File:
+Keep Path: 'false'
+Max Bin Age: 5 mins
+Maximum Group Size: 128 MB
+Maximum Number of Entries:
+Maximum number of Bins: '100'
+Merge Format: Binary Concatenation
+Merge Strategy: Bin-Packing Algorithm
+Minimum Group Size: 64 MB
+Minimum Number of Entries: '1'
+- name: PutHDFS
+  class: org.apache.nifi.processors.hadoop.PutHDFS
+  max concurrent tasks: 1
+  scheduling strategy: TIMER_DRIVEN
+  scheduling period: 0 sec
+  penalization period: 30 sec
+  yield period: 1 sec
+  run duration nanos: 0
+  auto-terminated relationships list:
+  - success
+  Properties:
+Block Size:
+Compression codec: NONE
+Conflict Resolution Strategy: fail
+Directory: /logs/${log.type}/${now():format('/MM/dd')}
+Hadoop Configuration Resources: conf/core-site.xml
+IO Buffer Size:
+Kerberos Keytab:
+Kerberos Principal:
+Kerberos Relogin Period: 4 hours
+Permissions umask:
+Remote Group:
+Remote Owner:
+Replication:
+- name: Tail App Log
+  class: org.apache.nifi.processors.standard.TailFile
+  max concurrent tasks: 1
+  scheduling strategy: TIMER_DRIVEN
+  scheduling period: 0 sec
+  penalization period: 30 sec
+  yield period: 1 sec
+  run duration nanos: 0
+  auto-terminated relationships list: []
+  Properties:
+File to Tail: logs/nifi-app.log
+Initial Start Position: Current Time
+Rolling Filename Pattern: nifi-app*
+State File: ./conf/state/737672eb-3f71-4ef9-bc86-dbc3645833d9
+- name: Tail User Log
+  class: org.apache.nifi.processors.standard.TailFile
+  max concurrent tasks: 1
+  scheduling strategy: TIMER_DRIVEN
+  scheduling period: 0 sec
+  penalization period: 30 sec
+  yield period: 1 sec
+  run duration nanos: 0
+  auto-terminated relationships list: []
+  Properties:
+File to Tail: logs/nifi-user.log
+Initial Start Position: 

[GitHub] nifi pull request #546: NIFI-1877, NIFI-1306: Add fields to FlowFile for FIF...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/546


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #601: NIFI-2039: Provide a new ProcessSession.read() method that ...

[mattyb149]

Github user mattyb149 commented on the issue:

https://github.com/apache/nifi/pull/601
  
Reviewing...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request #21: MINIFI-41 - CLI utility for template.xml -> YA...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi/pull/21


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #601: NIFI-2039: Provide a new ProcessSession.read() method that ...

[mattyb149]

Github user mattyb149 commented on the issue:

https://github.com/apache/nifi/pull/601
  
+1 LGTM, ran all tests and manually verified error messages, etc. Merging 
into master, thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #601: NIFI-2039: Provide a new ProcessSession.read() metho...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/601


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request #26: MINIFI-48 - Omitting queuePrioritizerClass fro...

[brosander]

GitHub user brosander opened a pull request:

https://github.com/apache/nifi-minifi/pull/26

MINIFI-48 - Omitting queuePrioritizerClass from flow if it isn't set …

…in schema

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/brosander/nifi-minifi MINIFI-48

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi/pull/26.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #26


commit 80cfbd645e7a8e3f30460c8b6c7b19601f826550
Author: Bryan Rosander 
Date:   2016-07-05T19:05:32Z

MINIFI-48 - Omitting queuePrioritizerClass from flow if it isn't set in 
schema




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request #27: MINIFI-21 - Reduce assembly footprint

[apiri]

GitHub user apiri opened a pull request:

https://github.com/apache/nifi-minifi/pull/27

MINIFI-21 - Reduce assembly footprint

Excluding duplicated dependencies transitively provided to the assembly.  
Excluding NiFi specific configuration files that dealt with user facing 
components removed in MiNiFi.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apiri/nifi-minifi MINIFI-21

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi/pull/27.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #27


commit be71c74ad4eb63bbad52dcb5c591b8f34b2efcec
Author: Aldrin Piri 
Date:   2016-07-05T17:57:53Z

MINIFI-21 MINIFI-24
Excluding duplicated dependencies transitively provided to the assembly.  
Excluding NiFi specific configuration files that dealt with user facing 
components removed in MiNiFi.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request #26: MINIFI-48 - Omitting queuePrioritizerClass fro...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi/pull/26


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[DISCUSS] MiNiFi's Initial Release, 0.0.1

[Aldrin Piri]

Hey folks,

We've had some nice work done on the MiNiFi Java implementation but would
love to start putting things through their paces and extend the community.
To that end, I would like to tackle our first release and volunteer to take
the RM duties.

MiNiFi has benefitted from the extensive body of work that the Apache NiFi
community has done and, building on that, provides another approach to data
collection and processing, establishing design and deploy of flows via
declarative configuration.  Currently, this is accomplished via YAML, but
interfaces are extensible to support additional/other means and evolve as
the community establishes a path forward.

As part of maintaining a smaller footprint, those processors included are
those located in the NiFi standard NAR to provide the core acquisition
process.  As MiNiFi continues on, we shall seek to maintain API
compatibility between NiFi and MiNiFi as per the efforts of NIFI-1896.  If
the base distribution of MiNiFi is not sufficient, additional NiFi
distribution or custom NARs can be additionally loaded.

There are a couple tickets in their final stages of review and after those
wrap up, I will look to give cover the LICENSE and NOTICE items as well as
augment and extend our existing NiFi release processes for any additional
needs that may arise.

For those who have not had the opportunity to scope out the intents of
MiNiFi or could use a refresher, many of the points from the community's
original discussion on this effort can be located on its wiki page [1].
Please feel free to comment and add any additional thoughts either there or
on JIRA [2].

Thanks!
--Aldrin

[1] https://cwiki.apache.org/confluence/display/NIFI/MiNiFi
[2] https://issues.apache.org/jira/browse/MINIFI

[GitHub] nifi-minifi pull request #27: MINIFI-21 - Reduce assembly footprint

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi/pull/27


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #604: [NIFI-2038] Make component buttons larger and add gr...

[scottyaslan]

GitHub user scottyaslan opened a pull request:

https://github.com/apache/nifi/pull/604

[NIFI-2038] Make component buttons larger and add grip across bottom



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/scottyaslan/nifi devBranch

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/604.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #604


commit 2138e5a77996684e282c166e6987d26df340dfe7
Author: Scott Aslan 
Date:   2016-07-05T21:25:46Z

[NIFI-2038] Make component buttons larger and add grip across bottom




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #605: NIFI-2160 fixed service startup ordering

[olegz]

GitHub user olegz opened a pull request:

https://github.com/apache/nifi/pull/605

NIFI-2160 fixed service startup ordering



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/olegz/nifi NIFI-2160-0.x

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/605.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #605


commit 81687cba18eb63f35f8d20cf3dc4b39232e8307f
Author: Oleg Zhurakousky 
Date:   2016-07-05T21:47:51Z

NIFI-2160 fixed service startup ordering




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #502: Nifi-1972 Apache Ignite Put Cache Processor

[mans2singh]

Github user mans2singh commented on the issue:

https://github.com/apache/nifi/pull/502
  
Hey @pvillard31 @aldrin - Just wanted to check if you have any 
suggestions/recommendations for this pull request.
Thanks


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #606: NIFI-2160 fixed service startup ordering (MASTER)

[olegz]

GitHub user olegz opened a pull request:

https://github.com/apache/nifi/pull/606

NIFI-2160 fixed service startup ordering (MASTER)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/olegz/nifi NIFI-2160

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/606.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #606


commit 7501278a252faabc9aae1f7eea84ef11fb2eac50
Author: Oleg Zhurakousky 
Date:   2016-07-06T00:08:34Z

NIFI-2160 fixed service startup ordering (MASTER)




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #563: NIFI-2078: External state management.

[ijokarumawak]

Github user ijokarumawak commented on the issue:

https://github.com/apache/nifi/pull/563
  
@JPercivall Thanks for your time to review.

I've looked at every existing Processors, Reporting tasks, and Controller 
services, but couldn't find other component needing external state management 
from NiFi.

If you know particular component should implement this, please let me know.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [DISCUSS] MiNiFi's Initial Release, 0.0.1

[Andre]

Aldrin,

This is great news!

Are there plans of releasing minifi-cpp anytime soon?



On Wed, Jul 6, 2016 at 6:53 AM, Aldrin Piri  wrote:

> Hey folks,
>
> We've had some nice work done on the MiNiFi Java implementation but would
> love to start putting things through their paces and extend the community.
> To that end, I would like to tackle our first release and volunteer to take
> the RM duties.
>
> MiNiFi has benefitted from the extensive body of work that the Apache NiFi
> community has done and, building on that, provides another approach to data
> collection and processing, establishing design and deploy of flows via
> declarative configuration.  Currently, this is accomplished via YAML, but
> interfaces are extensible to support additional/other means and evolve as
> the community establishes a path forward.
>
> As part of maintaining a smaller footprint, those processors included are
> those located in the NiFi standard NAR to provide the core acquisition
> process.  As MiNiFi continues on, we shall seek to maintain API
> compatibility between NiFi and MiNiFi as per the efforts of NIFI-1896.  If
> the base distribution of MiNiFi is not sufficient, additional NiFi
> distribution or custom NARs can be additionally loaded.
>
> There are a couple tickets in their final stages of review and after those
> wrap up, I will look to give cover the LICENSE and NOTICE items as well as
> augment and extend our existing NiFi release processes for any additional
> needs that may arise.
>
> For those who have not had the opportunity to scope out the intents of
> MiNiFi or could use a refresher, many of the points from the community's
> original discussion on this effort can be located on its wiki page [1].
> Please feel free to comment and add any additional thoughts either there or
> on JIRA [2].
>
> Thanks!
> --Aldrin
>
> [1] https://cwiki.apache.org/confluence/display/NIFI/MiNiFi
> [2] https://issues.apache.org/jira/browse/MINIFI
>

[GitHub] nifi issue #502: Nifi-1972 Apache Ignite Put Cache Processor

[pvillard31]

Github user pvillard31 commented on the issue:

https://github.com/apache/nifi/pull/502
  
Hey @mans2singh, I'm really busy this week and I won't be able to have a 
look before next week :(
In the meantime, you can fix the conflicts against master. And maybe @apiri 
will have a chance to have a look before I do.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---