[GitHub] nifi pull request: NIFI-1434 Prevent array index exception in PutE...

Github user rpmiskin commented on the pull request: https://github.com/apache/nifi/pull/187#issuecomment-176588865 @JPercivall I believe SMTPTransport will pick up the from address from a couple of other places in preference to the FROM header, so I don't think it should

[GitHub] nifi pull request: NIFI-1423 Allow to penalize FlowFiles to No Ret...

Github user apiri commented on the pull request: https://github.com/apache/nifi/pull/183#issuecomment-176597280 Agree with @JPercivall regarding the rollback to be able to get a full handle on rolled back files for a given session/execution to understand how the processor behaved.

Re: Are we thinking about Penalization all wrong?

Sorry any thoughts; “ I think it would be better/simpler to have one “out of service” concept to replace penalizing and yielding and when a plugin throws an exception then the plugin is deemed out of service, for a duration and so the connection to that plugin is disabled for the out of

Re: Are we thinking about Penalization all wrong?

Mark, I agree with all the points you mention about penalization being confusing, and I think the ability to apply a penalty to Flowfile's outside of a processor is a clearer way to express what is happening. I worry that having the penalty be a property of the connection would also be

[GitHub] nifi pull request: NIFI-1107 Multipart Uploads

Github user trkurc commented on the pull request: https://github.com/apache/nifi/pull/192#issuecomment-176290087 started reviewing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] nifi pull request: NIFI-210: Add ExecuteScript and InvokeScriptPro...

Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/185#discussion_r51151388 --- Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java --- @@ -0,0

Re: Are we thinking about Penalization all wrong?

I really like the idea of being able to have different penalty durations for each connection, and I think this would make things a bit clearer then relying on the developer to determine when to penalize things. On Thu, Jan 28, 2016 at 11:34 AM, Mark Payne wrote: > All, > >

Re: Are we thinking about Penalization all wrong?

Just to recap/level set... The distinct between yielding and penalization is important. Penalization is an action taken on a FlowFile because the FlowFile cannot be processed right now (like a naming conflict for instance). The Processor is indicating that it cannot process that specific FlowFile

[GitHub] nifi pull request: NIFI-1193: Add support for storing data in Hive...

Github user rdblue commented on the pull request: https://github.com/apache/nifi/pull/147#issuecomment-176281907 I just pushed a fix that @rickysaltzer came up with. The jars being pulled in aren't actually needed for the tests so we can exclude them. --- If your project is set up

[GitHub] nifi pull request: NIFI-1193: Add support for storing data in Hive...

Github user apiri commented on the pull request: https://github.com/apache/nifi/pull/147#issuecomment-176603003 @rdblue Thanks! Travis is giving the green thumb up. @rickysaltzer Would you be able to give this the same inspection/evaluation as before now that the build is

[GitHub] nifi pull request: NIFI-210: Add ExecuteScript and InvokeScriptPro...

Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/185#discussion_r51149080 --- Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java --- @@ -0,0

[GitHub] nifi pull request: NIFI-210: Add ExecuteScript and InvokeScriptPro...

Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/185#discussion_r51152200 --- Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java --- @@ -0,0

[GitHub] nifi pull request: NIFI-210: Add ExecuteScript and InvokeScriptPro...

Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/185#discussion_r51156774 --- Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/InvokeScriptProcessor.java --- @@

Re: Are we thinking about Penalization all wrong?

Is there currently a way to know how many times a FlowFile has been penalized? Do we have use cases where we want to penalize a FlowFile *n *number of times before sending it down an alternate relationship? I could imagine an API like penalizeOrTransfer(FlowFile flowFile, int numberOfTries,

Re: Are we thinking about Penalization all wrong?

I think the penalization being on the connection makes sense, but I'm not sure about taking penalization away from the processor altogether. If a processor can't get far enough to transfer a flowfile to a relationship, it can rollback to return the flowfile to the queue and optionally penalize

Re: Are we thinking about Penalization all wrong?

Joe, You bring up a great point. I realized after sending the initial e-mail that Processors still would need the ability to penalize a FlowFile in case of rollback. But I think this should be the only way that a Processor is able to penalize a FlowFile - to indicate that it will not process

Re: Are we thinking about Penalization all wrong?

That's a good point, Mark. I also agree that it's better to give the user control whenever possible. I imagine the RouteOnAttribute pattern to eventually "give up" on a FlowFile will be a common pattern, and so so we should account for that, rather than forcing the user into knowing this pattern.

Re: Are we thinking about Penalization all wrong?

The retry idea concerns me a bit. If we were to have a method like: penalizeOrTransfer(FlowFile flowFile, int numberOfTries, Relationship relationship) I think that leaves out some info - even if a FlowFile is penalized, it must be penalized and sent somewhere. So there would have to be a

[GitHub] nifi pull request: Nifi-Camel Integration

Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/186#issuecomment-176439666 Hello @joewitt PFA sample camel contexts.

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

Github user JPercivall commented on the pull request: https://github.com/apache/nifi/pull/180#issuecomment-176433804 In the latest commit there is more code coverage but there are still only a couple tests. Ideally I would like to see tests that work with each of the configuration

Re: Customer Questions

Kathy, Hello. NiFi's purpose is to act as a dataflow management tool. It certainly has support for scheduling of system executable commands which can do things like start/stop JVMs and initiate remote server reboots. F5 LTM provides a REST interface and NiFi's existing processors may well be

Re: Are we thinking about Penalization all wrong?

If we're willing to have a LoopFlowFile processor, why not consider a PenalizeFlowFile processor too? Just throwing it out for discussions sake, but penalization could ultimately be realized in multiple ways: a) by both the processor developer (and DFM via penalty duration), as it is done today;

Are there Travis CI interface issues?

*Is there a problem with GitHub server pulling status from the Travis CI build server?* I entered pull request #192 last night and this morning the PR list and PR#192 page show it as "Pending: The Travis CI build is

Re: Are there Travis CI interface issues?

GitHub was down at that time, probably prevented the Travis webhook from updating the status. Sent from my iPhone > On Jan 28, 2016, at 7:44 AM, Joe Skora wrote: > > *Is there a problem with GitHub server pulling status from the Travis CI > build server?* > > I entered pull

[GitHub] nifi pull request: NIFI-210: Add ExecuteScript and InvokeScriptPro...

Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/185#discussion_r51156891 --- Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/InvokeScriptProcessor.java --- @@

[GitHub] nifi pull request: NIFI-210: Add ExecuteScript and InvokeScriptPro...

Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/185#discussion_r51156976 --- Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/InvokeScriptProcessor.java --- @@

[GitHub] nifi pull request: NIFI-210: Add ExecuteScript and InvokeScriptPro...

Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/185#discussion_r51156818 --- Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/InvokeScriptProcessor.java --- @@

Re: Are we thinking about Penalization all wrong?

Matt thanks for your reply I guess what I am saying in that case - if there is an error in a FlowFile, then the processor that detects this cannot proceed so instead of calling an action to penalize the FlowFile it raises an exception OutOFServiceException or ProcessorException. You could have

[GitHub] nifi pull request: NIFI-1193: Add support for storing data in Hive...

Github user rdblue commented on the pull request: https://github.com/apache/nifi/pull/147#issuecomment-176297371 Looks like the build is working after the new commit. Thanks Ricky! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] nifi pull request: NIFI-210: Add ExecuteScript and InvokeScriptPro...

Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/185#discussion_r51156604 --- Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/InvokeScriptProcessor.java --- @@

[GitHub] nifi pull request: NIFI-1193: Add support for storing data in Hive...

Github user jskora commented on the pull request: https://github.com/apache/nifi/pull/147#issuecomment-176331346 I can also build master successfully now. Interestingly, it downloaded the hive-serde jar file from the Cloudera repository. Not sure why it didn't do that

[GitHub] nifi pull request: NIFI-1107 Multipart Uploads

Github user trkurc commented on the pull request: https://github.com/apache/nifi/pull/192#issuecomment-176546251 having trouble with my aws account - pretty much gone through the code, just trying to finish tests --- If your project is set up for it, you can reply to this email and

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r51129036 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearch.java

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r51129452 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearch.java

[GitHub] nifi pull request: NIFI-1275: Add processor(s) support for Elastic...

Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/180#discussion_r51130226 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearch.java

[GitHub] nifi pull request: NIFI-1423 Allow to penalize FlowFiles to No Ret...

Github user JPercivall commented on the pull request: https://github.com/apache/nifi/pull/183#issuecomment-176540108 When MockProcessSession.rollback(boolean)[1] is called it is, the boolean is telling the session whether or not the flowfiles should be penalized. In the event the

[GitHub] nifi pull request: NIFI-1423 Allow to penalize FlowFiles to No Ret...

Github user JPercivall commented on the pull request: https://github.com/apache/nifi/pull/183#issuecomment-176544259 I know this isn't part of the ticket but, there are numerous lines in the unit tests that say " // getMyFlowFiles();". I believe they're relicts from before the

[GitHub] nifi pull request: NIFI-1434 Prevent array index exception in PutE...

Github user JPercivall commented on the pull request: https://github.com/apache/nifi/pull/187#issuecomment-176558911 There 100% needs to be better unit testing for the PutEmail processor and I agree that this ticket is outside the scope of that. I've created a new one here[1].

[GitHub] nifi pull request: NIFI-1423 Allow to penalize FlowFiles to No Ret...

Github user JPercivall commented on the pull request: https://github.com/apache/nifi/pull/183#issuecomment-176533014 I believe flowfiles should be removed from the penalized list when MockProcessSession.remove() is called:

[GitHub] nifi pull request: NIFI-1423 Allow to penalize FlowFiles to No Ret...

Github user JPercivall commented on the pull request: https://github.com/apache/nifi/pull/183#issuecomment-176546233 Did a maven contrib check build and verified the functionality using a test http server. Once the comments are addressed I can merge in. --- If your project is set up

[GitHub] nifi pull request: NIFI-1333 fixed FlowController shutdown deadloc...

Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/148 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] nifi pull request: NIFI-1333 fixed FlowController shutdown deadloc...

Github user mcgilman commented on the pull request: https://github.com/apache/nifi/pull/148#issuecomment-176187058 With the github outage last night, ended up merging today. Thanks @trkurc and @olegz !!! --- If your project is set up for it, you can reply to this email and have your

[GitHub] nifi pull request: Nifi-Camel Integration

Github user olegz commented on a diff in the pull request: https://github.com/apache/nifi/pull/186#discussion_r51128670 --- Diff: nifi-nar-bundles/nifi-camel-bundle/nifi-camel-processors/src/main/java/org/apache/nifi/processors/camel/CamelProcessor.java --- @@ -0,0 +1,240 @@

[GitHub] nifi pull request: Nifi-Camel Integration

Github user olegz commented on a diff in the pull request: https://github.com/apache/nifi/pull/186#discussion_r51128900 --- Diff: nifi-nar-bundles/nifi-camel-bundle/nifi-camel-processors/src/main/java/org/apache/nifi/processors/camel/CamelProcessor.java --- @@ -0,0 +1,240 @@

[GitHub] nifi pull request: Nifi-Camel Integration

Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/186#issuecomment-176232391 @apiri Again OOM in travis: testExecuteIngestAndUpdate(org.apache.nifi.processors.standard.TestExecuteStreamCommand) Time elapsed: 1.717 sec <<<

[GitHub] nifi pull request: NIFI-1337: Add Riemann Reporting Task

Github user joewitt commented on a diff in the pull request: https://github.com/apache/nifi/pull/188#discussion_r51226391 --- Diff: nifi-nar-bundles/nifi-riemann-bundle/nifi-riemann-nar/pom.xml --- @@ -1,34 +1,40 @@ + Licensed to the Apache Software Foundation

[GitHub] nifi pull request: NIFI-1337: Add Riemann Reporting Task

Github user joewitt commented on a diff in the pull request: https://github.com/apache/nifi/pull/188#discussion_r51226519 --- Diff: nifi-nar-bundles/nifi-riemann-bundle/nifi-riemann-reporting-task/src/main/java/org/apache/nifi/reporting/riemann/RiemannReportingTask.java --- @@

[GitHub] nifi pull request: NIFI-1337: Add Riemann Reporting Task

Github user joewitt commented on a diff in the pull request: https://github.com/apache/nifi/pull/188#discussion_r51226675 --- Diff: nifi-nar-bundles/nifi-riemann-bundle/nifi-riemann-reporting-task/src/main/java/org/apache/nifi/reporting/riemann/RiemannReportingTask.java --- @@

[GitHub] nifi pull request: NIFI-1337: Add Riemann Reporting Task

Github user joewitt commented on a diff in the pull request: https://github.com/apache/nifi/pull/188#discussion_r51226918 --- Diff: nifi-nar-bundles/nifi-riemann-bundle/nifi-riemann-reporting-task/src/main/java/org/apache/nifi/reporting/riemann/RiemannReportingTask.java --- @@

[GitHub] nifi pull request: NIFI-1337: Add Riemann Reporting Task

Github user joewitt commented on a diff in the pull request: https://github.com/apache/nifi/pull/188#discussion_r51227063 --- Diff: nifi-nar-bundles/nifi-riemann-bundle/nifi-riemann-reporting-task/src/main/java/org/apache/nifi/reporting/riemann/metrics/MetricsService.java --- @@