Re: State Management

2016-01-11 Thread Mark Payne
In case any of you are following along here... In updating Processors to use the new state management API, I found that there were a few use cases that were a bit hard to accommodate with the proposed API so I have updated the API a bit, making it simpler, so that state is just retrieved/set by

Re: State Management

2016-01-11 Thread xmlking
Some open source projects are using Hazelcast's distributed map for state management. It provides standard Java map API. http://docs.hazelcast.org/docs/3.5/manual/html/map.html Sumo Sent from my iPad > On Jan 11, 2016, at 5:37 AM, Mark Payne wrote: > > In case any of you are following along h

Re: question regarding input stream from reddit.com

2016-01-11 Thread Joe Percivall
Hello Shahzad, Unfortunately the "stream" functionality of pushshift.io doesn't fit into any current NiFi processor. Processors work by having an "OnTrigger" method that is used to create FlowFiles with each call. This works nicely for aspects of the pushshit.io api like "https://api.pushshift

Re: [DISCUSS] Proposal for an Apache NiFi sub project - MiNiFi

2016-01-11 Thread Sumanth Chinthagunta
Good idea. There will be many possibilities if we can make MiNIFi run on android / iOS or other embedded devices. Wonder how back-pressure works in this kind of distributed setup. I was reading about reactivesocket project, This project is trying to solve reactive / back-pressure problem over n

Re: question regarding input stream from reddit.com

2016-01-11 Thread Aldrin Piri
Shahzad, Joe is correct in that we do not have anything that maps directly to this data stream source. As a means of getting the data into a NiFi flow, you could also consider https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ExecuteProcess/index.html processor

Re: Nifi + Oracle redo files?

2016-01-11 Thread jasondene
regarding: "Are you aware of any libraries that are open source friendly and support Oracle's redo logs?" No, I'm not aware of any such libraries that would meet our need. Will continue to search... thank you! -- View this message in context: http://apache-nifi-developer-list.39713.n7.nabbl

[GitHub] nifi pull request: NIFI-1283 Fixing ControllerStatusReportingTask ...

2016-01-11 Thread jvwing
GitHub user jvwing opened a pull request: https://github.com/apache/nifi/pull/166 NIFI-1283 Fixing ControllerStatusReportingTask logger name ControllerStatusReportingTask was using an abbreviated class name to prefix its loggers, "ControllerStatusReportingTask", instead of the fully

Patch Submitted - Re: Nifi-1325 - Enhancing Nifi AWS S3 for cross account access - Refactoring Nifi-AWS Processor credentials to use credentials provider

2016-01-11 Thread M Singh
Hey Folks: I've uploaded a patch for nifi-1325 (https://issues.apache.org/jira/browse/NIFI-1325).  I've added unit test cases and added integration tests (ignored in the checkin since they require aws resources arns) - that uses the credentials provider controller and they passed. Please let me

Re: Patch Submitted - Re: Nifi-1325 - Enhancing Nifi AWS S3 for cross account access - Refactoring Nifi-AWS Processor credentials to use credentials provider

2016-01-11 Thread Aldrin Piri
Mans, Thanks for your continued efforts on this. I'll get this applied and start scoping it out. On Mon, Jan 11, 2016 at 4:54 PM, M Singh wrote: > Hey Folks: > I've uploaded a patch for nifi-1325 ( > https://issues.apache.org/jira/browse/NIFI-1325). I've added unit test > cases and added inte

Re: Groovy unit tests

2016-01-11 Thread Andy LoPresto
Thanks to everyone who weighed in. This feature is documented in NIFI-1365 [1] and there is a patch available [2]. The tests do not run by default and are triggered with a Java variable named `groovy` being set to `test`. It can be invoked as follows: `mvn clean test -Dgroovy=test` [1] https:/

Re: [DISCUSS] roadmap for the next 6-12 months

2016-01-11 Thread Sean Busbey
On Fri, Jan 8, 2016 at 8:29 AM, Joe Witt wrote: > We should also have a discussion on how long we should be committed to > supporting the 0.x line and what that means. We need to document a > commitment for the community. Worth a dedicated thread? Presuming that we're going to use "1.0.0" as t