FOSDEM 2016 - take action by 4th of December 2015

2015-11-30 Thread Roman Shaposhnik
As most of you probably know FOSDEM 2016 (the biggest, 100% free open source developer conference) is right around the corner: https://fosdem.org/2016/ We hope to have an ASF booth and we would love to see as many ASF projects as possible present at various tracks (AKA Developer rooms): htt

Re: remote command execution via SSH?

2015-11-30 Thread Sumanth Chinthagunta
Sure Joe. I will create Jira tickets for those processors . I am also working on to move groovy lib dependency to parent nar level to keep processor nars sleek. Sumo Sent from my iPhone > On Nov 30, 2015, at 7:25 AM, Joe Percivall > wrote: > > Hey Sumo, > > I don't know much about this us

Re: Provenance tuning

2015-11-30 Thread Joe Witt
Ian, You can effectively turn off the NiFi provenance reporter by switching to an in-memory repository. So for example edit ./conf/nif.properties Change this line nifi.provenance.repository.implementation=org.apache.nifi.provenance.PersistentProvenanceRepository To nifi.provenance.rep

Re: Provenance tuning

2015-11-30 Thread ianwork
My instance is running on a VM with 8cpu's. The repositories are all stored on the same virtual disks. a snapshot of io when the throttling kicks in Device: tpskB_read/skB_wrtn/skB_readkB_wrtn vda 133.40 0.00 56982.40 0 284912 On Mon,

Re: Provenance tuning

2015-11-30 Thread Aldrin Piri
Ian, Could you provide information as to how your repositories are configured? Specifically, what the layout is to physical volume(s)/disks? On Monday, November 30, 2015, ianwork wrote: > I am also experiencing a slowdown with the provenance reporter and would > like > information on tuning. M

[GitHub] nifi pull request: added embedded Kafka server and tests

2015-11-30 Thread olegz
GitHub user olegz opened a pull request: https://github.com/apache/nifi/pull/134 added embedded Kafka server and tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/olegz/nifi NIFI-1219 Alternatively you can review and apply t

Re: Provenance tuning

2015-11-30 Thread ianwork
I am also experiencing a slowdown with the provenance reporter and would like information on tuning. My application is performing log processing of 20k flowfiles a second. As a workaround batch processing was implemented but I would also like to see if it can be done without batching. I would pre

Re: Clustered Site-toSite

2015-11-30 Thread Matthew Clarke
What are the debug logs telling you on the NCM and Nodes when you have it configured as (4)? I would be curious as to what the debug logs say on the sending NiFi when you add a RPG using http://56.72.192.81:8080/nifi? If it accepts the URL and adds the RPG to the graph, then the communication wit

Re: Should we have an easier way to implement ListenBlah processors?

2015-11-30 Thread Andre
Hi Joe, Damn! I should have suspected it... :-( Anyhow, the code is starting to take shape here : https://github.com/trixpan/nifi-lumberjack-bundle (I will send a PR when I get closer to complete) I believe I've cleaned most of the netty dependencies, except for the issues I faced trying to get

[GitHub] nifi pull request: NIFI-989: Support size() operation in distribut...

2015-11-30 Thread joemeszaros
Github user joemeszaros commented on the pull request: https://github.com/apache/nifi/pull/94#issuecomment-160670895 Ok, so we should wait for the Java 8 migration. We can reopen the PR at any time. --- If your project is set up for it, you can reply to this email and have your repl

[GitHub] nifi pull request: NIFI-989: Support size() operation in distribut...

2015-11-30 Thread joemeszaros
Github user joemeszaros closed the pull request at: https://github.com/apache/nifi/pull/94 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

Re: Clustered Site-toSite

2015-11-30 Thread Edgardo Vega
I am going to describe the machines and then what we tried. We have a cluster of 4 machines Hostnameinternal ip external ip master 176.0.1.128 56.72.192.81 node-1 176.0.1.130 56.72.192.123 node-2 176.0.1.131 56.72.192.45 node-3 17

Re: remote command execution via SSH?

2015-11-30 Thread Joe Percivall
Hey Sumo, I don't know much about this use-case but just taking a quick look the processors in that github repo they seem to be potentially a great addition to NiFi! I think you should consider creating a Jira and working this there. It would a lot easier to get feedback and have a record of i