Re: Testing a nifi flow via junit

2016-01-04 Thread Aldrin Piri
Hello Vincent, This is something that does not exist and there have been a few threads on this topic [1][2]. Summarily, these tools do not currently exist due to the preference of using the interactive and real-time command and control over the flow as well as the increasing difficulty of

Testing a nifi flow via junit

2016-01-04 Thread Vincent Russell
All, I see that there is a way to test a single processor with the TestRunner (StandardProcessorTestRunner) class, but is there a way to set up an integration test to test a complete flow or a subset of a flow? Thank you, Vincent

Re: Upgrade 0.3.0 to 0.4.1

2016-01-04 Thread Aldrin Piri
brew update is certainly doing most of the heavy lifting. Additionally, use a `brew outdated` to show which installed formula are currently lagging behind the current version. Also very helpful, but not one of the commands listed in the example usage listing. On Mon, Jan 4, 2016 at 11:10 PM,

Re: Data Ingestion forLarge Source Files and Masking

2016-01-04 Thread Joe Witt
Obaid, Really happy you're seeing the performance you need. That works out to about 110MB/s on average over that period. Any chance you have a 1GB NIC? If you really want to have fun with performance tuning you can use things like iostat and other commands to observe disk, network, cpu.

Re: Upgrade 0.3.0 to 0.4.1

2016-01-04 Thread Christopher Hamm
when does brew update? On Jan 4, 2016 9:08 PM, "Joe Witt" wrote: > Hello Chakri, > > Take a look here > https://cwiki.apache.org/confluence/display/NIFI/Upgrading+NiFi > > And here > https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance > > Thanks > Joe > >

Re: Data Ingestion forLarge Source Files and Masking

2016-01-04 Thread obaidul karim
Hi Joe, Just completed by test with 100GB data (on a local RAID 5 disk on a single server). I was able to load 100GB data within 15 minutes(awesome!!) using below flow. This throughput is enough to load 10TB data in a day with a single and simple machine. During the test, server disk I/O went up

Re: Upgrade 0.3.0 to 0.4.1

2016-01-04 Thread Christopher Hamm
brew update and brew doctor is my friend. On Jan 4, 2016 10:11 PM, "Aldrin Piri" wrote: > This occurs as shortly after the release is official and the formula > update gets incorporated into the main homebrew repository by their group > of committers. > > 0.4.1 is available

Re: Upgrade 0.3.0 to 0.4.1

2016-01-04 Thread Joe Witt
Hello Chakri, Take a look here https://cwiki.apache.org/confluence/display/NIFI/Upgrading+NiFi And here https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance Thanks Joe On Mon, Jan 4, 2016 at 8:02 PM, Chakrader Dewaragatla wrote: > Hi –

polling nifi status -- nifi reports not running every 10 to 20 min

2016-01-04 Thread Whitmore, Mattie [USA]
Hi, In order to be immediately alerted when/if nifi goes down we are polling bin/nifi.sh status every 5 min. It is stating that the server is down every 10 to 20 min, however I cannot repeat this message if I ask for a status myself, and the UI does not seem to be experiencing any

Re: polling nifi status -- nifi reports not running every 10 to 20 min

2016-01-04 Thread Juan Sequeiros
Good afternoon, We monitor our instance through a cron and checking its command output using service nifi status. I can only assume that you might have external factors causing the result to not meet true. I suggest that you also send the return result to a file to inspect it closer. So if you