Re: Open / Close Gate examples?

2019-05-28 Thread Evan Graham
Hi Dave, You should be able to do this by sending a Notify with Signal Counter Delta = 0 immediately after the Wait to close the gate and then another Notify with Signal Counter Delta = 1 after you have finished processing to reopen the gate. You will need to create a separate Notify with Signal

Re: Debugging NiFi Flows in Docker - problem with ListFile?

2019-05-28 Thread Stephen Greszczyszyn
OK, it seems like NiFi was actually trying to find files in the directory, although there were way too many and it was running out of memory... I guess I need to run something to archive the old logs and start with only the new ones. Thanks for your help Mark! On Tue, 28 May 2019 at 21:33,

Re: Debugging NiFi Flows in Docker - problem with ListFile?

2019-05-28 Thread Stephen Greszczyszyn
OK, thanks, this us useful information, I did also test the filenames with a regex parser, but didn't know NiFi was different. I really don't need an expression anyway as I want to get all the files. I'm wondering if the problem is that I have too many files in the directory, so I'll just set

RE: Open / Close Gate examples?

2019-05-28 Thread David Gallagher
Thanks, Shawn – I had seen your post to the mailing list and that’s where I got the gist. It’s really close to what I need to do, but it seems to be designed for a scenario where the data is arriving on a given interval. In my case, I could have multiple messages arriving at the Wait processor

Re: Debugging NiFi Flows in Docker - problem with ListFile?

2019-05-28 Thread Mark Payne
Stephen, Egrep evaluates regex'es very differently than NiFi does. With egrep, any line that contains text matching the regex will be output. With NiFi, however, the entire line of text must match the regex exactly in order to be selected. So '^c.r' will not match but '^c.r.*' will. Thanks

Re: Debugging NiFi Flows in Docker - problem with ListFile?

2019-05-28 Thread Stephen Greszczyszyn
I don't think that is the problem as I have checked the regex and can find the files (this is from inside the container so I also can read the directory): nifi@4f614c6c67c4:/data/sftp/cdr$ ls -1t | head -5 | egrep '^c.r' cdr_AP2-HKG-CCP-UCM_12_201905281539_35251

Re: Debugging NiFi Flows in Docker - problem with ListFile?

2019-05-28 Thread Mark Payne
Stephen, The regex that you have there in interpreted by NiFi as "the filename must consist of 3 characters, beginning with a c and ending with an r". What you want is "the filename must begin with a c, followed by any character, followed by an r, followed by anything else." So you'd want

Debugging NiFi Flows in Docker - problem with ListFile?

2019-05-28 Thread Stephen Greszczyszyn
Hi there, I'm running NiFi in docker. I'm trying to enable flow debugs and also to determine if NiFi can actually see in my data directory path. I'm trying to use a simple recursive list file processor to find the files based on a regular expression "^c.r". I can docker exec into the container

Re: Would a currency parser record path function be useful?

2019-05-28 Thread Andrew Grande
Honestly, I have seen so many unique ways to mangle the currency format, that I'm not even sure one can reliably parse one anymore. But more importantly, isn't it the responsibility of a representation layer to format and pretty print the value? Just have hard time seeing a widespread need for

Would a currency parser record path function be useful?

2019-05-28 Thread Mike Thomsen
We have a controller service that does some basic cleanup on currency to take a string and turn it into a float or double. It's not intended to be anything like the Java Money API, it just purges everything except raw currency-related characters and formats the text into a float or double. Would