NIFI var.startsWith("ch") bug?

i've found our system picks up files that would normaly be considered temporary/hidden files. files such as ".somename.txt" i've set the pickup file filter to exclude those but to no avail. i added a route on attribute to route to bucketA if it starts with a "." and everything else to another buck

Apache NIFI insert to MySQL help

I’m trying to do some stuff and send it to my MySQL database. I’m getting an exception (see below) but I’m fairly sure it’s just because I don’t have the right string or file located where it’s supposed to go. I’m not sure what to put in for the database connection URL or the driver class name..

RegEx not catching all tags

http://prntscr.com/basrzy the above is a screenshot showing a hashtags var only containing the first instance of a hashtag. i want to get a list of ALL hashtags from twitter.text not just the first one. i'm fairly sure my RegEx is wrong... here's what i have. (#{1}[a-zA-Z0-9_]*) i'm using https:

Re: RegEx not catching all tags

Thanks. I did some more reading in the documentation and Nifi's documentation says it only returns the first one. HOWEVER... The Jain object returned had an element of tags already! $.entities.hashtags.*.text or... Something. I got it working late last night! -Sven Davison (sent fr

getFile Content as json element

i've got a flow where i'm reading a file and setting various attributes. now i want to wrap the content of the origional file into a "content" element w/in my JSON object. this is invalid json but you'll get the idea. { filename: "somefile", fileTime: blahSomeDateString content:[ need cont

Re: getFile Content as json element

] > } > > The $1 is a back-reference that will reference whatever is in the first > capturing group of your Regular Expression (in this case, > the entire content of the FlowFile). > > Thanks > -Mark > > > > On Jun 1, 2016, at 1:10 PM, Sven Davison wrote: > > >

GetHTTP->ExtractText (Regex/User problem?)

I have looked at the example for extracting text. I seen the example pulls the content between the tags. I’ve changed it to pull from the tags w/o problem. The problem I’m having is pulling form something a bit more specific. I’m sure the problem is with my understanding/usage of REGEX. I’m t

RE: GetHTTP->ExtractText (Regex/User problem?)

ven,  give this a try: (.*?)<\/div> On Mon, Jun 20, 2016 at 10:25 AM, Sven Davison wrote: I have looked at the example for extracting text. I seen the example pulls the content between the tags. I’ve changed it to pull from the tags w/o problem. The problem I’m having is pulling form so

RE: GetHTTP->ExtractText (Regex/User problem?)

nts/org.apache.nifi.GetHTMLElement/index.html  these may be a little heavier on the processing, but will certainly save you a lot of problems with parsing. This lets you use css selectors against html, which is more intuitive and robust to parse HTML. Simon On 20 Jun 2016, at 18:43, Sven Davison wrote: I had tr

ExecuteProcess (fetch output)

I’ve been trying to run a script and fetch said output from the script as a variable work with. The process executes and I see the content in the body if I send it off to a LogAttribute process but I’m not sure how to get the output of the script as a variable. I want to be able to use this cont

Re: ExecuteProcess (fetch output)

t; of extracttext, but haven't found an example of it in use. -Sven Davison (sent from my iPhone) > On Jul 3, 2016, at 7:43 AM, Andrew Grande wrote: > > Sven, take a look at ExtractText component, it will allow you to promote a > result into an attribute. > > Andr

Re: ExecuteProcess (fetch output)

I tried regex of "." To grab everything. Works as intended on regexr.com. Not in nifi -Sven Davison (sent from my iPhone) > On Jul 3, 2016, at 9:04 AM, Sven Davison wrote: > > I've been looking at it. Google doesn't have many examples. Maybe I'm putt

Re: ExecuteProcess (fetch output)

No problem. The grouping was messing me up. I got it working in s round about way. Inserts into database. So for now I'll just do sql magic on the inserts/updates. Thanks! -Sven Davison (sent from my iPhone) > On Jul 3, 2016, at 9:53 AM, Matt Burgess wrote: > > A single do

Stuck Thread Monitor?

I have a custom processor that has a stuck thread from time to time. Once the thread's stuck, it fails to process data. Restarting the NiFi instance is enough to fix it... for a while. Without changing the processor, is there a way to monitor stuck threads w/in NiFi?

JsonSplit Question/Help

I’m trying to store each of the hash tags from a twitter post. They can be 0 or more and I’ve found the array via JSON but I’m not sure how to reference them other than via $twitter.hashtags[0].text that’s not very dynamic… as I never know how many there will be. Here’s a screenshot showing th

RE: JsonSplit Question/Help

Does it just become $twitter.hashtags without the index locators then? I’m not sure how to reference the content of the flowfile after it’s been split off. It LOOKS like it’s still the original full list vice element of the array. Sent from Mail for Windows 10 From: Matt Burgess

RE: JsonSplit Question/Help

WOOT! It took a minute but.. I got it! Thanks for the help! http://prntscr.com/bxx5jc Sent from Mail for Windows 10 From: Matt Burgess

NiFi PutSFTP

When it asks for the key.. is this just a regular openssh key? https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.PutSFTP/index.html

NiFi cluster error

I had a working cluster.. now when i try to restart a couple of the child nodes... it stops durring spin up. If i dont start the master node, the child will run until i DO start the master, then it dies. The message is the same in both conditions. "Failed to connect node to cluster due to: java.la

PutSQL ERROR bulletin

I’m getting several inserts but every once in a while (every 1-2 minutes or so)… I get this error. Anyone know what might cause this? PutSQL[id=b8d54aa6-567a-4686-96bc-1c00e5d43461] Failed to update database due to a failed batch update. There were a total of 1 FlowFiles that failed, 0 that suc

RE: PutSQL ERROR bulletin

Actualy… I’m fairly sure I found it. I sent stuff off to “logAttribute” processor and found the input is not escaped. http://prntscr.com/c51je3 Sent from Mail for Windows 10 From: Sven Davison

dynamic getTwitter ?

i have a GetTwitter processor which works wonders. I'm tracking a few people and a couple hash tags but i'm also pulling all hashtags out of the posts and tracking how many times i saw it and when the last time was that i saw it. example tweet: "hello world #earth #usa" if i'm watching #usa, i'll

Re: dynamic getTwitter ?

rg/jira/secure/CreateIssue!default.jspa > > Andy LoPresto > alopre...@apache.org > *alopresto.apa...@gmail.com * > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Aug 25, 2016, at 9:10 AM, Sven Davison wrote: > > i have a GetTwitter processor whic

Re: dynamic getTwitter ?

certainly > lessen the effect). Currently, filtering and such is very much integrated > with the lifecycle of the processor. A more dynamic processor could be > achieved, but will come with a few caveats. > > On Thu, Aug 25, 2016 at 1:03 PM, Sven Davison > wrote: > >&g

Re: dynamic getTwitter ?

t list. > > Matt Burgess has written a number of articles about this that are very > helpful [1][2]. > > [1] https://funnifi.blogspot.com/2016/02/executescript- > processor-hello-world.html > [2] https://funnifi.blogspot.com/2016/02/writing-reusable- > scripted-processors-in.html > &

Splitting Array out of JSON

I have a JSON object that i'm pulling back from ( http://redisq.zkillboard.com/listen.php ) i can get the victim.. by *$.package.killmail.victim.character.name * i want to pull something specific out of the attackers array. like all of attacker names.

Re: Splitting Array out of JSON

back against your provided input as you > enter the expression. > > Andy LoPresto > alopre...@apache.org > *alopresto.apa...@gmail.com * > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Jan 16, 2017, at 4:38 PM, Sven Davison wrote: > > I hav

Scientific Notation conversion?

I have a json object the SOMETIMES contains a scientific notation for the value. I want to have a nice or "human readable" number w/ thousands separator. example values: 1 2.23456789E6 2.1234567891E10 15.123456789E7 what i thought of doing was checking for the existance of E. then forking it

parsing html

I'm looking to parse some HTML. It's not the cleanest but i know that my content is always on line 10 of the file. I could use splittext then compare it to ensure it starts with XYZBeginningString, i supose.. but i'm looking for something w/ less overhead. Especially knowing the content is always o

Re: parsing html

d as you said, > decent overhead, especially if the desired content is early in the > flowfile. > > Andy LoPresto > alopre...@apache.org > *alopresto.apa...@gmail.com * > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Apr 11, 2017, at 9:38 AM,