Re: Putdatabase Record - Value too large for column

One way you might be able to get there would be to add a SplitRecord on the failure relationship of the processor and have it loop its output back to the processor so you can narrow down which record was failing. On Sun, Oct 25, 2020 at 1:58 PM KhajaAsmath Mohammed wrote: > > Hi, > > I am using p

Re: Putdatabase Record - Value too large for column

Hi Mike, I do the same during the failures but it is hard to know which Column it is. We have around 90 columns and this error can occur to any of those fields. If the error message mentions about the column that would be really helpful to expand our size. Thanks, Asmath Sent from my iPhone

Run Nifi in IntelliJ to debug?

Hi Is it possible to run Nifi from inside IntelliJ with debugging such that I can hit the app from my browser and trigger breakpoints? If anyone has done this can you please share any info? Thanks in advance! Darren Sent from my Verizon, Samsung Galaxy smartphone Get Outlook for Android

Re: Run Nifi in IntelliJ to debug?

Yes, that's a pretty common operation amongst NiFi developers. In conf/bootstrap.conf there's a section called Enable Remote Debugging and a commented-out line something like: java.arg.debug=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005 You can remove the comment from that li

Re: Run Nifi in IntelliJ to debug?

Sorry I misread the part where you wanted to run NiFi inside IntelliJ, I was talking about running it externally (from the command-line, e.g.) and connecting the IntelliJ debugger. I haven't run NiFi itself using IntelliJ, maybe someone else can chime in for that. On Mon, Oct 26, 2020 at 12:03 PM

RE: NiFi 1.11.4 HDFS/HBASE Processor Errors After Kerberos Ticket Expires

That seemed to have fixed it. Thank you! From: Mark Payne Sent: Thursday, October 22, 2020 11:54 To: users@nifi.apache.org Subject: Re: NiFi 1.11.4 HDFS/HBASE Processor Errors After Kerberos Ticket Expires This line in your bootstrap.conf appears to be the problem: java.arg.16=-Djava

Re: Run Nifi in IntelliJ to debug?

Thanks Matt. I think if i can attach remotely and step through the code that will satisfy my needs. Let me give it a try. I also found how to run mvnDebug and attach to that from intellij. Just need to find a maven goal that runs nifi but i havent seen one yet. Sent from my Verizon, Samsung Gal

Re: Run Nifi in IntelliJ to debug?

Hi, Darren - I am not aware of a method of running a full NiFi instance directly in IntelliJ, but I use the method Matt mentioned: attaching IntelliJ as a remote debugger to a running NiFi instance. I put together a guide a few years back for folks unfamiliar with this process or new to Intell

Re: Run Nifi in IntelliJ to debug?

Hi! Here instructions for running MiNiFi (debugging) within IntelliJ IDEA, practically the same steps for NiFi. shorturl.at/bkloI -- Javi Roman Twitter: @javiromanrh GitHub: github.com/javiroman Linkedin: es.linkedin.com/in/javiroman Big Data Blog: dataintensive.info On Mon, Oct 26, 2020 at 6

something not closing correctly in a session

I have a groovy scripts that I am running inside of a ExecuteGroovyScript processor (nifi v 1.12.0) I've been following the examples at https://community.cloudera.com/t5/Community-Articles/ExecuteScript-Cookbook-part-2/ta-p/249018, and I have something like: flowFile = session.get() if(!flowFi

Re: something not closing correctly in a session

Hello, You can't call session.transfer inside the OutputStreamCallback, it needs to be done after the OutputStreamCallback is complete so that you are transferring the updated reference to flowFile. Thanks, Bryan On Mon, Oct 26, 2020 at 3:49 PM Greene (US), Geoffrey N wrote: > > I have a groo

Re: Run Nifi in IntelliJ to debug?

Darren, I was out this morning and didn't see your plea until I got in just now. Here's a step by step I wrote up for both IntelliJ IDEA and Eclipse (I'm more an IntelliJ guy). It also covers using an IP tunnel. https://www.javahotchocolate.com/notes/nifi.html#20160323 On 10/26/20 9:52 AM, D

RE: [EXTERNAL] Re: something not closing correctly in a session

Thank you! That was it -Original Message- From: Bryan Bende [mailto:bbe...@gmail.com] Sent: Monday, October 26, 2020 3:55 PM To: users@nifi.apache.org Subject: [EXTERNAL] Re: something not closing correctly in a session This message was sent from outside of Boeing. Please do not click l

Re: Run Nifi in IntelliJ to debug?

If you want to use the Docker image, add "-e NIFI_JVM_DEBUGGER=1" and map port 8000 to something on your machine. On Mon, Oct 26, 2020 at 4:10 PM Russell Bateman wrote: > > Darren, > > I was out this morning and didn't see your plea until I got in just now. > Here's a step by step I wrote up for

Re: Run Nifi in IntelliJ to debug?

Kevin/Russel Thanks for the info. I did set things up this way. IntelliJ does connect to the nifi jvm and nifi runs and works but intellij isnt breaking on code it should. I did set the module where the code/classes are located (in the remote connection dialog) and i see the exception im track

Re: Run Nifi in IntelliJ to debug?

Darren, This is just Apache NiFi code out of NARs you want to step through or is it yours? You haven't stripped debug information or anything, right? Russ On 10/26/20 5:30 PM, Darren Govoni wrote: Kevin/Russel Thanks for the info. I did set things up this way. IntelliJ does connect to the

Re: Run Nifi in IntelliJ to debug?

Correct. Primarily the nifi-web-api module and AccessResource class. For starters. Sent from my Verizon, Samsung Galaxy smartphone Get Outlook for Android From: Russell Bateman Sent: Monday, October 26, 2020 7:37:13 PM To: Darren Govoni ;

Re: Run Nifi in IntelliJ to debug?

Hmmm... It's rare that I debug NiFi code. And it's also rare that I debug my own in that context since the NiFi test runner allows me to fend off most surprises via my JUnit tests. I think back in 2016, I was debugging a start-up problem involving NiFi start-up and incompatibility with the Jav

Re: Run Nifi in IntelliJ to debug?

Are you using a binary derived from the source code in your IDE? Like a 1.12.1 binary and the source code from the release? On Mon, Oct 26, 2020 at 7:47 PM Russell Bateman wrote: > > Hmmm... It's rare that I debug NiFi code. And it's also rare that I debug my > own in that context since the NiFi

Re: NIFI - Wait before merging the files

I have a case where I have a single stream of data items that need merged into a single file. I do this by setting the number of bins in merge to 1 and using the cron scheduler to run the merge every 15 min. I never got the bin age to work the way I wanted. I set the number of flowfiles to inclu

Re: NIFI - Wait before merging the files

Thanks David, I have the same issue. Was never able to get it work with merge record directly . Since my flow has stream of data, I can’t implement cron in between the job. I might have to see if groovy script has any option to wait for specified amount of time before merge operation Sent from

Re: NIFI - Wait before merging the files

Can you expand on your use case? Do you need to wait for a specific amount of time after receiving the first flow file? Why would a scheduled run every 2 min not work? Is the issue that you need all related flowfiles merged together? Dave On Mon, Oct 26, 2020, 9:25 PM KhajaAsmath Mohammed wro