Re: nifi-hive-nar-1.7.1.nar will not load

This has been biting a few users lately, not sure when it changed exactly, but the Hive NAR uses a version of Snappy that tries to extract the native Snappy library into a location pointed to by the "java.io.tmpdir" variable, which IIRC is /tmp/. The /tmp directory sometimes has a noexec restrictio

Hive w/ Kerberos Authentication starts failing after a week

We are seeing frequent failures of our Hive DBCP connections after a week of use when using Kerberos with Principal/Keytab. We've tried with both the Credential Service and without (though in looking at the code, there should be no difference). It looks like the tickets are expiring and renewal

Re: High CPU load upon starting non-connected Out Port inside PG

Thanks, good to know! We had a rather complex flow and took us a while to figure this one out :) best KT tor. 26. jul. 2018 kl. 16:16 skrev Mark Payne : > KT, > > I can confirm that this is the behavior I'm seeing as well. I went ahead > and created a JIRA [1] > for this. I think the bug really

Re: nifi-hive-nar-1.7.1.nar will not load

Here is the error: 2018-07-26 18:48:49,013 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider org.apache.nifi.processors.hive.PutHiveStreaming could not be instantiated java.util.ServiceConfigurationErr

Re: nifi-hive-nar-1.7.1.nar will not load

Geoff Dont think it is a known issue. Many of us tested the build/startup/etc.. Can you share the log output? Thanks On Thu, Jul 26, 2018 at 2:08 PM, geoff.craig wrote: > Hello, > > I did a clean install of NiFi 1.7.1 and nifi-hive-nar-1.7.1.nar will not > load. It throws a Java error. I ha

Re: nifi-hive-nar-1.7.1.nar will not load

Geoff, What error did you get? Please share it with us here. On Thu, Jul 26, 2018 at 11:38 PM geoff.craig wrote: > Hello, > > I did a clean install of NiFi 1.7.1 and nifi-hive-nar-1.7.1.nar will not > load. It throws a Java error. I had to remove it from the folder to get > NiFi running. Is

nifi-hive-nar-1.7.1.nar will not load

Hello, I did a clean install of NiFi 1.7.1 and nifi-hive-nar-1.7.1.nar will not load. It throws a Java error. I had to remove it from the folder to get NiFi running. Is this a known issue? -- Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/

Re: NiFi Data Usage via Rest API

Ryan, That is correct. Would just clarify that when you say "SEND events are when they are leaving the system" -- the data is being sent to an external system, but it is not being dropped from NiFi. So you could send the data to 10 different places. A "DROP" event indicates that NiFi is now fini

Re: NiFi Data Usage via Rest API

Hi Mark, Thanks for the explanation on this; this is what I was looking for. So it sounds like Provenance info is the way to go (as mentioned by Mike [thanks Mike]). I will have to do a little more research on the Provenance events, but it sounds like RECEIVE events are for when something is comin

Re: High CPU load upon starting non-connected Out Port inside PG

KT, I can confirm that this is the behavior I'm seeing as well. I went ahead and created a JIRA [1] for this. I think the bug really is in the fact that we allow you to start the Port at all. Just like some Processors are annotated as Requiring Input in order to be valid, ports should be too (u

Re: NiFi Data Usage via Rest API

Hey Ryan, The stats that you are seeing here is a rolling 5-minute window. The "bytesReceived" indicates the number of bytes that were received from external systems (i.e., the number of bytes reported as Provenance RECEIVE events). The "bytesSent' indicates the number of bytes that were sent t

High CPU load upon starting non-connected Out Port inside PG

Hi, First, thanks alot for a great product! :) My issue is this. Create a PG, inside it create an out-port and connect it to another out-port outside the PG. Start the out-port inside the PG. My CPU load then sky-rockets (from ~5-10% to 200-300% on my laptop to 500-1000% on my servers) :/ If I ho

Re: NiFi Data Usage via Rest API

Hi Matt, The use case that I am investigating is fairly simplistic (and I may be naive about it). I am only looking for the amount of data that has came in to the cluster (across all PG's) and out of the cluster for a given time period (or a way to derive based on a time period). I do not want to

RE: Unable to see Nifi data lineage in Atlas

Hi, While looking at the logs, I found out that ReportingLineageToAtlas is not able to construct KafkaProducer. It throws the following logs - org.apache.kafka.common.KafkaException: Failed to construct kafka producer at org.apache.kafka.clients.producer.KafkaProducer.(KafkaPr

Re: NiFi Data Usage via Rest API

Matt, Our main use, which provenance data handles well, is figuring out **what** data was handled. We drop everything but DROP out of convenience because we have no known scenarios where data will be removed before it reaches the end of the flow. FWIW, this is what inspired the record stats proce