Re: Problem with InvokeHTTP not timing out

[Jens M. Kofoed]

Thanks for the reply Mark,
I have tried dumping the threads, but I'm not able to find information
about that processor. and unfortunately the system having problems is
running on a sensitive system with no internet access and I'm not allowed
to transfer data from that system to the internet.
Looking in the thread file for InvokeHTTP I can find 3 Timer-Driven Process
Thread-nn with an ID which I can't relate to the process in the UI. 1
thread is RUNNING and 2 are WAITING.
After I stop and terminate the processor and dump to a new file one of the
waiting processes is gone, but the other 2 (1 running and 1 waiting) are
also in the new dump file.
Looking at the dump files and comparing the 2 waiting processors there is
nothing which pops right in my face saying here are some things wrong.

kind regards
Jens M. Kofoed


Den tor. 15. okt. 2020 kl. 16.17 skrev Mark Payne :

> Jens,
>
> If you encounter an issue where a processor seems ’stuck’ the best course
> of action is generally to grab a thread dump (bin/nifi.sh dump
> thread-dump1.txt) and attach that. It will show exactly what the processor
> is doing at that time and help to understand why where it’s getting stuck.
>
> Thanks
> -Mark
>
>
> On Oct 13, 2020, at 6:46 AM, Jens M. Kofoed 
> wrote:
>
> Hi community
> I have some issues with the InvokeHTTP process. Sometimes the process does
> not receive a response from the web server and the process hangs in a
> waiting state without timing out.
> I use nifi version 1.12.1, and the settings for the InvokeHTTP process is
> as follow:
> Penalty duration 30 secs
> Yield duration 1 sec
> Scheduling strategy: Timer driven
> concurrent Taskts: 1
> Run schedule: 1 sec
> Run duration: 0ms
> HTTP metode: GET
> Connection timeout: 5 secs
> Read timeout: 30 secs
> Idle timeout: 2 mins.
> Max Idle connections 1
>
> Looking in the nifi-app.log file for debug messages I can see many
> requests followed by a response from the web server. When the UI no longer
> showed any files going through I tried to stop the process and looked in
> the log file.
> In the logfile I can see the request to the web server, but no response. I
> am expecting a read timeout but nothing happens.
> When trying to stop the process in the UI, the process goes from 1 task to
> 2 tasks and after 10 min. the process has still not stopped. So I have to
> Terminate it.
>
> Can anyone please help?
> I have tried to create a bug report, but it is difficult when there are no
> error messages.
> https://issues.apache.org/jira/browse/NIFI-7899
>
> Kind regards
> Jens M. Kofoed
>
>
>

Re: Securing MiNiFi

[scotty]

You were right. Thanks for your help.



--
Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/

Re: Securing MiNiFi

[Andy LoPresto]

We’ll try to clean up the NiFi docs, but vendor documentation is out of our 
control. Glad it’s working for you now. 

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
He/Him
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Oct 21, 2020, at 17:46, scotty  wrote:
> 
> That was it!  I didn't have nifi.remote.input.http.enabled= true.
> 
> I was going off of the link below where it says "By default, it is set to
> true." I guess I misread it to mean that if nothing was specifically entered
> there then it would default to true. 
> 
> https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.1.2/bk_administration/content/site_to_site_properties.html
> 
> 
> 
> Anyway, thank you, greatly, for the help.
> 
> 
> 
> 
> --
> Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/

Re: Securing MiNiFi

[scotty]

That was it!  I didn't have nifi.remote.input.http.enabled= true.

I was going off of the link below where it says "By default, it is set to
true." I guess I misread it to mean that if nothing was specifically entered
there then it would default to true. 

https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.1.2/bk_administration/content/site_to_site_properties.html



Anyway, thank you, greatly, for the help.




--
Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/

PrometheusReportingTask Metrics

[Ritch, David]

Is there a description of each of the metrics provided by
the PrometheusReportingTask available somewhere?

Thank you!

-- 

David Ritch
Principal Software Engineer

WaveStrike/Analytics Capabilities Division





300 Sentinel Dr. Ste. 210

Annapolis Junction, MD 20701
Email dri...@novetta.com

Office (443) 661-4810x1173
Mobile (443) 718-9327

Re: NiFi 1.11.4 HDFS/HBASE Processor Errors After Kerberos Ticket Expires

[Joe Witt]

If there is nothing in the logs but they stop working I suspect the issue
is related the default prompt for name.  Update settings in bootstrap is
most likely needed.

Thanks

On Wed, Oct 21, 2020 at 3:36 PM Peter Turcsanyi 
wrote:

> Are there any exception stack traces in the log when the processors fail /
> before that?
>
> On Thu, Oct 22, 2020 at 12:28 AM  wrote:
>
>> Hello!
>>
>>
>>
>> We’re running into a problem with NiFi 1.11.4.
>>
>>
>>
>> Our HBASE/HDFS/Parquet processors are all configured with a master
>> KeytabCredentialsService that is pointing to a Kerberos principal and
>> keytab file.
>>
>>
>>
>> The environment’s /etc/krb5.conf file has the line renew_lifetime = 7d
>> commented out due to an issue with Java-OpenJDK (that is apparently fixed
>> but still shows up) causing “MESSAGE STREAM MODIFIED (41)” errors to appear
>> whenever we have it uncommented.
>>
>>
>>
>> When NiFi starts, it is able to kinit with the Kerberos KDC and is issued
>> a 24 hour ticket. Everything works fine right up until that ticket expires.
>> Once the ticket expires, all of our HDFS/HBASE/Parquet processors start
>> failing.
>>
>>
>>
>> I haven’t been able to find anything in our logs around the timeframe,
>> but I can’t turn on debug logging for this because the logs are
>> tremendously large when we do that (approximately 100-200 MB per minute and
>> the problem only occurs at the 24 hour mark).
>>
>>
>>
>> How would we go about troubleshooting this issue?
>>
>>
>>
>> Environment:
>>
>> Red Hat Enterprise Linux 7.9
>>
>> Apache NiFi 1.11.4
>>
>> java-11-openjdk 11.0.8.10-1.el7
>>
>>
>>
>> Thanks!
>>
>>
>>
>>
>>
>

Re: NiFi 1.11.4 HDFS/HBASE Processor Errors After Kerberos Ticket Expires

[Peter Turcsanyi]

Are there any exception stack traces in the log when the processors fail /
before that?

On Thu, Oct 22, 2020 at 12:28 AM  wrote:

> Hello!
>
>
>
> We’re running into a problem with NiFi 1.11.4.
>
>
>
> Our HBASE/HDFS/Parquet processors are all configured with a master
> KeytabCredentialsService that is pointing to a Kerberos principal and
> keytab file.
>
>
>
> The environment’s /etc/krb5.conf file has the line renew_lifetime = 7d
> commented out due to an issue with Java-OpenJDK (that is apparently fixed
> but still shows up) causing “MESSAGE STREAM MODIFIED (41)” errors to appear
> whenever we have it uncommented.
>
>
>
> When NiFi starts, it is able to kinit with the Kerberos KDC and is issued
> a 24 hour ticket. Everything works fine right up until that ticket expires.
> Once the ticket expires, all of our HDFS/HBASE/Parquet processors start
> failing.
>
>
>
> I haven’t been able to find anything in our logs around the timeframe, but
> I can’t turn on debug logging for this because the logs are tremendously
> large when we do that (approximately 100-200 MB per minute and the problem
> only occurs at the 24 hour mark).
>
>
>
> How would we go about troubleshooting this issue?
>
>
>
> Environment:
>
> Red Hat Enterprise Linux 7.9
>
> Apache NiFi 1.11.4
>
> java-11-openjdk 11.0.8.10-1.el7
>
>
>
> Thanks!
>
>
>
>
>

NiFi 1.11.4 HDFS/HBASE Processor Errors After Kerberos Ticket Expires

[jw4306295]

Hello!

 

We're running into a problem with NiFi 1.11.4.

 

Our HBASE/HDFS/Parquet processors are all configured with a master
KeytabCredentialsService that is pointing to a Kerberos principal and keytab
file. 

 

The environment's /etc/krb5.conf file has the line renew_lifetime = 7d
commented out due to an issue with Java-OpenJDK (that is apparently fixed
but still shows up) causing "MESSAGE STREAM MODIFIED (41)" errors to appear
whenever we have it uncommented.

 

When NiFi starts, it is able to kinit with the Kerberos KDC and is issued a
24 hour ticket. Everything works fine right up until that ticket expires.
Once the ticket expires, all of our HDFS/HBASE/Parquet processors start
failing. 

 

I haven't been able to find anything in our logs around the timeframe, but I
can't turn on debug logging for this because the logs are tremendously large
when we do that (approximately 100-200 MB per minute and the problem only
occurs at the 24 hour mark). 

 

How would we go about troubleshooting this issue?

 

Environment:

Red Hat Enterprise Linux 7.9

Apache NiFi 1.11.4

java-11-openjdk 11.0.8.10-1.el7

 

Thanks! 

 

 

Re: Securing MiNiFi

[Andy LoPresto]

I think you need to at least populate one of, if not both, 
nifi.remote.input.socket.port= and nifi.remote.input.http.enabled=. If there is 
no socket port specified, and HTTP transmission is not enabled, I don’t think 
S2S will work. 

I also don’t see any keystore or truststore configured; did you just not share 
that portion?


Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
He/Him
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Oct 21, 2020, at 11:18 AM, scotty  wrote:
> 
> Hi,
> 
> Here's the info:
> 
> # Site to Site properties
> 
> nifi.remote.input.host=mydomain.com
> nifi.remote.input.secure=true
> nifi.remote.input.socket.port=
> nifi.remote.input.http.enabled=
> nifi.remote.input.http.transaction.ttl=30 sec
> nifi.remote.contents.cache.expiration=30 secs
> 
> # web properties #
> 
> nifi.web.war.directory=./lib
> #nifi.web.http.host=mydomain.com
> #nifi.web.http.port=8080
> nifi.web.http.network.interface.default=
> nifi.web.https.host=mydomain.com
> nifi.web.https.port=9443
> nifi.web.https.network.interface.default=
> nifi.web.jetty.working.directory=./work/jetty
> nifi.web.jetty.threads=200
> nifi.web.max.header.size=16 KB
> 
> 
> 
> 
> 
> --
> Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/

Re: Securing MiNiFi

[scotty]

Hi,

Here's the info:

# Site to Site properties

nifi.remote.input.host=mydomain.com
nifi.remote.input.secure=true
nifi.remote.input.socket.port=
nifi.remote.input.http.enabled=
nifi.remote.input.http.transaction.ttl=30 sec
nifi.remote.contents.cache.expiration=30 secs

# web properties #

nifi.web.war.directory=./lib
#nifi.web.http.host=mydomain.com
#nifi.web.http.port=8080
nifi.web.http.network.interface.default=
nifi.web.https.host=mydomain.com
nifi.web.https.port=9443
nifi.web.https.network.interface.default=
nifi.web.jetty.working.directory=./work/jetty
nifi.web.jetty.threads=200
nifi.web.max.header.size=16 KB





--
Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/

NiFi on Kubernetes

[muhyid72]

Dear All,
I am trying to run standalone NiFi 1.12.1 (not cluster) on Kubernetes. I am
planning to migrate my NiFi instance from VM to Kubernetes. 
I did container PoC on Docker and it was successfully completed. Storage
side was really simple I used mount parameter for persistency required
folders like conf, state, flowfile_repository etc. and then it worked
(-"-mount
source=nifi-flowfile_repository,target=/opt/nifi/nifi-current/flowfile_repository").
I jumped to Kubernetes PoC. I deployed official image to Kubernetes
(standalone one replica, not cluster). It was successfully run. I moved on
to next step for persistent disk requirement.
I used Persistent Volume and Persistent Volume Claim with NFS but it didn't
work correctly. It seems, It doesn't support NFS type "Persistent Volume"
and "Persistent Volume Claim" for mounting the storage.
I saw some topics in the forum about Statefullset and Helm but I am making
PoC on my local environment therefore I would like to learn, is there any
guidance or documentation about NiFi on Kubernetes?
My PoC Environment:
Apache Nifi 1.12.1 Official Image (https://hub.docker.com/r/apache/nifi/)
1 master 2 worker nodes (on VirtualBox)
Deployment is standalone not cluster therefore I am using standart
deployment yaml with single replica



--
Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/