Re: Integrate NiFi auth using OKTA SAML
Thanks David. I will try then share it again. :) Regards, DongCheol Gong 2024년 3월 5일 (화) 오후 10:38, David Handermann 님이 작성: > Thanks for following up on this issue, the additional logging is helpful. > > The example URL provided does not include the port number ( > https://nifi.my-site.com) which seems to imply the presence of a gateway > or reverse proxy server in front of NiFi. This configuration is supported, > but in some cases, it can require including the port number in the URL, the > default value being 443. > > For reference, here is the Spring Security code that is producing the > error shown in the log: > > > https://github.com/spring-projects/spring-security/blob/69527f9a9c6ded890763d67d992cbcbb3a393162/saml2/saml2-service-provider/src/main/java/org/springframework/security/saml2/provider/service/authentication/OpenSaml4AuthenticationProvider.java#L388 > > If you are using a reverse proxy server, it is important that all of the > applicable X-Proxy headers are configured so that NiFi can generate the > correct URL for comparison. > > See the Proxy Configuration section of the Admin Guide for more details: > > > https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#proxy_configuration > > Regards, > David Handermann > > On Mon, Mar 4, 2024 at 7:14 PM DC Gong wrote: > >> Hi guys. >> >> I solved and share my story. >> >> I was able to check the logs that the destination information was >> different as shown below, but there was no problem with the settings in >> OKTA. >> >> >> 2024-03-04 12:14:33,051 DEBUG [NiFi Web Server-26] >> o.s.s.s.p.s.a.OpenSamlAuthenticationProvider Found 2 validation errors in >> SAML response [id4756651808328737370315028]: [[invalid_destination] Invalid >> destination [https://nifi.my-site.com/nifi-api/access/saml/login/consumer] >> for SAML response [id4756651808328737370315028], [invalid_assertion] >> Invalid assertion [id4756651808686833990238847] for SAML response >> [id4756651808328737370315028]: No subject confirmation methods were met for >> assertion with ID 'id4756651808686833990238847'] >> 2024-03-04 12:14:33,051 TRACE [NiFi Web Server-26] >> o.s.s.s.p.s.s.f.Saml2WebSsoAuthenticationFilter Failed to process >> authentication request >> org.springframework.security.saml2.provider.service.authentication.Saml2AuthenticationException: >> Invalid destination [ >> https://nifi.my-site.com/nifi-api/access/saml/login/consumer] for SAML >> response [id4756651808328737370315028] >> >> >> >> It was so strange that I tried lowering the version of NiFi. >> The version of NiFi that was causing the problem was 1.25.0, but I >> changed it to 1.15.0 and it worked fine. >> >> I haven't figured out exactly what the problem is, but I'll put that off >> until later and share my story. >> I realize this isn't a root cause fix, but it's one of the quickest >> things you can try to troubleshoot. >> >> Have a great day everyone. >> >> Regards, >> DongCheol Gong >> >> 2024년 3월 1일 (금) 오후 11:44, DC Gong 님이 작성: >> >>> Thanks David, >>> >>> I know it's not going to be easy to resolve my issue. >>> I'll change the loglevel as you suggested and test again. >>> >>> Have a nice and happy weekend. >>> >>> Regard, >>> DongCheol Gong >>> >>> 2024년 3월 1일 (금) 오전 10:36, David Handermann 님이 >>> 작성: >>> Thanks for providing some background on the issue with SAML configuration. The following post describes the steps required for configuring NiFi to integrate with Okta, including example configuration settings: https://exceptionfactory.com/posts/2022/11/30/integrating-apache-nifi-with-okta-saml-authentication/ It is difficult to determine the problem based on the logs provided. As a next step, enabling debug logging for the org.springframework.security logger should provide additional details about the SAML handshake process. Regards, David Handermann On Wed, Feb 28, 2024 at 9:25 PM DC Gong wrote: > Hi, > I’m trying to get an OKTA SAML integration for NiFi. > I set up nifi.properties using the information provided by okta. > The domain information is dummy for security reasons. > I set up the entityId and ACS information in okta correctly. > > > > nifi.security.user.saml.idp.metadata.url= > https://okta-site.com/nifi/okta-saml/metadata.xml > nifi.security.user.saml.sp.entity.id=mysite-entity-id > nifi.security.user.saml.identity.attribute.name= > nifi.security.user.saml.group.attribute.name= > nifi.security.user.saml.request.signing.enabled=false > nifi.security.user.saml.want.assertions.signed=true > nifi.security.user.saml.signature.algorithm= > http://www.w3.org/2001/04/xmldsig-more#rsa-sha256 > nifi.security.user.saml.authentication.expiration=12 hours > nifi.security.user.saml.single.logout.enabled=false > nifi.security.user.saml.http.client.truststore.strategy=JDK >
Re: SELinux and NiFi
Similar situation here. Not a devops guy. So I guess it’s fair to say this is expected behavior if policies are not written to allow nifi.sh and such to access the folders NiFi needs to access. Thanks for the help.Sent from my iPhoneOn Mar 8, 2024, at 5:38 PM, Russell Bateman wrote: That's what lies in those "SELinux policies." I think it's simple: Use SELinux to lock the filesystem (and other stuff) up so no one can get in or go around. Then create specific policies that allow, in this case, NiFi, access to its filesystem (like /opt/nifi/current-nifi/) so that it can do work. Obviously, when you install NiFi, things can get complicated like where do you want each repository to live--you'll have to provide NiFi access to each place, no longer a single filesystem. This is handled by DevOps guys and not me (I just write custom processors), but if you get real pointed, I can ask them better questions they can answer. Russ On 3/8/24 15:04, Mike Thomsen wrote: I think the admin told me that even a simple nifi.sh start won’t work. Just won’t even start the script and it is marked executable. I was wondering if there were any gotchas to getting a basic setup running. Sent from my iPhone On Mar 8, 2024, at 4:29 PM, Russell Bateman wrote: We have run on CentOS with SELinux set to enforcing and have run NiFi in that environment for probably 8 or 9 years now. We do install some SELinux policies that allow NiFi to access the filesystem underneath itself and not outside that filesystem. What specifically are you asking? On 3/8/24 14:04, Mike Thomsen wrote: Does anyone have experience setting up NiFi w/ SELinux set to "enforcing?"
Re: SELinux and NiFi
That's what lies in those "SELinux policies." I think it's simple: Use SELinux to lock the filesystem (and other stuff) up so no one can get in or go around. Then create specific policies that allow, in this case, NiFi, access to its filesystem (like //opt/nifi/current-nifi//) so that it can do work. Obviously, when you install NiFi, things can get complicated like where do you want each repository to live--you'll have to provide NiFi access to each place, no longer a single filesystem. This is handled by DevOps guys and not me (I just write custom processors), but if you get real pointed, I can ask them better questions they can answer. Russ On 3/8/24 15:04, Mike Thomsen wrote: I think the admin told me that even a simple nifi.sh start won’t work. Just won’t even start the script and it is marked executable. I was wondering if there were any gotchas to getting a basic setup running. Sent from my iPhone On Mar 8, 2024, at 4:29 PM, Russell Bateman wrote: We have run on CentOS with SELinux set to enforcing and have run NiFi in that environment for probably 8 or 9 years now. We do install some SELinux policies that allow NiFi to access the filesystem underneath itself and not outside that filesystem. What specifically are you asking? On 3/8/24 14:04, Mike Thomsen wrote: Does anyone have experience setting up NiFi w/ SELinux set to "enforcing?"
Re: SELinux and NiFi
I think the admin told me that even a simple nifi.sh start won’t work. Just won’t even start the script and it is marked executable. I was wondering if there were any gotchas to getting a basic setup running. Sent from my iPhone > On Mar 8, 2024, at 4:29 PM, Russell Bateman wrote: > > We have run on CentOS with SELinux set to enforcing and have run NiFi in > that environment for probably 8 or 9 years now. We do install some SELinux > policies that allow NiFi to access the filesystem underneath itself and not > outside that filesystem. > > What specifically are you asking? > >> On 3/8/24 14:04, Mike Thomsen wrote: >> Does anyone have experience setting up NiFi w/ SELinux set to "enforcing?" >
Re: SELinux and NiFi
We have run on CentOS with SELinux set to enforcing and have run NiFi in that environment for probably 8 or 9 years now. We do install some SELinux policies that allow NiFi to access the filesystem underneath itself and not outside that filesystem. What specifically are you asking? On 3/8/24 14:04, Mike Thomsen wrote: Does anyone have experience setting up NiFi w/ SELinux set to "enforcing?"
SELinux and NiFi
Just did a search through the docs on Google and nothing came up for SELinux. Does anyone have experience setting up NiFi w/ SELinux set to "enforcing?" Thanks, Mike
Re: Why are my journal files so large on node 2 of cluster?
Dave, When you say that the journal files are huge, I presume you mean the FlowFile repository? There are generally 4 things that can cause this: - OutOfMemoryError causing the FlowFile repo not to properly checkpoint - Out of Disk Space causing the FlowFile repo not to properly checkpoint - Out of open file handles causing the FlowFile repo not to properly checkpoint - Creating a lot of huge attributes on your FlowFiles. The first 3 situations can be identified by looking for errors in the logs. For the third one, you need to understand whether or not you’re creating huge FlowFile attributes. Generally, attributes should be very small - 100-200 characters or less, ideally. It’s possible that you have a flow that creates huge attributes but the flow is only running on the Primary Node, and Node 2 is your Primary Node, which would cause this to occur only on this node. Thanks -Mark > On Mar 7, 2024, at 9:24 PM, David Early via users > wrote: > > I have a massive issue: I have a 2 node cluster (using 5 external zookeepers > on other boxes), and for some reason on node 2 I have MASSIVE journal files. > > I am round robbining data between the nodes, but for some reason node 2 just > fills up. This is the second time this has happened this week. > > What should I do? nifi.properties are the same on both systems (except for > local host names).. > > Any ideas of what might be causing one node to overload? > > Dave > >
Re: RestLookupService question
Yes, I agree it is an improvement. Ticket submitted: https://issues.apache.org/jira/browse/NIFI-12877 Thanks Roman On Fri, 8 Mar 2024 at 02:35, Lehel Boér wrote: > The service currently does not support sensitive dynamic properties. It's > not a bug, but you can submit a ticket for the improvement. Look for the > > *Dynamic Properties:* > Supports Sensitive Dynamic Properties: *Yes* > > in the processor usage. > Lehel > -- > *From:* Roman Wesołowski > *Sent:* Thursday, March 7, 2024 17:48 > *To:* users@nifi.apache.org > *Subject:* Re: RestLookupService question > > Hi, > I have an additional question regarding RestLookupService. I will use this > thread. > When we add a header in InvokeHTTP processor we can set "Sensitive Value" > Yes/No, however during adding dynamic values (tokens) in RestLookupService > there is no way to make it sensitive it is grayed out. Shouldn't it be a > bug as well? > Or is there any idea behind it? > > Thanks, > > Roman > > > > On Thu, 7 Mar 2024 at 19:03, Lehel Boér wrote: > > The response code is only used in debug logs, there is room to improve > error handling, > Thanks for reporting! > -- > *From:* Gregory Foreman > *Sent:* Thursday, March 7, 2024 12:53 > *To:* users@nifi.apache.org > *Subject:* Re: RestLookupService question > > I assume this is a bug. I just created NIFI-12875 ( > https://issues.apache.org/jira/browse/NIFI-12875) to track. > > On Mar 1, 2024, at 8:04 AM, Gregory Foreman < > gfore...@spinnerconsulting.com> wrote: > > Hello: > > The RestLookupService service seems to treat all http response status > codes the same. Is this intentional? I would assume the LookupRecord > processor would like to know if the service is returning errors so it could > route accordingly. > > Thanks, > Greg > > >
