I have two NiFi clusters - an internal one (within my organization) and an external one (on a VPS in the cloud). The internal cluster is configured with Kerberos for authentication and LDAPs for authorization. The external cluster uses basic username/password authentication/authorization. User authorization by certificates is NOT used and not planned (this is important). I am trying to set up Site-to-Site between these two NiFi clusters. I followed this guide (Option 2): https://community.cloudera.com/t5/Community-Articles/Site-To-Site-communication-between-secured-HTTPS-and/ta-p/244752 Data is being sent from the internal cluster to the external one. The external (cloud) cluster is behind a reverse proxy (Nginx), and the nodes themselves are not exposed to the internet. The common address (on Nginx) is dev.my-esb.ru. In the RPG, I specify the common address of the external cluster - https://dev.my-esb.ru/nifi (because I can't specify node addresses, as they are not directly accessible on the internet). The protocol is HTTP. I see that a lock icon appears in the RPG (indicating "Site-to-Site is secure"). The problem is that in this case, the external cluster seems to think that it is being contacted by Nginx rather than by the nodes of the internal cluster. The setup looks like this: (internal_cluster_nodes) > (ext_nginx) > (external_cluster_nodes) In the logs of the internal cluster, I see the following: 2024-09-02 18:26:40,461 WARN [Remote Process Group b358637f-0191-1000-0000-00007a676d5a Thread-1] o.a.n.r.util.SiteToSiteRestApiClient Failed to get controller from https://dev.my-esb.ru/nifi-api org.apache.nifi.remote.util.SiteToSiteRestApiClient$HttpGetFailedException: response code 401:Unauthorized with explanation: null
2024-09-02 18:26:40,538 ERROR [Remote Process Group b358637f-0191-1000-0000-00007a676d5a Thread-1] o.a.n.remote.StandardRemoteProcessGroup org.apache.nifi.remote.StandardRemoteProcessGroup$InitializationTask@46d12aaf Failed to request account: got unexpected response code of 401:Unauthorized 2024-09-02 18:27:18,123 WARN [Timer-Driven Process Thread-2] o.a.n.r.util.SiteToSiteRestApiClient Failed to get controller from https://dev.my-esb.ru/nifi-api org.apache.nifi.remote.util.SiteToSiteRestApiClient$HttpGetFailedException: response code 401:Unauthorized with explanation: null 2024-09-02 18:27:18,124 WARN [Timer-Driven Process Thread-2] o.apache.nifi.controller.FlowController Unable to communicate with remote instance RemoteProcessGroup[https://dev.my-esb.ru/nifi] org.apache.nifi.controller.exception.CommunicationsException: org.apache.nifi.controller.exception.CommunicationsException: Unable to communicate with Remote NiFi at URI https://dev.my-esb.ru/nifi due to: response code 401:Unauthorized with explanation: null Caused by: org.apache.nifi.controller.exception.CommunicationsException: Unable to communicate with Remote NiFi at URI https://dev.my-esb.ru/nifi due to: response code 401:Unauthorized with explanation: null In the external Nginx access logs, I see this: 1.1.1.53 - - [02/Sep/2024:18:43:38 +0300] "GET /nifi-api/site-to-site HTTP/1.1" 401 12 "-" "Apache-HttpClient/4.5.14 (Java/21.0.4)" 1.1.1.53 - - [02/Sep/2024:18:43:38 +0300] "POST /nifi-api/controller/users HTTP/1.1" 401 12 "-" "Jersey/3.1.7 (HttpUrlConnection 21.0.4)" 1.1.1.53 is the IP of the router of the internal cluster. In the request logs of the external cluster, I get the following logs: 10.0.0.10 - anonymousUser [02/Sep/2024:16:10:19 +0000] "GET /nifi-api/site-to-site HTTP/1.1" 401 12 "-" "Apache-HttpClient/4.5.14 (Java/21.0.4)" 10.0.0.10 is the IP of the external Nginx. I understand why this may happen - the external NiFi does not trust Nginx for Site-to-Site connections. If I understand correctly, for this to work, the internal cluster must "authenticate" with the external one using a certificate. To achieve this, I added the root certificate of the internal cluster to the truststore of the external one. There is an interesting nuance. Nginx CAN pass its own certificate when forwarding requests. There are corresponding settings - proxy_ssl_certificate and proxy_ssl_certificate_key. And if this certificate is added to the truststore of the external cluster, Site-to-Site starts working properly! BUT! In this case, I cannot log into the NiFi UI, because it now thinks the user is logging in with the Nginx certificate and sees that this user lacks the necessary permissions. It is then not possible to log in using a username and password. Here are the relevant settings for the external cluster (for the first node as an example). nifi.properties: # Site to Site properties nifi.remote.input.host=dev-node-1.my-esb.ru nifi.remote.input.secure=true nifi.remote.input.socket.port=10443 nifi.remote.input.http.enabled=true nifi.remote.input.http.transaction.ttl=30 sec nifi.remote.contents.cache.expiration=30 secs nifi.remote.route.http.external.when=${X-ProxyHost:equals('dev.my-esb.ru')} nifi.remote.route.http.external.hostname=${s2s.target.hostname} nifi.remote.route.http.external.port=443 nifi.remote.route.http.external.secure=true nifi.web.https.host=dev-node-1.my-esb.ru nifi.web.https.port=8443 nifi.web.proxy.context.path= nifi.web.proxy.host=dev.my-esb.ru:443 nifi.cluster.node.address=dev-node-1.my-esb.ru External Nginx configuration: upstream dev_my-esb_ru { ip_hash; # Use ip_hash for stable connections server dev-node-1.my-esb.ru:8443; server dev-node-2.my-esb.ru:8443; server dev-node-3.my-esb.ru:8443; } # If the target node is not specified, use one from the cluster map $http_host $nifi { dev-node-1.my-esb.ru:443 "dev-node-1.my-esb.ru:8443"; dev-node-2.my-esb.ru:443 "dev-node-2.my-esb.ru:8443"; dev-node-3.my-esb.ru:443 "dev-node-3.my-esb.ru:8443"; default "dev_my-esb_ru"; } resolver 127.0.0.1; server { listen 443 ssl; server_name dev.my-esb.ru; ssl_certificate /etc/letsencrypt/live/dev.my-esb.ru/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/dev.my-esb.ru/privkey.pem; # proxy_ssl_certificate /etc/letsencrypt/live/dev.my-esb.ru/fullchain.pem; # proxy_ssl_certificate_key /etc/letsencrypt/live/dev.my-esb.ru/privkey.pem; ssl_protocols TLSv1.2 TLSv1.3; ssl_prefer_server_ciphers on; ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384; # Additional security settings add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always; ssl_stapling on; ssl_stapling_verify on; resolver 1.1.1.1 1.0.0.1 valid=300s; resolver_timeout 5s; # Logging access_log /etc/nginx/nifi_access.log; error_log /etc/nginx/nifi_error.log warn; location / { proxy_pass https://$nifi; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Original-URI $request_uri; proxy_set_header X-ProxyScheme https; proxy_set_header X-ProxyHost $host; proxy_set_header X-ProxyPort 443; # proxy_set_header X-ProxyContextPath /; proxy_set_header X-ProxiedEntitiesChain $ssl_client_s_dn; # WebSocket settings proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; # Timeouts proxy_connect_timeout 90; proxy_send_timeout 90; proxy_read_timeout 90; # SSL settings for the backend proxy_ssl_verify off; proxy_ssl_trusted_certificate /etc/nginx/certs/ca_chain.pem; proxy_ssl_server_name on; proxy_ssl_name $host; proxy_ssl_session_reuse on; } } TL;DR I'm trying to set up a Site-to-Site connection between two NiFi clusters: an internal cluster with Kerberos and LDAPs, and an external cluster behind an Nginx reverse proxy using basic authentication. The problem arises because the external NiFi cluster sees the requests coming from Nginx, not the internal cluster nodes, leading to authentication issues. When I enable Nginx to forward its certificate, Site-to-Site works, but then I can't log in to the NiFi UI due to conflicting user authentication. I’m looking for guidance on how to configure Site-to-Site properly under these conditions without losing UI access.
