GitHub user jmhublar edited a discussion: Replication broken after upgrade from 
2.11.2 to 3.0.2

Greetings,

I'm hoping the community can set me on the right track here. . .

We upgraded our Pulsar clusters to version 3.0.2 and have been encountering 
issues with namespace-scoped replication ever since. The replication between 
clusters (cluster_a and cluster_b ) has started failing, and we noticed this 
shortly after the upgrade.
Problem Details:

It seems like the brokers in our cluster_a cluster aren't passing the OAuth2 
parameters required to authenticate with the other instance in cluster_b . Upon 
closer inspection, we noticed that the cluster configuration in cluster_a lacks 
the necessary authentication parameters:

```
{
  "serviceUrl": "http://pulsar-proxy.apps-01.sbx.cluster_b.example.com:8080";,
  "serviceUrlTls": 
"https://pulsar-proxy.apps-01.sbx.cluster_b.example.com:8443";,
  "brokerServiceUrl": 
"pulsar://pulsar-proxy.apps-01.sbx.cluster_b.example.com:6650",
  "brokerServiceUrlTls": 
"pulsar+ssl://pulsar-proxy.apps-01.sbx.cluster_b.example.com:6651",
  "authenticationPlugin": "",
  "authenticationParameters": "",
  ...
}
```

The errors we’re seeing on the cluster_b cluster are like this:

```
WARN  org.apache.pulsar.proxy.server.ProxyConnection - [/10.191.231.31:14887] 
Unable to authenticate
javax.naming.AuthenticationException: No anonymous role, and no authentication 
provider configured
```

To address this, we tried updating the cluster configuration for cluster_a to 
include OAuth2 parameters, hoping that it would resolve the issue. It is worth 
noting that cluster_a and cluster_b. use the same auth endpoint and tokens.  
Below is a representation of the updated configuration:



{
  "serviceUrl": "http://pulsar-proxy.apps-01.sbx.cluster_b.example.com:8080";,
  "serviceUrlTls": 
"https://pulsar-proxy.apps-01.sbx.cluster_b.example.com:8443";,
  "brokerServiceUrl": 
"pulsar://pulsar-proxy.apps-01.sbx.cluster_b.example.com:6650",
  "brokerServiceUrlTls": 
"pulsar+ssl://pulsar-proxy.apps-01.sbx.cluster_b.example.com:6651",
  "authenticationPlugin": 
"org.apache.pulsar.client.impl.auth.oauth2.AuthenticationOAuth2",
  "authenticationParameters": "{\"issuerUrl\": 
\"https://oauth.example.com/[REDACTED]\",\"privateKey\": 
\"file:///pulsar/privatekey/proxy-oauth.json\",\"audience\": \"pulsar\"}",
  "brokerClientTlsEnabled": true,
  "tlsAllowInsecureConnection": true,
  ...
}

Despite this, we’re still seeing the same authentication errors when attempting 
replication.
Questions:

    Does this configuration seem correct? We're essentially trying to mirror 
the broker's internal settings for OAuth2.
    If we make changes to the cluster configuration (like updating OAuth2 
parameters), do we need to reinitialize or reset the replication configuration 
on the cluster for those changes to take effect?

Any insights or advice from those who've encountered similar issues would be 
greatly appreciated. Thanks in advance for your help!

GitHub link: https://github.com/apache/pulsar/discussions/23558

----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]

Reply via email to