GitHub user jmhublar edited a discussion: Replication broken after upgrade from
2.11.2 to 3.0.2
Greetings,
I'm hoping the community can set me on the right track here. . .
We upgraded our Pulsar clusters to version 3.0.2 and have been encountering
issues with namespace-scoped replication ever since. The replication between
clusters (cluster_a and cluster_b ) has started failing, and we noticed this
shortly after the upgrade. We continue to observe this with 3.0.7
Problem Details:
It seems like the brokers in our cluster_a cluster aren't passing the OAuth2
parameters required to authenticate with the other instance in cluster_b . Upon
closer inspection, we noticed that the cluster configuration in cluster_a lacks
the necessary authentication parameters:
```
{
"serviceUrl": "http://pulsar-proxy.apps-01.sbx.cluster_b.example.com:8080";,
"serviceUrlTls":
"https://pulsar-proxy.apps-01.sbx.cluster_b.example.com:8443";,
"brokerServiceUrl":
"pulsar://pulsar-proxy.apps-01.sbx.cluster_b.example.com:6650",
"brokerServiceUrlTls":
"pulsar+ssl://pulsar-proxy.apps-01.sbx.cluster_b.example.com:6651",
"authenticationPlugin": "",
"authenticationParameters": "",
...
}
```
The errors we’re seeing on the cluster_b cluster are like this:
```
WARN org.apache.pulsar.proxy.server.ProxyConnection - [/10.191.231.31:14887]
Unable to authenticate
javax.naming.AuthenticationException: No anonymous role, and no authentication
provider configured
```
To address this, we tried updating the cluster configuration for cluster_a to
include OAuth2 parameters, hoping that it would resolve the issue. It is worth
noting that cluster_a and cluster_b. use the same auth endpoint and tokens.
Below is a representation of the updated configuration:
```
{
"serviceUrl": "http://pulsar-proxy.apps-01.sbx.cluster_b.example.com:8080";,
"serviceUrlTls":
"https://pulsar-proxy.apps-01.sbx.cluster_b.example.com:8443";,
"brokerServiceUrl":
"pulsar://pulsar-proxy.apps-01.sbx.cluster_b.example.com:6650",
"brokerServiceUrlTls":
"pulsar+ssl://pulsar-proxy.apps-01.sbx.cluster_b.example.com:6651",
"authenticationPlugin":
"org.apache.pulsar.client.impl.auth.oauth2.AuthenticationOAuth2",
"authenticationParameters": "{\"issuerUrl\":
\"https://oauth.example.com/[REDACTED]\",\"privateKey\":
\"file:///pulsar/privatekey/proxy-oauth.json\",\"audience\": \"pulsar\"}",
"brokerClientTlsEnabled": true,
"tlsAllowInsecureConnection": true,
...
}
```
Despite this, we’re still seeing the same authentication errors when attempting
replication.
Questions:
- Does this configuration seem correct? We're essentially trying to mirror the
broker's internal settings for OAuth2.
- If we make changes to the cluster configuration (like updating OAuth2
parameters), do we need to reinitialize or reset the replication configuration
on the cluster for those changes to take effect?
Any insights or advice from those who've encountered similar issues would be
greatly appreciated. Thanks in advance for your help!
GitHub link: https://github.com/apache/pulsar/discussions/23558
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
