This is an automated email from the ASF dual-hosted git repository.
rxl pushed a commit to branch branch-2.6
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit bbcb4eed12e5084aed6555a2ed97dfdb0e5c4913
Author: Matteo Merli <mme...@apache.org>
AuthorDate: Sat Jun 27 16:39:46 2020 -0700
Use hostname for bookie rackawareness mapping (#7361)
### Motivation
In #5607 the `useHostName()` was added with `return false`. That means that
the rackaware policy will try to resolve the Bookies hostname into an IP and
then use that IP to figure out which rack the bookie belongs.
There are 2 problems:
1. The IP won't match the hostname which is recorded in the `/bookies`
z-node
2. If there is an error in resolving the bookie hostname (eg: transient
DNS error), an NPE exception will be triggered and the BK client will never
realize that this bookie was ever seen as available in the cluster.
```
java.lang.NullPointerException: null
at
org.apache.bookkeeper.net.NetUtils.resolveNetworkLocation(NetUtils.java:77)
~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0]
at
org.apache.bookkeeper.client.TopologyAwareEnsemblePlacementPolicy.resolveNetworkLocation(TopologyAwareEnsemblePlacementPolicy.java:779)
~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0]
at
org.apache.bookkeeper.client.TopologyAwareEnsemblePlacementPolicy.createBookieNode(TopologyAwareEnsemblePlacementPolicy.java:775)
~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0]
at
org.apache.bookkeeper.client.TopologyAwareEnsemblePlacementPolicy.handleBookiesThatJoined(TopologyAwareEnsemblePlacementPolicy.java:707)
[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0]
at
org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl.handleBookiesThatJoined(RackawareEnsemblePlacementPolicyImpl.java:79)
[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0]
at
org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicy.handleBookiesThatJoined(RackawareEnsemblePlacementPolicy.java:246)
[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0]
at
org.apache.bookkeeper.client.TopologyAwareEnsemblePlacementPolicy.onClusterChanged(TopologyAwareEnsemblePlacementPolicy.java:654)
[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0]
at
org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl.onClusterChanged(RackawareEnsemblePlacementPolicyImpl.java:79)
[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0]
at
org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicy.onClusterChanged(RackawareEnsemblePlacementPolicy.java:89)
[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0]
at
org.apache.bookkeeper.client.BookieWatcherImpl.processReadOnlyBookiesChanged(BookieWatcherImpl.java:190)
[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0]
at
org.apache.bookkeeper.client.BookieWatcherImpl.lambda$initialBlockingBookieRead$2(BookieWatcherImpl.java:209)
[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0]
at
org.apache.bookkeeper.discover.ZKRegistrationClient$WatchTask.accept(ZKRegistrationClient.java:139)
[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0]
at
org.apache.bookkeeper.discover.ZKRegistrationClient$WatchTask.accept(ZKRegistrationClient.java:62)
[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0]
```
The exception is thrown at 77, since `getAddress()` yealds a `null` given
that the address is unresolved.
```java
74 if (dnsResolver.useHostName()) {
75 names.add(addr.getHostName());
76 } else {
77 names.add(addr.getAddress().getHostAddress());
78 }
```
The default implementation for the `DnsResolver.useHostName()` is to return
true.
(cherry picked from commit e073465d70e0910ea533aefa5cb4af07423f22b9)
---
.../org/apache/pulsar/zookeeper/ZkBookieRackAffinityMapping.java | 5 -----
1 file changed, 5 deletions(-)
diff --git
a/pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMapping.java
b/pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMapping.java
index 7b9f314..93400e9 100644
---
a/pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMapping.java
+++
b/pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMapping.java
@@ -215,11 +215,6 @@ public class ZkBookieRackAffinityMapping extends
AbstractDNSToSwitchMapping
}
@Override
- public boolean useHostName() {
- return false;
- }
-
- @Override
public void onUpdate(String path, BookiesRackConfiguration data, Stat
stat) {
if (rackawarePolicy != null) {
LOG.info("Bookie rack info updated to {}. Notifying rackaware
policy.", data.toString());
