[
https://issues.apache.org/jira/browse/GIRAPH-882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357880#comment-15357880
]
Sergey Edunov commented on GIRAPH-882:
--------------------------------------
Well, we don't need to worry about multiple zookeepers anymore, because we can
only have one, and single host name can not exceed 255 characters (RFC 1035).
On many systems this limit is actually lower. You can try:
$getconf HOST_NAME_MAX
64 - this is on my linux box
255 - this is on my mac
Also, note if you want to properly take care of a single host name being that
long, you need to fix much more than what has been fixed so far. See for
example
ZooKeeperManager#createCandidateStamp(), it has this line:
Path myCandidacyPath = new Path(
taskDirectory, myHostname +
HOSTNAME_TASK_SEPARATOR + taskPartition);
or
ZooKeeperManager#onlineZooKeeperServer(), it has this line:
Path myReadyPath = new Path(
serverDirectory, myHostname +
HOSTNAME_TASK_SEPARATOR + taskPartition +
HOSTNAME_TASK_SEPARATOR + zkBasePort);
we use host names pretty much everywhere, if this is what you want to fix when
much more needs to be done.
> List of zookeeper connection strings is trimmed by Hadoop counters.
> -------------------------------------------------------------------
>
> Key: GIRAPH-882
> URL: https://issues.apache.org/jira/browse/GIRAPH-882
> Project: Giraph
> Issue Type: Bug
> Components: zookeeper
> Affects Versions: 1.1.0, 1.2.0
> Reporter: Lukas Nalezenec
> Attachments: GIRAPH-882-rev2.patch, GIRAPH-882-rev3.patch,
> GIRAPH-882.patch, testrun.log
>
>
> We are running job with quorum of 3 zookeepers. Each serves has got long name
> (turing452.fi.callan.de:22181). Connection strings are stored to Hadoop
> Counters (for example:
> turing452.fi.callan.de:22181,turing124.fi.callan.de:22181,turing488.fi.callan.de:22181)
> but since name of counter is limited to ~63 character the connection string
> is trimmed (turing452.fi.callan.de:22181,turing124.fi.callan.de:22181,turin).
> 14/03/18 23:44:41 INFO zookeeper.ZooKeeper: Client
> environment:user.name=hadoop
> 14/03/18 23:44:41 INFO zookeeper.ZooKeeper: Initiating client connection,
> connectString=turing452.fi.callan.de:22181,turing124.fi.callan.de:22181,turin
> sessionTimeout=60000
> Exception in thread "main" java.net.UnknownHostException: turin
> at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
> at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901)
> at
> java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
> at java.net.InetAddress.getAllByName0(InetAddress.java:1246)
> at java.net.InetAddress.getAllByName(InetAddress.java:1162)
> at java.net.InetAddress.getAllByName(InetAddress.java:1098)
> at
> org.apache.zookeeper.client.StaticHostProvider.<init>(StaticHostProvider.java:60)
> at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:445)
> at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380)
> at org.apache.giraph.zk.ZooKeeperExt.<init>(ZooKeeperExt.java:114)
> at
> org.apache.giraph.job.JobProgressTracker.<init>(JobProgressTracker.java:69)
> at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:255)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
