[ https://issues.apache.org/jira/browse/DRILL-4543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15218386#comment-15218386 ]
Paul Rogers commented on DRILL-4543: ------------------------------------ Yes, let's move the ports issue to another JIRA. > Advertise Drill-bit ports, status, capabilities in ZooKeeper > ------------------------------------------------------------ > > Key: DRILL-4543 > URL: https://issues.apache.org/jira/browse/DRILL-4543 > Project: Apache Drill > Issue Type: Improvement > Components: Server > Reporter: Paul Rogers > Fix For: 2.0.0 > > > Today Drill uses ZooKeeper (ZK) to advertise the existence of a Drill-bit, > providing the host name/IP Address of the Drill-bit and the ports used, > encoded in Protobuf format. All other information (status, CPUs, memory) are > assumed to be the same across all Drill-bits in the cluster as specified in > the Drill config file. (Amended to reflect 1.6 behavior.) > Moving forward, as Drill becomes more sophisticated, Drill should advertise > the specifics of each Drill-bit so that one Drill bit can differ from another. > For example, when running on YARN, we need a way for Drill to gracefully shut > down. Advertising a status of Ready or Unavailable will help. Ready is the > normal state. Unavailable means the Drill-bit will finish in-flight queries, > but won't accept new ones. (The actual status is a separate enhancement.) > In a YARN cluster, Drill should take advantage of machines with more memory, > but live with machines with less. (Perhaps some are newer, some are older or > more heavily loaded.) Drill should use ZK to identify its available memory > and CPUs so that the planner can use them. (Use of the info is a separate > enhancement.) > There may be times when two drill bits run on a single machine. If so, they > must use separate ports. So, each Drill-bit should advertise its ports in ZK. > For backward compatibility, the information is optional; if not present, the > receiver should assume the information defaults to that in the config file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)