[
https://issues.apache.org/jira/browse/DRILL-4543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15216644#comment-15216644
]
John Omernik commented on DRILL-4543:
-------------------------------------
I think one difference is that with HOCON we'd have to explicitly set the
drill-override to have the ENV variable as its value. I guess this shouldn't
hurt. We'd just have to make two changes, define what we want as ENV variables,
and then update the drill-override on each drill bit to have the setting use
the ENV variable instead of a hard coded port. I guess from a simplicity point
of view, I always saw the drill-override as a cluster wide settings file, as it
makes it harder to read/grok if there are lots of variables. On the other hand,
it's not more difficult than doing things implicitly with defaults, therefore,
as long as Drill ok with the different settings, for us to do this, all we have
to do define what environmental variables we want to use and then set them,
while also updating the drill-override.
Then yes, the only other thing is the data port, getting the ability to
explicitly set that.
> Advertise Drill-bit ports, status, capabilities in ZooKeeper
> ------------------------------------------------------------
>
> Key: DRILL-4543
> URL: https://issues.apache.org/jira/browse/DRILL-4543
> Project: Apache Drill
> Issue Type: Improvement
> Components: Server
> Reporter: Paul Rogers
> Fix For: 2.0.0
>
>
> Today Drill uses ZooKeeper (ZK) to advertise the existence of a Drill-bit,
> providing just the host name/IP Address of the Drill-bit. All other
> information (ports, status, capabilities) are assumed to be the same across
> all Drill-bits in the cluster as specified in the Drill config file.
> Moving forward, as Drill becomes more sophisticated, Drill should advertise
> the specifics of each Drill-bit so that one Drill bit can differ from another.
> For example, when running on YARN, we need a way for Drill to gracefully shut
> down. Advertising a status of Ready or Unavailable will help. Ready is the
> normal state. Unavailable means the Drill-bit will finish in-flight queries,
> but won't accept new ones. (The actual status is a separate enhancement.)
> In a YARN cluster, Drill should take advantage of machines with more memory,
> but live with machines with less. (Perhaps some are newer, some are older or
> more heavily loaded.) Drill should use ZK to identify its available memory
> and CPUs so that the planner can use them. (Use of the info is a separate
> enhancement.)
> There may be times when two drill bits run on a single machine. If so, they
> must use separate ports. So, each Drill-bit should advertise its ports in ZK.
> For backward compatibility, the information is optional; if not present, the
> receiver should assume the information defaults to that in the config file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
