[
https://issues.apache.org/jira/browse/DRILL-4591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Rogers updated DRILL-4591:
-------------------------------
Summary: Extend config system with distrib, site, node property files
(was: Extend config system with site, node property files)
> Extend config system with distrib, site, node property files
> ------------------------------------------------------------
>
> Key: DRILL-4591
> URL: https://issues.apache.org/jira/browse/DRILL-4591
> Project: Apache Drill
> Issue Type: Improvement
> Reporter: Paul Rogers
> Priority: Minor
>
> Today Drill provides the drill-override.conf file to set Drill properties,
> and the drill-env.sh file to provide custom launch properties. Today, most
> users seem to have a copy of DRILL_HOME per node, and thus they copy these
> two files per-node.
> The result is that the two files act as both the overall "site" configuration
> (for all nodes) and the "per-node" configuration for that one node.
> The approach works, but is awkward. Ideally, provide the option to have two
> files: one for the site, one for the node.
> drill-override.conf, drill-env.sh: For the site
> drill-override.conf, drill-env.sh: For the particular node
> The site-wide site-env.sh might, say, point to custom locations for log,
> conf, pid or other file locations, or set custom JVM options.
> Similarly, the per-node drill-node.conf might customize the per-node memory
> or other resources if one node has a different amount of physical memory or
> other resources.
> The improvement becomes more important if a user employs NFS, MapR FS or YARN
> to automatically deploy the site-wide files. In that case, the site files
> cannot also act as per-node files.
> Existing installs have only one each of drill-override.conf, drill-env.sh,
> copied to each node and possibly modified. For compatibility, the existing
> files become the site-wide files. When converting to a split-setup, copy the
> site-wide files as before. But, rather than modifying the site-wide files
> per-node, create a new per-load location, create new per-node files, and
> place the per-node modifications in those files. The per-node files need not
> change on following upgrades, just copy the site-wide files and that's it.
> We assume that DRILL_CONF_DIR points to the site-wide files. It defaults to
> $DRILL_HOME/conf or in /etc/drill/conf. To use node-specific files, the admin
> must set up a new, per-node location and identify that location using a
> proposed DRILL_NODE_CONF_DIR variable.
> The config files should be optional: skip them if they do not exist (or, for
> node files, skip them if DRILL_NODE_CONF_DIR is not set.)
> When loading, per-node files are loaded after the site-wide files (so that
> per-node values override site-wide values.)
> The change should be fully backward compatible. The change will also work
> with a YARN solution that "localizes" (copies to the local node) the
> site-wide files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
