rip out MapR stuff
Project: http://git-wip-us.apache.org/repos/asf/drill/repo Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/913b998b Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/913b998b Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/913b998b Branch: refs/heads/gh-pages Commit: 913b998b531ee73fcb05084e55ec84381d4b4d3b Parents: 80bbe06 Author: Kristine Hahn <kh...@maprtech.com> Authored: Thu May 7 16:06:15 2015 -0700 Committer: Kristine Hahn <kh...@maprtech.com> Committed: Thu May 7 16:06:15 2015 -0700 ---------------------------------------------------------------------- .../050-configuring-multitenant-resources.md | 49 ++------------------ 1 file changed, 5 insertions(+), 44 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/drill/blob/913b998b/_docs/configure-drill/050-configuring-multitenant-resources.md ---------------------------------------------------------------------- diff --git a/_docs/configure-drill/050-configuring-multitenant-resources.md b/_docs/configure-drill/050-configuring-multitenant-resources.md index f8ca673..2acd0e7 100644 --- a/_docs/configure-drill/050-configuring-multitenant-resources.md +++ b/_docs/configure-drill/050-configuring-multitenant-resources.md @@ -2,25 +2,9 @@ title: "Configuring Multitenant Resources" parent: "Configuring a Multitenant Cluster" --- -Drill operations are memory and CPU-intensive. Currently, Drill resources are managed outside of any cluster management service, such as the MapR warden service. In a multitenant or any other type of cluster, YARN-enabled or not, you configure memory and memory usage limits for Drill by modifying drill-env.sh as described in ["Configuring Drill Memory"]({{site.baseurl}}/docs/configuring-drill-memory). +Drill operations are memory and CPU-intensive. Currently, Drill resources are managed outside of any cluster management service. In a multitenant or any other type of cluster, YARN-enabled or not, you configure memory and memory usage limits for Drill by modifying drill-env.sh as described in ["Configuring Drill Memory"]({{site.baseurl}}/docs/configuring-drill-memory). -Configure a multitenant cluster to account for resources required for Drill. For example, on a MapR cluster, ensure warden accounts for resources required for Drill. Configuring `drill-env.sh` allocates resources for Drill to use during query execution, while configuring the following properties in `warden-drill-bits.conf` prevents warden from committing the resources to other processes. - - service.heapsize.min=<some value in MB> - service.heapsize.max=<some value in MB> - service.heapsize.percent=<a whole number> - -{% include startimportant.html %}Set the `service.heapsize` properties in `warden.drill-bits.conf` regardless of whether you changed defaults in `drill-env.sh` or not.{% include endimportant.html %} - -The section, ["Configuring Drill in a YARN-enabled MapR Cluster"]({{site.baseurl}}/docs/configuring-multitenant-resources#configuring-drill-in-a-yarn-enabled-mapr-cluster) shows an example of setting the `service.heapsize` properties. The `service.heapsize.percent` is the percentage of memory for the service bounded by minimum and maximum values. Typically, users change `service.heapsize.percent`. Using a percentage has the advantage of increasing or decreasing resources according to different node's configuration. For more information about the `service.heapsize` properties, see the section, ["warden.<servicename>.conf"](http://doc.mapr.com/display/MapR/warden.%3Cservicename%3E.conf). - -You need to statically partition the cluster to designate which partition handles which workload. To configure resources for Drill in a MapR cluster, modify one or more of the following files in `/opt/mapr/conf/conf.d` that the installation process creates. - -* `warden.drill-bits.conf` -* `warden.nodemanager.conf` -* `warden.resourcemanager.conf` - -Configure Drill memory by modifying `warden.drill-bits.conf` in YARN and non-YARN clusters. Configure other resources by modifying `warden.nodemanager.conf `and `warden.resourcemanager.conf `in a YARN-enabled cluster. +Configure a multitenant cluster manager to account for resources required for Drill. Configuring `drill-env.sh` allocates resources for Drill to use during query execution. It might be necessary to configure the cluster manager from committing the resources to other processes. ## Configuring Drill in a YARN-enabled MapR Cluster @@ -49,34 +33,11 @@ ResourceManager and NodeManager memory in `warden.resourcemanager.conf` and service.heapsize.max=325 service.heapsize.percent=2 -Change these settings for NodeManager and ResourceManager to reconfigure the total memory required for YARN services to run. If you want to place an upper limit on memory set YARN_NODEMANAGER_HEAPSIZE or YARN_RESOURCEMANAGER_HEAPSIZE environment variable in `/opt/mapr/hadoop/hadoop-2.5.1/etc/hadoop/yarn-env.sh`. The `-Xmx` option is not set, allowing memory on to grow as needed. - -### MapReduce v1 Resources - -The following default settings in `/opt/mapr/conf/warden.conf` control MapReduce v1 memory: - - mr1.memory.percent=50 - mr1.cpu.percent=50 - mr1.disk.percent=50 - -Modify these settings to reconfigure MapReduce v1 resources to suit your application needs, as described in section ["Resource Allocation for Jobs and Applications"](http://doc.mapr.com/display/MapR/Resource+Allocation+for+Jobs+and+Applications) of the MapR documentation. Remaining memory is given to YARN applications. - - -### MapReduce v2 and other Resources - -You configure memory for each service by setting three values in `warden.conf`. - - service.command.<servicename>.heapsize.percent - service.command.<servicename>.heapsize.max - service.command.<servicename>.heapsize.min - -Configure memory for other services in the same manner, as described in [MapR documentation](http://doc.mapr.com/display/MapR/warden.%3Cservicename%3E.conf) +Change these settings for NodeManager and ResourceManager to reconfigure the total memory required for YARN services to run. If you want to place an upper limit on memory set YARN_NODEMANAGER_HEAPSIZE or YARN_RESOURCEMANAGER_HEAPSIZE environment variable. Do not set the `-Xmx` option to allow the heap to grow as needed. -For more information about managing memory in a MapR cluster, see the following sections in the MapR documentation: +### MapReduce Resources -* [Memory Allocation for Nodes](http://doc.mapr.com/display/MapR/Memory+Allocation+for+Nodes) -* [Cluster Resource Allocation](http://doc.mapr.com/display/MapR/Cluster+Resource+Allocation) -* [Customizing Memory Settings for MapReduce v1](http://doc.mapr.com/display/MapR/Customize+Memory+Settings+for+MapReduce+v1) +Modify MapReduce memory to suit your application needs. Remaining memory is typically given to YARN applications. ## How to Manage Drill CPU Resources Currently, you do not manage CPU resources within Drill. [Use Linux `cgroups`](http://en.wikipedia.org/wiki/Cgroups) to manage the CPU resources. \ No newline at end of file
