Author: buildbot Date: Sat Oct 11 02:12:09 2014 New Revision: 925331 Log: Staging update by buildbot for slider
Modified: websites/staging/slider/trunk/content/ (props changed) websites/staging/slider/trunk/content/docs/security.html Propchange: websites/staging/slider/trunk/content/ ------------------------------------------------------------------------------ --- cms:source-revision (original) +++ cms:source-revision Sat Oct 11 02:12:09 2014 @@ -1 +1 @@ -1630615 +1631012 Modified: websites/staging/slider/trunk/content/docs/security.html ============================================================================== --- websites/staging/slider/trunk/content/docs/security.html (original) +++ websites/staging/slider/trunk/content/docs/security.html Sat Oct 11 02:12:09 2014 @@ -183,11 +183,11 @@ listed at the bottom. </p> <h2 id="concepts">Concepts</h2> <p>Slider runs in secure clusters, but with restrictions</p> <ol> -<li>The keytabs to allow a worker to authenticate with the master must - be distributed in advance: Slider does not attempt to pass these around.</li> +<li>The keytabs to allow the AM and components to authenticate with the master are + either distributed in advance or can be distributed by Slider.</li> <li>Until the location of Slider node instances can be strictly limited to - a set of nodes (a future YARN feature), the keytabs must be passed to - all the nodes in the cluster in advance, <em>and made available to the + a set of nodes (a future YARN feature), the keytabs are required by + all the nodes in the cluster, <em>and made available to the user creating the cluster</em></li> <li>due to the way that HBase and accumulo authenticate worker nodes to the masters, any HBase node running on a server must authenticate as @@ -242,10 +242,11 @@ kerberos identities.</li> <ol> <li>The user is expected to have their own Kerberos principal, and have used <code>kinit</code> or equivalent to authenticate with Kerberos and gain a (time-bounded) TGT</li> -<li>The user is expected to have their own principals for every host in the cluster of the form - username/hostname@REALM</li> -<li>A keytab must be generated which contains all these principals -and distributed - to all the nodes in the cluster with read access permissions to the user.</li> +<li>The user is expected to have principals for every host in the cluster of the form + username/hostname@REALM for component aunthentication. The AM authentication requirements + can be satisfied with a non-host based principal (username@REALM).</li> +<li>Separate keytabs should be generated for the AM, which contains the AM login principal, and the service components, which contain all the service principals. The keytabs can be manually distributed + to all the nodes in the cluster with read access permissions to the user, or the user may elect to leverage the Slider keytab distribution mechanism.</li> <li>When the user creates a secure cluster, they provide the standard HBase kerberos options to identify the principals to use and the keytab location.</li> </ol> @@ -263,6 +264,56 @@ user <code>r-x</code> for the group and rights of the user that created the cluster.</p> <p>The Application Master will read in the JSON cluster specification file, and instantiate the relevant number of componentss. </p> +<h3 id="the-keytab-distributionaccess-options">The Keytab distribution/access Options</h3> +<p>The AM has been modified to leverage keytabs for authenticating rather than relying on delegation-token based authentication mechanisms. In order to perform this login the AM requires access to a keytab file that contains the principal representing the user identity to be associated with the launched application instance. There are two mechanisms supported for keytab access and/or distribution:</p> +<h4 id="local-keytab-file-access">Local Keytab file access:</h4> +<p>An application deployer may choose to pre-distribute the keytab files required to the node manager hosts in a yarn cluster. In that instance the appConfig.json requires the following property:</p> +<div class="codehilite"><pre><span class="p">.</span> <span class="p">.</span> <span class="p">.</span> +"<span class="n">components</span>"<span class="p">:</span> <span class="p">{</span> + "<span class="n">slider</span><span class="o">-</span><span class="n">appmaster</span>"<span class="p">:</span> <span class="p">{</span> + "<span class="n">jvm</span><span class="p">.</span><span class="n">heapsize</span>"<span class="p">:</span> "256<span class="n">M</span>"<span class="p">,</span> + "<span class="n">slider</span><span class="p">.</span><span class="n">am</span><span class="p">.</span><span class="n">keytab</span><span class="p">.</span><span class="n">local</span><span class="p">.</span><span class="n">path</span>"<span class="p">:</span> "<span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">security</span><span class="o">/</span><span class="n">keytabs</span><span class="o">/</span><span class="n">hbase</span><span class="p">.</span><span class="n">headless</span><span class="p">.</span><span class="n">keytab</span>" + <span class="p">}</span> +<span class="p">}</span> +</pre></div> + + +<p>The âslider.am.keytab.local.pathâ property provides the full path to the keytab file location and is mandatory for the local lookup mechanism. In this scenario the distribution of keytab files for the AM AND the application itself is the purview of the application deployer. So, for example, for an hbase deployment, the hbase site service keytab will have to be distributed as well and indicated in the hbase-site properties:</p> +<div class="codehilite"><pre> <span class="p">.</span> <span class="p">.</span> <span class="p">.</span> + "<span class="n">site</span><span class="p">.</span><span class="n">hbase</span><span class="o">-</span><span class="n">site</span><span class="p">.</span><span class="n">hbase</span><span class="p">.</span><span class="n">master</span><span class="p">.</span><span class="n">kerberos</span><span class="p">.</span><span class="n">principal</span>"<span class="p">:</span> "<span class="n">hbase</span><span class="o">/</span><span class="n">_HOST</span><span class="p">@</span><span class="n">EXAMPLE</span><span class="p">.</span><span class="n">COM</span>"<span class="p">,</span> + "<span class="n">site</span><span class="p">.</span><span class="n">hbase</span><span class="o">-</span><span class="n">site</span><span class="p">.</span><span class="n">hbase</span><span class="p">.</span><span class="n">master</span><span class="p">.</span><span class="n">keytab</span><span class="p">.</span><span class="n">file</span>"<span class="p">:</span> "<span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">security</span><span class="o">/</span><span class="n">keytabs</span><span class="o">/</span><span class="n">hbase</span><span class="p">.</span><span class="n">service</span><span class="p">.</span><span class="n">keytab</span>"<span class="p">,</span> + <span class="p">.</span> <span class="p">.</span> <span class="p">.</span> +</pre></div> + + +<h4 id="slider-keytab-distribution">Slider keytab distribution:</h4> +<p>The deployer can select to upload the keytab files for the AM and the application to an HDFS directory (with appropriate permissions set) and slider will localize the keytab files to locations accessible by the AM or the application containers:</p> +<div class="codehilite"><pre><span class="p">.</span> <span class="p">.</span> <span class="p">.</span> +"<span class="n">components</span>"<span class="p">:</span> <span class="p">{</span> + "<span class="n">slider</span><span class="o">-</span><span class="n">appmaster</span>"<span class="p">:</span> <span class="p">{</span> + "<span class="n">jvm</span><span class="p">.</span><span class="n">heapsize</span>"<span class="p">:</span> "256<span class="n">M</span>"<span class="p">,</span> + "<span class="n">slider</span><span class="p">.</span><span class="n">hdfs</span><span class="p">.</span><span class="n">keytab</span><span class="p">.</span><span class="n">dir</span>"<span class="p">:</span> "<span class="p">.</span><span class="n">slider</span><span class="o">/</span><span class="n">keytabs</span><span class="o">/</span><span class="n">hbase</span>"<span class="p">,</span> + "<span class="n">slider</span><span class="p">.</span><span class="n">am</span><span class="p">.</span><span class="n">login</span><span class="p">.</span><span class="n">keytab</span><span class="p">.</span><span class="n">name</span>"<span class="p">:</span> "<span class="n">hbase</span><span class="p">.</span><span class="n">headless</span><span class="p">.</span><span class="n">keytab</span>" + <span class="p">}</span> +<span class="p">}</span> +</pre></div> + + +<p>The âslider.hdfs.keytab.dirâ points to an HDFS path, relative to the userâs home directory (e.g. /users/hbase), in which slider can find all keytab files required for both AM login as well as application services (e.g. for hbase that would be the headless keytab for the AM and the service keytab for the HBase application components). If no value is specified, a default location of â.slider/keytabs/<cluster name>â is assumed. + The âslider.am.login.keytab.nameâ is the name of the keytab file (mandatory property), found within the specified directory, that the AM will use to lookup up the login principal and authenticate.</p> +<p>If leveraging the slider-based distribution mechanism, the keytab files for components will be accessible from a âkeytabsâ sub-directory of the container work folder, e.g.:</p> +<div class="codehilite"><pre> . . . + "site.hbase-site.hbase.master.kerberos.principal": "hbase/_h...@example.com", + "site.hbase-site.hbase.master.keytab.file": "<span class="cp">${</span><span class="n">AGENT_WORK_ROOT</span><span class="cp">}</span>/keytabs/hbase.service.keytab", + . . . +</pre></div> + + +<p>For both mechanisms above, the principal name used for authentication is either:</p> +<ul> +<li>The principal name established on the client side before invocation of the Slider CLI (the principal used to âkinitâ) or</li> +<li>The value specified for a âslider.keytab.principal.nameâ property. </li> +</ul> <h2 id="securing-communications-between-the-slider-client-and-the-slider-am">Securing communications between the Slider Client and the Slider AM.</h2> <p>When the AM is deployed in a secure cluster, it automatically uses Kerberos-authorized RPC channels. The client must acquire a