Author: lmccay
Date: Wed Sep 24 14:18:28 2014
New Revision: 1627328
URL: http://svn.apache.org/r1627328
Log:
KNOX-429 updates to user guide for 0.5.0 via Sumit Gupta
Modified:
knox/site/books/knox-0-4-0/deployment-overview.png
knox/site/books/knox-0-4-0/deployment-provider.png
knox/site/books/knox-0-4-0/deployment-service.png
knox/site/books/knox-0-4-0/runtime-overview.png
knox/site/books/knox-0-4-0/runtime-request-processing.png
knox/site/books/knox-0-5-0/knox-0-5-0.html
knox/site/index.html
knox/site/issue-tracking.html
knox/site/license.html
knox/site/mail-lists.html
knox/site/project-info.html
knox/site/team-list.html
knox/trunk/books/0.5.0/book_getting-started.md
knox/trunk/books/0.5.0/config.md
knox/trunk/books/0.5.0/config_authn.md
knox/trunk/books/0.5.0/quick_start.md
Modified: knox/site/books/knox-0-4-0/deployment-overview.png
URL:
http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/deployment-overview.png?rev=1627328&r1=1627327&r2=1627328&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-4-0/deployment-provider.png
URL:
http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/deployment-provider.png?rev=1627328&r1=1627327&r2=1627328&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-4-0/deployment-service.png
URL:
http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/deployment-service.png?rev=1627328&r1=1627327&r2=1627328&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-4-0/runtime-overview.png
URL:
http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/runtime-overview.png?rev=1627328&r1=1627327&r2=1627328&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-4-0/runtime-request-processing.png
URL:
http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/runtime-request-processing.png?rev=1627328&r1=1627327&r2=1627328&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-5-0/knox-0-5-0.html
URL:
http://svn.apache.org/viewvc/knox/site/books/knox-0-5-0/knox-0-5-0.html?rev=1627328&r1=1627327&r2=1627328&view=diff
==============================================================================
--- knox/site/books/knox-0-5-0/knox-0-5-0.html (original)
+++ knox/site/books/knox-0-5-0/knox-0-5-0.html Wed Sep 24 14:18:28 2014
@@ -76,20 +76,20 @@
<li>Do Hadoop with Knox</li>
</ol><h3><a id="1+-+Requirements"></a>1 - Requirements</h3><h4><a
id="Java"></a>Java</h4><p>Java 1.6 or later is required for the Knox Gateway
runtime. Use the command below to check the version of Java installed on the
system where Knox will be running.</p>
<pre><code>java -version
-</code></pre><h4><a id="Hadoop"></a>Hadoop</h4><p>Knox 0.4.0 supports Hadoop
2.x, the quick start instructions assume a Hadoop 2.x virtual machine based
environment. </p><h3><a id="2+-+Download+Hadoop+2.x+VM"></a>2 - Download Hadoop
2.x VM</h3><p>The quick start provides a link to download Hadoop 2.0 based
Hortonworks virtual machine <a
href="http://hortonworks.com/products/hdp-2/#install";>Sandbox</a>. Please note
Knox supports other Hadoop distributions and is configurable against a full
blown Hadoop cluster. Configuring Knox for Hadoop 2.x version, or Hadoop
deployed in EC2 or a custom Hadoop cluster is documented in advance deployment
guide.</p><h3><a id="3+-+Download+Apache+Knox+Gateway"></a>3 - Download Apache
Knox Gateway</h3><p>Download one of the distributions below from the <a
href="http://www.apache.org/dyn/closer.cgi/knox";>Apache mirrors</a>.</p>
+</code></pre><h4><a id="Hadoop"></a>Hadoop</h4><p>Knox 0.5.0 supports Hadoop
2.x, the quick start instructions assume a Hadoop 2.x virtual machine based
environment. </p><h3><a id="2+-+Download+Hadoop+2.x+VM"></a>2 - Download Hadoop
2.x VM</h3><p>The quick start provides a link to download Hadoop 2.0 based
Hortonworks virtual machine <a
href="http://hortonworks.com/products/hdp-2/#install";>Sandbox</a>. Please note
Knox supports other Hadoop distributions and is configurable against a full
blown Hadoop cluster. Configuring Knox for Hadoop 2.x version, or Hadoop
deployed in EC2 or a custom Hadoop cluster is documented in advance deployment
guide.</p><h3><a id="3+-+Download+Apache+Knox+Gateway"></a>3 - Download Apache
Knox Gateway</h3><p>Download one of the distributions below from the <a
href="http://www.apache.org/dyn/closer.cgi/knox";>Apache mirrors</a>.</p>
<ul>
- <li>Source archive: <a
href="http://www.apache.org/dyn/closer.cgi/knox/0.4.0/knox-0.4.0-src.zip";>knox-0.4.0-src.zip</a>
(<a
href="http://www.apache.org/dyn/closer.cgi/knox/0.4.0/knox-0.4.0-src.zip.asc";>PGP
signature</a>, <a
href="http://www.apache.org/dyn/closer.cgi/knox/0.4.0/knox-0.4.0-src.zip.sha";>SHA1
digest</a>, <a
href="http://www.apache.org/dyn/closer.cgi/knox/0.4.0/knox-0.4.0-src.zip.md5";>MD5
digest</a>)</li>
- <li>Binary archive: <a
href="http://www.apache.org/dyn/closer.cgi/knox/0.4.0/knox-0.4.0.zip";>knox-0.4.0.zip</a>
(<a
href="http://www.apache.org/dyn/closer.cgi/knox/0.4.0/knox-0.4.0.zip.asc";>PGP
signature</a>, <a
href="http://www.apache.org/dyn/closer.cgi/knox/0.4.0/knox-0.4.0.zip.sha";>SHA1
digest</a>, <a
href="http://www.apache.org/dyn/closer.cgi/knox/0.4.0/knox-0.4.0.zip.md5";>MD5
digest</a>)</li>
+ <li>Source archive: <a
href="http://www.apache.org/dyn/closer.cgi/knox/0.5.0/knox-0.4.0-src.zip";>knox-0.5.0-src.zip</a>
(<a
href="http://www.apache.org/dyn/closer.cgi/knox/0.5.0/knox-0.5.0-src.zip.asc";>PGP
signature</a>, <a
href="http://www.apache.org/dyn/closer.cgi/knox/0.5.0/knox-0.5.0-src.zip.sha";>SHA1
digest</a>, <a
href="http://www.apache.org/dyn/closer.cgi/knox/0.5.0/knox-0.5.0-src.zip.md5";>MD5
digest</a>)</li>
+ <li>Binary archive: <a
href="http://www.apache.org/dyn/closer.cgi/knox/0.5.0/knox-0.5.0.zip";>knox-0.5.0.zip</a>
(<a
href="http://www.apache.org/dyn/closer.cgi/knox/0.5.0/knox-0.5.0.zip.asc";>PGP
signature</a>, <a
href="http://www.apache.org/dyn/closer.cgi/knox/0.5.0/knox-0.5.0.zip.sha";>SHA1
digest</a>, <a
href="http://www.apache.org/dyn/closer.cgi/knox/0.5.0/knox-0.5.0.zip.md5";>MD5
digest</a>)</li>
</ul><p>Apache Knox Gateway releases are available under the <a
href="http://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version
2.0</a>. See the NOTICE file contained in each release artifact for applicable
copyright attribution notices.</p><h3><a id="Verify"></a>Verify</h3><p>While
recommended, verify is an optional step. You can verify the integrity of any
downloaded files using the PGP signatures. Please read <a
href="http://httpd.apache.org/dev/verification.html";>Verifying Apache HTTP
Server Releases</a> for more information on why you should verify our
releases.</p><p>The PGP signatures can be verified using PGP or GPG. First
download the KEYS file as well as the .asc signature files for the relevant
release packages. Make sure you get these files from the main distribution
directory linked above, rather than from a mirror. Then verify the signatures
using one of the methods below.</p>
<pre><code>% pgpk -a KEYS
-% pgpv knox-0.4.0.zip.asc
+% pgpv knox-0.5.0.zip.asc
</code></pre><p>or</p>
<pre><code>% pgp -ka KEYS
-% pgp knox-0.4.0.zip.asc
+% pgp knox-0.5.0.zip.asc
</code></pre><p>or</p>
<pre><code>% gpg --import KEYS
-% gpg --verify knox-0.4.0.zip.asc
-</code></pre><h3><a id="4+-+Start+Hadoop+virtual+machine"></a>4 - Start Hadoop
virtual machine</h3><p>Start the Hadoop virtual machine.</p><h3><a
id="5+-+Install+Knox"></a>5 - Install Knox</h3><p>The steps required to install
the gateway will vary depending upon which distribution format (zip | rpm) was
downloaded. In either case you will end up with a directory where the gateway
is installed. This directory will be referred to as your
<code>{GATEWAY_HOME}</code> throughout this document.</p><h4><a
id="ZIP"></a>ZIP</h4><p>If you downloaded the Zip distribution you can simply
extract the contents into a directory. The example below provides a command
that can be executed to do this. Note the <code>{VERSION}</code> portion of the
command must be replaced with an actual Apache Knox Gateway version number.
This might be 0.4.0 for example and must patch the value in the file
downloaded.</p>
+% gpg --verify knox-0.5.0.zip.asc
+</code></pre><h3><a id="5+-+Start+Hadoop+virtual+machine"></a>5 - Start Hadoop
virtual machine</h3><p>Start the Hadoop virtual machine.</p><h3><a
id="5+-+Install+Knox"></a>5 - Install Knox</h3><p>The steps required to install
the gateway will vary depending upon which distribution format (zip | rpm) was
downloaded. In either case you will end up with a directory where the gateway
is installed. This directory will be referred to as your
<code>{GATEWAY_HOME}</code> throughout this document.</p><h4><a
id="ZIP"></a>ZIP</h4><p>If you downloaded the Zip distribution you can simply
extract the contents into a directory. The example below provides a command
that can be executed to do this. Note the <code>{VERSION}</code> portion of the
command must be replaced with an actual Apache Knox Gateway version number.
This might be 0.4.0 for example and must patch the value in the file
downloaded.</p>
<pre><code>jar xf knox-{VERSION}.zip
</code></pre><p>This will create a directory <code>knox-{VERSION}</code> in
your current directory. The directory <code>knox-{VERSION}</code> will
considered your <code>{GATEWAY_HOME}</code></p><h3><a
id="6+-+Start+LDAP+embedded+in+Knox"></a>6 - Start LDAP embedded in
Knox</h3><p>Knox comes with an LDAP server for demonstration purposes.</p>
<pre><code>cd {GATEWAY_HOME}
@@ -121,7 +121,7 @@ Server: Jetty(6.1.26)
{"accessTime":0,"blockSize":0,"group":"hdfs","length":0,"modificationTime":1350596040075,"owner":"hdfs","pathSuffix":"tmp","permission":"777","replication":0,"type":"DIRECTORY"},
{"accessTime":0,"blockSize":0,"group":"hdfs","length":0,"modificationTime":1350595857178,"owner":"hdfs","pathSuffix":"user","permission":"755","replication":0,"type":"DIRECTORY"}
]}}
-</code></pre><h4><a id="Submit+a+MR+job+via+Knox."></a>Submit a MR job via
Knox.</h4><h4><a id="Get+status+of+a+MR+job+via+Knox."></a>Get status of a MR
job via Knox.</h4><h4><a id="Cancel+a+MR+job+via+Knox."></a>Cancel a MR job via
Knox.</h4><h3><a id="More+Examples"></a>More Examples</h3><h2><a
id="Apache+Knox+Details"></a>Apache Knox Details</h2><p>This section provides
everything you need to know to get the Knox gateway up and running against a
Hadoop cluster.</p><h4><a id="Hadoop"></a>Hadoop</h4><p>An existing Hadoop 2.x
cluster is required for Knox 0.4.0 to sit in front of and protect. It is
possible to use a Hadoop cluster deployed on EC2 but this will require
additional configuration not covered here. It is also possible to protect
access to a services of a Hadoop cluster that is secured with kerberos. This
too requires additional configuration that is described in other sections of
this guide. See <a href="#Supported+Services">Supported Services</a> for
details on what is s
upported for this release.</p><p>The Hadoop cluster should be ensured to have
at least WebHDFS, WebHCat (i.e. Templeton) and Oozie configured, deployed and
running. HBase/Stargate and Hive can also be accessed via the Knox Gateway
given the proper versions and configuration.</p><p>The instructions that follow
assume a few things:</p>
+</code></pre><h4><a id="Submit+a+MR+job+via+Knox."></a>Submit a MR job via
Knox.</h4><h4><a id="Get+status+of+a+MR+job+via+Knox."></a>Get status of a MR
job via Knox.</h4><h4><a id="Cancel+a+MR+job+via+Knox."></a>Cancel a MR job via
Knox.</h4><h3><a id="More+Examples"></a>More Examples</h3><h2><a
id="Apache+Knox+Details"></a>Apache Knox Details</h2><p>This section provides
everything you need to know to get the Knox gateway up and running against a
Hadoop cluster.</p><h4><a id="Hadoop"></a>Hadoop</h4><p>An existing Hadoop 2.x
cluster is required for Knox 0.5.0 to sit in front of and protect. It is
possible to use a Hadoop cluster deployed on EC2 but this will require
additional configuration not covered here. It is also possible to protect
access to a services of a Hadoop cluster that is secured with kerberos. This
too requires additional configuration that is described in other sections of
this guide. See <a href="#Supported+Services">Supported Services</a> for
details on what is s
upported for this release.</p><p>The Hadoop cluster should be ensured to have
at least WebHDFS, WebHCat (i.e. Templeton) and Oozie configured, deployed and
running. HBase/Stargate and Hive can also be accessed via the Knox Gateway
given the proper versions and configuration.</p><p>The instructions that follow
assume a few things:</p>
<ol>
<li>The gateway is <em>not</em> collocated with the Hadoop clusters
themselves.</li>
<li>The host names and IP addresses of the cluster services are accessible
by the gateway where ever it happens to be running.</li>
@@ -264,6 +264,12 @@ Server: Jetty(6.1.26)
<td><img src="check.png" alt="y"/> </td>
<td><img src="check.png" alt="y"/> </td>
</tr>
+ <tr>
+ <td>Yarn </td>
+ <td>2.5.0 </td>
+ <td><img src="check.png" alt="y"/> </td>
+ <td><img src="check.png" alt="y"/> </td>
+ </tr>
</tbody>
</table><h3><a id="More+Examples"></a>More Examples</h3><p>These examples
provide more detail about how to access various Apache Hadoop services via the
Apache Knox Gateway.</p>
<ul>
@@ -272,6 +278,7 @@ Server: Jetty(6.1.26)
<li><a href="#Oozie+Examples">Oozie Examples</a></li>
<li><a href="#HBase+Examples">HBase Examples</a></li>
<li><a href="#Hive+Examples">Hive Examples</a></li>
+ <li><a href="#Yarn+Examples">Yarn Examples</a></li>
</ul><h2><a id="Gateway+Details"></a>Gateway Details</h2><p>This section
describes the details of the Knox Gateway itself. Including: </p>
<ul>
<li>How URLs are mapped between a gateway that services multiple Hadoop
clusters and the clusters themselves</li>
@@ -336,7 +343,7 @@ https://{gateway-host}:{gateway-port}/{g
<url>http://localhost:50070/webhdfs</url>
</service>
</code></pre>
-<dl><dt>/topology/service</dt><dd>Provider information about a particular
service within the Hadoop cluster. Not all services are necessarily exposed as
gateway endpoints.</dd><dt>/topology/service/role</dt><dd>Identifies the role
of this service. Currently supported roles are: WEBHDFS, WEBHCAT, WEBHBASE,
OOZIE, HIVE, NAMENODE, JOBTRACKER Additional service roles can be supported via
plugins.</dd><dt>topology/service/url</dt><dd>The URL identifying the location
of a particular service within the Hadoop cluster.</dd>
+<dl><dt>/topology/service</dt><dd>Provider information about a particular
service within the Hadoop cluster. Not all services are necessarily exposed as
gateway endpoints.</dd><dt>/topology/service/role</dt><dd>Identifies the role
of this service. Currently supported roles are: WEBHDFS, WEBHCAT, WEBHBASE,
OOZIE, HIVE, NAMENODE, JOBTRACKER, RESOURCEMANAGER Additional service roles can
be supported via plugins.</dd><dt>topology/service/url</dt><dd>The URL
identifying the location of a particular service within the Hadoop cluster.</dd>
</dl><h4><a id="Hostmap+Provider"></a>Hostmap Provider</h4><p>The purpose of
the Hostmap provider is to handle situations where host are known by one name
within the cluster and another name externally. This frequently occurs when
virtual machines are used and in particular when using cloud hosting services.
Currently, the Hostmap provider is configured as part of the topology file. The
basic structure is shown below.</p>
<pre><code><topology>
<gateway>
@@ -615,7 +622,7 @@ ldapRealm=org.apache.shiro.realm.ldap.Jn
ldapRealm.contextFactory.authenticationMechanism=simple
ldapRealm.contextFactory.url=ldap://localhost:33389
ldapRealm.userDnTemplate=uid={0},ou=people,dc=hadoop,dc=apache,dc=org
-</code></pre><p>In order to fit into the context of an INI file format, at
deployment time we interrogate the paramaters provided in the provider
configuration and parse the INI section out of the paramter names. The
following provider config illustrates this approach. Notice that the section
names in the above shiro.ini match the beginning of the param names that are in
the following config:</p>
+</code></pre><p>In order to fit into the context of an INI file format, at
deployment time we interrogate the parameters provided in the provider
configuration and parse the INI section out of the parameter names. The
following provider config illustrates this approach. Notice that the section
names in the above shiro.ini match the beginning of the param names that are in
the following config:</p>
<pre><code><gateway>
<provider>
<role>authentication</role>
@@ -642,7 +649,7 @@ ldapRealm.userDnTemplate=uid={0},ou=peop
<value>authcBasic</value>
</param>
</provider>
-</code></pre><p>This happens to be the way that we are currently configuring
Shiro for BASIC/LDAP authentication. This same config approach may be used to
achieve other authentication mechanisms or variations on this one. We however
have not tested additional uses for it for this release.</p><h4><a
id="LDAP+Configuration"></a>LDAP Configuration</h4><p>This section discusses
the LDAP configuration used above for the Shiro Provider. Some of these
configuration elements will need to be customized to reflect your deployment
environment.</p><p><strong>main.ldapRealm</strong> - this element indicates the
fully qualified classname of the Shiro realm to be used in authenticating the
user. The classname provided by default in the sample is the
<code>org.apache.shiro.realm.ldap.JndiLdapRealm</code> this implementation
provides us with the ability to authenticate but by default has authorization
disabled. In order to provide authorization - which is seen by Shiro as
dependent on an LDAP schema
that is specific to each organization - an extension of JndiLdapRealm is
generally used to override and implement the doGetAuhtorizationInfo method. In
this particular release we are providing a simple authorization provider that
can be used along with the Shiro authentication
provider.</p><p><strong>main.ldapRealm.userDnTemplate</strong> - in order to
bind a simple username to an LDAP server that generally requires a full
distinguished name (DN), we must provide the template into which the simple
username will be inserted. This template allows for the creation of a DN by
injecting the simple username into the common name (CN) portion of the DN.
<strong>This element will need to be customized to reflect your deployment
environment.</strong> The template provided in the sample is only an example
and is valid only within the LDAP schema distributed with Knox and is
represented by the users.ldif file in the {GATEWAY_HOME}conf
directory.</p><p><strong>main.ldapRealm.contextFactory.url<
/strong> - this element is the URL that represents the host and port of LDAP
server. It also includes the scheme of the protocol to use. This may be either
ldap or ldaps depending on whether you are communicating with the LDAP over SSL
(higly recommended). <strong>This element will need to be cusomized to reflect
your deployment
environment.</strong>.</p><p><strong>main.ldapRealm.contextFactory.authenticationMechanism</strong>
- this element indicates the type of authentication that should be performed
against the LDAP server. The current default value is <code>simple</code> which
indicates a simple bind operation. This element should not need to be modified
and no mechanism other than a simple bind has been tested for this particular
release.</p><p><strong>urls./</strong>** - this element represents a single
URL_Ant_Path_Expression and the value the Shiro filter chain to apply to it.
This particular sample indicates that all paths into the application have the
same Shiro filter cha
in applied. The paths are relative to the application context path. The use of
the value <code>authcBasic</code> here indicates that BASIC authentication is
expected for every path into the application. Adding an additional Shiro filter
to that chain for validating that the request isSecure() and over SSL can be
achieved by changing the value to <code>ssl, authcBasic</code>. It is not
likely that you need to change this element for your environment.</p><h4><a
id="Active+Directory+-+Special+Note"></a>Active Directory - Special
Note</h4><p>You would use LDAP configuration as documented above to
authenticate against Active Directory as well.</p><p>Some Active Directory
specifc things to keep in mind:</p><p>Typical AD main.ldapRealm.userDnTemplate
value looks slightly different, such as
cn={0},cn=users,DC=lab,DC=sample,dc=com</p><p>Please compare this with a
typical Apache DS main.ldapRealm.userDnTemplate value and make note of the
difference. uid={0},ou=people,dc=hadoop,dc=apache,dc=
org</p><p>If your AD is configured to authenticate based on just the cn and
password and does not require user DN, you do not have to specify value for
main.ldapRealm.userDnTemplate.</p><h4><a
id="LDAP+over+SSL+(LDAPS)+Configuration"></a>LDAP over SSL (LDAPS)
Configuration</h4><p>In order to communicate with your LDAP server over SSL
(again, highly recommended), you will need to modify the topology file in a
couple ways and possibly provision some keying material.</p>
+</code></pre><p>This happens to be the way that we are currently configuring
Shiro for BASIC/LDAP authentication. This same config approach may be used to
achieve other authentication mechanisms or variations on this one. We however
have not tested additional uses for it for this release.</p><h4><a
id="LDAP+Configuration"></a>LDAP Configuration</h4><p>This section discusses
the LDAP configuration used above for the Shiro Provider. Some of these
configuration elements will need to be customized to reflect your deployment
environment.</p><p><strong>main.ldapRealm</strong> - this element indicates the
fully qualified classname of the Shiro realm to be used in authenticating the
user. The classname provided by default in the sample is the
<code>org.apache.shiro.realm.ldap.JndiLdapRealm</code> this implementation
provides us with the ability to authenticate but by default has authorization
disabled. In order to provide authorization - which is seen by Shiro as
dependent on an LDAP schema
that is specific to each organization - an extension of JndiLdapRealm is
generally used to override and implement the doGetAuhtorizationInfo method. In
this particular release we are providing a simple authorization provider that
can be used along with the Shiro authentication
provider.</p><p><strong>main.ldapRealm.userDnTemplate</strong> - in order to
bind a simple username to an LDAP server that generally requires a full
distinguished name (DN), we must provide the template into which the simple
username will be inserted. This template allows for the creation of a DN by
injecting the simple username into the common name (CN) portion of the DN.
<strong>This element will need to be customized to reflect your deployment
environment.</strong> The template provided in the sample is only an example
and is valid only within the LDAP schema distributed with Knox and is
represented by the users.ldif file in the {GATEWAY_HOME}/conf
directory.</p><p><strong>main.ldapRealm.contextFactory.url
</strong> - this element is the URL that represents the host and port of LDAP
server. It also includes the scheme of the protocol to use. This may be either
ldap or ldaps depending on whether you are communicating with the LDAP over SSL
(higly recommended). <strong>This element will need to be cusomized to reflect
your deployment
environment.</strong>.</p><p><strong>main.ldapRealm.contextFactory.authenticationMechanism</strong>
- this element indicates the type of authentication that should be performed
against the LDAP server. The current default value is <code>simple</code> which
indicates a simple bind operation. This element should not need to be modified
and no mechanism other than a simple bind has been tested for this particular
release.</p><p><strong>urls./</strong>** - this element represents a single
URL_Ant_Path_Expression and the value the Shiro filter chain to apply to it.
This particular sample indicates that all paths into the application have the
same Shiro filter ch
ain applied. The paths are relative to the application context path. The use
of the value <code>authcBasic</code> here indicates that BASIC authentication
is expected for every path into the application. Adding an additional Shiro
filter to that chain for validating that the request isSecure() and over SSL
can be achieved by changing the value to <code>ssl, authcBasic</code>. It is
not likely that you need to change this element for your environment.</p><h4><a
id="Active+Directory+-+Special+Note"></a>Active Directory - Special
Note</h4><p>You would use LDAP configuration as documented above to
authenticate against Active Directory as well.</p><p>Some Active Directory
specifc things to keep in mind:</p><p>Typical AD main.ldapRealm.userDnTemplate
value looks slightly different, such as
cn={0},cn=users,DC=lab,DC=sample,dc=com</p><p>Please compare this with a
typical Apache DS main.ldapRealm.userDnTemplate value and make note of the
difference. uid={0},ou=people,dc=hadoop,dc=apache,dc
=org</p><p>If your AD is configured to authenticate based on just the cn and
password and does not require user DN, you do not have to specify value for
main.ldapRealm.userDnTemplate.</p><h4><a
id="LDAP+over+SSL+(LDAPS)+Configuration"></a>LDAP over SSL (LDAPS)
Configuration</h4><p>In order to communicate with your LDAP server over SSL
(again, highly recommended), you will need to modify the topology file in a
couple ways and possibly provision some keying material.</p>
<ol>
<li><strong>main.ldapRealm.contextFactory.url</strong> must be changed to
have the <code>ldaps</code> protocol scheme and the port must be the SSL
listener port on your LDAP server.</li>
<li>Identity certificate (keypair) provisioned to LDAP server - your LDAP
server specific documentation should indicate what is requried for providing a
cert or keypair to represent the LDAP server identity to connecting
clients.</li>
Modified: knox/site/index.html
URL:
http://svn.apache.org/viewvc/knox/site/index.html?rev=1627328&r1=1627327&r2=1627328&view=diff
==============================================================================
--- knox/site/index.html (original)
+++ knox/site/index.html Wed Sep 24 14:18:28 2014
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2014-09-17 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2014-09-24 -->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css"
media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20140917" />
+ <meta name="Date-Revision-yyyymmdd" content="20140924" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a
href="https://cwiki.apache.org/confluence/display/KNOX/Index";
class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published:
2014-09-17</span>
+ | <span id="publishDate">Last Published:
2014-09-24</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/issue-tracking.html
URL:
http://svn.apache.org/viewvc/knox/site/issue-tracking.html?rev=1627328&r1=1627327&r2=1627328&view=diff
==============================================================================
--- knox/site/issue-tracking.html (original)
+++ knox/site/issue-tracking.html Wed Sep 24 14:18:28 2014
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2014-09-17 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2014-09-24 -->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css"
media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20140917" />
+ <meta name="Date-Revision-yyyymmdd" content="20140924" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a
href="https://cwiki.apache.org/confluence/display/KNOX/Index";
class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published:
2014-09-17</span>
+ | <span id="publishDate">Last Published:
2014-09-24</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/license.html
URL:
http://svn.apache.org/viewvc/knox/site/license.html?rev=1627328&r1=1627327&r2=1627328&view=diff
==============================================================================
--- knox/site/license.html (original)
+++ knox/site/license.html Wed Sep 24 14:18:28 2014
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2014-09-17 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2014-09-24 -->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css"
media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20140917" />
+ <meta name="Date-Revision-yyyymmdd" content="20140924" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a
href="https://cwiki.apache.org/confluence/display/KNOX/Index";
class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published:
2014-09-17</span>
+ | <span id="publishDate">Last Published:
2014-09-24</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/mail-lists.html
URL:
http://svn.apache.org/viewvc/knox/site/mail-lists.html?rev=1627328&r1=1627327&r2=1627328&view=diff
==============================================================================
--- knox/site/mail-lists.html (original)
+++ knox/site/mail-lists.html Wed Sep 24 14:18:28 2014
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2014-09-17 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2014-09-24 -->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css"
media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20140917" />
+ <meta name="Date-Revision-yyyymmdd" content="20140924" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a
href="https://cwiki.apache.org/confluence/display/KNOX/Index";
class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published:
2014-09-17</span>
+ | <span id="publishDate">Last Published:
2014-09-24</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/project-info.html
URL:
http://svn.apache.org/viewvc/knox/site/project-info.html?rev=1627328&r1=1627327&r2=1627328&view=diff
==============================================================================
--- knox/site/project-info.html (original)
+++ knox/site/project-info.html Wed Sep 24 14:18:28 2014
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2014-09-17 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2014-09-24 -->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css"
media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20140917" />
+ <meta name="Date-Revision-yyyymmdd" content="20140924" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a
href="https://cwiki.apache.org/confluence/display/KNOX/Index";
class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published:
2014-09-17</span>
+ | <span id="publishDate">Last Published:
2014-09-24</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/team-list.html
URL:
http://svn.apache.org/viewvc/knox/site/team-list.html?rev=1627328&r1=1627327&r2=1627328&view=diff
==============================================================================
--- knox/site/team-list.html (original)
+++ knox/site/team-list.html Wed Sep 24 14:18:28 2014
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2014-09-17 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2014-09-24 -->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css"
media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20140917" />
+ <meta name="Date-Revision-yyyymmdd" content="20140924" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a
href="https://cwiki.apache.org/confluence/display/KNOX/Index";
class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published:
2014-09-17</span>
+ | <span id="publishDate">Last Published:
2014-09-24</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/trunk/books/0.5.0/book_getting-started.md
URL:
http://svn.apache.org/viewvc/knox/trunk/books/0.5.0/book_getting-started.md?rev=1627328&r1=1627327&r2=1627328&view=diff
==============================================================================
--- knox/trunk/books/0.5.0/book_getting-started.md (original)
+++ knox/trunk/books/0.5.0/book_getting-started.md Wed Sep 24 14:18:28 2014
@@ -21,7 +21,7 @@ This section provides everything you nee
#### Hadoop ####
-An existing Hadoop 2.x cluster is required for Knox 0.4.0 to sit in front of
and protect.
+An existing Hadoop 2.x cluster is required for Knox 0.5.0 to sit in front of
and protect.
It is possible to use a Hadoop cluster deployed on EC2 but this will require
additional configuration not covered here.
It is also possible to protect access to a services of a Hadoop cluster that
is secured with kerberos.
This too requires additional configuration that is described in other sections
of this guide.
@@ -80,6 +80,7 @@ This table enumerates the versions of va
| HBase/Stargate | 0.98.0 | ![y] | ![y] |
| Hive (via WebHCat) | 0.13.0 | ![y] | ![y] |
| Hive (via JDBC) | 0.13.0 | ![y] | ![y] |
+| Yarn | 2.5.0 | ![y] | ![y] |
### More Examples ###
@@ -91,3 +92,4 @@ These examples provide more detail about
* #[Oozie Examples]
* #[HBase Examples]
* #[Hive Examples]
+* #[Yarn Examples]
Modified: knox/trunk/books/0.5.0/config.md
URL:
http://svn.apache.org/viewvc/knox/trunk/books/0.5.0/config.md?rev=1627328&r1=1627327&r2=1627328&view=diff
==============================================================================
--- knox/trunk/books/0.5.0/config.md (original)
+++ knox/trunk/books/0.5.0/config.md Wed Sep 24 14:18:28 2014
@@ -106,7 +106,7 @@ Not all services are necessarily exposed
/topology/service/role
: Identifies the role of this service.
-Currently supported roles are: WEBHDFS, WEBHCAT, WEBHBASE, OOZIE, HIVE,
NAMENODE, JOBTRACKER
+Currently supported roles are: WEBHDFS, WEBHCAT, WEBHBASE, OOZIE, HIVE,
NAMENODE, JOBTRACKER, RESOURCEMANAGER
Additional service roles can be supported via plugins.
topology/service/url
Modified: knox/trunk/books/0.5.0/config_authn.md
URL:
http://svn.apache.org/viewvc/knox/trunk/books/0.5.0/config_authn.md?rev=1627328&r1=1627327&r2=1627328&view=diff
==============================================================================
--- knox/trunk/books/0.5.0/config_authn.md (original)
+++ knox/trunk/books/0.5.0/config_authn.md Wed Sep 24 14:18:28 2014
@@ -61,7 +61,7 @@ The following example illustrates a conf
ldapRealm.contextFactory.url=ldap://localhost:33389
ldapRealm.userDnTemplate=uid={0},ou=people,dc=hadoop,dc=apache,dc=org
-In order to fit into the context of an INI file format, at deployment time we
interrogate the paramaters provided in the provider configuration and parse the
INI section out of the paramter names. The following provider config
illustrates this approach. Notice that the section names in the above shiro.ini
match the beginning of the param names that are in the following config:
+In order to fit into the context of an INI file format, at deployment time we
interrogate the parameters provided in the provider configuration and parse the
INI section out of the parameter names. The following provider config
illustrates this approach. Notice that the section names in the above shiro.ini
match the beginning of the param names that are in the following config:
<gateway>
<provider>
@@ -98,7 +98,7 @@ This section discusses the LDAP configur
**main.ldapRealm** - this element indicates the fully qualified classname of
the Shiro realm to be used in authenticating the user. The classname provided
by default in the sample is the `org.apache.shiro.realm.ldap.JndiLdapRealm`
this implementation provides us with the ability to authenticate but by default
has authorization disabled. In order to provide authorization - which is seen
by Shiro as dependent on an LDAP schema that is specific to each organization -
an extension of JndiLdapRealm is generally used to override and implement the
doGetAuhtorizationInfo method. In this particular release we are providing a
simple authorization provider that can be used along with the Shiro
authentication provider.
-**main.ldapRealm.userDnTemplate** - in order to bind a simple username to an
LDAP server that generally requires a full distinguished name (DN), we must
provide the template into which the simple username will be inserted. This
template allows for the creation of a DN by injecting the simple username into
the common name (CN) portion of the DN. **This element will need to be
customized to reflect your deployment environment.** The template provided in
the sample is only an example and is valid only within the LDAP schema
distributed with Knox and is represented by the users.ldif file in the
{GATEWAY_HOME}conf directory.
+**main.ldapRealm.userDnTemplate** - in order to bind a simple username to an
LDAP server that generally requires a full distinguished name (DN), we must
provide the template into which the simple username will be inserted. This
template allows for the creation of a DN by injecting the simple username into
the common name (CN) portion of the DN. **This element will need to be
customized to reflect your deployment environment.** The template provided in
the sample is only an example and is valid only within the LDAP schema
distributed with Knox and is represented by the users.ldif file in the
{GATEWAY_HOME}/conf directory.
**main.ldapRealm.contextFactory.url** - this element is the URL that
represents the host and port of LDAP server. It also includes the scheme of the
protocol to use. This may be either ldap or ldaps depending on whether you are
communicating with the LDAP over SSL (higly recommended). **This element will
need to be cusomized to reflect your deployment environment.**.
Modified: knox/trunk/books/0.5.0/quick_start.md
URL:
http://svn.apache.org/viewvc/knox/trunk/books/0.5.0/quick_start.md?rev=1627328&r1=1627327&r2=1627328&view=diff
==============================================================================
--- knox/trunk/books/0.5.0/quick_start.md (original)
+++ knox/trunk/books/0.5.0/quick_start.md Wed Sep 24 14:18:28 2014
@@ -41,7 +41,7 @@ Use the command below to check the versi
#### Hadoop ####
-Knox 0.4.0 supports Hadoop 2.x, the quick start instructions assume a Hadoop
2.x virtual machine based environment.
+Knox 0.5.0 supports Hadoop 2.x, the quick start instructions assume a Hadoop
2.x virtual machine based environment.
### 2 - Download Hadoop 2.x VM ###
@@ -53,17 +53,17 @@ Configuring Knox for Hadoop 2.x version,
Download one of the distributions below from the [Apache mirrors][mirror].
-* Source archive: [knox-0.4.0-src.zip][src-zip] ([PGP signature][src-pgp],
[SHA1 digest][src-sha], [MD5 digest][src-md5])
-* Binary archive: [knox-0.4.0.zip][bin-zip] ([PGP signature][bin-pgp], [SHA1
digest][bin-sha], [MD5 digest][bin-md5])
+* Source archive: [knox-0.5.0-src.zip][src-zip] ([PGP signature][src-pgp],
[SHA1 digest][src-sha], [MD5 digest][src-md5])
+* Binary archive: [knox-0.5.0.zip][bin-zip] ([PGP signature][bin-pgp], [SHA1
digest][bin-sha], [MD5 digest][bin-md5])
-[src-zip]: http://www.apache.org/dyn/closer.cgi/knox/0.4.0/knox-0.4.0-src.zip
-[src-sha]:
http://www.apache.org/dyn/closer.cgi/knox/0.4.0/knox-0.4.0-src.zip.sha
-[src-pgp]:
http://www.apache.org/dyn/closer.cgi/knox/0.4.0/knox-0.4.0-src.zip.asc
-[src-md5]:
http://www.apache.org/dyn/closer.cgi/knox/0.4.0/knox-0.4.0-src.zip.md5
-[bin-zip]: http://www.apache.org/dyn/closer.cgi/knox/0.4.0/knox-0.4.0.zip
-[bin-pgp]: http://www.apache.org/dyn/closer.cgi/knox/0.4.0/knox-0.4.0.zip.asc
-[bin-sha]: http://www.apache.org/dyn/closer.cgi/knox/0.4.0/knox-0.4.0.zip.sha
-[bin-md5]: http://www.apache.org/dyn/closer.cgi/knox/0.4.0/knox-0.4.0.zip.md5
+[src-zip]: http://www.apache.org/dyn/closer.cgi/knox/0.5.0/knox-0.4.0-src.zip
+[src-sha]:
http://www.apache.org/dyn/closer.cgi/knox/0.5.0/knox-0.5.0-src.zip.sha
+[src-pgp]:
http://www.apache.org/dyn/closer.cgi/knox/0.5.0/knox-0.5.0-src.zip.asc
+[src-md5]:
http://www.apache.org/dyn/closer.cgi/knox/0.5.0/knox-0.5.0-src.zip.md5
+[bin-zip]: http://www.apache.org/dyn/closer.cgi/knox/0.5.0/knox-0.5.0.zip
+[bin-pgp]: http://www.apache.org/dyn/closer.cgi/knox/0.5.0/knox-0.5.0.zip.asc
+[bin-sha]: http://www.apache.org/dyn/closer.cgi/knox/0.5.0/knox-0.5.0.zip.sha
+[bin-md5]: http://www.apache.org/dyn/closer.cgi/knox/0.5.0/knox-0.5.0.zip.md5
Apache Knox Gateway releases are available under the [Apache License, Version
2.0][asl].
See the NOTICE file contained in each release artifact for applicable
copyright attribution notices.
@@ -80,19 +80,19 @@ Make sure you get these files from the m
Then verify the signatures using one of the methods below.
% pgpk -a KEYS
- % pgpv knox-0.4.0.zip.asc
+ % pgpv knox-0.5.0.zip.asc
or
% pgp -ka KEYS
- % pgp knox-0.4.0.zip.asc
+ % pgp knox-0.5.0.zip.asc
or
% gpg --import KEYS
- % gpg --verify knox-0.4.0.zip.asc
+ % gpg --verify knox-0.5.0.zip.asc
-### 4 - Start Hadoop virtual machine ###
+### 5 - Start Hadoop virtual machine ###
Start the Hadoop virtual machine.
