Author: buildbot Date: Mon Mar 2 19:24:03 2015 New Revision: 941978 Log: Staging update by buildbot for gora
Modified: websites/staging/gora/trunk/content/ (props changed) websites/staging/gora/trunk/content/current/gora-shims.html Propchange: websites/staging/gora/trunk/content/ ------------------------------------------------------------------------------ --- cms:source-revision (original) +++ cms:source-revision Mon Mar 2 19:24:03 2015 @@ -1 +1 @@ -1663395 +1663401 Modified: websites/staging/gora/trunk/content/current/gora-shims.html ============================================================================== --- websites/staging/gora/trunk/content/current/gora-shims.html (original) +++ websites/staging/gora/trunk/content/current/gora-shims.html Mon Mar 2 19:24:03 2015 @@ -185,25 +185,21 @@ into your software stack. </p> <li><a href="#gora-shims-distribution">gora-shims-distribution</a><ul> <li><a href="#description">Description</a></li> <li><a href="#dependency-definition">Dependency Definition</a></li> -<li><a href="#direct-dependency">Direct Dependency</a></li> </ul> </li> <li><a href="#gora-shims-hadoop">gora-shims-hadoop</a><ul> <li><a href="#description_1">Description</a></li> -<li><a href="#dependency">Dependency</a></li> -<li><a href="#direct-dependency_1">Direct Dependency</a></li> +<li><a href="#dependency-definition_1">Dependency Definition</a></li> </ul> </li> <li><a href="#gora-shims-hadoop1">gora-shims-hadoop1</a><ul> <li><a href="#description_2">Description</a></li> -<li><a href="#dependency_1">Dependency</a></li> -<li><a href="#direct-dependency_2">Direct Dependency</a></li> +<li><a href="#dependency-definition_2">Dependency Definition</a></li> </ul> </li> <li><a href="#gora-shims-hadoop2">gora-shims-hadoop2</a><ul> <li><a href="#description_3">Description</a></li> -<li><a href="#dependency_2">Dependency</a></li> -<li><a href="#direct-dependency_3">Direct Dependency</a></li> +<li><a href="#dependency-definition_3">Dependency Definition</a></li> </ul> </li> </ul> @@ -216,19 +212,102 @@ on all of the other shims modules but co should be used as below.</strong></p> <h2 id="dependency-definition">Dependency Definition</h2> <p><a href="http://search.maven.org/#artifactdetails|org.apache.gora|gora-shims-distribution|0.6|bundle">Click Here</a></p> -<h2 id="direct-dependency">Direct Dependency</h2> <h1 id="gora-shims-hadoop">gora-shims-hadoop</h1> <h2 id="description_1">Description</h2> -<h2 id="dependency">Dependency</h2> -<h2 id="direct-dependency_1">Direct Dependency</h2> +<p>This module contains functional Java code for enabling dynamic selection of the Hadoop environment based upon the +presence of <strong>org.apache.hadoop.util.VersionInfo</strong>. This is a two-part process where we first obtain the <a href="https://github.com/apache/gora/blob/master/gora-shims-hadoop/src/main/java/org/apache/gora/shims/hadoop/HadoopShimFactory.java#L79-L94">Hadoop +Major version</a>. +We then <a href="https://github.com/apache/gora/blob/master/gora-shims-hadoop/src/main/java/org/apache/gora/shims/hadoop/HadoopShimFactory.java#L56-L77">select the appropriate Gora Shim</a> +based on detecting the Hadoop Major version. This code looks the following:</p> +<div class="codehilite"><pre><span class="cm">/**</span> +<span class="cm"> * Get the Hadoop major version number.</span> +<span class="cm"> *</span> +<span class="cm"> * @return The major version number of Hadoop.</span> +<span class="cm"> */</span> +<span class="n">public</span> <span class="n">String</span> <span class="n">getMajorVersion</span><span class="p">()</span> <span class="p">{</span> + <span class="n">String</span> <span class="n">vers</span> <span class="o">=</span> <span class="n">VersionInfo</span><span class="p">.</span><span class="n">getVersion</span><span class="p">();</span> + <span class="n">String</span><span class="p">[]</span> <span class="n">parts</span> <span class="o">=</span> <span class="n">vers</span><span class="p">.</span><span class="n">split</span><span class="p">(</span><span class="s">"</span><span class="se">\\</span><span class="s">."</span><span class="p">);</span> + <span class="k">if</span> <span class="p">(</span><span class="n">parts</span><span class="p">.</span><span class="n">length</span> <span class="o"><</span> <span class="mh">2</span><span class="p">)</span> <span class="p">{</span> + <span class="n">throw</span> <span class="k">new</span> <span class="n">RuntimeException</span><span class="p">(</span><span class="s">"Unable to parse Hadoop version: "</span> + <span class="o">+</span> <span class="n">vers</span> <span class="o">+</span> <span class="s">" (expected X.Y.* format)"</span><span class="p">);</span> + <span class="p">}</span> + <span class="k">return</span> <span class="n">parts</span><span class="p">[</span><span class="mh">0</span><span class="p">];</span> +<span class="p">}</span> + +<span class="cm">/**</span> +<span class="cm"> * Get the Hadoop shim for the Hadoop version on the class path. In case it</span> +<span class="cm"> * fails to obtain an appropriate shim (i.e. unsupported Hadoop version), it</span> +<span class="cm"> * throws a {@link RuntimeException}.</span> +<span class="cm"> *</span> +<span class="cm"> * Note that this method is potentially costly.</span> +<span class="cm"> *</span> +<span class="cm"> * @return A newly created instance of a {@link HadoopShim}.</span> +<span class="cm"> */</span> +<span class="n">public</span> <span class="n">HadoopShim</span> <span class="n">getHadoopShim</span><span class="p">()</span> <span class="p">{</span> + <span class="n">String</span> <span class="n">version</span> <span class="o">=</span> <span class="n">getMajorVersion</span><span class="p">();</span> + <span class="n">String</span> <span class="n">className</span> <span class="o">=</span> <span class="no">HADOOP_VERSION_TO_IMPL_MAP</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">version</span><span class="p">);</span> + <span class="n">try</span> <span class="p">{</span> + <span class="n">Class</span><span class="o"><?></span> <span class="n">class1</span> <span class="o">=</span> <span class="n">Class</span><span class="p">.</span><span class="n">forName</span><span class="p">(</span><span class="n">className</span><span class="p">);</span> + <span class="k">return</span> <span class="n">HadoopShim</span><span class="p">.</span><span class="k">class</span><span class="p">.</span><span class="n">cast</span><span class="p">(</span><span class="n">class1</span><span class="p">.</span><span class="n">newInstance</span><span class="p">());</span> + <span class="p">}</span> <span class="n">catch</span> <span class="p">(</span><span class="n">Exception</span> <span class="n">e</span><span class="p">)</span> <span class="p">{</span> + <span class="n">throw</span> <span class="k">new</span> <span class="n">RuntimeException</span><span class="p">(</span> + <span class="s">"Could not load Hadoop shim for version "</span> <span class="o">+</span> <span class="n">version</span> + <span class="o">+</span> <span class="s">", className="</span> <span class="o">+</span> <span class="n">className</span><span class="p">,</span> <span class="n">e</span><span class="p">);</span> + <span class="p">}</span> +<span class="p">}</span> +</pre></div> + + +<h2 id="dependency-definition_1">Dependency Definition</h2> +<p><a href="http://search.maven.org/#artifactdetails|org.apache.gora|gora-shims-hadoop|0.6|bundle">Click Here</a></p> <h1 id="gora-shims-hadoop1">gora-shims-hadoop1</h1> <h2 id="description_2">Description</h2> -<h2 id="dependency_1">Dependency</h2> -<h2 id="direct-dependency_2">Direct Dependency</h2> +<p>This module provides all functionality for Hadoop 1.X support in Gora. The Hadoop version is specified within +the project <a href="https://github.com/apache/gora/blob/master/pom.xml">pom.xml</a> within the <strong>${hadoop1-version}</strong> variable.</p> +<p>The actual <a href="https://github.com/apache/gora/blob/master/gora-shims-hadoop1/src/main/java/org/apache/gora/shims/hadoop1/HadoopShim1.java">code implementation</a> +is very simple. Essentially it consists of two Java methods</p> +<div class="codehilite"><pre> <span class="o">/**</span> + <span class="o">*</span> <span class="p">{@</span><span class="n">inheritDoc</span><span class="p">}</span> + <span class="o">*/</span> + <span class="n">public</span> <span class="n">Job</span> <span class="n">createJob</span><span class="p">(</span><span class="n">Configuration</span> <span class="n">configuration</span><span class="p">)</span> <span class="n">throws</span> <span class="n">IOException</span> <span class="p">{</span> + <span class="k">return</span> <span class="n">new</span> <span class="n">Job</span><span class="p">(</span><span class="n">configuration</span><span class="p">);</span> + <span class="p">}</span> + <span class="o">/**</span> + <span class="o">*</span> <span class="p">{@</span><span class="n">inheritDoc</span><span class="p">}</span> + <span class="o">*/</span> + <span class="n">public</span> <span class="n">JobContext</span> <span class="n">createJobContext</span><span class="p">(</span><span class="n">Configuration</span> <span class="n">configuration</span><span class="p">)</span> <span class="p">{</span> + <span class="k">return</span> <span class="n">new</span> <span class="n">JobContext</span><span class="p">(</span><span class="n">configuration</span><span class="p">,</span> <span class="n">null</span><span class="p">);</span> + <span class="p">}</span> +</pre></div> + + +<h2 id="dependency-definition_2">Dependency Definition</h2> +<p><a href="http://search.maven.org/#artifactdetails|org.apache.gora|gora-shims-hadoop1|0.6|bundle">Click Here</a></p> <h1 id="gora-shims-hadoop2">gora-shims-hadoop2</h1> <h2 id="description_3">Description</h2> -<h2 id="dependency_2">Dependency</h2> -<h2 id="direct-dependency_3">Direct Dependency</h2> +<p>This module provides all functionality for Hadoop 2.X support in Gora. The Hadoop version is specified within +the project <a href="https://github.com/apache/gora/blob/master/pom.xml">pom.xml</a> within the <strong>${hadoop2-version}</strong> variable.</p> +<p>The actual <a href="https://github.com/apache/gora/blob/master/gora-shims-hadoop1/src/main/java/org/apache/gora/shims/hadoop1/HadoopShim1.java">code implementation</a> +is very simple. Essentially it consists of two Java methods + /<strong> + * {@inheritDoc} + <em> + * Use the Hadoop 2.x way of creating a {@link Job} object. + </em>/ + public Job createJob(Configuration configuration) throws IOException { + Job instance = Job.getInstance(configuration); + return instance; + } + /</strong> + * {@inheritDoc} + <em> + * Use the Hadoop 2.x way of creating a {@link JobContext} object. + </em>/ + public JobContext createJobContext(Configuration configuration) { + return new JobContextImpl(configuration, null); + }</p> +<h2 id="dependency-definition_3">Dependency Definition</h2> +<p><a href="http://search.maven.org/#artifactdetails|org.apache.gora|gora-shims-hadoop2|0.6|bundle">Click Here</a></p> </div> <!-- /container (main block) -->