Modified: calcite/site/docs/howto.html URL: http://svn.apache.org/viewvc/calcite/site/docs/howto.html?rev=1739659&r1=1739658&r2=1739659&view=diff ============================================================================== --- calcite/site/docs/howto.html (original) +++ calcite/site/docs/howto.html Sun Apr 17 21:26:04 2016 @@ -4,7 +4,7 @@ <meta charset="UTF-8"> <title>HOWTO</title> <meta name="viewport" content="width=device-width,initial-scale=1"> - <meta name="generator" content="Jekyll v2.4.0"> + <meta name="generator" content="Jekyll v3.0.3"> <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900"> <link rel="stylesheet" href="/css/screen.css"> <link rel="icon" type="image/x-icon" href="/favicon.ico"> @@ -813,40 +813,40 @@ adapters.</p> <ul id="markdown-toc"> - <li><a href="#building-from-a-source-distribution">Building from a source distribution</a></li> - <li><a href="#building-from-git">Building from git</a></li> - <li><a href="#running-tests">Running tests</a></li> - <li><a href="#running-integration-tests">Running integration tests</a> <ul> - <li><a href="#vm-preparation">VM preparation</a></li> - <li><a href="#vm-management">VM management</a></li> - <li><a href="#suggested-test-flow">Suggested test flow</a></li> - <li><a href="#integration-tests-technical-details">Integration tests technical details</a></li> + <li><a href="#building-from-a-source-distribution" id="markdown-toc-building-from-a-source-distribution">Building from a source distribution</a></li> + <li><a href="#building-from-git" id="markdown-toc-building-from-git">Building from git</a></li> + <li><a href="#running-tests" id="markdown-toc-running-tests">Running tests</a></li> + <li><a href="#running-integration-tests" id="markdown-toc-running-integration-tests">Running integration tests</a> <ul> + <li><a href="#vm-preparation" id="markdown-toc-vm-preparation">VM preparation</a></li> + <li><a href="#vm-management" id="markdown-toc-vm-management">VM management</a></li> + <li><a href="#suggested-test-flow" id="markdown-toc-suggested-test-flow">Suggested test flow</a></li> + <li><a href="#integration-tests-technical-details" id="markdown-toc-integration-tests-technical-details">Integration tests technical details</a></li> </ul> </li> - <li><a href="#contributing">Contributing</a></li> - <li><a href="#getting-started">Getting started</a></li> - <li><a href="#tracing">Tracing</a></li> - <li><a href="#csv-adapter">CSV adapter</a></li> - <li><a href="#mongodb-adapter">MongoDB adapter</a></li> - <li><a href="#splunk-adapter">Splunk adapter</a></li> - <li><a href="#implementing-an-adapter">Implementing an adapter</a> <ul> - <li><a href="#testing-adapter-in-java">Testing adapter in Java</a></li> + <li><a href="#contributing" id="markdown-toc-contributing">Contributing</a></li> + <li><a href="#getting-started" id="markdown-toc-getting-started">Getting started</a></li> + <li><a href="#tracing" id="markdown-toc-tracing">Tracing</a></li> + <li><a href="#csv-adapter" id="markdown-toc-csv-adapter">CSV adapter</a></li> + <li><a href="#mongodb-adapter" id="markdown-toc-mongodb-adapter">MongoDB adapter</a></li> + <li><a href="#splunk-adapter" id="markdown-toc-splunk-adapter">Splunk adapter</a></li> + <li><a href="#implementing-an-adapter" id="markdown-toc-implementing-an-adapter">Implementing an adapter</a> <ul> + <li><a href="#testing-adapter-in-java" id="markdown-toc-testing-adapter-in-java">Testing adapter in Java</a></li> </ul> </li> - <li><a href="#advanced-topics-for-developers">Advanced topics for developers</a> <ul> - <li><a href="#javatypefactory">JavaTypeFactory</a></li> - <li><a href="#rebuilding-generated-protocol-buffer-code">Rebuilding generated Protocol Buffer code</a></li> + <li><a href="#advanced-topics-for-developers" id="markdown-toc-advanced-topics-for-developers">Advanced topics for developers</a> <ul> + <li><a href="#javatypefactory" id="markdown-toc-javatypefactory">JavaTypeFactory</a></li> + <li><a href="#rebuilding-generated-protocol-buffer-code" id="markdown-toc-rebuilding-generated-protocol-buffer-code">Rebuilding generated Protocol Buffer code</a></li> </ul> </li> - <li><a href="#advanced-topics-for-committers">Advanced topics for committers</a> <ul> - <li><a href="#set-up-pgp-signing-keys-for-calcite-committers">Set up PGP signing keys (for Calcite committers)</a></li> - <li><a href="#making-a-snapshot-for-calcite-committers">Making a snapshot (for Calcite committers)</a></li> - <li><a href="#making-a-release-for-calcite-committers">Making a release (for Calcite committers)</a></li> - <li><a href="#cleaning-up-after-a-failed-release-attempt-for-calcite-committers">Cleaning up after a failed release attempt (for Calcite committers)</a></li> - <li><a href="#validate-a-release">Validate a release</a></li> - <li><a href="#get-approval-for-a-release-via-apache-voting-process-for-calcite-committers">Get approval for a release via Apache voting process (for Calcite committers)</a></li> - <li><a href="#publishing-a-release-for-calcite-committers">Publishing a release (for Calcite committers)</a></li> - <li><a href="#publish-the-web-site">Publishing the web site (for Calcite committers)</a></li> + <li><a href="#advanced-topics-for-committers" id="markdown-toc-advanced-topics-for-committers">Advanced topics for committers</a> <ul> + <li><a href="#set-up-pgp-signing-keys-for-calcite-committers" id="markdown-toc-set-up-pgp-signing-keys-for-calcite-committers">Set up PGP signing keys (for Calcite committers)</a></li> + <li><a href="#making-a-snapshot-for-calcite-committers" id="markdown-toc-making-a-snapshot-for-calcite-committers">Making a snapshot (for Calcite committers)</a></li> + <li><a href="#making-a-release-for-calcite-committers" id="markdown-toc-making-a-release-for-calcite-committers">Making a release (for Calcite committers)</a></li> + <li><a href="#cleaning-up-after-a-failed-release-attempt-for-calcite-committers" id="markdown-toc-cleaning-up-after-a-failed-release-attempt-for-calcite-committers">Cleaning up after a failed release attempt (for Calcite committers)</a></li> + <li><a href="#validate-a-release" id="markdown-toc-validate-a-release">Validate a release</a></li> + <li><a href="#get-approval-for-a-release-via-apache-voting-process-for-calcite-committers" id="markdown-toc-get-approval-for-a-release-via-apache-voting-process-for-calcite-committers">Get approval for a release via Apache voting process (for Calcite committers)</a></li> + <li><a href="#publishing-a-release-for-calcite-committers" id="markdown-toc-publishing-a-release-for-calcite-committers">Publishing a release (for Calcite committers)</a></li> + <li><a href="#publish-the-web-site" id="markdown-toc-publish-the-web-site">Publishing the web site (for Calcite committers)</a></li> </ul> </li> </ul> @@ -856,13 +856,13 @@ adapters.</p> <p>Prerequisites are maven (3.2.1 or later) and Java (JDK 1.7 or later, 1.8 preferred) on your path.</p> -<p>Unpack the source distribution <code>.tar.gz</code> or <code>.zip</code> file, -<code>cd</code> to the root directory of the unpacked source, +<p>Unpack the source distribution <code class="highlighter-rouge">.tar.gz</code> or <code class="highlighter-rouge">.zip</code> file, +<code class="highlighter-rouge">cd</code> to the root directory of the unpacked source, then build using maven:</p> -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>tar xvfz calcite-1.7.0-source.tar.gz -<span class="nv">$ </span><span class="nb">cd </span>calcite-1.7.0 -<span class="nv">$ </span>mvn install</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>tar xvfz calcite-1.7.0-source.tar.gz +<span class="gp">$ </span><span class="nb">cd </span>calcite-1.7.0 +<span class="gp">$ </span>mvn install</code></pre></figure> <p><a href="#running-tests">Running tests</a> describes how to run more or fewer tests.</p> @@ -873,26 +873,26 @@ tests.</p> and Java (JDK 1.7 or later, 1.8 preferred) on your path.</p> <p>Create a local copy of the github repository, -<code>cd</code> to its root directory, +<code class="highlighter-rouge">cd</code> to its root directory, then build using maven:</p> -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>git clone git://github.com/apache/calcite.git -<span class="nv">$ </span><span class="nb">cd </span>calcite -<span class="nv">$ </span>mvn install</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git clone git://github.com/apache/calcite.git +<span class="gp">$ </span><span class="nb">cd </span>calcite +<span class="gp">$ </span>mvn install</code></pre></figure> <p>Calcite includes a number of machine-generated codes. By default, these are regenerated on every build, but this has the negative side-effect of causing a re-compilation of the entire project when the non-machine-generated code has not changed. To make sure incremental compilation still works as intended, -provide the <code>skipGenerate</code> command line option with your maven command. -If you invoke the <code>clean</code> lifecycle phase, you must not specify the -<code>skipGenerate</code> option as it will not recompile the necessary code for the build +provide the <code class="highlighter-rouge">skipGenerate</code> command line option with your maven command. +If you invoke the <code class="highlighter-rouge">clean</code> lifecycle phase, you must not specify the +<code class="highlighter-rouge">skipGenerate</code> option as it will not recompile the necessary code for the build to succeed.</p> -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>mvn clean -<span class="nv">$ </span>mvn package +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>mvn clean +<span class="gp">$ </span>mvn package ... hacks ... -<span class="nv">$ </span>mvn package -DskipGenerate</code></pre></div> +<span class="gp">$ </span>mvn package -DskipGenerate</code></pre></figure> <p><a href="#running-tests">Running tests</a> describes how to run more or fewer tests.</p> @@ -900,32 +900,32 @@ tests.</p> <h2 id="running-tests">Running tests</h2> <p>The test suite will run by default when you build, unless you specify -<code>-DskipTests</code>:</p> +<code class="highlighter-rouge">-DskipTests</code>:</p> -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>mvn -DskipTests clean install -<span class="nv">$ </span>mvn <span class="nb">test</span></code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>mvn -DskipTests clean install +<span class="gp">$ </span>mvn <span class="nb">test</span></code></pre></figure> <p>There are other options that control which tests are run, and in what environment, as follows.</p> <ul> - <li><code>-Dcalcite.test.db=DB</code> (where db is <code>h2</code>, <code>hsqldb</code>, <code>mysql</code>, or <code>postgresql</code>) allows you + <li><code class="highlighter-rouge">-Dcalcite.test.db=DB</code> (where db is <code class="highlighter-rouge">h2</code>, <code class="highlighter-rouge">hsqldb</code>, <code class="highlighter-rouge">mysql</code>, or <code class="highlighter-rouge">postgresql</code>) allows you to change the JDBC data source for the test suite. Calciteâs test suite requires a JDBC data source populated with the foodmart data set. <ul> - <li><code>hsqldb</code>, the default, uses an in-memory hsqldb database.</li> + <li><code class="highlighter-rouge">hsqldb</code>, the default, uses an in-memory hsqldb database.</li> <li>All others access a test virtual machine (see <a href="#running-integration-tests">integration tests</a> below). -<code>mysql</code> and <code>postgresql</code> might be somewhat faster than hsqldb, but you need +<code class="highlighter-rouge">mysql</code> and <code class="highlighter-rouge">postgresql</code> might be somewhat faster than hsqldb, but you need to populate it (i.e. provision a VM).</li> </ul> </li> - <li><code>-Dcalcite.debug</code> prints extra debugging information to stdout.</li> - <li><code>-Dcalcite.test.slow</code> enables tests that take longer to execute. For + <li><code class="highlighter-rouge">-Dcalcite.debug</code> prints extra debugging information to stdout.</li> + <li><code class="highlighter-rouge">-Dcalcite.test.slow</code> enables tests that take longer to execute. For example, there are tests that create virtual TPC-H and TPC-DS schemas in-memory and run tests from those benchmarks.</li> - <li><code>-Dcalcite.test.splunk</code> enables tests that run against Splunk. + <li><code class="highlighter-rouge">-Dcalcite.test.splunk</code> enables tests that run against Splunk. Splunk must be installed and running.</li> </ul> @@ -946,20 +946,20 @@ The VM includes H2, HSQLDB, MySQL, Mongo <p>1) Clone https://github.com/vlsi/calcite-test-dataset.git at the same level as calcite repository. For instance:</p> -<div class="highlight"><pre><code class="language-bash" data-lang="bash">code +<figure class="highlight"><pre><code class="language-bash" data-lang="bash">code +-- calcite - +-- calcite-test-dataset</code></pre></div> + +-- calcite-test-dataset</code></pre></figure> <p>Note: integration tests search for ../calcite-test-dataset or ../../calcite-test-dataset. You can specify full path via calcite.test.dataset system property.</p> <p>2) Build and start the VM:</p> -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">cd </span>calcite-test-dataset <span class="o">&&</span> mvn install</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">cd </span>calcite-test-dataset <span class="o">&&</span> mvn install</code></pre></figure> <h3 id="vm-management">VM management</h3> -<p>Test VM is provisioned by Vagrant, so regular Vagrant <code>vagrant up</code> and <code>vagrant halt</code> should be used to start and stop the VM. +<p>Test VM is provisioned by Vagrant, so regular Vagrant <code class="highlighter-rouge">vagrant up</code> and <code class="highlighter-rouge">vagrant halt</code> should be used to start and stop the VM. The connection strings for different databases are listed in <a href="https://github.com/vlsi/calcite-test-dataset";>calcite-test-dataset</a> readme.</p> <h3 id="suggested-test-flow">Suggested test flow</h3> @@ -969,19 +969,19 @@ The connection strings for different dat <p>Command line:</p> <ul> - <li>Executing regular unit tests (does not require external data): no change. <code>mvn test</code> or <code>mvn install</code>.</li> - <li>Executing all tests, for all the DBs: <code>mvn verify -Pit</code>. <code>it</code> stands for âintegration-testâ. <code>mvn install -Pit</code> works as well.</li> - <li>Executing just tests for external DBs, excluding unit tests: <code>mvn -Dtest=foo -DfailIfNoTests=false -Pit verify</code></li> - <li>Executing just MongoDB tests: <code>cd mongo; mvn verify -Pit</code></li> + <li>Executing regular unit tests (does not require external data): no change. <code class="highlighter-rouge">mvn test</code> or <code class="highlighter-rouge">mvn install</code>.</li> + <li>Executing all tests, for all the DBs: <code class="highlighter-rouge">mvn verify -Pit</code>. <code class="highlighter-rouge">it</code> stands for âintegration-testâ. <code class="highlighter-rouge">mvn install -Pit</code> works as well.</li> + <li>Executing just tests for external DBs, excluding unit tests: <code class="highlighter-rouge">mvn -Dtest=foo -DfailIfNoTests=false -Pit verify</code></li> + <li>Executing just MongoDB tests: <code class="highlighter-rouge">cd mongo; mvn verify -Pit</code></li> </ul> <p>From within IDE:</p> <ul> <li>Executing regular unit tests: no change.</li> - <li>Executing MongoDB tests: run <code>MongoAdapterIT.java</code> as usual (no additional properties are required)</li> - <li>Executing MySQL tests: run <code>JdbcTest</code> and <code>JdbcAdapterTest</code> with setting <code>-Dcalcite.test.db=mysql</code></li> - <li>Executing PostgreSQL tests: run <code>JdbcTest</code> and <code>JdbcAdapterTest</code> with setting <code>-Dcalcite.test.db=postgresql</code></li> + <li>Executing MongoDB tests: run <code class="highlighter-rouge">MongoAdapterIT.java</code> as usual (no additional properties are required)</li> + <li>Executing MySQL tests: run <code class="highlighter-rouge">JdbcTest</code> and <code class="highlighter-rouge">JdbcAdapterTest</code> with setting <code class="highlighter-rouge">-Dcalcite.test.db=mysql</code></li> + <li>Executing PostgreSQL tests: run <code class="highlighter-rouge">JdbcTest</code> and <code class="highlighter-rouge">JdbcAdapterTest</code> with setting <code class="highlighter-rouge">-Dcalcite.test.db=postgresql</code></li> </ul> <h3 id="integration-tests-technical-details">Integration tests technical details</h3> @@ -989,7 +989,7 @@ The connection strings for different dat <p>Tests with external data are executed at mavenâs integration-test phase. We do not currently use pre-integration-test/post-integration-test, however we could use that in future. The verification of build pass/failure is performed at verify phase. -Integration tests should be named <code>...IT.java</code>, so they are not picked up on unit test execution.</p> +Integration tests should be named <code class="highlighter-rouge">...IT.java</code>, so they are not picked up on unit test execution.</p> <h2 id="contributing">Contributing</h2> @@ -1003,26 +1003,26 @@ Integration tests should be named <code> <p>To enable tracing, add the following flags to the java command line:</p> -<p><code>-Dcalcite.debug=true</code></p> +<p><code class="highlighter-rouge">-Dcalcite.debug=true</code></p> <p>The first flag causes Calcite to print the Java code it generates (to execute queries) to stdout. It is especially useful if you are debugging mysterious problems like this:</p> -<p><code>Exception in thread "main" java.lang.ClassCastException: Integer cannot be cast to Long +<p><code class="highlighter-rouge">Exception in thread "main" java.lang.ClassCastException: Integer cannot be cast to Long at Baz$1$1.current(Unknown Source)</code></p> <p>By default, Calcite uses the Log4j bindings for SLF4J. There is a provided configuration -file which outputs logging at the INFO level to the console in <code>core/src/test/resources/log4j.properties</code>. +file which outputs logging at the INFO level to the console in <code class="highlighter-rouge">core/src/test/resources/log4j.properties</code>. You can modify the level for the rootLogger to increase verbosity or change the level for a specific class if you so choose.</p> -<div class="highlight"><pre><code class="language-properties" data-lang="properties"><span class="c"># Change rootLogger level to WARN</span> -<span class="na">log4j.rootLogger</span><span class="o">=</span><span class="s">WARN, A1</span> -<span class="c"># Increase level to DEBUG for RelOptPlanner</span> -<span class="na">log4j.logger.org.apache.calcite.plan.RelOptPlanner</span><span class="o">=</span><span class="s">DEBUG</span> -<span class="c"># Increase level to TRACE for HepPlanner</span> -<span class="na">log4j.logger.org.apache.calcite.plan.hep.HepPlanner</span><span class="o">=</span><span class="s">TRACE</span></code></pre></div> +<figure class="highlight"><pre><code class="language-properties" data-lang="properties"><span class="c"># Change rootLogger level to WARN +</span><span class="py">log4j.rootLogger</span><span class="p">=</span><span class="s">WARN, A1</span> +<span class="c"># Increase level to DEBUG for RelOptPlanner +</span><span class="py">log4j.logger.org.apache.calcite.plan.RelOptPlanner</span><span class="p">=</span><span class="s">DEBUG</span> +<span class="c"># Increase level to TRACE for HepPlanner +</span><span class="py">log4j.logger.org.apache.calcite.plan.hep.HepPlanner</span><span class="p">=</span><span class="s">TRACE</span></code></pre></figure> <h2 id="csv-adapter">CSV adapter</h2> @@ -1037,113 +1037,113 @@ and <a href="http://www.mongodb.org/down <p>Import MongoDBâs zipcode data set into MongoDB:</p> -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>curl -o /tmp/zips.json http://media.mongodb.org/zips.json -<span class="nv">$ </span>mongoimport --db <span class="nb">test</span> --collection zips --file /tmp/zips.json -Tue Jun <span class="m">4</span> 16:24:14.190 check <span class="m">9</span> 29470 -Tue Jun <span class="m">4</span> 16:24:14.469 imported <span class="m">29470</span> objects</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>curl -o /tmp/zips.json http://media.mongodb.org/zips.json +<span class="gp">$ </span>mongoimport --db <span class="nb">test</span> --collection zips --file /tmp/zips.json +Tue Jun 4 16:24:14.190 check 9 29470 +Tue Jun 4 16:24:14.469 imported 29470 objects</code></pre></figure> <p>Log into MongoDB to check itâs there:</p> -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>mongo +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>mongo MongoDB shell version: 2.4.3 connecting to: <span class="nb">test</span> -> db.zips.find<span class="o">()</span>.limit<span class="o">(</span>3<span class="o">)</span> -<span class="o">{</span> <span class="s2">"city"</span> : <span class="s2">"ACMAR"</span>, <span class="s2">"loc"</span> : <span class="o">[</span> -86.51557, 33.584132 <span class="o">]</span>, <span class="s2">"pop"</span> : 6055, <span class="s2">"state"</span> : <span class="s2">"AL"</span>, <span class="s2">"_id"</span> : <span class="s2">"35004"</span> <span class="o">}</span> -<span class="o">{</span> <span class="s2">"city"</span> : <span class="s2">"ADAMSVILLE"</span>, <span class="s2">"loc"</span> : <span class="o">[</span> -86.959727, 33.588437 <span class="o">]</span>, <span class="s2">"pop"</span> : 10616, <span class="s2">"state"</span> : <span class="s2">"AL"</span>, <span class="s2">"_id"</span> : <span class="s2">"35005"</span> <span class="o">}</span> -<span class="o">{</span> <span class="s2">"city"</span> : <span class="s2">"ADGER"</span>, <span class="s2">"loc"</span> : <span class="o">[</span> -87.167455, 33.434277 <span class="o">]</span>, <span class="s2">"pop"</span> : 3205, <span class="s2">"state"</span> : <span class="s2">"AL"</span>, <span class="s2">"_id"</span> : <span class="s2">"35006"</span> <span class="o">}</span> -> <span class="nb">exit</span> -bye</code></pre></div> +<span class="gp">> </span>db.zips.find<span class="o">()</span>.limit<span class="o">(</span>3<span class="o">)</span> +<span class="o">{</span> <span class="s2">"city"</span> : <span class="s2">"ACMAR"</span>, <span class="s2">"loc"</span> : <span class="o">[</span> -86.51557, 33.584132 <span class="o">]</span>, <span class="s2">"pop"</span> : 6055, <span class="s2">"state"</span> : <span class="s2">"AL"</span>, <span class="s2">"_id"</span> : <span class="s2">"35004"</span> <span class="o">}</span> +<span class="o">{</span> <span class="s2">"city"</span> : <span class="s2">"ADAMSVILLE"</span>, <span class="s2">"loc"</span> : <span class="o">[</span> -86.959727, 33.588437 <span class="o">]</span>, <span class="s2">"pop"</span> : 10616, <span class="s2">"state"</span> : <span class="s2">"AL"</span>, <span class="s2">"_id"</span> : <span class="s2">"35005"</span> <span class="o">}</span> +<span class="o">{</span> <span class="s2">"city"</span> : <span class="s2">"ADGER"</span>, <span class="s2">"loc"</span> : <span class="o">[</span> -87.167455, 33.434277 <span class="o">]</span>, <span class="s2">"pop"</span> : 3205, <span class="s2">"state"</span> : <span class="s2">"AL"</span>, <span class="s2">"_id"</span> : <span class="s2">"35006"</span> <span class="o">}</span> +<span class="gp">> </span><span class="nb">exit +</span>bye</code></pre></figure> <p>Connect using the <a href="https://github.com/apache/calcite/blob/master/mongodb/src/test/resources/mongo-zips-model.json";>mongo-zips-model.json</a> Calcite model:</p> -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>./sqlline -sqlline> !connect jdbc:calcite:model<span class="o">=</span>mongodb/target/test-classes/mongo-zips-model.json admin admin +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>./sqlline +<span class="gp">sqlline> </span>!connect jdbc:calcite:model<span class="o">=</span>mongodb/target/test-classes/mongo-zips-model.json admin admin Connecting to jdbc:calcite:model<span class="o">=</span>mongodb/target/test-classes/mongo-zips-model.json Connected to: Calcite <span class="o">(</span>version 1.x.x<span class="o">)</span> Driver: Calcite JDBC Driver <span class="o">(</span>version 1.x.x<span class="o">)</span> -Autocommit status: <span class="nb">true</span> -Transaction isolation: TRANSACTION_REPEATABLE_READ -sqlline> !tables +Autocommit status: <span class="nb">true +</span>Transaction isolation: TRANSACTION_REPEATABLE_READ +<span class="gp">sqlline> </span>!tables +------------+--------------+-----------------+---------------+ -<span class="p">|</span> TABLE_CAT <span class="p">|</span> TABLE_SCHEM <span class="p">|</span> TABLE_NAME <span class="p">|</span> TABLE_TYPE <span class="p">|</span> +| TABLE_CAT | TABLE_SCHEM | TABLE_NAME | TABLE_TYPE | +------------+--------------+-----------------+---------------+ -<span class="p">|</span> null <span class="p">|</span> mongo_raw <span class="p">|</span> zips <span class="p">|</span> TABLE <span class="p">|</span> -<span class="p">|</span> null <span class="p">|</span> mongo_raw <span class="p">|</span> system.indexes <span class="p">|</span> TABLE <span class="p">|</span> -<span class="p">|</span> null <span class="p">|</span> mongo <span class="p">|</span> ZIPS <span class="p">|</span> VIEW <span class="p">|</span> -<span class="p">|</span> null <span class="p">|</span> metadata <span class="p">|</span> COLUMNS <span class="p">|</span> SYSTEM_TABLE <span class="p">|</span> -<span class="p">|</span> null <span class="p">|</span> metadata <span class="p">|</span> TABLES <span class="p">|</span> SYSTEM_TABLE <span class="p">|</span> +| null | mongo_raw | zips | TABLE | +| null | mongo_raw | system.indexes | TABLE | +| null | mongo | ZIPS | VIEW | +| null | metadata | COLUMNS | SYSTEM_TABLE | +| null | metadata | TABLES | SYSTEM_TABLE | +------------+--------------+-----------------+---------------+ -sqlline> <span class="k">select</span> count<span class="o">(</span>*<span class="o">)</span> from zips<span class="p">;</span> +<span class="gp">sqlline> </span><span class="k">select </span>count<span class="o">(</span><span class="k">*</span><span class="o">)</span> from zips; +---------+ -<span class="p">|</span> EXPR<span class="nv">$0</span> <span class="p">|</span> +| EXPR<span class="nv">$0</span> | +---------+ -<span class="p">|</span> <span class="m">29467</span> <span class="p">|</span> +| 29467 | +---------+ -<span class="m">1</span> row selected <span class="o">(</span>0.746 seconds<span class="o">)</span> -sqlline> !quit +1 row selected <span class="o">(</span>0.746 seconds<span class="o">)</span> +<span class="gp">sqlline> </span>!quit Closing: org.apache.calcite.jdbc.FactoryJdbc41<span class="nv">$CalciteConnectionJdbc41</span> -<span class="err">$</span></code></pre></div> +<span class="err">$</span></code></pre></figure> <h2 id="splunk-adapter">Splunk adapter</h2> <p>To run the test suite and sample queries against Splunk, -load Splunkâs <code>tutorialdata.zip</code> data set as described in +load Splunkâs <code class="highlighter-rouge">tutorialdata.zip</code> data set as described in <a href="http://docs.splunk.com/Documentation/Splunk/6.0.2/PivotTutorial/GetthetutorialdataintoSplunk";>the Splunk tutorial</a>.</p> <p>(This step is optional, but it provides some interesting data for the sample queries. It is also necessary if you intend to run the test suite, using -<code>-Dcalcite.test.splunk=true</code>.)</p> +<code class="highlighter-rouge">-Dcalcite.test.splunk=true</code>.)</p> <h2 id="implementing-an-adapter">Implementing an adapter</h2> -<p>New adapters can be created by implementing <code>CalcitePrepare.Context</code>:</p> +<p>New adapters can be created by implementing <code class="highlighter-rouge">CalcitePrepare.Context</code>:</p> -<div class="highlight"><pre><code class="language-java" data-lang="java"><span class="kn">import</span> <span class="nn">org.apache.calcite.adapter.java.JavaTypeFactory</span><span class="o">;</span> +<figure class="highlight"><pre><code class="language-java" data-lang="java"><span class="kn">import</span> <span class="nn">org.apache.calcite.adapter.java.JavaTypeFactory</span><span class="o">;</span> <span class="kn">import</span> <span class="nn">org.apache.calcite.jdbc.CalcitePrepare</span><span class="o">;</span> <span class="kn">import</span> <span class="nn">org.apache.calcite.jdbc.CalciteSchema</span><span class="o">;</span> <span class="kd">public</span> <span class="kd">class</span> <span class="nc">AdapterContext</span> <span class="kd">implements</span> <span class="n">CalcitePrepare</span><span class="o">.</span><span class="na">Context</span> <span class="o">{</span> <span class="nd">@Override</span> - <span class="kd">public</span> <span class="n">JavaTypeFactory</span> <span class="nf">getTypeFactory</span><span class="o">()</span> <span class="o">{</span> + <span class="kd">public</span> <span class="n">JavaTypeFactory</span> <span class="n">getTypeFactory</span><span class="o">()</span> <span class="o">{</span> <span class="c1">// adapter implementation</span> <span class="k">return</span> <span class="n">typeFactory</span><span class="o">;</span> <span class="o">}</span> <span class="nd">@Override</span> - <span class="kd">public</span> <span class="n">CalciteSchema</span> <span class="nf">getRootSchema</span><span class="o">()</span> <span class="o">{</span> + <span class="kd">public</span> <span class="n">CalciteSchema</span> <span class="n">getRootSchema</span><span class="o">()</span> <span class="o">{</span> <span class="c1">// adapter implementation</span> <span class="k">return</span> <span class="n">rootSchema</span><span class="o">;</span> <span class="o">}</span> -<span class="o">}</span></code></pre></div> +<span class="o">}</span></code></pre></figure> <h3 id="testing-adapter-in-java">Testing adapter in Java</h3> <p>The example below shows how SQL query can be submitted to -<code>CalcitePrepare</code> with a custom context (<code>AdapterContext</code> in this +<code class="highlighter-rouge">CalcitePrepare</code> with a custom context (<code class="highlighter-rouge">AdapterContext</code> in this case). Calcite prepares and implements the query execution, using the -resources provided by the <code>Context</code>. <code>CalcitePrepare.PrepareResult</code> +resources provided by the <code class="highlighter-rouge">Context</code>. <code class="highlighter-rouge">CalcitePrepare.PrepareResult</code> provides access to the underlying enumerable and methods for enumeration. The enumerable itself can naturally be some adapter specific implementation.</p> -<div class="highlight"><pre><code class="language-java" data-lang="java"><span class="kn">import</span> <span class="nn">org.apache.calcite.jdbc.CalcitePrepare</span><span class="o">;</span> +<figure class="highlight"><pre><code class="language-java" data-lang="java"><span class="kn">import</span> <span class="nn">org.apache.calcite.jdbc.CalcitePrepare</span><span class="o">;</span> <span class="kn">import</span> <span class="nn">org.apache.calcite.prepare.CalcitePrepareImpl</span><span class="o">;</span> <span class="kn">import</span> <span class="nn">org.junit.Test</span><span class="o">;</span> <span class="kd">public</span> <span class="kd">class</span> <span class="nc">AdapterContextTest</span> <span class="o">{</span> <span class="nd">@Test</span> - <span class="kd">public</span> <span class="kt">void</span> <span class="nf">testSelectAllFromTable</span><span class="o">()</span> <span class="o">{</span> - <span class="n">AdapterContext</span> <span class="n">ctx</span> <span class="o">=</span> <span class="k">new</span> <span class="nf">AdapterContext</span><span class="o">();</span> - <span class="n">String</span> <span class="n">sql</span> <span class="o">=</span> <span class="s">"SELECT * FROM TABLENAME"</span><span class="o">;</span> + <span class="kd">public</span> <span class="kt">void</span> <span class="n">testSelectAllFromTable</span><span class="o">()</span> <span class="o">{</span> + <span class="n">AdapterContext</span> <span class="n">ctx</span> <span class="o">=</span> <span class="k">new</span> <span class="n">AdapterContext</span><span class="o">();</span> + <span class="n">String</span> <span class="n">sql</span> <span class="o">=</span> <span class="s">"SELECT * FROM TABLENAME"</span><span class="o">;</span> <span class="n">Class</span> <span class="n">elementType</span> <span class="o">=</span> <span class="n">Object</span><span class="o">[].</span><span class="na">class</span><span class="o">;</span> <span class="n">CalcitePrepare</span><span class="o">.</span><span class="na">PrepareResult</span><span class="o"><</span><span class="n">Object</span><span class="o">></span> <span class="n">prepared</span> <span class="o">=</span> - <span class="k">new</span> <span class="nf">CalcitePrepareImpl</span><span class="o">().</span><span class="na">prepareSql</span><span class="o">(</span><span class="n">ctx</span><span class="o">,</span> <span class="n">sql</span><span class="o">,</span> <span class="kc">null</span><span class="o">,</span> <span class="n">elementType</span><span class="o">,</span> <span class="o">-</span><span class="mi">1</span><span class="o">);</span> + <span class="k">new</span> <span class="n">CalcitePrepareImpl</span><span class="o">().</span><span class="na">prepareSql</span><span class="o">(</span><span class="n">ctx</span><span class="o">,</span> <span class="n">sql</span><span class="o">,</span> <span class="kc">null</span><span class="o">,</span> <span class="n">elementType</span><span class="o">,</span> <span class="o">-</span><span class="mi">1</span><span class="o">);</span> <span class="n">Object</span> <span class="n">enumerable</span> <span class="o">=</span> <span class="n">prepared</span><span class="o">.</span><span class="na">getExecutable</span><span class="o">();</span> <span class="c1">// etc.</span> <span class="o">}</span> -<span class="o">}</span></code></pre></div> +<span class="o">}</span></code></pre></figure> <h1 id="advanced-topics-for-developers">Advanced topics for developers</h1> @@ -1153,13 +1153,13 @@ these topics if you are just building fr <h2 id="javatypefactory">JavaTypeFactory</h2> -<p>When Calcite compares types (instances of <code>RelDataType</code>), it requires them to be the same +<p>When Calcite compares types (instances of <code class="highlighter-rouge">RelDataType</code>), it requires them to be the same object. If there are two distinct type instances that refer to the same Java type, Calcite may fail to recognize that they match. It is recommended to:</p> <ul> - <li>Use a single instance of <code>JavaTypeFactory</code> within the calcite context;</li> + <li>Use a single instance of <code class="highlighter-rouge">JavaTypeFactory</code> within the calcite context;</li> <li>Store the types so that the same object is always returned for the same type.</li> </ul> @@ -1179,16 +1179,16 @@ need to be re-generated only every build <p>First, install Protobuf 3.0:</p> -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>wget https://github.com/google/protobuf/releases/download/v3.0.0-beta-1/protobuf-java-3.0.0-beta-1.tar.gz -<span class="nv">$ </span>tar xf protobuf-java-3.0.0-beta-1.tar.gz <span class="o">&&</span> <span class="nb">cd </span>protobuf-3.0.0-beta-1 -<span class="nv">$ </span>./configure -<span class="nv">$ </span>make -<span class="nv">$ </span>sudo make install</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>wget https://github.com/google/protobuf/releases/download/v3.0.0-beta-1/protobuf-java-3.0.0-beta-1.tar.gz +<span class="gp">$ </span>tar xf protobuf-java-3.0.0-beta-1.tar.gz <span class="o">&&</span> <span class="nb">cd </span>protobuf-3.0.0-beta-1 +<span class="gp">$ </span>./configure +<span class="gp">$ </span>make +<span class="gp">$ </span>sudo make install</code></pre></figure> <p>Then, re-generate the compiled code:</p> -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span><span class="nb">cd </span>avatica -<span class="nv">$ </span>./src/main/scripts/generate-protobuf.sh</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span><span class="nb">cd </span>avatica +<span class="gp">$ </span>./src/main/scripts/generate-protobuf.sh</code></pre></figure> <h1 id="advanced-topics-for-committers">Advanced topics for committers</h1> @@ -1198,11 +1198,11 @@ particular release managers.</p> <h2 id="set-up-pgp-signing-keys-for-calcite-committers">Set up PGP signing keys (for Calcite committers)</h2> <p>Follow instructions <a href="http://www.apache.org/dev/release-signing";>here</a> to -create a key pair. (On Mac OS X, I did <code>brew install gpg</code> and -<code>gpg --gen-key</code>.)</p> +create a key pair. (On Mac OS X, I did <code class="highlighter-rouge">brew install gpg</code> and +<code class="highlighter-rouge">gpg --gen-key</code>.)</p> -<p>Add your public key to the <code>KEYS</code> file by following instructions in -the <code>KEYS</code> file.</p> +<p>Add your public key to the <code class="highlighter-rouge">KEYS</code> file by following instructions in +the <code class="highlighter-rouge">KEYS</code> file.</p> <h2 id="making-a-snapshot-for-calcite-committers">Making a snapshot (for Calcite committers)</h2> @@ -1211,19 +1211,19 @@ the <code>KEYS</code> file.</p> <ul> <li>Set up signing keys as described above.</li> <li>Make sure you are using JDK 1.7 (not 1.8).</li> - <li>Make sure build and tests succeed with <code>-Dcalcite.test.db=hsqldb</code> (the default)</li> + <li>Make sure build and tests succeed with <code class="highlighter-rouge">-Dcalcite.test.db=hsqldb</code> (the default)</li> </ul> -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Set passphrase variable without putting it into shell history</span> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Set passphrase variable without putting it into shell history</span> <span class="nb">read</span> -s GPG_PASSPHRASE <span class="c"># Make sure that there are no junk files in the sandbox</span> git clean -xn mvn clean -mvn -Papache-release -Dgpg.passphrase<span class="o">=</span><span class="k">${</span><span class="nv">GPG_PASSPHRASE</span><span class="k">}</span> install</code></pre></div> +mvn -Papache-release -Dgpg.passphrase<span class="o">=</span><span class="k">${</span><span class="nv">GPG_PASSPHRASE</span><span class="k">}</span> install</code></pre></figure> -<p>When the dry-run has succeeded, change <code>install</code> to <code>deploy</code>.</p> +<p>When the dry-run has succeeded, change <code class="highlighter-rouge">install</code> to <code class="highlighter-rouge">deploy</code>.</p> <h2 id="making-a-release-for-calcite-committers">Making a release (for Calcite committers)</h2> @@ -1232,31 +1232,31 @@ mvn -Papache-release -Dgpg.passphrase<sp <ul> <li>Set up signing keys as described above.</li> <li>Make sure you are using JDK 1.7 (not 1.8).</li> - <li>Check that <code>README</code> and <code>site/_docs/howto.md</code> have the correct version number.</li> - <li>Set <code>version.major</code> and <code>version.minor</code> in <code>pom.xml</code>.</li> - <li>Make sure build and tests succeed, including with <code>-P it,it-oracle</code>.</li> - <li>Make sure that <code>mvn javadoc:javadoc javadoc:test-javadoc</code> succeeds + <li>Check that <code class="highlighter-rouge">README</code> and <code class="highlighter-rouge">site/_docs/howto.md</code> have the correct version number.</li> + <li>Set <code class="highlighter-rouge">version.major</code> and <code class="highlighter-rouge">version.minor</code> in <code class="highlighter-rouge">pom.xml</code>.</li> + <li>Make sure build and tests succeed, including with <code class="highlighter-rouge">-P it,it-oracle</code>.</li> + <li>Make sure that <code class="highlighter-rouge">mvn javadoc:javadoc javadoc:test-javadoc</code> succeeds (i.e. gives no errors; warnings are OK)</li> - <li>Make sure that <code>mvn apache-rat:check</code> succeeds. (It will be run as part of + <li>Make sure that <code class="highlighter-rouge">mvn apache-rat:check</code> succeeds. (It will be run as part of the release, but itâs better to trouble-shoot early.)</li> <li>Decide the supported configurations of JDK, operating system and Guava. These will probably be the same as those described in the release notes of the previous release. Document them in the release -notes. To test Guava version x.y, specify <code>-Dguava.version=x.y</code></li> +notes. To test Guava version x.y, specify <code class="highlighter-rouge">-Dguava.version=x.y</code></li> <li>Optional extra tests: <ul> - <li><code>-Dcalcite.test.db=mysql</code></li> - <li><code>-Dcalcite.test.db=hsqldb</code></li> - <li><code>-Dcalcite.test.slow</code></li> - <li><code>-Dcalcite.test.mongodb</code></li> - <li><code>-Dcalcite.test.splunk</code></li> + <li><code class="highlighter-rouge">-Dcalcite.test.db=mysql</code></li> + <li><code class="highlighter-rouge">-Dcalcite.test.db=hsqldb</code></li> + <li><code class="highlighter-rouge">-Dcalcite.test.slow</code></li> + <li><code class="highlighter-rouge">-Dcalcite.test.mongodb</code></li> + <li><code class="highlighter-rouge">-Dcalcite.test.splunk</code></li> </ul> </li> <li>Trigger a <a href="https://scan.coverity.com/projects/2966";>Coverity scan</a> -by merging the latest code into the <code>julianhyde/coverity_scan</code> branch, +by merging the latest code into the <code class="highlighter-rouge">julianhyde/coverity_scan</code> branch, and when it completes, make sure that there are no important issues.</li> - <li>Add release notes to <code>site/_docs/history.md</code>. Include the commit history, + <li>Add release notes to <code class="highlighter-rouge">site/_docs/history.md</code>. Include the commit history, and say which versions of Java, Guava and operating systems the release is tested against.</li> <li>Make sure that @@ -1266,14 +1266,14 @@ a fix version assigned (most likely the just about to release)</li> </ul> -<p>Create a release branch named after the release, e.g. <code>branch-1.1</code>, and push it to Apache.</p> +<p>Create a release branch named after the release, e.g. <code class="highlighter-rouge">branch-1.1</code>, and push it to Apache.</p> -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>git checkout -b branch-X.Y -<span class="nv">$ </span>git push -u origin branch-X.Y</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="gp">$ </span>git checkout -b branch-X.Y +<span class="gp">$ </span>git push -u origin branch-X.Y</code></pre></figure> <p>We will use the branch for the entire the release process. Meanwhile, we do not allow commits to the master branch. After the release is -final, we can use <code>git merge --ff-only</code> to append the changes on the +final, we can use <code class="highlighter-rouge">git merge --ff-only</code> to append the changes on the release branch onto the master branch. (Apache does not allow reverts to the master branch, which makes it difficult to clean up the kind of messy commits that inevitably happen while you are trying to finalize @@ -1286,7 +1286,7 @@ that the release process will complete a <p>If any of the steps fail, clean up (see below), fix the problem, and start again from the top.</p> -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Set passphrase variable without putting it into shell history</span> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Set passphrase variable without putting it into shell history</span> <span class="nb">read</span> -s GPG_PASSPHRASE <span class="c"># Make sure that there are no junk files in the sandbox</span> @@ -1294,12 +1294,12 @@ git clean -xn mvn clean <span class="c"># Do a dry run of the release:prepare step, which sets version numbers.</span> -mvn -DdryRun<span class="o">=</span><span class="nb">true</span> -DskipTests -DreleaseVersion<span class="o">=</span>X.Y.Z -DdevelopmentVersion<span class="o">=</span>X.Y.Z+1-SNAPSHOT -Papache-release -Darguments<span class="o">=</span><span class="s2">"-Dgpg.passphrase=${GPG_PASSPHRASE}"</span> release:prepare 2><span class="p">&</span><span class="m">1</span> <span class="p">|</span> tee /tmp/prepare-dry.log</code></pre></div> +mvn -DdryRun<span class="o">=</span><span class="nb">true</span> -DskipTests -DreleaseVersion<span class="o">=</span>X.Y.Z -DdevelopmentVersion<span class="o">=</span>X.Y.Z+1-SNAPSHOT -Papache-release -Darguments<span class="o">=</span><span class="s2">"-Dgpg.passphrase=</span><span class="k">${</span><span class="nv">GPG_PASSPHRASE</span><span class="k">}</span><span class="s2">"</span> release:prepare 2>&1 | tee /tmp/prepare-dry.log</code></pre></figure> <p>Check the artifacts:</p> <ul> - <li>In the <code>target</code> directory should be these 8 files, among others: + <li>In the <code class="highlighter-rouge">target</code> directory should be these 8 files, among others: <ul> <li>apache-calcite-X.Y.Z-src.tar.gz</li> <li>apache-calcite-X.Y.Z-src.tar.gz.asc</li> @@ -1311,40 +1311,40 @@ mvn -DdryRun<span class="o">=</span><spa <li>apache-calcite-X.Y.Z-src.zip.sha1</li> </ul> </li> - <li>Note that the file names start <code>apache-calcite-</code>.</li> - <li>In the two source distros <code>.tar.gz</code> and <code>.zip</code> (currently there is + <li>Note that the file names start <code class="highlighter-rouge">apache-calcite-</code>.</li> + <li>In the two source distros <code class="highlighter-rouge">.tar.gz</code> and <code class="highlighter-rouge">.zip</code> (currently there is no binary distro), check that all files belong to a directory called -<code>apache-calcite-X.Y.Z-src</code>.</li> - <li>That directory must contain files <code>NOTICE</code>, <code>LICENSE</code>, -<code>README</code>, <code>README.md</code> +<code class="highlighter-rouge">apache-calcite-X.Y.Z-src</code>.</li> + <li>That directory must contain files <code class="highlighter-rouge">NOTICE</code>, <code class="highlighter-rouge">LICENSE</code>, +<code class="highlighter-rouge">README</code>, <code class="highlighter-rouge">README.md</code> <ul> - <li>Check that the version in <code>README</code> is correct</li> + <li>Check that the version in <code class="highlighter-rouge">README</code> is correct</li> </ul> </li> <li>In each .jar (for example -<code>core/target/calcite-core-X.Y.Z.jar</code> and -<code>mongodb/target/calcite-mongodb-X.Y.Z-sources.jar</code>), check -that the <code>META-INF</code> directory contains <code>DEPENDENCIES</code>, <code>LICENSE</code>, -<code>NOTICE</code> and <code>git.properties</code></li> - <li>In each .jar, check that <code>org-apache-calcite-jdbc.properties</code> is -present and does not contain un-substituted <code>${...}</code> variables</li> +<code class="highlighter-rouge">core/target/calcite-core-X.Y.Z.jar</code> and +<code class="highlighter-rouge">mongodb/target/calcite-mongodb-X.Y.Z-sources.jar</code>), check +that the <code class="highlighter-rouge">META-INF</code> directory contains <code class="highlighter-rouge">DEPENDENCIES</code>, <code class="highlighter-rouge">LICENSE</code>, +<code class="highlighter-rouge">NOTICE</code> and <code class="highlighter-rouge">git.properties</code></li> + <li>In each .jar, check that <code class="highlighter-rouge">org-apache-calcite-jdbc.properties</code> is +present and does not contain un-substituted <code class="highlighter-rouge">${...}</code> variables</li> <li>Check PGP, per <a href="https://httpd.apache.org/dev/verification.html";>this</a></li> </ul> -<p>Now, remove the <code>-DdryRun</code> flag and run the release for real.</p> +<p>Now, remove the <code class="highlighter-rouge">-DdryRun</code> flag and run the release for real.</p> -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Prepare sets the version numbers, creates a tag, and pushes it to git.</span> -mvn -DdryRun<span class="o">=</span><span class="nb">false</span> -DskipTests -DreleaseVersion<span class="o">=</span>X.Y.Z -DdevelopmentVersion<span class="o">=</span>X.Y.Z+1-SNAPSHOT -Papache-release -Darguments<span class="o">=</span><span class="s2">"-Dgpg.passphrase=${GPG_PASSPHRASE}"</span> release:prepare 2><span class="p">&</span><span class="m">1</span> <span class="p">|</span> tee /tmp/prepare.log +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Prepare sets the version numbers, creates a tag, and pushes it to git.</span> +mvn -DdryRun<span class="o">=</span><span class="nb">false</span> -DskipTests -DreleaseVersion<span class="o">=</span>X.Y.Z -DdevelopmentVersion<span class="o">=</span>X.Y.Z+1-SNAPSHOT -Papache-release -Darguments<span class="o">=</span><span class="s2">"-Dgpg.passphrase=</span><span class="k">${</span><span class="nv">GPG_PASSPHRASE</span><span class="k">}</span><span class="s2">"</span> release:prepare 2>&1 | tee /tmp/prepare.log <span class="c"># Perform checks out the tagged version, builds, and deploys to the staging repository</span> -mvn -DskipTests -Papache-release -Darguments<span class="o">=</span><span class="s2">"-Dgpg.passphrase=${GPG_PASSPHRASE}"</span> release:perform 2><span class="p">&</span><span class="m">1</span> <span class="p">|</span> tee /tmp/perform.log</code></pre></div> +mvn -DskipTests -Papache-release -Darguments<span class="o">=</span><span class="s2">"-Dgpg.passphrase=</span><span class="k">${</span><span class="nv">GPG_PASSPHRASE</span><span class="k">}</span><span class="s2">"</span> release:perform 2>&1 | tee /tmp/perform.log</code></pre></figure> <p>Verify the staged artifacts in the Nexus repository:</p> <ul> <li>Go to <a href="https://repository.apache.org/";>https://repository.apache.org/</a> and login</li> - <li>Under <code>Build Promotion</code>, click <code>Staging Repositories</code></li> - <li>In the <code>Staging Repositories</code> tab there should be a line with profile <code>org.apache.calcite</code></li> + <li>Under <code class="highlighter-rouge">Build Promotion</code>, click <code class="highlighter-rouge">Staging Repositories</code></li> + <li>In the <code class="highlighter-rouge">Staging Repositories</code> tab there should be a line with profile <code class="highlighter-rouge">org.apache.calcite</code></li> <li>Navigate through the artifact tree and make sure the .jar, .pom, .asc files are present</li> <li>Check the box on in the first column of the row, and press the âCloseâ button to publish the repository at @@ -1355,7 +1355,7 @@ https://repository.apache.org/content/re <p>Upload the artifacts via subversion to a staging area, https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-X.Y.Z-rcN:</p> -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Create a subversion workspace, if you haven't already</span> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Create a subversion workspace, if you haven't already</span> mkdir -p ~/dist/dev <span class="nb">pushd</span> ~/dist/dev svn co https://dist.apache.org/repos/dist/dev/calcite @@ -1364,16 +1364,16 @@ svn co https://dist.apache.org/repos/dis <span class="c"># Move the files into a directory</span> <span class="nb">cd </span>target mkdir ~/dist/dev/calcite/apache-calcite-X.Y.Z-rcN -mv apache-calcite-* ~/dist/dev/calcite/apache-calcite-X.Y.Z-rcN +mv apache-calcite-<span class="k">*</span> ~/dist/dev/calcite/apache-calcite-X.Y.Z-rcN <span class="c"># Check in</span> <span class="nb">cd</span> ~/dist/dev/calcite svn add apache-calcite-X.Y.Z-rcN -svn ci</code></pre></div> +svn ci</code></pre></figure> <h2 id="cleaning-up-after-a-failed-release-attempt-for-calcite-committers">Cleaning up after a failed release attempt (for Calcite committers)</h2> -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Make sure that the tag you are about to generate does not already</span> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Make sure that the tag you are about to generate does not already</span> <span class="c"># exist (due to a failed release attempt)</span> git tag @@ -1387,53 +1387,53 @@ mvn release:clean <span class="c"># Check whether there are modified files and if so, go back to the</span> <span class="c"># original git commit</span> git status -git reset --hard HEAD</code></pre></div> +git reset --hard HEAD</code></pre></figure> <h2 id="validate-a-release">Validate a release</h2> -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Check that the signing key (e.g. 2AD3FAE3) is pushed</span> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Check that the signing key (e.g. 2AD3FAE3) is pushed</span> gpg --recv-keys key <span class="c"># Check keys</span> curl -O https://dist.apache.org/repos/dist/release/calcite/KEYS <span class="c"># Sign/check md5 and sha1 hashes</span> -<span class="c"># (Assumes your O/S has 'md5' and 'sha1' commands.)</span> -<span class="k">function</span> checkHash<span class="o">()</span> <span class="o">{</span> - <span class="nb">cd</span> <span class="s2">"$1"</span> - <span class="k">for</span> i in *.<span class="o">{</span>zip,pom,gz<span class="o">}</span><span class="p">;</span> <span class="k">do</span> - <span class="k">if</span> <span class="o">[</span> ! -f <span class="nv">$i</span> <span class="o">]</span><span class="p">;</span> <span class="k">then</span> - <span class="k">continue</span> - <span class="k">fi</span> - <span class="k">if</span> <span class="o">[</span> -f <span class="nv">$i</span>.md5 <span class="o">]</span><span class="p">;</span> <span class="k">then</span> - <span class="k">if</span> <span class="o">[</span> <span class="s2">"$(cat $i.md5)"</span> <span class="o">=</span> <span class="s2">"$(md5 -q $i)"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then</span> - <span class="nb">echo</span> <span class="nv">$i</span>.md5 present and correct - <span class="k">else</span> - <span class="nb">echo</span> <span class="nv">$i</span>.md5 does not match - <span class="k">fi</span> - <span class="k">else</span> - md5 -q <span class="nv">$i</span> > <span class="nv">$i</span>.md5 +<span class="c"># (Assumes your O/S has 'md5' and 'sha1' commands.)</span> +<span class="k">function </span>checkHash<span class="o">()</span> <span class="o">{</span> + <span class="nb">cd</span> <span class="s2">"</span><span class="nv">$1</span><span class="s2">"</span> + <span class="k">for </span>i <span class="k">in</span> <span class="k">*</span>.<span class="o">{</span>zip,pom,gz<span class="o">}</span>; <span class="k">do + if</span> <span class="o">[</span> ! -f <span class="nv">$i</span> <span class="o">]</span>; <span class="k">then + continue + fi + if</span> <span class="o">[</span> -f <span class="nv">$i</span>.md5 <span class="o">]</span>; <span class="k">then + if</span> <span class="o">[</span> <span class="s2">"</span><span class="k">$(</span>cat <span class="nv">$i</span>.md5<span class="k">)</span><span class="s2">"</span> <span class="o">=</span> <span class="s2">"</span><span class="k">$(</span>md5 -q <span class="nv">$i</span><span class="k">)</span><span class="s2">"</span> <span class="o">]</span>; <span class="k">then + </span><span class="nb">echo</span> <span class="nv">$i</span>.md5 present and correct + <span class="k">else + </span><span class="nb">echo</span> <span class="nv">$i</span>.md5 does not match + <span class="k">fi + else + </span>md5 -q <span class="nv">$i</span> > <span class="nv">$i</span>.md5 <span class="nb">echo</span> <span class="nv">$i</span>.md5 created - <span class="k">fi</span> - <span class="k">if</span> <span class="o">[</span> -f <span class="nv">$i</span>.sha1 <span class="o">]</span><span class="p">;</span> <span class="k">then</span> - <span class="k">if</span> <span class="o">[</span> <span class="s2">"$(cat $i.sha1)"</span> <span class="o">=</span> <span class="s2">"$(sha1 -q $i)"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then</span> - <span class="nb">echo</span> <span class="nv">$i</span>.sha1 present and correct - <span class="k">else</span> - <span class="nb">echo</span> <span class="nv">$i</span>.sha1 does not match - <span class="k">fi</span> - <span class="k">else</span> - sha1 -q <span class="nv">$i</span> > <span class="nv">$i</span>.sha1 + <span class="k">fi + if</span> <span class="o">[</span> -f <span class="nv">$i</span>.sha1 <span class="o">]</span>; <span class="k">then + if</span> <span class="o">[</span> <span class="s2">"</span><span class="k">$(</span>cat <span class="nv">$i</span>.sha1<span class="k">)</span><span class="s2">"</span> <span class="o">=</span> <span class="s2">"</span><span class="k">$(</span>sha1 -q <span class="nv">$i</span><span class="k">)</span><span class="s2">"</span> <span class="o">]</span>; <span class="k">then + </span><span class="nb">echo</span> <span class="nv">$i</span>.sha1 present and correct + <span class="k">else + </span><span class="nb">echo</span> <span class="nv">$i</span>.sha1 does not match + <span class="k">fi + else + </span>sha1 -q <span class="nv">$i</span> > <span class="nv">$i</span>.sha1 <span class="nb">echo</span> <span class="nv">$i</span>.sha1 created - <span class="k">fi</span> - <span class="k">done</span> + <span class="k">fi + done</span> <span class="o">}</span> -checkHash apache-calcite-X.Y.Z-rcN</code></pre></div> +checkHash apache-calcite-X.Y.Z-rcN</code></pre></figure> <h2 id="get-approval-for-a-release-via-apache-voting-process-for-calcite-committers">Get approval for a release via Apache voting process (for Calcite committers)</h2> <p>Release vote on dev list</p> -<div class="highlight"><pre><code class="language-text" data-lang="text">To: [email protected] +<figure class="highlight"><pre><code class="language-text" data-lang="text">To: [email protected] Subject: [VOTE] Release apache-calcite-X.Y.Z (release candidate N) Hi all, @@ -1470,7 +1470,7 @@ The vote is open for the next 72 hours a at least three +1 PMC votes are cast. [ ] +1 Release this package as Apache Calcite X.Y.Z -[ ] 0 I don't feel strongly about it, but I'm okay with the release +[ ] 0 I don't feel strongly about it, but I'm okay with the release [ ] -1 Do not release this package because... @@ -1478,11 +1478,11 @@ Here is my vote: +1 (binding) -Julian</code></pre></div> +Julian</code></pre></figure> <p>After vote finishes, send out the result:</p> -<div class="highlight"><pre><code class="language-text" data-lang="text">Subject: [RESULT] [VOTE] Release apache-calcite-X.Y.Z (release candidate N) +<figure class="highlight"><pre><code class="language-text" data-lang="text">Subject: [RESULT] [VOTE] Release apache-calcite-X.Y.Z (release candidate N) To: [email protected] Thanks to everyone who has tested the release candidate and given @@ -1507,7 +1507,7 @@ There was some feedback during voting. I thread to discuss. -Julian</code></pre></div> +Julian</code></pre></figure> <p>Use the <a href="http://s.apache.org";>Apache URL shortener</a> to generate shortened URLs for the vote proposal and result emails. Examples: @@ -1543,12 +1543,12 @@ Uncheck âSend mail for this update� <p>Check the artifacts into svn.</p> -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Get the release candidate.</span> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Get the release candidate.</span> mkdir -p ~/dist/dev <span class="nb">cd</span> ~/dist/dev svn co https://dist.apache.org/repos/dist/dev/calcite -<span class="c"># Copy the artifacts. Note that the copy does not have '-rcN' suffix.</span> +<span class="c"># Copy the artifacts. Note that the copy does not have '-rcN' suffix.</span> mkdir -p ~/dist/release <span class="nb">cd</span> ~/dist/release svn co https://dist.apache.org/repos/dist/release/calcite @@ -1557,7 +1557,7 @@ cp -rp ../../dev/calcite/apache-calcite- svn add apache-calcite-X.Y.Z <span class="c"># Check in.</span> -svn ci</code></pre></div> +svn ci</code></pre></figure> <p>Svnpubsub will publish to the <a href="https://dist.apache.org/repos/dist/release/calcite";>release repo</a> and propagate to the @@ -1565,16 +1565,16 @@ svn ci</code></pre></div> <p>If there are now more than 2 releases, clear out the oldest ones:</p> -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">cd</span> ~/dist/release/calcite +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">cd</span> ~/dist/release/calcite svn rm apache-calcite-X.Y.Z -svn ci</code></pre></div> +svn ci</code></pre></figure> <p>The old releases will remain available in the <a href="http://archive.apache.org/dist/calcite/";>release archive</a>.</p> <p>Add a release note by copying <a href="https://github.com/apache/calcite/blob/master/site/_posts/2015-11-10-release-1.5.0.md";>site/_posts/2015-11-10-release-1.5.0.md</a>, -generate the javadoc and copy to <code>site/target/apidocs</code> and <code>site/target/testapidocs</code>, +generate the javadoc and copy to <code class="highlighter-rouge">site/target/apidocs</code> and <code class="highlighter-rouge">site/target/testapidocs</code>, <a href="#publish-the-web-site">publish the site</a>, and check that it appears in the contents in <a href="http://localhost:4000/news/";>news</a>.</p> @@ -1668,74 +1668,50 @@ as a template. Be sure to include a brie - - - - - - - - - - - - - - - - - - - - - - - - <li class=""><a href="/docs/index.html">Background</a></li> @@ -1746,116 +1722,78 @@ as a template. Be sure to include a brie - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - <li class=""><a href="/docs/tutorial.html">Tutorial</a></li> @@ -1866,20 +1804,14 @@ as a template. Be sure to include a brie - - - - - - <li class=""><a href="/docs/algebra.html">Algebra</a></li> @@ -1898,14 +1830,10 @@ as a template. Be sure to include a brie - - - - <li class=""><a href="/docs/adapter.html">Adapters</a></li> @@ -1916,104 +1844,70 @@ as a template. Be sure to include a brie - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - <li class=""><a href="/docs/stream.html">Streaming</a></li> @@ -2024,80 +1918,54 @@ as a template. Be sure to include a brie - - - - - - - - - - - - - - - - - - - - - - - - - - <li class=""><a href="/docs/lattice.html">Lattices</a></li> @@ -2116,38 +1984,26 @@ as a template. Be sure to include a brie - - - - - - - - - - - - <li class=""><a href="/docs/avatica_overview.html">Overview</a></li> @@ -2158,50 +2014,34 @@ as a template. Be sure to include a brie - - - - - - - - - - - - - - - - <li class=""><a href="/docs/avatica_roadmap.html">Roadmap</a></li> @@ -2212,32 +2052,22 @@ as a template. Be sure to include a brie - - - - - - - - - - <li class=""><a href="/docs/avatica_json_reference.html">JSON Reference</a></li> @@ -2248,44 +2078,30 @@ as a template. Be sure to include a brie - - - - - - - - - - - - - - <li class=""><a href="/docs/avatica_protobuf_reference.html">Protobuf Reference</a></li> @@ -2304,98 +2120,66 @@ as a template. Be sure to include a brie - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - <li class=""><a href="/docs/reference.html">SQL language</a></li> @@ -2406,86 +2190,58 @@ as a template. Be sure to include a brie - - - - - - - - - - - - - - - - - - - - - - - - - - - - <li class=""><a href="/docs/model.html">JSON models</a></li> @@ -2496,68 +2252,46 @@ as a template. Be sure to include a brie - - - - - - - - - - - - - - - - - - - - - - <li class="current"><a href="/docs/howto.html">HOWTO</a></li> @@ -2576,62 +2310,42 @@ as a template. Be sure to include a brie - - - - - - - - - - - - - - - - - - - - <li class=""><a href="/docs/history.html">History</a></li> @@ -2642,92 +2356,62 @@ as a template. Be sure to include a brie - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - <li class=""><a href="/docs/powered_by.html">Powered By Calcite</a></li> @@ -2738,143 +2422,161 @@ as a template. Be sure to include a brie - - - - - - - - - <li class=""><a href="/docs/api.html">API</a></li> - - - - - - - - - - - + + - - - + + + - + + + - + + + - + + + - + + + - + + + + + + + + + + + + - + + + - + + + - + + + - + + + - + + + - + + + - + + + - + + + - + - <li class=""><a href="/docs/testapi.html">Test API</a></li> - + + + </ul>
