Modified: kylin/site/docs/tutorial/web.html URL: http://svn.apache.org/viewvc/kylin/site/docs/tutorial/web.html?rev=1735396&r1=1735395&r2=1735396&view=diff ============================================================================== --- kylin/site/docs/tutorial/web.html (original) +++ kylin/site/docs/tutorial/web.html Thu Mar 17 10:34:37 2016 @@ -535,6 +535,10 @@ + + + + @@ -853,6 +857,10 @@ + + + + <li><a href="/docs/tutorial/kylin_sample.html" class="list-group-item-lay pjaxlink" id="navlist">Quick Start with Sample Cube</a></li> @@ -1167,6 +1175,10 @@ + + + + <li><a href="/docs/tutorial/web.html" class="list-group-item-lay pjaxlink" id="navlist">Kylin Web Interface</a></li> @@ -1257,6 +1269,10 @@ + + + + <li><a href="/docs/tutorial/tableau.html" class="list-group-item-lay pjaxlink" id="navlist">Tableau 8</a></li> @@ -1351,6 +1367,10 @@ + + + + <li><a href="/docs/tutorial/tableau_91.html" class="list-group-item-lay pjaxlink" id="navlist">Tableau 9</a></li> @@ -1437,6 +1457,10 @@ + + + + <li><a href="/docs/tutorial/powerbi.html" class="list-group-item-lay pjaxlink" id="navlist">MS Excel and Power BI</a></li> @@ -1519,10 +1543,94 @@ + + + + <li><a href="/docs/tutorial/odbc.html" class="list-group-item-lay pjaxlink" id="navlist">Kylin ODBC Driver</a></li> + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + <li><a href="/docs/tutorial/kylin_client_tool.html" class="list-group-item-lay pjaxlink" id="navlist">Kylin Client Tool Tutorial</a></li> + + +
Modified: kylin/site/docs15/howto/howto_upgrade.html URL: http://svn.apache.org/viewvc/kylin/site/docs15/howto/howto_upgrade.html?rev=1735396&r1=1735395&r2=1735396&view=diff ============================================================================== --- kylin/site/docs15/howto/howto_upgrade.html (original) +++ kylin/site/docs15/howto/howto_upgrade.html Thu Mar 17 10:34:37 2016 @@ -1652,87 +1652,112 @@ <article class="post-content" > - <h2 id="upgrade-from-v13-to-v15">Upgrade from v1.3 to v1.5</h2> + <h2 id="upgrade-from-prior-15-to-v151">Upgrade from prior 1.5 to v1.5.1</h2> -<p>From v1.3 to v1.5, Kylinâs cube data is backward compatible, but metadata has been refactored as new schema, to support new features on cubing and query enhancement. So if you want to deploy v1.5 on your v1.3 base, you need to upgrade the metadata as following steps:</p> +<p>Kylin 1.5.1 is not backward compatible in terms of metadata. (The built cubes are still functional after metadata upgrade) So if you want to deploy v1.5.x code on your prior 1.5 metadata store (in the following text weâll use v1.3.0 as example), you need to upgrade the metadata as following steps:</p> + +<h4 id="backup-metadata-on-v130">1. Backup metadata on v1.3.0</h4> -<h4 id="backup-metadata-on-v13">1. Backup metadata on v1.3</h4> <p>To avoid data loss during the upgrade, a backup at the very beginning is always suggested. In case of upgrade failure, you can roll back to original state with the backup.</p> -<div class="highlighter-rouge"><pre class="highlight"><code>$KYLIN_HOME/bin/metastore.sh backup +<div class="highlighter-rouge"><pre class="highlight"><code>export KYLIN_HOME="<path_of_1_3_0_installation>" +$KYLIN_HOME/bin/metastore.sh backup </code></pre> </div> -<p>It will print the backup folder, take it down and make sure it will not be deleted before the upgrade finished. If there is no âmetastore.shâ, you can use HBaseâs snapshot command to do backup:</p> -<div class="highlighter-rouge"><pre class="highlight"><code>hbase shell -snapshot 'kylin_metadata', 'kylin_metadata_backup20160101' +<p>It will print the backup folder, write it down and make sure it will not be deleted before the upgrade finished. Weâll later reference this folder as BACKUP_FOLDER.</p> + +<h4 id="stop-kylin-v130-instance">2. Stop Kylin v1.3.0 instance</h4> + +<p>Before deploying Kylin v1.5.1 instance, you need to stop the old instance. Note that end users cannot access kylin service from this point.</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>$KYLIN_HOME/bin/kylin.sh stop </code></pre> </div> -<p>Here âkylin_metadataâ is the default kylin metadata table name, replace it with the right table name of your Kylin metastore.</p> -<h4 id="stop-kylin-v13-instance">2. Stop Kylin v1.3 instance</h4> -<p>Before deploying Kylin v1.5 instance, you need to stop the old instance. Note that end users cannot access kylin service from this point.</p> +<h4 id="install-kylin-v151-and-copy-back-conf">3. Install Kylin v1.5.1 and copy back âconfâ</h4> -<div class="highlighter-rouge"><pre class="highlight"><code>$KYLIN_HOME/bin/kylin.sh stop +<p>Download the new Kylin v1.5.1 binary package from Kylinâs download page; Extract it to a different folder other than current KYLIN_HOME; Before copy back the âconfâ folder, do a compare and merge between the old and new kylin.properties to ensure newly introduced property will be kept.</p> + +<h4 id="automaticly-upgrade-metadata">4. Automaticly upgrade metadata</h4> + +<p>Kylin v1.5.1 package provides a tool for metadata automaticly upgrade. In this upgrade, all cubesâ metadata will be updated to v1.5.1 compatible format. The treatment of empty cubes and non-empty cubes are different though. For empty cubes, weâll upgrade the cubeâs storage engine and cubing engine to the latest, so that new features will be enabled for the new coming cubing jobs. But those non-empty cubes carries legacy cube segments, so weâll remain its old storage engine and cubing engine. In other word, the non-empty cubes will not enjoy the performance and storage wise gains released in 1.5.x versions. Check the last section to see how to deal with non-empty cubes.<br /> +To avoid corrupting the metadata store, metadata upgrade is performed against a copy of the local metadata backup, i.e. a copy of BACKUP_FOLDER.</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>export KYLIN_HOME="<path_of_1_5_0_installation>" +$KYLIN_HOME/bin/kylin.sh org.apache.kylin.cube.upgrade.entry.CubeMetadataUpgradeEntry_v_1_5_1 <path_of_BACKUP_FOLDER> </code></pre> </div> -<p>#### 3. Install Kylin v1.5 and copy back âconfâ<br /> -Download the new Kylin v1.5 binary package from Kylinâs download page; Extract it to a different folder other than current KYLIN_HOME; Before copy back the âconfâ folder, do a compare and merge between the old and new kylin.properties to ensure newly introduced property will be kept.</p> -<h4 id="optional-4-upgrading-metadata-will-not-bring-new-features-of-v15-to-existing-cube-built-with-v13-engine-if-you-want-to-leverage-those-features-please-refer-to-highlight-part">(Optional) 4. Upgrading metadata will not bring new features of v1.5 to existing cube built with v1.3 engine. If you want to leverage those features, please refer to <a href="">Highlight</a> part.</h4> +<p>The above commands will first copy the BACKUP_FOLDER to ${BACKUP_FOLDER}_workspace, and perform the upgrade against the workspace folder at local disk. Check the output, if no error happened, then you have a 1.5.1 compatible metadata saved in the workspace folder now. Otherwise the upgrade process is not successful, please donât take further actions. <br /> +The next thing to do is to override the metatdata store with the new metadata in workspace:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>$KYLIN_HOME/bin/metastore.sh reset +$KYLIN_HOME/bin/metastore.sh restore <path_of_workspace> +</code></pre> +</div> -<h4 id="automaticly-upgrade-metadata">5. Automaticly upgrade metadata</h4> -<p>Kylin v1.5 package provides a script for metadata automaticly upgrade. In this upgrade, empty cubes will be updated to v1.5 version and all new features are enabled for them. But those non-empty cubes are not able to use those new features.</p> +<p>The last thing to do is to upgrade all cubesâ coprocessor:</p> -<div class="highlighter-rouge"><pre class="highlight"><code>export KYLIN_HOME="<path_of_new_installation>" -$KYLIN_HOME/bin/upgrade_v2.sh +<div class="highlighter-rouge"><pre class="highlight"><code>$KYLIN_HOME/bin/kylin.sh org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI $KYLIN_HOME/lib/kylin-coprocessor*.jar all </code></pre> </div> -<p>After this, the metadata in hbase table has been applied with new metadata schema.</p> -<h4 id="start-kylin-v15-instance">6. Start Kylin v1.5 instance</h4> +<h4 id="start-kylin-v151-instance">6. Start Kylin v1.5.1 instance</h4> + <div class="highlighter-rouge"><pre class="highlight"><code>$KYLIN_HOME/bin/kylin.sh start </code></pre> </div> + <p>Check the log and open web UI to see if the upgrade succeeded.</p> <h2 id="rollback-if-the-upgrade-is-failed">Rollback if the upgrade is failed</h2> -<p>If the new version couldnât startup normally, you need to roll back to orignal v1.3 version. The steps are as followed:</p> -<h4 id="stop-kylin-v15-instance">1. Stop Kylin v1.5 instance</h4> +<p>If the new version couldnât startup normally, you need to roll back to orignal v1.3.0 version. The steps are as followed:</p> + +<h4 id="stop-kylin-v151-instance">1. Stop Kylin v1.5.1 instance</h4> <div class="highlighter-rouge"><pre class="highlight"><code>$KYLIN_HOME/bin/kylin.sh stop </code></pre> </div> -<p>#### 2. Restore 1.3 metadata from backup folder</p> -<div class="highlighter-rouge"><pre class="highlight"><code>export KYLIN_HOME="<path_of_1.3_installation>" -$KYLIN_HOME/bin/metastore.sh restore <backup_folder> +<h4 id="restore-130-metadata-from-backup-folder">2. Restore 1.3.0 metadata from backup folder</h4> + +<div class="highlighter-rouge"><pre class="highlight"><code>export KYLIN_HOME="<path_of_1_3_0_installation>" +$KYLIN_HOME/bin/metastore.sh reset +$KYLIN_HOME/bin/metastore.sh restore <path_of_BACKUP_FOLDER> </code></pre> </div> -<p>#### 3. Deploy coprocessor of v1.3<br /> -Since coprocessor of used HTable are upgraded as v1.5, you need to manually downgrade them with this command.</p> -<div class="highlighter-rouge"><pre class="highlight"><code>$KYLIN_HOME/bin/kylin.sh org.apache.kylin.job.tools.DeployCoprocessorCLI $KYLIN_HOME/lib/kylin-coprocessor*.jar -all +<h4 id="deploy-coprocessor-of-v130">3. Deploy coprocessor of v1.3.0</h4> + +<p>Since coprocessor of used HTable are upgraded as v1.5.1, you need to manually downgrade them with this command.</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>$KYLIN_HOME/bin/kylin.sh org.apache.kylin.job.tools.DeployCoprocessorCLI $KYLIN_HOME/lib/kylin-coprocessor*.jar all </code></pre> </div> -<h4 id="start-kylin-v13-instance">4. Start Kylin v1.3 instance</h4> +<h4 id="start-kylin-v130-instance">4. Start Kylin v1.3.0 instance</h4> <div class="highlighter-rouge"><pre class="highlight"><code>$KYLIN_HOME/bin/kylin.sh start </code></pre> </div> -<h2 id="highlights">Highlights</h2> -<p>Since old cubes built with v1.3 cannot leverage new features of v1.5. But if you must have them on your cubes, you can choose one of these solutions:<br /> -#### 1. Rebuilt cubes<br /> -If the cost of rebuilding is acceptable, if you purge the cube before Step 4(Running Upgrade Scripts). After upgrade done, you need to manually rebuilt those segments by yourself.<br /> -#### 2. Use hybrid model<br /> -If you donât want to rebuild any cube, but want to leverage new features for new data. You can use hybrid model, which contains not only your old cube, but also an empty cube which has same model with the old one. For the empty cube, you can do incremental building with v2 features. For the old cube, you can refresh existing segments only.</p> +<h2 id="for-non-empty-cubes">For non-empty cubes</h2> + +<p>Since old cubes built with v1.3.0 cannot leverage new features of v1.5.1. But if you must have them on your cubes, you can choose one of these solutions:</p> + +<h4 id="rebuild-cubes">1. Rebuild cubes</h4> + +<p>This is the simplest way: If the cost of rebuilding is acceptable, if you purge the cube before Metadata Upgrade. After upgrade done, you need to manually rebuild those segments by yourself.</p> + +<h4 id="use-hybrid-model">2. Use hybrid model</h4> + +<p>If you canât rebuild any segments, but want to leverage new features for new segments. You can use hybrid model, which contains not only your old segments, but also an new empty cube which has same model with the old one. For the empty cube, you can do incremental building with new v1.5.x features. For the old cube, you can refresh existing segments only.</p> <p>Here is the command to create hybrid model:</p> -<div class="highlighter-rouge"><pre class="highlight"><code>export KYLIN_HOME="<path_of_v1.5_installation>" +<div class="highlighter-rouge"><pre class="highlight"><code>export KYLIN_HOME="<path_of_1_5_0_installation>" $KYLIN_HOME/bin/kylin.sh org.apache.kylin.storage.hbase.util.ExtendCubeToHybridCLI <project_name> <cube_name> </code></pre> </div>
