kylin git commit: minor, fix MeasureDecoder getPeekLength method

2016-06-07 Thread liyang 
Repository: kylin
Updated Branches:
  refs/heads/master 800fbd218 -> 6788ee6e1


minor, fix MeasureDecoder getPeekLength method

Signed-off-by: Li Yang 


Project: http://git-wip-us.apache.org/repos/asf/kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/6788ee6e
Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/6788ee6e
Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/6788ee6e

Branch: refs/heads/master
Commit: 6788ee6e1aa925334c79bb422a09ea7957363537
Parents: 800fbd2
Author: Roger Shi 
Authored: Wed Jun 8 11:32:53 2016 +0800
Committer: Li Yang 
Committed: Wed Jun 8 11:40:53 2016 +0800

--
 .../src/main/java/org/apache/kylin/measure/MeasureDecoder.java| 3 +++
 1 file changed, 3 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/kylin/blob/6788ee6e/core-metadata/src/main/java/org/apache/kylin/measure/MeasureDecoder.java
--
diff --git 
a/core-metadata/src/main/java/org/apache/kylin/measure/MeasureDecoder.java 
b/core-metadata/src/main/java/org/apache/kylin/measure/MeasureDecoder.java
index 87b4603..9656020 100644
--- a/core-metadata/src/main/java/org/apache/kylin/measure/MeasureDecoder.java
+++ b/core-metadata/src/main/java/org/apache/kylin/measure/MeasureDecoder.java
@@ -78,8 +78,11 @@ public class MeasureDecoder {
 
 public int[] getPeekLength(ByteBuffer buf) {
 int[] length = new int[nMeasures];
+int offset = 0;
 for (int i = 0; i < nMeasures; i++) {
 length[i] = serializers[i].peekLength(buf);
+offset += length[i];
+buf.position(offset);
 }
 return length;
 }

kylin git commit: update download page for v1.5.2.1

2016-06-07 Thread shaofengshi 
Repository: kylin
Updated Branches:
  refs/heads/document 19138dcd5 -> b918189a4


update download page for v1.5.2.1


Project: http://git-wip-us.apache.org/repos/asf/kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/b918189a
Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/b918189a
Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/b918189a

Branch: refs/heads/document
Commit: b918189a44f2f3e8512c16459fb699f89bd1558a
Parents: 19138dc
Author: shaofengshi 
Authored: Tue Jun 7 22:25:47 2016 +0800
Committer: shaofengshi 
Committed: Tue Jun 7 22:25:47 2016 +0800

--
 website/_docs15/release_notes.md | 11 +++
 website/download/index.cn.md | 24 
 website/download/index.md| 16 
 3 files changed, 31 insertions(+), 20 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kylin/blob/b918189a/website/_docs15/release_notes.md
--
diff --git a/website/_docs15/release_notes.md b/website/_docs15/release_notes.md
index e212373..19094be 100644
--- a/website/_docs15/release_notes.md
+++ b/website/_docs15/release_notes.md
@@ -15,6 +15,17 @@ or send to Apache Kylin mailing list:
 * User relative: [u...@kylin.apache.org](mailto:u...@kylin.apache.org)
 * Development relative: [d...@kylin.apache.org](mailto:d...@kylin.apache.org)
 
+## v1.5.2.1 - 2016-06-07
+_Tag:_ [kylin-1.5.2.1](https://github.com/apache/kylin/tree/kylin-1.5.2.1)
+
+This is a hot-fix version on v1.5.2, no new feature introduced, please upgrade 
to this version;
+
+__Bug__
+
+* [KYLIN-1758] - createLookupHiveViewMaterializationStep will create 
intermediate table for fact table
+* [KYLIN-1739] - kylin_job_conf_inmem.xml can impact non-inmem MR job
+
+
 ## v1.5.2 - 2016-05-26
 _Tag:_ [kylin-1.5.2](https://github.com/apache/kylin/tree/kylin-1.5.2)
 

http://git-wip-us.apache.org/repos/asf/kylin/blob/b918189a/website/download/index.cn.md
--
diff --git a/website/download/index.cn.md b/website/download/index.cn.md
index 7060250..f60496a 100644
--- a/website/download/index.cn.md
+++ b/website/download/index.cn.md
@@ -3,31 +3,31 @@ layout: download-cn
 title: 下载
 ---
 
-__1.5.2 最新发布(源代码)__
+__最新发布(源代码)__
 
 最新发布的Apache Kylin™可以从ASF网站下载：:
 
-* [Apache Kylin 
v1.5.2](http://www.apache.org/dyn/closer.cgi/kylin/apache-kylin-1.5.2/)
+* [Apache Kylin 
v1.5.2.1](http://www.apache.org/dyn/closer.cgi/kylin/apache-kylin-1.5.2.1/)
 * [发布日志](http://kylin.apache.org/docs15/release_notes.html)
-* Git 标签: [kylin-1.5.2](https://github.com/apache/kylin/tree/kylin-1.5.2)
-* Git Commit: 
[af2646b72fbb6dc81699ad6661303fd612a2eebf](https://github.com/apache/kylin/commit/af2646b72fbb6dc81699ad6661303fd612a2eebf)
+* Git 标签: 
[kylin-1.5.2.1](https://github.com/apache/kylin/tree/kylin-1.5.2.1)
+* Git Commit: 
[71a847bbe97ec4e6f23b1bf271132d6ff9955a96](https://github.com/apache/kylin/commit/71a847bbe97ec4e6f23b1bf271132d6ff9955a96)
 
-__1.5.2 二进制包 for HBase 0.98/0.99__
+__1.5.2.1 二进制包 for HBase 0.98/0.99__
 
 为方便使用，我们提供预打包的二进制安装包：
 
-* 
[apache-kylin-1.5.2-bin.tar.gz](https://dist.apache.org/repos/dist/release/kylin/apache-kylin-1.5.1/apache-kylin-1.5.2-bin.tar.gz)
+* 
[apache-kylin-1.5.2.1-bin.tar.gz](https://dist.apache.org/repos/dist/release/kylin/apache-kylin-1.5.1/apache-kylin-1.5.2.1-bin.tar.gz)
 * [安装帮助](http://kylin.apache.org/docs15/install)
 
-__1.5.2 二进制包 for HBase 1.x__
+__1.5.2.1 二进制包 for HBase 1.x__
 
-* 
[apache-kylin-1.5.2-HBase1.x-bin.tar.gz](https://dist.apache.org/repos/dist/release/kylin/apache-kylin-1.5.1/apache-kylin-1.5.2-HBase1.x-bin.tar.gz)
-* Git commit 
[e85b49823e3cfea027cca497e898064fbefbd449](https://github.com/apache/kylin/commit/e85b49823e3cfea027cca497e898064fbefbd449)
+* 
[apache-kylin-1.5.2.1-HBase1.x-bin.tar.gz](https://dist.apache.org/repos/dist/release/kylin/apache-kylin-1.5.1/apache-kylin-1.5.2.1-HBase1.x-bin.tar.gz)
+* Git commit 
[cf4d2940b67d622eacd2ac9a913b221091a35c2e](https://github.com/apache/kylin/commit/cf4d2940b67d622eacd2ac9a913b221091a35c2e)
 
-__1.5.2 二进制包 for CDH 5.7__
+__1.5.2.1 二进制包 for CDH 5.7__
 
-* 
[apache-kylin-1.5.2-cdh5.7-bin.tar.gz](https://dist.apache.org/repos/dist/release/kylin/apache-kylin-1.5.1/apache-kylin-1.5.2-cdh5.7-bin.tar.gz)
-* Git commit 
[e0fbb3417063fe33b99e82724e19e107787b2e62](https://github.com/apache/kylin/commit/e0fbb3417063fe33b99e82724e19e107787b2e62)
+* 
[apache-kylin-1.5.2.1-cdh5.7-bin.tar.gz](https://dist.apache.org/repos/dist/release/kylin/apache-kylin-1.5.1/apache-kylin-1.5.2.1-cdh5.7-bin.tar.gz)
+* Git commit

svn commit: r13923 - in /release/kylin/apache-kylin-1.5.2.1: apache-kylin-1.5.2.1-HBase1.x-bin.tar.gz apache-kylin-1.5.2.1-HBase1.x.bin.tar.gz

2016-06-07 Thread shaofengshi 
Author: shaofengshi
Date: Tue Jun  7 14:23:58 2016
New Revision: 13923

Log:
fix wrong file name

Added:
release/kylin/apache-kylin-1.5.2.1/apache-kylin-1.5.2.1-HBase1.x-bin.tar.gz
  - copied unchanged from r13922, 
release/kylin/apache-kylin-1.5.2.1/apache-kylin-1.5.2.1-HBase1.x.bin.tar.gz
Removed:
release/kylin/apache-kylin-1.5.2.1/apache-kylin-1.5.2.1-HBase1.x.bin.tar.gz

[1/2] kylin git commit: KYLIN-1762 Query threw NPE with 3 or more join conditions

2016-06-07 Thread liyang 
Repository: kylin
Updated Branches:
  refs/heads/master 0c9496859 -> 800fbd218


KYLIN-1762 Query threw NPE with 3 or more join conditions

Signed-off-by: Yang Li 


Project: http://git-wip-us.apache.org/repos/asf/kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/4e52d2fc
Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/4e52d2fc
Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/4e52d2fc

Branch: refs/heads/master
Commit: 4e52d2fcb92aa337a47546d2fb28b1c5acbb79dd
Parents: 0c94968
Author: sunyerui 
Authored: Sat Jun 4 23:04:48 2016 +0800
Committer: Yang Li 
Committed: Tue Jun 7 21:35:06 2016 +0800

--
 .../src/test/resources/query/sql/query100.sql   | 28 
 .../apache/kylin/query/relnode/OLAPJoinRel.java |  9 +--
 2 files changed, 35 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kylin/blob/4e52d2fc/kylin-it/src/test/resources/query/sql/query100.sql
--
diff --git a/kylin-it/src/test/resources/query/sql/query100.sql 
b/kylin-it/src/test/resources/query/sql/query100.sql
new file mode 100644
index 000..1edf92d
--- /dev/null
+++ b/kylin-it/src/test/resources/query/sql/query100.sql
@@ -0,0 +1,28 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one
+-- or more contributor license agreements.  See the NOTICE file
+-- distributed with this work for additional information
+-- regarding copyright ownership.  The ASF licenses this file
+-- to you under the Apache License, Version 2.0 (the
+-- "License"); you may not use this file except in compliance
+-- with the License.  You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+select t1.leaf_categ_id, max_price, min_price, sum_price
+from
+(select leaf_categ_id, sum(price) as sum_price from test_kylin_fact group by 
leaf_categ_id) t1
+join
+(select leaf_categ_id, max(price) as max_price from test_kylin_fact group by 
leaf_categ_id) t2
+on t1.leaf_categ_id = t2.leaf_categ_id
+join
+(select leaf_categ_id, min(price) as min_price from test_kylin_fact group by 
leaf_categ_id) t3
+on t1.leaf_categ_id = t3.leaf_categ_id
+order by t1.leaf_categ_id

http://git-wip-us.apache.org/repos/asf/kylin/blob/4e52d2fc/query/src/main/java/org/apache/kylin/query/relnode/OLAPJoinRel.java
--
diff --git 
a/query/src/main/java/org/apache/kylin/query/relnode/OLAPJoinRel.java 
b/query/src/main/java/org/apache/kylin/query/relnode/OLAPJoinRel.java
index efe404b..4b293bc 100644
--- a/query/src/main/java/org/apache/kylin/query/relnode/OLAPJoinRel.java
+++ b/query/src/main/java/org/apache/kylin/query/relnode/OLAPJoinRel.java
@@ -121,12 +121,17 @@ public class OLAPJoinRel extends EnumerableJoin 
implements OLAPRel {
 implementor.visitChild(this.left, this);
 if (this.context != implementor.getContext() || ((OLAPRel) 
this.left).hasSubQuery()) {
 this.hasSubQuery = true;
-implementor.freeContext();
+// child join node didn't allocated a new context, and free 
context should be skipped
+if (!(this.left instanceof OLAPJoinRel)) {
+implementor.freeContext();
+}
 }
 implementor.visitChild(this.right, this);
 if (this.context != implementor.getContext() || ((OLAPRel) 
this.right).hasSubQuery()) {
 this.hasSubQuery = true;
-implementor.freeContext();
+if (!(this.right instanceof OLAPJoinRel)) {
+implementor.freeContext();
+}
 }
 
 this.columnRowType = buildColumnRowType();

[kylin] Git Push Summary

2016-06-07 Thread shaofengshi 
Repository: kylin
Updated Tags:  refs/tags/kylin-1.5.2.1-HBase1.x [created] 2b2b3a228

[kylin] Git Push Summary

2016-06-07 Thread shaofengshi 
Repository: kylin
Updated Tags:  refs/tags/kylin-2.0-alpha [deleted] e159079f4

[kylin] Git Push Summary

2016-06-07 Thread shaofengshi 
Repository: kylin
Updated Tags:  refs/tags/v1.5.2-release [deleted] 354a13a36

[kylin] Git Push Summary

2016-06-07 Thread shaofengshi 
Repository: kylin
Updated Tags:  refs/tags/kylin-1.5.2.1-cdh5.7 [created] e760242da
  refs/tags/kylin-2.0-alpha [created] e159079f4
  refs/tags/v1.5.2-release [created] 354a13a36

[kylin] Git Push Summary

2016-06-07 Thread shaofengshi 
Repository: kylin
Updated Branches:
  refs/heads/kylin-1.5.2.1-cdh5.7-branch [deleted] e760242da

[3/3] kylin git commit: KYLIN-1528 Create a branch for v1.5 with HBase 1.x API

2016-06-07 Thread shaofengshi 
KYLIN-1528 Create a branch for v1.5 with HBase 1.x API


Project: http://git-wip-us.apache.org/repos/asf/kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/add37856
Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/add37856
Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/add37856

Branch: refs/heads/kylin-1.5.2.1-cdh5.7-branch
Commit: add37856b0e9701c1ff55351c09f02525582a99d
Parents: 71a847b
Author: shaofengshi 
Authored: Wed Mar 23 17:07:05 2016 +0800
Committer: shaofengshi 
Committed: Tue Jun 7 16:45:29 2016 +0800

--
 .../kylin/job/hadoop/invertedindex/IITest.java  | 10 ++-
 .../apache/kylin/common/KylinConfigBase.java|  5 +-
 examples/test_case_data/sandbox/hbase-site.xml  | 19 +---
 .../test_case_data/sandbox/kylin_job_conf.xml   | 86 +-
 examples/test_case_data/sandbox/mapred-site.xml | 23 +++--
 .../kylin/provision/BuildCubeWithEngine.java| 54 +--
 .../kylin/provision/BuildIIWithStream.java  | 13 +--
 .../hbase/ii/ITInvertedIndexHBaseTest.java  |  9 +-
 pom.xml | 18 ++--
 .../kylin/rest/security/AclHBaseStorage.java|  4 +-
 .../rest/security/MockAclHBaseStorage.java  |  8 +-
 .../apache/kylin/rest/security/MockHTable.java  | 95 
 .../rest/security/RealAclHBaseStorage.java  |  9 +-
 .../apache/kylin/rest/service/AclService.java   | 25 +++---
 .../apache/kylin/rest/service/CubeService.java  | 38 +++-
 .../apache/kylin/rest/service/QueryService.java | 21 +++--
 .../apache/kylin/rest/service/UserService.java  | 16 ++--
 .../kylin/storage/hbase/HBaseConnection.java| 36 
 .../kylin/storage/hbase/HBaseResourceStore.java | 29 +++---
 .../storage/hbase/cube/SimpleHBaseStore.java| 20 ++---
 .../hbase/cube/v1/CubeSegmentTupleIterator.java | 11 +--
 .../storage/hbase/cube/v1/CubeStorageQuery.java |  4 +-
 .../hbase/cube/v1/HBaseClientKVIterator.java| 11 +--
 .../hbase/cube/v1/RegionScannerAdapter.java | 11 ++-
 .../cube/v1/SerializedHBaseTupleIterator.java   |  4 +-
 .../observer/AggregateRegionObserver.java   |  4 +-
 .../observer/AggregationScanner.java| 15 +++-
 .../observer/ObserverAggregationCache.java  | 15 ++--
 .../coprocessor/observer/ObserverEnabler.java   |  4 +-
 .../hbase/cube/v2/CubeHBaseEndpointRPC.java | 11 +--
 .../storage/hbase/cube/v2/CubeHBaseScanRPC.java |  9 +-
 .../coprocessor/endpoint/CubeVisitService.java  |  4 +-
 .../storage/hbase/ii/IICreateHTableJob.java | 13 +--
 .../hbase/ii/InvertedIndexStorageQuery.java |  6 +-
 .../endpoint/EndpointTupleIterator.java | 13 +--
 .../ii/coprocessor/endpoint/IIEndpoint.java |  4 +-
 .../storage/hbase/steps/CubeHTableUtil.java | 16 ++--
 .../storage/hbase/steps/DeprecatedGCStep.java   | 24 ++---
 .../storage/hbase/steps/HBaseCuboidWriter.java  |  7 +-
 .../hbase/steps/HBaseStreamingOutput.java   |  9 +-
 .../kylin/storage/hbase/steps/MergeGCStep.java  | 24 ++---
 .../storage/hbase/util/CleanHtableCLI.java  | 13 +--
 .../storage/hbase/util/CubeMigrationCLI.java| 38 
 .../hbase/util/CubeMigrationCheckCLI.java   | 17 ++--
 .../hbase/util/DeployCoprocessorCLI.java| 23 ++---
 .../hbase/util/ExtendCubeToHybridCLI.java   |  8 +-
 .../hbase/util/GridTableHBaseBenchmark.java | 34 +++
 .../kylin/storage/hbase/util/HBaseClean.java| 18 ++--
 .../hbase/util/HBaseRegionSizeCalculator.java   | 42 -
 .../kylin/storage/hbase/util/HBaseUsage.java| 10 ++-
 .../storage/hbase/util/HbaseStreamingInput.java | 31 +++
 .../hbase/util/HtableAlterMetadataCLI.java  |  9 +-
 .../storage/hbase/util/OrphanHBaseCleanJob.java | 18 ++--
 .../kylin/storage/hbase/util/PingHBaseCLI.java  | 15 ++--
 .../kylin/storage/hbase/util/RowCounterCLI.java | 11 +--
 .../storage/hbase/util/StorageCleanupJob.java   | 16 ++--
 .../storage/hbase/util/UpdateHTableHostCLI.java | 17 ++--
 .../observer/AggregateRegionObserverTest.java   | 31 +++
 .../v1/filter/TestFuzzyRowFilterV2EndToEnd.java |  5 +-
 59 files changed, 530 insertions(+), 583 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kylin/blob/add37856/assembly/src/test/java/org/apache/kylin/job/hadoop/invertedindex/IITest.java
--
diff --git 
a/assembly/src/test/java/org/apache/kylin/job/hadoop/invertedindex/IITest.java 
b/assembly/src/test/java/org/apache/kylin/job/hadoop/invertedindex/IITest.java
index da25143..5d2cfc4 100644
--- 
a/assembly/src/test/java/org/apache/kylin/job/hadoop/invertedindex/IITest.java
+++ 
b/assembly/src/test/java/org/apache/kylin/job/hadoop/invertedindex/IITest.java
@@ -38,6 +38,7 @@ import org.apache.commons.lang.NotImplementedException;
 import org.apache.hadoop.hbase.Cell;

[1/3] kylin git commit: KYLIN-1672 support kylin on cdh 5.7

2016-06-07 Thread shaofengshi 
Repository: kylin
Updated Branches:
  refs/heads/kylin-1.5.2.1-cdh5.7-branch [created] e760242da


KYLIN-1672 support kylin on cdh 5.7

Signed-off-by: Li Yang 


Project: http://git-wip-us.apache.org/repos/asf/kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/e760242d
Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/e760242d
Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/e760242d

Branch: refs/heads/kylin-1.5.2.1-cdh5.7-branch
Commit: e760242daad739b66813dc2c2d8a52bf74c0ec22
Parents: add3785
Author: Lynne Jiang 
Authored: Mon May 16 03:33:27 2016 -0700
Committer: shaofengshi 
Committed: Tue Jun 7 16:45:29 2016 +0800

--
 dev-support/test_all_against_hdp_2_2_4_2_2.sh   |   0
 .../kylin/engine/mr/steps/MockupMapContext.java |  15 +-
 examples/test_case_data/sandbox/core-site.xml   | 146 +++---
 examples/test_case_data/sandbox/hbase-site.xml  | 162 ++
 examples/test_case_data/sandbox/hdfs-site.xml   | 259 ++
 examples/test_case_data/sandbox/mapred-site.xml | 396 ++-
 examples/test_case_data/sandbox/yarn-site.xml   | 496 ++-
 pom.xml |  18 +-
 server/pom.xml  |  36 ++
 .../storage/hbase/steps/MockupMapContext.java   |  19 +-
 10 files changed, 418 insertions(+), 1129 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kylin/blob/e760242d/dev-support/test_all_against_hdp_2_2_4_2_2.sh
--
diff --git a/dev-support/test_all_against_hdp_2_2_4_2_2.sh 
b/dev-support/test_all_against_hdp_2_2_4_2_2.sh
old mode 100644
new mode 100755

http://git-wip-us.apache.org/repos/asf/kylin/blob/e760242d/engine-mr/src/test/java/org/apache/kylin/engine/mr/steps/MockupMapContext.java
--
diff --git 
a/engine-mr/src/test/java/org/apache/kylin/engine/mr/steps/MockupMapContext.java
 
b/engine-mr/src/test/java/org/apache/kylin/engine/mr/steps/MockupMapContext.java
index 847071d..9900465 100644
--- 
a/engine-mr/src/test/java/org/apache/kylin/engine/mr/steps/MockupMapContext.java
+++ 
b/engine-mr/src/test/java/org/apache/kylin/engine/mr/steps/MockupMapContext.java
@@ -77,6 +77,7 @@ public class MockupMapContext {
 outKV[0] = key;
 outKV[1] = value;
 }
+
 }
 
 @Override
@@ -99,6 +100,7 @@ public class MockupMapContext {
 throw new NotImplementedException();
 }
 
+
 @Override
 public float getProgress() {
 throw new NotImplementedException();
@@ -195,17 +197,17 @@ public class MockupMapContext {
 }
 
 @Override
-public RawComparator getSortComparator() {
+public boolean userClassesTakesPrecedence() {
 throw new NotImplementedException();
 }
 
 @Override
-public String getJar() {
+public RawComparator getSortComparator() {
 throw new NotImplementedException();
 }
 
 @Override
-public RawComparator getGroupingComparator() {
+public String getJar() {
 throw new NotImplementedException();
 }
 
@@ -221,7 +223,7 @@ public class MockupMapContext {
 
 @Override
 public boolean getProfileEnabled() {
-throw new NotImplementedException();
+return false;
 }
 
 @Override
@@ -308,6 +310,11 @@ public class MockupMapContext {
 public RawComparator getCombinerKeyGroupingComparator() {
 throw new NotImplementedException();
 }
+
+@Override
+public RawComparator getGroupingComparator() {
+return null;
+}
 });
 }
 }

http://git-wip-us.apache.org/repos/asf/kylin/blob/e760242d/examples/test_case_data/sandbox/core-site.xml
--
diff --git a/examples/test_case_data/sandbox/core-site.xml 
b/examples/test_case_data/sandbox/core-site.xml
index 9aa588c..6162406 100644
--- a/examples/test_case_data/sandbox/core-site.xml
+++ b/examples/test_case_data/sandbox/core-site.xml
@@ -14,152 +14,146 @@
   See the License for the specific language governing permissions and
   limitations under the License.
 -->
+
 
-
 
 fs.defaultFS
-hdfs://sandbox.hortonworks.com:8020
-true
+hdfs://quickstart.cloudera:8020
 
-
 
 fs.trash.interval
-360
+1
 
-
 
-
ha.failover-controller.active-standby-elector.zk.op.retries
-120
+

[6/8] kylin git commit: [maven-release-plugin] prepare release kylin-1.5.2.1

2016-06-07 Thread shaofengshi 
[maven-release-plugin] prepare release kylin-1.5.2.1


Project: http://git-wip-us.apache.org/repos/asf/kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/71a847bb
Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/71a847bb
Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/71a847bb

Branch: refs/heads/1.5.x-HBase1.x
Commit: 71a847bbe97ec4e6f23b1bf271132d6ff9955a96
Parents: 404c530
Author: shaofengshi 
Authored: Sat Jun 4 11:51:30 2016 +
Committer: shaofengshi 
Committed: Sat Jun 4 11:51:30 2016 +

--
 assembly/pom.xml | 2 +-
 atopcalcite/pom.xml  | 2 +-
 core-common/pom.xml  | 2 +-
 core-cube/pom.xml| 2 +-
 core-dictionary/pom.xml  | 2 +-
 core-job/pom.xml | 2 +-
 core-metadata/pom.xml| 2 +-
 core-storage/pom.xml | 2 +-
 engine-mr/pom.xml| 2 +-
 engine-spark/pom.xml | 2 +-
 engine-streaming/pom.xml | 2 +-
 invertedindex/pom.xml| 2 +-
 jdbc/pom.xml | 2 +-
 kylin-it/pom.xml | 2 +-
 pom.xml  | 2 +-
 query/pom.xml| 2 +-
 server/pom.xml   | 2 +-
 source-hive/pom.xml  | 2 +-
 source-kafka/pom.xml | 2 +-
 storage-hbase/pom.xml| 2 +-
 tool/pom.xml | 2 +-
 21 files changed, 21 insertions(+), 21 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kylin/blob/71a847bb/assembly/pom.xml
--
diff --git a/assembly/pom.xml b/assembly/pom.xml
index 2aa8c87..204035b 100644
--- a/assembly/pom.xml
+++ b/assembly/pom.xml
@@ -21,7 +21,7 @@
 
 kylin
 org.apache.kylin
-1.5.2.1-SNAPSHOT
+1.5.2.1
 
 
 4.0.0

http://git-wip-us.apache.org/repos/asf/kylin/blob/71a847bb/atopcalcite/pom.xml
--
diff --git a/atopcalcite/pom.xml b/atopcalcite/pom.xml
index ec23344..7779201 100644
--- a/atopcalcite/pom.xml
+++ b/atopcalcite/pom.xml
@@ -27,7 +27,7 @@
 
 org.apache.kylin
 kylin
-1.5.2.1-SNAPSHOT
+1.5.2.1
 
 
 

http://git-wip-us.apache.org/repos/asf/kylin/blob/71a847bb/core-common/pom.xml
--
diff --git a/core-common/pom.xml b/core-common/pom.xml
index 381192a..5030d5a 100644
--- a/core-common/pom.xml
+++ b/core-common/pom.xml
@@ -27,7 +27,7 @@
 
 org.apache.kylin
 kylin
-1.5.2.1-SNAPSHOT
+1.5.2.1
 
 
 

http://git-wip-us.apache.org/repos/asf/kylin/blob/71a847bb/core-cube/pom.xml
--
diff --git a/core-cube/pom.xml b/core-cube/pom.xml
index 0fa5ffe..f4e0c71 100644
--- a/core-cube/pom.xml
+++ b/core-cube/pom.xml
@@ -27,7 +27,7 @@
 
 org.apache.kylin
 kylin
-1.5.2.1-SNAPSHOT
+1.5.2.1
 
 
 

http://git-wip-us.apache.org/repos/asf/kylin/blob/71a847bb/core-dictionary/pom.xml
--
diff --git a/core-dictionary/pom.xml b/core-dictionary/pom.xml
index 56d70e7..a6cd19c 100644
--- a/core-dictionary/pom.xml
+++ b/core-dictionary/pom.xml
@@ -27,7 +27,7 @@
 
 org.apache.kylin
 kylin
-1.5.2.1-SNAPSHOT
+1.5.2.1
 
 
 

http://git-wip-us.apache.org/repos/asf/kylin/blob/71a847bb/core-job/pom.xml
--
diff --git a/core-job/pom.xml b/core-job/pom.xml
index 8a0ee7d..383303c 100644
--- a/core-job/pom.xml
+++ b/core-job/pom.xml
@@ -27,7 +27,7 @@
 
 org.apache.kylin
 kylin
-1.5.2.1-SNAPSHOT
+1.5.2.1
 
 
 

http://git-wip-us.apache.org/repos/asf/kylin/blob/71a847bb/core-metadata/pom.xml
--
diff --git a/core-metadata/pom.xml b/core-metadata/pom.xml
index 76d709c..184dc5a 100644
--- a/core-metadata/pom.xml
+++ b/core-metadata/pom.xml
@@ -27,7 +27,7 @@
 
 org.apache.kylin
 kylin
-1.5.2.1-SNAPSHOT
+1.5.2.1
 
 
 

http://git-wip-us.apache.org/repos/asf/kylin/blob/71a847bb/core-storage/pom.xml
--
diff --git a/core-storage/pom.xml b/core-storage/pom.xml
index b9daa31..4948e59 100644
--- a/core-storage/pom.xml
+++ b/core-storage/pom.xml
@@ -27,7 +27,7 @@
 
 org.apache.kylin
 kylin
-1.5.2.1-SNAPSHOT
+1.5.2.1
 
 
 

http://git-wip-us.apache.org/repos/asf/kylin/blob/71a847bb/engine-mr/pom.xml
--
diff --git a/engine-mr/pom.xml b/engine-mr/pom.xml
index 2d97310..dcd3e2f 100644
--- a/engine-mr/pom.xml

[8/8] kylin git commit: KYLIN-1528 Create a branch for v1.5 with HBase 1.x API

2016-06-07 Thread shaofengshi 
KYLIN-1528 Create a branch for v1.5 with HBase 1.x API


Project: http://git-wip-us.apache.org/repos/asf/kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/cf4d2940
Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/cf4d2940
Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/cf4d2940

Branch: refs/heads/1.5.x-HBase1.x
Commit: cf4d2940b67d622eacd2ac9a913b221091a35c2e
Parents: 71a847b
Author: shaofengshi 
Authored: Wed Mar 23 17:07:05 2016 +0800
Committer: shaofengshi 
Committed: Tue Jun 7 16:29:01 2016 +0800

--
 .../kylin/job/hadoop/invertedindex/IITest.java  | 10 ++-
 .../apache/kylin/common/KylinConfigBase.java|  5 +-
 examples/test_case_data/sandbox/hbase-site.xml  | 19 +---
 .../test_case_data/sandbox/kylin_job_conf.xml   | 86 +-
 examples/test_case_data/sandbox/mapred-site.xml | 23 +++--
 .../kylin/provision/BuildCubeWithEngine.java| 54 +--
 .../kylin/provision/BuildIIWithStream.java  | 13 +--
 .../hbase/ii/ITInvertedIndexHBaseTest.java  |  9 +-
 pom.xml | 18 ++--
 .../kylin/rest/security/AclHBaseStorage.java|  4 +-
 .../rest/security/MockAclHBaseStorage.java  |  8 +-
 .../apache/kylin/rest/security/MockHTable.java  | 95 
 .../rest/security/RealAclHBaseStorage.java  |  9 +-
 .../apache/kylin/rest/service/AclService.java   | 25 +++---
 .../apache/kylin/rest/service/CubeService.java  | 38 +++-
 .../apache/kylin/rest/service/QueryService.java | 21 +++--
 .../apache/kylin/rest/service/UserService.java  | 16 ++--
 .../kylin/storage/hbase/HBaseConnection.java| 36 
 .../kylin/storage/hbase/HBaseResourceStore.java | 29 +++---
 .../storage/hbase/cube/SimpleHBaseStore.java| 20 ++---
 .../hbase/cube/v1/CubeSegmentTupleIterator.java | 11 +--
 .../storage/hbase/cube/v1/CubeStorageQuery.java |  4 +-
 .../hbase/cube/v1/HBaseClientKVIterator.java| 11 +--
 .../hbase/cube/v1/RegionScannerAdapter.java | 11 ++-
 .../cube/v1/SerializedHBaseTupleIterator.java   |  4 +-
 .../observer/AggregateRegionObserver.java   |  4 +-
 .../observer/AggregationScanner.java| 15 +++-
 .../observer/ObserverAggregationCache.java  | 15 ++--
 .../coprocessor/observer/ObserverEnabler.java   |  4 +-
 .../hbase/cube/v2/CubeHBaseEndpointRPC.java | 11 +--
 .../storage/hbase/cube/v2/CubeHBaseScanRPC.java |  9 +-
 .../coprocessor/endpoint/CubeVisitService.java  |  4 +-
 .../storage/hbase/ii/IICreateHTableJob.java | 13 +--
 .../hbase/ii/InvertedIndexStorageQuery.java |  6 +-
 .../endpoint/EndpointTupleIterator.java | 13 +--
 .../ii/coprocessor/endpoint/IIEndpoint.java |  4 +-
 .../storage/hbase/steps/CubeHTableUtil.java | 16 ++--
 .../storage/hbase/steps/DeprecatedGCStep.java   | 24 ++---
 .../storage/hbase/steps/HBaseCuboidWriter.java  |  7 +-
 .../hbase/steps/HBaseStreamingOutput.java   |  9 +-
 .../kylin/storage/hbase/steps/MergeGCStep.java  | 24 ++---
 .../storage/hbase/util/CleanHtableCLI.java  | 13 +--
 .../storage/hbase/util/CubeMigrationCLI.java| 38 
 .../hbase/util/CubeMigrationCheckCLI.java   | 17 ++--
 .../hbase/util/DeployCoprocessorCLI.java| 23 ++---
 .../hbase/util/ExtendCubeToHybridCLI.java   |  8 +-
 .../hbase/util/GridTableHBaseBenchmark.java | 34 +++
 .../kylin/storage/hbase/util/HBaseClean.java| 18 ++--
 .../hbase/util/HBaseRegionSizeCalculator.java   | 42 -
 .../kylin/storage/hbase/util/HBaseUsage.java| 10 ++-
 .../storage/hbase/util/HbaseStreamingInput.java | 31 +++
 .../hbase/util/HtableAlterMetadataCLI.java  |  9 +-
 .../storage/hbase/util/OrphanHBaseCleanJob.java | 18 ++--
 .../kylin/storage/hbase/util/PingHBaseCLI.java  | 15 ++--
 .../kylin/storage/hbase/util/RowCounterCLI.java | 11 +--
 .../storage/hbase/util/StorageCleanupJob.java   | 16 ++--
 .../storage/hbase/util/UpdateHTableHostCLI.java | 17 ++--
 .../observer/AggregateRegionObserverTest.java   | 31 +++
 .../v1/filter/TestFuzzyRowFilterV2EndToEnd.java |  5 +-
 59 files changed, 530 insertions(+), 583 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kylin/blob/cf4d2940/assembly/src/test/java/org/apache/kylin/job/hadoop/invertedindex/IITest.java
--
diff --git 
a/assembly/src/test/java/org/apache/kylin/job/hadoop/invertedindex/IITest.java 
b/assembly/src/test/java/org/apache/kylin/job/hadoop/invertedindex/IITest.java
index da25143..5d2cfc4 100644
--- 
a/assembly/src/test/java/org/apache/kylin/job/hadoop/invertedindex/IITest.java
+++ 
b/assembly/src/test/java/org/apache/kylin/job/hadoop/invertedindex/IITest.java
@@ -38,6 +38,7 @@ import org.apache.commons.lang.NotImplementedException;
 import org.apache.hadoop.hbase.Cell;
 import

[1/8] kylin git commit: KYLIN-1739 Do not cache and reuse hadoop configuration in HadoopUtil [Forced Update!]

2016-06-07 Thread shaofengshi 
Repository: kylin
Updated Branches:
  refs/heads/1.5.x-HBase1.x e85b49823 -> cf4d2940b (forced update)


KYLIN-1739 Do not cache and reuse hadoop configuration in HadoopUtil


Project: http://git-wip-us.apache.org/repos/asf/kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/0860934f
Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/0860934f
Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/0860934f

Branch: refs/heads/1.5.x-HBase1.x
Commit: 0860934f8bf7e30256d2c59c862c63d6b77d87ed
Parents: af2646b
Author: Li Yang 
Authored: Thu May 26 14:00:01 2016 +0800
Committer: shaofengshi 
Committed: Fri Jun 3 18:30:47 2016 +0800

--
 .../src/main/java/org/apache/kylin/engine/mr/HadoopUtil.java  | 3 +--
 .../apache/kylin/storage/hbase/cube/v2/CubeHBaseEndpointRPC.java  | 2 +-
 2 files changed, 2 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kylin/blob/0860934f/engine-mr/src/main/java/org/apache/kylin/engine/mr/HadoopUtil.java
--
diff --git a/engine-mr/src/main/java/org/apache/kylin/engine/mr/HadoopUtil.java 
b/engine-mr/src/main/java/org/apache/kylin/engine/mr/HadoopUtil.java
index 9ce2bab..c638d99 100644
--- a/engine-mr/src/main/java/org/apache/kylin/engine/mr/HadoopUtil.java
+++ b/engine-mr/src/main/java/org/apache/kylin/engine/mr/HadoopUtil.java
@@ -41,8 +41,7 @@ public class HadoopUtil {
 public static Configuration getCurrentConfiguration() {
 if (hadoopConfig.get() == null) {
 Configuration conf = healSickConfig(new Configuration());
-
-hadoopConfig.set(conf);
+return conf;
 }
 return hadoopConfig.get();
 }

http://git-wip-us.apache.org/repos/asf/kylin/blob/0860934f/storage-hbase/src/main/java/org/apache/kylin/storage/hbase/cube/v2/CubeHBaseEndpointRPC.java
--
diff --git 
a/storage-hbase/src/main/java/org/apache/kylin/storage/hbase/cube/v2/CubeHBaseEndpointRPC.java
 
b/storage-hbase/src/main/java/org/apache/kylin/storage/hbase/cube/v2/CubeHBaseEndpointRPC.java
index 12b1fa8..f04d46e 100644
--- 
a/storage-hbase/src/main/java/org/apache/kylin/storage/hbase/cube/v2/CubeHBaseEndpointRPC.java
+++ 
b/storage-hbase/src/main/java/org/apache/kylin/storage/hbase/cube/v2/CubeHBaseEndpointRPC.java
@@ -90,7 +90,7 @@ public class CubeHBaseEndpointRPC extends CubeHBaseRPC {
 this.expectedSize = expectedSize;
 this.queue = new ArrayBlockingQueue(expectedSize);
 
-this.timeout = 
HadoopUtil.getCurrentConfiguration().getInt(HConstants.HBASE_RPC_TIMEOUT_KEY, 
HConstants.DEFAULT_HBASE_RPC_TIMEOUT);
+this.timeout = 
HBaseConnection.getCurrentHBaseConfiguration().getInt(HConstants.HBASE_RPC_TIMEOUT_KEY,
 HConstants.DEFAULT_HBASE_RPC_TIMEOUT);
 this.timeout *= 
KylinConfig.getInstanceFromEnv().getCubeVisitTimeoutTimes();
 
 if (BackdoorToggles.getQueryTimeout() != -1) {

[4/8] kylin git commit: KYLIN-1758 add the hive set statements

2016-06-07 Thread shaofengshi 
KYLIN-1758 add the hive set statements


Project: http://git-wip-us.apache.org/repos/asf/kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/a1d17e21
Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/a1d17e21
Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/a1d17e21

Branch: refs/heads/1.5.x-HBase1.x
Commit: a1d17e215561089bb5958b84e876a74fc60f092f
Parents: ef1d13f
Author: shaofengshi 
Authored: Sat Jun 4 16:22:38 2016 +0800
Committer: shaofengshi 
Committed: Sat Jun 4 16:22:38 2016 +0800

--
 .../org/apache/kylin/job/JoinedFlatTable.java   | 56 +++-
 .../apache/kylin/source/hive/HiveMRInput.java   | 14 +
 2 files changed, 44 insertions(+), 26 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kylin/blob/a1d17e21/core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java
--
diff --git a/core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java 
b/core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java
index 5886325..f13a48c 100644
--- a/core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java
+++ b/core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java
@@ -55,6 +55,36 @@ public class JoinedFlatTable {
 return storageDfsDir + "/" + intermediateTableDesc.getTableName();
 }
 
+public static String generateHiveSetStatements(JobEngineConfig 
engineConfig) throws IOException {
+StringBuilder buffer = new StringBuilder();
+File hadoopPropertiesFile = new 
File(engineConfig.getHiveConfFilePath());
+
+if (hadoopPropertiesFile.exists()) {
+DocumentBuilderFactory factory = 
DocumentBuilderFactory.newInstance();
+DocumentBuilder builder;
+Document doc;
+try {
+builder = factory.newDocumentBuilder();
+doc = builder.parse(hadoopPropertiesFile);
+NodeList nl = doc.getElementsByTagName("property");
+for (int i = 0; i < nl.getLength(); i++) {
+String name = 
doc.getElementsByTagName("name").item(i).getFirstChild().getNodeValue();
+String value = 
doc.getElementsByTagName("value").item(i).getFirstChild().getNodeValue();
+if (!name.equals("tmpjars")) {
+buffer.append("SET " + name + "=" + value + ";\n");
+}
+}
+
+} catch (ParserConfigurationException e) {
+throw new IOException(e);
+} catch (SAXException e) {
+throw new IOException(e);
+}
+}
+
+return buffer.toString();
+}
+
 public static String generateCreateTableStatement(IJoinedFlatTableDesc 
intermediateTableDesc, String storageDfsDir) {
 StringBuilder ddl = new StringBuilder();
 
@@ -86,32 +116,6 @@ public class JoinedFlatTable {
 
 public static String generateInsertDataStatement(IJoinedFlatTableDesc 
intermediateTableDesc, JobEngineConfig engineConfig) throws IOException {
 StringBuilder sql = new StringBuilder();
-
-File hadoopPropertiesFile = new 
File(engineConfig.getHiveConfFilePath());
-
-if (hadoopPropertiesFile.exists()) {
-DocumentBuilderFactory factory = 
DocumentBuilderFactory.newInstance();
-DocumentBuilder builder;
-Document doc;
-try {
-builder = factory.newDocumentBuilder();
-doc = builder.parse(hadoopPropertiesFile);
-NodeList nl = doc.getElementsByTagName("property");
-for (int i = 0; i < nl.getLength(); i++) {
-String name = 
doc.getElementsByTagName("name").item(i).getFirstChild().getNodeValue();
-String value = 
doc.getElementsByTagName("value").item(i).getFirstChild().getNodeValue();
-if (name.equals("tmpjars") == false) {
-sql.append("SET " + name + "=" + value + 
";").append("\n");
-}
-}
-
-} catch (ParserConfigurationException e) {
-throw new IOException(e);
-} catch (SAXException e) {
-throw new IOException(e);
-}
-}
-
 sql.append("INSERT OVERWRITE TABLE " + 
intermediateTableDesc.getTableName() + " " + 
generateSelectDataStatement(intermediateTableDesc) + ";").append("\n");
 
 return sql.toString();

http://git-wip-us.apache.org/repos/asf/kylin/blob/a1d17e21/source-hive/src/main/java/org/apache/kylin/source/hive/HiveMRInput.java
--
diff --git 
a/source-hive/src/main/java/org/apache/kylin/source/hive/HiveMRInput.java

[3/8] kylin git commit: update version number

2016-06-07 Thread shaofengshi 
update version number


Project: http://git-wip-us.apache.org/repos/asf/kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/ef1d13f8
Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/ef1d13f8
Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/ef1d13f8

Branch: refs/heads/1.5.x-HBase1.x
Commit: ef1d13f8482b755b59c077c5b1ae1898e2e05e66
Parents: 6688d64
Author: shaofengshi 
Authored: Sat Jun 4 08:24:38 2016 +0800
Committer: shaofengshi 
Committed: Sat Jun 4 08:24:38 2016 +0800

--
 assembly/pom.xml | 2 +-
 atopcalcite/pom.xml  | 2 +-
 core-common/pom.xml  | 2 +-
 core-cube/pom.xml| 2 +-
 core-dictionary/pom.xml  | 2 +-
 core-job/pom.xml | 2 +-
 core-metadata/pom.xml| 2 +-
 core-storage/pom.xml | 2 +-
 engine-mr/pom.xml| 2 +-
 engine-spark/pom.xml | 2 +-
 engine-streaming/pom.xml | 2 +-
 invertedindex/pom.xml| 2 +-
 jdbc/pom.xml | 2 +-
 kylin-it/pom.xml | 2 +-
 pom.xml  | 4 ++--
 query/pom.xml| 2 +-
 server/pom.xml   | 2 +-
 source-hive/pom.xml  | 2 +-
 source-kafka/pom.xml | 2 +-
 storage-hbase/pom.xml| 2 +-
 tool/pom.xml | 2 +-
 21 files changed, 22 insertions(+), 22 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kylin/blob/ef1d13f8/assembly/pom.xml
--
diff --git a/assembly/pom.xml b/assembly/pom.xml
index b6e3675..204035b 100644
--- a/assembly/pom.xml
+++ b/assembly/pom.xml
@@ -21,7 +21,7 @@
 
 kylin
 org.apache.kylin
-1.5.2
+1.5.2.1
 
 
 4.0.0

http://git-wip-us.apache.org/repos/asf/kylin/blob/ef1d13f8/atopcalcite/pom.xml
--
diff --git a/atopcalcite/pom.xml b/atopcalcite/pom.xml
index 7c70df3..7779201 100644
--- a/atopcalcite/pom.xml
+++ b/atopcalcite/pom.xml
@@ -27,7 +27,7 @@
 
 org.apache.kylin
 kylin
-1.5.2
+1.5.2.1
 
 
 

http://git-wip-us.apache.org/repos/asf/kylin/blob/ef1d13f8/core-common/pom.xml
--
diff --git a/core-common/pom.xml b/core-common/pom.xml
index 7647621..5030d5a 100644
--- a/core-common/pom.xml
+++ b/core-common/pom.xml
@@ -27,7 +27,7 @@
 
 org.apache.kylin
 kylin
-1.5.2
+1.5.2.1
 
 
 

http://git-wip-us.apache.org/repos/asf/kylin/blob/ef1d13f8/core-cube/pom.xml
--
diff --git a/core-cube/pom.xml b/core-cube/pom.xml
index 2309872..f4e0c71 100644
--- a/core-cube/pom.xml
+++ b/core-cube/pom.xml
@@ -27,7 +27,7 @@
 
 org.apache.kylin
 kylin
-1.5.2
+1.5.2.1
 
 
 

http://git-wip-us.apache.org/repos/asf/kylin/blob/ef1d13f8/core-dictionary/pom.xml
--
diff --git a/core-dictionary/pom.xml b/core-dictionary/pom.xml
index 1af0091..a6cd19c 100644
--- a/core-dictionary/pom.xml
+++ b/core-dictionary/pom.xml
@@ -27,7 +27,7 @@
 
 org.apache.kylin
 kylin
-1.5.2
+1.5.2.1
 
 
 

http://git-wip-us.apache.org/repos/asf/kylin/blob/ef1d13f8/core-job/pom.xml
--
diff --git a/core-job/pom.xml b/core-job/pom.xml
index 864bf1b..383303c 100644
--- a/core-job/pom.xml
+++ b/core-job/pom.xml
@@ -27,7 +27,7 @@
 
 org.apache.kylin
 kylin
-1.5.2
+1.5.2.1
 
 
 

http://git-wip-us.apache.org/repos/asf/kylin/blob/ef1d13f8/core-metadata/pom.xml
--
diff --git a/core-metadata/pom.xml b/core-metadata/pom.xml
index f65129d..184dc5a 100644
--- a/core-metadata/pom.xml
+++ b/core-metadata/pom.xml
@@ -27,7 +27,7 @@
 
 org.apache.kylin
 kylin
-1.5.2
+1.5.2.1
 
 
 

http://git-wip-us.apache.org/repos/asf/kylin/blob/ef1d13f8/core-storage/pom.xml
--
diff --git a/core-storage/pom.xml b/core-storage/pom.xml
index 2f08dd9..4948e59 100644
--- a/core-storage/pom.xml
+++ b/core-storage/pom.xml
@@ -27,7 +27,7 @@
 
 org.apache.kylin
 kylin
-1.5.2
+1.5.2.1
 
 
 

http://git-wip-us.apache.org/repos/asf/kylin/blob/ef1d13f8/engine-mr/pom.xml
--
diff --git a/engine-mr/pom.xml b/engine-mr/pom.xml
index 379eaa6..dcd3e2f 100644
--- a/engine-mr/pom.xml
+++ b/engine-mr/pom.xml
@@ -27,7 +27,7 @@
 
 org.apache.kylin
 kylin
-1.5.2
+

kylin git commit: KYLIN-1752 Add an option to fail cube build job when source table is empty

2016-06-07 Thread shaofengshi 
Repository: kylin
Updated Branches:
  refs/heads/master 9f1029a76 -> d2e96bda0


KYLIN-1752 Add an option to fail cube build job when source table is empty

Signed-off-by: shaofengshi 


Project: http://git-wip-us.apache.org/repos/asf/kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/d2e96bda
Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/d2e96bda
Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/d2e96bda

Branch: refs/heads/master
Commit: d2e96bda0a5848815cf4430746276bc9e896bad7
Parents: 9f1029a
Author: gaodayue 
Authored: Fri Jun 3 18:17:43 2016 +0800
Committer: shaofengshi 
Committed: Tue Jun 7 15:09:29 2016 +0800

--
 .../apache/kylin/common/KylinConfigBase.java|  4 
 .../source/hive/CreateFlatHiveTableStep.java| 22 +---
 2 files changed, 18 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kylin/blob/d2e96bda/core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java
--
diff --git 
a/core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java 
b/core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java
index 68e3b6c..7664c66 100644
--- a/core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java
+++ b/core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java
@@ -363,6 +363,10 @@ abstract public class KylinConfigBase implements 
Serializable {
 return getOptional("kylin.job.cmd.extra.args");
 }
 
+public boolean isEmptySegmentAllowed() {
+return 
Boolean.parseBoolean(getOptional("kylin.job.allow.empty.segment", "true"));
+}
+
 public String getOverrideHiveTableLocation(String table) {
 return getOptional("hive.table.location." + table.toUpperCase());
 }

http://git-wip-us.apache.org/repos/asf/kylin/blob/d2e96bda/source-hive/src/main/java/org/apache/kylin/source/hive/CreateFlatHiveTableStep.java
--
diff --git 
a/source-hive/src/main/java/org/apache/kylin/source/hive/CreateFlatHiveTableStep.java
 
b/source-hive/src/main/java/org/apache/kylin/source/hive/CreateFlatHiveTableStep.java
index e9b9994..443de99 100644
--- 
a/source-hive/src/main/java/org/apache/kylin/source/hive/CreateFlatHiveTableStep.java
+++ 
b/source-hive/src/main/java/org/apache/kylin/source/hive/CreateFlatHiveTableStep.java
@@ -23,9 +23,11 @@ import java.io.InputStream;
 public class CreateFlatHiveTableStep extends AbstractExecutable {
 private final BufferedLogger stepLogger = new BufferedLogger(logger);
 
-private long readRowCountFromFile(Path file) throws IOException {
-FileSystem fs = FileSystem.get(file.toUri(), 
HadoopUtil.getCurrentConfiguration());
-InputStream in = fs.open(file);
+private long readRowCountFromFile() throws IOException {
+Path rowCountFile = new Path(getRowCountOutputDir(), "00_0");
+
+FileSystem fs = FileSystem.get(rowCountFile.toUri(), 
HadoopUtil.getCurrentConfiguration());
+InputStream in = fs.open(rowCountFile);
 try {
 String content = IOUtils.toString(in);
 return Long.valueOf(content.trim()); // strip the '\n' character
@@ -35,9 +37,7 @@ public class CreateFlatHiveTableStep extends 
AbstractExecutable {
 }
 }
 
-private int determineNumReducer(KylinConfig config) throws IOException {
-Path rowCountFile = new Path(getRowCountOutputDir(), "00_0");
-long rowCount = readRowCountFromFile(rowCountFile);
+private int determineNumReducer(KylinConfig config, long rowCount) throws 
IOException {
 int mapperInputRows = config.getHadoopJobMapperInputRows();
 
 int numReducers = Math.round(rowCount / ((float) mapperInputRows));
@@ -78,8 +78,14 @@ public class CreateFlatHiveTableStep extends 
AbstractExecutable {
 protected ExecuteResult doWork(ExecutableContext context) throws 
ExecuteException {
 KylinConfig config = getCubeSpecificConfig();
 try {
-
-int numReducers = determineNumReducer(config);
+long rowCount = readRowCountFromFile();
+if (!config.isEmptySegmentAllowed() && rowCount == 0) {
+stepLogger.log("Detect upstream hive table is empty, " +
+"fail the job because 
\"kylin.job.allow.empty.segment\" = \"false\"");
+return new ExecuteResult(ExecuteResult.State.ERROR, 
stepLogger.getBufferedLog());
+}
+
+int numReducers = determineNumReducer(config, rowCount);
 createFlatHiveTable(config, numReducers);
 return new ExecuteResult(ExecuteResult.State.SUCCEED, 
stepLogger.getBufferedLog());

kylin git commit: KYLIN-1769 add override property not works in CubeDesigner

2016-06-07 Thread zhongjian 
Repository: kylin
Updated Branches:
  refs/heads/master e924fc439 -> 9f1029a76


KYLIN-1769 add override property not works in CubeDesigner


Project: http://git-wip-us.apache.org/repos/asf/kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/9f1029a7
Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/9f1029a7
Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/9f1029a7

Branch: refs/heads/master
Commit: 9f1029a76f053d2fc83a1623db125fce648662f6
Parents: e924fc4
Author: Jason 
Authored: Tue Jun 7 14:56:24 2016 +0800
Committer: Jason 
Committed: Tue Jun 7 14:56:24 2016 +0800

--
 webapp/app/js/model/cubeDescModel.js | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/kylin/blob/9f1029a7/webapp/app/js/model/cubeDescModel.js
--
diff --git a/webapp/app/js/model/cubeDescModel.js 
b/webapp/app/js/model/cubeDescModel.js
index 6f25326..a6e02ac 100644
--- a/webapp/app/js/model/cubeDescModel.js
+++ b/webapp/app/js/model/cubeDescModel.js
@@ -56,7 +56,8 @@ KylinApp.service('CubeDescModel', function () {
   "status_need_notify":['ERROR', 'DISCARDED', 'SUCCEED'],
   "auto_merge_time_ranges": [60480, 241920],
   "engine_type": 2,
-  "storage_type":2
+  "storage_type":2,
+  "override_kylin_properties":{}
 };
 
 return cubeMeta;