[GitHub] coveralls commented on issue #195: KYLIN-3446 Connect to HBase out of Spark

2018-08-14 Thread GitBox 
coveralls commented on issue #195: KYLIN-3446 Connect to HBase out of Spark
URL: https://github.com/apache/kylin/pull/195#issuecomment-413080321
 
 
   ## Pull Request Test Coverage Report for [Build 
3440](https://coveralls.io/builds/18490029)
   
   * **0** of **61**   **(0.0%)**  changed or added relevant lines in **4** 
files are covered.
   * **12** unchanged lines in **6** files lost coverage.
   * Overall coverage decreased (**-0.01%**) to **23.012%**
   
   ---
   
   |  Changes Missing Coverage | Covered Lines | Changed/Added Lines | % |
   | :-|--||---: |
   | 
[storage-hbase/src/main/java/org/apache/kylin/storage/hbase/steps/HBaseSparkSteps.java](https://coveralls.io/builds/18490029/source?filename=storage-hbase%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstorage%2Fhbase%2Fsteps%2FHBaseSparkSteps.java#L51)
 | 0 | 1 | 0.0%
   | 
[storage-hbase/src/main/java/org/apache/kylin/storage/hbase/steps/SparkCubeHFile.java](https://coveralls.io/builds/18490029/source?filename=storage-hbase%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstorage%2Fhbase%2Fsteps%2FSparkCubeHFile.java#L91)
 | 0 | 10 | 0.0%
   | 
[storage-hbase/src/main/java/org/apache/kylin/storage/hbase/steps/HBaseJobSteps.java](https://coveralls.io/builds/18490029/source?filename=storage-hbase%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstorage%2Fhbase%2Fsteps%2FHBaseJobSteps.java#L62)
 | 0 | 15 | 0.0%
   | 
[storage-hbase/src/main/java/org/apache/kylin/storage/hbase/steps/CreateHTableJob.java](https://coveralls.io/builds/18490029/source?filename=storage-hbase%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstorage%2Fhbase%2Fsteps%2FCreateHTableJob.java#L71)
 | 0 | 35 | 0.0%
   
   
   |  Files with Coverage Reduction | New Missed Lines | % |
   | :-|--|--: |
   | 
[storage-hbase/src/main/java/org/apache/kylin/storage/hbase/steps/HBaseJobSteps.java](https://coveralls.io/builds/18490029/source?filename=storage-hbase%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstorage%2Fhbase%2Fsteps%2FHBaseJobSteps.java#L64)
 | 1 | 0.0% |
   | 
[core-metadata/src/main/java/org/apache/kylin/source/datagen/ColumnGenerator.java](https://coveralls.io/builds/18490029/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fdatagen%2FColumnGenerator.java#L317)
 | 1 | 81.08% |
   | 
[storage-hbase/src/main/java/org/apache/kylin/storage/hbase/steps/SparkCubeHFile.java](https://coveralls.io/builds/18490029/source?filename=storage-hbase%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstorage%2Fhbase%2Fsteps%2FSparkCubeHFile.java#L105)
 | 1 | 4.63% |
   | 
[core-metadata/src/main/java/org/apache/kylin/metadata/cachesync/Broadcaster.java](https://coveralls.io/builds/18490029/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetadata%2Fcachesync%2FBroadcaster.java#L149)
 | 1 | 57.71% |
   | 
[core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/18490029/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L555)
 | 4 | 77.81% |
   | 
[storage-hbase/src/main/java/org/apache/kylin/storage/hbase/steps/CreateHTableJob.java](https://coveralls.io/builds/18490029/source?filename=storage-hbase%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstorage%2Fhbase%2Fsteps%2FCreateHTableJob.java#L84)
 | 4 | 0.0% |
   
   
   |  Totals | [![Coverage 
Status](https://coveralls.io/builds/18490029/badge)](https://coveralls.io/builds/18490029)
 |
   | :-- | --: |
   | Change from base [Build 3439](https://coveralls.io/builds/18473770): |  
-0.01% |
   | Covered Lines: | 15697 |
   | Relevant Lines: | 68211 |
   
   ---
   #   - [Coveralls](https://coveralls.io)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #195: KYLIN-3446 Connect to HBase out of Spark

2018-08-14 Thread GitBox 
codecov-io edited a comment on issue #195: KYLIN-3446 Connect to HBase out of 
Spark
URL: https://github.com/apache/kylin/pull/195#issuecomment-412249918
 
 
   # [Codecov](https://codecov.io/gh/apache/kylin/pull/195?src=pr=h1) Report
   > Merging [#195](https://codecov.io/gh/apache/kylin/pull/195?src=pr=desc) 
into 
[master](https://codecov.io/gh/apache/kylin/commit/648379867850f4dba6187efe6fd22e2b57d5580e?src=pr=desc)
 will **decrease** coverage by `<.01%`.
   > The diff coverage is `0%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/kylin/pull/195/graphs/tree.svg?width=650=pr=JawVgbgsVo=150)](https://codecov.io/gh/apache/kylin/pull/195?src=pr=tree)
   
   ```diff
   @@ Coverage Diff  @@
   ## master #195  +/-   ##
   
   - Coverage21%   20.99%   -0.01% 
 Complexity 4296 4296  
   
 Files  1071 1071  
 Lines 6817568211  +36 
 Branches   9893 9893  
   
   + Hits  1432114322   +1 
   - Misses5248152516  +35 
 Partials   1373 1373
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/kylin/pull/195?src=pr=tree) | Coverage Δ 
| Complexity Δ | |
   |---|---|---|---|
   | 
[...che/kylin/storage/hbase/steps/CreateHTableJob.java](https://codecov.io/gh/apache/kylin/pull/195/diff?src=pr=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9zdGVwcy9DcmVhdGVIVGFibGVKb2IuamF2YQ==)
 | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: |
   | 
[...pache/kylin/storage/hbase/steps/HBaseJobSteps.java](https://codecov.io/gh/apache/kylin/pull/195/diff?src=pr=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9zdGVwcy9IQmFzZUpvYlN0ZXBzLmphdmE=)
 | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: |
   | 
[...che/kylin/storage/hbase/steps/HBaseSparkSteps.java](https://codecov.io/gh/apache/kylin/pull/195/diff?src=pr=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9zdGVwcy9IQmFzZVNwYXJrU3RlcHMuamF2YQ==)
 | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: |
   | 
[...ache/kylin/storage/hbase/steps/SparkCubeHFile.java](https://codecov.io/gh/apache/kylin/pull/195/diff?src=pr=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9zdGVwcy9TcGFya0N1YmVIRmlsZS5qYXZh)
 | `3.7% <0%> (-0.04%)` | `0 <0> (ø)` | |
   | 
[...g/apache/kylin/source/datagen/ColumnGenerator.java](https://codecov.io/gh/apache/kylin/pull/195/diff?src=pr=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc291cmNlL2RhdGFnZW4vQ29sdW1uR2VuZXJhdG9yLmphdmE=)
 | `70.94% <0%> (-1.36%)` | `8% <0%> (ø)` | |
   | 
[...g/apache/kylin/metadata/cachesync/Broadcaster.java](https://codecov.io/gh/apache/kylin/pull/195/diff?src=pr=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWV0YWRhdGEvY2FjaGVzeW5jL0Jyb2FkY2FzdGVyLmphdmE=)
 | `51.74% <0%> (-1%)` | `22% <0%> (ø)` | |
   | 
[...lin/dict/lookup/cache/RocksDBLookupTableCache.java](https://codecov.io/gh/apache/kylin/pull/195/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGVDYWNoZS5qYXZh)
 | `76.16% <0%> (-0.52%)` | `27% <0%> (ø)` | |
   | 
[...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/195/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh)
 | `69.3% <0%> (+1.82%)` | `7% <0%> (ø)` | :arrow_down: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/kylin/pull/195?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/kylin/pull/195?src=pr=footer). Last 
update 
[6483798...3147052](https://codecov.io/gh/apache/kylin/pull/195?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Re: Apache Kylin meetup on 8/11 @Beijing

2018-08-14 Thread Ashish Singhi 
Hi Shaofeng,

Is the presentation uploaded for reference anywhere ?

Regards,
Ashish

On Wed, Aug 8, 2018 at 11:10 AM, ShaoFeng Shi 
wrote:

> Hello, Kylin users,
>
> This Saturday (8/11) we will have a meetup event at Meituan.com, in Beijing
> China. Engineers from Meituan, XiaoMi(mi.com), Beike (ke.com, originally
> Lianjian), and Kyligence will share their topics:
>
> 1. Apache Kylin with Druid as storage in Meituan
> 2. Apache Kylin practices from Beike
> 3. Lambda OLAP architecture in XiaoMi
> 4. Apache Kylin smart modeling and optimization
>
> It is open for registration. But we got the message that the air
> conditioner might be unavailable in the weekend, so it might be a little
> warm...
>
> Here is the event link:
>
> *http://www.huodongxing.com/event/7452131278400
> *
>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>

[GitHub] momoxixi commented on issue #198: KYLIN-3499 Update HBaseResourceStore.java

2018-08-14 Thread GitBox 
momoxixi commented on issue #198: KYLIN-3499 Update HBaseResourceStore.java
URL: https://github.com/apache/kylin/pull/198#issuecomment-412849363
 
 
   @shaofengshi  OK, I have been add a JIRA, please check it, thank you.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (KYLIN-3499) resource inconsistent caused by hbase rpc timeout

2018-08-14 Thread ulysses you (JIRA) 
ulysses you created KYLIN-3499:
--

 Summary: resource inconsistent caused by hbase rpc timeout
 Key: KYLIN-3499
 URL: https://issues.apache.org/jira/browse/KYLIN-3499
 Project: Kylin
  Issue Type: Bug
  Components: Storage - HBase
Affects Versions: all
Reporter: ulysses you
 Fix For: Future
 Attachments: ca7a61b2fdbfc321e7c896c3d34fd4d4c48e3eed.patch, error.png

KYLIN use hbase to persist resouce metadata. When hbase rpc timeout, kylin 
would rollback the resource timestamp. It caused resource inconsistent between 
kylin's memory and hbase if hbase already put the resource.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] codecov-io commented on issue #199: KYLIN-3497 Make JDBC Module more testable

2018-08-14 Thread GitBox 
codecov-io commented on issue #199: KYLIN-3497 Make JDBC Module more testable
URL: https://github.com/apache/kylin/pull/199#issuecomment-412819042
 
 
   # [Codecov](https://codecov.io/gh/apache/kylin/pull/199?src=pr=h1) Report
   > :exclamation: No coverage uploaded for pull request base 
(`master@ddfed27`). [Click here to learn what that 
means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
   > The diff coverage is `76.92%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/kylin/pull/199/graphs/tree.svg?height=150=650=JawVgbgsVo=pr)](https://codecov.io/gh/apache/kylin/pull/199?src=pr=tree)
   
   ```diff
   @@Coverage Diff@@
   ## master #199   +/-   ##
   =
 Coverage  ?   21.45%   
 Complexity? 4357   
   =
 Files ? 1069   
 Lines ?67824   
 Branches  ? 9836   
   =
 Hits  ?14553   
 Misses?51889   
 Partials  ? 1382
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/kylin/pull/199?src=pr=tree) | Coverage Δ 
| Complexity Δ | |
   |---|---|---|---|
   | 
[...main/java/org/apache/kylin/jdbc/IRemoteClient.java](https://codecov.io/gh/apache/kylin/pull/199/diff?src=pr=tree#diff-amRiYy9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vamRiYy9JUmVtb3RlQ2xpZW50LmphdmE=)
 | `100% <ø> (ø)` | `0 <0> (?)` | |
   | 
[...n/java/org/apache/kylin/jdbc/KylinJdbcFactory.java](https://codecov.io/gh/apache/kylin/pull/199/diff?src=pr=tree#diff-amRiYy9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vamRiYy9LeWxpbkpkYmNGYWN0b3J5LmphdmE=)
 | `72.22% <ø> (ø)` | `7 <0> (?)` | |
   | 
[...ain/java/org/apache/kylin/jdbc/KylinResultSet.java](https://codecov.io/gh/apache/kylin/pull/199/diff?src=pr=tree#diff-amRiYy9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vamRiYy9LeWxpblJlc3VsdFNldC5qYXZh)
 | `86.95% <100%> (ø)` | `5 <0> (?)` | |
   | 
[...in/java/org/apache/kylin/jdbc/KylinConnection.java](https://codecov.io/gh/apache/kylin/pull/199/diff?src=pr=tree#diff-amRiYy9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vamRiYy9LeWxpbkNvbm5lY3Rpb24uamF2YQ==)
 | `73.91% <50%> (ø)` | `12 <2> (?)` | |
   | 
[...c/main/java/org/apache/kylin/jdbc/KylinClient.java](https://codecov.io/gh/apache/kylin/pull/199/diff?src=pr=tree#diff-amRiYy9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vamRiYy9LeWxpbkNsaWVudC5qYXZh)
 | `67.67% <80%> (ø)` | `33 <8> (?)` | |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/kylin/pull/199?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/kylin/pull/199?src=pr=footer). Last 
update 
[ddfed27...c23a8c0](https://codecov.io/gh/apache/kylin/pull/199?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] shaofengshi closed pull request #197: KYLIN-3317, replace UUID.randomUUID with new UUID(ThreadLocalRandom.current().nextLong(), ThreadLocalRandom.current().nextLong())

2018-08-14 Thread GitBox 
shaofengshi closed pull request #197: KYLIN-3317, replace UUID.randomUUID with 
new UUID(ThreadLocalRandom.current().nextLong(), 
ThreadLocalRandom.current().nextLong())
URL: https://github.com/apache/kylin/pull/197
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/core-common/src/main/java/org/apache/kylin/common/QueryContext.java 
b/core-common/src/main/java/org/apache/kylin/common/QueryContext.java
index 66b9871344..ef288c7d6d 100644
--- a/core-common/src/main/java/org/apache/kylin/common/QueryContext.java
+++ b/core-common/src/main/java/org/apache/kylin/common/QueryContext.java
@@ -22,11 +22,11 @@
 import java.util.List;
 import java.util.Map;
 import java.util.Set;
-import java.util.UUID;
 import java.util.concurrent.atomic.AtomicBoolean;
 import java.util.concurrent.atomic.AtomicLong;
 
 import org.apache.kylin.common.exceptions.KylinTimeoutException;
+import org.apache.kylin.common.util.RandomUtil;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -67,7 +67,7 @@
 }
 
 QueryContext(long startMills) {
-queryId = UUID.randomUUID().toString();
+queryId = RandomUtil.randomUUID().toString();
 queryStartMillis = startMills;
 }
 
diff --git 
a/core-common/src/main/java/org/apache/kylin/common/persistence/ResourceStore.java
 
b/core-common/src/main/java/org/apache/kylin/common/persistence/ResourceStore.java
index a71db459bd..36bb595489 100644
--- 
a/core-common/src/main/java/org/apache/kylin/common/persistence/ResourceStore.java
+++ 
b/core-common/src/main/java/org/apache/kylin/common/persistence/ResourceStore.java
@@ -32,7 +32,6 @@
 import java.util.LinkedHashMap;
 import java.util.List;
 import java.util.NavigableSet;
-import java.util.UUID;
 import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.ConcurrentMap;
 
@@ -42,6 +41,7 @@
 import org.apache.kylin.common.StorageURL;
 import org.apache.kylin.common.util.ClassUtil;
 import org.apache.kylin.common.util.OptionsHelper;
+import org.apache.kylin.common.util.RandomUtil;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -147,7 +147,7 @@ final public KylinConfig getConfig() {
 abstract protected NavigableSet listResourcesImpl(String 
folderPath, boolean recursive) throws IOException;
 
 protected String createMetaStoreUUID() throws IOException {
-return UUID.randomUUID().toString();
+return RandomUtil.randomUUID().toString();
 }
 
 public String getMetaStoreUUID() throws IOException {
diff --git 
a/core-common/src/main/java/org/apache/kylin/common/persistence/RootPersistentEntity.java
 
b/core-common/src/main/java/org/apache/kylin/common/persistence/RootPersistentEntity.java
index 39c29955ef..a4877f2e28 100644
--- 
a/core-common/src/main/java/org/apache/kylin/common/persistence/RootPersistentEntity.java
+++ 
b/core-common/src/main/java/org/apache/kylin/common/persistence/RootPersistentEntity.java
@@ -21,10 +21,10 @@
 import java.io.Serializable;
 import java.text.DateFormat;
 import java.text.SimpleDateFormat;
-import java.util.UUID;
 
 import org.apache.commons.lang.time.FastDateFormat;
 import org.apache.kylin.common.KylinVersion;
+import org.apache.kylin.common.util.RandomUtil;
 
 import com.fasterxml.jackson.annotation.JsonAutoDetect;
 import com.fasterxml.jackson.annotation.JsonAutoDetect.Visibility;
@@ -99,7 +99,7 @@ public void setLastModified(long lastModified) {
 }
 
 public void updateRandomUuid() {
-setUuid(UUID.randomUUID().toString());
+setUuid(RandomUtil.randomUUID().toString());
 }
 
 public boolean isCachedAndShared() {
diff --git 
a/core-common/src/main/java/org/apache/kylin/common/util/RandomUtil.java 
b/core-common/src/main/java/org/apache/kylin/common/util/RandomUtil.java
new file mode 100644
index 00..00be8dd4c3
--- /dev/null
+++ b/core-common/src/main/java/org/apache/kylin/common/util/RandomUtil.java
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package

[GitHub] shaofengshi commented on issue #198: Update HBaseResourceStore.java

2018-08-14 Thread GitBox 
shaofengshi commented on issue #198: Update HBaseResourceStore.java
URL: https://github.com/apache/kylin/pull/198#issuecomment-412816072
 
 
   @momoxixi hello, thanks for the reporting; Could you please create a JIRA 
for the problem, and then update the commit message as "KYLIN- 
". Thanks!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] shaofengshi closed pull request #178: KYLIN-3470 Add cache for execute and execute_output

2018-08-14 Thread GitBox 
shaofengshi closed pull request #178: KYLIN-3470 Add cache for execute and 
execute_output
URL: https://github.com/apache/kylin/pull/178
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/core-job/src/main/java/org/apache/kylin/job/JobSearchResult.java 
b/core-job/src/main/java/org/apache/kylin/job/JobSearchResult.java
new file mode 100644
index 00..bd4326dd3e
--- /dev/null
+++ b/core-job/src/main/java/org/apache/kylin/job/JobSearchResult.java
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+package org.apache.kylin.job;
+
+public class JobSearchResult implements Comparable {
+private String id;
+
+private String jobName;
+
+private String cubeName;
+
+private long lastModified;
+
+public String getId() {
+return id;
+}
+
+public void setId(String id) {
+this.id = id;
+}
+
+public String getJobName() {
+return jobName;
+}
+
+public void setJobName(String jobName) {
+this.jobName = jobName;
+}
+
+public String getCubeName() {
+return cubeName;
+}
+
+public void setCubeName(String cubeName) {
+this.cubeName = cubeName;
+}
+
+public long getLastModified() {
+return lastModified;
+}
+
+public void setLastModified(long lastModified) {
+this.lastModified = lastModified;
+}
+
+@Override
+public int compareTo(JobSearchResult o) {
+return o.lastModified < this.lastModified ? -1 : o.lastModified > 
this.lastModified ? 1 : 0;
+}
+}
diff --git a/core-job/src/main/java/org/apache/kylin/job/dao/ExecutableDao.java 
b/core-job/src/main/java/org/apache/kylin/job/dao/ExecutableDao.java
index 1b34aa1ca7..0cc6c8e5fb 100644
--- a/core-job/src/main/java/org/apache/kylin/job/dao/ExecutableDao.java
+++ b/core-job/src/main/java/org/apache/kylin/job/dao/ExecutableDao.java
@@ -28,7 +28,11 @@
 import org.apache.kylin.common.persistence.JsonSerializer;
 import org.apache.kylin.common.persistence.ResourceStore;
 import org.apache.kylin.common.persistence.Serializer;
+import org.apache.kylin.common.util.AutoReadWriteLock;
 import org.apache.kylin.job.exception.PersistentException;
+import org.apache.kylin.metadata.cachesync.Broadcaster;
+import org.apache.kylin.metadata.cachesync.CachedCrudAssist;
+import org.apache.kylin.metadata.cachesync.CaseInsensitiveStringCache;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -55,9 +59,140 @@ static ExecutableDao newInstance(KylinConfig config) throws 
IOException {
 
 private ResourceStore store;
 
-private ExecutableDao(KylinConfig config) {
+private CaseInsensitiveStringCache executableDigestMap;
+
+private CaseInsensitiveStringCache 
executableOutputDigestMap;
+
+private CachedCrudAssist executableDigestCrud;
+
+private CachedCrudAssist executableOutputDigestCrud;
+
+private AutoReadWriteLock executableDigestMapLock = new 
AutoReadWriteLock();
+
+private AutoReadWriteLock executableOutputDigestMapLock = new 
AutoReadWriteLock();
+
+private ExecutableDao(KylinConfig config) throws IOException {
 logger.info("Using metadata url: " + config);
 this.store = ResourceStore.getStore(config);
+this.executableDigestMap = new CaseInsensitiveStringCache<>(config, 
"execute");
+this.executableDigestCrud = new CachedCrudAssist(store, 
ResourceStore.EXECUTE_RESOURCE_ROOT, "",
+ExecutablePO.class, executableDigestMap, false) {
+@Override
+public ExecutablePO reloadAt(String path) {
+try {
+ExecutablePO executablePO = readJobResource(path);
+if (executablePO == null) {
+logger.warn("No job found at " + path + ", returning 
null");
+executableDigestMap.removeLocal(resourceName(path));
+return null;
+}
+
+// create a digest
+

[GitHub] shaofengshi closed pull request #160: KYLIN-3434 Support prepare statement in Kylin server side

2018-08-14 Thread GitBox 
shaofengshi closed pull request #160: KYLIN-3434 Support prepare statement in 
Kylin server side
URL: https://github.com/apache/kylin/pull/160
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java 
b/core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java
index 3ae6c2d48b..f4aad5e6bc 100644
--- a/core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java
+++ b/core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java
@@ -1377,6 +1377,14 @@ public String getQueryAccessController() {
 return getOptional("kylin.query.access-controller", null);
 }
 
+public int getQueryMaxCacheStatementNum() {
+return 
Integer.parseInt(this.getOptional("kylin.query.statement-cache-max-num", 
String.valueOf(5)));
+}
+
+public int getQueryMaxCacheStatementInstancePerKey() {
+return 
Integer.parseInt(this.getOptional("kylin.query.statement-cache-max-num-per-key",
 String.valueOf(50)));
+}
+
 public int getDimCountDistinctMaxCardinality() {
 return 
Integer.parseInt(getOptional("kylin.query.max-dimension-count-distinct", 
"500"));
 }
diff --git a/core-cube/src/main/java/org/apache/kylin/cube/CubeManager.java 
b/core-cube/src/main/java/org/apache/kylin/cube/CubeManager.java
index 3f4c576053..8d395af6a7 100755
--- a/core-cube/src/main/java/org/apache/kylin/cube/CubeManager.java
+++ b/core-cube/src/main/java/org/apache/kylin/cube/CubeManager.java
@@ -264,6 +264,7 @@ public CubeInstance updateCubeStatus(CubeInstance cube, 
RealizationStatusEnum ne
 cube = cube.latestCopyForWrite(); // get a latest copy
 CubeUpdate update = new CubeUpdate(cube);
 update.setStatus(newStatus);
+ProjectManager.getInstance(config).touchProject(cube.getProject());
 return updateCube(update);
 }
 }
diff --git 
a/core-metadata/src/main/java/org/apache/kylin/metadata/project/ProjectManager.java
 
b/core-metadata/src/main/java/org/apache/kylin/metadata/project/ProjectManager.java
index 5122fd8401..4f1de33c2b 100644
--- 
a/core-metadata/src/main/java/org/apache/kylin/metadata/project/ProjectManager.java
+++ 
b/core-metadata/src/main/java/org/apache/kylin/metadata/project/ProjectManager.java
@@ -355,6 +355,18 @@ public void removeExtFilterFromProject(String filterName, 
String projectName) th
 save(projectInstance);
 }
 }
+
+/**
+ * change the last project modify time
+ * @param projectName
+ * @throws IOException
+ */
+public void touchProject(String projectName) throws IOException {
+try (AutoLock lock = prjMapLock.lockForWrite()) {
+ProjectInstance projectInstance = getProject(projectName);
+save(projectInstance);
+}
+}
 
 private ProjectInstance save(ProjectInstance prj) throws IOException {
 crud.save(prj);
diff --git a/pom.xml b/pom.xml
index 854fd3b39e..1888a7f5e7 100644
--- a/pom.xml
+++ b/pom.xml
@@ -80,6 +80,7 @@
 1.3.3
 3.1.1
 3.2.2
+2.5.0
 
 
 2.9.5
@@ -507,6 +508,11 @@
 ${commons-collections.version}
 provided
 
+
+org.apache.commons
+commons-pool2
+${commons-pool.version}
+
 
 
 
diff --git a/server-base/pom.xml b/server-base/pom.xml
index 455ae786f4..baa6433f7e 100644
--- a/server-base/pom.xml
+++ b/server-base/pom.xml
@@ -85,6 +85,10 @@
 net.sf.supercsv
 super-csv
 
+
+org.apache.commons
+commons-pool2
+
 
 
 org.apache.kylin
diff --git 
a/server-base/src/main/java/org/apache/kylin/rest/request/PrepareSqlRequest.java
 
b/server-base/src/main/java/org/apache/kylin/rest/request/PrepareSqlRequest.java
index 97a486326a..48e382ae31 100644
--- 
a/server-base/src/main/java/org/apache/kylin/rest/request/PrepareSqlRequest.java
+++ 
b/server-base/src/main/java/org/apache/kylin/rest/request/PrepareSqlRequest.java
@@ -28,6 +28,7 @@
  * 
  */
 public class PrepareSqlRequest extends SQLRequest {
+private boolean enableStatementCache = true;
 
 public PrepareSqlRequest() {
 super();
@@ -43,6 +44,14 @@ public void setParams(StateParam[] params) {
 this.params = params;
 }
 
+public boolean isEnableStatementCache() {
+return enableStatementCache;
+}
+
+public void setEnableStatementCache(boolean enableStatementCache) {
+this.enableStatementCache = enableStatementCache;
+}
+
 public static class StateParam implements Serializable {

[GitHub] asfgit commented on issue #199: KYLIN-3497 Make JDBC Module more testable

2018-08-14 Thread GitBox 
asfgit commented on issue #199: KYLIN-3497 Make JDBC Module more testable
URL: https://github.com/apache/kylin/pull/199#issuecomment-412808466
 
 
   Can one of the admins verify this patch?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ian4hu opened a new pull request #199: KYLIN-3497 Make JDBC Module more testable

2018-08-14 Thread GitBox 
ian4hu opened a new pull request #199: KYLIN-3497 Make JDBC Module more testable
URL: https://github.com/apache/kylin/pull/199
 
 
   De-coupling IRemoteClient with KylinConnection.
   Add dependencies mocking to improve test coverage.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (KYLIN-3498) Chinese translation of community, development, blog docs

2018-08-14 Thread Na Zhai (JIRA) 
Na Zhai created KYLIN-3498:
--

 Summary: Chinese translation of community, development, blog docs
 Key: KYLIN-3498
 URL: https://issues.apache.org/jira/browse/KYLIN-3498
 Project: Kylin
  Issue Type: Improvement
  Components: Documentation
Reporter: Na Zhai
Assignee: Na Zhai


There are many active Chinese users in Kylin community, a Chinese translation 
of community, development, blog docs will be very helpful to Kylin newbies and 
experienced Kylin users as well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] asfgit commented on issue #198: Update HBaseResourceStore.java

2018-08-14 Thread GitBox 
asfgit commented on issue #198: Update HBaseResourceStore.java
URL: https://github.com/apache/kylin/pull/198#issuecomment-412765757
 
 
   Can one of the admins verify this patch?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] momoxixi opened a new pull request #198: Update HBaseResourceStore.java

2018-08-14 Thread GitBox 
momoxixi opened a new pull request #198: Update HBaseResourceStore.java
URL: https://github.com/apache/kylin/pull/198
 
 
   Avoid resource inconsistent caused by hbase rpc timeout


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services