[jira] [Commented] (KYLIN-3669) Add log to GTStreamAggregateScanner
[
https://issues.apache.org/jira/browse/KYLIN-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16694203#comment-16694203
]
ASF GitHub Bot commented on KYLIN-3669:
---
shaofengshi commented on a change in pull request #352: KYLIN-3669, add log to
indicate the case when using "GTStreamAggregateScanner".
URL: https://github.com/apache/kylin/pull/352#discussion_r235249257
##
File path:
core-storage/src/main/java/org/apache/kylin/storage/gtrecord/SegmentCubeTupleIterator.java
##
@@ -92,6 +92,7 @@ public SegmentCubeTupleIterator(CubeSegmentScanner scanner,
Cuboid cuboid, Set Add log to GTStreamAggregateScanner
> ---
>
> Key: KYLIN-3669
> URL: https://issues.apache.org/jira/browse/KYLIN-3669
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Jiatao Tao
>Assignee: Jiatao Tao
>Priority: Trivial
>
> There's no log printing in GTStreamAggregateScanner, and you know nothing
> about whether you go to this path or not, it's pretty unfriendly for trouble
> shooting.
> Code in _SegmentCubeTupleIterator_.
> {code:java}
> boolean hasMultiplePartitions = records instanceof
> SortMergedPartitionResultIterator;
> if (hasMultiplePartitions && context.isStreamAggregateEnabled()) {
> ...
> // input records are ordered, leverage stream aggregator to produce
> possibly fewer records
> try (GTStreamAggregateScanner aggregator = new
> GTStreamAggregateScanner(inputScanner, scanRequest)) {
> result = aggregator.valuesIterator(gtDimsIdx, gtMetricsIdx);
> } catch (IOException ioe) {
> // implementation of close method of anonymous IGTScanner is empty,
> no way throw exception
> throw new IllegalStateException("IOException is not expected here.",
> ioe);
> }
> return result;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3669) Add log to GTStreamAggregateScanner
[
https://issues.apache.org/jira/browse/KYLIN-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16694205#comment-16694205
]
ASF subversion and git services commented on KYLIN-3669:
Commit 78a5f349e8f744edeed8072b0dc82ce7d21e052f in kylin's branch
refs/heads/master from [~Aron.tao]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=78a5f34 ]
KYLIN-3669, add log to indicate the case when using GTStreamAggregateScanner.
> Add log to GTStreamAggregateScanner
> ---
>
> Key: KYLIN-3669
> URL: https://issues.apache.org/jira/browse/KYLIN-3669
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Jiatao Tao
>Assignee: Jiatao Tao
>Priority: Trivial
>
> There's no log printing in GTStreamAggregateScanner, and you know nothing
> about whether you go to this path or not, it's pretty unfriendly for trouble
> shooting.
> Code in _SegmentCubeTupleIterator_.
> {code:java}
> boolean hasMultiplePartitions = records instanceof
> SortMergedPartitionResultIterator;
> if (hasMultiplePartitions && context.isStreamAggregateEnabled()) {
> ...
> // input records are ordered, leverage stream aggregator to produce
> possibly fewer records
> try (GTStreamAggregateScanner aggregator = new
> GTStreamAggregateScanner(inputScanner, scanRequest)) {
> result = aggregator.valuesIterator(gtDimsIdx, gtMetricsIdx);
> } catch (IOException ioe) {
> // implementation of close method of anonymous IGTScanner is empty,
> no way throw exception
> throw new IllegalStateException("IOException is not expected here.",
> ioe);
> }
> return result;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3669) Add log to GTStreamAggregateScanner
[
https://issues.apache.org/jira/browse/KYLIN-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16694204#comment-16694204
]
ASF GitHub Bot commented on KYLIN-3669:
---
shaofengshi closed pull request #352: KYLIN-3669, add log to indicate the case
when using "GTStreamAggregateScanner".
URL: https://github.com/apache/kylin/pull/352
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/core-storage/src/main/java/org/apache/kylin/storage/gtrecord/SegmentCubeTupleIterator.java
b/core-storage/src/main/java/org/apache/kylin/storage/gtrecord/SegmentCubeTupleIterator.java
index 629c02563b..028d105a17 100644
---
a/core-storage/src/main/java/org/apache/kylin/storage/gtrecord/SegmentCubeTupleIterator.java
+++
b/core-storage/src/main/java/org/apache/kylin/storage/gtrecord/SegmentCubeTupleIterator.java
@@ -92,6 +92,7 @@ public SegmentCubeTupleIterator(CubeSegmentScanner scanner,
Cuboid cuboid, Set Add log to GTStreamAggregateScanner
> ---
>
> Key: KYLIN-3669
> URL: https://issues.apache.org/jira/browse/KYLIN-3669
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Jiatao Tao
>Assignee: Jiatao Tao
>Priority: Trivial
>
> There's no log printing in GTStreamAggregateScanner, and you know nothing
> about whether you go to this path or not, it's pretty unfriendly for trouble
> shooting.
> Code in _SegmentCubeTupleIterator_.
> {code:java}
> boolean hasMultiplePartitions = records instanceof
> SortMergedPartitionResultIterator;
> if (hasMultiplePartitions && context.isStreamAggregateEnabled()) {
> ...
> // input records are ordered, leverage stream aggregator to produce
> possibly fewer records
> try (GTStreamAggregateScanner aggregator = new
> GTStreamAggregateScanner(inputScanner, scanRequest)) {
> result = aggregator.valuesIterator(gtDimsIdx, gtMetricsIdx);
> } catch (IOException ioe) {
> // implementation of close method of anonymous IGTScanner is empty,
> no way throw exception
> throw new IllegalStateException("IOException is not expected here.",
> ioe);
> }
> return result;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] shaofengshi closed pull request #352: KYLIN-3669, add log to indicate the case when using "GTStreamAggregateScanner".
shaofengshi closed pull request #352: KYLIN-3669, add log to indicate the case when using "GTStreamAggregateScanner". URL: https://github.com/apache/kylin/pull/352 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/core-storage/src/main/java/org/apache/kylin/storage/gtrecord/SegmentCubeTupleIterator.java b/core-storage/src/main/java/org/apache/kylin/storage/gtrecord/SegmentCubeTupleIterator.java index 629c02563b..028d105a17 100644 --- a/core-storage/src/main/java/org/apache/kylin/storage/gtrecord/SegmentCubeTupleIterator.java +++ b/core-storage/src/main/java/org/apache/kylin/storage/gtrecord/SegmentCubeTupleIterator.java @@ -92,6 +92,7 @@ public SegmentCubeTupleIterator(CubeSegmentScanner scanner, Cuboid cuboid, Set
[GitHub] shaofengshi commented on a change in pull request #352: KYLIN-3669, add log to indicate the case when using "GTStreamAggregateScanner".
shaofengshi commented on a change in pull request #352: KYLIN-3669, add log to indicate the case when using "GTStreamAggregateScanner". URL: https://github.com/apache/kylin/pull/352#discussion_r235249257 ## File path: core-storage/src/main/java/org/apache/kylin/storage/gtrecord/SegmentCubeTupleIterator.java ## @@ -92,6 +92,7 @@ public SegmentCubeTupleIterator(CubeSegmentScanner scanner, Cuboid cuboid, Set
[jira] [Commented] (KYLIN-3666) Mege cube step 2: Update dictionary throws IllegalStateException
[
https://issues.apache.org/jira/browse/KYLIN-3666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16694184#comment-16694184
]
ASF GitHub Bot commented on KYLIN-3666:
---
coveralls edited a comment on issue #347: KYLIN-3666 HDFS metadata url not be
recognized
URL: https://github.com/apache/kylin/pull/347#issuecomment-439015589
## Pull Request Test Coverage Report for [Build
3870](https://coveralls.io/builds/20216347)
* **0** of **4** **(0.0%)** changed or added relevant lines in **2**
files are covered.
* **3** unchanged lines in **3** files lost coverage.
* Overall coverage decreased (**-0.0004%**) to **25.914%**
---
| Changes Missing Coverage | Covered Lines | Changed/Added Lines | % |
| :-|--||---: |
|
[engine-mr/src/main/java/org/apache/kylin/engine/mr/steps/UpdateDictionaryStep.java](https://coveralls.io/builds/20216347/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fsteps%2FUpdateDictionaryStep.java#L71)
| 0 | 1 | 0.0%
|
[engine-mr/src/main/java/org/apache/kylin/engine/mr/common/AbstractHadoopJob.java](https://coveralls.io/builds/20216347/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fcommon%2FAbstractHadoopJob.java#L493)
| 0 | 3 | 0.0%
| Files with Coverage Reduction | New Missed Lines | % |
| :-|--|--: |
|
[engine-mr/src/main/java/org/apache/kylin/engine/mr/common/AbstractHadoopJob.java](https://coveralls.io/builds/20216347/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fcommon%2FAbstractHadoopJob.java#L528)
| 1 | 15.38% |
|
[core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/20216347/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62)
| 1 | 81.08% |
|
[core-metadata/src/main/java/org/apache/kylin/source/datagen/ColumnGenerator.java](https://coveralls.io/builds/20216347/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fdatagen%2FColumnGenerator.java#L319)
| 1 | 81.08% |
| Totals | [](https://coveralls.io/builds/20216347)
|
| :-- | --: |
| Change from base [Build 3869](https://coveralls.io/builds/20215506): |
-0.0004% |
| Covered Lines: | 17648 |
| Relevant Lines: | 68103 |
---
# 💛 - [Coveralls](https://coveralls.io)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Mege cube step 2: Update dictionary throws IllegalStateException
>
>
> Key: KYLIN-3666
> URL: https://issues.apache.org/jira/browse/KYLIN-3666
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
>Affects Versions: v2.5.0
> Environment: cdh5.11.0
>Reporter: Xi Chen
>Assignee: Chao Long
>Priority: Major
> Fix For: v2.5.2
>
>
> MERGE CUBE job failed in Step 2, throwing such exception:
>
> {code:java}
> java.lang.IllegalStateException: Metadata uri :
> kylin_metadata_my_service@hdfs,path=viewfs://hadoop-footest/kylin/kylin_metadata_my_service/kylin-9583f16b-93cb-9461-a58c-43c848007972/kylin_sales_cube_clone_spark/metadata
> is not recognized
> at org.apache.kylin.common.KylinConfig.decideUriType(KylinConfig.java:172)
> at
> org.apache.kylin.common.KylinConfig.createInstanceFromUri(KylinConfig.java:182)
> at
> org.apache.kylin.engine.mr.steps.UpdateDictionaryStep.doWork(UpdateDictionaryStep.java:70)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:163)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:69)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:163)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:113)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.IllegalStateException: Metadata uri :
> kylin_metadata_my_service@hdfs,path=viewfs://hadoop-footest/kylin/kylin_metadata_my_service/kylin-9583f16b-93cb-9461-a58c-43c848007972/kylin_sales_cube_clone_spark/metadata
> looks like a file but it's neither a file n
[jira] [Commented] (KYLIN-3666) Mege cube step 2: Update dictionary throws IllegalStateException
[
https://issues.apache.org/jira/browse/KYLIN-3666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16694185#comment-16694185
]
ASF GitHub Bot commented on KYLIN-3666:
---
codecov-io commented on issue #347: KYLIN-3666 HDFS metadata url not be
recognized
URL: https://github.com/apache/kylin/pull/347#issuecomment-440520921
# [Codecov](https://codecov.io/gh/apache/kylin/pull/347?src=pr&el=h1) Report
> Merging [#347](https://codecov.io/gh/apache/kylin/pull/347?src=pr&el=desc)
into
[master](https://codecov.io/gh/apache/kylin/commit/7105e5ef1403a571922efe1661cc3d62680d560a?src=pr&el=desc)
will **decrease** coverage by `<.01%`.
> The diff coverage is `0%`.
[](https://codecov.io/gh/apache/kylin/pull/347?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master#347 +/- ##
===
- Coverage 23.61% 23.6% -0.01%
+ Complexity 48264825 -1
===
Files 11331133
Lines 68102 68103 +1
Branches 96599659
===
- Hits 16079 16078 -1
- Misses50458 50459 +1
- Partials 15651566 +1
```
| [Impacted
Files](https://codecov.io/gh/apache/kylin/pull/347?src=pr&el=tree) | Coverage Δ
| Complexity Δ | |
|---|---|---|---|
|
[...ache/kylin/engine/mr/common/AbstractHadoopJob.java](https://codecov.io/gh/apache/kylin/pull/347/diff?src=pr&el=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvY29tbW9uL0Fic3RyYWN0SGFkb29wSm9iLmphdmE=)
| `15.13% <0%> (-0.04%)` | `2 <0> (ø)` | |
|
[...he/kylin/engine/mr/steps/UpdateDictionaryStep.java](https://codecov.io/gh/apache/kylin/pull/347/diff?src=pr&el=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvc3RlcHMvVXBkYXRlRGljdGlvbmFyeVN0ZXAuamF2YQ==)
| `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: |
|
[...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/347/diff?src=pr&el=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==)
| `72.97% <0%> (-5.41%)` | `6% <0%> (-1%)` | |
|
[...g/apache/kylin/source/datagen/ColumnGenerator.java](https://codecov.io/gh/apache/kylin/pull/347/diff?src=pr&el=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc291cmNlL2RhdGFnZW4vQ29sdW1uR2VuZXJhdG9yLmphdmE=)
| `70.94% <0%> (-1.36%)` | `8% <0%> (ø)` | |
|
[.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://codecov.io/gh/apache/kylin/pull/347/diff?src=pr&el=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9UcmVlQ3Vib2lkU2NoZWR1bGVyLmphdmE=)
| `66.15% <0%> (+2.3%)` | `0% <0%> (ø)` | :arrow_down: |
--
[Continue to review full report at
Codecov](https://codecov.io/gh/apache/kylin/pull/347?src=pr&el=continue).
> **Legend** - [Click here to learn
more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by
[Codecov](https://codecov.io/gh/apache/kylin/pull/347?src=pr&el=footer). Last
update
[7105e5e...46bb2bb](https://codecov.io/gh/apache/kylin/pull/347?src=pr&el=lastupdated).
Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Mege cube step 2: Update dictionary throws IllegalStateException
>
>
> Key: KYLIN-3666
> URL: https://issues.apache.org/jira/browse/KYLIN-3666
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
>Affects Versions: v2.5.0
> Environment: cdh5.11.0
>Reporter: Xi Chen
>Assignee: Chao Long
>Priority: Major
> Fix For: v2.5.2
>
>
> MERGE CUBE job failed in Step 2, throwing such exception:
>
> {code:java}
> java.lang.IllegalStateException: Metadata uri :
> kylin_metadata_my_service@hdfs,path=viewfs://hadoop-footest/kylin/kylin_metadata_my_service/kylin-9583f16b-93cb-9461-a58c-43c848007972/kylin_sales_cube_clone_spark/metadata
> is not recognized
> at org.apache.kylin.common.KylinConfig.decideUriType(KylinConfig.jav
[GitHub] codecov-io commented on issue #347: KYLIN-3666 HDFS metadata url not be recognized
codecov-io commented on issue #347: KYLIN-3666 HDFS metadata url not be recognized URL: https://github.com/apache/kylin/pull/347#issuecomment-440520921 # [Codecov](https://codecov.io/gh/apache/kylin/pull/347?src=pr&el=h1) Report > Merging [#347](https://codecov.io/gh/apache/kylin/pull/347?src=pr&el=desc) into [master](https://codecov.io/gh/apache/kylin/commit/7105e5ef1403a571922efe1661cc3d62680d560a?src=pr&el=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `0%`. [](https://codecov.io/gh/apache/kylin/pull/347?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master#347 +/- ## === - Coverage 23.61% 23.6% -0.01% + Complexity 48264825 -1 === Files 11331133 Lines 68102 68103 +1 Branches 96599659 === - Hits 16079 16078 -1 - Misses50458 50459 +1 - Partials 15651566 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/347?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...ache/kylin/engine/mr/common/AbstractHadoopJob.java](https://codecov.io/gh/apache/kylin/pull/347/diff?src=pr&el=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvY29tbW9uL0Fic3RyYWN0SGFkb29wSm9iLmphdmE=) | `15.13% <0%> (-0.04%)` | `2 <0> (ø)` | | | [...he/kylin/engine/mr/steps/UpdateDictionaryStep.java](https://codecov.io/gh/apache/kylin/pull/347/diff?src=pr&el=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvc3RlcHMvVXBkYXRlRGljdGlvbmFyeVN0ZXAuamF2YQ==) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/347/diff?src=pr&el=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==) | `72.97% <0%> (-5.41%)` | `6% <0%> (-1%)` | | | [...g/apache/kylin/source/datagen/ColumnGenerator.java](https://codecov.io/gh/apache/kylin/pull/347/diff?src=pr&el=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc291cmNlL2RhdGFnZW4vQ29sdW1uR2VuZXJhdG9yLmphdmE=) | `70.94% <0%> (-1.36%)` | `8% <0%> (ø)` | | | [.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://codecov.io/gh/apache/kylin/pull/347/diff?src=pr&el=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9UcmVlQ3Vib2lkU2NoZWR1bGVyLmphdmE=) | `66.15% <0%> (+2.3%)` | `0% <0%> (ø)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/347?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/347?src=pr&el=footer). Last update [7105e5e...46bb2bb](https://codecov.io/gh/apache/kylin/pull/347?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] coveralls edited a comment on issue #347: KYLIN-3666 HDFS metadata url not be recognized
coveralls edited a comment on issue #347: KYLIN-3666 HDFS metadata url not be recognized URL: https://github.com/apache/kylin/pull/347#issuecomment-439015589 ## Pull Request Test Coverage Report for [Build 3870](https://coveralls.io/builds/20216347) * **0** of **4** **(0.0%)** changed or added relevant lines in **2** files are covered. * **3** unchanged lines in **3** files lost coverage. * Overall coverage decreased (**-0.0004%**) to **25.914%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/steps/UpdateDictionaryStep.java](https://coveralls.io/builds/20216347/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fsteps%2FUpdateDictionaryStep.java#L71) | 0 | 1 | 0.0% | [engine-mr/src/main/java/org/apache/kylin/engine/mr/common/AbstractHadoopJob.java](https://coveralls.io/builds/20216347/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fcommon%2FAbstractHadoopJob.java#L493) | 0 | 3 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/common/AbstractHadoopJob.java](https://coveralls.io/builds/20216347/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fcommon%2FAbstractHadoopJob.java#L528) | 1 | 15.38% | | [core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/20216347/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62) | 1 | 81.08% | | [core-metadata/src/main/java/org/apache/kylin/source/datagen/ColumnGenerator.java](https://coveralls.io/builds/20216347/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fdatagen%2FColumnGenerator.java#L319) | 1 | 81.08% | | Totals | [](https://coveralls.io/builds/20216347) | | :-- | --: | | Change from base [Build 3869](https://coveralls.io/builds/20215506): | -0.0004% | | Covered Lines: | 17648 | | Relevant Lines: | 68103 | --- # 💛 - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3669) Add log to GTStreamAggregateScanner
[
https://issues.apache.org/jira/browse/KYLIN-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16694092#comment-16694092
]
ASF GitHub Bot commented on KYLIN-3669:
---
Aaron commented on a change in pull request #352: KYLIN-3669, add log to
indicate the case when using "GTStreamAggregateScanner".
URL: https://github.com/apache/kylin/pull/352#discussion_r235229952
##
File path:
core-storage/src/main/java/org/apache/kylin/storage/gtrecord/SegmentCubeTupleIterator.java
##
@@ -92,6 +92,7 @@ public SegmentCubeTupleIterator(CubeSegmentScanner scanner,
Cuboid cuboid, Sethttp://apache-kylin.74782.x6.nabble.com/Re-doubt-about-measure-of-processedRowCount-tc12250.html
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add log to GTStreamAggregateScanner
> ---
>
> Key: KYLIN-3669
> URL: https://issues.apache.org/jira/browse/KYLIN-3669
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Jiatao Tao
>Assignee: Jiatao Tao
>Priority: Trivial
>
> There's no log printing in GTStreamAggregateScanner, and you know nothing
> about whether you go to this path or not, it's pretty unfriendly for trouble
> shooting.
> Code in _SegmentCubeTupleIterator_.
> {code:java}
> boolean hasMultiplePartitions = records instanceof
> SortMergedPartitionResultIterator;
> if (hasMultiplePartitions && context.isStreamAggregateEnabled()) {
> ...
> // input records are ordered, leverage stream aggregator to produce
> possibly fewer records
> try (GTStreamAggregateScanner aggregator = new
> GTStreamAggregateScanner(inputScanner, scanRequest)) {
> result = aggregator.valuesIterator(gtDimsIdx, gtMetricsIdx);
> } catch (IOException ioe) {
> // implementation of close method of anonymous IGTScanner is empty,
> no way throw exception
> throw new IllegalStateException("IOException is not expected here.",
> ioe);
> }
> return result;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3669) Add log to GTStreamAggregateScanner
[
https://issues.apache.org/jira/browse/KYLIN-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16694091#comment-16694091
]
ASF GitHub Bot commented on KYLIN-3669:
---
Aaron commented on a change in pull request #352: KYLIN-3669, add log to
indicate the case when using "GTStreamAggregateScanner".
URL: https://github.com/apache/kylin/pull/352#discussion_r235229952
##
File path:
core-storage/src/main/java/org/apache/kylin/storage/gtrecord/SegmentCubeTupleIterator.java
##
@@ -92,6 +92,7 @@ public SegmentCubeTupleIterator(CubeSegmentScanner scanner,
Cuboid cuboid, Sethttp://apache-kylin.74782.x6.nabble.com/Re-doubt-about-measure-of-processedRowCount-tc12250.html#none
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add log to GTStreamAggregateScanner
> ---
>
> Key: KYLIN-3669
> URL: https://issues.apache.org/jira/browse/KYLIN-3669
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Jiatao Tao
>Assignee: Jiatao Tao
>Priority: Trivial
>
> There's no log printing in GTStreamAggregateScanner, and you know nothing
> about whether you go to this path or not, it's pretty unfriendly for trouble
> shooting.
> Code in _SegmentCubeTupleIterator_.
> {code:java}
> boolean hasMultiplePartitions = records instanceof
> SortMergedPartitionResultIterator;
> if (hasMultiplePartitions && context.isStreamAggregateEnabled()) {
> ...
> // input records are ordered, leverage stream aggregator to produce
> possibly fewer records
> try (GTStreamAggregateScanner aggregator = new
> GTStreamAggregateScanner(inputScanner, scanRequest)) {
> result = aggregator.valuesIterator(gtDimsIdx, gtMetricsIdx);
> } catch (IOException ioe) {
> // implementation of close method of anonymous IGTScanner is empty,
> no way throw exception
> throw new IllegalStateException("IOException is not expected here.",
> ioe);
> }
> return result;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] Aaaaaaron commented on a change in pull request #352: KYLIN-3669, add log to indicate the case when using "GTStreamAggregateScanner".
Aaron commented on a change in pull request #352: KYLIN-3669, add log to indicate the case when using "GTStreamAggregateScanner". URL: https://github.com/apache/kylin/pull/352#discussion_r235229952 ## File path: core-storage/src/main/java/org/apache/kylin/storage/gtrecord/SegmentCubeTupleIterator.java ## @@ -92,6 +92,7 @@ public SegmentCubeTupleIterator(CubeSegmentScanner scanner, Cuboid cuboid, Sethttp://apache-kylin.74782.x6.nabble.com/Re-doubt-about-measure-of-processedRowCount-tc12250.html#none This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] Aaaaaaron commented on a change in pull request #352: KYLIN-3669, add log to indicate the case when using "GTStreamAggregateScanner".
Aaron commented on a change in pull request #352: KYLIN-3669, add log to indicate the case when using "GTStreamAggregateScanner". URL: https://github.com/apache/kylin/pull/352#discussion_r235229952 ## File path: core-storage/src/main/java/org/apache/kylin/storage/gtrecord/SegmentCubeTupleIterator.java ## @@ -92,6 +92,7 @@ public SegmentCubeTupleIterator(CubeSegmentScanner scanner, Cuboid cuboid, Sethttp://apache-kylin.74782.x6.nabble.com/Re-doubt-about-measure-of-processedRowCount-tc12250.html This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3696) TOPN度量在同一个模型下2个cube同时开启统计值不准与真实值差得较多
[
https://issues.apache.org/jira/browse/KYLIN-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693944#comment-16693944
]
Shaofeng SHI commented on KYLIN-3696:
-
[~yangwei] how about the result with MR engine?
> TOPN度量在同一个模型下2个cube同时开启统计值不准与真实值差得较多
>
>
> Key: KYLIN-3696
> URL: https://issues.apache.org/jira/browse/KYLIN-3696
> Project: Kylin
> Issue Type: Bug
> Components: Measure - TopN
>Affects Versions: v2.5.1
>Reporter: yangwei
>Priority: Major
> Attachments: image-2018-11-20-10-57-28-546.png,
> image-2018-11-20-11-01-25-120.png, image-2018-11-20-11-27-43-750.png
>
>
> 我使用的是v2.5.1，度量topN使用上出现不准的总量。
> 问题再现：
> 一，二个cube使用同一个模型就是同一张物理事实表。
> 二，二个cube同时包含相同的topN度量
> 三，二个cube状态都是Ready
> 目前我暂时的解决方法是在其中一个cube去掉一个topN度量
> 同一个sql在hive与kylin里查的的结果对不上相差很远,下面给出sql
> SELECT IP ,
> SUM(ACCESS_COUNT) c
> FROM API_ACCESS
> WHERE TAG_DATE = CAST('2018-11-19' AS DATE)
> group by ip
> ORDER BY
> c DESC
> LIMIT 10;
> 二个cube中的度量:
> cube1:
> !image-2018-11-20-10-57-28-546.png!
> cube2:
> !image-2018-11-20-11-01-25-120.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3669) Add log to GTStreamAggregateScanner
[
https://issues.apache.org/jira/browse/KYLIN-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693939#comment-16693939
]
ASF GitHub Bot commented on KYLIN-3669:
---
shaofengshi commented on a change in pull request #352: KYLIN-3669, add log to
indicate the case when using "GTStreamAggregateScanner".
URL: https://github.com/apache/kylin/pull/352#discussion_r235217694
##
File path:
core-storage/src/main/java/org/apache/kylin/storage/gtrecord/SegmentCubeTupleIterator.java
##
@@ -92,6 +92,7 @@ public SegmentCubeTupleIterator(CubeSegmentScanner scanner,
Cuboid cuboid, Set Add log to GTStreamAggregateScanner
> ---
>
> Key: KYLIN-3669
> URL: https://issues.apache.org/jira/browse/KYLIN-3669
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Jiatao Tao
>Assignee: Jiatao Tao
>Priority: Trivial
>
> There's no log printing in GTStreamAggregateScanner, and you know nothing
> about whether you go to this path or not, it's pretty unfriendly for trouble
> shooting.
> Code in _SegmentCubeTupleIterator_.
> {code:java}
> boolean hasMultiplePartitions = records instanceof
> SortMergedPartitionResultIterator;
> if (hasMultiplePartitions && context.isStreamAggregateEnabled()) {
> ...
> // input records are ordered, leverage stream aggregator to produce
> possibly fewer records
> try (GTStreamAggregateScanner aggregator = new
> GTStreamAggregateScanner(inputScanner, scanRequest)) {
> result = aggregator.valuesIterator(gtDimsIdx, gtMetricsIdx);
> } catch (IOException ioe) {
> // implementation of close method of anonymous IGTScanner is empty,
> no way throw exception
> throw new IllegalStateException("IOException is not expected here.",
> ioe);
> }
> return result;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] shaofengshi commented on a change in pull request #352: KYLIN-3669, add log to indicate the case when using "GTStreamAggregateScanner".
shaofengshi commented on a change in pull request #352: KYLIN-3669, add log to indicate the case when using "GTStreamAggregateScanner". URL: https://github.com/apache/kylin/pull/352#discussion_r235217694 ## File path: core-storage/src/main/java/org/apache/kylin/storage/gtrecord/SegmentCubeTupleIterator.java ## @@ -92,6 +92,7 @@ public SegmentCubeTupleIterator(CubeSegmentScanner scanner, Cuboid cuboid, Set
[jira] [Commented] (KYLIN-3697) check port availability when starts kylin instance
[
https://issues.apache.org/jira/browse/KYLIN-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693934#comment-16693934
]
ASF GitHub Bot commented on KYLIN-3697:
---
shaofengshi closed pull request #354: KYLIN-3697 minor, only kylin.sh start
check-env
URL: https://github.com/apache/kylin/pull/354
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/build/bin/kylin.sh b/build/bin/kylin.sh
index b65cb958a5..b04707815c 100755
--- a/build/bin/kylin.sh
+++ b/build/bin/kylin.sh
@@ -27,7 +27,6 @@ if [ "$verbose" = true ]; then
shift
fi
-source ${dir}/check-env.sh
mkdir -p ${KYLIN_HOME}/logs
mkdir -p ${KYLIN_HOME}/ext
@@ -66,7 +65,9 @@ then
quit "Kylin is running, stop it first"
fi
fi
-
+
+source ${dir}/check-env.sh
+
tomcat_root=${dir}/../tomcat
export tomcat_root
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> check port availability when starts kylin instance
> --
>
> Key: KYLIN-3697
> URL: https://issues.apache.org/jira/browse/KYLIN-3697
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.5.1
>Reporter: May Zhou
>Priority: Minor
>
> As I ran the script and it turned out with no error message, and I ran the
> kylin.sh start script, the console output indicated that a kylin instance had
> successfully started, but when I visit the :/kylin, I found
> another service was running rather than kylin.
>
> I can only find the error message in the kylin.out, which is not so
> user-friendly.
> > Caused by: java.net.BindException: Address already in use (Bind failed)
> at java.net.PlainSocketImpl.socketBind(Native Method)
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
> at java.net.ServerSocket.bind(ServerSocket.java:375)
> I am wondering why check-env does not check the availability of port, and the
> error message could be threw out more user-friendly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3697) check port availability when starts kylin instance
[ https://issues.apache.org/jira/browse/KYLIN-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693935#comment-16693935 ] ASF subversion and git services commented on KYLIN-3697: Commit 7105e5ef1403a571922efe1661cc3d62680d560a in kylin's branch refs/heads/master from DDDQ [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=7105e5e ] KYLIN-3697 minor, only kylin.sh start check-env > check port availability when starts kylin instance > -- > > Key: KYLIN-3697 > URL: https://issues.apache.org/jira/browse/KYLIN-3697 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.1 >Reporter: May Zhou >Priority: Minor > > As I ran the script and it turned out with no error message, and I ran the > kylin.sh start script, the console output indicated that a kylin instance had > successfully started, but when I visit the :/kylin, I found > another service was running rather than kylin. > > I can only find the error message in the kylin.out, which is not so > user-friendly. > > Caused by: java.net.BindException: Address already in use (Bind failed) > at java.net.PlainSocketImpl.socketBind(Native Method) > at > java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387) > at java.net.ServerSocket.bind(ServerSocket.java:375) > I am wondering why check-env does not check the availability of port, and the > error message could be threw out more user-friendly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] shaofengshi closed pull request #354: KYLIN-3697 minor, only kylin.sh start check-env
shaofengshi closed pull request #354: KYLIN-3697 minor, only kylin.sh start
check-env
URL: https://github.com/apache/kylin/pull/354
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/build/bin/kylin.sh b/build/bin/kylin.sh
index b65cb958a5..b04707815c 100755
--- a/build/bin/kylin.sh
+++ b/build/bin/kylin.sh
@@ -27,7 +27,6 @@ if [ "$verbose" = true ]; then
shift
fi
-source ${dir}/check-env.sh
mkdir -p ${KYLIN_HOME}/logs
mkdir -p ${KYLIN_HOME}/ext
@@ -66,7 +65,9 @@ then
quit "Kylin is running, stop it first"
fi
fi
-
+
+source ${dir}/check-env.sh
+
tomcat_root=${dir}/../tomcat
export tomcat_root
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (KYLIN-3697) check port availability when starts kylin instance
[ https://issues.apache.org/jira/browse/KYLIN-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693331#comment-16693331 ] ASF GitHub Bot commented on KYLIN-3697: --- codecov-io commented on issue #354: KYLIN-3697 minor, only kylin.sh start check-env URL: https://github.com/apache/kylin/pull/354#issuecomment-440300354 # [Codecov](https://codecov.io/gh/apache/kylin/pull/354?src=pr&el=h1) Report > Merging [#354](https://codecov.io/gh/apache/kylin/pull/354?src=pr&el=desc) into [master](https://codecov.io/gh/apache/kylin/commit/5aea3d40646ed578f5a35fde290ebab560790785?src=pr&el=desc) will **increase** coverage by `<.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/kylin/pull/354?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #354 +/- ## + Coverage 23.6% 23.61% +<.01% - Complexity 4824 4826 +2 Files 1133 1133 Lines 6810268102 Branches 9659 9659 + Hits 1607716082 +5 + Misses5045850456 -2 + Partials 1567 1564 -3 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/354?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...ain/java/org/apache/kylin/cube/model/CubeDesc.java](https://codecov.io/gh/apache/kylin/pull/354/diff?src=pr&el=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL21vZGVsL0N1YmVEZXNjLmphdmE=) | `64.28% <0%> (ø)` | `179% <0%> (ø)` | :arrow_down: | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/354/diff?src=pr&el=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `70.21% <0%> (+0.6%)` | `7% <0%> (ø)` | :arrow_down: | | [...a/org/apache/kylin/dict/Number2BytesConverter.java](https://codecov.io/gh/apache/kylin/pull/354/diff?src=pr&el=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L051bWJlcjJCeXRlc0NvbnZlcnRlci5qYXZh) | `82.53% <0%> (+0.79%)` | `18% <0%> (+1%)` | :arrow_up: | | [...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/354/diff?src=pr&el=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==) | `78.37% <0%> (+5.4%)` | `7% <0%> (+1%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/354?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/354?src=pr&el=footer). Last update [5aea3d4...c463662](https://codecov.io/gh/apache/kylin/pull/354?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > check port availability when starts kylin instance > -- > > Key: KYLIN-3697 > URL: https://issues.apache.org/jira/browse/KYLIN-3697 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.1 >Reporter: May Zhou >Priority: Minor > > As I ran the script and it turned out with no error message, and I ran the > kylin.sh start script, the console output indicated that a kylin instance had > successfully started, but when I visit the :/kylin, I found > another service was running rather than kylin. > > I can only find the error message in the kylin.out, which is not so > user-friendly. > > Caused by: java.net.BindException: Address already in use (Bind failed) > at java.net.PlainSocketImpl.socketBind(Native Method) > at > java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387) > at java.net.ServerSocket.bind(ServerSocket.java:375) > I am wondering why check-env does not check the availability of port, and the > error message could be threw out more user-friendly. -- This message was sent by Atlass
[GitHub] codecov-io commented on issue #354: KYLIN-3697 minor, only kylin.sh start check-env
codecov-io commented on issue #354: KYLIN-3697 minor, only kylin.sh start check-env URL: https://github.com/apache/kylin/pull/354#issuecomment-440300354 # [Codecov](https://codecov.io/gh/apache/kylin/pull/354?src=pr&el=h1) Report > Merging [#354](https://codecov.io/gh/apache/kylin/pull/354?src=pr&el=desc) into [master](https://codecov.io/gh/apache/kylin/commit/5aea3d40646ed578f5a35fde290ebab560790785?src=pr&el=desc) will **increase** coverage by `<.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/kylin/pull/354?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #354 +/- ## + Coverage 23.6% 23.61% +<.01% - Complexity 4824 4826 +2 Files 1133 1133 Lines 6810268102 Branches 9659 9659 + Hits 1607716082 +5 + Misses5045850456 -2 + Partials 1567 1564 -3 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/354?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...ain/java/org/apache/kylin/cube/model/CubeDesc.java](https://codecov.io/gh/apache/kylin/pull/354/diff?src=pr&el=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL21vZGVsL0N1YmVEZXNjLmphdmE=) | `64.28% <0%> (ø)` | `179% <0%> (ø)` | :arrow_down: | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/354/diff?src=pr&el=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `70.21% <0%> (+0.6%)` | `7% <0%> (ø)` | :arrow_down: | | [...a/org/apache/kylin/dict/Number2BytesConverter.java](https://codecov.io/gh/apache/kylin/pull/354/diff?src=pr&el=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L051bWJlcjJCeXRlc0NvbnZlcnRlci5qYXZh) | `82.53% <0%> (+0.79%)` | `18% <0%> (+1%)` | :arrow_up: | | [...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/354/diff?src=pr&el=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==) | `78.37% <0%> (+5.4%)` | `7% <0%> (+1%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/354?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/354?src=pr&el=footer). Last update [5aea3d4...c463662](https://codecov.io/gh/apache/kylin/pull/354?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3698) check-env.sh should print more details about checking items
[ https://issues.apache.org/jira/browse/KYLIN-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693325#comment-16693325 ] May Zhou commented on KYLIN-3698: - As this issue is not that urgent, I would like to contribute a patch myself. :) > check-env.sh should print more details about checking items > --- > > Key: KYLIN-3698 > URL: https://issues.apache.org/jira/browse/KYLIN-3698 > Project: Kylin > Issue Type: Improvement >Affects Versions: v2.5.1 >Reporter: May Zhou >Priority: Minor > > In the current version, when users run _check-env.sh_, if there's no error > message, it means everything is OK. > From my perspective, adding more details about the checking items when > executing check-env.sh is better. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3698) check-env.sh should print more details about checking items
May Zhou created KYLIN-3698: --- Summary: check-env.sh should print more details about checking items Key: KYLIN-3698 URL: https://issues.apache.org/jira/browse/KYLIN-3698 Project: Kylin Issue Type: Improvement Affects Versions: v2.5.1 Reporter: May Zhou In the current version, when users run _check-env.sh_, if there's no error message, it means everything is OK. >From my perspective, adding more details about the checking items when >executing check-env.sh is better. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] asfgit commented on issue #354: minor, only kylin.sh start check-env
asfgit commented on issue #354: minor, only kylin.sh start check-env URL: https://github.com/apache/kylin/pull/354#issuecomment-440292695 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] DingqianZhou opened a new pull request #354: minor, only kylin.sh start check-env
DingqianZhou opened a new pull request #354: minor, only kylin.sh start check-env URL: https://github.com/apache/kylin/pull/354 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3697) check port availability when starts kylin instance
[ https://issues.apache.org/jira/browse/KYLIN-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693264#comment-16693264 ] ASF GitHub Bot commented on KYLIN-3697: --- codecov-io commented on issue #353: KYLIN-3697 Check port availability when starts kylin instance URL: https://github.com/apache/kylin/pull/353#issuecomment-440281351 # [Codecov](https://codecov.io/gh/apache/kylin/pull/353?src=pr&el=h1) Report > Merging [#353](https://codecov.io/gh/apache/kylin/pull/353?src=pr&el=desc) into [master](https://codecov.io/gh/apache/kylin/commit/4e0340737fc39364c23c6b7d6d5f604a8e2eca7f?src=pr&el=desc) will **increase** coverage by `<.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/kylin/pull/353?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #353 +/- ## + Coverage 23.6% 23.61% +<.01% - Complexity 4824 4825 +1 Files 1133 1133 Lines 6810268102 Branches 9659 9659 + Hits 1607716080 +3 + Misses5045850457 -1 + Partials 1567 1565 -2 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/353?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...ain/java/org/apache/kylin/cube/model/CubeDesc.java](https://codecov.io/gh/apache/kylin/pull/353/diff?src=pr&el=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL21vZGVsL0N1YmVEZXNjLmphdmE=) | `64.28% <0%> (ø)` | `179% <0%> (ø)` | :arrow_down: | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/353/diff?src=pr&el=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `70.21% <0%> (+0.6%)` | `7% <0%> (ø)` | :arrow_down: | | [...a/org/apache/kylin/dict/Number2BytesConverter.java](https://codecov.io/gh/apache/kylin/pull/353/diff?src=pr&el=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L051bWJlcjJCeXRlc0NvbnZlcnRlci5qYXZh) | `82.53% <0%> (+0.79%)` | `18% <0%> (+1%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/353?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/353?src=pr&el=footer). Last update [4e03407...28d1427](https://codecov.io/gh/apache/kylin/pull/353?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > check port availability when starts kylin instance > -- > > Key: KYLIN-3697 > URL: https://issues.apache.org/jira/browse/KYLIN-3697 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.1 >Reporter: May Zhou >Priority: Minor > > As I ran the script and it turned out with no error message, and I ran the > kylin.sh start script, the console output indicated that a kylin instance had > successfully started, but when I visit the :/kylin, I found > another service was running rather than kylin. > > I can only find the error message in the kylin.out, which is not so > user-friendly. > > Caused by: java.net.BindException: Address already in use (Bind failed) > at java.net.PlainSocketImpl.socketBind(Native Method) > at > java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387) > at java.net.ServerSocket.bind(ServerSocket.java:375) > I am wondering why check-env does not check the availability of port, and the > error message could be threw out more user-friendly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] codecov-io commented on issue #353: KYLIN-3697 Check port availability when starts kylin instance
codecov-io commented on issue #353: KYLIN-3697 Check port availability when starts kylin instance URL: https://github.com/apache/kylin/pull/353#issuecomment-440281351 # [Codecov](https://codecov.io/gh/apache/kylin/pull/353?src=pr&el=h1) Report > Merging [#353](https://codecov.io/gh/apache/kylin/pull/353?src=pr&el=desc) into [master](https://codecov.io/gh/apache/kylin/commit/4e0340737fc39364c23c6b7d6d5f604a8e2eca7f?src=pr&el=desc) will **increase** coverage by `<.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/kylin/pull/353?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #353 +/- ## + Coverage 23.6% 23.61% +<.01% - Complexity 4824 4825 +1 Files 1133 1133 Lines 6810268102 Branches 9659 9659 + Hits 1607716080 +3 + Misses5045850457 -1 + Partials 1567 1565 -2 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/353?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...ain/java/org/apache/kylin/cube/model/CubeDesc.java](https://codecov.io/gh/apache/kylin/pull/353/diff?src=pr&el=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL21vZGVsL0N1YmVEZXNjLmphdmE=) | `64.28% <0%> (ø)` | `179% <0%> (ø)` | :arrow_down: | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/353/diff?src=pr&el=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `70.21% <0%> (+0.6%)` | `7% <0%> (ø)` | :arrow_down: | | [...a/org/apache/kylin/dict/Number2BytesConverter.java](https://codecov.io/gh/apache/kylin/pull/353/diff?src=pr&el=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L051bWJlcjJCeXRlc0NvbnZlcnRlci5qYXZh) | `82.53% <0%> (+0.79%)` | `18% <0%> (+1%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/353?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/353?src=pr&el=footer). Last update [4e03407...28d1427](https://codecov.io/gh/apache/kylin/pull/353?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3697) check port availability when starts kylin instance
[
https://issues.apache.org/jira/browse/KYLIN-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693251#comment-16693251
]
ASF GitHub Bot commented on KYLIN-3697:
---
shaofengshi closed pull request #353: KYLIN-3697 Check port availability when
starts kylin instance
URL: https://github.com/apache/kylin/pull/353
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/build/bin/check-env.sh b/build/bin/check-env.sh
index cd745d4526..5b14e0af7d 100755
--- a/build/bin/check-env.sh
+++ b/build/bin/check-env.sh
@@ -80,4 +80,7 @@ then
then
quit "Failed to create $SPARK_HISTORYLOG_DIR. Please make sure the
user has right to access $SPARK_HISTORYLOG_DIR"
fi
-fi
\ No newline at end of file
+fi
+
+${KYLIN_HOME}/bin/check-port-availability.sh || exit 1;
+
diff --git a/build/bin/check-port-availability.sh
b/build/bin/check-port-availability.sh
new file mode 100644
index 00..023fdef3b9
--- /dev/null
+++ b/build/bin/check-port-availability.sh
@@ -0,0 +1,30 @@
+#!/bin/bash
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+source $(cd -P -- "$(dirname -- "$0")" && pwd -P)/header.sh
+
+
+# get port from configuraton
+kylin_port=`grep " check port availability when starts kylin instance
> --
>
> Key: KYLIN-3697
> URL: https://issues.apache.org/jira/browse/KYLIN-3697
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.5.1
>Reporter: May Zhou
>Priority: Minor
>
> As I ran the script and it turned out with no error message, and I ran the
> kylin.sh start script, the console output indicated that a kylin instance had
> successfully started, but when I visit the :/kylin, I found
> another service was running rather than kylin.
>
> I can only find the error message in the kylin.out, which is not so
> user-friendly.
> > Caused by: java.net.BindException: Address already in use (Bind failed)
> at java.net.PlainSocketImpl.socketBind(Native Method)
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
> at java.net.ServerSocket.bind(ServerSocket.java:375)
> I am wondering why check-env does not check the availability of port, and the
> error message could be threw out more user-friendly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3697) check port availability when starts kylin instance
[ https://issues.apache.org/jira/browse/KYLIN-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693252#comment-16693252 ] ASF subversion and git services commented on KYLIN-3697: Commit 5aea3d40646ed578f5a35fde290ebab560790785 in kylin's branch refs/heads/master from DDDQ [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=5aea3d4 ] KYLIN-3697:check port availability when starts kylin instance > check port availability when starts kylin instance > -- > > Key: KYLIN-3697 > URL: https://issues.apache.org/jira/browse/KYLIN-3697 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.1 >Reporter: May Zhou >Priority: Minor > > As I ran the script and it turned out with no error message, and I ran the > kylin.sh start script, the console output indicated that a kylin instance had > successfully started, but when I visit the :/kylin, I found > another service was running rather than kylin. > > I can only find the error message in the kylin.out, which is not so > user-friendly. > > Caused by: java.net.BindException: Address already in use (Bind failed) > at java.net.PlainSocketImpl.socketBind(Native Method) > at > java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387) > at java.net.ServerSocket.bind(ServerSocket.java:375) > I am wondering why check-env does not check the availability of port, and the > error message could be threw out more user-friendly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] shaofengshi closed pull request #353: KYLIN-3697 Check port availability when starts kylin instance
shaofengshi closed pull request #353: KYLIN-3697 Check port availability when
starts kylin instance
URL: https://github.com/apache/kylin/pull/353
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/build/bin/check-env.sh b/build/bin/check-env.sh
index cd745d4526..5b14e0af7d 100755
--- a/build/bin/check-env.sh
+++ b/build/bin/check-env.sh
@@ -80,4 +80,7 @@ then
then
quit "Failed to create $SPARK_HISTORYLOG_DIR. Please make sure the
user has right to access $SPARK_HISTORYLOG_DIR"
fi
-fi
\ No newline at end of file
+fi
+
+${KYLIN_HOME}/bin/check-port-availability.sh || exit 1;
+
diff --git a/build/bin/check-port-availability.sh
b/build/bin/check-port-availability.sh
new file mode 100644
index 00..023fdef3b9
--- /dev/null
+++ b/build/bin/check-port-availability.sh
@@ -0,0 +1,30 @@
+#!/bin/bash
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+source $(cd -P -- "$(dirname -- "$0")" && pwd -P)/header.sh
+
+
+# get port from configuraton
+kylin_port=`grep "
[jira] [Commented] (KYLIN-3697) check port availability when starts kylin instance
[ https://issues.apache.org/jira/browse/KYLIN-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693233#comment-16693233 ] ASF GitHub Bot commented on KYLIN-3697: --- DingqianZhou opened a new pull request #353: KYLIN-3697 Check port availability when starts kylin instance URL: https://github.com/apache/kylin/pull/353 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > check port availability when starts kylin instance > -- > > Key: KYLIN-3697 > URL: https://issues.apache.org/jira/browse/KYLIN-3697 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.1 >Reporter: May Zhou >Priority: Minor > > As I ran the script and it turned out with no error message, and I ran the > kylin.sh start script, the console output indicated that a kylin instance had > successfully started, but when I visit the :/kylin, I found > another service was running rather than kylin. > > I can only find the error message in the kylin.out, which is not so > user-friendly. > > Caused by: java.net.BindException: Address already in use (Bind failed) > at java.net.PlainSocketImpl.socketBind(Native Method) > at > java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387) > at java.net.ServerSocket.bind(ServerSocket.java:375) > I am wondering why check-env does not check the availability of port, and the > error message could be threw out more user-friendly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3697) check port availability when starts kylin instance
[ https://issues.apache.org/jira/browse/KYLIN-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693234#comment-16693234 ] ASF GitHub Bot commented on KYLIN-3697: --- asfgit commented on issue #353: KYLIN-3697 Check port availability when starts kylin instance URL: https://github.com/apache/kylin/pull/353#issuecomment-440273296 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > check port availability when starts kylin instance > -- > > Key: KYLIN-3697 > URL: https://issues.apache.org/jira/browse/KYLIN-3697 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.1 >Reporter: May Zhou >Priority: Minor > > As I ran the script and it turned out with no error message, and I ran the > kylin.sh start script, the console output indicated that a kylin instance had > successfully started, but when I visit the :/kylin, I found > another service was running rather than kylin. > > I can only find the error message in the kylin.out, which is not so > user-friendly. > > Caused by: java.net.BindException: Address already in use (Bind failed) > at java.net.PlainSocketImpl.socketBind(Native Method) > at > java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387) > at java.net.ServerSocket.bind(ServerSocket.java:375) > I am wondering why check-env does not check the availability of port, and the > error message could be threw out more user-friendly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] asfgit commented on issue #353: KYLIN-3697 Check port availability when starts kylin instance
asfgit commented on issue #353: KYLIN-3697 Check port availability when starts kylin instance URL: https://github.com/apache/kylin/pull/353#issuecomment-440273296 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] DingqianZhou opened a new pull request #353: KYLIN-3697 Check port availability when starts kylin instance
DingqianZhou opened a new pull request #353: KYLIN-3697 Check port availability when starts kylin instance URL: https://github.com/apache/kylin/pull/353 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3669) Add log to GTStreamAggregateScanner
[
https://issues.apache.org/jira/browse/KYLIN-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693016#comment-16693016
]
ASF GitHub Bot commented on KYLIN-3669:
---
Aaron opened a new pull request #352: KYLIN-3669, add log to indicate the
case when using "GTStreamAggregateScanner".
URL: https://github.com/apache/kylin/pull/352
…Scanner.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add log to GTStreamAggregateScanner
> ---
>
> Key: KYLIN-3669
> URL: https://issues.apache.org/jira/browse/KYLIN-3669
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Jiatao Tao
>Assignee: Jiatao Tao
>Priority: Trivial
>
> There's no log printing in GTStreamAggregateScanner, and you know nothing
> about whether you go to this path or not, it's pretty unfriendly for trouble
> shooting.
> Code in _SegmentCubeTupleIterator_.
> {code:java}
> boolean hasMultiplePartitions = records instanceof
> SortMergedPartitionResultIterator;
> if (hasMultiplePartitions && context.isStreamAggregateEnabled()) {
> ...
> // input records are ordered, leverage stream aggregator to produce
> possibly fewer records
> try (GTStreamAggregateScanner aggregator = new
> GTStreamAggregateScanner(inputScanner, scanRequest)) {
> result = aggregator.valuesIterator(gtDimsIdx, gtMetricsIdx);
> } catch (IOException ioe) {
> // implementation of close method of anonymous IGTScanner is empty,
> no way throw exception
> throw new IllegalStateException("IOException is not expected here.",
> ioe);
> }
> return result;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3669) Add log to GTStreamAggregateScanner
[
https://issues.apache.org/jira/browse/KYLIN-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693017#comment-16693017
]
ASF GitHub Bot commented on KYLIN-3669:
---
asfgit commented on issue #352: KYLIN-3669, add log to indicate the case when
using "GTStreamAggregateScanner".
URL: https://github.com/apache/kylin/pull/352#issuecomment-440223647
Can one of the admins verify this patch?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add log to GTStreamAggregateScanner
> ---
>
> Key: KYLIN-3669
> URL: https://issues.apache.org/jira/browse/KYLIN-3669
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Jiatao Tao
>Assignee: Jiatao Tao
>Priority: Trivial
>
> There's no log printing in GTStreamAggregateScanner, and you know nothing
> about whether you go to this path or not, it's pretty unfriendly for trouble
> shooting.
> Code in _SegmentCubeTupleIterator_.
> {code:java}
> boolean hasMultiplePartitions = records instanceof
> SortMergedPartitionResultIterator;
> if (hasMultiplePartitions && context.isStreamAggregateEnabled()) {
> ...
> // input records are ordered, leverage stream aggregator to produce
> possibly fewer records
> try (GTStreamAggregateScanner aggregator = new
> GTStreamAggregateScanner(inputScanner, scanRequest)) {
> result = aggregator.valuesIterator(gtDimsIdx, gtMetricsIdx);
> } catch (IOException ioe) {
> // implementation of close method of anonymous IGTScanner is empty,
> no way throw exception
> throw new IllegalStateException("IOException is not expected here.",
> ioe);
> }
> return result;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] Aaaaaaron opened a new pull request #352: KYLIN-3669, add log to indicate the case when using "GTStreamAggregateScanner".
Aaron opened a new pull request #352: KYLIN-3669, add log to indicate the case when using "GTStreamAggregateScanner". URL: https://github.com/apache/kylin/pull/352 …Scanner. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #352: KYLIN-3669, add log to indicate the case when using "GTStreamAggregateScanner".
asfgit commented on issue #352: KYLIN-3669, add log to indicate the case when using "GTStreamAggregateScanner". URL: https://github.com/apache/kylin/pull/352#issuecomment-440223647 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3636) Wrong "storage_type" in CubeDesc causing cube building error
[
https://issues.apache.org/jira/browse/KYLIN-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692991#comment-16692991
]
ASF subversion and git services commented on KYLIN-3636:
Commit 0fe1eb2ebd77b37023228d700ac35378e1713bd0 in kylin's branch
refs/heads/2.5.x from shaofengshi
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=0fe1eb2 ]
KYLIN-3636 set default storage_type to 2
> Wrong "storage_type" in CubeDesc causing cube building error
>
>
> Key: KYLIN-3636
> URL: https://issues.apache.org/jira/browse/KYLIN-3636
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v2.4.1
>Reporter: KANG-SEN LU
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.5.2
>
>
> Hi, ShaoFeng:
>
> Thanks for the reply. I missed this email and not responded earlier, I am
> sorry.
>
> I tried to reproduce this problem with the sample database, and it did not
> happen.
>
> So I am hoping by collecting enough "clue", someone can figure out why this
> problem occurred.
>
> --
> I issued the following query at the sample project to exercise the topn
> aggregation:
>
> select seller_id, SUM(price) as total from kylin_sales group by seller_id
> order by total limit 5;
>
> With my own added debugging, I saw the following log in the kylin.log: (the
> query worked OK).
>
> 2018-10-16 16:18:19,963 INFO [Query a747f16f-4b12-cc97-08d2-9b45c27a529f-90]
> model.FunctionDesc:59 : KSL2,
> getRewriteFieldName=_KY_SUM_KYLIN_SALES_PRICE_
> 2018-10-16 16:18:19,963 INFO [kylin-coproc--pool12-t1]
> v2.CubeHBaseEndpointRPC:217 : Query-a747f16f-4b12-cc97-08d2-9b45c27a529f:
> send request to the init region server anovadata4.anovadata.local on table
> ANOVA_KYLIN_25X_K758MEAWJG
> 2018-10-16 16:18:19,963 INFO [Query a747f16f-4b12-cc97-08d2-9b45c27a529f-90]
> topn.TopNMeasureType:399 : KSL888: in TopNMeasureType.java, sumFieldName=
> _KY_SUM_KYLIN_SALES_PRICE_
>
>
> When I was executing my project query, I issued the following select
> statement:
>
> SELECT ZETTICSDW.A_VL_HOURLY_V.IMSIID \"ZETTICSDW_A_VL_HOURLY_V_IMSIID\",
> SUM(ZETTICSDW.A_VL_HOURLY_V.SIG_EVENT_COUNT)
> \"vl_aggs_model___USERS_BY_ERROR_3XX\" FROM ZETTICSDW.A_VL_HOURLY_V inner
> JOIN ZETTICSDW.T_VL_TRANSACTION_RULE_V ON (
> ZETTICSDW.A_VL_HOURLY_V.CAUSE_CODE_KEY =
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.CAUSE_CODE_KEY AND
> ZETTICSDW.A_VL_HOURLY_V.REASON_CODE_KEY =
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.REASON_CODE_KEY AND
> ZETTICSDW.A_VL_HOURLY_V.TRANSACTION_TYPE_KEY =
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.TRANSACTION_TYPE_KEY)
> WHERE ((ZETTICSDW.A_VL_HOURLY_V.THEDATE = '20180209') AND
> ((ZETTICSDW.A_VL_HOURLY_V.THEHOUR >= '02') AND
> (ZETTICSDW.A_VL_HOURLY_V.THEHOUR <= '03'))) AND
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.DISPLAY_STRING LIKE '%+3%'
> GROUP BY ZETTICSDW.A_VL_HOURLY_V.IMSIID
> ORDER BY \"vl_aggs_model___USERS_BY_ERROR_3XX\"
> LIMIT 25
>
> An exception occurred within the method "private ColumnRowType
> buildColumnRowType()" of
> "query/src/main/java/org/apache/kylin/query/relnode/OLAPTableScan.java".
>
> if (columns.size() != rowType.getFieldCount()) {
> throw new IllegalStateException("RowType=" +
> rowType.getFieldCount() + ", ColumnRowType=" + columns.size());
> }
>
> It printed "RowType=133, ColumnRowType=132".
> The RowType list contains one extra column name: "ANY
> _KY_SUM_1_3a1aedef_SIG_EVENT_COUNT_".
>
>
> I think this has something to do with the bug fix "KYLIN-3359 Support
> sum(expression) if possible".
>
> After this bug fix was submitted, I noticed that a lot of column name was
> added into rowType like "_KY_SUM_XXX".
>
> This strange column name "_KY_SUM_1_3a1aedef_SIG_EVENT_COUNT_" is very
> similar.
>
> I also found that this extra column name, only existed in RowType but not in
> ColumnRowType, was added in the method "public void
> implementRewrite(RewriteImplementor implementor)" within "OLAPJoinRel.java".
>
> With my own debug statement, I saw this debug text in kylin.log:
>
> relnode.OLAPJoinRel:362 : KSL54: newField= #132:
> _KY_SUM_1_6735969a_SIG_EVENT_COUNT_ ANY
>
>
> I hope someone with a deeper technical knowledge in kylin query engine can
> figure out what was causing the problem I have seen.
>
> Thanks again.
>
> Kang-sen
>
> *From:* ShaoFeng Shi [[mailto:shaofeng...@apache.org]]
> *Sent:* Friday, October 05, 2018 9:59 PM
> *To:* user <[u...@kylin.apache.org|mailto:u...@kylin.apache.org]>
> *Subject:* Re: any body see topn in kylin 2.5.1 working?
>
> Hi Kang-Sen,
>
> Didn't see this; Can you reproduce the problem with the sample cube?
>
> Kang-Sen Lu <[k...@anovadata.com|mai
[jira] [Commented] (KYLIN-3636) Wrong "storage_type" in CubeDesc causing cube building error
[
https://issues.apache.org/jira/browse/KYLIN-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692993#comment-16692993
]
ASF subversion and git services commented on KYLIN-3636:
Commit 5be8474c656852501bbbe8c1724adf616a4fe6d1 in kylin's branch
refs/heads/2.4.x from shaofengshi
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=5be8474 ]
KYLIN-3636 set default storage_type to 2
> Wrong "storage_type" in CubeDesc causing cube building error
>
>
> Key: KYLIN-3636
> URL: https://issues.apache.org/jira/browse/KYLIN-3636
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v2.4.1
>Reporter: KANG-SEN LU
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.5.2
>
>
> Hi, ShaoFeng:
>
> Thanks for the reply. I missed this email and not responded earlier, I am
> sorry.
>
> I tried to reproduce this problem with the sample database, and it did not
> happen.
>
> So I am hoping by collecting enough "clue", someone can figure out why this
> problem occurred.
>
> --
> I issued the following query at the sample project to exercise the topn
> aggregation:
>
> select seller_id, SUM(price) as total from kylin_sales group by seller_id
> order by total limit 5;
>
> With my own added debugging, I saw the following log in the kylin.log: (the
> query worked OK).
>
> 2018-10-16 16:18:19,963 INFO [Query a747f16f-4b12-cc97-08d2-9b45c27a529f-90]
> model.FunctionDesc:59 : KSL2,
> getRewriteFieldName=_KY_SUM_KYLIN_SALES_PRICE_
> 2018-10-16 16:18:19,963 INFO [kylin-coproc--pool12-t1]
> v2.CubeHBaseEndpointRPC:217 : Query-a747f16f-4b12-cc97-08d2-9b45c27a529f:
> send request to the init region server anovadata4.anovadata.local on table
> ANOVA_KYLIN_25X_K758MEAWJG
> 2018-10-16 16:18:19,963 INFO [Query a747f16f-4b12-cc97-08d2-9b45c27a529f-90]
> topn.TopNMeasureType:399 : KSL888: in TopNMeasureType.java, sumFieldName=
> _KY_SUM_KYLIN_SALES_PRICE_
>
>
> When I was executing my project query, I issued the following select
> statement:
>
> SELECT ZETTICSDW.A_VL_HOURLY_V.IMSIID \"ZETTICSDW_A_VL_HOURLY_V_IMSIID\",
> SUM(ZETTICSDW.A_VL_HOURLY_V.SIG_EVENT_COUNT)
> \"vl_aggs_model___USERS_BY_ERROR_3XX\" FROM ZETTICSDW.A_VL_HOURLY_V inner
> JOIN ZETTICSDW.T_VL_TRANSACTION_RULE_V ON (
> ZETTICSDW.A_VL_HOURLY_V.CAUSE_CODE_KEY =
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.CAUSE_CODE_KEY AND
> ZETTICSDW.A_VL_HOURLY_V.REASON_CODE_KEY =
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.REASON_CODE_KEY AND
> ZETTICSDW.A_VL_HOURLY_V.TRANSACTION_TYPE_KEY =
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.TRANSACTION_TYPE_KEY)
> WHERE ((ZETTICSDW.A_VL_HOURLY_V.THEDATE = '20180209') AND
> ((ZETTICSDW.A_VL_HOURLY_V.THEHOUR >= '02') AND
> (ZETTICSDW.A_VL_HOURLY_V.THEHOUR <= '03'))) AND
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.DISPLAY_STRING LIKE '%+3%'
> GROUP BY ZETTICSDW.A_VL_HOURLY_V.IMSIID
> ORDER BY \"vl_aggs_model___USERS_BY_ERROR_3XX\"
> LIMIT 25
>
> An exception occurred within the method "private ColumnRowType
> buildColumnRowType()" of
> "query/src/main/java/org/apache/kylin/query/relnode/OLAPTableScan.java".
>
> if (columns.size() != rowType.getFieldCount()) {
> throw new IllegalStateException("RowType=" +
> rowType.getFieldCount() + ", ColumnRowType=" + columns.size());
> }
>
> It printed "RowType=133, ColumnRowType=132".
> The RowType list contains one extra column name: "ANY
> _KY_SUM_1_3a1aedef_SIG_EVENT_COUNT_".
>
>
> I think this has something to do with the bug fix "KYLIN-3359 Support
> sum(expression) if possible".
>
> After this bug fix was submitted, I noticed that a lot of column name was
> added into rowType like "_KY_SUM_XXX".
>
> This strange column name "_KY_SUM_1_3a1aedef_SIG_EVENT_COUNT_" is very
> similar.
>
> I also found that this extra column name, only existed in RowType but not in
> ColumnRowType, was added in the method "public void
> implementRewrite(RewriteImplementor implementor)" within "OLAPJoinRel.java".
>
> With my own debug statement, I saw this debug text in kylin.log:
>
> relnode.OLAPJoinRel:362 : KSL54: newField= #132:
> _KY_SUM_1_6735969a_SIG_EVENT_COUNT_ ANY
>
>
> I hope someone with a deeper technical knowledge in kylin query engine can
> figure out what was causing the problem I have seen.
>
> Thanks again.
>
> Kang-sen
>
> *From:* ShaoFeng Shi [[mailto:shaofeng...@apache.org]]
> *Sent:* Friday, October 05, 2018 9:59 PM
> *To:* user <[u...@kylin.apache.org|mailto:u...@kylin.apache.org]>
> *Subject:* Re: any body see topn in kylin 2.5.1 working?
>
> Hi Kang-Sen,
>
> Didn't see this; Can you reproduce the problem with the sample cube?
>
> Kang-Sen Lu <[k...@anovadata.com|mai
[jira] [Commented] (KYLIN-3636) Wrong "storage_type" in CubeDesc causing cube building error
[
https://issues.apache.org/jira/browse/KYLIN-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692990#comment-16692990
]
ASF subversion and git services commented on KYLIN-3636:
Commit 4e0340737fc39364c23c6b7d6d5f604a8e2eca7f in kylin's branch
refs/heads/master from shaofengshi
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=4e03407 ]
KYLIN-3636 set default storage_type to 2
> Wrong "storage_type" in CubeDesc causing cube building error
>
>
> Key: KYLIN-3636
> URL: https://issues.apache.org/jira/browse/KYLIN-3636
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v2.4.1
>Reporter: KANG-SEN LU
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.5.2
>
>
> Hi, ShaoFeng:
>
> Thanks for the reply. I missed this email and not responded earlier, I am
> sorry.
>
> I tried to reproduce this problem with the sample database, and it did not
> happen.
>
> So I am hoping by collecting enough "clue", someone can figure out why this
> problem occurred.
>
> --
> I issued the following query at the sample project to exercise the topn
> aggregation:
>
> select seller_id, SUM(price) as total from kylin_sales group by seller_id
> order by total limit 5;
>
> With my own added debugging, I saw the following log in the kylin.log: (the
> query worked OK).
>
> 2018-10-16 16:18:19,963 INFO [Query a747f16f-4b12-cc97-08d2-9b45c27a529f-90]
> model.FunctionDesc:59 : KSL2,
> getRewriteFieldName=_KY_SUM_KYLIN_SALES_PRICE_
> 2018-10-16 16:18:19,963 INFO [kylin-coproc--pool12-t1]
> v2.CubeHBaseEndpointRPC:217 : Query-a747f16f-4b12-cc97-08d2-9b45c27a529f:
> send request to the init region server anovadata4.anovadata.local on table
> ANOVA_KYLIN_25X_K758MEAWJG
> 2018-10-16 16:18:19,963 INFO [Query a747f16f-4b12-cc97-08d2-9b45c27a529f-90]
> topn.TopNMeasureType:399 : KSL888: in TopNMeasureType.java, sumFieldName=
> _KY_SUM_KYLIN_SALES_PRICE_
>
>
> When I was executing my project query, I issued the following select
> statement:
>
> SELECT ZETTICSDW.A_VL_HOURLY_V.IMSIID \"ZETTICSDW_A_VL_HOURLY_V_IMSIID\",
> SUM(ZETTICSDW.A_VL_HOURLY_V.SIG_EVENT_COUNT)
> \"vl_aggs_model___USERS_BY_ERROR_3XX\" FROM ZETTICSDW.A_VL_HOURLY_V inner
> JOIN ZETTICSDW.T_VL_TRANSACTION_RULE_V ON (
> ZETTICSDW.A_VL_HOURLY_V.CAUSE_CODE_KEY =
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.CAUSE_CODE_KEY AND
> ZETTICSDW.A_VL_HOURLY_V.REASON_CODE_KEY =
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.REASON_CODE_KEY AND
> ZETTICSDW.A_VL_HOURLY_V.TRANSACTION_TYPE_KEY =
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.TRANSACTION_TYPE_KEY)
> WHERE ((ZETTICSDW.A_VL_HOURLY_V.THEDATE = '20180209') AND
> ((ZETTICSDW.A_VL_HOURLY_V.THEHOUR >= '02') AND
> (ZETTICSDW.A_VL_HOURLY_V.THEHOUR <= '03'))) AND
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.DISPLAY_STRING LIKE '%+3%'
> GROUP BY ZETTICSDW.A_VL_HOURLY_V.IMSIID
> ORDER BY \"vl_aggs_model___USERS_BY_ERROR_3XX\"
> LIMIT 25
>
> An exception occurred within the method "private ColumnRowType
> buildColumnRowType()" of
> "query/src/main/java/org/apache/kylin/query/relnode/OLAPTableScan.java".
>
> if (columns.size() != rowType.getFieldCount()) {
> throw new IllegalStateException("RowType=" +
> rowType.getFieldCount() + ", ColumnRowType=" + columns.size());
> }
>
> It printed "RowType=133, ColumnRowType=132".
> The RowType list contains one extra column name: "ANY
> _KY_SUM_1_3a1aedef_SIG_EVENT_COUNT_".
>
>
> I think this has something to do with the bug fix "KYLIN-3359 Support
> sum(expression) if possible".
>
> After this bug fix was submitted, I noticed that a lot of column name was
> added into rowType like "_KY_SUM_XXX".
>
> This strange column name "_KY_SUM_1_3a1aedef_SIG_EVENT_COUNT_" is very
> similar.
>
> I also found that this extra column name, only existed in RowType but not in
> ColumnRowType, was added in the method "public void
> implementRewrite(RewriteImplementor implementor)" within "OLAPJoinRel.java".
>
> With my own debug statement, I saw this debug text in kylin.log:
>
> relnode.OLAPJoinRel:362 : KSL54: newField= #132:
> _KY_SUM_1_6735969a_SIG_EVENT_COUNT_ ANY
>
>
> I hope someone with a deeper technical knowledge in kylin query engine can
> figure out what was causing the problem I have seen.
>
> Thanks again.
>
> Kang-sen
>
> *From:* ShaoFeng Shi [[mailto:shaofeng...@apache.org]]
> *Sent:* Friday, October 05, 2018 9:59 PM
> *To:* user <[u...@kylin.apache.org|mailto:u...@kylin.apache.org]>
> *Subject:* Re: any body see topn in kylin 2.5.1 working?
>
> Hi Kang-Sen,
>
> Didn't see this; Can you reproduce the problem with the sample cube?
>
> Kang-Sen Lu <[k...@anovadata.com|ma
[jira] [Updated] (KYLIN-3636) Wrong "storage_type" in CubeDesc causing cube building error
[
https://issues.apache.org/jira/browse/KYLIN-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3636:
Fix Version/s: (was: v2.6.0)
v2.5.2
Summary: Wrong "storage_type" in CubeDesc causing cube building error
(was: in kylin 2.4.1 and 2.5.1 topn aggregation query caused exception)
> Wrong "storage_type" in CubeDesc causing cube building error
>
>
> Key: KYLIN-3636
> URL: https://issues.apache.org/jira/browse/KYLIN-3636
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v2.4.1
>Reporter: KANG-SEN LU
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.5.2
>
>
> Hi, ShaoFeng:
>
> Thanks for the reply. I missed this email and not responded earlier, I am
> sorry.
>
> I tried to reproduce this problem with the sample database, and it did not
> happen.
>
> So I am hoping by collecting enough "clue", someone can figure out why this
> problem occurred.
>
> --
> I issued the following query at the sample project to exercise the topn
> aggregation:
>
> select seller_id, SUM(price) as total from kylin_sales group by seller_id
> order by total limit 5;
>
> With my own added debugging, I saw the following log in the kylin.log: (the
> query worked OK).
>
> 2018-10-16 16:18:19,963 INFO [Query a747f16f-4b12-cc97-08d2-9b45c27a529f-90]
> model.FunctionDesc:59 : KSL2,
> getRewriteFieldName=_KY_SUM_KYLIN_SALES_PRICE_
> 2018-10-16 16:18:19,963 INFO [kylin-coproc--pool12-t1]
> v2.CubeHBaseEndpointRPC:217 : Query-a747f16f-4b12-cc97-08d2-9b45c27a529f:
> send request to the init region server anovadata4.anovadata.local on table
> ANOVA_KYLIN_25X_K758MEAWJG
> 2018-10-16 16:18:19,963 INFO [Query a747f16f-4b12-cc97-08d2-9b45c27a529f-90]
> topn.TopNMeasureType:399 : KSL888: in TopNMeasureType.java, sumFieldName=
> _KY_SUM_KYLIN_SALES_PRICE_
>
>
> When I was executing my project query, I issued the following select
> statement:
>
> SELECT ZETTICSDW.A_VL_HOURLY_V.IMSIID \"ZETTICSDW_A_VL_HOURLY_V_IMSIID\",
> SUM(ZETTICSDW.A_VL_HOURLY_V.SIG_EVENT_COUNT)
> \"vl_aggs_model___USERS_BY_ERROR_3XX\" FROM ZETTICSDW.A_VL_HOURLY_V inner
> JOIN ZETTICSDW.T_VL_TRANSACTION_RULE_V ON (
> ZETTICSDW.A_VL_HOURLY_V.CAUSE_CODE_KEY =
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.CAUSE_CODE_KEY AND
> ZETTICSDW.A_VL_HOURLY_V.REASON_CODE_KEY =
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.REASON_CODE_KEY AND
> ZETTICSDW.A_VL_HOURLY_V.TRANSACTION_TYPE_KEY =
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.TRANSACTION_TYPE_KEY)
> WHERE ((ZETTICSDW.A_VL_HOURLY_V.THEDATE = '20180209') AND
> ((ZETTICSDW.A_VL_HOURLY_V.THEHOUR >= '02') AND
> (ZETTICSDW.A_VL_HOURLY_V.THEHOUR <= '03'))) AND
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.DISPLAY_STRING LIKE '%+3%'
> GROUP BY ZETTICSDW.A_VL_HOURLY_V.IMSIID
> ORDER BY \"vl_aggs_model___USERS_BY_ERROR_3XX\"
> LIMIT 25
>
> An exception occurred within the method "private ColumnRowType
> buildColumnRowType()" of
> "query/src/main/java/org/apache/kylin/query/relnode/OLAPTableScan.java".
>
> if (columns.size() != rowType.getFieldCount()) {
> throw new IllegalStateException("RowType=" +
> rowType.getFieldCount() + ", ColumnRowType=" + columns.size());
> }
>
> It printed "RowType=133, ColumnRowType=132".
> The RowType list contains one extra column name: "ANY
> _KY_SUM_1_3a1aedef_SIG_EVENT_COUNT_".
>
>
> I think this has something to do with the bug fix "KYLIN-3359 Support
> sum(expression) if possible".
>
> After this bug fix was submitted, I noticed that a lot of column name was
> added into rowType like "_KY_SUM_XXX".
>
> This strange column name "_KY_SUM_1_3a1aedef_SIG_EVENT_COUNT_" is very
> similar.
>
> I also found that this extra column name, only existed in RowType but not in
> ColumnRowType, was added in the method "public void
> implementRewrite(RewriteImplementor implementor)" within "OLAPJoinRel.java".
>
> With my own debug statement, I saw this debug text in kylin.log:
>
> relnode.OLAPJoinRel:362 : KSL54: newField= #132:
> _KY_SUM_1_6735969a_SIG_EVENT_COUNT_ ANY
>
>
> I hope someone with a deeper technical knowledge in kylin query engine can
> figure out what was causing the problem I have seen.
>
> Thanks again.
>
> Kang-sen
>
> *From:* ShaoFeng Shi [[mailto:shaofeng...@apache.org]]
> *Sent:* Friday, October 05, 2018 9:59 PM
> *To:* user <[u...@kylin.apache.org|mailto:u...@kylin.apache.org]>
> *Subject:* Re: any body see topn in kylin 2.5.1 working?
>
> Hi Kang-Sen,
>
> Didn't see this; Can you reproduce the problem with the sample cube?
>
> Kang-Sen Lu <[k...@anovadata.com|mailto:k...@anovadata.com]> 于2018年10月6日周六
> 上午12:21写道：
> I have been running kylin 2.2.
[jira] [Updated] (KYLIN-3672) Performance is poor when multiple queries occur in short period
[
https://issues.apache.org/jira/browse/KYLIN-3672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3672:
Fix Version/s: (was: v2.6.0)
v2.5.2
> Performance is poor when multiple queries occur in short period
> ---
>
> Key: KYLIN-3672
> URL: https://issues.apache.org/jira/browse/KYLIN-3672
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v2.5.0
> Environment: CentOS 6.7, HBase 1.2.0+cdh5.14.2+456
>Reporter: Zongwei Li
>Assignee: Zongwei Li
>Priority: Critical
> Labels: patch, performance
> Fix For: v2.5.2
>
> Attachments: KYLIN-3672.master.002.patch, TrendChartAfterFix.png,
> TrendChartBeforeFix.png, codeChangedCausedThisBug.png, jstackBeforeBugFix.log
>
>
> Hi, Kylin Team
> We found one Kylin performance bug during performance tuning for our BI
> report integrate with Kylin.
>
> +Background+
> Our BI report show customer usage report to enterprise customers, provide 15
> usage charts in report page.
> Each chart need send API request to Kylin with different SQLs. So it means
> for one user, it will trigger 15 API calls(by JDBC) to Kylin.
> For our product scale, we need support at least 20 users to review the report
> at same time for each Kylin query node.
> So it means each Kylin node should be able to handle 15 * 20 = 300 queries
> per second.
>
> +Performance Report+
> To reduce the network impact. We built up Kylin cluster and testing machine
> in the same network with Hadoop system.
> We use gatling and Jmeter tools to do several round testing, result as follow.
>
> |Thread|Handled Queries (in 60 seconds)|Handled Queries (per second)|Mean
> Response Time
> (ms)|
> |1|773|13|77|
> |15|3245|54|279|
> |25|3844|64|390|
> |50|4912|82|612|
> |75|5405|90|841|
> |100|5436|91|1108|
> |150|5434|91|1688|
>
> And draw the trend chart as follow:
> !TrendChartBeforeFix.png!
>
> +Conclusion+
> From the trend, when the thread count reach 75, the handled queries per
> second reaches peak data 90, and cannot improved by increase the thread count.
> Each Kylin query engine can handle 90 queries per second, it means only
> support 90/15 = 6 users to review report page at same time.
> Even we setup 3 query nodes, can extend to 18 users at same time, this
> performance capacity cannot meet our business requirement.
>
> +Analyze+
> From test result, response for one thread is fast, but as the thread
> increase, throughput of Kylin not increased as we expected.
> We have full code review for Kylin query engine, and use Jstack and JProfile
> to do analyze, found the root cause for this performance bottleneck.
> This is one regression bug introduced by new feature involved one year before.
> With bug fixing, one Kylin node can handle 350+ queries per second. Submit
> this bug for contribute patch to Kylin.
> +Jstack Log Analyze+
> We use Jstack to capture thread info during performance testing. Already
> attach one of them 'jstackBeforeBugFix.log'.
> From the log, we can found that
> One thread locked at sun.misc.URLClassPath.getNextLoader. TID is
> {color:#ff}*0x00048007a180*{color}
>
> {{"Query e9c44a2d-6226-ff3b-f984-ce8489107d79-3425"}} {{#}}{{3425}} {{daemon
> prio=}}{{5}} {{os_prio=}}{{0}} {{tid=}}{{0x0472b000}}
> {{nid=}}{{0x1433}} {{waiting }}{{for}} {{monitor entry
> [}}\\\{{0x7f272e40d000}}{{]}}
>
> {{ }}{{java.lang.Thread.State: BLOCKED (on object monitor)}}
>
> {{}}{{at
> sun.misc.URLClassPath.getNextLoader(URLClassPath.java:}}{{469}}{{)}}
>
> {{}}{{- locked <}}{{0x00048007a180}}{{> (a sun.misc.URLClassPath)}}
>
> {{}}{{at
> sun.misc.URLClassPath.findResource(URLClassPath.java:}}{{214}}{{)}}
>
> {{}}{{at
> java.net.URLClassLoader$}}{{2}}{{.run(URLClassLoader.java:}}{{569}}{{)}}
>
> {{}}{{at
> java.net.URLClassLoader$}}{{2}}{{.run(URLClassLoader.java:}}{{567}}{{)}}
>
> {{}}{{at java.security.AccessController.doPrivileged(Native Method)}}
>
> {{}}{{at
> java.net.URLClassLoader.findResource(URLClassLoader.java:}}{{566}}{{)}}
>
> {{}}{{at
> java.lang.ClassLoader.getResource(ClassLoader.java:}}{{1096}}{{)}}
>
> {{}}{{at
> java.lang.ClassLoader.getResource(ClassLoader.java:}}{{1091}}{{)}}
>
> {{}}{{at
> org.apache.catalina.loader.WebappClassLoaderBase.getResource(WebappClassLoaderBase.java:}}{{1666}}{{)}}
>
> {{}}{{at
> org.apache.kylin.common.KylinConfig.buildSiteOrderedProps(KylinConfig.java:}}{{338}}{{)}}
>
> 43 threads waiting to lock <{color:#ff}*0x00048007a180*{color}>
>
> {{"Query f1f0bbec-a3f7-04b2-1ac6-fd3e03a0232d-4002"}} {{#}}{{4002}} {{daemon
> prio=}}{{5}} {{os_prio=}}{{0}} {{tid=}}{
[jira] [Updated] (KYLIN-3676) Update custom Calctie
[ https://issues.apache.org/jira/browse/KYLIN-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3676: Fix Version/s: (was: v2.6.0) v2.5.2 > Update custom Calctie > - > > Key: KYLIN-3676 > URL: https://issues.apache.org/jira/browse/KYLIN-3676 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v2.4.0 >Reporter: Lijun Cao >Assignee: Lijun Cao >Priority: Major > Fix For: v2.5.2 > > > The Tomcat version is upgraded to 8.5 in 2.5.x-hadoop3.1 branch. However, due > to the mechanism of the class load in Tomcat 8.5 is different from Tomcat 7, > current custom Calcite got some issues when working on Tomcat 8.5 env. So it > should be upgrade to adapt to Tomcat 8.5. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3678) CacheStateChecker may remove a cache file that under building
[
https://issues.apache.org/jira/browse/KYLIN-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3678:
Fix Version/s: (was: v2.6.0)
v2.5.2
> CacheStateChecker may remove a cache file that under building
> -
>
> Key: KYLIN-3678
> URL: https://issues.apache.org/jira/browse/KYLIN-3678
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
>Affects Versions: v2.4.0, v2.4.1, v2.5.0, v2.5.1
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.5.2
>
>
> Kylin test failed with such error:
> {code:java}
> 2018-11-09 02:15:24,379 DEBUG [main] cachesync.CachedCrudAssist:127 : Loaded
> 1 ExternalFilterDesc(s) out of 1 resource
> 2018-11-09 02:15:24,380 WARN [main] common.KylinConfigBase:77 : KYLIN_HOME
> was not set
> 2018-11-09 02:15:24,380 INFO [main] cache.RocksDBLookupBuilder:66 : create
> new rocksdb
> folder:lookup_cache/rocksdb/DEFAULT.TEST_COUNTRY/f19bc17c-d41d-a4be-b561-f6bd275f4c90/db
> for table cache:DEFAULT.TEST_COUNTRY
> 2018-11-09 02:15:24,380 INFO [main] cache.RocksDBLookupBuilder:69 : start to
> build lookup table:DEFAULT.TEST_COUNTRY to rocks
> db:lookup_cache/rocksdb/DEFAULT.TEST_COUNTRY/f19bc17c-d41d-a4be-b561-f6bd275f4c90/db
> 2018-11-09 02:15:26,814 WARN [lookup-cache-state-checker-1]
> common.KylinConfigBase:77 : KYLIN_HOME was not set
> 2018-11-09 02:15:26,814 INFO [lookup-cache-state-checker-1]
> cache.RocksDBLookupTableCache:334 : check snapshot local cache state, local
> path:lookup_cache/rocksdb
> 2018-11-09 02:15:26,814 INFO [lookup-cache-state-checker-1]
> cache.RocksDBLookupTableCache:361 : removed cache
> file:/var/jenkins/workspace/kylin-manual-ci/core-dictionary/lookup_cache/rocksdb/DEFAULT.TEST_COUNTRY/f19bc17c-d41d-a4be-b561-f6bd275f4c90,
> it is not referred by any cube
> 2018-11-09 02:15:28,474 ERROR [main] cache.RocksDBLookupBuilder:77 : error
> when put data to rocksDB
> org.rocksdb.RocksDBException: While open a file for random read:
> lookup_cache/rocksdb/DEFAULT.TEST_COUNTRY/f19bc17c-d41d-a4be-b561-f6bd275f4c90/db/18.sst:
> No such file or directory
> at org.rocksdb.RocksDB.put(Native Method)
> at org.rocksdb.RocksDB.put(RocksDB.java:453)
> at
> org.apache.kylin.dict.lookup.cache.RocksDBLookupBuilder.build(RocksDBLookupBuilder.java:74)
> at
> org.apache.kylin.dict.lookup.cache.RocksDBLookupTableCacheTest.testRestoreCacheFromFiles(RocksDBLookupTableCacheTest.java:115)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(
[jira] [Commented] (KYLIN-3690) New streaming backend implementation
[ https://issues.apache.org/jira/browse/KYLIN-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692947#comment-16692947 ] Ma Gang commented on KYLIN-3690: Thanks Hubert, the PR should be ready around Nov 30th, and the following process will be: code review, test, and release, when it can be release depends on the readiness of the feature. > New streaming backend implementation > > > Key: KYLIN-3690 > URL: https://issues.apache.org/jira/browse/KYLIN-3690 > Project: Kylin > Issue Type: Sub-task >Reporter: Ma Gang >Assignee: Ma Gang >Priority: Major > > New streaming backend implementation -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (KYLIN-3675) Unknown host exception when building cube of create HTable steps.
[
https://issues.apache.org/jira/browse/KYLIN-3675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI closed KYLIN-3675.
---
Resolution: Fixed
Please upgrade to 2.5.1 or above.
> Unknown host exception when building cube of create HTable steps.
> -
>
> Key: KYLIN-3675
> URL: https://issues.apache.org/jira/browse/KYLIN-3675
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine, Storage - HBase
>Affects Versions: v2.5.0
>Reporter: Jacky Woo
>Priority: Major
> Attachments: hbase.hdfs.xml, kylin.properties
>
>
> Hi, all
>
> I had "UnknownHostException" when building cube. Below is stack trace:
>
> {panel:title=stack trace:}
> 2018-11-08 18:44:55,069 ERROR [Scheduler 321750220 Job
> 42a75dbe-4b37-bb8a-8361-0bab7bcea106-849] common.HadoopShellExecutable:65 :
> error execute
> HadoopShellExecutable{id=42a75dbe-4b37-bb8a-8361-0bab7bcea106-04, name=Create
> HTable, st
> ate=RUNNING}
> org.apache.hadoop.hbase.DoNotRetryIOException:
> org.apache.hadoop.hbase.DoNotRetryIOException: java.net.UnknownHostException:
> data-batch-hdfs Set hbase.table.sanity.checks to false at conf or table
> descriptor if you want to bypass san
> ity checks
> at
> org.apache.hadoop.hbase.master.HMaster.warnOrThrowExceptionForFailure(HMaster.java:1785)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1646)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1576)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:469)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:55682)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2183)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
> at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:236)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:254)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:150)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4313)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4305)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsyncV2(HBaseAdmin.java:768)
> at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:689)
> at
> org.apache.kylin.storage.hbase.steps.CubeHTableUtil.createHTable(CubeHTableUtil.java:107)
> at
> org.apache.kylin.storage.hbase.steps.CreateHTableJob.run(CreateHTableJob.java:120)
> at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92)
> at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:163)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:69)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:163)
> at
> org.apache.kylin.job.impl.threadpool.DistributedScheduler$JobRunner.run(DistributedScheduler.java:111)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {panel}
>
>
> I use kylin 2.5.0.cdh.5.7
> There are two hdfs in my deployment:
> "data-batch-hdfs" : for cube building
> "hbase-hdfs" : for hbase
> And I use "data-batch-hdfs" as defaultFS in core-site.xml.
>
> {color:#FF}*The error occurs at "Create HTable" step, but it is
> incidental not always.*{color}
>
> I did some research:
>
> 1. According the error stack trace, error is thrown from
> "CubeHTableUtil.createHTable" method.
> So I added some log to print HBase table desc and found coprocessor jar
> location is in "data-batch-hdfs" which is not expected.(coprocessor s
[jira] [Updated] (KYLIN-3669) Add log to GTStreamAggregateScanner
[
https://issues.apache.org/jira/browse/KYLIN-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiatao Tao updated KYLIN-3669:
--
Summary: Add log to GTStreamAggregateScanner (was: Add logs to
GTStreamAggregateScanner)
> Add log to GTStreamAggregateScanner
> ---
>
> Key: KYLIN-3669
> URL: https://issues.apache.org/jira/browse/KYLIN-3669
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Jiatao Tao
>Assignee: Jiatao Tao
>Priority: Trivial
>
> There's no log printing in GTStreamAggregateScanner, and you know nothing
> about whether you go to this path or not, it's pretty unfriendly for trouble
> shooting.
> Code in _SegmentCubeTupleIterator_.
> {code:java}
> boolean hasMultiplePartitions = records instanceof
> SortMergedPartitionResultIterator;
> if (hasMultiplePartitions && context.isStreamAggregateEnabled()) {
> ...
> // input records are ordered, leverage stream aggregator to produce
> possibly fewer records
> try (GTStreamAggregateScanner aggregator = new
> GTStreamAggregateScanner(inputScanner, scanRequest)) {
> result = aggregator.valuesIterator(gtDimsIdx, gtMetricsIdx);
> } catch (IOException ioe) {
> // implementation of close method of anonymous IGTScanner is empty,
> no way throw exception
> throw new IllegalStateException("IOException is not expected here.",
> ioe);
> }
> return result;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3697) check port availability when starts kylin instance
[ https://issues.apache.org/jira/browse/KYLIN-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692872#comment-16692872 ] May Zhou commented on KYLIN-3697: - As this issue is not that urgent, I would like to contribute a patch myself :) > check port availability when starts kylin instance > -- > > Key: KYLIN-3697 > URL: https://issues.apache.org/jira/browse/KYLIN-3697 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.1 >Reporter: May Zhou >Priority: Minor > > As I ran the script and it turned out with no error message, and I ran the > kylin.sh start script, the console output indicated that a kylin instance had > successfully started, but when I visit the :/kylin, I found > another service was running rather than kylin. > > I can only find the error message in the kylin.out, which is not so > user-friendly. > > Caused by: java.net.BindException: Address already in use (Bind failed) > at java.net.PlainSocketImpl.socketBind(Native Method) > at > java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387) > at java.net.ServerSocket.bind(ServerSocket.java:375) > I am wondering why check-env does not check the availability of port, and the > error message could be threw out more user-friendly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3697) check port availability when starts kylin instance
[ https://issues.apache.org/jira/browse/KYLIN-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] May Zhou updated KYLIN-3697: Priority: Minor (was: Critical) > check port availability when starts kylin instance > -- > > Key: KYLIN-3697 > URL: https://issues.apache.org/jira/browse/KYLIN-3697 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.1 >Reporter: May Zhou >Priority: Minor > > As I ran the script and it turned out with no error message, and I ran the > kylin.sh start script, the console output indicated that a kylin instance had > successfully started, but when I visit the :/kylin, I found > another service was running rather than kylin. > > I can only find the error message in the kylin.out, which is not so > user-friendly. > > Caused by: java.net.BindException: Address already in use (Bind failed) > at java.net.PlainSocketImpl.socketBind(Native Method) > at > java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387) > at java.net.ServerSocket.bind(ServerSocket.java:375) > I am wondering why check-env does not check the availability of port, and the > error message could be threw out more user-friendly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3559) Use Splitter for splitting String
[ https://issues.apache.org/jira/browse/KYLIN-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI reassigned KYLIN-3559: --- Assignee: Wu Bin (was: Na Zhai) Bin, please go ahead. Thanks for making Kylin better! > Use Splitter for splitting String > - > > Key: KYLIN-3559 > URL: https://issues.apache.org/jira/browse/KYLIN-3559 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Assignee: Wu Bin >Priority: Major > Fix For: v2.6.0 > > > See http://errorprone.info/bugpattern/StringSplitter for why Splitter is > preferred . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3559) Use Splitter for splitting String
[ https://issues.apache.org/jira/browse/KYLIN-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692856#comment-16692856 ] Wu Bin edited comment on KYLIN-3559 at 11/20/18 8:44 AM: - Is there anyone working on this issue? May I take a try? And one more question: I found org.apache.commons.lang.StringUtils.split are using in our code. Which splitter should we leverage? was (Author: skywind2006): Is there anyone working on this issue? May I take a try? > Use Splitter for splitting String > - > > Key: KYLIN-3559 > URL: https://issues.apache.org/jira/browse/KYLIN-3559 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Assignee: Na Zhai >Priority: Major > Fix For: v2.6.0 > > > See http://errorprone.info/bugpattern/StringSplitter for why Splitter is > preferred . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3697) check port availability when starts kylin instance
May Zhou created KYLIN-3697: --- Summary: check port availability when starts kylin instance Key: KYLIN-3697 URL: https://issues.apache.org/jira/browse/KYLIN-3697 Project: Kylin Issue Type: Bug Affects Versions: v2.5.1 Reporter: May Zhou As I ran the script and it turned out with no error message, and I ran the kylin.sh start script, the console output indicated that a kylin instance had successfully started, but when I visit the :/kylin, I found another service was running rather than kylin. I can only find the error message in the kylin.out, which is not so user-friendly. > Caused by: java.net.BindException: Address already in use (Bind failed) at java.net.PlainSocketImpl.socketBind(Native Method) at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387) at java.net.ServerSocket.bind(ServerSocket.java:375) I am wondering why check-env does not check the availability of port, and the error message could be threw out more user-friendly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3559) Use Splitter for splitting String
[ https://issues.apache.org/jira/browse/KYLIN-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692856#comment-16692856 ] Wu Bin commented on KYLIN-3559: --- Is there anyone working on this issue? May I take a try? > Use Splitter for splitting String > - > > Key: KYLIN-3559 > URL: https://issues.apache.org/jira/browse/KYLIN-3559 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Assignee: Na Zhai >Priority: Major > Fix For: v2.6.0 > > > See http://errorprone.info/bugpattern/StringSplitter for why Splitter is > preferred . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3290) Avoid calling Class#newInstance
[ https://issues.apache.org/jira/browse/KYLIN-3290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692817#comment-16692817 ] ASF subversion and git services commented on KYLIN-3290: Commit cede58b5325bdb8b17dd8f0f1d59f8e0ded52ceb in kylin's branch refs/heads/master from 武斌 [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=cede58b ] KYLIN-3290 Leverage getDecalredConstructor().newInstance() instead of newInstance() > Avoid calling Class#newInstance > --- > > Key: KYLIN-3290 > URL: https://issues.apache.org/jira/browse/KYLIN-3290 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Assignee: Wu Bin >Priority: Minor > Labels: jdk > Fix For: v2.6.0 > > > Class#newInstance is deprecated starting in Java 9 - > https://bugs.openjdk.java.net/browse/JDK-6850612 - because it may throw > undeclared checked exceptions. > The suggested replacement is getDeclaredConstructor().newInstance(), which > wraps the checked exceptions in InvocationException. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3290) Avoid calling Class#newInstance
[
https://issues.apache.org/jira/browse/KYLIN-3290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692816#comment-16692816
]
ASF GitHub Bot commented on KYLIN-3290:
---
shaofengshi closed pull request #351: KYLIN-3290 Leverage
getDecalredConstructor().newInstance() instead of newInstance()
URL: https://github.com/apache/kylin/pull/351
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/core-common/src/main/java/org/apache/kylin/common/util/ClassUtil.java
b/core-common/src/main/java/org/apache/kylin/common/util/ClassUtil.java
index cd78fe0e01..5019b21053 100644
--- a/core-common/src/main/java/org/apache/kylin/common/util/ClassUtil.java
+++ b/core-common/src/main/java/org/apache/kylin/common/util/ClassUtil.java
@@ -85,7 +85,7 @@ private static String forRenamedClass(String name) {
public static Object newInstance(String clz) {
try {
-return forName(clz, Object.class).newInstance();
+return forName(clz,
Object.class).getDeclaredConstructor().newInstance();
} catch (Exception e) {
throw new RuntimeException(e);
}
diff --git
a/core-dictionary/src/main/java/org/apache/kylin/dict/DictionaryInfoSerializer.java
b/core-dictionary/src/main/java/org/apache/kylin/dict/DictionaryInfoSerializer.java
index ee88ea7cb3..96d34a1439 100644
---
a/core-dictionary/src/main/java/org/apache/kylin/dict/DictionaryInfoSerializer.java
+++
b/core-dictionary/src/main/java/org/apache/kylin/dict/DictionaryInfoSerializer.java
@@ -21,6 +21,7 @@
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
+import java.lang.reflect.InvocationTargetException;
import org.apache.kylin.common.persistence.Serializer;
import org.apache.kylin.common.util.ClassUtil;
@@ -63,14 +64,19 @@ public DictionaryInfo deserialize(DataInputStream in)
throws IOException {
if (infoOnly == false) {
Dictionary dict;
try {
-dict = (Dictionary)
ClassUtil.forName(obj.getDictionaryClass(), Dictionary.class).newInstance();
+dict = (Dictionary)
ClassUtil.forName(obj.getDictionaryClass(),
Dictionary.class).getDeclaredConstructor().newInstance();
} catch (InstantiationException e) {
throw new RuntimeException(e);
} catch (IllegalAccessException e) {
throw new RuntimeException(e);
} catch (ClassNotFoundException e) {
throw new RuntimeException(e);
+} catch (NoSuchMethodException e) {
+throw new RuntimeException(e);
+} catch (InvocationTargetException e) {
+throw new RuntimeException(e);
}
+
dict.readFields(in);
obj.setDictionaryObject(dict);
}
diff --git
a/core-dictionary/src/main/java/org/apache/kylin/dict/DictionarySerializer.java
b/core-dictionary/src/main/java/org/apache/kylin/dict/DictionarySerializer.java
index 72dc9953c3..23f2b88608 100644
---
a/core-dictionary/src/main/java/org/apache/kylin/dict/DictionarySerializer.java
+++
b/core-dictionary/src/main/java/org/apache/kylin/dict/DictionarySerializer.java
@@ -41,7 +41,7 @@ private DictionarySerializer() {
try {
final DataInputStream dataInputStream = new
DataInputStream(inputStream);
final String type = dataInputStream.readUTF();
-final Dictionary dictionary = ClassUtil.forName(type,
Dictionary.class).newInstance();
+final Dictionary dictionary = ClassUtil.forName(type,
Dictionary.class).getDeclaredConstructor().newInstance();
dictionary.readFields(dataInputStream);
return dictionary;
} catch (Exception e) {
diff --git
a/core-dictionary/src/main/java/org/apache/kylin/dict/NumberDictionary.java
b/core-dictionary/src/main/java/org/apache/kylin/dict/NumberDictionary.java
index 1377e8ef13..8e5979289d 100644
--- a/core-dictionary/src/main/java/org/apache/kylin/dict/NumberDictionary.java
+++ b/core-dictionary/src/main/java/org/apache/kylin/dict/NumberDictionary.java
@@ -49,7 +49,7 @@ protected boolean isNullObjectForm(T value) {
@Override
protected void setConverterByName(String converterName) throws Exception {
converterName = "org.apache.kylin.dict.Number2BytesConverter";
-this.bytesConvert = ClassUtil.forName(converterName,
BytesConverter.class).newInstance();
+this.bytesConvert = ClassUtil.forName(converterName,
BytesConverter.class).getDeclaredConstructor().newInstance();
((Number2BytesConverter)this.bytesConvert).setMaxDigitsBeforeDecimalPoint(Number2BytesConverter.MAX_DIGITS
[GitHub] shaofengshi closed pull request #351: KYLIN-3290 Leverage getDecalredConstructor().newInstance() instead of newInstance()
shaofengshi closed pull request #351: KYLIN-3290 Leverage
getDecalredConstructor().newInstance() instead of newInstance()
URL: https://github.com/apache/kylin/pull/351
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/core-common/src/main/java/org/apache/kylin/common/util/ClassUtil.java
b/core-common/src/main/java/org/apache/kylin/common/util/ClassUtil.java
index cd78fe0e01..5019b21053 100644
--- a/core-common/src/main/java/org/apache/kylin/common/util/ClassUtil.java
+++ b/core-common/src/main/java/org/apache/kylin/common/util/ClassUtil.java
@@ -85,7 +85,7 @@ private static String forRenamedClass(String name) {
public static Object newInstance(String clz) {
try {
-return forName(clz, Object.class).newInstance();
+return forName(clz,
Object.class).getDeclaredConstructor().newInstance();
} catch (Exception e) {
throw new RuntimeException(e);
}
diff --git
a/core-dictionary/src/main/java/org/apache/kylin/dict/DictionaryInfoSerializer.java
b/core-dictionary/src/main/java/org/apache/kylin/dict/DictionaryInfoSerializer.java
index ee88ea7cb3..96d34a1439 100644
---
a/core-dictionary/src/main/java/org/apache/kylin/dict/DictionaryInfoSerializer.java
+++
b/core-dictionary/src/main/java/org/apache/kylin/dict/DictionaryInfoSerializer.java
@@ -21,6 +21,7 @@
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
+import java.lang.reflect.InvocationTargetException;
import org.apache.kylin.common.persistence.Serializer;
import org.apache.kylin.common.util.ClassUtil;
@@ -63,14 +64,19 @@ public DictionaryInfo deserialize(DataInputStream in)
throws IOException {
if (infoOnly == false) {
Dictionary dict;
try {
-dict = (Dictionary)
ClassUtil.forName(obj.getDictionaryClass(), Dictionary.class).newInstance();
+dict = (Dictionary)
ClassUtil.forName(obj.getDictionaryClass(),
Dictionary.class).getDeclaredConstructor().newInstance();
} catch (InstantiationException e) {
throw new RuntimeException(e);
} catch (IllegalAccessException e) {
throw new RuntimeException(e);
} catch (ClassNotFoundException e) {
throw new RuntimeException(e);
+} catch (NoSuchMethodException e) {
+throw new RuntimeException(e);
+} catch (InvocationTargetException e) {
+throw new RuntimeException(e);
}
+
dict.readFields(in);
obj.setDictionaryObject(dict);
}
diff --git
a/core-dictionary/src/main/java/org/apache/kylin/dict/DictionarySerializer.java
b/core-dictionary/src/main/java/org/apache/kylin/dict/DictionarySerializer.java
index 72dc9953c3..23f2b88608 100644
---
a/core-dictionary/src/main/java/org/apache/kylin/dict/DictionarySerializer.java
+++
b/core-dictionary/src/main/java/org/apache/kylin/dict/DictionarySerializer.java
@@ -41,7 +41,7 @@ private DictionarySerializer() {
try {
final DataInputStream dataInputStream = new
DataInputStream(inputStream);
final String type = dataInputStream.readUTF();
-final Dictionary dictionary = ClassUtil.forName(type,
Dictionary.class).newInstance();
+final Dictionary dictionary = ClassUtil.forName(type,
Dictionary.class).getDeclaredConstructor().newInstance();
dictionary.readFields(dataInputStream);
return dictionary;
} catch (Exception e) {
diff --git
a/core-dictionary/src/main/java/org/apache/kylin/dict/NumberDictionary.java
b/core-dictionary/src/main/java/org/apache/kylin/dict/NumberDictionary.java
index 1377e8ef13..8e5979289d 100644
--- a/core-dictionary/src/main/java/org/apache/kylin/dict/NumberDictionary.java
+++ b/core-dictionary/src/main/java/org/apache/kylin/dict/NumberDictionary.java
@@ -49,7 +49,7 @@ protected boolean isNullObjectForm(T value) {
@Override
protected void setConverterByName(String converterName) throws Exception {
converterName = "org.apache.kylin.dict.Number2BytesConverter";
-this.bytesConvert = ClassUtil.forName(converterName,
BytesConverter.class).newInstance();
+this.bytesConvert = ClassUtil.forName(converterName,
BytesConverter.class).getDeclaredConstructor().newInstance();
((Number2BytesConverter)this.bytesConvert).setMaxDigitsBeforeDecimalPoint(Number2BytesConverter.MAX_DIGITS_BEFORE_DECIMAL_POINT_LEGACY);
}
diff --git
a/core-dictionary/src/main/java/org/apache/kylin/dict/NumberDictionary2.java
b/core-dictionary/src/main/java/org/apache/kylin/dict/NumberDictionary2.java
index 0efe9362ca..24700ca48d 100644
--- a/core-dic
