[jira] [Assigned] (KYLIN-2732) Introduce base cuboid as a new input for cubing job
[ https://issues.apache.org/jira/browse/KYLIN-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong reassigned KYLIN-2732: - Assignee: Wang Ken (was: Zhong Yanghong) > Introduce base cuboid as a new input for cubing job > --- > > Key: KYLIN-2732 > URL: https://issues.apache.org/jira/browse/KYLIN-2732 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Wang Ken >Priority: Major > Fix For: v2.3.0 > > > If the design for an existing cube is not good, which means its prebuilt > cuboid set is not suitable for query, there should be a way to adjust this > cuboid set. This is phase two of cube planner, also called cube optimization. > Its data source should be base cuboid rather than the original data source, > like hive, for the original one may be changed. Thus for cubing jobs, like > cube optimization, we introduce a new input , base cuboid formatted by > sequence file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-2723) Introduce metrics collector for query & job metrics
[ https://issues.apache.org/jira/browse/KYLIN-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong reassigned KYLIN-2723: - Assignee: Wang Ken (was: Zhong Yanghong) > Introduce metrics collector for query & job metrics > --- > > Key: KYLIN-2723 > URL: https://issues.apache.org/jira/browse/KYLIN-2723 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Wang Ken >Priority: Major > Fix For: v2.3.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-2987) Add 'auto.purge=true' when creating intermediate hive table or redistribute a hive table
[ https://issues.apache.org/jira/browse/KYLIN-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-2987: Fix Version/s: (was: v2.3.0) > Add 'auto.purge=true' when creating intermediate hive table or redistribute a > hive table > > > Key: KYLIN-2987 > URL: https://issues.apache.org/jira/browse/KYLIN-2987 > Project: Kylin > Issue Type: Improvement >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Trivial > Attachments: APACHE-KYLIN-2987.patch > > > At kylin side, we can add auto.purge=true when creating intermediate table. > However, to make ‘auto.purge’ effective for “insert overwrite table”, we > still need one patch for hive. > https://issues.apache.org/jira/browse/HIVE-15880 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (KYLIN-2987) Add 'auto.purge=true' when creating intermediate hive table or redistribute a hive table
[ https://issues.apache.org/jira/browse/KYLIN-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI reopened KYLIN-2987: - I have the same quesiton as Kaige; As this is unclarified, keep this issue as open. > Add 'auto.purge=true' when creating intermediate hive table or redistribute a > hive table > > > Key: KYLIN-2987 > URL: https://issues.apache.org/jira/browse/KYLIN-2987 > Project: Kylin > Issue Type: Improvement >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Trivial > Attachments: APACHE-KYLIN-2987.patch > > > At kylin side, we can add auto.purge=true when creating intermediate table. > However, to make ‘auto.purge’ effective for “insert overwrite table”, we > still need one patch for hive. > https://issues.apache.org/jira/browse/HIVE-15880 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-2903) Support cardinality calculation for Hive view
[ https://issues.apache.org/jira/browse/KYLIN-2903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413257#comment-16413257 ] Shaofeng SHI commented on KYLIN-2903: - Hi Gang, what's the status of this issue? > Support cardinality calculation for Hive view > - > > Key: KYLIN-2903 > URL: https://issues.apache.org/jira/browse/KYLIN-2903 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Wang, Gang >Assignee: Wang, Gang >Priority: Minor > Attachments: > 0001-KYLIN-2903-support-cardinality-calculation-for-Hive-.patch > > > Currently, Kylin leverage HCatlog to calculate column cardinality for Hive > tables. While, HCatlog does not support Hive view actually. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3001) Fix the wrong Cache key issue
[
https://issues.apache.org/jira/browse/KYLIN-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI reassigned KYLIN-3001:
---
Assignee: Pan, Julian (was: liyang)
> Fix the wrong Cache key issue
> --
>
> Key: KYLIN-3001
> URL: https://issues.apache.org/jira/browse/KYLIN-3001
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v2.1.0
>Reporter: Pan, Julian
>Assignee: Pan, Julian
>Priority: Major
> Fix For: v2.3.0
>
> Attachments: KYLIN_3001_Cache_key_issue.patch
>
>
> Currently the cacheKey generate by SQLRequest getCacheKey:
> cacheKey = Lists.newArrayList(sql.replaceAll("\\s+", ""), project, offset,
> limit, acceptPartial,
> backdoorToggles);
> Which will remove all the space and break line.
> If I have a sql:
> select a, sum(c) from table
> --where a > 0 group by a;
> and another sql:
> select a, sum(c) from table
> --where a > 0
> group by a;
> The two sql will share same cache key.
> But the first sql should be break. If you query second sql, and then query
> the first again. It will return result.
> And another issue is if remove all space, which will mix a as b and aasb.
> I think the better way is just replace multiple space to one space
> Should I create a patch? Or just provide the fix:
> cacheKey = Lists.newArrayList(sql.replaceAll("[ ]", " "), project, offset,
> limit, acceptPartial,
> backdoorToggles);
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3293) FixedLenHexDimEnc return a wrong code length leads to cut bytes error.
[ https://issues.apache.org/jira/browse/KYLIN-3293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412996#comment-16412996 ] ASF GitHub Bot commented on KYLIN-3293: --- coveralls commented on issue #123: KYLIN-3293, fix FixedLenHexDimEnc that return a wrong code length lea… URL: https://github.com/apache/kylin/pull/123#issuecomment-375964186 ## Pull Request Test Coverage Report for [Build 3086](https://coveralls.io/builds/16165173) * **0** of **2** **(0.0%)** changed or added relevant lines in **1** file are covered. * **3** unchanged lines in **2** files lost coverage. * Overall coverage decreased (**-0.002%**) to **23.871%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-metadata/src/main/java/org/apache/kylin/dimension/FixedLenHexDimEnc.java](https://coveralls.io/builds/16165173/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdimension%2FFixedLenHexDimEnc.java#L277) | 0 | 2 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/16165173/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L129) | 1 | 68.5% | | [core-metadata/src/main/java/org/apache/kylin/dimension/FixedLenHexDimEnc.java](https://coveralls.io/builds/16165173/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdimension%2FFixedLenHexDimEnc.java#L278) | 2 | 82.95% | | Totals | [](https://coveralls.io/builds/16165173) | | :-- | --: | | Change from base [Build 3078](https://coveralls.io/builds/16139781): | -0.002% | | Covered Lines: | 14339 | | Relevant Lines: | 60069 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > FixedLenHexDimEnc return a wrong code length leads to cut bytes error. > -- > > Key: KYLIN-3293 > URL: https://issues.apache.org/jira/browse/KYLIN-3293 > Project: Kylin > Issue Type: Bug >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Major > > FixedLenHexDimEnc lost its byteLen when deserializing, so in > GTRecord#loadColumnsFromColumnBlocks, the byte after cutting are wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3293) FixedLenHexDimEnc return a wrong code length leads to cut bytes error.
[ https://issues.apache.org/jira/browse/KYLIN-3293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412995#comment-16412995 ] ASF GitHub Bot commented on KYLIN-3293: --- codecov-io commented on issue #123: KYLIN-3293, fix FixedLenHexDimEnc that return a wrong code length lea… URL: https://github.com/apache/kylin/pull/123#issuecomment-375964107 # [Codecov](https://codecov.io/gh/apache/kylin/pull/123?src=pr=h1) Report > Merging [#123](https://codecov.io/gh/apache/kylin/pull/123?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/a7a0baa319c8f609be0142e7098141f518b972fb?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `0%`. [](https://codecov.io/gh/apache/kylin/pull/123?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #123 +/- ## - Coverage 21.83% 21.83% -0.01% Complexity 3886 3886 Files 999 999 Lines 6006760069 +2 Branches 8675 8675 - Hits 1311713115 -2 - Misses4572945732 +3 - Partials 1221 1222 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/123?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...he/kylin/cube/gridtable/TrimmedCubeCodeSystem.java](https://codecov.io/gh/apache/kylin/pull/123/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2dyaWR0YWJsZS9UcmltbWVkQ3ViZUNvZGVTeXN0ZW0uamF2YQ==) | `62% <ø> (ø)` | `5 <0> (ø)` | :arrow_down: | | [.../org/apache/kylin/dimension/FixedLenHexDimEnc.java](https://codecov.io/gh/apache/kylin/pull/123/diff?src=pr=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vZGltZW5zaW9uL0ZpeGVkTGVuSGV4RGltRW5jLmphdmE=) | `82.17% <0%> (-1.3%)` | `22 <0> (ø)` | | | [.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://codecov.io/gh/apache/kylin/pull/123/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9UcmVlQ3Vib2lkU2NoZWR1bGVyLmphdmE=) | `64.56% <0%> (-0.79%)` | `0% <0%> (ø)` | | | [...g/apache/kylin/metadata/cachesync/Broadcaster.java](https://codecov.io/gh/apache/kylin/pull/123/diff?src=pr=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWV0YWRhdGEvY2FjaGVzeW5jL0Jyb2FkY2FzdGVyLmphdmE=) | `55.22% <0%> (-0.5%)` | `21% <0%> (ø)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/123?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/123?src=pr=footer). Last update [a7a0baa...1e5c7a6](https://codecov.io/gh/apache/kylin/pull/123?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > FixedLenHexDimEnc return a wrong code length leads to cut bytes error. > -- > > Key: KYLIN-3293 > URL: https://issues.apache.org/jira/browse/KYLIN-3293 > Project: Kylin > Issue Type: Bug >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Major > > FixedLenHexDimEnc lost its byteLen when deserializing, so in > GTRecord#loadColumnsFromColumnBlocks, the byte after cutting are wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3293) FixedLenHexDimEnc return a wrong code length leads to cut bytes error.
[ https://issues.apache.org/jira/browse/KYLIN-3293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412987#comment-16412987 ] ASF GitHub Bot commented on KYLIN-3293: --- tttMelody opened a new pull request #123: KYLIN-3293, fix FixedLenHexDimEnc that return a wrong code length lea… URL: https://github.com/apache/kylin/pull/123 …ds to cut bytes error. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > FixedLenHexDimEnc return a wrong code length leads to cut bytes error. > -- > > Key: KYLIN-3293 > URL: https://issues.apache.org/jira/browse/KYLIN-3293 > Project: Kylin > Issue Type: Bug >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Major > > FixedLenHexDimEnc lost its byteLen when deserializing, so in > GTRecord#loadColumnsFromColumnBlocks, the byte after cutting are wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3293) FixedLenHexDimEnc return a wrong code length leads to cut bytes error.
[ https://issues.apache.org/jira/browse/KYLIN-3293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412988#comment-16412988 ] ASF GitHub Bot commented on KYLIN-3293: --- asfgit commented on issue #123: KYLIN-3293, fix FixedLenHexDimEnc that return a wrong code length lea… URL: https://github.com/apache/kylin/pull/123#issuecomment-375963165 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > FixedLenHexDimEnc return a wrong code length leads to cut bytes error. > -- > > Key: KYLIN-3293 > URL: https://issues.apache.org/jira/browse/KYLIN-3293 > Project: Kylin > Issue Type: Bug >Reporter: jiatao.tao >Assignee: jiatao.tao >Priority: Major > > FixedLenHexDimEnc lost its byteLen when deserializing, so in > GTRecord#loadColumnsFromColumnBlocks, the byte after cutting are wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3311) Segments overlap error (refactor write conflict exception)
[ https://issues.apache.org/jira/browse/KYLIN-3311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyang updated KYLIN-3311: -- Summary: Segments overlap error (refactor write conflict exception) (was: Segments overlap error) > Segments overlap error (refactor write conflict exception) > -- > > Key: KYLIN-3311 > URL: https://issues.apache.org/jira/browse/KYLIN-3311 > Project: Kylin > Issue Type: Bug > Components: Metadata >Affects Versions: v2.3.0 >Reporter: xujing >Priority: Major > Labels: build > Attachments: Segments_Overlap_ErrorLog.txt > > > when "updateCubeWithRetry" method be called at first time , > line newSegs.validate();was passed . > then > cube = crud.save(cube);seem with error throw exception > write conflict to update cube at try 0 ,will retry... > while retry "updateCubeWithRetry" start > line newSegs.validate();was not passed . > throw exception > Segments overlap: [2018031800_2018031900] and > sales_order_channel[2018031800_201803 > 1900] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
