[jira] [Commented] (KYLIN-4020) fix_length rowkey encode without sepecified length can be saved but cause CreateHTable step failed
[ https://issues.apache.org/jira/browse/KYLIN-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16852644#comment-16852644 ] Yuzhang QIU commented on KYLIN-4020: Same as https://issues.apache.org/jira/browse/KYLIN-3893. But I just fix this bug from frontend > fix_length rowkey encode without sepecified length can be saved but cause > CreateHTable step failed > -- > > Key: KYLIN-4020 > URL: https://issues.apache.org/jira/browse/KYLIN-4020 > Project: Kylin > Issue Type: Improvement > Components: Metadata >Affects Versions: v2.5.2 >Reporter: Yuzhang QIU >Priority: Major > > Hi dear team: > Just as title said. > Maybe there should have more strict check for advanced settings, I think. > How do you think about this? > If there already have same JIRA，please inform me and close this one. > > Best regards > > yuzhang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3924) Build cube on step 'build Dimension Dictionary ', the contents of non-dimensional columns are also checked and built
[ https://issues.apache.org/jira/browse/KYLIN-3924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16852618#comment-16852618 ] wangxiaojing edited comment on KYLIN-3924 at 5/31/19 3:42 AM: -- Hello ShaoFeng,[~Shaofengshi] I am sorry , I responded to you so late. The values of some fields in my lookup table are really large. For example, the memo field is a text type, but this may often be encountered in tables. Can we just check and build the dimension columns and the necessary metric column, and the others can be set as a null value? This way, on the one hand, it can avoid situations where tables due to large content fields (non-dimension fields) can not be used as lookup tables,on the other hand,it can reduce the time and size of the build dictionary. was (Author: wangxiaojing): Hello ShaoFeng, I am sorry , I responded to you so late. The values of some fields in my lookup table are really large. For example, the memo field is a text type, but this may often be encountered in tables. Can we just check and build the dimension columns and the necessary metric column, and the others can be set as a null value? This way, on the one hand, it can avoid situations where tables due to large content fields (non-dimension fields) can not be used as lookup tables,on the other hand,it can reduce the time and size of the build dictionary. > Build cube on step 'build Dimension Dictionary ', the contents of > non-dimensional columns are also checked and built > > > Key: KYLIN-3924 > URL: https://issues.apache.org/jira/browse/KYLIN-3924 > Project: Kylin > Issue Type: Bug > Components: Job Engine, Tools, Build and Test >Affects Versions: v2.6.1 >Reporter: wangxiaojing >Priority: Major > Labels: build, patch > Attachments: builderr.log > > > Hi all, > I use version 2.6.1 , when build cube on step 'build Dimension Dictionary' > for lookup tables ,it throws some exceptions,it seems checks the content size > of each field in the table while building Dimension Dictionary, and build all > fields in the table with a dictionary. > Suggestion: > 1. Large content fields may not be dimension columns, in fact, it is not > necessary to check the value size of non-dimension columns，only check > dimension columns. > 2. If the value of some fields in the dimension table is too long, the > exception is not clear, can not be located the problem column quickly. > 3. There is no need to build dictionaries for non-dimensional columns -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3924) Build cube on step 'build Dimension Dictionary ', the contents of non-dimensional columns are also checked and built
[ https://issues.apache.org/jira/browse/KYLIN-3924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16852618#comment-16852618 ] wangxiaojing commented on KYLIN-3924: - Hello ShaoFeng, I am sorry , I responded to you so late. The values of some fields in my lookup table are really large. For example, the memo field is a text type, but this may often be encountered in tables. Can we just check and build the dimension columns and the necessary metric column, and the others can be set as a null value? This way, on the one hand, it can avoid situations where tables due to large content fields (non-dimension fields) can not be used as lookup tables,on the other hand,it can reduce the time and size of the build dictionary. > Build cube on step 'build Dimension Dictionary ', the contents of > non-dimensional columns are also checked and built > > > Key: KYLIN-3924 > URL: https://issues.apache.org/jira/browse/KYLIN-3924 > Project: Kylin > Issue Type: Bug > Components: Job Engine, Tools, Build and Test >Affects Versions: v2.6.1 >Reporter: wangxiaojing >Priority: Major > Labels: build, patch > Attachments: builderr.log > > > Hi all, > I use version 2.6.1 , when build cube on step 'build Dimension Dictionary' > for lookup tables ,it throws some exceptions,it seems checks the content size > of each field in the table while building Dimension Dictionary, and build all > fields in the table with a dictionary. > Suggestion: > 1. Large content fields may not be dimension columns, in fact, it is not > necessary to check the value size of non-dimension columns，only check > dimension columns. > 2. If the value of some fields in the dimension table is too long, the > exception is not clear, can not be located the problem column quickly. > 3. There is no need to build dictionaries for non-dimensional columns -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-4022) when Adhoc Push Down then Unrecognized column type: DECIMAL(xx,xx)
[ https://issues.apache.org/jira/browse/KYLIN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16852584#comment-16852584 ] jinguowei commented on KYLIN-4022: -- and integer has same problem > when Adhoc Push Down then Unrecognized column type: DECIMAL(xx,xx) > -- > > Key: KYLIN-4022 > URL: https://issues.apache.org/jira/browse/KYLIN-4022 > Project: Kylin > Issue Type: Improvement >Reporter: jinguowei >Assignee: jinguowei >Priority: Major > Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-4022) when Adhoc Push Down then Unrecognized column type: DECIMAL(xx,xx)
[ https://issues.apache.org/jira/browse/KYLIN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jinguowei updated KYLIN-4022: - Attachment: screenshot-3.png > when Adhoc Push Down then Unrecognized column type: DECIMAL(xx,xx) > -- > > Key: KYLIN-4022 > URL: https://issues.apache.org/jira/browse/KYLIN-4022 > Project: Kylin > Issue Type: Improvement >Reporter: jinguowei >Assignee: jinguowei >Priority: Major > Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-4022) when Adhoc Push Down then Unrecognized column type: DECIMAL(xx,xx)
[ https://issues.apache.org/jira/browse/KYLIN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16852583#comment-16852583 ] jinguowei commented on KYLIN-4022: -- when xxx_cnm column type is decimal(38,6) hive> desc test.test_name; OK stat_dt string xxx_cnm decimal(38,6) dt string # Partition Information # col_name data_type dt string Time taken: 1.167 seconds === then push donw query result !screenshot-1.png! !screenshot-2.png! !screenshot-3.png! > when Adhoc Push Down then Unrecognized column type: DECIMAL(xx,xx) > -- > > Key: KYLIN-4022 > URL: https://issues.apache.org/jira/browse/KYLIN-4022 > Project: Kylin > Issue Type: Improvement >Reporter: jinguowei >Assignee: jinguowei >Priority: Major > Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-4022) when Adhoc Push Down then Unrecognized column type: DECIMAL(xx,xx)
[ https://issues.apache.org/jira/browse/KYLIN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jinguowei updated KYLIN-4022: - Attachment: screenshot-2.png > when Adhoc Push Down then Unrecognized column type: DECIMAL(xx,xx) > -- > > Key: KYLIN-4022 > URL: https://issues.apache.org/jira/browse/KYLIN-4022 > Project: Kylin > Issue Type: Improvement >Reporter: jinguowei >Assignee: jinguowei >Priority: Major > Attachments: screenshot-1.png, screenshot-2.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-4022) when Adhoc Push Down then Unrecognized column type: DECIMAL(xx,xx)
[ https://issues.apache.org/jira/browse/KYLIN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jinguowei updated KYLIN-4022: - Attachment: screenshot-1.png > when Adhoc Push Down then Unrecognized column type: DECIMAL(xx,xx) > -- > > Key: KYLIN-4022 > URL: https://issues.apache.org/jira/browse/KYLIN-4022 > Project: Kylin > Issue Type: Improvement >Reporter: jinguowei >Assignee: jinguowei >Priority: Major > Attachments: screenshot-1.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-4024) when ad-hoc Push Down by presto engine Unrecognized column type: INTEGER,TIME,VARBINARY
wangxiaojing created KYLIN-4024: --- Summary: when ad-hoc Push Down by presto engine Unrecognized column type: INTEGER,TIME,VARBINARY Key: KYLIN-4024 URL: https://issues.apache.org/jira/browse/KYLIN-4024 Project: Kylin Issue Type: Bug Components: Query Engine Affects Versions: v2.6.2, all, v3.0.0 Reporter: wangxiaojing Fix For: Future Hello , When do ad-hoc Push Down query by presto engine ,it throws Unrecognized column type,like INTEGER,TIME,VARBINARY 。 The field of the table that appears in the hive is defined as an int type. If the Presto query is used, the INTEGER type will be returned. At this time, kylin ad-hoc push down will not recognize this type. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-2620) Check for "ORDER BY LIMIT" clause when rewrite SUM query as TOPN
[ https://issues.apache.org/jira/browse/KYLIN-2620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851728#comment-16851728 ] ASF GitHub Bot commented on KYLIN-2620: --- nichunen commented on pull request #647: KYLIN-2620 TopN can't match when multi sort columns URL: https://github.com/apache/kylin/pull/647 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Check for "ORDER BY LIMIT" clause when rewrite SUM query as TOPN > > > Key: KYLIN-2620 > URL: https://issues.apache.org/jira/browse/KYLIN-2620 > Project: Kylin > Issue Type: Bug > Components: Measure - TopN >Reporter: Lin Tingmao >Assignee: Chao Long >Priority: Major > Fix For: v3.0.0, v2.6.3 > > Attachments: cube_desc.png, sql.png > > > When running the following query > select sum(measure) from table group by col_id > if there exists TOPN(measure, group by col_id) measure, > TopNMeasureType.isTopNCompatibleSum()will pass, so the SUM is rewritten > to TOPN. This confuses the user since they may expect a accurate result for > every distinct value of group by column(s). > Kylin should check if "ORDER BY col_id LIMIT topncapacity" is present in the > query to determine whether to rewrite. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-2620) Check for "ORDER BY LIMIT" clause when rewrite SUM query as TOPN
[ https://issues.apache.org/jira/browse/KYLIN-2620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851727#comment-16851727 ] ASF subversion and git services commented on KYLIN-2620: Commit 670c2a98a82a699481eceddb685168104d5b185c in kylin's branch refs/heads/master from chao long [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=670c2a9 ] KYLIN-2620 TopN can't match when multi sort columns > Check for "ORDER BY LIMIT" clause when rewrite SUM query as TOPN > > > Key: KYLIN-2620 > URL: https://issues.apache.org/jira/browse/KYLIN-2620 > Project: Kylin > Issue Type: Bug > Components: Measure - TopN >Reporter: Lin Tingmao >Assignee: Chao Long >Priority: Major > Fix For: v3.0.0, v2.6.3 > > Attachments: cube_desc.png, sql.png > > > When running the following query > select sum(measure) from table group by col_id > if there exists TOPN(measure, group by col_id) measure, > TopNMeasureType.isTopNCompatibleSum()will pass, so the SUM is rewritten > to TOPN. This confuses the user since they may expect a accurate result for > every distinct value of group by column(s). > Kylin should check if "ORDER BY col_id LIMIT topncapacity" is present in the > query to determine whether to rewrite. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] nichunen merged pull request #647: KYLIN-2620 TopN can't match when multi sort columns
nichunen merged pull request #647: KYLIN-2620 TopN can't match when multi sort columns URL: https://github.com/apache/kylin/pull/647 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (KYLIN-2620) Check for "ORDER BY LIMIT" clause when rewrite SUM query as TOPN
[ https://issues.apache.org/jira/browse/KYLIN-2620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-2620: Fix Version/s: (was: Future) v2.6.3 > Check for "ORDER BY LIMIT" clause when rewrite SUM query as TOPN > > > Key: KYLIN-2620 > URL: https://issues.apache.org/jira/browse/KYLIN-2620 > Project: Kylin > Issue Type: Bug > Components: Measure - TopN >Reporter: Lin Tingmao >Assignee: Chao Long >Priority: Major > Fix For: v3.0.0, v2.6.3 > > Attachments: cube_desc.png, sql.png > > > When running the following query > select sum(measure) from table group by col_id > if there exists TOPN(measure, group by col_id) measure, > TopNMeasureType.isTopNCompatibleSum()will pass, so the SUM is rewritten > to TOPN. This confuses the user since they may expect a accurate result for > every distinct value of group by column(s). > Kylin should check if "ORDER BY col_id LIMIT topncapacity" is present in the > query to determine whether to rewrite. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] ZhengshuaiPENG commented on a change in pull request #633: KYLIN-3994: StorageCleanupJob may delete cube id data of new built segment because of cube cache in CubeManager
ZhengshuaiPENG commented on a change in pull request #633: KYLIN-3994:
StorageCleanupJob may delete cube id data of new built segment because of cube
cache in CubeManager
URL: https://github.com/apache/kylin/pull/633#discussion_r288895268
##
File path:
server-base/src/main/java/org/apache/kylin/rest/job/MetadataCleanupJob.java
##
@@ -122,7 +122,7 @@ public MetadataCleanupJob(KylinConfig config) {
// exclude resources in use
Set activeResources = Sets.newHashSet();
-for (CubeInstance cube : cubeManager.listAllCubes()) {
+for (CubeInstance cube : cubeManager.reloadAndListAllCubes()) {
Review comment:
In fact here when getting the `toDeleteCandidates`, it filters candidates
who were generated before NEW_RESOURCE_THRESHOLD_MS (12 hours). I think don't
need to reload cube here. But it's OK
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (KYLIN-3893) Cube build failed for wrong row key column description
[
https://issues.apache.org/jira/browse/KYLIN-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851693#comment-16851693
]
ASF subversion and git services commented on KYLIN-3893:
Commit 475df34fc27969414936e59a5ee2efd23cf2bcbe in kylin's branch
refs/heads/master from Na Zhai
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=475df34 ]
KYLIN-3893 Add validation for the encoding length and the type of encoding
(#622)
* KYLIN-3893 Add validation for the encoding length and the type of encoding
* KYLIN-3893 Code review
> Cube build failed for wrong row key column description
> --
>
> Key: KYLIN-3893
> URL: https://issues.apache.org/jira/browse/KYLIN-3893
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: Future
>
>
> User created wrong RowKeyColDesc, eg,
> RowKeyColDesc\{column=MYSQL_FEEDBACK_USER_AUDIT.DATE,
> encoding=integer:undefined}
> which cause the cube build forever.
>
> {code:java}
> org.apache.kylin.engine.mr.exception.HadoopShellException:
> java.lang.NumberFormatException: For input string: "undefined" at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:580) at
> java.lang.Integer.parseInt(Integer.java:615) at
> org.apache.kylin.dimension.IntegerDimEnc$Factory.createDimensionEncoding(IntegerDimEnc.java:65)
> at
> org.apache.kylin.dimension.DimensionEncodingFactory.create(DimensionEncodingFactory.java:65)
> at org.apache.kylin.cube.kv.CubeDimEncMap.get(CubeDimEncMap.java:74)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMapFromRowCount(CubeStatsReader.java:206)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMap(CubeStatsReader.java:170)
> at
> org.apache.kylin.storage.hbase.steps.CreateHTableJob.run(CreateHTableJob.java:102)
> at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92) at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)result code:2 at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:73)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3893) Cube build failed for wrong row key column description
[
https://issues.apache.org/jira/browse/KYLIN-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851691#comment-16851691
]
ASF subversion and git services commented on KYLIN-3893:
Commit 475df34fc27969414936e59a5ee2efd23cf2bcbe in kylin's branch
refs/heads/master from Na Zhai
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=475df34 ]
KYLIN-3893 Add validation for the encoding length and the type of encoding
(#622)
* KYLIN-3893 Add validation for the encoding length and the type of encoding
* KYLIN-3893 Code review
> Cube build failed for wrong row key column description
> --
>
> Key: KYLIN-3893
> URL: https://issues.apache.org/jira/browse/KYLIN-3893
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: Future
>
>
> User created wrong RowKeyColDesc, eg,
> RowKeyColDesc\{column=MYSQL_FEEDBACK_USER_AUDIT.DATE,
> encoding=integer:undefined}
> which cause the cube build forever.
>
> {code:java}
> org.apache.kylin.engine.mr.exception.HadoopShellException:
> java.lang.NumberFormatException: For input string: "undefined" at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:580) at
> java.lang.Integer.parseInt(Integer.java:615) at
> org.apache.kylin.dimension.IntegerDimEnc$Factory.createDimensionEncoding(IntegerDimEnc.java:65)
> at
> org.apache.kylin.dimension.DimensionEncodingFactory.create(DimensionEncodingFactory.java:65)
> at org.apache.kylin.cube.kv.CubeDimEncMap.get(CubeDimEncMap.java:74)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMapFromRowCount(CubeStatsReader.java:206)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMap(CubeStatsReader.java:170)
> at
> org.apache.kylin.storage.hbase.steps.CreateHTableJob.run(CreateHTableJob.java:102)
> at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92) at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)result code:2 at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:73)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3893) Cube build failed for wrong row key column description
[
https://issues.apache.org/jira/browse/KYLIN-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851690#comment-16851690
]
ASF subversion and git services commented on KYLIN-3893:
Commit 475df34fc27969414936e59a5ee2efd23cf2bcbe in kylin's branch
refs/heads/master from Na Zhai
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=475df34 ]
KYLIN-3893 Add validation for the encoding length and the type of encoding
(#622)
* KYLIN-3893 Add validation for the encoding length and the type of encoding
* KYLIN-3893 Code review
> Cube build failed for wrong row key column description
> --
>
> Key: KYLIN-3893
> URL: https://issues.apache.org/jira/browse/KYLIN-3893
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: Future
>
>
> User created wrong RowKeyColDesc, eg,
> RowKeyColDesc\{column=MYSQL_FEEDBACK_USER_AUDIT.DATE,
> encoding=integer:undefined}
> which cause the cube build forever.
>
> {code:java}
> org.apache.kylin.engine.mr.exception.HadoopShellException:
> java.lang.NumberFormatException: For input string: "undefined" at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:580) at
> java.lang.Integer.parseInt(Integer.java:615) at
> org.apache.kylin.dimension.IntegerDimEnc$Factory.createDimensionEncoding(IntegerDimEnc.java:65)
> at
> org.apache.kylin.dimension.DimensionEncodingFactory.create(DimensionEncodingFactory.java:65)
> at org.apache.kylin.cube.kv.CubeDimEncMap.get(CubeDimEncMap.java:74)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMapFromRowCount(CubeStatsReader.java:206)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMap(CubeStatsReader.java:170)
> at
> org.apache.kylin.storage.hbase.steps.CreateHTableJob.run(CreateHTableJob.java:102)
> at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92) at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)result code:2 at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:73)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [kylin] nichunen merged pull request #622: KYLIN-3893 Add validation for the encoding length and the type of encoding
nichunen merged pull request #622: KYLIN-3893 Add validation for the encoding length and the type of encoding URL: https://github.com/apache/kylin/pull/622 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3893) Cube build failed for wrong row key column description
[
https://issues.apache.org/jira/browse/KYLIN-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851688#comment-16851688
]
ASF GitHub Bot commented on KYLIN-3893:
---
nichunen commented on pull request #622: KYLIN-3893 Add validation for the
encoding length and the type of encoding
URL: https://github.com/apache/kylin/pull/622
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Cube build failed for wrong row key column description
> --
>
> Key: KYLIN-3893
> URL: https://issues.apache.org/jira/browse/KYLIN-3893
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: Future
>
>
> User created wrong RowKeyColDesc, eg,
> RowKeyColDesc\{column=MYSQL_FEEDBACK_USER_AUDIT.DATE,
> encoding=integer:undefined}
> which cause the cube build forever.
>
> {code:java}
> org.apache.kylin.engine.mr.exception.HadoopShellException:
> java.lang.NumberFormatException: For input string: "undefined" at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:580) at
> java.lang.Integer.parseInt(Integer.java:615) at
> org.apache.kylin.dimension.IntegerDimEnc$Factory.createDimensionEncoding(IntegerDimEnc.java:65)
> at
> org.apache.kylin.dimension.DimensionEncodingFactory.create(DimensionEncodingFactory.java:65)
> at org.apache.kylin.cube.kv.CubeDimEncMap.get(CubeDimEncMap.java:74)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMapFromRowCount(CubeStatsReader.java:206)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMap(CubeStatsReader.java:170)
> at
> org.apache.kylin.storage.hbase.steps.CreateHTableJob.run(CreateHTableJob.java:102)
> at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92) at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)result code:2 at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:73)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [kylin] nichunen commented on a change in pull request #644: KYLIN-4004, fix table acl when use groups
nichunen commented on a change in pull request #644: KYLIN-4004, fix table acl
when use groups
URL: https://github.com/apache/kylin/pull/644#discussion_r288916499
##
File path:
core-metadata/src/main/java/org/apache/kylin/metadata/acl/TableACL.java
##
@@ -63,9 +64,24 @@ public String resourceName() {
Set tableBlackList = new
TreeSet<>(String.CASE_INSENSITIVE_ORDER);
tableBlackList.addAll(userTableBlackList.getTableBlackList(username));
//if user is in group, add group's black list
-for (String group : groups) {
-
tableBlackList.addAll(groupTableBlackList.getTableBlackList(group));
+List> tableBlackListForGroups = Lists.newArrayList();
Review comment:
Seems "tableBlackListForGroups" never used?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (KYLIN-3994) StorageCleanupJob may delete data of newly built segment because of cube cache in CubeManager
[ https://issues.apache.org/jira/browse/KYLIN-3994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851656#comment-16851656 ] Shaofeng SHI commented on KYLIN-3994: - I agree with Zhengshuai; The current PR only narrow down the possibility of reading a dirty data, not solve the problem totally. > StorageCleanupJob may delete data of newly built segment because of cube > cache in CubeManager > - > > Key: KYLIN-3994 > URL: https://issues.apache.org/jira/browse/KYLIN-3994 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.2 >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Major > Fix For: v2.6.3 > > > In our production cluster, we found that the cube id data of a new-built > segment is deleted by the StorageCleanupJob. > After checking the code of cleanUnusedHdfsFiles in StorageCleanupJob, we > found that there is a bug here: CubeManager read all cube meta in > initiation and cache it for later > listAllCubes operations, the metadata will be out of data after listing the > hdfs working dir. > So the working directory of a finished job may be deleted unexpectedly. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3994) StorageCleanupJob may delete data of newly built segment because of cube cache in CubeManager
[ https://issues.apache.org/jira/browse/KYLIN-3994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3994: Fix Version/s: v2.6.3 Summary: StorageCleanupJob may delete data of newly built segment because of cube cache in CubeManager (was: StorageCleanupJob may delete cube id data of new built segment because of cube cache in CubeManager) > StorageCleanupJob may delete data of newly built segment because of cube > cache in CubeManager > - > > Key: KYLIN-3994 > URL: https://issues.apache.org/jira/browse/KYLIN-3994 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.2 >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Major > Fix For: v2.6.3 > > > In our production cluster, we found that the cube id data of a new-built > segment is deleted by the StorageCleanupJob. > After checking the code of cleanUnusedHdfsFiles in StorageCleanupJob, we > found that there is a bug here: CubeManager read all cube meta in > initiation and cache it for later > listAllCubes operations, the metadata will be out of data after listing the > hdfs working dir. > So the working directory of a finished job may be deleted unexpectedly. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-4004) Multi user groups table permission error
[ https://issues.apache.org/jira/browse/KYLIN-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4004: Fix Version/s: (was: v2.6.1) v2.6.3 > Multi user groups table permission error > > > Key: KYLIN-4004 > URL: https://issues.apache.org/jira/browse/KYLIN-4004 > Project: Kylin > Issue Type: Bug > Components: Others >Affects Versions: all >Reporter: haifeng wang >Priority: Major > Fix For: v2.6.3 > > > When a user has multiple user groups, When collecting the blacklist of the > table, Using the union of the blacklists of each group -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] asfgit commented on issue #667: KYLIN-4032 return timestamp and date without timezone in jdbc
asfgit commented on issue #667: KYLIN-4032 return timestamp and date without timezone in jdbc URL: https://github.com/apache/kylin/pull/667#issuecomment-497246811 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] chenzhx opened a new pull request #667: KYLIN-4032 return timestamp and date without timezone in jdbc
chenzhx opened a new pull request #667: KYLIN-4032 return timestamp and date without timezone in jdbc URL: https://github.com/apache/kylin/pull/667 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (KYLIN-4023) timestamp and date with timezone in jdbc
Zhixiong Chen created KYLIN-4023: Summary: timestamp and date with timezone in jdbc Key: KYLIN-4023 URL: https://issues.apache.org/jira/browse/KYLIN-4023 Project: Kylin Issue Type: Bug Components: Driver - JDBC Reporter: Zhixiong Chen Assignee: Zhixiong Chen query with timestamp or date type column the result with timezone in jdbc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3994) StorageCleanupJob may delete cube id data of new built segment because of cube cache in CubeManager
[ https://issues.apache.org/jira/browse/KYLIN-3994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851618#comment-16851618 ] PENG Zhengshuai commented on KYLIN-3994: Reload cube list is a solution. But I think a more safety solution is to add NEW-BORN resource protection everywhere. That means when getting the candidates to delete, filter them if they are NEW-BORN resources (just generated very recently) > StorageCleanupJob may delete cube id data of new built segment because of > cube cache in CubeManager > --- > > Key: KYLIN-3994 > URL: https://issues.apache.org/jira/browse/KYLIN-3994 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.2 >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Major > > In our production cluster, we found that the cube id data of a new-built > segment is deleted by the StorageCleanupJob. > After checking the code of cleanUnusedHdfsFiles in StorageCleanupJob, we > found that there is a bug here: CubeManager read all cube meta in > initiation and cache it for later > listAllCubes operations, the metadata will be out of data after listing the > hdfs working dir. > So the working directory of a finished job may be deleted unexpectedly. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] ZhengshuaiPENG commented on a change in pull request #633: KYLIN-3994: StorageCleanupJob may delete cube id data of new built segment because of cube cache in CubeManager
ZhengshuaiPENG commented on a change in pull request #633: KYLIN-3994:
StorageCleanupJob may delete cube id data of new built segment because of cube
cache in CubeManager
URL: https://github.com/apache/kylin/pull/633#discussion_r288895567
##
File path: core-cube/src/main/java/org/apache/kylin/cube/CubeManager.java
##
@@ -188,6 +188,11 @@ public void onEntityChange(Broadcaster broadcaster,
String entity, Event event,
}
}
+public List reloadAndListAllCubes() throws IOException {
+crud.reloadAll();
Review comment:
It's better to add lock here to avoid W/R conflict
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] ZhengshuaiPENG commented on a change in pull request #633: KYLIN-3994: StorageCleanupJob may delete cube id data of new built segment because of cube cache in CubeManager
ZhengshuaiPENG commented on a change in pull request #633: KYLIN-3994:
StorageCleanupJob may delete cube id data of new built segment because of cube
cache in CubeManager
URL: https://github.com/apache/kylin/pull/633#discussion_r288895268
##
File path:
server-base/src/main/java/org/apache/kylin/rest/job/MetadataCleanupJob.java
##
@@ -122,7 +122,7 @@ public MetadataCleanupJob(KylinConfig config) {
// exclude resources in use
Set activeResources = Sets.newHashSet();
-for (CubeInstance cube : cubeManager.listAllCubes()) {
+for (CubeInstance cube : cubeManager.reloadAndListAllCubes()) {
Review comment:
In fact here when getting the `toDeleteCandidates`, it filters at candidates
who were generated before NEW_RESOURCE_THRESHOLD_MS (12 hours). I think don't
need to reload cube here. But it's OK
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (KYLIN-3913) Remove getAllOutputs api in ExecutableManager to avoid OOM for large metadata
[ https://issues.apache.org/jira/browse/KYLIN-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851609#comment-16851609 ] ASF GitHub Bot commented on KYLIN-3913: --- ZhengshuaiPENG commented on pull request #630: #KYLIN-3913 Remove getAllOutputs api in ExecutableManager to avoid OOM for large metadata URL: https://github.com/apache/kylin/pull/630 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Remove getAllOutputs api in ExecutableManager to avoid OOM for large metadata > - > > Key: KYLIN-3913 > URL: https://issues.apache.org/jira/browse/KYLIN-3913 > Project: Kylin > Issue Type: Improvement >Reporter: Liu Shaohui >Assignee: PENG Zhengshuai >Priority: Major > > In a big cluster, there will be many job info left in the metadata. The kylin > server will be OOM when search the jobs with a long time range. The reason is > that ExecutableManager will load all job output info into memory when search > a job. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] ZhengshuaiPENG closed pull request #630: #KYLIN-3913 Remove getAllOutputs api in ExecutableManager to avoid OOM for large metadata
ZhengshuaiPENG closed pull request #630: #KYLIN-3913 Remove getAllOutputs api in ExecutableManager to avoid OOM for large metadata URL: https://github.com/apache/kylin/pull/630 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] ZhengshuaiPENG commented on issue #630: #KYLIN-3913 Remove getAllOutputs api in ExecutableManager to avoid OOM for large metadata
ZhengshuaiPENG commented on issue #630: #KYLIN-3913 Remove getAllOutputs api in ExecutableManager to avoid OOM for large metadata URL: https://github.com/apache/kylin/pull/630#issuecomment-497240632 Not a good design, close it for now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3980) Cube planner cuboid id error when dimensions too many
[ https://issues.apache.org/jira/browse/KYLIN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851607#comment-16851607 ] ASF subversion and git services commented on KYLIN-3980: Commit b08935205a7aa1a00b6c09ef4625f45eacf9b013 in kylin's branch refs/heads/master from liapan [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=b089352 ] KYLIN-3980 Cube planner cuboid id error when dimensions too many > Cube planner cuboid id error when dimensions too many > - > > Key: KYLIN-3980 > URL: https://issues.apache.org/jira/browse/KYLIN-3980 > Project: Kylin > Issue Type: Bug >Reporter: Pan, Julian >Assignee: Pan, Julian >Priority: Major > Fix For: v2.6.3 > > > Cube planner cuboid id error when dimensions beyond too many. > The issue due to cuboid is the Number and Number in javascript max value is > 2^53-1. > [https://www.ecma-international.org/ecma-262/5.1/#sec-8.5] > If the cuboid id is greater than Number.MAX_SAFE_INTEGER(9007199254740991), > then it will get error cuboid in page, which impact optimize job. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3980) Cube planner cuboid id error when dimensions too many
[ https://issues.apache.org/jira/browse/KYLIN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851606#comment-16851606 ] ASF GitHub Bot commented on KYLIN-3980: --- nichunen commented on pull request #628: KYLIN-3980 Cube planner cuboid id error when dimensions too many URL: https://github.com/apache/kylin/pull/628 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Cube planner cuboid id error when dimensions too many > - > > Key: KYLIN-3980 > URL: https://issues.apache.org/jira/browse/KYLIN-3980 > Project: Kylin > Issue Type: Bug >Reporter: Pan, Julian >Assignee: Pan, Julian >Priority: Major > Fix For: v2.6.3 > > > Cube planner cuboid id error when dimensions beyond too many. > The issue due to cuboid is the Number and Number in javascript max value is > 2^53-1. > [https://www.ecma-international.org/ecma-262/5.1/#sec-8.5] > If the cuboid id is greater than Number.MAX_SAFE_INTEGER(9007199254740991), > then it will get error cuboid in page, which impact optimize job. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3980) Cube planner cuboid id error when dimensions too many
[ https://issues.apache.org/jira/browse/KYLIN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3980: Fix Version/s: v2.6.3 > Cube planner cuboid id error when dimensions too many > - > > Key: KYLIN-3980 > URL: https://issues.apache.org/jira/browse/KYLIN-3980 > Project: Kylin > Issue Type: Bug >Reporter: Pan, Julian >Assignee: Pan, Julian >Priority: Major > Fix For: v2.6.3 > > > Cube planner cuboid id error when dimensions beyond too many. > The issue due to cuboid is the Number and Number in javascript max value is > 2^53-1. > [https://www.ecma-international.org/ecma-262/5.1/#sec-8.5] > If the cuboid id is greater than Number.MAX_SAFE_INTEGER(9007199254740991), > then it will get error cuboid in page, which impact optimize job. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] nichunen merged pull request #628: KYLIN-3980 Cube planner cuboid id error when dimensions too many
nichunen merged pull request #628: KYLIN-3980 Cube planner cuboid id error when dimensions too many URL: https://github.com/apache/kylin/pull/628 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] coveralls edited a comment on issue #622: KYLIN-3893 Add validation for the encoding length and the type of encoding
coveralls edited a comment on issue #622: KYLIN-3893 Add validation for the encoding length and the type of encoding URL: https://github.com/apache/kylin/pull/622#issuecomment-487300031 ## Pull Request Test Coverage Report for [Build 4557](https://coveralls.io/builds/23688064) * **0** of **42** **(0.0%)** changed or added relevant lines in **10** files are covered. * **1393** unchanged lines in **41** files lost coverage. * Overall coverage increased (+**0.5%**) to **28.283%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-metadata/src/main/java/org/apache/kylin/dimension/BooleanDimEnc.java](https://coveralls.io/builds/23688064/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdimension%2FBooleanDimEnc.java#L66) | 0 | 1 | 0.0% | [core-metadata/src/main/java/org/apache/kylin/dimension/TimeDimEnc.java](https://coveralls.io/builds/23688064/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdimension%2FTimeDimEnc.java#L46) | 0 | 1 | 0.0% | [core-metadata/src/main/java/org/apache/kylin/dimension/FixedLenDimEnc.java](https://coveralls.io/builds/23688064/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdimension%2FFixedLenDimEnc.java#L57) | 0 | 4 | 0.0% | [core-metadata/src/main/java/org/apache/kylin/dimension/FixedLenHexDimEnc.java](https://coveralls.io/builds/23688064/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdimension%2FFixedLenHexDimEnc.java#L119) | 0 | 4 | 0.0% | [core-metadata/src/main/java/org/apache/kylin/dimension/IntDimEnc.java](https://coveralls.io/builds/23688064/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdimension%2FIntDimEnc.java#L61) | 0 | 4 | 0.0% | [core-metadata/src/main/java/org/apache/kylin/dimension/IntegerDimEnc.java](https://coveralls.io/builds/23688064/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdimension%2FIntegerDimEnc.java#L71) | 0 | 4 | 0.0% | [core-metadata/src/main/java/org/apache/kylin/dimension/OneMoreByteVLongDimEnc.java](https://coveralls.io/builds/23688064/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdimension%2FOneMoreByteVLongDimEnc.java#L66) | 0 | 4 | 0.0% | [core-metadata/src/main/java/org/apache/kylin/dimension/DimensionEncodingFactory.java](https://coveralls.io/builds/23688064/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdimension%2FDimensionEncodingFactory.java#L101) | 0 | 6 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/controller/CubeController.java](https://coveralls.io/builds/23688064/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fcontroller%2FCubeController.java#L630) | 0 | 6 | 0.0% | [core-metadata/src/main/java/org/apache/kylin/dimension/DateDimEnc.java](https://coveralls.io/builds/23688064/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdimension%2FDateDimEnc.java#L51) | 0 | 8 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTableCache.java](https://coveralls.io/builds/23688064/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTableCache.java#L337) | 1 | 82.05% | | [server-base/src/main/java/org/apache/kylin/rest/controller/CubeController.java](https://coveralls.io/builds/23688064/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fcontroller%2FCubeController.java#L639) | 1 | 0.0% | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/23688064/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L553) | 1 | 78.12% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/steps/MergeStatisticsWithOldStep.java](https://coveralls.io/builds/23688064/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fsteps%2FMergeStatisticsWithOldStep.java#L123) | 1 | 0.0% | | [core-metadata/src/main/java/org/apache/kylin/measure/MeasureTypeFactory.java](https://coveralls.io/builds/23688064/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmeasure%2FMeasureTypeFactory.java#L130) | 2 | 66.33% | | [storage-hbase/src/main/java/org/apache/kylin/storage/hbase/cube/v2/CubeHBaseEndpointRPC.java](https://coveralls.io/builds/23688064/source?filename=storage-hbase%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstorage%2Fhbase%2Fcube%2Fv2%2FCubeHBaseEndpointRPC.java#L568) | 3 | 7.91% | | [core-common/src/main/java/org/apache/kylin/common/util/HiveCmdBuilder.java](https://coveralls.io/builds/23688064/source?f
