[jira] [Updated] (KYLIN-1418) Memory hungry cube should select LAYER and INMEM cubing smartly
[ https://issues.apache.org/jira/browse/KYLIN-1418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyang updated KYLIN-1418: -- Description: At the time of writing, all memory hungry cube goes to LAYER cubing, which is too simple. The concern of memory hungry measure mainly comes from CI env, where physical memory is very limited. In real production env, memory hungry measures are usually not a problem for INMEM cubing. Suggest check the sum of measure memory footprint instead of the existence of memory hungry measure. was:At the time of writing, all memory hungry cube goes to LAYER cubing, which is too simple. > Memory hungry cube should select LAYER and INMEM cubing smartly > --- > > Key: KYLIN-1418 > URL: https://issues.apache.org/jira/browse/KYLIN-1418 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.1 >Reporter: liyang > Labels: newbie > > At the time of writing, all memory hungry cube goes to LAYER cubing, which is > too simple. The concern of memory hungry measure mainly comes from CI env, > where physical memory is very limited. In real production env, memory hungry > measures are usually not a problem for INMEM cubing. > Suggest check the sum of measure memory footprint instead of the existence of > memory hungry measure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1418) Memory hungry cube should select LAYER and INMEM cubing smartly
[ https://issues.apache.org/jira/browse/KYLIN-1418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyang updated KYLIN-1418: -- Labels: newbie (was: ) > Memory hungry cube should select LAYER and INMEM cubing smartly > --- > > Key: KYLIN-1418 > URL: https://issues.apache.org/jira/browse/KYLIN-1418 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.1 >Reporter: liyang > Labels: newbie > > At the time of writing, all memory hungry cube goes to LAYER cubing, which is > too simple. The concern of memory hungry measure mainly comes from CI env, > where physical memory is very limited. In real production env, memory hungry > measures are usually not a problem for INMEM cubing. > Suggest check the sum of measure memory footprint instead of the existence of > memory hungry measure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1420) Query returns empty result on partition column's boundary condition
Shaofeng SHI created KYLIN-1420: --- Summary: Query returns empty result on partition column's boundary condition Key: KYLIN-1420 URL: https://issues.apache.org/jira/browse/KYLIN-1420 Project: Kylin Issue Type: Bug Components: Query Engine Affects Versions: v2.0 Reporter: Shaofeng SHI Assignee: Shaofeng SHI Fix For: v2.0 This issue is reported by an user. The cube has a "month_end_dt" column as the partition column. In the first segment, it built the data from 2015-01-01 to 2016-01-01; The second segment it built the data after 2016-01-01. The queries which doesn't have filtering condition works as expected, such as: select month_end_dt,count(*) from fact_table group by month_end_dt order by month_end_dt; The result set includes the records for each month; But when filter on the month_end_date='2015-12-31' it is returning 0 records: select month_end_dt, count(*) from fact_table where month_end_dt='2015-12-31' group by month_end_dt After doing some investigation, identify it is a bug in v2 query engine: In CubeSegmentScanner.getSegmentStartAndEnd(), it uses -1 as 'roundingFlag' for cubeSeg.getDateRangeEnd(), which may get a smaller value than the cubeSegment's end date; While in GTScanRangePlanner.newScanRange(), it still treat the value as a closed condition, that caused the query on boundary condition returns empty directly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1419) NullPointerException occurs when query from subqueries with order by
[ https://issues.apache.org/jira/browse/KYLIN-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-1419: Attachment: KYLIN-1419.patch > NullPointerException occurs when query from subqueries with order by > > > Key: KYLIN-1419 > URL: https://issues.apache.org/jira/browse/KYLIN-1419 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v1.2 >Reporter: nichunen >Assignee: liyang > Fix For: v1.3 > > Attachments: KYLIN-1419.patch > > > Context may not have table scan which is created by OLAPJoinRel, so it does > not have realization. In > org.apache.kylin.query.relnode.OLAPSortRel#findMeasure, it visits > "this.context.realization" directly, so makes the NullPointerException. For > example, query like "select a.LSTG_FORMAT_NAME from (select > LEAF_CATEG_ID,LSTG_FORMAT_NAME from KYLIN_SALES)a, (select > LEAF_CATEG_ID,PART_DT from KYLIN_SALES)b order by a.LSTG_FORMAT_NAME". > Attached is the patch to fix, added a null protection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1419) NullPointerException occurs when query from subqueries with order by
nichunen created KYLIN-1419: --- Summary: NullPointerException occurs when query from subqueries with order by Key: KYLIN-1419 URL: https://issues.apache.org/jira/browse/KYLIN-1419 Project: Kylin Issue Type: Bug Components: Query Engine Affects Versions: v1.2 Reporter: nichunen Assignee: liyang Fix For: v1.3 Context may not have table scan which is created by OLAPJoinRel, so it does not have realization. In org.apache.kylin.query.relnode.OLAPSortRel#findMeasure, it visits "this.context.realization" directly, so makes the NullPointerException. For example, query like "select a.LSTG_FORMAT_NAME from (select LEAF_CATEG_ID,LSTG_FORMAT_NAME from KYLIN_SALES)a, (select LEAF_CATEG_ID,PART_DT from KYLIN_SALES)b order by a.LSTG_FORMAT_NAME". Attached is the patch to fix, added a null protection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1418) Memory hungry cube should select LAYER and INMEM cubing smartly
liyang created KYLIN-1418: - Summary: Memory hungry cube should select LAYER and INMEM cubing smartly Key: KYLIN-1418 URL: https://issues.apache.org/jira/browse/KYLIN-1418 Project: Kylin Issue Type: Bug Affects Versions: v2.1 Reporter: liyang At the time of writing, all memory hungry cube goes to LAYER cubing, which is too simple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1417) TimedJsonStreamParser is case sensitive for message's property name
Shaofeng SHI created KYLIN-1417: --- Summary: TimedJsonStreamParser is case sensitive for message's property name Key: KYLIN-1417 URL: https://issues.apache.org/jira/browse/KYLIN-1417 Project: Kylin Issue Type: Bug Components: streaming Affects Versions: v2.0 Reporter: Shaofeng SHI Assignee: Shaofeng SHI Fix For: v2.0 TimedJsonStreamParser is case sensitive for message's property name, this is wrong; It should be case insensitive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1038) Job retry on error
[ https://issues.apache.org/jira/browse/KYLIN-1038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15146547#comment-15146547 ] Shaofeng SHI commented on KYLIN-1038: - https://github.com/apache/kylin/commit/68a31eb77a6a29131c891d6f049df795e35d1f5e > Job retry on error > -- > > Key: KYLIN-1038 > URL: https://issues.apache.org/jira/browse/KYLIN-1038 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v2.0 >Reporter: hongbin ma >Assignee: Shaofeng SHI > Fix For: v2.1 > > > streaming cube will incur a lot of cube mergings, and some of them might fail > due to cluster resource shortage. proper reties would help to reduce manual > maintainance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1038) Job retry on error
[ https://issues.apache.org/jira/browse/KYLIN-1038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-1038: Fix Version/s: v2.1 Component/s: Job Engine Summary: Job retry on error (was: Streaming merge job retry on error) Update the summary, as this is a general feature for all kinds of job, not just merge job. Introduce a new parameter "kylin.job.retry", with default value 0; The int value indicates the max. retries on a job step failure. > Job retry on error > -- > > Key: KYLIN-1038 > URL: https://issues.apache.org/jira/browse/KYLIN-1038 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v2.0 >Reporter: hongbin ma >Assignee: Shaofeng SHI > Fix For: v2.1 > > > streaming cube will incur a lot of cube mergings, and some of them might fail > due to cluster resource shortage. proper reties would help to reduce manual > maintainance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-1405) Aggregation group validation
[ https://issues.apache.org/jira/browse/KYLIN-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyang resolved KYLIN-1405. --- Resolution: Fixed > Aggregation group validation > > > Key: KYLIN-1405 > URL: https://issues.apache.org/jira/browse/KYLIN-1405 > Project: Kylin > Issue Type: Improvement > Components: General >Reporter: Chen Luwei >Assignee: Chen Luwei >Priority: Minor > Fix For: v2.1 > > Attachments: cubeDesc.patch > > > To add more validations on aggregation group during the loading of cubes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1405) Aggregation group validation
[ https://issues.apache.org/jira/browse/KYLIN-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyang updated KYLIN-1405: -- Fix Version/s: v2.1 > Aggregation group validation > > > Key: KYLIN-1405 > URL: https://issues.apache.org/jira/browse/KYLIN-1405 > Project: Kylin > Issue Type: Improvement > Components: General >Reporter: Chen Luwei >Assignee: Chen Luwei >Priority: Minor > Fix For: v2.1 > > Attachments: cubeDesc.patch > > > To add more validations on aggregation group during the loading of cubes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1405) Aggregation group validation
[ https://issues.apache.org/jira/browse/KYLIN-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15146439#comment-15146439 ] liyang commented on KYLIN-1405: --- Patch merged, followed with a small review commit. > Aggregation group validation > > > Key: KYLIN-1405 > URL: https://issues.apache.org/jira/browse/KYLIN-1405 > Project: Kylin > Issue Type: Improvement > Components: General >Reporter: Chen Luwei >Assignee: Chen Luwei >Priority: Minor > Attachments: cubeDesc.patch > > > To add more validations on aggregation group during the loading of cubes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
