[jira] [Commented] (KYLIN-3882) kylin master build failed for pom issues
[
https://issues.apache.org/jira/browse/KYLIN-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836954#comment-16836954
]
nichunen commented on KYLIN-3882:
-
Committed to 2.6.x branch
> kylin master build failed for pom issues
>
>
> Key: KYLIN-3882
> URL: https://issues.apache.org/jira/browse/KYLIN-3882
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Major
> Fix For: v2.6.2
>
>
> As title.
> 1, Kyligence repo id : nexus conflicts with local maven settings.xml
> {code:java}
> [ERROR] Failed to execute goal on project kylin-core-metadata: Could not
> resolve dependencies for project
> org.apache.kylin:kylin-core-metadata:jar:3.0.0-SNAPSHOT: Failure to find
> org.apache.calcite:calcite-core:jar:1.16.0-kylin-r2 in
> http://nexus.x./nexus/content/groups/public was cached in the local
> repository, resolution will not be reattempted until the update interval of
> nexus has elapsed or updates are forced -> [Help 1]
> {code}
>
> 2, maven.compiler.source/target is not set
> {code:java}
> [INFO] Compiling 2 Scala sources and 18 Java sources to
> /ssd/liushaohui/workspace/computing/kylin/engine-spark/target/classes ...
> [WARNING] [Warn] : bootstrap class path not set in conjunction with -source
> 1.6
> [ERROR] [Error]
> /ssd/liushaohui/workspace/computing/kylin/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkBatchCubingJobBuilder2.java:148:
> diamond operator is not supported in -source 1.6
> (use -source 7 or higher to enable diamond operator)
> [ERROR] [Error]
> /ssd/liushaohui/workspace/computing/kylin/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkCubingByLayer.java:239:
> try-with-resources is not supported in -source 1.6
> (use -source 7 or higher to enable try-with-resources)
> [ERROR] [Error]
> /ssd/liushaohui/workspace/computing/kylin/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkCubingByLayer.java:251:
> diamond operator is not supported in -source 1.6
> (use -source 7 or higher to enable diamond operator){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3571) Not build Spark in Kylin's binary package
[ https://issues.apache.org/jira/browse/KYLIN-3571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836952#comment-16836952 ] nichunen commented on KYLIN-3571: - Committed to 2.6.x branch > Not build Spark in Kylin's binary package > - > > Key: KYLIN-3571 > URL: https://issues.apache.org/jira/browse/KYLIN-3571 > Project: Kylin > Issue Type: Improvement > Components: Environment >Reporter: Shaofeng SHI >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3830) Wrong result when 'SELECT SUM(dim1)' without set a relative metric of dim1.
[ https://issues.apache.org/jira/browse/KYLIN-3830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836947#comment-16836947 ] nichunen commented on KYLIN-3830: - Committed to 2.6.x branch > Wrong result when 'SELECT SUM(dim1)' without set a relative metric of dim1. > --- > > Key: KYLIN-3830 > URL: https://issues.apache.org/jira/browse/KYLIN-3830 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.2 >Reporter: Yuzhang QIU >Assignee: Yuzhang QIU >Priority: Major > Fix For: v2.6.2, v3.0.0-alpha > > > Hi, dear team: > I design an cube1 based on table table1 with dim1, dim2, dim3 and only > one metric count(1), and 'SELECT SUM(dim1) FROM table1 group by dim2', Kylin > process this SQL and return some result1. It seems ok. But as we know, Kylin > don't store the detail data, the dimensions' members have been encoded and > stored in Hbase as rowkey(cause I don't set any metric with an column). So, > is the result1 right? > Then, I clone cube1 to cube2, and set a metric SUM(dim1). the same SQL has > been passed to kylin and got result2. It's different from result1 at the > aggregation field. I also pass same SQL to hive and got result3, it's same > with result2. > Yes, I turn off the pushdown. > I think there are some problems. > I can't upload some picture of results for secret policy, sorry for that. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3817) Duration in Cube building is a negative number
[ https://issues.apache.org/jira/browse/KYLIN-3817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836953#comment-16836953 ] nichunen commented on KYLIN-3817: - Committed to 2.6.x branch > Duration in Cube building is a negative number > -- > > Key: KYLIN-3817 > URL: https://issues.apache.org/jira/browse/KYLIN-3817 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.6.0 >Reporter: Na Zhai >Assignee: Liu Shaohui >Priority: Major > Fix For: v2.6.2, v3.0.0-alpha > > Attachments: Duration error.png > > > After building the cube for a few minutes, click the Refresh button and the > duration of the job becomes negative. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (KYLIN-3571) Not build Spark in Kylin's binary package
[ https://issues.apache.org/jira/browse/KYLIN-3571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3571: Comment: was deleted (was: Committed to 2.6.x branch) > Not build Spark in Kylin's binary package > - > > Key: KYLIN-3571 > URL: https://issues.apache.org/jira/browse/KYLIN-3571 > Project: Kylin > Issue Type: Improvement > Components: Environment >Reporter: Shaofeng SHI >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3866) Whether to set mapreduce.application.classpath is determined by the user
[ https://issues.apache.org/jira/browse/KYLIN-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836951#comment-16836951 ] nichunen commented on KYLIN-3866: - Committed to 2.6.x branch > Whether to set mapreduce.application.classpath is determined by the user > > > Key: KYLIN-3866 > URL: https://issues.apache.org/jira/browse/KYLIN-3866 > Project: Kylin > Issue Type: Bug >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > Fix For: v2.6.2, v3.0.0-alpha > > Attachments: KYLIN-3866.master.001.patch > > > In the kylin environment, if there is no configuration parameter > "mapreduce.application.classpath", then kylin will use the command "mapred > classpath" to get the local classpath and set it to > "mapreduce.application.classpath", However, it will cause the following two > problems: > 1. The local classpath may not be the same as the MapReduce cluster. > 2. There is no need to configure the parameter > "mapreduce.application.classpath" on the client side in some cluster. > So, it may be a better choice for users to decide whether they need to > configure this parameter or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3571) Not build Spark in Kylin's binary package
[ https://issues.apache.org/jira/browse/KYLIN-3571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3571: Fix Version/s: (was: v2.6.1) v2.6.2 > Not build Spark in Kylin's binary package > - > > Key: KYLIN-3571 > URL: https://issues.apache.org/jira/browse/KYLIN-3571 > Project: Kylin > Issue Type: Improvement > Components: Environment >Reporter: Shaofeng SHI >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] hit-lacus edited a comment on issue #624: Allow user-specified time format using real-time
hit-lacus edited a comment on issue #624: Allow user-specified time format using real-time URL: https://github.com/apache/kylin/pull/624#issuecomment-491174708 Hi, could you please update your commit? @GuoNingNing 1. Use correct git user name/ user email which can link to your github profile page 2. Merge your commit into two commit, one for backend change, another for front end change 3. Create a JIRA, describe the problem you found, how you solve it, and attach manual test screen shot 4. Use KYLIN- in your git commit message Thank you so much~ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Reopened] (KYLIN-3571) Not build Spark in Kylin's binary package
[ https://issues.apache.org/jira/browse/KYLIN-3571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen reopened KYLIN-3571: - Reopen to reset fix version > Not build Spark in Kylin's binary package > - > > Key: KYLIN-3571 > URL: https://issues.apache.org/jira/browse/KYLIN-3571 > Project: Kylin > Issue Type: Improvement > Components: Environment >Reporter: Shaofeng SHI >Assignee: Chao Long >Priority: Major > Fix For: v2.6.1 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KYLIN-3571) Not build Spark in Kylin's binary package
[ https://issues.apache.org/jira/browse/KYLIN-3571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-3571. - Resolution: Fixed Committed to 2.6.x branch > Not build Spark in Kylin's binary package > - > > Key: KYLIN-3571 > URL: https://issues.apache.org/jira/browse/KYLIN-3571 > Project: Kylin > Issue Type: Improvement > Components: Environment >Reporter: Shaofeng SHI >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3818) After Cube disabled, auto-merge cube job still running
[ https://issues.apache.org/jira/browse/KYLIN-3818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836938#comment-16836938 ] ASF subversion and git services commented on KYLIN-3818: Commit d40f062a31851da4784f4d007192e305ce28b628 in kylin's branch refs/heads/2.6.x from GinaZhai [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=d40f062 ] KYLIN-3818 After Cube disabled, auto-merge cube job still running > After Cube disabled, auto-merge cube job still running > -- > > Key: KYLIN-3818 > URL: https://issues.apache.org/jira/browse/KYLIN-3818 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.6.0 >Reporter: Na Zhai >Assignee: Na Zhai >Priority: Major > Fix For: v3.0.0-alpha > > > *precondition* > There is a Cube that turns on the auto-merge feature. And it satisfied the > condition of the auto-merge. Then the job of merging segments begins. > After a few minutes, the job of merging segments goes into the error status, > so I discard the job. Then I disable this cube, but a new job of merging > segments begin to run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KYLIN-3818) After Cube disabled, auto-merge cube job still running
[ https://issues.apache.org/jira/browse/KYLIN-3818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-3818. - Resolution: Fixed Committed to 2.6.x branch > After Cube disabled, auto-merge cube job still running > -- > > Key: KYLIN-3818 > URL: https://issues.apache.org/jira/browse/KYLIN-3818 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.6.0 >Reporter: Na Zhai >Assignee: Na Zhai >Priority: Major > Fix For: v2.6.2, v3.0.0-alpha > > > *precondition* > There is a Cube that turns on the auto-merge feature. And it satisfied the > condition of the auto-merge. Then the job of merging segments begins. > After a few minutes, the job of merging segments goes into the error status, > so I discard the job. Then I disable this cube, but a new job of merging > segments begin to run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (KYLIN-3818) After Cube disabled, auto-merge cube job still running
[ https://issues.apache.org/jira/browse/KYLIN-3818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen reopened KYLIN-3818: - Reopen to add fix version > After Cube disabled, auto-merge cube job still running > -- > > Key: KYLIN-3818 > URL: https://issues.apache.org/jira/browse/KYLIN-3818 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.6.0 >Reporter: Na Zhai >Assignee: Na Zhai >Priority: Major > Fix For: v3.0.0-alpha > > > *precondition* > There is a Cube that turns on the auto-merge feature. And it satisfied the > condition of the auto-merge. Then the job of merging segments begins. > After a few minutes, the job of merging segments goes into the error status, > so I discard the job. Then I disable this cube, but a new job of merging > segments begin to run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3818) After Cube disabled, auto-merge cube job still running
[ https://issues.apache.org/jira/browse/KYLIN-3818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3818: Fix Version/s: v2.6.2 > After Cube disabled, auto-merge cube job still running > -- > > Key: KYLIN-3818 > URL: https://issues.apache.org/jira/browse/KYLIN-3818 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.6.0 >Reporter: Na Zhai >Assignee: Na Zhai >Priority: Major > Fix For: v2.6.2, v3.0.0-alpha > > > *precondition* > There is a Cube that turns on the auto-merge feature. And it satisfied the > condition of the auto-merge. Then the job of merging segments begins. > After a few minutes, the job of merging segments goes into the error status, > so I discard the job. Then I disable this cube, but a new job of merging > segments begin to run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] hit-lacus commented on issue #624: Allow user-specified time format using real-time
hit-lacus commented on issue #624: Allow user-specified time format using real-time URL: https://github.com/apache/kylin/pull/624#issuecomment-491174708 Hi, could you please update your commit? 1. Use correct git user name/ user email which can link to your github profile page 2. Merge your commit into two commit, one for backend change, another for front end change 3. Create a JIRA, describe the problem you found, how you solve it, and attach manual test screen shot 4. Use KYLIN- in your git commit message Thank you so much~ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (KYLIN-2620) Check for "ORDER BY LIMIT" clause when rewrite SUM query as TOPN
[ https://issues.apache.org/jira/browse/KYLIN-2620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-2620. - Resolution: Fixed Committed to 2.6.x branch > Check for "ORDER BY LIMIT" clause when rewrite SUM query as TOPN > > > Key: KYLIN-2620 > URL: https://issues.apache.org/jira/browse/KYLIN-2620 > Project: Kylin > Issue Type: Bug > Components: Measure - TopN >Reporter: Lin Tingmao >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > > When running the following query > select sum(measure) from table group by col_id > if there exists TOPN(measure, group by col_id) measure, > TopNMeasureType.isTopNCompatibleSum()will pass, so the SUM is rewritten > to TOPN. This confuses the user since they may expect a accurate result for > every distinct value of group by column(s). > Kylin should check if "ORDER BY col_id LIMIT topncapacity" is present in the > query to determine whether to rewrite. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3838) Retry mechanism is invalid when build with spark
[ https://issues.apache.org/jira/browse/KYLIN-3838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836922#comment-16836922 ] nichunen commented on KYLIN-3838: - Committed to 2.6.x branch > Retry mechanism is invalid when build with spark > > > Key: KYLIN-3838 > URL: https://issues.apache.org/jira/browse/KYLIN-3838 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > > description：http://mail-archives.apache.org/mod_mbox/kylin-dev/201902.mbox/%3C1550482835832-0.post%40n6.nabble.com%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3808) RestAPI /api/jobs always returns 0 for exec_start_time and exec_end_time and exec_interrupt_time fields
[
https://issues.apache.org/jira/browse/KYLIN-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836925#comment-16836925
]
nichunen commented on KYLIN-3808:
-
Committed to 2.6.x branch
> RestAPI /api/jobs always returns 0 for exec_start_time and exec_end_time and
> exec_interrupt_time fields
> ---
>
> Key: KYLIN-3808
> URL: https://issues.apache.org/jira/browse/KYLIN-3808
> Project: Kylin
> Issue Type: Bug
> Components: REST Service
>Affects Versions: v2.5.2
>Reporter: yaozq
>Assignee: yaozq
>Priority: Minor
> Fix For: v2.6.2
>
> Attachments:
> 0001-KYLIN-3808-fix-RestAPI-api-jobs-always-returns-0-for.patch
>
>
> GET
> [http://kylin-host:7070/kylin/api/jobs?cubeName=xxx&timeFilter=0&offset=0&limit=1]
> [
> {
> "uuid":"a89cf2d4-caaf-21fe-5cb9-63f1725d0e02",
> "last_modified":1549917430155,
> "version":"2.5.2.20500",
> "name":"BUILD CUBE - - 2019021100_2019021200 - GMT+08:00
> 2019-02-12 04:10:32",
> "type":"BUILD",
> "duration":1583,
> "related_cube":"",
> "display_cube_name":"",
> "related_segment":"898fc7c8-5e30-927c-da26-816ab4dbff08",
> {color:#FF}"exec_start_time":0,{color}
> {color:#FF}"exec_end_time":0,{color}
> {color:#FF}"exec_interrupt_time":0,{color}
> "mr_waiting":179,
> ..
>
> exec_start_time, exec_end_time and exec_interrupt_time fields always return 0
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3835) TableSchemaUpdateChecker didn't check models when reload table
[ https://issues.apache.org/jira/browse/KYLIN-3835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836919#comment-16836919 ] nichunen commented on KYLIN-3835: - Committed to 2.6.x branch > TableSchemaUpdateChecker didn't check models when reload table > -- > > Key: KYLIN-3835 > URL: https://issues.apache.org/jira/browse/KYLIN-3835 > Project: Kylin > Issue Type: Bug > Components: Metadata, REST Service >Affects Versions: v2.5.2 >Reporter: Yuzhang QIU >Assignee: Yuzhang QIU >Priority: Major > Fix For: v2.6.2 > > > 1. load table1 from hive. > 2. create model1 based on table1 and use table1.column1 as dimension1 > 3. alter table1.column1 to table1.column11 in hive. > 4. reload table1 successfully. (it's bug) > 5. swicth to model, the model1 still exist. I can create cube1 based on > model1 and launch a build job, of course, the job turn out error after a > period of time. (can't find table1.column1, etc) > 6. reload metadata in system page, the model1 is disappeared from Web UI, and > cube1 change to DESCBROKEN, and can't be deleted due to "null" (trace the > log, I found it's caused by null DataModelDesc in CubeInstance). > 7. I want to recreated the model1, but Kylin tell me model1 already existed > in current project. yes, I use 'sh bin/metadata.sh backup', I found the > model1's metadata is still stored in Hbase. > 8. I hacked the code, the reload table validation is checked in > TableSchemaUpdateChecker.allowLoad(), but it just check the used cubes. If a > model using the changed table without any cube based on it, the table can be > reloaded successfully! > I think it shouldn't be like this. > Best regard -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KYLIN-3788) Kylin's internal time zone conversions are various and confusing
[
https://issues.apache.org/jira/browse/KYLIN-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-3788.
-
Resolution: Fixed
> Kylin's internal time zone conversions are various and confusing
> -
>
> Key: KYLIN-3788
> URL: https://issues.apache.org/jira/browse/KYLIN-3788
> Project: Kylin
> Issue Type: Bug
> Components: REST Service, Web
>Affects Versions: v2.5.2
> Environment: Huawei FI
>Reporter: zhao jintao
>Assignee: zhao jintao
>Priority: Major
> Labels: TimeZone
> Fix For: v2.6.2
>
>
> Hi, All team
> I found that Kylin has a variety of time zone conversions inside, which is
> confusing. In some code, the time zone is "GMT" when converting time, and
> some code uses the time zone specified in the configuration file
> ("TimeZone.getTimeZone("GMT")" );
> For Example:
> in CubingJob.java, use TimeZone from configuration file:
> format.setTimeZone(TimeZone.getTimeZone(
> config.getTimeZone())),
> but in TimeUtil.java, use GMT TimeZone:private static TimeZone gmt =
> TimeZone.getTimeZone("GMT")
>
> When inport data from Kafka Streaming Data，kylin generate "Data_Start",
> "Hour_Start" uses the GMT time zone, They are some time zones later than
> the real time. Why not use the time zone specified in the configuration file?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Reopened] (KYLIN-3788) Kylin's internal time zone conversions are various and confusing
[
https://issues.apache.org/jira/browse/KYLIN-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen reopened KYLIN-3788:
-
reopen to set fix version
> Kylin's internal time zone conversions are various and confusing
> -
>
> Key: KYLIN-3788
> URL: https://issues.apache.org/jira/browse/KYLIN-3788
> Project: Kylin
> Issue Type: Bug
> Components: REST Service, Web
>Affects Versions: v2.5.2
> Environment: Huawei FI
>Reporter: zhao jintao
>Assignee: zhao jintao
>Priority: Major
> Labels: TimeZone
> Fix For: Future
>
>
> Hi, All team
> I found that Kylin has a variety of time zone conversions inside, which is
> confusing. In some code, the time zone is "GMT" when converting time, and
> some code uses the time zone specified in the configuration file
> ("TimeZone.getTimeZone("GMT")" );
> For Example:
> in CubingJob.java, use TimeZone from configuration file:
> format.setTimeZone(TimeZone.getTimeZone(
> config.getTimeZone())),
> but in TimeUtil.java, use GMT TimeZone:private static TimeZone gmt =
> TimeZone.getTimeZone("GMT")
>
> When inport data from Kafka Streaming Data，kylin generate "Data_Start",
> "Hour_Start" uses the GMT time zone, They are some time zones later than
> the real time. Why not use the time zone specified in the configuration file?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3788) Kylin's internal time zone conversions are various and confusing
[
https://issues.apache.org/jira/browse/KYLIN-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-3788:
Fix Version/s: (was: Future)
v2.6.2
> Kylin's internal time zone conversions are various and confusing
> -
>
> Key: KYLIN-3788
> URL: https://issues.apache.org/jira/browse/KYLIN-3788
> Project: Kylin
> Issue Type: Bug
> Components: REST Service, Web
>Affects Versions: v2.5.2
> Environment: Huawei FI
>Reporter: zhao jintao
>Assignee: zhao jintao
>Priority: Major
> Labels: TimeZone
> Fix For: v2.6.2
>
>
> Hi, All team
> I found that Kylin has a variety of time zone conversions inside, which is
> confusing. In some code, the time zone is "GMT" when converting time, and
> some code uses the time zone specified in the configuration file
> ("TimeZone.getTimeZone("GMT")" );
> For Example:
> in CubingJob.java, use TimeZone from configuration file:
> format.setTimeZone(TimeZone.getTimeZone(
> config.getTimeZone())),
> but in TimeUtil.java, use GMT TimeZone:private static TimeZone gmt =
> TimeZone.getTimeZone("GMT")
>
> When inport data from Kafka Streaming Data，kylin generate "Data_Start",
> "Hour_Start" uses the GMT time zone, They are some time zones later than
> the real time. Why not use the time zone specified in the configuration file?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [kylin] Qsimple commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
Qsimple commented on a change in pull request #629: KYLIN-3983 Add extra
metadata for measure. Add test case and example/…
URL: https://github.com/apache/kylin/pull/629#discussion_r282722596
##
File path: core-cube/src/main/java/org/apache/kylin/measure/MeasureManager.java
##
@@ -0,0 +1,698 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.measure;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import org.apache.kylin.common.KylinConfig;
+import org.apache.kylin.common.persistence.JsonSerializer;
+import org.apache.kylin.common.persistence.ResourceStore;
+import org.apache.kylin.common.persistence.Serializer;
+import org.apache.kylin.common.util.AutoReadWriteLock;
+import org.apache.kylin.cube.CubeInstance;
+import org.apache.kylin.cube.CubeManager;
+import org.apache.kylin.cube.CubeUpdate;
+import org.apache.kylin.cube.model.CubeDesc;
+import org.apache.kylin.metadata.MetadataConstants;
+import org.apache.kylin.metadata.cachesync.Broadcaster;
+import org.apache.kylin.metadata.cachesync.CachedCrudAssist;
+import org.apache.kylin.metadata.cachesync.CaseInsensitiveStringCache;
+import org.apache.kylin.metadata.model.ISegment;
+import org.apache.kylin.metadata.model.MeasureDesc;
+import org.apache.kylin.metadata.model.SegmentStatusEnum;
+import org.apache.kylin.metadata.project.ProjectInstance;
+import org.apache.kylin.metadata.project.ProjectManager;
+import org.apache.kylin.metadata.project.RealizationEntry;
+import org.apache.kylin.metadata.realization.RealizationType;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Locale;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import static org.apache.kylin.common.util.AutoReadWriteLock.AutoLock;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.CREATE;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.DROP;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.UPDATE;
+
+public class MeasureManager {
+
+public static final Serializer MEASURE_SERIALIZER = new
JsonSerializer<>(MeasureInstance.class);
+
+private static final Logger LOG =
LoggerFactory.getLogger(MeasureManager.class);
+public static final String MEASURE = "measure";
+public static final String CUBE = "cube";
+public static final String CUBE_DESC = "cube_desc";
+public static final String CUBE_MEASURE = "cube_measure";
+
+public static MeasureManager getInstance(KylinConfig config) {
+return config.getManager(MeasureManager.class);
+}
+
+// called by reflection
+static MeasureManager newInstance(KylinConfig config) throws IOException {
+return new MeasureManager(config);
+}
+
+// ===
+
+private KylinConfig config;
+
+// measure instance store path ==> MeasureInstance
+private CaseInsensitiveStringCache measureMap;
+private CubeL2Cache cubeL2Cache;
+
+private CachedCrudAssist crud;
+
+private AutoReadWriteLock measureMapLock = new AutoReadWriteLock();
+
+private MeasureManager(KylinConfig config) throws IOException {
+LOG.info("Initializing MeasureManager with config " + config);
+this.config = config;
+this.measureMap = new CaseInsensitiveStringCache<>(config, "measure");
+this.cubeL2Cache = new CubeL2Cache(getCubeManager());
+
+this.crud = new CachedCrudAssist(getStore(),
ResourceStore.MEASURE_RESOURCE_ROOT, MeasureInstance.class, measureMap) {
+@Override
+protected MeasureInstance initEntityAfterReload(MeasureInstance
entity, String resourceName) {
+CubeInstance cube =
getCubeManager().getCube(entity.getCubeName());
+entity.init(cube);
+
+return entity;
+}
+};
+
+// touch lower level metadata before registering my listener
+crud.reloadAll
[GitHub] [kylin] Qsimple commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
Qsimple commented on a change in pull request #629: KYLIN-3983 Add extra
metadata for measure. Add test case and example/…
URL: https://github.com/apache/kylin/pull/629#discussion_r282722143
##
File path: core-cube/src/main/java/org/apache/kylin/measure/MeasureManager.java
##
@@ -0,0 +1,698 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.measure;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import org.apache.kylin.common.KylinConfig;
+import org.apache.kylin.common.persistence.JsonSerializer;
+import org.apache.kylin.common.persistence.ResourceStore;
+import org.apache.kylin.common.persistence.Serializer;
+import org.apache.kylin.common.util.AutoReadWriteLock;
+import org.apache.kylin.cube.CubeInstance;
+import org.apache.kylin.cube.CubeManager;
+import org.apache.kylin.cube.CubeUpdate;
+import org.apache.kylin.cube.model.CubeDesc;
+import org.apache.kylin.metadata.MetadataConstants;
+import org.apache.kylin.metadata.cachesync.Broadcaster;
+import org.apache.kylin.metadata.cachesync.CachedCrudAssist;
+import org.apache.kylin.metadata.cachesync.CaseInsensitiveStringCache;
+import org.apache.kylin.metadata.model.ISegment;
+import org.apache.kylin.metadata.model.MeasureDesc;
+import org.apache.kylin.metadata.model.SegmentStatusEnum;
+import org.apache.kylin.metadata.project.ProjectInstance;
+import org.apache.kylin.metadata.project.ProjectManager;
+import org.apache.kylin.metadata.project.RealizationEntry;
+import org.apache.kylin.metadata.realization.RealizationType;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Locale;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import static org.apache.kylin.common.util.AutoReadWriteLock.AutoLock;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.CREATE;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.DROP;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.UPDATE;
+
+public class MeasureManager {
+
+public static final Serializer MEASURE_SERIALIZER = new
JsonSerializer<>(MeasureInstance.class);
+
+private static final Logger LOG =
LoggerFactory.getLogger(MeasureManager.class);
+public static final String MEASURE = "measure";
+public static final String CUBE = "cube";
+public static final String CUBE_DESC = "cube_desc";
+public static final String CUBE_MEASURE = "cube_measure";
Review comment:
When Listener receive "cube_measure", MeasureManager will batch-process the
MeasureInstances on the cube
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] Qsimple commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
Qsimple commented on a change in pull request #629: KYLIN-3983 Add extra
metadata for measure. Add test case and example/…
URL: https://github.com/apache/kylin/pull/629#discussion_r282721252
##
File path: core-cube/src/main/java/org/apache/kylin/measure/CubeL2Cache.java
##
@@ -0,0 +1,188 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.measure;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+import org.apache.kylin.cube.CubeInstance;
+import org.apache.kylin.cube.CubeManager;
+import org.apache.kylin.metadata.model.ISegment;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentSkipListMap;
+import java.util.stream.Collectors;
+
+public class CubeL2Cache {
+
+private static final Logger LOG =
LoggerFactory.getLogger(CubeL2Cache.class);
+
+private CubeManager cubeMgr;
+private Map cubeCaches = new
ConcurrentSkipListMap<>(String.CASE_INSENSITIVE_ORDER);
+
+
+public CubeL2Cache(CubeManager cubeMgr) {
+this.cubeMgr = cubeMgr;
+}
+
+// cat
+public List getMeasuresOnCubeAllowMiss(String cubeName) {
+CubeCache cache = cubeCaches.get(cubeName);
+cache = cache == null ? reloadCacheAllowMiss(cubeName) : cache;
Review comment:
ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] Qsimple commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
Qsimple commented on a change in pull request #629: KYLIN-3983 Add extra
metadata for measure. Add test case and example/…
URL: https://github.com/apache/kylin/pull/629#discussion_r282714563
##
File path:
core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java
##
@@ -2184,4 +2184,32 @@ public boolean isStreamingStandAloneMode() {
public String getLocalStorageImpl() {
return getOptional("kylin.stream.settled.storage", null);
}
+
+public String getHBaseColumnQualifierPrefix() {
Review comment:
ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] codecov-io commented on issue #637: KYLIN-3998 Make "bpus-min-benefit-ratio" configurable in cube planner phase 1
codecov-io commented on issue #637: KYLIN-3998 Make "bpus-min-benefit-ratio" configurable in cube planner phase 1 URL: https://github.com/apache/kylin/pull/637#issuecomment-491114316 # [Codecov](https://codecov.io/gh/apache/kylin/pull/637?src=pr&el=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@3f0ca84`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `0%`. [](https://codecov.io/gh/apache/kylin/pull/637?src=pr&el=tree) ```diff @@Coverage Diff@@ ## master #637 +/- ## = Coverage ? 25.37% Complexity? 5859 = Files ? 1379 Lines ?81871 Branches ?11479 = Hits ?20775 Misses?59073 Partials ? 2023 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/637?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../kylin/engine/mr/common/CuboidRecommenderUtil.java](https://codecov.io/gh/apache/kylin/pull/637/diff?src=pr&el=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvY29tbW9uL0N1Ym9pZFJlY29tbWVuZGVyVXRpbC5qYXZh) | `0% <0%> (ø)` | `0 <0> (?)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/637?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/637?src=pr&el=footer). Last update [3f0ca84...2f4841e](https://codecov.io/gh/apache/kylin/pull/637?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] Qsimple commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
Qsimple commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/… URL: https://github.com/apache/kylin/pull/629#discussion_r282714032 ## File path: core-cube/src/main/java/org/apache/kylin/cube/CubeManager.java ## @@ -238,7 +239,7 @@ public CubeInstance createCube(String cubeName, String projectName, CubeDesc des ProjectManager.getInstance(config).moveRealizationToProject(RealizationType.CUBE, cubeName, projectName, owner); - +getMeasureManager().createMeasuresOnCube(cube); Review comment: I don't understand this. why need to put "Create A" into "Update B". This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3998) Make "bpus-min-benefit-ratio" configurable in cube planner phase 1
[ https://issues.apache.org/jira/browse/KYLIN-3998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836800#comment-16836800 ] ASF GitHub Bot commented on KYLIN-3998: --- Wayne1c commented on pull request #637: KYLIN-3998 Make "bpus-min-benefit-ratio" configurable in cube planner phase 1 URL: https://github.com/apache/kylin/pull/637 …hase 1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Make "bpus-min-benefit-ratio" configurable in cube planner phase 1 > -- > > Key: KYLIN-3998 > URL: https://issues.apache.org/jira/browse/KYLIN-3998 > Project: Kylin > Issue Type: Improvement >Reporter: Chao Long >Assignee: Chao Long >Priority: Minor > Fix For: Future > > > When I was testing the impact of parameter “bpus-min-benefit-ratio” on > cube-planner phase 1, I found the parameter always uses the default value and > can't be overwritten. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] asfgit commented on issue #637: KYLIN-3998 Make "bpus-min-benefit-ratio" configurable in cube planner phase 1
asfgit commented on issue #637: KYLIN-3998 Make "bpus-min-benefit-ratio" configurable in cube planner phase 1 URL: https://github.com/apache/kylin/pull/637#issuecomment-491110024 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] Wayne1c opened a new pull request #637: KYLIN-3998 Make "bpus-min-benefit-ratio" configurable in cube planner phase 1
Wayne1c opened a new pull request #637: KYLIN-3998 Make "bpus-min-benefit-ratio" configurable in cube planner phase 1 URL: https://github.com/apache/kylin/pull/637 …hase 1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] Qsimple commented on issue #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
Qsimple commented on issue #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/… URL: https://github.com/apache/kylin/pull/629#issuecomment-490970062 Yes, I agree with you. Another PR will be prepared to replace this one. Tks for you, chunen This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] coveralls edited a comment on issue #631: Add additional unit tests
coveralls edited a comment on issue #631: Add additional unit tests URL: https://github.com/apache/kylin/pull/631#issuecomment-490178792 ## Pull Request Test Coverage Report for [Build 4480](https://coveralls.io/builds/23288694) * **0** of **0** changed or added relevant lines in **0** files are covered. * **380** unchanged lines in **11** files lost coverage. * Overall coverage increased (+**0.09%**) to **27.955%** --- | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/steps/MergeStatisticsWithOldStep.java](https://coveralls.io/builds/23288694/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fsteps%2FMergeStatisticsWithOldStep.java#L123) | 1 | 0.0% | | [core-job/src/main/java/org/apache/kylin/job/impl/threadpool/DefaultScheduler.java](https://coveralls.io/builds/23288694/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fthreadpool%2FDefaultScheduler.java#L194) | 2 | 80.23% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/common/CubeStatsWriter.java](https://coveralls.io/builds/23288694/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fcommon%2FCubeStatsWriter.java#L48) | 10 | 62.07% | | [source-jdbc/src/main/java/org/apache/kylin/source/jdbc/JdbcHiveInputBase.java](https://coveralls.io/builds/23288694/source?filename=source-jdbc%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fjdbc%2FJdbcHiveInputBase.java#L175) | 11 | 12.26% | | [core-metadata/src/main/java/org/apache/kylin/metadata/expression/CaseTupleExpression.java](https://coveralls.io/builds/23288694/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetadata%2Fexpression%2FCaseTupleExpression.java#L53) | 16 | 51.85% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/steps/MergeStatisticsStep.java](https://coveralls.io/builds/23288694/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fsteps%2FMergeStatisticsStep.java#L83) | 23 | 0.0% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/steps/MergeDictionaryMapper.java](https://coveralls.io/builds/23288694/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fsteps%2FMergeDictionaryMapper.java#L136) | 30 | 0.0% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/common/CubeStatsReader.java](https://coveralls.io/builds/23288694/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fcommon%2FCubeStatsReader.java#L169) | 52 | 0.0% | | [engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkMergingDictionary.java](https://coveralls.io/builds/23288694/source?filename=engine-spark%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fspark%2FSparkMergingDictionary.java#L92) | 59 | 0.0% | | [core-metadata/src/main/java/org/apache/kylin/metadata/filter/CompareTupleFilter.java](https://coveralls.io/builds/23288694/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetadata%2Ffilter%2FCompareTupleFilter.java#L64) | 62 | 33.13% | | Totals | [](https://coveralls.io/builds/23288694) | | :-- | --: | | Change from base [Build 4453](https://coveralls.io/builds/23203528): | 0.09% | | Covered Lines: | 22887 | | Relevant Lines: | 81871 | --- # 💛 - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
nichunen commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/… URL: https://github.com/apache/kylin/pull/629#discussion_r282483148 ## File path: core-cube/src/main/java/org/apache/kylin/cube/CubeManager.java ## @@ -238,7 +239,7 @@ public CubeInstance createCube(String cubeName, String projectName, CubeDesc des ProjectManager.getInstance(config).moveRealizationToProject(RealizationType.CUBE, cubeName, projectName, owner); - +getMeasureManager().createMeasuresOnCube(cube); Review comment: why not put the logic of update measure in updateSegmentsOnCube() This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
nichunen commented on a change in pull request #629: KYLIN-3983 Add extra
metadata for measure. Add test case and example/…
URL: https://github.com/apache/kylin/pull/629#discussion_r282517673
##
File path: core-cube/src/main/java/org/apache/kylin/measure/MeasureManager.java
##
@@ -0,0 +1,698 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.measure;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import org.apache.kylin.common.KylinConfig;
+import org.apache.kylin.common.persistence.JsonSerializer;
+import org.apache.kylin.common.persistence.ResourceStore;
+import org.apache.kylin.common.persistence.Serializer;
+import org.apache.kylin.common.util.AutoReadWriteLock;
+import org.apache.kylin.cube.CubeInstance;
+import org.apache.kylin.cube.CubeManager;
+import org.apache.kylin.cube.CubeUpdate;
+import org.apache.kylin.cube.model.CubeDesc;
+import org.apache.kylin.metadata.MetadataConstants;
+import org.apache.kylin.metadata.cachesync.Broadcaster;
+import org.apache.kylin.metadata.cachesync.CachedCrudAssist;
+import org.apache.kylin.metadata.cachesync.CaseInsensitiveStringCache;
+import org.apache.kylin.metadata.model.ISegment;
+import org.apache.kylin.metadata.model.MeasureDesc;
+import org.apache.kylin.metadata.model.SegmentStatusEnum;
+import org.apache.kylin.metadata.project.ProjectInstance;
+import org.apache.kylin.metadata.project.ProjectManager;
+import org.apache.kylin.metadata.project.RealizationEntry;
+import org.apache.kylin.metadata.realization.RealizationType;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Locale;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import static org.apache.kylin.common.util.AutoReadWriteLock.AutoLock;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.CREATE;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.DROP;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.UPDATE;
+
+public class MeasureManager {
+
+public static final Serializer MEASURE_SERIALIZER = new
JsonSerializer<>(MeasureInstance.class);
+
+private static final Logger LOG =
LoggerFactory.getLogger(MeasureManager.class);
+public static final String MEASURE = "measure";
+public static final String CUBE = "cube";
+public static final String CUBE_DESC = "cube_desc";
+public static final String CUBE_MEASURE = "cube_measure";
Review comment:
what is "cube_measure" for?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] nichunen commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
nichunen commented on a change in pull request #629: KYLIN-3983 Add extra
metadata for measure. Add test case and example/…
URL: https://github.com/apache/kylin/pull/629#discussion_r282458112
##
File path:
core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java
##
@@ -2184,4 +2184,32 @@ public boolean isStreamingStandAloneMode() {
public String getLocalStorageImpl() {
return getOptional("kylin.stream.settled.storage", null);
}
+
+public String getHBaseColumnQualifierPrefix() {
Review comment:
Configs should be grouped, configs start with "kylin.metadata.*" should be
in one group
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] nichunen commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
nichunen commented on a change in pull request #629: KYLIN-3983 Add extra
metadata for measure. Add test case and example/…
URL: https://github.com/apache/kylin/pull/629#discussion_r282502592
##
File path: core-cube/src/main/java/org/apache/kylin/measure/CubeL2Cache.java
##
@@ -0,0 +1,188 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.measure;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+import org.apache.kylin.cube.CubeInstance;
+import org.apache.kylin.cube.CubeManager;
+import org.apache.kylin.metadata.model.ISegment;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentSkipListMap;
+import java.util.stream.Collectors;
+
+public class CubeL2Cache {
+
+private static final Logger LOG =
LoggerFactory.getLogger(CubeL2Cache.class);
+
+private CubeManager cubeMgr;
+private Map cubeCaches = new
ConcurrentSkipListMap<>(String.CASE_INSENSITIVE_ORDER);
+
+
+public CubeL2Cache(CubeManager cubeMgr) {
+this.cubeMgr = cubeMgr;
+}
+
+// cat
+public List getMeasuresOnCubeAllowMiss(String cubeName) {
+CubeCache cache = cubeCaches.get(cubeName);
+cache = cache == null ? reloadCacheAllowMiss(cubeName) : cache;
Review comment:
redundant code
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] nichunen commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
nichunen commented on a change in pull request #629: KYLIN-3983 Add extra
metadata for measure. Add test case and example/…
URL: https://github.com/apache/kylin/pull/629#discussion_r282519903
##
File path: core-cube/src/main/java/org/apache/kylin/measure/MeasureManager.java
##
@@ -0,0 +1,698 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.measure;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import org.apache.kylin.common.KylinConfig;
+import org.apache.kylin.common.persistence.JsonSerializer;
+import org.apache.kylin.common.persistence.ResourceStore;
+import org.apache.kylin.common.persistence.Serializer;
+import org.apache.kylin.common.util.AutoReadWriteLock;
+import org.apache.kylin.cube.CubeInstance;
+import org.apache.kylin.cube.CubeManager;
+import org.apache.kylin.cube.CubeUpdate;
+import org.apache.kylin.cube.model.CubeDesc;
+import org.apache.kylin.metadata.MetadataConstants;
+import org.apache.kylin.metadata.cachesync.Broadcaster;
+import org.apache.kylin.metadata.cachesync.CachedCrudAssist;
+import org.apache.kylin.metadata.cachesync.CaseInsensitiveStringCache;
+import org.apache.kylin.metadata.model.ISegment;
+import org.apache.kylin.metadata.model.MeasureDesc;
+import org.apache.kylin.metadata.model.SegmentStatusEnum;
+import org.apache.kylin.metadata.project.ProjectInstance;
+import org.apache.kylin.metadata.project.ProjectManager;
+import org.apache.kylin.metadata.project.RealizationEntry;
+import org.apache.kylin.metadata.realization.RealizationType;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Locale;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import static org.apache.kylin.common.util.AutoReadWriteLock.AutoLock;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.CREATE;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.DROP;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.UPDATE;
+
+public class MeasureManager {
+
+public static final Serializer MEASURE_SERIALIZER = new
JsonSerializer<>(MeasureInstance.class);
+
+private static final Logger LOG =
LoggerFactory.getLogger(MeasureManager.class);
+public static final String MEASURE = "measure";
+public static final String CUBE = "cube";
+public static final String CUBE_DESC = "cube_desc";
+public static final String CUBE_MEASURE = "cube_measure";
+
+public static MeasureManager getInstance(KylinConfig config) {
+return config.getManager(MeasureManager.class);
+}
+
+// called by reflection
+static MeasureManager newInstance(KylinConfig config) throws IOException {
+return new MeasureManager(config);
+}
+
+// ===
+
+private KylinConfig config;
+
+// measure instance store path ==> MeasureInstance
+private CaseInsensitiveStringCache measureMap;
+private CubeL2Cache cubeL2Cache;
+
+private CachedCrudAssist crud;
+
+private AutoReadWriteLock measureMapLock = new AutoReadWriteLock();
+
+private MeasureManager(KylinConfig config) throws IOException {
+LOG.info("Initializing MeasureManager with config " + config);
+this.config = config;
+this.measureMap = new CaseInsensitiveStringCache<>(config, "measure");
+this.cubeL2Cache = new CubeL2Cache(getCubeManager());
+
+this.crud = new CachedCrudAssist(getStore(),
ResourceStore.MEASURE_RESOURCE_ROOT, MeasureInstance.class, measureMap) {
+@Override
+protected MeasureInstance initEntityAfterReload(MeasureInstance
entity, String resourceName) {
+CubeInstance cube =
getCubeManager().getCube(entity.getCubeName());
+entity.init(cube);
+
+return entity;
+}
+};
+
+// touch lower level metadata before registering my listener
+crud.reloadAl
[GitHub] [kylin] nichunen merged pull request #635: KYLIN-3934 add config for sqoop config null-string and null-non-string
nichunen merged pull request #635: KYLIN-3934 add config for sqoop config null-string and null-non-string URL: https://github.com/apache/kylin/pull/635 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3934) sqoop import param '--null-string' result in null value become blank string in hive table
[
https://issues.apache.org/jira/browse/KYLIN-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836427#comment-16836427
]
ASF GitHub Bot commented on KYLIN-3934:
---
nichunen commented on pull request #635: KYLIN-3934 add config for sqoop config
null-string and null-non-string
URL: https://github.com/apache/kylin/pull/635
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> sqoop import param '--null-string' result in null value become blank string
> in hive table
> -
>
> Key: KYLIN-3934
> URL: https://issues.apache.org/jira/browse/KYLIN-3934
> Project: Kylin
> Issue Type: Bug
> Components: Others
>Affects Versions: v2.6.0
>Reporter: wanghao
>Priority: Major
> Fix For: v2.6.2
>
>
> when column value from jdbc is null, sqoop will write it into hive table with
> blank string.
> eg
> jdbc:
> A | B
> 1 | 1
> 2 | 2
> a | null
>
> hive table:
> A | B
> 1 | 1
> 2 | 2
> a |
> because of this, when I use count(distinct B), it return 3, not 2, and it can
> lead to other problems
>
>
> {code:java}
> String cmd = String.format(Locale.ROOT,
> "%s/bin/sqoop import" + generateSqoopConfigArgString()
> + "--connect \"%s\" --driver %s --username %s --password %s --query \"%s AND
> \\$CONDITIONS\" "
> + "--target-dir %s/%s --split-by %s --boundary-query \"%s\" --null-string '' "
> + "--fields-terminated-by '%s' --num-mappers %d",
> sqoopHome, connectionUrl, driverClass, jdbcUser, jdbcPass, selectSql,
> jobWorkingDir, hiveTable,
> splitColumn, bquery, filedDelimiter, mapperNum);
> {code}
> the param '–null=string' should be '
> \\N' instead of blank string ''.
> I resolved this problem by replace the param. But it needs be configured in
> kylin.properties
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3934) sqoop import param '--null-string' result in null value become blank string in hive table
[
https://issues.apache.org/jira/browse/KYLIN-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836428#comment-16836428
]
ASF subversion and git services commented on KYLIN-3934:
Commit 3f0ca847e09532ff07d4258f92e30fec331e672c in kylin's branch
refs/heads/master from wanghao
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=3f0ca84 ]
KYLIN-3934 add config for sqoop config null-string and null-non-string
> sqoop import param '--null-string' result in null value become blank string
> in hive table
> -
>
> Key: KYLIN-3934
> URL: https://issues.apache.org/jira/browse/KYLIN-3934
> Project: Kylin
> Issue Type: Bug
> Components: Others
>Affects Versions: v2.6.0
>Reporter: wanghao
>Priority: Major
> Fix For: v2.6.2
>
>
> when column value from jdbc is null, sqoop will write it into hive table with
> blank string.
> eg
> jdbc:
> A | B
> 1 | 1
> 2 | 2
> a | null
>
> hive table:
> A | B
> 1 | 1
> 2 | 2
> a |
> because of this, when I use count(distinct B), it return 3, not 2, and it can
> lead to other problems
>
>
> {code:java}
> String cmd = String.format(Locale.ROOT,
> "%s/bin/sqoop import" + generateSqoopConfigArgString()
> + "--connect \"%s\" --driver %s --username %s --password %s --query \"%s AND
> \\$CONDITIONS\" "
> + "--target-dir %s/%s --split-by %s --boundary-query \"%s\" --null-string '' "
> + "--fields-terminated-by '%s' --num-mappers %d",
> sqoopHome, connectionUrl, driverClass, jdbcUser, jdbcPass, selectSql,
> jobWorkingDir, hiveTable,
> splitColumn, bquery, filedDelimiter, mapperNum);
> {code}
> the param '–null=string' should be '
> \\N' instead of blank string ''.
> I resolved this problem by replace the param. But it needs be configured in
> kylin.properties
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3998) Make "bpus-min-benefit-ratio" configurable in cube planner phase 1
[ https://issues.apache.org/jira/browse/KYLIN-3998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Long updated KYLIN-3998: - Description: When I was testing the impact of parameter “bpus-min-benefit-ratio” on cube-planner phase 1, I found the parameter always uses the default value and can't be overwritten. > Make "bpus-min-benefit-ratio" configurable in cube planner phase 1 > -- > > Key: KYLIN-3998 > URL: https://issues.apache.org/jira/browse/KYLIN-3998 > Project: Kylin > Issue Type: Improvement >Reporter: Chao Long >Assignee: Chao Long >Priority: Minor > Fix For: Future > > > When I was testing the impact of parameter “bpus-min-benefit-ratio” on > cube-planner phase 1, I found the parameter always uses the default value and > can't be overwritten. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3998) Make "bpus-min-benefit-ratio" configurable in cube planner phase 1
Chao Long created KYLIN-3998: Summary: Make "bpus-min-benefit-ratio" configurable in cube planner phase 1 Key: KYLIN-3998 URL: https://issues.apache.org/jira/browse/KYLIN-3998 Project: Kylin Issue Type: Improvement Reporter: Chao Long Assignee: Chao Long Fix For: Future -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-490) Support multiple column distinct count
[ https://issues.apache.org/jira/browse/KYLIN-490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836293#comment-16836293 ] hejian commented on KYLIN-490: -- hi,[~R0ger] [~yimingliu] [~Shaofengshi] what's new about this isue? I found the query engine can't parse it correctly in kylin-2.4.0, for example: the result of *count(distinct day_id, seller_id)* and *count(distinct day_id)* are same > Support multiple column distinct count > -- > > Key: KYLIN-490 > URL: https://issues.apache.org/jira/browse/KYLIN-490 > Project: Kylin > Issue Type: Improvement >Reporter: Luke Han >Assignee: Roger Shi >Priority: Major > Labels: github-import > Fix For: v2.0.0 > > Attachments: > 0001-KYLIN-490-support-multiple-column-distinct-count.patch, > 0002-KYLIN-490-support-multiple-column-distinct-count.patch > > > Make count distinct(A, B) work, currently calcite only support count distinct > on one column. > Imported from GitHub > Url: https://github.com/KylinOLAP/Kylin/issues/15 > Created by: [liyang-gmt8|https://github.com/liyang-gmt8] > Labels: enhancement, > Milestone: v0.6.2 Release > Assignee: [liyang-gmt8|https://github.com/liyang-gmt8] > Created at: Fri Oct 24 15:27:20 CST 2014 > State: open -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] GinaZhai opened a new pull request #636: Modify the display format of 'Get job list' api
GinaZhai opened a new pull request #636: Modify the display format of 'Get job list' api URL: https://github.com/apache/kylin/pull/636 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #636: Modify the display format of 'Get job list' api
asfgit commented on issue #636: Modify the display format of 'Get job list' api URL: https://github.com/apache/kylin/pull/636#issuecomment-490797168 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3989) Invalid temporary table path for kylin_metadata
[
https://issues.apache.org/jira/browse/KYLIN-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836148#comment-16836148
]
Frederic Souchu commented on KYLIN-3989:
No effect - tried:
{code:java}
maprfs:///apps/kylin/
/apps/kylin/{code}
The setting seems to be completely ignored (except for this HBase log line -
seems incorrect as well since HBase is actually working in
/apps/kylin_metadata!!).
> Invalid temporary table path for kylin_metadata
> ---
>
> Key: KYLIN-3989
> URL: https://issues.apache.org/jira/browse/KYLIN-3989
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.5.2
> Environment: MapR 6.1
> Hive 2.3
>Reporter: Frederic Souchu
>Priority: Major
>
> How to reproduce:
> * create a mode
> * define a cube
> * build the cube
> The cube building will fail with the following Hive logs:
> {code:java}
> USE default;
> No rows affected (0.067 seconds)
> 0: jdbc:hive2://x.com>
> 0: jdbc:hive2://x.com> DROP TABLE IF EXISTS
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e;
> No rows affected (0.023 seconds)
> 0: jdbc:hive2://x.com> CREATE EXTERNAL TABLE IF NOT EXISTS
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09
> e
> . . . . . . . . . . . . . . . . . . . . . . .> (
> . . . . . . . . . . . . . . . . . . . . . . .> PAYMENTS_GLOBALMERCHANTUID int
> . . . . . . . . . . . . . . . . . . . . . . .> )
> . . . . . . . . . . . . . . . . . . . . . . .> STORED AS SEQUENCEFILE
> . . . . . . . . . . . . . . . . . . . . . . .> LOCATION
> 'maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermedia
> te_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e';
> Error: org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception:
> java.io.IOException Error: Not a directory(20), file:
> kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e, user name:
> mapr, ID: 5000)
> {code}
> using metastore.sh to list content gives:
> {code}
> 2019-04-26 15:25:54,231 INFO [main] common.KylinConfig:100 : Loading
> kylin-defaults.properties from
> file:/opt/apache-kylin-2.5.2-bin-hbase1x/tool/kylin-tool-2.5.2.jar!/kylin-defaults.properties
> 2019-04-26 15:25:54,257 DEBUG [main] common.KylinConfig:327 : KYLIN_CONF
> property was not set, will seek KYLIN_HOME env variable
> 2019-04-26 15:25:54,263 INFO [main] common.KylinConfig:135 : Initialized a
> new KylinConfig from getInstanceFromEnv : 453523494
> 2019-04-26 15:25:54,376 INFO [main] persistence.ResourceStore:88 : Using
> metadata url /apps/kylin_metadata@hbase for resource store
> 2019-04-26 15:25:55,684 DEBUG [main] hbase.HBaseConnection:180 : Using the
> working dir FS for HBase: maprfs:///
> 2019-04-26 15:25:55,684 INFO [main] hbase.HBaseConnection:257 : connection
> is null or closed, creating a new one
> 2019-04-26 15:25:55,719 INFO [main] client.ConnectionFactory:272 :
> ConnectionFactory receives mapr.hbase.default.db(maprdb), set
> clusterType(MAPR_ONLY), user(mapr), hbase_admin_connect_at_construction(false)
> 2019-04-26 15:25:55,899 DEBUG [main] hbase.HBaseConnection:306 : HTable
> '/apps/kylin_metadata' already exists
> null
> 2019-04-26 15:25:56,400 INFO [close-hbase-conn] hbase.HBaseConnection:136 :
> Closing HBase connections...
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
