[jira] [Commented] (KYLIN-4279) Mapreduce Exception when building cube at step 3
[ https://issues.apache.org/jira/browse/KYLIN-4279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048447#comment-17048447 ] wangrupeng commented on KYLIN-4279: --- Hi, It seems like the problem of mapreduce job not kylin. I cannot find the root cause through the log you shared. You may check out your yarn log to get more details about your job failed infomation. Besides, you'd better tell us your environment information(hdp or cdh version). > Mapreduce Exception when building cube at step 3 > > > Key: KYLIN-4279 > URL: https://issues.apache.org/jira/browse/KYLIN-4279 > Project: Kylin > Issue Type: Bug > Components: Tools, Build and Test >Affects Versions: v2.6.4 >Reporter: Kanivel >Priority: Major > Attachments: Screenshot from 2019-12-05 12-55-22.png, Screenshot from > 2019-12-05 12-55-47.png > > > I have setup hadoop multinode cluster through ambari .ambari running as > master node and hive running in another aws instance.I have installed kylin > in master node and configured hive instance in kylin.properties. I can create > model,can create cube.But exception occur when build cube.Build fails at 3rd > step.When I check logs it shows Mapreduce exception.2nd step also taking more > time.I have 3 tables and each have 10 records.sales,customers,products.sales > is a fact table.customers and products are dimesions table..Please look at > the attached screenshots. > The below error I got, > org.apache.kylin.engine.mr.exception.MapReduceException: Counters: 13 Job > Counters Failed map tasks=4 Killed reduce tasks=3 Launched map tasks=4 Other > local map tasks=3 Data-local map tasks=1 Total time spent by all maps in > occupied slots (ms)=194628 Total time spent by all reduces in occupied slots > (ms)=0 Total time spent by all map tasks (ms)=27804 Total vcore-milliseconds > taken by all map tasks=27804 Total megabyte-milliseconds taken by all map > tasks=199299072 Map-Reduce Framework CPU time spent (ms)=0 Physical memory > (bytes) snapshot=0 Virtual memory (bytes) snapshot=0 Job Diagnostics:Task > failed task_1575385745059_0009_m_00 Job failed as tasks failed. > failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0 Failure task > Diagnostics: [2019-12-04 07:44:23.184]Exception from container-launch. > Container id: container_e03_1575385745059_0009_01_04 Exit code: 255 > [2019-12-04 07:44:23.186]Container exited with a non-zero exit code 255. > Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes > of stderr : [2019-12-04 07:44:23.186]Container exited with a non-zero exit > code 255. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last > 4096 bytes of stderr : at > org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:173) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (KYLIN-4341) by-level cuboid intermediate files are left behind and not cleaned up after job is complete
[
https://issues.apache.org/jira/browse/KYLIN-4341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048440#comment-17048440
]
wangrupeng edited comment on KYLIN-4341 at 3/1/20 1:31 AM:
---
Hi,
Kylin will not delete cuboids file on hdfs when segment building job finished,
because if this segment need to be merged, kylin won't calculate again, this
will save the calculation resource. If you are sure the segment won't be
merged, you can delete the cuboids files on hdfs yourself. :)
So it's not a bug.
was (Author: wangrupeng):
Hi,
Kylin will not delete cuboids file on hdfs when segment building job finished,
because if this segment need to be merged, kylin won't calculate again, this
will save the calculation resource. If you are sure the segment won't be
merged, you can delete the cuboids files on hdfs yourself. :)
> by-level cuboid intermediate files are left behind and not cleaned up after
> job is complete
> ---
>
> Key: KYLIN-4341
> URL: https://issues.apache.org/jira/browse/KYLIN-4341
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
>Affects Versions: v2.6.4
> Environment: Kylin 2.6.4, CenOS 7.6, HDP 2.6.5
>Reporter: Vsevolod Ostapenko
>Assignee: wangrupeng
>Priority: Major
>
> Setup: MR as a cube build engine and by-level cube build strategy (auto
> picked).
> Upon completion of a cube segment build job a number of intermediate files
> are still left behind.
> Namely, output of the MR-jobs that produce the base cuboid, subsequent level
> cuboids, as well as rowkey_stats from the hfile creation step.
> The files in question consume about the same amount of space in HDFS as the
> final hfile.
> This lead to wasted space in HDFS that is not released for as long as the
> corresponding cube segment is online. The only point the leaked space is
> released, is when segment is offlined and cleaned up as part of the segment
> retention.
> Sample output is as follows.
> {quote}$ hadoop fs -ls -R
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/
> drwxr-xr-x - kylin hdfs 0 2020-01-07 04:44
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89
> drwxr-xr-x - kylin hdfs 0 2020-01-07 04:26
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid
> drwxr-xr-x - kylin hdfs 0 2020-01-07 04:36
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid
> -rw-r--r-- 2 kylin hdfs 0 2020-01-07 04:36
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/_SUCCESS
> -rw-r--r-- 2 kylin hdfs 51570048 2020-01-07 04:35
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-0
> -rw-r--r-- 2 kylin hdfs 51477377 2020-01-07 04:36
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-1
> -rw-r--r-- 2 kylin hdfs 51615162 2020-01-07 04:35
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-2
> -rw-r--r-- 2 kylin hdfs 51591031 2020-01-07 04:36
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-3
> -rw-r--r-- 2 kylin hdfs 51648914 2020-01-07 04:35
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-4
> -rw-r--r-- 2 kylin hdfs 51532761 2020-01-07 04:36
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-5
> -rw-r--r-- 2 kylin hdfs 51455652 2020-01-07 04:35
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-6
> -rw-r--r-- 2 kylin hdfs 51552752 2020-01-07 04:36
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-7
> drwxr-xr-x - kylin hdfs 0 2020-01-07 04:25
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_base_cuboid
> -rw-r--r-- 2 kylin hdfs 0 2020-01-07 04:25
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_base_cuboid/_SUCCESS
> -rw-r--r-- 2 kylin hdfs 16293012 2020-01-07 04:25
>
[jira] [Commented] (KYLIN-4341) by-level cuboid intermediate files are left behind and not cleaned up after job is complete
[
https://issues.apache.org/jira/browse/KYLIN-4341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048440#comment-17048440
]
wangrupeng commented on KYLIN-4341:
---
Hi,
Kylin will not delete cuboids file on hdfs when segment building job finished,
because if this segment need to be merged, kylin won't calculate again, this
will save the calculation resource. If you are sure the segment won't be
merged, you can delete the cuboids files on hdfs yourself. :)
> by-level cuboid intermediate files are left behind and not cleaned up after
> job is complete
> ---
>
> Key: KYLIN-4341
> URL: https://issues.apache.org/jira/browse/KYLIN-4341
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
>Affects Versions: v2.6.4
> Environment: Kylin 2.6.4, CenOS 7.6, HDP 2.6.5
>Reporter: Vsevolod Ostapenko
>Assignee: wangrupeng
>Priority: Major
>
> Setup: MR as a cube build engine and by-level cube build strategy (auto
> picked).
> Upon completion of a cube segment build job a number of intermediate files
> are still left behind.
> Namely, output of the MR-jobs that produce the base cuboid, subsequent level
> cuboids, as well as rowkey_stats from the hfile creation step.
> The files in question consume about the same amount of space in HDFS as the
> final hfile.
> This lead to wasted space in HDFS that is not released for as long as the
> corresponding cube segment is online. The only point the leaked space is
> released, is when segment is offlined and cleaned up as part of the segment
> retention.
> Sample output is as follows.
> {quote}$ hadoop fs -ls -R
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/
> drwxr-xr-x - kylin hdfs 0 2020-01-07 04:44
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89
> drwxr-xr-x - kylin hdfs 0 2020-01-07 04:26
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid
> drwxr-xr-x - kylin hdfs 0 2020-01-07 04:36
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid
> -rw-r--r-- 2 kylin hdfs 0 2020-01-07 04:36
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/_SUCCESS
> -rw-r--r-- 2 kylin hdfs 51570048 2020-01-07 04:35
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-0
> -rw-r--r-- 2 kylin hdfs 51477377 2020-01-07 04:36
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-1
> -rw-r--r-- 2 kylin hdfs 51615162 2020-01-07 04:35
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-2
> -rw-r--r-- 2 kylin hdfs 51591031 2020-01-07 04:36
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-3
> -rw-r--r-- 2 kylin hdfs 51648914 2020-01-07 04:35
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-4
> -rw-r--r-- 2 kylin hdfs 51532761 2020-01-07 04:36
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-5
> -rw-r--r-- 2 kylin hdfs 51455652 2020-01-07 04:35
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-6
> -rw-r--r-- 2 kylin hdfs 51552752 2020-01-07 04:36
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-7
> drwxr-xr-x - kylin hdfs 0 2020-01-07 04:25
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_base_cuboid
> -rw-r--r-- 2 kylin hdfs 0 2020-01-07 04:25
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_base_cuboid/_SUCCESS
> -rw-r--r-- 2 kylin hdfs 16293012 2020-01-07 04:25
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_base_cuboid/part-r-0
> -rw-r--r-- 2 kylin hdfs 16283730 2020-01-07 04:25
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_base_cuboid/part-r-1
> -rw-r--r-- 2 kylin hdfs 16288965 2020-01-07 04:25
>
[GitHub] [kylin] YYTVicky opened a new pull request #1127: Suggest changes for security concern
YYTVicky opened a new pull request #1127: Suggest changes for security concern URL: https://github.com/apache/kylin/pull/1127 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Closed] (KYLIN-3969) BeelineHiveClient can't parse Hive meta data result set with partitioned table
[
https://issues.apache.org/jira/browse/KYLIN-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen closed KYLIN-3969.
---
> BeelineHiveClient can't parse Hive meta data result set with partitioned table
> --
>
> Key: KYLIN-3969
> URL: https://issues.apache.org/jira/browse/KYLIN-3969
> Project: Kylin
> Issue Type: Bug
> Components: Metadata
>Affects Versions: v2.6.1
> Environment: Hive 3.1.0, HiveServer2, Beeline（Along with Ambari
> 2.7.1.0)
> CentOS Linux release 7.6.1810
>Reporter: Tuo Zhu
>Assignee: Tuo Zhu
>Priority: Blocker
> Fix For: v2.6.3
>
>
> When creating a model, you can add tables without partitions, but can't add
> ones with partitions.
> Web UI pops："Oops. cannot get HiveTableMeta".
> I tried same table in Hive2 with cli, it works.
> log prints:
> {code:java}
> java.lang.RuntimeException: cannot get HiveTableMeta
> at
> org.apache.kylin.source.hive.HiveMetadataExplorer.loadTableMetadata(HiveMetadataExplorer.java:68)
> at
> org.apache.kylin.rest.service.TableService.extractHiveTableMeta(TableService.java:205)
> at
> org.apache.kylin.rest.service.TableService.loadHiveTablesToProject(TableService.java:131)
> at
> org.apache.kylin.rest.controller.TableController.loadHiveTables(TableController.java:113)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
> at
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
> at
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
> at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
> at
> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:661)
> at
> org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
>
[jira] [Commented] (KYLIN-3969) BeelineHiveClient can't parse Hive meta data result set with partitioned table
[
https://issues.apache.org/jira/browse/KYLIN-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048331#comment-17048331
]
nichunen commented on KYLIN-3969:
-
Fixed in 2.6.3
> BeelineHiveClient can't parse Hive meta data result set with partitioned table
> --
>
> Key: KYLIN-3969
> URL: https://issues.apache.org/jira/browse/KYLIN-3969
> Project: Kylin
> Issue Type: Bug
> Components: Metadata
>Affects Versions: v2.6.1
> Environment: Hive 3.1.0, HiveServer2, Beeline（Along with Ambari
> 2.7.1.0)
> CentOS Linux release 7.6.1810
>Reporter: Tuo Zhu
>Assignee: Tuo Zhu
>Priority: Blocker
> Fix For: v2.6.3
>
>
> When creating a model, you can add tables without partitions, but can't add
> ones with partitions.
> Web UI pops："Oops. cannot get HiveTableMeta".
> I tried same table in Hive2 with cli, it works.
> log prints:
> {code:java}
> java.lang.RuntimeException: cannot get HiveTableMeta
> at
> org.apache.kylin.source.hive.HiveMetadataExplorer.loadTableMetadata(HiveMetadataExplorer.java:68)
> at
> org.apache.kylin.rest.service.TableService.extractHiveTableMeta(TableService.java:205)
> at
> org.apache.kylin.rest.service.TableService.loadHiveTablesToProject(TableService.java:131)
> at
> org.apache.kylin.rest.controller.TableController.loadHiveTables(TableController.java:113)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
> at
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
> at
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
> at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
> at
> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:661)
> at
> org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
>
[jira] [Resolved] (KYLIN-3969) BeelineHiveClient can't parse Hive meta data result set with partitioned table
[
https://issues.apache.org/jira/browse/KYLIN-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-3969.
-
Resolution: Fixed
> BeelineHiveClient can't parse Hive meta data result set with partitioned table
> --
>
> Key: KYLIN-3969
> URL: https://issues.apache.org/jira/browse/KYLIN-3969
> Project: Kylin
> Issue Type: Bug
> Components: Metadata
>Affects Versions: v2.6.1
> Environment: Hive 3.1.0, HiveServer2, Beeline（Along with Ambari
> 2.7.1.0)
> CentOS Linux release 7.6.1810
>Reporter: Tuo Zhu
>Assignee: Tuo Zhu
>Priority: Blocker
> Fix For: v2.6.3
>
>
> When creating a model, you can add tables without partitions, but can't add
> ones with partitions.
> Web UI pops："Oops. cannot get HiveTableMeta".
> I tried same table in Hive2 with cli, it works.
> log prints:
> {code:java}
> java.lang.RuntimeException: cannot get HiveTableMeta
> at
> org.apache.kylin.source.hive.HiveMetadataExplorer.loadTableMetadata(HiveMetadataExplorer.java:68)
> at
> org.apache.kylin.rest.service.TableService.extractHiveTableMeta(TableService.java:205)
> at
> org.apache.kylin.rest.service.TableService.loadHiveTablesToProject(TableService.java:131)
> at
> org.apache.kylin.rest.controller.TableController.loadHiveTables(TableController.java:113)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
> at
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
> at
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
> at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
> at
> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:661)
> at
> org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
>
[jira] [Updated] (KYLIN-3969) BeelineHiveClient can't parse Hive meta data result set with partitioned table
[
https://issues.apache.org/jira/browse/KYLIN-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-3969:
Fix Version/s: v2.6.3
> BeelineHiveClient can't parse Hive meta data result set with partitioned table
> --
>
> Key: KYLIN-3969
> URL: https://issues.apache.org/jira/browse/KYLIN-3969
> Project: Kylin
> Issue Type: Bug
> Components: Metadata
>Affects Versions: v2.6.1
> Environment: Hive 3.1.0, HiveServer2, Beeline（Along with Ambari
> 2.7.1.0)
> CentOS Linux release 7.6.1810
>Reporter: Tuo Zhu
>Assignee: Tuo Zhu
>Priority: Blocker
> Fix For: v2.6.3
>
>
> When creating a model, you can add tables without partitions, but can't add
> ones with partitions.
> Web UI pops："Oops. cannot get HiveTableMeta".
> I tried same table in Hive2 with cli, it works.
> log prints:
> {code:java}
> java.lang.RuntimeException: cannot get HiveTableMeta
> at
> org.apache.kylin.source.hive.HiveMetadataExplorer.loadTableMetadata(HiveMetadataExplorer.java:68)
> at
> org.apache.kylin.rest.service.TableService.extractHiveTableMeta(TableService.java:205)
> at
> org.apache.kylin.rest.service.TableService.loadHiveTablesToProject(TableService.java:131)
> at
> org.apache.kylin.rest.controller.TableController.loadHiveTables(TableController.java:113)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
> at
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
> at
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
> at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
> at
> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:661)
> at
> org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
>
