[GitHub] [kylin] lgtm-com[bot] commented on pull request #1936: Test if this patch pass CI?
lgtm-com[bot] commented on PR #1936: URL: https://github.com/apache/kylin/pull/1936#issuecomment-1214549646 This pull request **fixes 1 alert** when merging cd21918bf4784955ddce39e856fa27d45e941045 into 9e28c8732a0bf50290584dc047fe48b73890bcae - [view on LGTM.com](https://lgtm.com/projects/g/apache/kylin/rev/pr-d58109c49dfe38389b539e3b73b513ee1b2ecd98) **fixed alerts:** * 1 for \`TrustManager\` that accepts all certificates -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@kylin.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kylin] lgtm-com[bot] commented on pull request #1936: Test if this patch pass CI?
lgtm-com[bot] commented on PR #1936: URL: https://github.com/apache/kylin/pull/1936#issuecomment-1214481449 This pull request **fixes 1 alert** when merging 83b7ebbe2057b3a3a8eb6f9dfee4cc47e0d33c0d into 9e28c8732a0bf50290584dc047fe48b73890bcae - [view on LGTM.com](https://lgtm.com/projects/g/apache/kylin/rev/pr-e930c14e5d9f69d4e4aecc0919f4218bd8cc1f66) **fixed alerts:** * 1 for \`TrustManager\` that accepts all certificates -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@kylin.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kylin] hit-lacus opened a new pull request, #1936: Test if this patch pass CI?
hit-lacus opened a new pull request, #1936: URL: https://github.com/apache/kylin/pull/1936 ## Proposed changes Test if this patch pass CI? ## Github Branch As most of the development works are on Kylin 4, we need to switch it as main branch. Apache Kylin community changes the branch settings on Github since 2021-08-04 : 1. The default branch _main_ is for **Kylin 4.x** (Parquet storage); 2. The original branch _master_ for **Kylin 3.x** (HBase storage) has been renamed to **kylin3** ; Please check [Intro to Kylin 4 architecture](https://kylin.apache.org/blog/2021/07/02/Apache-Kylin4-A-new-storage-and-compute-architecture/) and [INFRA-22166](https://issues.apache.org/jira/browse/INFRA-22166) if you are interested. ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have create an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@kylin.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (KYLIN-5224) kylin 4.0.1 does not support limit..offset (requires spark 3.4)
[
https://issues.apache.org/jira/browse/KYLIN-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liyang updated KYLIN-5224:
--
Summary: kylin 4.0.1 does not support limit..offset (requires spark 3.4)
(was: kylin 4.0.1 不支持分页查询)
> kylin 4.0.1 does not support limit..offset (requires spark 3.4)
> ---
>
> Key: KYLIN-5224
> URL: https://issues.apache.org/jira/browse/KYLIN-5224
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v4.0.1
>Reporter: liulei
>Priority: Major
>
> {
> "code": "999",
> "data": null,
> "msg": " mismatched input 'offset' expecting \{, ';'}(line 1, pos
> 55) == SQL == select * from MES_PROD.inspec_doc_report_data limit 10 offset
> 5 ---^^^ ",
> "stacktrace": "org.apache.kylin.rest.exception.InternalErrorException:
> mismatched input 'offset' expecting \{, ';'}(line 1, pos 55) == SQL ==
> select * from MES_PROD.inspec_doc_report_data limit 10 offset 5
> ---^^^ \n\tat
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:497)\n\tat
>
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:416)\n\tat
>
> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:93)\n\tat
> sun.reflect.GeneratedMethodAccessor184.invoke(Unknown Source)\n\tat
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat
> java.lang.reflect.Method.invoke(Method.java:498)\n\tat
> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)\n\tat
>
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)\n\tat
>
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)\n\tat
>
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:854)\n\tat
>
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:765)\n\tat
>
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)\n\tat
>
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)\n\tat
>
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)\n\tat
>
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)\n\tat
>
> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)\n\tat
> javax.servlet.http.HttpServlet.service(HttpServlet.java:647)\n\tat
> org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)\n\tat
> javax.servlet.http.HttpServlet.service(HttpServlet.java:728)\n\tat
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)\n\tat
>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)\n\tat
> org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)\n\tat
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)\n\tat
>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)\n\tat
>
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)\n\tat
>
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)\n\tat
>
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)\n\tat
>
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\n\tat
>
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)\n\tat
>
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\n\tat
>
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)\n\tat
>
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\n\tat
>
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)\n\tat
>
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\n\tat
>
>
[jira] [Commented] (KYLIN-5224) kylin 4.0.1 不支持分页查询
[
https://issues.apache.org/jira/browse/KYLIN-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579441#comment-17579441
]
liyang commented on KYLIN-5224:
---
then the request changes to support spark 3.4 :)
> kylin 4.0.1 不支持分页查询
> ---
>
> Key: KYLIN-5224
> URL: https://issues.apache.org/jira/browse/KYLIN-5224
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v4.0.1
>Reporter: liulei
>Priority: Major
>
> {
> "code": "999",
> "data": null,
> "msg": " mismatched input 'offset' expecting \{, ';'}(line 1, pos
> 55) == SQL == select * from MES_PROD.inspec_doc_report_data limit 10 offset
> 5 ---^^^ ",
> "stacktrace": "org.apache.kylin.rest.exception.InternalErrorException:
> mismatched input 'offset' expecting \{, ';'}(line 1, pos 55) == SQL ==
> select * from MES_PROD.inspec_doc_report_data limit 10 offset 5
> ---^^^ \n\tat
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:497)\n\tat
>
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:416)\n\tat
>
> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:93)\n\tat
> sun.reflect.GeneratedMethodAccessor184.invoke(Unknown Source)\n\tat
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat
> java.lang.reflect.Method.invoke(Method.java:498)\n\tat
> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)\n\tat
>
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)\n\tat
>
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)\n\tat
>
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:854)\n\tat
>
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:765)\n\tat
>
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)\n\tat
>
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)\n\tat
>
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)\n\tat
>
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)\n\tat
>
> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)\n\tat
> javax.servlet.http.HttpServlet.service(HttpServlet.java:647)\n\tat
> org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)\n\tat
> javax.servlet.http.HttpServlet.service(HttpServlet.java:728)\n\tat
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)\n\tat
>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)\n\tat
> org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)\n\tat
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)\n\tat
>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)\n\tat
>
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)\n\tat
>
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)\n\tat
>
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)\n\tat
>
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\n\tat
>
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)\n\tat
>
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\n\tat
>
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)\n\tat
>
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\n\tat
>
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)\n\tat
>
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\n\tat
>
> org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)\n\tat
>
>
[jira] [Closed] (KYLIN-5227) Kylin association query does not use ad hoc query, but push down spark query
[ https://issues.apache.org/jira/browse/KYLIN-5227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyang closed KYLIN-5227. - Resolution: Not A Problem > Kylin association query does not use ad hoc query, but push down spark query > > > Key: KYLIN-5227 > URL: https://issues.apache.org/jira/browse/KYLIN-5227 > Project: Kylin > Issue Type: Bug >Reporter: huang song >Priority: Major > Attachments: image-2022-08-12-09-24-42-891.png > > > The query statement is as follows > select： > b.boname, > h.hdrcode > from > b_booklist b > inner join b_holding h > on h.borecno=b.borecno > Build cube fields as follows， > All other steps are default > !image-2022-08-12-09-24-42-891.png! > Execute the push-down query of the above sql walk； > take a long time > I don't know where to build what's the problem -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (KYLIN-5212) cannot get HiveTableMeta
[ https://issues.apache.org/jira/browse/KYLIN-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyang closed KYLIN-5212. - Resolution: Not A Problem > cannot get HiveTableMeta > > > Key: KYLIN-5212 > URL: https://issues.apache.org/jira/browse/KYLIN-5212 > Project: Kylin > Issue Type: Bug >Reporter: huang song >Priority: Major > Attachments: screenshot-1.png > > > version: kylin4.0.1+hive2.3.9+hadoop3.2.0+spark3.1.1 > 2022-07-19 11:42:25,982 ERROR [http-bio-7070-exec-2] > controller.BasicController:65 : > org.apache.kylin.rest.exception.InternalErrorException: cannot get > HiveTableMeta > at > org.apache.kylin.rest.controller.TableController.loadHiveTables(TableController.java:132) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) > at > org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) > at > org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:854) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:765) > at > org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) > at > org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967) > at > org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) > at > org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) > at > org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:647) > at > org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:728) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:215) > at > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) > at >
[jira] [Commented] (KYLIN-5227) Kylin association query does not use ad hoc query, but push down spark query
[
https://issues.apache.org/jira/browse/KYLIN-5227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579388#comment-17579388
]
liyang commented on KYLIN-5227:
---
Without group by and aggregate function, this is a raw query. It is expected
that raw query will go to spark pushdown and may take longer than aggregated
queries that are answered by pre-computed data.
Things look normal suggest close this JIRA.
{code:java}
select：
b.boname,
h.hdrcode
from
b_booklist b
inner join b_holding h
on h.borecno=b.borecno{code}
> Kylin association query does not use ad hoc query, but push down spark query
>
>
> Key: KYLIN-5227
> URL: https://issues.apache.org/jira/browse/KYLIN-5227
> Project: Kylin
> Issue Type: Bug
>Reporter: huang song
>Priority: Major
> Attachments: image-2022-08-12-09-24-42-891.png
>
>
> The query statement is as follows
> select：
> b.boname,
> h.hdrcode
> from
> b_booklist b
> inner join b_holding h
> on h.borecno=b.borecno
> Build cube fields as follows，
> All other steps are default
> !image-2022-08-12-09-24-42-891.png!
> Execute the push-down query of the above sql walk；
> take a long time
> I don't know where to build what's the problem
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5224) kylin 4.0.1 不支持分页查询
[
https://issues.apache.org/jira/browse/KYLIN-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579335#comment-17579335
]
zhaoliu commented on KYLIN-5224:
spark 3.4.0 支持了"limit ... offset ..."
https://issues.apache.org/jira/browse/SPARK-28330
> kylin 4.0.1 不支持分页查询
> ---
>
> Key: KYLIN-5224
> URL: https://issues.apache.org/jira/browse/KYLIN-5224
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v4.0.1
>Reporter: liulei
>Priority: Major
>
> {
> "code": "999",
> "data": null,
> "msg": " mismatched input 'offset' expecting \{, ';'}(line 1, pos
> 55) == SQL == select * from MES_PROD.inspec_doc_report_data limit 10 offset
> 5 ---^^^ ",
> "stacktrace": "org.apache.kylin.rest.exception.InternalErrorException:
> mismatched input 'offset' expecting \{, ';'}(line 1, pos 55) == SQL ==
> select * from MES_PROD.inspec_doc_report_data limit 10 offset 5
> ---^^^ \n\tat
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:497)\n\tat
>
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:416)\n\tat
>
> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:93)\n\tat
> sun.reflect.GeneratedMethodAccessor184.invoke(Unknown Source)\n\tat
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat
> java.lang.reflect.Method.invoke(Method.java:498)\n\tat
> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)\n\tat
>
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)\n\tat
>
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)\n\tat
>
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:854)\n\tat
>
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:765)\n\tat
>
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)\n\tat
>
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)\n\tat
>
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)\n\tat
>
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)\n\tat
>
> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)\n\tat
> javax.servlet.http.HttpServlet.service(HttpServlet.java:647)\n\tat
> org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)\n\tat
> javax.servlet.http.HttpServlet.service(HttpServlet.java:728)\n\tat
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)\n\tat
>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)\n\tat
> org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)\n\tat
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)\n\tat
>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)\n\tat
>
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)\n\tat
>
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)\n\tat
>
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)\n\tat
>
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\n\tat
>
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)\n\tat
>
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\n\tat
>
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)\n\tat
>
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\n\tat
>
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)\n\tat
>
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\n\tat
>
>
[jira] [Commented] (KYLIN-5200) Kylin4 RAW Schema written to Parquet and read from Parquet are inconsistent
[
https://issues.apache.org/jira/browse/KYLIN-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579331#comment-17579331
]
ASF GitHub Bot commented on KYLIN-5200:
---
zhaoliu17 commented on PR #1896:
URL: https://github.com/apache/kylin/pull/1896#issuecomment-1214294560
构建时支持 raw 类型计算
> Kylin4 RAW Schema written to Parquet and read from Parquet are inconsistent
> ---
>
> Key: KYLIN-5200
> URL: https://issues.apache.org/jira/browse/KYLIN-5200
> Project: Kylin
> Issue Type: Bug
> Components: Metadata
>Affects Versions: v4.0.1
>Reporter: zhaoliu
>Priority: Major
>
> I created a cube on kylin version 4.0.1. One of the measures is defined as
> raw. When I query after building, I find that there are inconsistencies
> between parquet schema and spark schema. When building cube, the raw measure
> written to parquet is processed with spark max, and the datatype of Max is
> child Datatype, in my cube, child Datatype is decimal (19,4). However, when I
> query through SQL, raw is uniformly specified as binarytype in tablescanpaln.
> Therefore, I wonder if the structtype of raw in tablescanpaln also uses child
> dataType ？
> when build ,Raw type is child.dataType
> @see org.apache.kylin.engine.spark.job.CuboidAggregator
> {code:java}
> measure.expression.toUpperCase(Locale.ROOT) match {
> case "MAX" =>
> max(columns.head).as(id.toString)
> case "MIN" =>
> min(columns.head).as(id.toString)
> case "SUM" =>
> sum(columns.head).as(id.toString)
> case "COUNT" =>
> if (reuseLayout) {
> sum(columns.head).as(id.toString)
> } else {
> count(columns.head).as(id.toString)
> }
> case "COUNT_DISTINCT" =>
> // for test
> if (isSparkSql) {
> countDistinct(columns.head).as(id.toString)
> } else {
> val cdAggregate = getCountDistinctAggregate(columns,
> measure.returnType, reuseLayout)
> new Column(cdAggregate.toAggregateExpression()).as(id.toString)
> }
> case "TOP_N" =>
> // Uses new TopN aggregate function
> // located in
> kylin-spark-project/kylin-spark-common/src/main/scala/org/apache/spark/sql/udaf/TopN.scala
> val schema = StructType(measure.pra.map { col =>
> val dateType = col.dataType
> if (col == measure) {
> StructField(s"MEASURE_${col.columnName}", dateType)
> } else {
> StructField(s"DIMENSION_${col.columnName}", dateType)
> }
> })
> if (reuseLayout) {
> new Column(ReuseTopN(measure.returnType.precision, schema,
> columns.head.expr)
> .toAggregateExpression()).as(id.toString)
> } else {
> new Column(EncodeTopN(measure.returnType.precision, schema,
> columns.head.expr, columns.drop(1).map(_.expr))
> .toAggregateExpression()).as(id.toString)
> }
> case "PERCENTILE_APPROX" =>
> val udfName =
> UdfManager.register(measure.returnType.toKylinDataType, measure.expression,
> null, !reuseLayout)
> if (!reuseLayout) {
> callUDF(udfName, columns.head.cast(StringType)).as(id.toString)
> } else {
> callUDF(udfName, columns.head).as(id.toString)
> }
> case _ =>
> max(columns.head).as(id.toString) // Raw matcher here，but max
> dataType is child.dataType
> }
> }.toSeq
> {code}
> But when query，Raw StructType is BinaryType.
> @see org.apache.kylin.query.runtime.plans.TableScanPlan
> ，org.apache.spark.sql.utils.SparkTypeUtil
> {code:java}
> def toSparkType(dataTp: DataType, isSum: Boolean = false):
> org.apache.spark.sql.types.DataType = {
> dataTp.getName match {
> // org.apache.spark.sql.catalyst.expressions.aggregate.Sum#resultType
> case "decimal" =>
> if (isSum) {
> val i = dataTp.getPrecision + 10
> DecimalType(Math.min(DecimalType.MAX_PRECISION, i), dataTp.getScale)
> }
> else DecimalType(dataTp.getPrecision, dataTp.getScale)
> case "date" => DateType
> case "time" => DateType
> case "timestamp" => TimestampType
> case "datetime" => DateType
> case "tinyint" => if (isSum) LongType else ByteType
> case "smallint" => if (isSum) LongType else ShortType
> case "integer" => if (isSum) LongType else IntegerType
> case "int4" => if (isSum) LongType else IntegerType
> case "bigint" => LongType
> case "long8" => LongType
> case "float" => if (isSum) DoubleType else FloatType
> case "double" => DoubleType
> case tp if tp.startsWith("varchar") =>
[GitHub] [kylin] zhaoliu17 commented on pull request #1896: KYLIN-5200, support raw measure
zhaoliu17 commented on PR #1896: URL: https://github.com/apache/kylin/pull/1896#issuecomment-1214294560 构建时支持 raw 类型计算 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@kylin.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
