[jira] [Created] (KYLIN-3629) NullPointException throws when use preparedStatement cache in some case
Ma Gang created KYLIN-3629:
--
Summary: NullPointException throws when use preparedStatement
cache in some case
Key: KYLIN-3629
URL: https://issues.apache.org/jira/browse/KYLIN-3629
Project: Kylin
Issue Type: Bug
Components: Query Engine
Reporter: Ma Gang
Assignee: Ma Gang
NullPointException throws when use preparedStatement cache in some case that
there is no realization exists in the OLAPContext, for example the following
sql request will cause issue:
{color:#205081}{"sql":"select t1.lstg_site_id,t1.seller_id, count(1) from
(select lstg_site_id,seller_id,count(1) from TEST_KYLIN_FACT where
lstg_site_id=? and seller_id=? group by lstg_site_id,seller_id) t1 left join
(select lstg_site_id,seller_id,count(1) from TEST_KYLIN_FACT where
lstg_site_id=? and seller_id=? group by lstg_site_id,seller_id) t2 on
t1.lstg_site_id = t2.lstg_site_id group by
t1.lstg_site_id,t1.seller_id","offset":0,"limit":5,"params":[\{"className":"java.lang.Integer","value":"100"},\{"className":"java.lang.Integer","value":"1731"},\{"className":"java.lang.Integer","value":"100"},\{"className":"java.lang.Integer","value":"1732"}],"acceptPartial":true,"project":"default","backdoorToggles":
{}}{color}
the exception stacktrace is like:
{color:#205081}org.apache.kylin.rest.exception.InternalErrorException: Unknown
error.\n\tat
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:424)\n\tat
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:351)\n\tat
org.apache.kylin.rest.controller.QueryController.query(QueryController.java:86)\n\tat
sun.reflect.GeneratedMethodAccessor155.invoke(Unknown Source)\n\tat
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat
java.lang.reflect.Method.invoke(Method.java:498)\n\tat
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)\n\tat
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)\n\tat
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)\n\tat
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)\n\tat
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)\n\tat
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)\n\tat
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)\n\tat
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)\n\tat
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)\n\tat
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)\n\tat
javax.servlet.http.HttpServlet.service(HttpServlet.java:661)\n\tat
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)\n\tat
javax.servlet.http.HttpServlet.service(HttpServlet.java:742)\n\tat
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)\n\tat
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)\n\tat
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)\n\tat
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)\n\tat
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)\n\tat
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)\n\tat
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)\n\tat
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)\n\tat
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\n\tat
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)\n\tat
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\n\tat
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)\n\tat
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\n\tat
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)\n\tat
[GitHub] codecov-io commented on issue #288: KYLIN-3597 Improve code smell
codecov-io commented on issue #288: KYLIN-3597 Improve code smell URL: https://github.com/apache/kylin/pull/288#issuecomment-429588453 # [Codecov](https://codecov.io/gh/apache/kylin/pull/288?src=pr=h1) Report > Merging [#288](https://codecov.io/gh/apache/kylin/pull/288?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/fdafecc23a34e1b245dbe409cfd2fc1984717c55?src=pr=desc) will **increase** coverage by `0.01%`. > The diff coverage is `3.26%`. [](https://codecov.io/gh/apache/kylin/pull/288?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#288 +/- ## === + Coverage 21.29% 21.3% +0.01% Complexity 44424442 === Files 10871087 Lines 69953 69966 +13 Branches 10107 10107 === + Hits 14894 14907 +13 - Misses53650 53656 +6 + Partials 14091403 -6 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/288?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...age/hbase/lookup/LookupTableHFilesBulkLoadJob.java](https://codecov.io/gh/apache/kylin/pull/288/diff?src=pr=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9sb29rdXAvTG9va3VwVGFibGVIRmlsZXNCdWxrTG9hZEpvYi5qYXZh) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/288/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==) | `14.68% <2.77%> (ø)` | `36 <2> (ø)` | :arrow_down: | | [...c/main/java/org/apache/kylin/cube/CubeManager.java](https://codecov.io/gh/apache/kylin/pull/288/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL0N1YmVNYW5hZ2VyLmphdmE=) | `39.71% <5.26%> (-0.96%)` | `42 <0> (ø)` | | | [...lin/dict/lookup/cache/RocksDBLookupTableCache.java](https://codecov.io/gh/apache/kylin/pull/288/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGVDYWNoZS5qYXZh) | `76.68% <0%> (+0.51%)` | `27% <0%> (ø)` | :arrow_down: | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/288/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `70.21% <0%> (+3.64%)` | `7% <0%> (ø)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/288?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/288?src=pr=footer). Last update [fdafecc...45af68f](https://codecov.io/gh/apache/kylin/pull/288?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] coveralls edited a comment on issue #288: KYLIN-3597 Improve code smell
coveralls edited a comment on issue #288: KYLIN-3597 Improve code smell URL: https://github.com/apache/kylin/pull/288#issuecomment-429504585 ## Pull Request Test Coverage Report for [Build 3765](https://coveralls.io/builds/19508188) * **3** of **92** **(3.26%)** changed or added relevant lines in **3** files are covered. * **2** unchanged lines in **1** file lost coverage. * Overall coverage increased (+**0.006%**) to **23.314%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [storage-hbase/src/main/java/org/apache/kylin/storage/hbase/lookup/LookupTableHFilesBulkLoadJob.java](https://coveralls.io/builds/19508188/source?filename=storage-hbase%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstorage%2Fhbase%2Flookup%2FLookupTableHFilesBulkLoadJob.java#L88) | 0 | 1 | 0.0% | [core-cube/src/main/java/org/apache/kylin/cube/CubeManager.java](https://coveralls.io/builds/19508188/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2FCubeManager.java#L400) | 1 | 19 | 5.26% | [core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java](https://coveralls.io/builds/19508188/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2FKylinConfigBase.java#L93) | 2 | 72 | 2.78% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java](https://coveralls.io/builds/19508188/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2FKylinConfigBase.java#L988) | 2 | 15.82% | | Totals | [](https://coveralls.io/builds/19508188) | | :-- | --: | | Change from base [Build 3758](https://coveralls.io/builds/19483243): | 0.006% | | Covered Lines: | 16312 | | Relevant Lines: | 69966 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] shaofengshi commented on a change in pull request #286: KYLIN-3617 Use job's cache in job scheduler
shaofengshi commented on a change in pull request #286: KYLIN-3617 Use job's
cache in job scheduler
URL: https://github.com/apache/kylin/pull/286#discussion_r224976002
##
File path: core-job/src/main/java/org/apache/kylin/job/dao/ExecutableDao.java
##
@@ -391,4 +405,13 @@ public void deleteJobOutput(String uuid) throws
PersistentException {
throw new PersistentException(e);
}
}
+
+public void reloadAll() throws IOException {
+try (AutoReadWriteLock.AutoLock lock =
executableDigestMapLock.lockForWrite()) {
+executableDigestCrud.reloadAll();
Review comment:
Why this is in "lockForWrite()" but the next reload is in "lockForRead"?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
Re: LDAP Sync issue - Empty filter; nested exception is javax.naming.directory.InvalidSearchFilterException
What's your Kylin version? Which document are you following?
I just noticed the LDAP configuration document has been out of date for
some time. We will update it. But seems you're not following that -:).
BTW, there is no the configuration named
"kylin.security.ldap.user-search-filter" in Apache Kylin code base.
Jiatao Tao <245915...@qq.com> 于2018年10月13日周六 下午9:42写道：
> Hi,
> Can you try command '"ldapsearch" to get the users/groups you wanted
> first?
>
> ---
> Regards!
> Aron Tao
>
>
>
> On [DATE], "[NAME]" <[ADDRESS]> wrote:
>
> Hi - Please help us to solve the below issue to sync with LDAP
>
> Below is the kylin ldap configuration .
>
> kylin.security.profile=ldap
> kylin.security.acl.admin-role=ROLE_KYLIN-ADMIN-GROUP
>
> kylin.security.ldap.connection-server=ldap://:389
> kylin.security.ldap.connection-username=*
> kylin.security.ldap.connection-password=*
>
>
> kylin.security.ldap.user-search-base=OU=,OU=Applications,DC=,DC=com
>
> kylin.security.ldap.user-search-pattern=(&(cn={0})) --tried with many
> options but still same issue as below (CN=*,
> OU=Applications,OU=Groups,DC=bcbsfl,DC=com) (uid=*)
>
> #kylin.security.ldap.user-search-filter=CN=*,OU=Hadoop,DC=,DC=com
>
>
> kylin.security.ldap.user-group-search-base=OU=Requested,OU=Groups,DC=*,DC=com
>
>
>
> org.springframework.security.authentication.InternalAuthenticationServiceException:
> Empty filter; nested exception is
> javax.naming.directory.InvalidSearchFilterException: Empty filter;
> remaining
> name '/'
> at
>
> org.springframework.security.ldap.authentication.LdapAuthenticationProvider.doAuthentication(LdapAuthenticationProvider.java:206)
> at
>
> org.springframework.security.ldap.authentication.AbstractLdapAuthenticationProvider.authenticate(AbstractLdapAuthenticationProvider.java:85)
> at
>
> org.apache.kylin.rest.security.KylinAuthenticationProvider.authenticate(KylinAuthenticationProvider.java:94)
> at
>
> org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:174)
> at
>
> org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:199)
> at
>
> org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:180)
> at
>
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
>
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
>
> org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
> at
>
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
>
> org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
> at
>
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
>
> org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:64)
> at
>
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
>
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
>
> org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
>
>
> --
> Sent from: http://apache-kylin.74782.x6.nabble.com/
>
>
>
>
>
--
Best regards,
Shaofeng Shi 史少锋
[GitHub] shaofengshi commented on a change in pull request #288: KYLIN-3597 Improve code smell
shaofengshi commented on a change in pull request #288: KYLIN-3597 Improve code
smell
URL: https://github.com/apache/kylin/pull/288#discussion_r224960913
##
File path: core-cube/src/main/java/org/apache/kylin/cube/CubeManager.java
##
@@ -100,6 +100,9 @@ public static CubeManager getInstance(KylinConfig config) {
return config.getManager(CubeManager.class);
}
+private static final String FOR_CUBE = "For cube ";
+private static final String SEGMENT = ", segment ";
Review comment:
I don't think this worth to declare.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
Re: LDAP Sync issue - Empty filter; nested exception is javax.naming.directory.InvalidSearchFilterException
Hi,
Can you try command '"ldapsearch" to get the users/groups you wanted first?
---
Regards!
Aron Tao
On [DATE], "[NAME]" <[ADDRESS]> wrote:
Hi - Please help us to solve the below issue to sync with LDAP
Below is the kylin ldap configuration .
kylin.security.profile=ldap
kylin.security.acl.admin-role=ROLE_KYLIN-ADMIN-GROUP
kylin.security.ldap.connection-server=ldap://:389
kylin.security.ldap.connection-username=*
kylin.security.ldap.connection-password=*
kylin.security.ldap.user-search-base=OU=,OU=Applications,DC=,DC=com
kylin.security.ldap.user-search-pattern=(&(cn={0})) --tried with many
options but still same issue as below (CN=*,
OU=Applications,OU=Groups,DC=bcbsfl,DC=com) (uid=*)
#kylin.security.ldap.user-search-filter=CN=*,OU=Hadoop,DC=,DC=com
kylin.security.ldap.user-group-search-base=OU=Requested,OU=Groups,DC=*,DC=com
org.springframework.security.authentication.InternalAuthenticationServiceException:
Empty filter; nested exception is
javax.naming.directory.InvalidSearchFilterException: Empty filter; remaining
name '/'
at
org.springframework.security.ldap.authentication.LdapAuthenticationProvider.doAuthentication(LdapAuthenticationProvider.java:206)
at
org.springframework.security.ldap.authentication.AbstractLdapAuthenticationProvider.authenticate(AbstractLdapAuthenticationProvider.java:85)
at
org.apache.kylin.rest.security.KylinAuthenticationProvider.authenticate(KylinAuthenticationProvider.java:94)
at
org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:174)
at
org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:199)
at
org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:180)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:64)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
--
Sent from: http://apache-kylin.74782.x6.nabble.com/
[jira] [Created] (KYLIN-3628) The wrong result when a query with one lookup table
Na Zhai created KYLIN-3628: -- Summary: The wrong result when a query with one lookup table Key: KYLIN-3628 URL: https://issues.apache.org/jira/browse/KYLIN-3628 Project: Kylin Issue Type: Improvement Reporter: Na Zhai Assignee: Na Zhai Fix For: v2.6.0 Two cubes use the same lookup table, and then the lookup table data in Hive changes. One of Kylin's cubes builds the new data, and the other doesn't. When Kylin queries the lookup table, the data gets confused. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
