[jira] [Commented] (KYLIN-4391) Bump Flink version from 1.9.0 to 1.9.2
[ https://issues.apache.org/jira/browse/KYLIN-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049966#comment-17049966 ] Harvey Yue commented on KYLIN-4391: --- After discussion with Vino, I will take this task to verify building cube with Flink 1.9.2 and 1.10.0 version > Bump Flink version from 1.9.0 to 1.9.2 > -- > > Key: KYLIN-4391 > URL: https://issues.apache.org/jira/browse/KYLIN-4391 > Project: Kylin > Issue Type: Sub-task > Components: Flink Engine >Reporter: vinoyang >Assignee: vinoyang >Priority: Major > > Flink 1.9.2 is a stable version in 1.9 series. We should upgrade to this > version for flink engine. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (KYLIN-4391) Bump Flink version from 1.9.0 to 1.9.2
[ https://issues.apache.org/jira/browse/KYLIN-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harvey Yue reassigned KYLIN-4391: - Assignee: Harvey Yue (was: vinoyang) > Bump Flink version from 1.9.0 to 1.9.2 > -- > > Key: KYLIN-4391 > URL: https://issues.apache.org/jira/browse/KYLIN-4391 > Project: Kylin > Issue Type: Sub-task > Components: Flink Engine >Reporter: vinoyang >Assignee: Harvey Yue >Priority: Major > > Flink 1.9.2 is a stable version in 1.9 series. We should upgrade to this > version for flink engine. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3274) Please delete old releases from mirroring system
[ https://issues.apache.org/jira/browse/KYLIN-3274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-3274. --- Resolution: Fixed > Please delete old releases from mirroring system > > > Key: KYLIN-3274 > URL: https://issues.apache.org/jira/browse/KYLIN-3274 > Project: Kylin > Issue Type: Bug > Environment: https://dist.apache.org/repos/dist/release/kylin/ >Reporter: Sebb >Assignee: Chunen Ni >Priority: Major > > To reduce the load on the ASF mirrors, projects are required to delete old > releases [1] > Please can you remove all non-current releases? > [From the download page, it looks like all but the latest release (2.3.0) are > superseded.] > It's unfair to expect the 3rd party mirrors to carry old releases. > Note that older releases can still be linked from the download page, but such > links should use the archive server at: > https://archive.apache.org/dist/incubator/mxnet/ > Thanks! > [1] http://www.apache.org/dev/release.html#when-to-archive -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-3274) Please delete old releases from mirroring system
[ https://issues.apache.org/jira/browse/KYLIN-3274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049915#comment-17049915 ] nichunen commented on KYLIN-3274: - Done. Close this issue > Please delete old releases from mirroring system > > > Key: KYLIN-3274 > URL: https://issues.apache.org/jira/browse/KYLIN-3274 > Project: Kylin > Issue Type: Bug > Environment: https://dist.apache.org/repos/dist/release/kylin/ >Reporter: Sebb >Assignee: Chunen Ni >Priority: Major > > To reduce the load on the ASF mirrors, projects are required to delete old > releases [1] > Please can you remove all non-current releases? > [From the download page, it looks like all but the latest release (2.3.0) are > superseded.] > It's unfair to expect the 3rd party mirrors to carry old releases. > Note that older releases can still be linked from the download page, but such > links should use the archive server at: > https://archive.apache.org/dist/incubator/mxnet/ > Thanks! > [1] http://www.apache.org/dev/release.html#when-to-archive -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (KYLIN-4396) File Descriptor Leakage in MR Build Engine
[
https://issues.apache.org/jira/browse/KYLIN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049867#comment-17049867
]
Xiaoxiang Yu edited comment on KYLIN-4396 at 3/3/20 3:48 AM:
-
h2. How to Test
I deployed two kylin instance, one built from master, another built from master
too but apply my patch.
I schedule to submit about 20 building job(Realtime OLAP) for one hour, and
last for 12 hours.
After 12 hours, we compared the count of following command:
{code:java}
netstat -anp | grep PID | grep CLOSE_WAIT | wc -l{code}
h2. Before fix
!image-2020-03-03-11-47-15-845.png!
!image-2020-03-03-11-26-17-536.png!
!image-2020-03-03-11-26-47-382.png!
h2. After Fix !image-2020-03-03-11-28-31-161.png!
was (Author: hit_lacus):
h2. How to Test
I deployed two kylin instance, one built from master, another built from master
too but apply my patch.
I schedule to submit about 20 building job for one hour, and last for 12 hours.
After 12 hours, we compared the count of following command:
{code:java}
netstat -anp | grep PID | grep CLOSE_WAIT | wc -l{code}
h2. Before fix
!image-2020-03-03-11-47-15-845.png!
!image-2020-03-03-11-26-17-536.png!
!image-2020-03-03-11-26-47-382.png!
h2. After Fix !image-2020-03-03-11-28-31-161.png!
> File Descriptor Leakage in MR Build Engine
> --
>
> Key: KYLIN-4396
> URL: https://issues.apache.org/jira/browse/KYLIN-4396
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v3.0.0, v3.0.1
>Reporter: Xiaoxiang Yu
>Assignee: Xiaoxiang Yu
>Priority: Major
> Fix For: v3.1.0, v3.0.2
>
> Attachments: image-2020-03-03-11-26-17-536.png,
> image-2020-03-03-11-26-47-382.png, image-2020-03-03-11-27-35-009.png,
> image-2020-03-03-11-28-31-161.png, image-2020-03-03-11-47-15-845.png
>
>
> After runing for one or two days, Job Server failed to response to any http
> requests, but the process looks good, without high CPU pressure etc. In
> _kylin.out_, we find error message indicate that process has hit the limit of
> *FD.*
> After checking detail, we find many fds are not released after a long period,
> most of them are socket in state, that is *CLOSE_WAIT*.
> !https://user-images.githubusercontent.com/14030549/72678560-0fc7e580-3ae2-11ea-85ea-5dcb40e97bd7.png!
>
> !https://user-images.githubusercontent.com/14030549/72678638-c9bf5180-3ae2-11ea-880c-7f8e34d17a8c.png!
>
>
> {code:java}
> 一月 19, 2020 5:13:51 下午 org.apache.catalina.core.StandardWrapperValve invoke
> 严重: Servlet.service() for servlet [default] in context with path [/kylin]
> threw exception java.io.FileNotFoundException:
> /root/xiaoxiang.yu/deploy/kylin-all/tomcat/webapps/kylin/js/scripts.min.20191209061232.js
> (打开的文件过多) at java.io.FileInputStream.open0(Native Method) at
> java.io.FileInputStream.open(FileInputStream.java:195) at
> java.io.FileInputStream.(FileInputStream.java:138) at
> org.apache.naming.resources.FileDirContext$FileResource.streamContent(FileDirContext.java:1008)
> at
> org.apache.catalina.servlets.DefaultServlet.copy(DefaultServlet.java:2084) at
> org.apache.catalina.servlets.DefaultServlet.serveResource(DefaultServlet.java:1023)
> at
> org.apache.catalina.servlets.DefaultServlet.doGet(DefaultServlet.java:453) at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:624) at
> org.apache.catalina.servlets.DefaultServlet.service(DefaultServlet.java:433)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
>
[jira] [Comment Edited] (KYLIN-4396) File Descriptor Leakage in MR Build Engine
[
https://issues.apache.org/jira/browse/KYLIN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049867#comment-17049867
]
Xiaoxiang Yu edited comment on KYLIN-4396 at 3/3/20 3:47 AM:
-
h2. How to Test
I deployed two kylin instance, one built from master, another built from master
too but apply my patch.
I schedule to submit about 20 building job for one hour, and last for 12 hours.
After 12 hours, we compared the count of following command:
{code:java}
netstat -anp | grep PID | grep CLOSE_WAIT | wc -l{code}
h2. Before fix
!image-2020-03-03-11-47-15-845.png!
!image-2020-03-03-11-26-17-536.png!
!image-2020-03-03-11-26-47-382.png!
h2. After Fix !image-2020-03-03-11-28-31-161.png!
was (Author: hit_lacus):
h2. How to Test
I deployed two kylin instance, one built from master, another built from master
too but apply my patch.
I schedule to submit about 20 building job for one hour, and last for 12 hours.
After 12 hours, we compared the count of following command:
{code:java}
netstat -anp | grep PID | grep CLOSE_WAIT | wc -l{code}
h2. Before fix
!image-2020-03-03-11-26-17-536.png!
!image-2020-03-03-11-26-47-382.png!
h2. After Fix !image-2020-03-03-11-28-31-161.png!
> File Descriptor Leakage in MR Build Engine
> --
>
> Key: KYLIN-4396
> URL: https://issues.apache.org/jira/browse/KYLIN-4396
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v3.0.0, v3.0.1
>Reporter: Xiaoxiang Yu
>Assignee: Xiaoxiang Yu
>Priority: Major
> Fix For: v3.1.0, v3.0.2
>
> Attachments: image-2020-03-03-11-26-17-536.png,
> image-2020-03-03-11-26-47-382.png, image-2020-03-03-11-27-35-009.png,
> image-2020-03-03-11-28-31-161.png, image-2020-03-03-11-47-15-845.png
>
>
> After runing for one or two days, Job Server failed to response to any http
> requests, but the process looks good, without high CPU pressure etc. In
> _kylin.out_, we find error message indicate that process has hit the limit of
> *FD.*
> After checking detail, we find many fds are not released after a long period,
> most of them are socket in state, that is *CLOSE_WAIT*.
> !https://user-images.githubusercontent.com/14030549/72678560-0fc7e580-3ae2-11ea-85ea-5dcb40e97bd7.png!
>
> !https://user-images.githubusercontent.com/14030549/72678638-c9bf5180-3ae2-11ea-880c-7f8e34d17a8c.png!
>
>
> {code:java}
> 一月 19, 2020 5:13:51 下午 org.apache.catalina.core.StandardWrapperValve invoke
> 严重: Servlet.service() for servlet [default] in context with path [/kylin]
> threw exception java.io.FileNotFoundException:
> /root/xiaoxiang.yu/deploy/kylin-all/tomcat/webapps/kylin/js/scripts.min.20191209061232.js
> (打开的文件过多) at java.io.FileInputStream.open0(Native Method) at
> java.io.FileInputStream.open(FileInputStream.java:195) at
> java.io.FileInputStream.(FileInputStream.java:138) at
> org.apache.naming.resources.FileDirContext$FileResource.streamContent(FileDirContext.java:1008)
> at
> org.apache.catalina.servlets.DefaultServlet.copy(DefaultServlet.java:2084) at
> org.apache.catalina.servlets.DefaultServlet.serveResource(DefaultServlet.java:1023)
> at
> org.apache.catalina.servlets.DefaultServlet.doGet(DefaultServlet.java:453) at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:624) at
> org.apache.catalina.servlets.DefaultServlet.service(DefaultServlet.java:433)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
[jira] [Updated] (KYLIN-4396) File Descriptor Leakage in MR Build Engine
[
https://issues.apache.org/jira/browse/KYLIN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoxiang Yu updated KYLIN-4396:
Fix Version/s: v3.1.0
> File Descriptor Leakage in MR Build Engine
> --
>
> Key: KYLIN-4396
> URL: https://issues.apache.org/jira/browse/KYLIN-4396
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v3.0.0, v3.0.1
>Reporter: Xiaoxiang Yu
>Assignee: Xiaoxiang Yu
>Priority: Major
> Fix For: v3.1.0, v3.0.2
>
> Attachments: image-2020-03-03-11-26-17-536.png,
> image-2020-03-03-11-26-47-382.png, image-2020-03-03-11-27-35-009.png,
> image-2020-03-03-11-28-31-161.png
>
>
> After runing for one or two days, Job Server failed to response to any http
> requests, but the process looks good, without high CPU pressure etc. In
> _kylin.out_, we find error message indicate that process has hit the limit of
> *FD.*
> After checking detail, we find many fds are not released after a long period,
> most of them are socket in state, that is *CLOSE_WAIT*.
> !https://user-images.githubusercontent.com/14030549/72678560-0fc7e580-3ae2-11ea-85ea-5dcb40e97bd7.png!
>
> !https://user-images.githubusercontent.com/14030549/72678638-c9bf5180-3ae2-11ea-880c-7f8e34d17a8c.png!
>
>
> {code:java}
> 一月 19, 2020 5:13:51 下午 org.apache.catalina.core.StandardWrapperValve invoke
> 严重: Servlet.service() for servlet [default] in context with path [/kylin]
> threw exception java.io.FileNotFoundException:
> /root/xiaoxiang.yu/deploy/kylin-all/tomcat/webapps/kylin/js/scripts.min.20191209061232.js
> (打开的文件过多) at java.io.FileInputStream.open0(Native Method) at
> java.io.FileInputStream.open(FileInputStream.java:195) at
> java.io.FileInputStream.(FileInputStream.java:138) at
> org.apache.naming.resources.FileDirContext$FileResource.streamContent(FileDirContext.java:1008)
> at
> org.apache.catalina.servlets.DefaultServlet.copy(DefaultServlet.java:2084) at
> org.apache.catalina.servlets.DefaultServlet.serveResource(DefaultServlet.java:1023)
> at
> org.apache.catalina.servlets.DefaultServlet.doGet(DefaultServlet.java:453) at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:624) at
> org.apache.catalina.servlets.DefaultServlet.service(DefaultServlet.java:433)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
>
[jira] [Updated] (KYLIN-4396) File Descriptor Leakage in MR Build Engine
[
https://issues.apache.org/jira/browse/KYLIN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoxiang Yu updated KYLIN-4396:
Fix Version/s: (was: v3.1.0)
v3.0.2
> File Descriptor Leakage in MR Build Engine
> --
>
> Key: KYLIN-4396
> URL: https://issues.apache.org/jira/browse/KYLIN-4396
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v3.0.0, v3.0.1
>Reporter: Xiaoxiang Yu
>Assignee: Xiaoxiang Yu
>Priority: Major
> Fix For: v3.0.2
>
> Attachments: image-2020-03-03-11-26-17-536.png,
> image-2020-03-03-11-26-47-382.png, image-2020-03-03-11-27-35-009.png,
> image-2020-03-03-11-28-31-161.png
>
>
> After runing for one or two days, Job Server failed to response to any http
> requests, but the process looks good, without high CPU pressure etc. In
> _kylin.out_, we find error message indicate that process has hit the limit of
> *FD.*
> After checking detail, we find many fds are not released after a long period,
> most of them are socket in state, that is *CLOSE_WAIT*.
> !https://user-images.githubusercontent.com/14030549/72678560-0fc7e580-3ae2-11ea-85ea-5dcb40e97bd7.png!
>
> !https://user-images.githubusercontent.com/14030549/72678638-c9bf5180-3ae2-11ea-880c-7f8e34d17a8c.png!
>
>
> {code:java}
> 一月 19, 2020 5:13:51 下午 org.apache.catalina.core.StandardWrapperValve invoke
> 严重: Servlet.service() for servlet [default] in context with path [/kylin]
> threw exception java.io.FileNotFoundException:
> /root/xiaoxiang.yu/deploy/kylin-all/tomcat/webapps/kylin/js/scripts.min.20191209061232.js
> (打开的文件过多) at java.io.FileInputStream.open0(Native Method) at
> java.io.FileInputStream.open(FileInputStream.java:195) at
> java.io.FileInputStream.(FileInputStream.java:138) at
> org.apache.naming.resources.FileDirContext$FileResource.streamContent(FileDirContext.java:1008)
> at
> org.apache.catalina.servlets.DefaultServlet.copy(DefaultServlet.java:2084) at
> org.apache.catalina.servlets.DefaultServlet.serveResource(DefaultServlet.java:1023)
> at
> org.apache.catalina.servlets.DefaultServlet.doGet(DefaultServlet.java:453) at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:624) at
> org.apache.catalina.servlets.DefaultServlet.service(DefaultServlet.java:433)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
> at
>
[jira] [Comment Edited] (KYLIN-4396) File Descriptor Leakage in MR Build Engine
[
https://issues.apache.org/jira/browse/KYLIN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049867#comment-17049867
]
Xiaoxiang Yu edited comment on KYLIN-4396 at 3/3/20 3:43 AM:
-
h2. How to Test
I deployed two kylin instance, one built from master, another built from master
too but apply my patch.
I schedule to submit about 20 building job for one hour, and last for 12 hours.
After 12 hours, we compared the count of following command:
{code:java}
netstat -anp | grep PID | grep CLOSE_WAIT | wc -l{code}
h2. Before fix
!image-2020-03-03-11-26-17-536.png!
!image-2020-03-03-11-26-47-382.png!
h2. After Fix !image-2020-03-03-11-28-31-161.png!
was (Author: hit_lacus):
h2. Before fix
!image-2020-03-03-11-26-17-536.png!
!image-2020-03-03-11-26-47-382.png!
h2. After Fix !image-2020-03-03-11-28-31-161.png!
> File Descriptor Leakage in MR Build Engine
> --
>
> Key: KYLIN-4396
> URL: https://issues.apache.org/jira/browse/KYLIN-4396
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v3.0.0, v3.0.1
>Reporter: Xiaoxiang Yu
>Assignee: Xiaoxiang Yu
>Priority: Major
> Fix For: v3.1.0
>
> Attachments: image-2020-03-03-11-26-17-536.png,
> image-2020-03-03-11-26-47-382.png, image-2020-03-03-11-27-35-009.png,
> image-2020-03-03-11-28-31-161.png
>
>
> After runing for one or two days, Job Server failed to response to any http
> requests, but the process looks good, without high CPU pressure etc. In
> _kylin.out_, we find error message indicate that process has hit the limit of
> *FD.*
> After checking detail, we find many fds are not released after a long period,
> most of them are socket in state, that is *CLOSE_WAIT*.
> !https://user-images.githubusercontent.com/14030549/72678560-0fc7e580-3ae2-11ea-85ea-5dcb40e97bd7.png!
>
> !https://user-images.githubusercontent.com/14030549/72678638-c9bf5180-3ae2-11ea-880c-7f8e34d17a8c.png!
>
>
> {code:java}
> 一月 19, 2020 5:13:51 下午 org.apache.catalina.core.StandardWrapperValve invoke
> 严重: Servlet.service() for servlet [default] in context with path [/kylin]
> threw exception java.io.FileNotFoundException:
> /root/xiaoxiang.yu/deploy/kylin-all/tomcat/webapps/kylin/js/scripts.min.20191209061232.js
> (打开的文件过多) at java.io.FileInputStream.open0(Native Method) at
> java.io.FileInputStream.open(FileInputStream.java:195) at
> java.io.FileInputStream.(FileInputStream.java:138) at
> org.apache.naming.resources.FileDirContext$FileResource.streamContent(FileDirContext.java:1008)
> at
> org.apache.catalina.servlets.DefaultServlet.copy(DefaultServlet.java:2084) at
> org.apache.catalina.servlets.DefaultServlet.serveResource(DefaultServlet.java:1023)
> at
> org.apache.catalina.servlets.DefaultServlet.doGet(DefaultServlet.java:453) at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:624) at
> org.apache.catalina.servlets.DefaultServlet.service(DefaultServlet.java:433)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
>
[jira] [Comment Edited] (KYLIN-4396) File Descriptor Leakage in MR Build Engine
[
https://issues.apache.org/jira/browse/KYLIN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049867#comment-17049867
]
Xiaoxiang Yu edited comment on KYLIN-4396 at 3/3/20 3:29 AM:
-
h2. Before fix
!image-2020-03-03-11-26-17-536.png!
!image-2020-03-03-11-26-47-382.png!
h2. After Fix !image-2020-03-03-11-28-31-161.png!
was (Author: hit_lacus):
h2. Before fi
!image-2020-03-03-11-26-17-536.png!
!image-2020-03-03-11-26-47-382.png!
h2. After Fix !image-2020-03-03-11-28-31-161.png!
> File Descriptor Leakage in MR Build Engine
> --
>
> Key: KYLIN-4396
> URL: https://issues.apache.org/jira/browse/KYLIN-4396
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v3.0.0, v3.0.1
>Reporter: Xiaoxiang Yu
>Assignee: Xiaoxiang Yu
>Priority: Major
> Fix For: v3.1.0
>
> Attachments: image-2020-03-03-11-26-17-536.png,
> image-2020-03-03-11-26-47-382.png, image-2020-03-03-11-27-35-009.png,
> image-2020-03-03-11-28-31-161.png
>
>
> After runing for one or two days, Job Server failed to response to any http
> requests, but the process looks good, without high CPU pressure etc. In
> _kylin.out_, we find error message indicate that process has hit the limit of
> *FD.*
> After checking detail, we find many fds are not released after a long period,
> most of them are socket in state, that is *CLOSE_WAIT*.
> !https://user-images.githubusercontent.com/14030549/72678560-0fc7e580-3ae2-11ea-85ea-5dcb40e97bd7.png!
>
> !https://user-images.githubusercontent.com/14030549/72678638-c9bf5180-3ae2-11ea-880c-7f8e34d17a8c.png!
>
>
> {code:java}
> 一月 19, 2020 5:13:51 下午 org.apache.catalina.core.StandardWrapperValve invoke
> 严重: Servlet.service() for servlet [default] in context with path [/kylin]
> threw exception java.io.FileNotFoundException:
> /root/xiaoxiang.yu/deploy/kylin-all/tomcat/webapps/kylin/js/scripts.min.20191209061232.js
> (打开的文件过多) at java.io.FileInputStream.open0(Native Method) at
> java.io.FileInputStream.open(FileInputStream.java:195) at
> java.io.FileInputStream.(FileInputStream.java:138) at
> org.apache.naming.resources.FileDirContext$FileResource.streamContent(FileDirContext.java:1008)
> at
> org.apache.catalina.servlets.DefaultServlet.copy(DefaultServlet.java:2084) at
> org.apache.catalina.servlets.DefaultServlet.serveResource(DefaultServlet.java:1023)
> at
> org.apache.catalina.servlets.DefaultServlet.doGet(DefaultServlet.java:453) at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:624) at
> org.apache.catalina.servlets.DefaultServlet.service(DefaultServlet.java:433)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
>
[jira] [Commented] (KYLIN-4396) File Descriptor Leakage in MR Build Engine
[
https://issues.apache.org/jira/browse/KYLIN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049867#comment-17049867
]
Xiaoxiang Yu commented on KYLIN-4396:
-
h2. Before fi
!image-2020-03-03-11-26-17-536.png!
!image-2020-03-03-11-26-47-382.png!
h2. After Fix !image-2020-03-03-11-28-31-161.png!
> File Descriptor Leakage in MR Build Engine
> --
>
> Key: KYLIN-4396
> URL: https://issues.apache.org/jira/browse/KYLIN-4396
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v3.0.0, v3.0.1
>Reporter: Xiaoxiang Yu
>Assignee: Xiaoxiang Yu
>Priority: Major
> Fix For: v3.1.0
>
> Attachments: image-2020-03-03-11-26-17-536.png,
> image-2020-03-03-11-26-47-382.png, image-2020-03-03-11-27-35-009.png,
> image-2020-03-03-11-28-31-161.png
>
>
> After runing for one or two days, Job Server failed to response to any http
> requests, but the process looks good, without high CPU pressure etc. In
> _kylin.out_, we find error message indicate that process has hit the limit of
> *FD.*
> After checking detail, we find many fds are not released after a long period,
> most of them are socket in state, that is *CLOSE_WAIT*.
> !https://user-images.githubusercontent.com/14030549/72678560-0fc7e580-3ae2-11ea-85ea-5dcb40e97bd7.png!
>
> !https://user-images.githubusercontent.com/14030549/72678638-c9bf5180-3ae2-11ea-880c-7f8e34d17a8c.png!
>
>
> {code:java}
> 一月 19, 2020 5:13:51 下午 org.apache.catalina.core.StandardWrapperValve invoke
> 严重: Servlet.service() for servlet [default] in context with path [/kylin]
> threw exception java.io.FileNotFoundException:
> /root/xiaoxiang.yu/deploy/kylin-all/tomcat/webapps/kylin/js/scripts.min.20191209061232.js
> (打开的文件过多) at java.io.FileInputStream.open0(Native Method) at
> java.io.FileInputStream.open(FileInputStream.java:195) at
> java.io.FileInputStream.(FileInputStream.java:138) at
> org.apache.naming.resources.FileDirContext$FileResource.streamContent(FileDirContext.java:1008)
> at
> org.apache.catalina.servlets.DefaultServlet.copy(DefaultServlet.java:2084) at
> org.apache.catalina.servlets.DefaultServlet.serveResource(DefaultServlet.java:1023)
> at
> org.apache.catalina.servlets.DefaultServlet.doGet(DefaultServlet.java:453) at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:624) at
> org.apache.catalina.servlets.DefaultServlet.service(DefaultServlet.java:433)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
>
[jira] [Updated] (KYLIN-4396) File Descriptor Leakage in MR Build Engine
[
https://issues.apache.org/jira/browse/KYLIN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoxiang Yu updated KYLIN-4396:
Attachment: image-2020-03-03-11-28-31-161.png
> File Descriptor Leakage in MR Build Engine
> --
>
> Key: KYLIN-4396
> URL: https://issues.apache.org/jira/browse/KYLIN-4396
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v3.0.0, v3.0.1
>Reporter: Xiaoxiang Yu
>Assignee: Xiaoxiang Yu
>Priority: Major
> Fix For: v3.1.0
>
> Attachments: image-2020-03-03-11-26-17-536.png,
> image-2020-03-03-11-26-47-382.png, image-2020-03-03-11-27-35-009.png,
> image-2020-03-03-11-28-31-161.png
>
>
> After runing for one or two days, Job Server failed to response to any http
> requests, but the process looks good, without high CPU pressure etc. In
> _kylin.out_, we find error message indicate that process has hit the limit of
> *FD.*
> After checking detail, we find many fds are not released after a long period,
> most of them are socket in state, that is *CLOSE_WAIT*.
> !https://user-images.githubusercontent.com/14030549/72678560-0fc7e580-3ae2-11ea-85ea-5dcb40e97bd7.png!
>
> !https://user-images.githubusercontent.com/14030549/72678638-c9bf5180-3ae2-11ea-880c-7f8e34d17a8c.png!
>
>
> {code:java}
> 一月 19, 2020 5:13:51 下午 org.apache.catalina.core.StandardWrapperValve invoke
> 严重: Servlet.service() for servlet [default] in context with path [/kylin]
> threw exception java.io.FileNotFoundException:
> /root/xiaoxiang.yu/deploy/kylin-all/tomcat/webapps/kylin/js/scripts.min.20191209061232.js
> (打开的文件过多) at java.io.FileInputStream.open0(Native Method) at
> java.io.FileInputStream.open(FileInputStream.java:195) at
> java.io.FileInputStream.(FileInputStream.java:138) at
> org.apache.naming.resources.FileDirContext$FileResource.streamContent(FileDirContext.java:1008)
> at
> org.apache.catalina.servlets.DefaultServlet.copy(DefaultServlet.java:2084) at
> org.apache.catalina.servlets.DefaultServlet.serveResource(DefaultServlet.java:1023)
> at
> org.apache.catalina.servlets.DefaultServlet.doGet(DefaultServlet.java:453) at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:624) at
> org.apache.catalina.servlets.DefaultServlet.service(DefaultServlet.java:433)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
>
[jira] [Updated] (KYLIN-4396) File Descriptor Leakage in MR Build Engine
[
https://issues.apache.org/jira/browse/KYLIN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoxiang Yu updated KYLIN-4396:
Attachment: image-2020-03-03-11-27-35-009.png
> File Descriptor Leakage in MR Build Engine
> --
>
> Key: KYLIN-4396
> URL: https://issues.apache.org/jira/browse/KYLIN-4396
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v3.0.0, v3.0.1
>Reporter: Xiaoxiang Yu
>Assignee: Xiaoxiang Yu
>Priority: Major
> Fix For: v3.1.0
>
> Attachments: image-2020-03-03-11-26-17-536.png,
> image-2020-03-03-11-26-47-382.png, image-2020-03-03-11-27-35-009.png
>
>
> After runing for one or two days, Job Server failed to response to any http
> requests, but the process looks good, without high CPU pressure etc. In
> _kylin.out_, we find error message indicate that process has hit the limit of
> *FD.*
> After checking detail, we find many fds are not released after a long period,
> most of them are socket in state, that is *CLOSE_WAIT*.
> !https://user-images.githubusercontent.com/14030549/72678560-0fc7e580-3ae2-11ea-85ea-5dcb40e97bd7.png!
>
> !https://user-images.githubusercontent.com/14030549/72678638-c9bf5180-3ae2-11ea-880c-7f8e34d17a8c.png!
>
>
> {code:java}
> 一月 19, 2020 5:13:51 下午 org.apache.catalina.core.StandardWrapperValve invoke
> 严重: Servlet.service() for servlet [default] in context with path [/kylin]
> threw exception java.io.FileNotFoundException:
> /root/xiaoxiang.yu/deploy/kylin-all/tomcat/webapps/kylin/js/scripts.min.20191209061232.js
> (打开的文件过多) at java.io.FileInputStream.open0(Native Method) at
> java.io.FileInputStream.open(FileInputStream.java:195) at
> java.io.FileInputStream.(FileInputStream.java:138) at
> org.apache.naming.resources.FileDirContext$FileResource.streamContent(FileDirContext.java:1008)
> at
> org.apache.catalina.servlets.DefaultServlet.copy(DefaultServlet.java:2084) at
> org.apache.catalina.servlets.DefaultServlet.serveResource(DefaultServlet.java:1023)
> at
> org.apache.catalina.servlets.DefaultServlet.doGet(DefaultServlet.java:453) at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:624) at
> org.apache.catalina.servlets.DefaultServlet.service(DefaultServlet.java:433)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
>
[jira] [Assigned] (KYLIN-4399) Redirect http url is not override as https when using vip
[
https://issues.apache.org/jira/browse/KYLIN-4399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhong Yanghong reassigned KYLIN-4399:
-
Assignee: Julian Pan (was: Zhong Yanghong)
> Redirect http url is not override as https when using vip
> -
>
> Key: KYLIN-4399
> URL: https://issues.apache.org/jira/browse/KYLIN-4399
> Project: Kylin
> Issue Type: Improvement
> Components: Website
>Reporter: Zhong Yanghong
>Assignee: Julian Pan
>Priority: Minor
>
> Redirect http url is not override as https when using vip, when user input:
> https://${vip_name}/kylin/, user will be redirect to sso url do
> authentication, after authenticated, the sso server will redirect url to
> https://${vip_name}/kylin/saml/sso, after process it will redirect to
> http://${vip_name}/kylin/, there should be some redirect override
> configuration in kylinSecurity.xml that can override the location from
> http://${vip_name}:7070/kylin to https://${vip_name}/kylin/
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4396) File Descriptor Leakage in MR Build Engine
[
https://issues.apache.org/jira/browse/KYLIN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoxiang Yu updated KYLIN-4396:
Attachment: image-2020-03-03-11-26-17-536.png
> File Descriptor Leakage in MR Build Engine
> --
>
> Key: KYLIN-4396
> URL: https://issues.apache.org/jira/browse/KYLIN-4396
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v3.0.0, v3.0.1
>Reporter: Xiaoxiang Yu
>Assignee: Xiaoxiang Yu
>Priority: Major
> Fix For: v3.1.0
>
> Attachments: image-2020-03-03-11-26-17-536.png,
> image-2020-03-03-11-26-47-382.png
>
>
> After runing for one or two days, Job Server failed to response to any http
> requests, but the process looks good, without high CPU pressure etc. In
> _kylin.out_, we find error message indicate that process has hit the limit of
> *FD.*
> After checking detail, we find many fds are not released after a long period,
> most of them are socket in state, that is *CLOSE_WAIT*.
> !https://user-images.githubusercontent.com/14030549/72678560-0fc7e580-3ae2-11ea-85ea-5dcb40e97bd7.png!
>
> !https://user-images.githubusercontent.com/14030549/72678638-c9bf5180-3ae2-11ea-880c-7f8e34d17a8c.png!
>
>
> {code:java}
> 一月 19, 2020 5:13:51 下午 org.apache.catalina.core.StandardWrapperValve invoke
> 严重: Servlet.service() for servlet [default] in context with path [/kylin]
> threw exception java.io.FileNotFoundException:
> /root/xiaoxiang.yu/deploy/kylin-all/tomcat/webapps/kylin/js/scripts.min.20191209061232.js
> (打开的文件过多) at java.io.FileInputStream.open0(Native Method) at
> java.io.FileInputStream.open(FileInputStream.java:195) at
> java.io.FileInputStream.(FileInputStream.java:138) at
> org.apache.naming.resources.FileDirContext$FileResource.streamContent(FileDirContext.java:1008)
> at
> org.apache.catalina.servlets.DefaultServlet.copy(DefaultServlet.java:2084) at
> org.apache.catalina.servlets.DefaultServlet.serveResource(DefaultServlet.java:1023)
> at
> org.apache.catalina.servlets.DefaultServlet.doGet(DefaultServlet.java:453) at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:624) at
> org.apache.catalina.servlets.DefaultServlet.service(DefaultServlet.java:433)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
>
[jira] [Updated] (KYLIN-4396) File Descriptor Leakage in MR Build Engine
[
https://issues.apache.org/jira/browse/KYLIN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoxiang Yu updated KYLIN-4396:
Attachment: image-2020-03-03-11-26-47-382.png
> File Descriptor Leakage in MR Build Engine
> --
>
> Key: KYLIN-4396
> URL: https://issues.apache.org/jira/browse/KYLIN-4396
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v3.0.0, v3.0.1
>Reporter: Xiaoxiang Yu
>Assignee: Xiaoxiang Yu
>Priority: Major
> Fix For: v3.1.0
>
> Attachments: image-2020-03-03-11-26-17-536.png,
> image-2020-03-03-11-26-47-382.png
>
>
> After runing for one or two days, Job Server failed to response to any http
> requests, but the process looks good, without high CPU pressure etc. In
> _kylin.out_, we find error message indicate that process has hit the limit of
> *FD.*
> After checking detail, we find many fds are not released after a long period,
> most of them are socket in state, that is *CLOSE_WAIT*.
> !https://user-images.githubusercontent.com/14030549/72678560-0fc7e580-3ae2-11ea-85ea-5dcb40e97bd7.png!
>
> !https://user-images.githubusercontent.com/14030549/72678638-c9bf5180-3ae2-11ea-880c-7f8e34d17a8c.png!
>
>
> {code:java}
> 一月 19, 2020 5:13:51 下午 org.apache.catalina.core.StandardWrapperValve invoke
> 严重: Servlet.service() for servlet [default] in context with path [/kylin]
> threw exception java.io.FileNotFoundException:
> /root/xiaoxiang.yu/deploy/kylin-all/tomcat/webapps/kylin/js/scripts.min.20191209061232.js
> (打开的文件过多) at java.io.FileInputStream.open0(Native Method) at
> java.io.FileInputStream.open(FileInputStream.java:195) at
> java.io.FileInputStream.(FileInputStream.java:138) at
> org.apache.naming.resources.FileDirContext$FileResource.streamContent(FileDirContext.java:1008)
> at
> org.apache.catalina.servlets.DefaultServlet.copy(DefaultServlet.java:2084) at
> org.apache.catalina.servlets.DefaultServlet.serveResource(DefaultServlet.java:1023)
> at
> org.apache.catalina.servlets.DefaultServlet.doGet(DefaultServlet.java:453) at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:624) at
> org.apache.catalina.servlets.DefaultServlet.service(DefaultServlet.java:433)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
>
[jira] [Assigned] (KYLIN-4398) Display Kylin Version on UI
[ https://issues.apache.org/jira/browse/KYLIN-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong reassigned KYLIN-4398: - Assignee: Julian Pan (was: Zhong Yanghong) > Display Kylin Version on UI > --- > > Key: KYLIN-4398 > URL: https://issues.apache.org/jira/browse/KYLIN-4398 > Project: Kylin > Issue Type: Improvement > Components: Web >Reporter: Zhong Yanghong >Assignee: Julian Pan >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KYLIN-4399) Redirect http url is not override as https when using vip
Zhong Yanghong created KYLIN-4399:
-
Summary: Redirect http url is not override as https when using vip
Key: KYLIN-4399
URL: https://issues.apache.org/jira/browse/KYLIN-4399
Project: Kylin
Issue Type: Improvement
Components: Website
Reporter: Zhong Yanghong
Assignee: Zhong Yanghong
Redirect http url is not override as https when using vip, when user input:
https://${vip_name}/kylin/, user will be redirect to sso url do authentication,
after authenticated, the sso server will redirect url to
https://${vip_name}/kylin/saml/sso, after process it will redirect to
http://${vip_name}/kylin/, there should be some redirect override configuration
in LB that can override the location from http://${vip_name}:7070/kylin to
https://${vip_name}/kylin/
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4399) Redirect http url is not override as https when using vip
[
https://issues.apache.org/jira/browse/KYLIN-4399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhong Yanghong updated KYLIN-4399:
--
Description: Redirect http url is not override as https when using vip,
when user input: https://${vip_name}/kylin/, user will be redirect to sso url
do authentication, after authenticated, the sso server will redirect url to
https://${vip_name}/kylin/saml/sso, after process it will redirect to
http://${vip_name}/kylin/, there should be some redirect override configuration
in kylinSecurity.xml that can override the location from
http://${vip_name}:7070/kylin to https://${vip_name}/kylin/ (was: Redirect
http url is not override as https when using vip, when user input:
https://${vip_name}/kylin/, user will be redirect to sso url do authentication,
after authenticated, the sso server will redirect url to
https://${vip_name}/kylin/saml/sso, after process it will redirect to
http://${vip_name}/kylin/, there should be some redirect override configuration
in LB that can override the location from http://${vip_name}:7070/kylin to
https://${vip_name}/kylin/)
> Redirect http url is not override as https when using vip
> -
>
> Key: KYLIN-4399
> URL: https://issues.apache.org/jira/browse/KYLIN-4399
> Project: Kylin
> Issue Type: Improvement
> Components: Website
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Minor
>
> Redirect http url is not override as https when using vip, when user input:
> https://${vip_name}/kylin/, user will be redirect to sso url do
> authentication, after authenticated, the sso server will redirect url to
> https://${vip_name}/kylin/saml/sso, after process it will redirect to
> http://${vip_name}/kylin/, there should be some redirect override
> configuration in kylinSecurity.xml that can override the location from
> http://${vip_name}:7070/kylin to https://${vip_name}/kylin/
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-3937) 从 kafka 流构建 cube ,发生死锁
[ https://issues.apache.org/jira/browse/KYLIN-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049850#comment-17049850 ] 风语者 commented on KYLIN-3937: 我在用 bitmap 计算精确 uv 的时候, 会出现这个问题. 如果计算精确 uv 的话, 应该是 kylin 在第三步计算全局字典的时候, 需要在 zk 里面加锁, 前一个批次没有加工完, 后续加工无法获取锁导致加工无法进行下去的. 在计算字典的时候, kylin 应该是在 kylin 的 server 里面计算字典, 如果计算全局字典的话, 那需要的内存是不是就越来越大? 如果给的内存不够大, 比如我现在 给的 16 G, 计算不了多久, 就会停止在第三步,无法进行下去, 不知道是不是因为个原因导致的? (数据加工量级 数十亿级别每天) > 从 kafka 流构建 cube ,发生死锁 > -- > > Key: KYLIN-3937 > URL: https://issues.apache.org/jira/browse/KYLIN-3937 > Project: Kylin > Issue Type: Bug > Components: Job Engine, Metadata, Spark Engine >Affects Versions: v2.6.0 > Environment: kylin: 2.6.0 > hdp: 2.5.0.0-1245 >Reporter: 风语者 >Priority: Critical > Attachments: deadlock_kylin.png, kylin_monitors.png > > > 你好，用 kylin 从 kafka 流构建 cube, 每五分钟 构建一次，从日志看到好像发生了死锁现象。 > 目前为止，发生了再次，第一次我将所有正在进行的构建全部 discard , 然后死锁现象还是存在，目前不知道该怎么处理这个问题。烦请查看下吧。多谢多谢。 > 用 spark 进行构建，到第三步：Build Dimension Dictionary 时，发生了死锁。 > 发生死锁时，没有发生 cube 构建错误现象。 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-2971) Fix the wrong "Realization Names" in logQuery when hit cache
[
https://issues.apache.org/jira/browse/KYLIN-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-2971:
Fix Version/s: (was: v2.3.0)
v3.0.1
v2.6.6
v3.1.0
> Fix the wrong "Realization Names" in logQuery when hit cache
>
>
> Key: KYLIN-2971
> URL: https://issues.apache.org/jira/browse/KYLIN-2971
> Project: Kylin
> Issue Type: Bug
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v3.1.0, v3.0.1, v2.6.6
>
> Attachments: APACHE-KYLIN-2971.patch
>
>
> OLAPContext.clearThreadLocalContexts() should be called before a query
> starts. And we should consider query hit cache.
> Example,
> # query Q1 hit project P1 and cube C1;
> # then query Q2 hit project P2 and cube C2;
> # then Q1 comes again, it will show project P1 and {color:#f79232}cube
> C2{color}. However, it should be {color:#f79232}cube C1{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (KYLIN-4398) Display Kylin Version on UI
Zhong Yanghong created KYLIN-4398: - Summary: Display Kylin Version on UI Key: KYLIN-4398 URL: https://issues.apache.org/jira/browse/KYLIN-4398 Project: Kylin Issue Type: Improvement Components: Web Reporter: Zhong Yanghong Assignee: Zhong Yanghong -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (KYLIN-2971) Fix the wrong "Realization Names" in logQuery when hit cache
[
https://issues.apache.org/jira/browse/KYLIN-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhong Yanghong reopened KYLIN-2971:
---
> Fix the wrong "Realization Names" in logQuery when hit cache
>
>
> Key: KYLIN-2971
> URL: https://issues.apache.org/jira/browse/KYLIN-2971
> Project: Kylin
> Issue Type: Bug
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.3.0
>
> Attachments: APACHE-KYLIN-2971.patch
>
>
> OLAPContext.clearThreadLocalContexts() should be called before a query
> starts. And we should consider query hit cache.
> Example,
> # query Q1 hit project P1 and cube C1;
> # then query Q2 hit project P2 and cube C2;
> # then Q1 comes again, it will show project P1 and {color:#f79232}cube
> C2{color}. However, it should be {color:#f79232}cube C1{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-2971) Fix the wrong "Realization Names" in logQuery when hit cache
[
https://issues.apache.org/jira/browse/KYLIN-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049826#comment-17049826
]
Zhong Yanghong commented on KYLIN-2971:
---
It seems this patch is overridden by other patches.
> Fix the wrong "Realization Names" in logQuery when hit cache
>
>
> Key: KYLIN-2971
> URL: https://issues.apache.org/jira/browse/KYLIN-2971
> Project: Kylin
> Issue Type: Bug
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.3.0
>
> Attachments: APACHE-KYLIN-2971.patch
>
>
> OLAPContext.clearThreadLocalContexts() should be called before a query
> starts. And we should consider query hit cache.
> Example,
> # query Q1 hit project P1 and cube C1;
> # then query Q2 hit project P2 and cube C2;
> # then Q1 comes again, it will show project P1 and {color:#f79232}cube
> C2{color}. However, it should be {color:#f79232}cube C1{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] lgtm-com[bot] commented on issue #1134: KYLIN-4396 Close FileReader in SaveDictStep
lgtm-com[bot] commented on issue #1134: KYLIN-4396 Close FileReader in SaveDictStep URL: https://github.com/apache/kylin/pull/1134#issuecomment-593496307 This pull request **fixes 1 alert** when merging 9f0029d7edfed1080947a38ba131daacae8eca50 into 8ce3deafa1498d4f39bfb9b36e84da7627065296 - [view on LGTM.com](https://lgtm.com/projects/g/apache/kylin/rev/pr-02fc6a1dcd2004138a417e21d46910e6544218e1) **fixed alerts:** * 1 for Potential input resource leak This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] codecov-io commented on issue #1134: KYLIN-4396 Close FileReader in SaveDictStep
codecov-io commented on issue #1134: KYLIN-4396 Close FileReader in SaveDictStep URL: https://github.com/apache/kylin/pull/1134#issuecomment-593489557 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1134?src=pr=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@7e89e95`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `0%`. [](https://codecov.io/gh/apache/kylin/pull/1134?src=pr=tree) ```diff @@Coverage Diff@@ ## master#1134 +/- ## = Coverage ? 25.06% Complexity? 6249 = Files ? 1446 Lines ?88273 Branches ?12351 = Hits ?22127 Misses?63976 Partials ? 2170 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1134?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...apache/kylin/engine/mr/streaming/SaveDictStep.java](https://codecov.io/gh/apache/kylin/pull/1134/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvc3RyZWFtaW5nL1NhdmVEaWN0U3RlcC5qYXZh) | `0% <0%> (ø)` | `0 <0> (?)` | | | [.../apache/kylin/engine/mr/streaming/DictsReader.java](https://codecov.io/gh/apache/kylin/pull/1134/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvc3RyZWFtaW5nL0RpY3RzUmVhZGVyLmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...n/engine/mr/streaming/ColumnarSplitDictReader.java](https://codecov.io/gh/apache/kylin/pull/1134/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvc3RyZWFtaW5nL0NvbHVtbmFyU3BsaXREaWN0UmVhZGVyLmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1134?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/1134?src=pr=footer). Last update [7e89e95...9f0029d](https://codecov.io/gh/apache/kylin/pull/1134?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4341) by-level cuboid intermediate files are left behind and not cleaned up after job is complete
[
https://issues.apache.org/jira/browse/KYLIN-4341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049369#comment-17049369
]
Vsevolod Ostapenko commented on KYLIN-4341:
---
@[~wangrupeng]
I have to respectfully disagree with this naive explanation of this not being a
bug.
I did not request segment merging in the cube configuration, therefore files
are expected to be removed.
Removing files manually is not a constructive proposition. Any manual
management of intermediate files is an operational burden. Plus, it's not
properly documented.
> by-level cuboid intermediate files are left behind and not cleaned up after
> job is complete
> ---
>
> Key: KYLIN-4341
> URL: https://issues.apache.org/jira/browse/KYLIN-4341
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
>Affects Versions: v2.6.4
> Environment: Kylin 2.6.4, CenOS 7.6, HDP 2.6.5
>Reporter: Vsevolod Ostapenko
>Assignee: wangrupeng
>Priority: Major
>
> Setup: MR as a cube build engine and by-level cube build strategy (auto
> picked).
> Upon completion of a cube segment build job a number of intermediate files
> are still left behind.
> Namely, output of the MR-jobs that produce the base cuboid, subsequent level
> cuboids, as well as rowkey_stats from the hfile creation step.
> The files in question consume about the same amount of space in HDFS as the
> final hfile.
> This lead to wasted space in HDFS that is not released for as long as the
> corresponding cube segment is online. The only point the leaked space is
> released, is when segment is offlined and cleaned up as part of the segment
> retention.
> Sample output is as follows.
> {quote}$ hadoop fs -ls -R
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/
> drwxr-xr-x - kylin hdfs 0 2020-01-07 04:44
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89
> drwxr-xr-x - kylin hdfs 0 2020-01-07 04:26
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid
> drwxr-xr-x - kylin hdfs 0 2020-01-07 04:36
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid
> -rw-r--r-- 2 kylin hdfs 0 2020-01-07 04:36
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/_SUCCESS
> -rw-r--r-- 2 kylin hdfs 51570048 2020-01-07 04:35
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-0
> -rw-r--r-- 2 kylin hdfs 51477377 2020-01-07 04:36
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-1
> -rw-r--r-- 2 kylin hdfs 51615162 2020-01-07 04:35
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-2
> -rw-r--r-- 2 kylin hdfs 51591031 2020-01-07 04:36
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-3
> -rw-r--r-- 2 kylin hdfs 51648914 2020-01-07 04:35
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-4
> -rw-r--r-- 2 kylin hdfs 51532761 2020-01-07 04:36
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-5
> -rw-r--r-- 2 kylin hdfs 51455652 2020-01-07 04:35
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-6
> -rw-r--r-- 2 kylin hdfs 51552752 2020-01-07 04:36
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_1_cuboid/part-r-7
> drwxr-xr-x - kylin hdfs 0 2020-01-07 04:25
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_base_cuboid
> -rw-r--r-- 2 kylin hdfs 0 2020-01-07 04:25
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_base_cuboid/_SUCCESS
> -rw-r--r-- 2 kylin hdfs 16293012 2020-01-07 04:25
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_base_cuboid/part-r-0
> -rw-r--r-- 2 kylin hdfs 16283730 2020-01-07 04:25
> /user/kylin/26x/test_kylin_26x_metadata/kylin-d2cff21e-9f59-6ac8-df34-656a3479433f/vl_aggs_cube_89/cuboid/level_base_cuboid/part-r-1
> -rw-r--r-- 2 kylin hdfs 16288965 2020-01-07 04:25
>
[jira] [Commented] (KYLIN-4394) There are several CVEs in the project dependencies
[ https://issues.apache.org/jira/browse/KYLIN-4394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049355#comment-17049355 ] XuCongying commented on KYLIN-4394: --- There are more details, and you can check it in "Attachments" (apache-kylin_CVE-report.md) I found that the buggy methods of the CVEs are in the program execution path of your project, which makes your project at risk. I have suggested some version updates. Here is the detailed information: * *Vulnerable Dependency:* org.apache.hadoop : hadoop-common : 2.7.1 * *Call Chain to Buggy Methods:* ** *Some files in your project call the library method org.apache.hadoop.conf.Configuration.getLong(java.lang.String,long), which can reach the buggy method of [CVE-2017-15713|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-15713].* *** Files in your project: storage-hbase/src/main/java/org/apache/kylin/storage/hbase/steps/HFileOutputFormat3.java *** One of the possible call chain: org.apache.hadoop.conf.Configuration.getLong(java.lang.String,long) org.apache.hadoop.conf.Configuration.getTrimmed(java.lang.String) org.apache.hadoop.conf.Configuration.get(java.lang.String) org.apache.hadoop.conf.Configuration.substituteVars(java.lang.String) [buggy method] ** Files in your project: storage-hbase/src/main/java/org/apache/kylin/storage/hbase/HBaseConnection.java *** One of the possible call chain: org.apache.hadoop.conf.Configuration.getTrimmedStringCollection(java.lang.String) org.apache.hadoop.conf.Configuration.get(java.lang.String) org.apache.hadoop.conf.Configuration.substituteVars(java.lang.String) [buggy method] ** *Some files in your project call the library method org.apache.hadoop.security.UserGroupInformation.getCurrentUser(), which can reach the buggy method of [CVE-2017-15713|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-15713].* *** Files in your project: storage-hbase/src/main/java/org/apache/kylin/storage/hbase/util/PingHBaseCLI.java *** One of the possible call chain: org.apache.hadoop.security.UserGroupInformation.getCurrentUser() org.apache.hadoop.security.UserGroupInformation.getLoginUser() org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(javax.security.auth.Subject) org.apache.hadoop.security.UserGroupInformation.ensureInitialized() org.apache.hadoop.security.UserGroupInformation.initialize(org.apache.hadoop.conf.Configuration,boolean) org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(org.apache.hadoop.conf.Configuration) org.apache.hadoop.conf.Configuration.get(java.lang.String,java.lang.String) org.apache.hadoop.conf.Configuration.substituteVars(java.lang.String) [buggy method] ** *Some files in your project call the library method org.apache.hadoop.conf.Configuration.get(java.lang.String,java.lang.String), which can reach the buggy method of [CVE-2017-15713|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-15713].* *** Files in your project: storage-hbase/src/main/java/org/apache/kylin/storage/hbase/steps/HFileOutputFormat3.java, engine-mr/src/main/java/org/apache/kylin/engine/mr/common/AbstractHadoopJob.java, storage-hbase/src/main/java/org/apache/kylin/storage/hbase/HBaseConnection.java *** One of the possible call chain: org.apache.hadoop.conf.Configuration.get(java.lang.String,java.lang.String) org.apache.hadoop.conf.Configuration.substituteVars(java.lang.String) [buggy method] ** *Some files in your project call the library method org.apache.hadoop.io.SequenceFile.Reader.next(org.apache.hadoop.io.Writable,org.apache.hadoop.io.Writable), which can reach the buggy method of [CVE-2017-15713|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-15713].* *** Files in your project: engine-mr/src/main/java/org/apache/kylin/engine/mr/common/CubeStatsReader.java, core-common/src/main/java/org/apache/kylin/common/util/HadoopUtil.java *** One of the possible call chain: org.apache.hadoop.io.SequenceFile.Reader.next(org.apache.hadoop.io.Writable,org.apache.hadoop.io.Writable) [buggy method] ** *Some files in your project call the library method org.apache.hadoop.util.ToolRunner.run(org.apache.hadoop.util.Tool,java.lang.String[]), which can reach the buggy method of [CVE-2017-15713|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-15713].* *** Files in your project: engine-mr/src/main/java/org/apache/kylin/engine/mr/common/AbstractHadoopJob.java *** One of the possible call chain: org.apache.hadoop.util.ToolRunner.run(org.apache.hadoop.util.Tool,java.lang.String[]) org.apache.hadoop.util.ToolRunner.run(org.apache.hadoop.conf.Configuration,org.apache.hadoop.util.Tool,java.lang.String[]) org.apache.hadoop.util.GenericOptionsParser.(org.apache.hadoop.conf.Configuration,java.lang.String[]) org.apache.hadoop.util.GenericOptionsParser.(org.apache.hadoop.conf.Configuration,org.apache.commons.cli.Options,java.lang.String[])
[jira] [Updated] (KYLIN-4394) There are several CVEs in the project dependencies
[ https://issues.apache.org/jira/browse/KYLIN-4394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XuCongying updated KYLIN-4394: -- Attachment: apache-kylin_CVE-report.md > There are several CVEs in the project dependencies > -- > > Key: KYLIN-4394 > URL: https://issues.apache.org/jira/browse/KYLIN-4394 > Project: Kylin > Issue Type: Bug > Components: Security >Reporter: XuCongying >Assignee: Yaqian Zhang >Priority: Major > Attachments: apache-kylin_CVE-report.md > > > I noticed some of your libraries contained CVEs. I suggest updating their > versions to increase the security of your project. The following is a > detailed content. > * *Vulnerable Library Version:* org.scala-lang : scala-compiler : 2.11.0 > *CVE ID:* > [CVE-2017-15288|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-15288] > *Import Path:* engine-flink/pom.xml, kylin-it/pom.xml, engine-spark/pom.xml > *Suggested Safe Versions:* 2.11.12, 2.12.10, 2.12.4, 2.12.5, 2.12.6, 2.12.7, > 2.12.8, 2.12.9, 2.13.0, 2.13.0-M1, 2.13.0-M2, 2.13.0-M3, 2.13.0-M3-f73b161, > 2.13.0-M4, 2.13.0-M4-pre-20d3c21, 2.13.0-M5, 2.13.0-M5-1775dba, > 2.13.0-M5-5eef812, 2.13.0-M5-6e0cba7, 2.13.0-RC1, 2.13.0-RC2, 2.13.0-RC3, > 2.13.1 > * *Vulnerable Library Version:* org.apache.tomcat : tomcat-catalina : 7.0.91 > *CVE ID:* > [CVE-2016-8735|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-8735], > [CVE-2019-0232|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-0232], > [CVE-2016-6794|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6794], > [CVE-2016-6816|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6816], > [CVE-2016-8745|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-8745], > [CVE-2019-17563|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17563] > *Import Path:* tomcat-ext/pom.xml, server/pom.xml, server-base/pom.xml > *Suggested Safe Versions:* 10.0.0-M1, 7.0.100, 9.0.30, 9.0.31 > * *Vulnerable Library Version:* com.h2database : h2 : 1.4.196 *CVE ID:* > [CVE-2018-10054|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-10054], > > [CVE-2018-14335|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-14335] > *Import Path:* server/pom.xml, kylin-it/pom.xml, source-jdbc/pom.xml, > source-hive/pom.xml, datasource-sdk/pom.xml *Suggested Safe Versions:* > 1.4.198, 1.4.199, 1.4.200 > * *Vulnerable Library Version:* com.google.guava : guava : 14.0 *CVE ID:* > [CVE-2018-10237|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-10237] > *Import Path:* core-storage/pom.xml, stream-receiver/pom.xml, > server/pom.xml, core-cube/pom.xml, core-metadata/pom.xml, jdbc/pom.xml, > tool-assembly/pom.xml, core-metrics/pom.xml *Suggested Safe Versions:* > 24.1.1-android, 24.1.1-jre, 25.0-android, 25.0-jre, 25.1-android, 25.1-jre, > 26.0-android, 26.0-jre, 27.0-android, 27.0-jre, 27.0.1-android, 27.0.1-jre, > 27.1-android, 27.1-jre, 28.0-android, 28.0-jre, 28.1-android, 28.1-jre, > 28.2-android, 28.2-jre > * *Vulnerable Library Version:* org.apache.hive.hcatalog : > hive-hcatalog-core : 1.2.1 *CVE ID:* > [CVE-2015-7521|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-7521] > *Import Path:* metrics-reporter-hive/pom.xml, assembly/pom.xml, > server/pom.xml, kylin-it/pom.xml, source-jdbc/pom.xml, source-hive/pom.xml, > server-base/pom.xml *Suggested Safe Versions:* 1.2.2, 2.0.0, 2.0.1, 2.1.0, > 2.1.1, 2.2.0, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.3.6, 3.0.0, 3.1.0, > 3.1.1, 3.1.2 > * *Vulnerable Library Version:* org.apache.spark : spark-core_2.11 : 2.3.2 > *CVE ID:* > [CVE-2017-7678|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-7678], > [CVE-2018-3826|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-3826], > [CVE-2018-11770|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-11770], > > [CVE-2019-10099|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-10099] > *Import Path:* server/pom.xml, kylin-it/pom.xml, engine-spark/pom.xml, > storage-hbase/pom.xml *Suggested Safe Versions:* 2.4.5 > * *Vulnerable Library Version:* org.apache.kafka : kafka_2.11 : 1.0.0 *CVE > ID:* > [CVE-2018-1288|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1288], > [CVE-2019-17196|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17196] > *Import Path:* assembly/pom.xml, source-kafka/pom.xml, kylin-it/pom.xml, > stream-source-kafka/pom.xml, metrics-reporter-kafka/pom.xml *Suggested Safe > Versions:* 2.1.1, 2.2.0, 2.2.1, 2.2.2, 2.3.0, 2.3.1, 2.4.0 > * *Vulnerable Library Version:* org.apache.hive : hive-jdbc : 1.2.1 *CVE > ID:* > [CVE-2016-3083|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-3083], > [CVE-2015-7521|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-7521], >
[jira] [Updated] (KYLIN-4394) There are several CVEs in the project dependencies
[ https://issues.apache.org/jira/browse/KYLIN-4394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XuCongying updated KYLIN-4394: -- Description: I noticed some of your libraries contained CVEs. I suggest updating their versions to increase the security of your project. The following is a detailed content. * *Vulnerable Library Version:* org.scala-lang : scala-compiler : 2.11.0 *CVE ID:* [CVE-2017-15288|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-15288] *Import Path:* engine-flink/pom.xml, kylin-it/pom.xml, engine-spark/pom.xml *Suggested Safe Versions:* 2.11.12, 2.12.10, 2.12.4, 2.12.5, 2.12.6, 2.12.7, 2.12.8, 2.12.9, 2.13.0, 2.13.0-M1, 2.13.0-M2, 2.13.0-M3, 2.13.0-M3-f73b161, 2.13.0-M4, 2.13.0-M4-pre-20d3c21, 2.13.0-M5, 2.13.0-M5-1775dba, 2.13.0-M5-5eef812, 2.13.0-M5-6e0cba7, 2.13.0-RC1, 2.13.0-RC2, 2.13.0-RC3, 2.13.1 * *Vulnerable Library Version:* org.apache.tomcat : tomcat-catalina : 7.0.91 *CVE ID:* [CVE-2016-8735|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-8735], [CVE-2019-0232|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-0232], [CVE-2016-6794|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6794], [CVE-2016-6816|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6816], [CVE-2016-8745|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-8745], [CVE-2019-17563|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17563] *Import Path:* tomcat-ext/pom.xml, server/pom.xml, server-base/pom.xml *Suggested Safe Versions:* 10.0.0-M1, 7.0.100, 9.0.30, 9.0.31 * *Vulnerable Library Version:* com.h2database : h2 : 1.4.196 *CVE ID:* [CVE-2018-10054|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-10054], [CVE-2018-14335|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-14335] *Import Path:* server/pom.xml, kylin-it/pom.xml, source-jdbc/pom.xml, source-hive/pom.xml, datasource-sdk/pom.xml *Suggested Safe Versions:* 1.4.198, 1.4.199, 1.4.200 * *Vulnerable Library Version:* com.google.guava : guava : 14.0 *CVE ID:* [CVE-2018-10237|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-10237] *Import Path:* core-storage/pom.xml, stream-receiver/pom.xml, server/pom.xml, core-cube/pom.xml, core-metadata/pom.xml, jdbc/pom.xml, tool-assembly/pom.xml, core-metrics/pom.xml *Suggested Safe Versions:* 24.1.1-android, 24.1.1-jre, 25.0-android, 25.0-jre, 25.1-android, 25.1-jre, 26.0-android, 26.0-jre, 27.0-android, 27.0-jre, 27.0.1-android, 27.0.1-jre, 27.1-android, 27.1-jre, 28.0-android, 28.0-jre, 28.1-android, 28.1-jre, 28.2-android, 28.2-jre * *Vulnerable Library Version:* org.apache.hive.hcatalog : hive-hcatalog-core : 1.2.1 *CVE ID:* [CVE-2015-7521|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-7521] *Import Path:* metrics-reporter-hive/pom.xml, assembly/pom.xml, server/pom.xml, kylin-it/pom.xml, source-jdbc/pom.xml, source-hive/pom.xml, server-base/pom.xml *Suggested Safe Versions:* 1.2.2, 2.0.0, 2.0.1, 2.1.0, 2.1.1, 2.2.0, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.3.6, 3.0.0, 3.1.0, 3.1.1, 3.1.2 * *Vulnerable Library Version:* org.apache.spark : spark-core_2.11 : 2.3.2 *CVE ID:* [CVE-2017-7678|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-7678], [CVE-2018-3826|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-3826], [CVE-2018-11770|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-11770], [CVE-2019-10099|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-10099] *Import Path:* server/pom.xml, kylin-it/pom.xml, engine-spark/pom.xml, storage-hbase/pom.xml *Suggested Safe Versions:* 2.4.5 * *Vulnerable Library Version:* org.apache.kafka : kafka_2.11 : 1.0.0 *CVE ID:* [CVE-2018-1288|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1288], [CVE-2019-17196|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17196] *Import Path:* assembly/pom.xml, source-kafka/pom.xml, kylin-it/pom.xml, stream-source-kafka/pom.xml, metrics-reporter-kafka/pom.xml *Suggested Safe Versions:* 2.1.1, 2.2.0, 2.2.1, 2.2.2, 2.3.0, 2.3.1, 2.4.0 * *Vulnerable Library Version:* org.apache.hive : hive-jdbc : 1.2.1 *CVE ID:* [CVE-2016-3083|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-3083], [CVE-2015-7521|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-7521], [CVE-2018-1282|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1282] *Import Path:* server/pom.xml, source-jdbc/pom.xml, source-hive/pom.xml *Suggested Safe Versions:* 2.3.3, 2.3.4, 2.3.5, 2.3.6, 3.0.0, 3.1.0, 3.1.1, 3.1.2 * *Vulnerable Library Version:* org.apache.hadoop : hadoop-hdfs : 2.7.1 *CVE ID:* [CVE-2016-5001|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-5001], [CVE-2018-11768|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-11768] *Import Path:* metrics-reporter-hive/pom.xml, assembly/pom.xml, stream-core/pom.xml, stream-receiver/pom.xml, server/pom.xml, kylin-it/pom.xml, engine-mr/pom.xml,
[GitHub] [kylin] hit-lacus opened a new pull request #1134: KYLIN-4396 Close FileReader in SaveDictStep
hit-lacus opened a new pull request #1134: KYLIN-4396 Close FileReader in SaveDictStep URL: https://github.com/apache/kylin/pull/1134 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4396) File Descriptor Leakage in MR Build Engine
[
https://issues.apache.org/jira/browse/KYLIN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049343#comment-17049343
]
ASF GitHub Bot commented on KYLIN-4396:
---
hit-lacus commented on pull request #1134: KYLIN-4396 Close FileReader in
SaveDictStep
URL: https://github.com/apache/kylin/pull/1134
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> File Descriptor Leakage in MR Build Engine
> --
>
> Key: KYLIN-4396
> URL: https://issues.apache.org/jira/browse/KYLIN-4396
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v3.0.0, v3.0.1
>Reporter: Xiaoxiang Yu
>Assignee: Xiaoxiang Yu
>Priority: Major
> Fix For: v3.1.0
>
>
> After runing for one or two days, Job Server failed to response to any http
> requests, but the process looks good, without high CPU pressure etc. In
> _kylin.out_, we find error message indicate that process has hit the limit of
> *FD.*
> After checking detail, we find many fds are not released after a long period,
> most of them are socket in state, that is *CLOSE_WAIT*.
> !https://user-images.githubusercontent.com/14030549/72678560-0fc7e580-3ae2-11ea-85ea-5dcb40e97bd7.png!
>
> !https://user-images.githubusercontent.com/14030549/72678638-c9bf5180-3ae2-11ea-880c-7f8e34d17a8c.png!
>
>
> {code:java}
> 一月 19, 2020 5:13:51 下午 org.apache.catalina.core.StandardWrapperValve invoke
> 严重: Servlet.service() for servlet [default] in context with path [/kylin]
> threw exception java.io.FileNotFoundException:
> /root/xiaoxiang.yu/deploy/kylin-all/tomcat/webapps/kylin/js/scripts.min.20191209061232.js
> (打开的文件过多) at java.io.FileInputStream.open0(Native Method) at
> java.io.FileInputStream.open(FileInputStream.java:195) at
> java.io.FileInputStream.(FileInputStream.java:138) at
> org.apache.naming.resources.FileDirContext$FileResource.streamContent(FileDirContext.java:1008)
> at
> org.apache.catalina.servlets.DefaultServlet.copy(DefaultServlet.java:2084) at
> org.apache.catalina.servlets.DefaultServlet.serveResource(DefaultServlet.java:1023)
> at
> org.apache.catalina.servlets.DefaultServlet.doGet(DefaultServlet.java:453) at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:624) at
> org.apache.catalina.servlets.DefaultServlet.service(DefaultServlet.java:433)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
> at
>
[jira] [Assigned] (KYLIN-4396) File Descriptor Leakage in MR Build Engine
[
https://issues.apache.org/jira/browse/KYLIN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoxiang Yu reassigned KYLIN-4396:
---
Assignee: Xiaoxiang Yu
> File Descriptor Leakage in MR Build Engine
> --
>
> Key: KYLIN-4396
> URL: https://issues.apache.org/jira/browse/KYLIN-4396
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v3.0.0, v3.0.1
>Reporter: Xiaoxiang Yu
>Assignee: Xiaoxiang Yu
>Priority: Major
> Fix For: v3.1.0
>
>
> After runing for one or two days, Job Server failed to response to any http
> requests, but the process looks good, without high CPU pressure etc. In
> _kylin.out_, we find error message indicate that process has hit the limit of
> *FD.*
> After checking detail, we find many fds are not released after a long period,
> most of them are socket in state, that is *CLOSE_WAIT*.
> !https://user-images.githubusercontent.com/14030549/72678560-0fc7e580-3ae2-11ea-85ea-5dcb40e97bd7.png!
>
> !https://user-images.githubusercontent.com/14030549/72678638-c9bf5180-3ae2-11ea-880c-7f8e34d17a8c.png!
>
>
> {code:java}
> 一月 19, 2020 5:13:51 下午 org.apache.catalina.core.StandardWrapperValve invoke
> 严重: Servlet.service() for servlet [default] in context with path [/kylin]
> threw exception java.io.FileNotFoundException:
> /root/xiaoxiang.yu/deploy/kylin-all/tomcat/webapps/kylin/js/scripts.min.20191209061232.js
> (打开的文件过多) at java.io.FileInputStream.open0(Native Method) at
> java.io.FileInputStream.open(FileInputStream.java:195) at
> java.io.FileInputStream.(FileInputStream.java:138) at
> org.apache.naming.resources.FileDirContext$FileResource.streamContent(FileDirContext.java:1008)
> at
> org.apache.catalina.servlets.DefaultServlet.copy(DefaultServlet.java:2084) at
> org.apache.catalina.servlets.DefaultServlet.serveResource(DefaultServlet.java:1023)
> at
> org.apache.catalina.servlets.DefaultServlet.doGet(DefaultServlet.java:453) at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:624) at
> org.apache.catalina.servlets.DefaultServlet.service(DefaultServlet.java:433)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
>
[jira] [Commented] (KYLIN-4392) Bump Flink version from 1.9.2 to 1.10.0
[ https://issues.apache.org/jira/browse/KYLIN-4392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049329#comment-17049329 ] vinoyang commented on KYLIN-4392: - [~shaofengshi] It seems this issue does not duplicate with KYLIN-4391. This issue tries to bump flink version from 1.9.2 to 1.10.0. However, KYLIN-4391 tries to bump flink version from 1.9.0 to 1.9.2. > Bump Flink version from 1.9.2 to 1.10.0 > --- > > Key: KYLIN-4392 > URL: https://issues.apache.org/jira/browse/KYLIN-4392 > Project: Kylin > Issue Type: Sub-task > Components: Flink Engine >Reporter: vinoyang >Assignee: vinoyang >Priority: Major > > Recently, Flink has release v1.10.0. It's a major version that contains some > great features and big changes. We can tentatively upgrade to v1.10.0 after > we upgrade to v1.9.2 firstly. If we find some fatal issues, we can fall back > to v1.9.2. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] coveralls commented on issue #1131: KYLIN-4387 Flink cubing merge step failed
coveralls commented on issue #1131: KYLIN-4387 Flink cubing merge step failed URL: https://github.com/apache/kylin/pull/1131#issuecomment-593457434 ## Pull Request Test Coverage Report for [Build 5682](https://coveralls.io/builds/29075025) * **0** of **30** **(0.0%)** changed or added relevant lines in **1** file are covered. * No unchanged relevant lines lost coverage. * Overall coverage decreased (**-0.002%**) to **27.523%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [engine-flink/src/main/java/org/apache/kylin/engine/flink/FlinkCubingMerge.java](https://coveralls.io/builds/29075025/source?filename=engine-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fflink%2FFlinkCubingMerge.java#L153) | 0 | 30 | 0.0% | Totals | [](https://coveralls.io/builds/29075025) | | :-- | --: | | Change from base [Build 5678](https://coveralls.io/builds/29064652): | -0.002% | | Covered Lines: | 24298 | | Relevant Lines: | 88284 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Closed] (KYLIN-2787) SUM(CASE WHEN...) does not supported
[
https://issues.apache.org/jira/browse/KYLIN-2787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shao Feng Shi closed KYLIN-2787.
Resolution: Fixed
> SUM(CASE WHEN...) does not supported
>
>
> Key: KYLIN-2787
> URL: https://issues.apache.org/jira/browse/KYLIN-2787
> Project: Kylin
> Issue Type: Bug
> Components: REST Service
>Affects Versions: v1.6.0
> Environment: jdk1.7.0_80, hbase-0.98.6-cdh5.3.9,
> apache-kylin-1.6.0-bin,
>Reporter: Shendandan
>Assignee: Jason Zhong
>Priority: Major
>
> Use a calculated field in tableau, but it can not work. There is an error in
> kylin.log:
> {{[Kylin][ODBC 1.0(w) Driver]The REST query request failed, the error
> message is: Error while executing SQL "SELECT SUM((CASE WHEN (( EXTRACT(YEAR
> FROM a.column1) = 2017) AND ( EXTRACT(MONTH FROM a.column1) = 6)) THEN
> a.column2 ELSE 0 END)) AS "sum_Calculation_FIIAIBABGCIFEFEJ_ok" FROM table a
> LEFT JOIN table b ON (a. column3= b.column1) LEFT JOIN table c ON (a.
> column4= c.column1) LEFT JOIN table d ON (a.column5 = d.column1) LEFT JOIN
> table e ON (a.column6 = e.column1) ": Can't find any realization. Please
> confirm with providers. SQL digest: fact table
> a ,group by [],filter on [],with aggregates[FunctionDesc [expression=SUM,
> parameter=null, returnType=null]].}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (KYLIN-2821) Fix bug in GTScanRequest to init columns in right order
[ https://issues.apache.org/jira/browse/KYLIN-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-2821. Resolution: Incomplete > Fix bug in GTScanRequest to init columns in right order > --- > > Key: KYLIN-2821 > URL: https://issues.apache.org/jira/browse/KYLIN-2821 > Project: Kylin > Issue Type: Bug >Reporter: Xiaqing Wang >Assignee: Xiaqing Wang >Priority: Major > > In initiation of GTScanRequest, we should validate filter columns before > computing selected column blocks according to columns. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-2817) The problem with Kerberos when building Cube in the kylin read and write separation environment
[ https://issues.apache.org/jira/browse/KYLIN-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-2817. Resolution: Incomplete > The problem with Kerberos when building Cube in the kylin read and write > separation environment > --- > > Key: KYLIN-2817 > URL: https://issues.apache.org/jira/browse/KYLIN-2817 > Project: Kylin > Issue Type: Bug > Components: Environment >Affects Versions: v2.0.0 > Environment: CDH：5.7.1 > Kylin：apache-kylin-2.0.0-bin >Reporter: jiangshouzhuang >Assignee: Hongbin Ma >Priority: Major > Fix For: Future > > > In building kylin read and write separation environment, I encountered some > Kerberos permissions problems. > The Kylin's environment is as follows: > CDH 5.7.1 Cluster A: Yarn,HDFS,Hive. HDFS is HA, Nameservice is nn-idc. > CDH 5.7.1 Cluster B: Yarn,HDFS,HBase,Kylin，HDFS is HA, Nameservice is kylin-ns > Cluster A and Cluster B use a common KDC. > When I use kylin to build the cube, the error as following: > #19 Step Name: Convert Cuboid Data to HFile > java.io.IOException: Failed to run job : Failed to renew token: Kind: > HDFS_DELEGATION_TOKEN, Service: ha-hdfs:kylin-ns, Ident: > (HDFS_DELEGATION_TOKEN token 12 for kylin_manager_user) > at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:301) > at > org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:244) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) > at org.apache.hadoop.mapreduce.Job.submit(Job.java:1304) > at > org.apache.kylin.engine.mr.common.AbstractHadoopJob.waitForCompletion(AbstractHadoopJob.java:149) > at > org.apache.kylin.storage.hbase.steps.CubeHFileJob.run(CubeHFileJob.java:106) > at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:102) > at > org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:123) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:124) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:64) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:124) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:142) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > The problem is similar to that on Yarn's issues: > https://issues.apache.org/jira/browse/YARN-3021 > So I think that Apache Kylin supports the environment of reading and writing > separation. How can I solve this problem and configure it? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-2779) Segment size estimation is wrong for bitmap measure
[ https://issues.apache.org/jira/browse/KYLIN-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-2779. Resolution: Fixed The estimation was improved a lot in Kylin 2.5. Please try to upgrade to a newer version. > Segment size estimation is wrong for bitmap measure > --- > > Key: KYLIN-2779 > URL: https://issues.apache.org/jira/browse/KYLIN-2779 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.0.0 >Reporter: Alexander Sterligov >Assignee: Alexander Sterligov >Priority: Major > Labels: scope > > If bitmap is used for distinct count, then size estimation is wrong for about > 4 times. > The problem is fixed by kylin.cube.size-estimate-countdistinct-ratio=0.2. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-2829) Union two subquery under join make SQL return null values
[
https://issues.apache.org/jira/browse/KYLIN-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shao Feng Shi closed KYLIN-2829.
Resolution: Incomplete
> Union two subquery under join make SQL return null values
> -
>
> Key: KYLIN-2829
> URL: https://issues.apache.org/jira/browse/KYLIN-2829
> Project: Kylin
> Issue Type: Bug
>Reporter: Roger Shi
>Priority: Major
>
> SQL like:
> {code:sql}
> select sum(TEST_A.PRICE) as ITEM_CNT
> FROM TEST_KYLIN_FACT as TEST_A
> join (
> select * from ITEM_CNT where CAL_DT < DATE '2012-06-01'
> union
> select * from ITEM_CNT where CAL_DT > DATE '2013-06-01'
> ) TEST_B
> on TEST_A.TRANS_ID = TEST_B.TRANS_ID
> group by TEST_A.SELLER_ID
> {code}
> returns null values.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (KYLIN-1446) Visualization in Kylin Insight page doesn't work for line chart
[ https://issues.apache.org/jira/browse/KYLIN-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-1446. Resolution: Won't Fix > Visualization in Kylin Insight page doesn't work for line chart > --- > > Key: KYLIN-1446 > URL: https://issues.apache.org/jira/browse/KYLIN-1446 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v1.4.0, v1.5.0 > Environment: PRODUCTION >Reporter: Hao Chen >Assignee: Jason Zhong >Priority: Major > > Visualization with Line chart in Kylin Insight page doesn't work. > When submitting a query to kylin on insight page, and select chart type as > line chart, the option selection controller doesn't allow to choose dimension > while metric selection works. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-1000) i18n and localization support
[ https://issues.apache.org/jira/browse/KYLIN-1000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-1000. Resolution: Won't Fix > i18n and localization support > - > > Key: KYLIN-1000 > URL: https://issues.apache.org/jira/browse/KYLIN-1000 > Project: Kylin > Issue Type: Wish > Components: Web >Affects Versions: v1.0, v1.4.0 >Reporter: Luke Han >Assignee: yongjie zhao >Priority: Major > > There are many use cases of Apache Kylin are requiring to support other > language rather than English, like Chinese, Japanese, EU side and others. > Open this JIRA to tracking internationalization and localization of Kylin. > The first phase is to support multi-language of web UI, like Chinese. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-1131) Enabling saving incomplete cube
[ https://issues.apache.org/jira/browse/KYLIN-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-1131. Resolution: Won't Fix > Enabling saving incomplete cube > --- > > Key: KYLIN-1131 > URL: https://issues.apache.org/jira/browse/KYLIN-1131 > Project: Kylin > Issue Type: Wish > Components: Web >Reporter: Haowei Ji >Assignee: Jason Zhong >Priority: Major > > When adding cubes, the cube could only be saved when it comes to the last > step, and if user switch to other functions and back, it will disappear, > seems the web does not caching what user has just done, it's annoying to > start up again, especially when some cubes are complicated. So suggest to > caching the cube and adding save button in the process of creating the cube. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-299) Easy config switch according to different deploy env
[ https://issues.apache.org/jira/browse/KYLIN-299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-299. --- Resolution: Won't Fix > Easy config switch according to different deploy env > > > Key: KYLIN-299 > URL: https://issues.apache.org/jira/browse/KYLIN-299 > Project: Kylin > Issue Type: Wish > Components: Tools, Build and Test >Reporter: Luke Han >Assignee: Dong Li >Priority: Major > Labels: github-import > > Easy config switch according to different deploy env (dev,qa,prod) > Imported from GitHub > Url: https://github.com/KylinOLAP/Kylin/issues/206 > Created by: [janzhongi|https://github.com/janzhongi] > Labels: newfeature, > Assignee: [janzhongi|https://github.com/janzhongi] > Created at: Wed Dec 10 10:46:23 CST 2014 > State: open -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3045) Too many open files raised from new Configuration()
[ https://issues.apache.org/jira/browse/KYLIN-3045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-3045. Resolution: Cannot Reproduce > Too many open files raised from new Configuration() > --- > > Key: KYLIN-3045 > URL: https://issues.apache.org/jira/browse/KYLIN-3045 > Project: Kylin > Issue Type: Bug >Reporter: liyang >Assignee: liyang >Priority: Major > Labels: scope > > Try to refactor MapReduceExecutable to avoid re-creating Configuration too > many times. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3002) Use Spark as default engine in web
[ https://issues.apache.org/jira/browse/KYLIN-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-3002. Resolution: Won't Fix > Use Spark as default engine in web > -- > > Key: KYLIN-3002 > URL: https://issues.apache.org/jira/browse/KYLIN-3002 > Project: Kylin > Issue Type: Improvement > Components: Spark Engine, Web >Reporter: Kaisen Kang >Assignee: Kaisen Kang >Priority: Major > > After KYLIN-2997, like KYLIN-2963, we could use Spark as default engine for > none-global-dict cube. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3304) Unstable UT CacheServiceTest
[ https://issues.apache.org/jira/browse/KYLIN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-3304. Resolution: Fixed > Unstable UT CacheServiceTest > > > Key: KYLIN-3304 > URL: https://issues.apache.org/jira/browse/KYLIN-3304 > Project: Kylin > Issue Type: Test > Components: Tools, Build and Test >Reporter: Shao Feng Shi >Priority: Major > > Kylin integration test is unstable, sometimes failed at the CacheServiceTest: > > *00:48:11* Failed tests: *00:48:11* > CacheServiceTest.testCubeCRUD:237*00:48:11* Tests in error: *00:48:11* > CacheServiceTest.testMetaCRUD:316->waitForCounterAndClear:156 Runtime > timeout*00:48:11* *00:48:11* Tests run: 93, Failures: 1, Errors: 1, Skipped: 1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-2408) GTRecord work on column block rather than individual columns
[ https://issues.apache.org/jira/browse/KYLIN-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-2408. Resolution: Incomplete > GTRecord work on column block rather than individual columns > > > Key: KYLIN-2408 > URL: https://issues.apache.org/jira/browse/KYLIN-2408 > Project: Kylin > Issue Type: Improvement >Reporter: liyang >Assignee: Jiatao Tao >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3341) SQL queries take up too much memory
[ https://issues.apache.org/jira/browse/KYLIN-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-3341. Resolution: Incomplete > SQL queries take up too much memory > --- > > Key: KYLIN-3341 > URL: https://issues.apache.org/jira/browse/KYLIN-3341 > Project: Kylin > Issue Type: Improvement > Components: Storage - HBase >Affects Versions: v2.0.0 >Reporter: RenZhiMin >Priority: Major > Attachments: log.txt > > > Some SQL will take up too much memory in the application, the service is not > stable, after analysis because the SQL query need to scan the relevant > segemnt corresponding hbase table, is through the thread asynchronous request > related to SQL cuboid corresponding region. After each segment makes a > request, wait for the data to process the hbase callbacks. > But sometimes SQL involves more segments, and before all the segments make > requests, there is a large amount of data returned, leading to memory alarm > and service instability. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3350) Build cube incrementally from a file list
[ https://issues.apache.org/jira/browse/KYLIN-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-3350. Resolution: Won't Fix > Build cube incrementally from a file list > - > > Key: KYLIN-3350 > URL: https://issues.apache.org/jira/browse/KYLIN-3350 > Project: Kylin > Issue Type: New Feature > Components: Job Engine >Reporter: Shao Feng Shi >Assignee: Shao Feng Shi >Priority: Major > > This a new requirement. > In some cases, upstream components dump the new data into files, and then > expect Kylin can consume data from a given file list, instead of from Hive. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3432) Merge segments from HBase snapshots
[ https://issues.apache.org/jira/browse/KYLIN-3432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-3432. Resolution: Won't Fix > Merge segments from HBase snapshots > --- > > Key: KYLIN-3432 > URL: https://issues.apache.org/jira/browse/KYLIN-3432 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Shao Feng Shi >Priority: Major > > HBase introduces the MultiTableSnapshotInputFormat since 1.2 ; Kylin can > merge segments with this method, so that the cuboid files can be deleted. > > # HBASE-13356 > # Sample code: > https://github.com/apache/hbase/blob/master/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapred/MultiTableSnapshotInputFormat.java -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3405) kylin mandatory dimension
[ https://issues.apache.org/jira/browse/KYLIN-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-3405. Resolution: Not A Problem > kylin mandatory dimension > - > > Key: KYLIN-3405 > URL: https://issues.apache.org/jira/browse/KYLIN-3405 > Project: Kylin > Issue Type: Bug >Reporter: Lemont >Priority: Major > > I created a simple cube on kylin 2.3.1，the table has three > dimensions:pt,biz_time and periods_time ，one measure > :online_rider > Something trouble me is that when i set biz_time to the mandatory > dimension,the sql will execute fail. > The sql is : > select floor((1524931200 - biz_time)/(30*86400)), count(DISTINCT > online_rider) onlineRiders > from rider_view where pt = '20180530' and periods_time in(0) and biz_time > >1524931200 > group by floor((1524931200 - biz_time)*1.0/(30*86400)) . > The error is : > null while executing sql. > But when I remove the condition of periods_time in(0) or floor function the > sql will execute success. > I don't know what's the relationship between the two. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3532) Inconsistent results of the query
[ https://issues.apache.org/jira/browse/KYLIN-3532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-3532. Resolution: Incomplete > Inconsistent results of the query > - > > Key: KYLIN-3532 > URL: https://issues.apache.org/jira/browse/KYLIN-3532 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Reporter: Na Zhai >Priority: Major > > When I query with sql "select count(*) from KYLIN_CATEGORY_GROUPINGS where > KYLIN_CATEGORY_GROUPINGS.KYLIN_GROUPINGS_CRE_DATE > DATE '2012-09-02'". The > query result shows 0 which is correct. When I query with sql "select count(*) > from KYLIN_CATEGORY_GROUPINGS where > KYLIN_CATEGORY_GROUPINGS.KYLIN_GROUPINGS_CRE_DATE < DATE '2012-09-02'". The > query result shows 0 which is wrong. It should be 14. When I query with sql > "select count(*) from KYLIN_CATEGORY_GROUPINGS where > KYLIN_CATEGORY_GROUPINGS.KYLIN_GROUPINGS_CRE_DATE > DATE '2010-09-02'". The > query result shows 14 which is correct. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-2669) Cube is ready but insight tables not result
[ https://issues.apache.org/jira/browse/KYLIN-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-2669. Resolution: Cannot Reproduce > Cube is ready but insight tables not result > --- > > Key: KYLIN-2669 > URL: https://issues.apache.org/jira/browse/KYLIN-2669 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v1.6.0 >Reporter: YUNFEI CHEN >Assignee: liyang >Priority: Major > Attachments: logs.tar.gz > > > As reported by Cheney( cyf_2...@sina.com) > Hi All, > I use kylin v1.6.0. add Cube and bulid successful.Cube status is Ready. but > in Insight tabs, tables display no result.It happened sometime,not all the > time. > has anyone encountered a similar situation, how to resovle it? > Best Regards > Cheney Chen -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (KYLIN-3274) Please delete old releases from mirroring system
[ https://issues.apache.org/jira/browse/KYLIN-3274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi reassigned KYLIN-3274: Assignee: Chunen Ni (was: Shao Feng Shi) Chunen, I see there are a couple of old packages, could you please drop the old versions? > Please delete old releases from mirroring system > > > Key: KYLIN-3274 > URL: https://issues.apache.org/jira/browse/KYLIN-3274 > Project: Kylin > Issue Type: Bug > Environment: https://dist.apache.org/repos/dist/release/kylin/ >Reporter: Sebb >Assignee: Chunen Ni >Priority: Major > > To reduce the load on the ASF mirrors, projects are required to delete old > releases [1] > Please can you remove all non-current releases? > [From the download page, it looks like all but the latest release (2.3.0) are > superseded.] > It's unfair to expect the 3rd party mirrors to carry old releases. > Note that older releases can still be linked from the download page, but such > links should use the archive server at: > https://archive.apache.org/dist/incubator/mxnet/ > Thanks! > [1] http://www.apache.org/dev/release.html#when-to-archive -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3563) Web UI style upgrade
[ https://issues.apache.org/jira/browse/KYLIN-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-3563. Resolution: Won't Fix > Web UI style upgrade > > > Key: KYLIN-3563 > URL: https://issues.apache.org/jira/browse/KYLIN-3563 > Project: Kylin > Issue Type: Improvement > Components: Web >Reporter: Shao Feng Shi >Assignee: Roger >Priority: Major > > Kylin's web GUI style almost has no change since the first day. We can > introduce some change to make it fresh. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3591) 每次执行到"Convert Cuboid Data to HFile"这里时都会失败
[ https://issues.apache.org/jira/browse/KYLIN-3591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-3591. Resolution: Incomplete > 每次执行到"Convert Cuboid Data to HFile"这里时都会失败 > -- > > Key: KYLIN-3591 > URL: https://issues.apache.org/jira/browse/KYLIN-3591 > Project: Kylin > Issue Type: Bug > Components: Job Engine > Environment: Hadoop 2.7.2 ，HBase 1.1.1 ， Zookeeper 3.4.12 >Reporter: jiajingsi >Priority: Major > > 我测试过2.4和2.5.0版本，但是都是同样的问题。每次创建cube后，执行Monitor时，任务跑到""Convert Cuboid Data to > HFile""这里都会失败，日志显示的是： > org.apache.kylin.engine.mr.exception.MapReduceException: > at > org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:173) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:163) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:69) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:163) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:113) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > > 有人遇到过类似的问题吗？请帮忙看一下 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-3614) kylin2.4维度表字段内容超长报错
[ https://issues.apache.org/jira/browse/KYLIN-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049264#comment-17049264 ] Shao Feng Shi commented on KYLIN-3614: -- No plan to support CDH as dev env. You can try to use the community release. > kylin2.4维度表字段内容超长报错 > --- > > Key: KYLIN-3614 > URL: https://issues.apache.org/jira/browse/KYLIN-3614 > Project: Kylin > Issue Type: Bug > Components: Job Engine, Tools, Build and Test >Affects Versions: v2.4.0 >Reporter: wlxie >Priority: Major > Labels: build > Attachments: 维度表字段内容超长.txt > > > 各位老师好， > kylin2.4关联维度表创建cube，如果维度表某些字段值超长会报错，报错信息请参考附件。 > 对此有两个建议。 > 1. 报错信息不友好，没有告诉用户是哪个表哪个字段超长，导致定位问题困难。 > 2. 在cube中未使用到的字段建议不要做字段长度检查，目前是所有字段都有去检查，包括并未使用到的字段。 > 谢谢 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-4392) Bump Flink version from 1.9.2 to 1.10.0
[ https://issues.apache.org/jira/browse/KYLIN-4392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-4392. Resolution: Duplicate > Bump Flink version from 1.9.2 to 1.10.0 > --- > > Key: KYLIN-4392 > URL: https://issues.apache.org/jira/browse/KYLIN-4392 > Project: Kylin > Issue Type: Sub-task > Components: Flink Engine >Reporter: vinoyang >Assignee: vinoyang >Priority: Major > > Recently, Flink has release v1.10.0. It's a major version that contains some > great features and big changes. We can tentatively upgrade to v1.10.0 after > we upgrade to v1.9.2 firstly. If we find some fatal issues, we can fall back > to v1.9.2. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (KYLIN-4393) There are several CVEs in the project dependencies
[ https://issues.apache.org/jira/browse/KYLIN-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi reassigned KYLIN-4393: Component/s: Security Assignee: Yaqian Zhang > There are several CVEs in the project dependencies > -- > > Key: KYLIN-4393 > URL: https://issues.apache.org/jira/browse/KYLIN-4393 > Project: Kylin > Issue Type: Bug > Components: Security >Reporter: XuCongying >Assignee: Yaqian Zhang >Priority: Major > > Hi~ I noticed some of your libraries contained CVEs. I suggest updating their > versions to increase the security of your project. The following is a > detailed content. > > Vulnerable Library Version: org.scala-lang : scala-compiler : 2.11.0 > CVE ID: > [CVE-2017-15288](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-15288) > Import Path: engine-flink/pom.xml, kylin-it/pom.xml, engine-spark/pom.xml > Suggested Safe Versions: 2.11.12, 2.12.10, 2.12.4, 2.12.5, 2.12.6, 2.12.7, > 2.12.8, 2.12.9, 2.13.0, 2.13.0-M1, 2.13.0-M2, 2.13.0-M3, 2.13.0-M3-f73b161, > 2.13.0-M4, 2.13.0-M4-pre-20d3c21, 2.13.0-M5, 2.13.0-M5-1775dba, > 2.13.0-M5-5eef812, 2.13.0-M5-6e0cba7, 2.13.0-RC1, 2.13.0-RC2, 2.13.0-RC3, > 2.13.1 > Vulnerable Library Version: org.apache.tomcat : tomcat-catalina : 7.0.91 > CVE ID: > [CVE-2016-8735](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-8735), > > [CVE-2019-0232](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-0232), > > [CVE-2016-6794](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6794), > > [CVE-2016-6816](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6816), > > [CVE-2016-8745](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-8745), > > [CVE-2019-17563](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17563) > Import Path: tomcat-ext/pom.xml, server/pom.xml, server-base/pom.xml > Suggested Safe Versions: 10.0.0-M1, 7.0.100, 9.0.30, 9.0.31 > Vulnerable Library Version: com.h2database : h2 : 1.4.196 > CVE ID: > [CVE-2018-10054](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-10054), > > [CVE-2018-14335](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-14335) > Import Path: server/pom.xml, kylin-it/pom.xml, source-jdbc/pom.xml, > source-hive/pom.xml, datasource-sdk/pom.xml > Suggested Safe Versions: 1.4.198, 1.4.199, 1.4.200 > Vulnerable Library Version: com.google.guava : guava : 14.0 > CVE ID: > [CVE-2018-10237](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-10237) > Import Path: core-storage/pom.xml, stream-receiver/pom.xml, server/pom.xml, > core-cube/pom.xml, core-metadata/pom.xml, jdbc/pom.xml, > tool-assembly/pom.xml, core-metrics/pom.xml > Suggested Safe Versions: 24.1.1-android, 24.1.1-jre, 25.0-android, > 25.0-jre, 25.1-android, 25.1-jre, 26.0-android, 26.0-jre, 27.0-android, > 27.0-jre, 27.0.1-android, 27.0.1-jre, 27.1-android, 27.1-jre, 28.0-android, > 28.0-jre, 28.1-android, 28.1-jre, 28.2-android, 28.2-jre > Vulnerable Library Version: org.apache.hive.hcatalog : hive-hcatalog-core : > 1.2.1 > CVE ID: > [CVE-2015-7521](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-7521) > Import Path: metrics-reporter-hive/pom.xml, assembly/pom.xml, > server/pom.xml, kylin-it/pom.xml, source-jdbc/pom.xml, source-hive/pom.xml, > server-base/pom.xml > Suggested Safe Versions: 1.2.2, 2.0.0, 2.0.1, 2.1.0, 2.1.1, 2.2.0, 2.3.0, > 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.3.6, 3.0.0, 3.1.0, 3.1.1, 3.1.2 > > Vulnerable Library Version: org.apache.spark : spark-core_2.11 : 2.3.2 > CVE ID: > [CVE-2017-7678](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-7678), > > [CVE-2018-3826](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-3826), > > [CVE-2018-11770](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-11770), > > [CVE-2019-10099](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-10099) > Import Path: server/pom.xml, kylin-it/pom.xml, engine-spark/pom.xml, > storage-hbase/pom.xml > Suggested Safe Versions: 2.4.5 > > Vulnerable Library Version: org.apache.kafka : kafka_2.11 : 1.0.0 > CVE ID: > [CVE-2018-1288](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1288), > > [CVE-2019-17196](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17196) > Import Path: assembly/pom.xml, source-kafka/pom.xml, kylin-it/pom.xml, > stream-source-kafka/pom.xml, metrics-reporter-kafka/pom.xml > Suggested Safe Versions: 2.1.1, 2.2.0, 2.2.1, 2.2.2, 2.3.0, 2.3.1, 2.4.0 > > Vulnerable Library Version: org.apache.hive : hive-jdbc : 1.2.1 > CVE ID: > [CVE-2016-3083](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-3083), > > [CVE-2015-7521](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-7521), >
[jira] [Updated] (KYLIN-4132) Kylin needn't use "org.apache.directory.api.util.Strings" to import api-util.jar
[ https://issues.apache.org/jira/browse/KYLIN-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi updated KYLIN-4132: - Fix Version/s: v3.0.2 > Kylin needn't use "org.apache.directory.api.util.Strings" to import > api-util.jar > > > Key: KYLIN-4132 > URL: https://issues.apache.org/jira/browse/KYLIN-4132 > Project: Kylin > Issue Type: Improvement > Components: REST Service > Environment: Fusion Insight >Reporter: zhao jintao >Assignee: wangrupeng >Priority: Major > Fix For: v3.0.2 > > Original Estimate: 168h > Remaining Estimate: 168h > > Hi Team: > I read kylin code find that JobService class and ProjectService class import > "org.apache.directory.api.util.Strings" use the function of > "Strings.isEmpty(String)" from "api-util.jar" to judge whether a string is > empty. > This function can be replaced by importing > "org.apache.commons.lang.StringUtils" use "StringUtils.isEmpty(String)" from > “common-lang.jar”. > The "api-util.jar" was dependent by hbase-client. When kylin build, > there isn't "api-util.jar" in kylin.war. When start kylin process, it will be > imported from hbase client lib. But if hbase client doesn't dependency > "api-util.jar", kylin running failed and will throw exception. > In fact, I encountered this problem in the latest Fusion-Insignt version. > Kylin start failed because there isn't "api-util.jar" in hbase client lib. > > So I suggest replace "org.apache.directory.api.util.Strings.isEmpty(String)" > to "org.apache.commons.lang.StringUtils.isEmpty(String)" to avoid this > problem. > Best wishes > Yours zjt. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4132) Kylin needn't use "org.apache.directory.api.util.Strings" to import api-util.jar
[ https://issues.apache.org/jira/browse/KYLIN-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi resolved KYLIN-4132. -- Resolution: Fixed > Kylin needn't use "org.apache.directory.api.util.Strings" to import > api-util.jar > > > Key: KYLIN-4132 > URL: https://issues.apache.org/jira/browse/KYLIN-4132 > Project: Kylin > Issue Type: Improvement > Components: REST Service > Environment: Fusion Insight >Reporter: zhao jintao >Assignee: wangrupeng >Priority: Major > Fix For: v3.0.2 > > Original Estimate: 168h > Remaining Estimate: 168h > > Hi Team: > I read kylin code find that JobService class and ProjectService class import > "org.apache.directory.api.util.Strings" use the function of > "Strings.isEmpty(String)" from "api-util.jar" to judge whether a string is > empty. > This function can be replaced by importing > "org.apache.commons.lang.StringUtils" use "StringUtils.isEmpty(String)" from > “common-lang.jar”. > The "api-util.jar" was dependent by hbase-client. When kylin build, > there isn't "api-util.jar" in kylin.war. When start kylin process, it will be > imported from hbase client lib. But if hbase client doesn't dependency > "api-util.jar", kylin running failed and will throw exception. > In fact, I encountered this problem in the latest Fusion-Insignt version. > Kylin start failed because there isn't "api-util.jar" in hbase client lib. > > So I suggest replace "org.apache.directory.api.util.Strings.isEmpty(String)" > to "org.apache.commons.lang.StringUtils.isEmpty(String)" to avoid this > problem. > Best wishes > Yours zjt. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4387) Flink cubing merge step failed
[ https://issues.apache.org/jira/browse/KYLIN-4387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049258#comment-17049258 ] ASF GitHub Bot commented on KYLIN-4387: --- harveyyue commented on pull request #1131: KYLIN-4387 Flink cubing merge step failed URL: https://github.com/apache/kylin/pull/1131 Please ignore the pull request #1125, because the branch had the extra commit issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Flink cubing merge step failed > -- > > Key: KYLIN-4387 > URL: https://issues.apache.org/jira/browse/KYLIN-4387 > Project: Kylin > Issue Type: Bug > Components: Flink Engine >Reporter: Harvey Yue >Assignee: Harvey Yue >Priority: Major > Fix For: v3.1.0 > > > When job running this step face following issue, I guess which have two class > loader to loading the class *InstantiatorStrategy* when flink execute the > reduceGroup operator > Workaround：using ByteArray instead of Text in reduceGroup operator > The kylin binary including the flink core jar is the root cause, we need to > change the flink related dependence scope to provided. > > The program finished with the following exception: > org.apache.flink.client.program.ProgramInvocationException: The main method > caused an error: error execute > org.apache.kylin.engine.flink.FlinkCubingMerge. Root cause: Job failed. > (JobID: 7a40a92561cfa972e3c5b975db993fce) > at > org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:593) > at > org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438) > at > org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274) > at > org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746) > at > org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273) > at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205) > at > org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1010) > at > org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1083) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) > at > org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) > at > org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1083) > Caused by: java.lang.RuntimeException: error execute > org.apache.kylin.engine.flink.FlinkCubingMerge. Root cause: Job failed. > (JobID: 7a40a92561cfa972e3c5b975db993fce) > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42) > at org.apache.kylin.common.util.FlinkEntry.main(FlinkEntry.java:46) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:576) > ... 12 more > Caused by: org.apache.flink.client.program.ProgramInvocationException: Job > failed. (JobID: 7a40a92561cfa972e3c5b975db993fce) > at > org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:262) > at > org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:338) > at > org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:326) > at > org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:62) > at > org.apache.kylin.engine.flink.FlinkCubingMerge.execute(FlinkCubingMerge.java:235) > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37) > ... 18 more > Caused by: org.apache.flink.runtime.client.JobExecutionException: Job > execution failed. > at > org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146) > at > org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:259) > ... 23 more > Caused by:
[jira] [Commented] (KYLIN-4387) Flink cubing merge step failed
[ https://issues.apache.org/jira/browse/KYLIN-4387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049257#comment-17049257 ] ASF GitHub Bot commented on KYLIN-4387: --- shaofengshi commented on pull request #1125: KYLIN-4387 Flink cubing merge step failed URL: https://github.com/apache/kylin/pull/1125 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Flink cubing merge step failed > -- > > Key: KYLIN-4387 > URL: https://issues.apache.org/jira/browse/KYLIN-4387 > Project: Kylin > Issue Type: Bug > Components: Flink Engine >Reporter: Harvey Yue >Assignee: Harvey Yue >Priority: Major > Fix For: v3.1.0 > > > When job running this step face following issue, I guess which have two class > loader to loading the class *InstantiatorStrategy* when flink execute the > reduceGroup operator > Workaround：using ByteArray instead of Text in reduceGroup operator > The kylin binary including the flink core jar is the root cause, we need to > change the flink related dependence scope to provided. > > The program finished with the following exception: > org.apache.flink.client.program.ProgramInvocationException: The main method > caused an error: error execute > org.apache.kylin.engine.flink.FlinkCubingMerge. Root cause: Job failed. > (JobID: 7a40a92561cfa972e3c5b975db993fce) > at > org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:593) > at > org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438) > at > org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274) > at > org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746) > at > org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273) > at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205) > at > org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1010) > at > org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1083) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) > at > org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) > at > org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1083) > Caused by: java.lang.RuntimeException: error execute > org.apache.kylin.engine.flink.FlinkCubingMerge. Root cause: Job failed. > (JobID: 7a40a92561cfa972e3c5b975db993fce) > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42) > at org.apache.kylin.common.util.FlinkEntry.main(FlinkEntry.java:46) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:576) > ... 12 more > Caused by: org.apache.flink.client.program.ProgramInvocationException: Job > failed. (JobID: 7a40a92561cfa972e3c5b975db993fce) > at > org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:262) > at > org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:338) > at > org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:326) > at > org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:62) > at > org.apache.kylin.engine.flink.FlinkCubingMerge.execute(FlinkCubingMerge.java:235) > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37) > ... 18 more > Caused by: org.apache.flink.runtime.client.JobExecutionException: Job > execution failed. > at > org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146) > at > org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:259) > ... 23 more > Caused by: java.lang.LinkageError: loader constraint violation: when > resolving method >
[GitHub] [kylin] harveyyue opened a new pull request #1131: KYLIN-4387 Flink cubing merge step failed
harveyyue opened a new pull request #1131: KYLIN-4387 Flink cubing merge step failed URL: https://github.com/apache/kylin/pull/1131 Please ignore the pull request #1125, because the branch had the extra commit issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] shaofengshi closed pull request #1125: KYLIN-4387 Flink cubing merge step failed
shaofengshi closed pull request #1125: KYLIN-4387 Flink cubing merge step failed URL: https://github.com/apache/kylin/pull/1125 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] shaofengshi commented on issue #1125: KYLIN-4387 Flink cubing merge step failed
shaofengshi commented on issue #1125: KYLIN-4387 Flink cubing merge step failed URL: https://github.com/apache/kylin/pull/1125#issuecomment-593432722 I should close this one, and reopen the other. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] shaofengshi commented on a change in pull request #1125: KYLIN-4387 Flink cubing merge step failed
shaofengshi commented on a change in pull request #1125: KYLIN-4387 Flink
cubing merge step failed
URL: https://github.com/apache/kylin/pull/1125#discussion_r386427647
##
File path: pom.xml
##
@@ -748,6 +751,24 @@
${kryo.version}
provided
+
+
+org.apache.flink
+flink-java
+${flink.version}
+provided
+
+
+org.apache.flink
+flink-scala_2.11
+${flink.version}
+provided
+
+
+org.apache.flink
+flink-hadoop-compatibility_2.11
+${flink.version}
Review comment:
Does this also need be "provided"?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (KYLIN-4387) Flink cubing merge step failed
[ https://issues.apache.org/jira/browse/KYLIN-4387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049248#comment-17049248 ] ASF GitHub Bot commented on KYLIN-4387: --- shaofengshi commented on pull request #1131: KYLIN-4387 Flink cubing merge step failed URL: https://github.com/apache/kylin/pull/1131 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Flink cubing merge step failed > -- > > Key: KYLIN-4387 > URL: https://issues.apache.org/jira/browse/KYLIN-4387 > Project: Kylin > Issue Type: Bug > Components: Flink Engine >Reporter: Harvey Yue >Assignee: Harvey Yue >Priority: Major > Fix For: v3.1.0 > > > When job running this step face following issue, I guess which have two class > loader to loading the class *InstantiatorStrategy* when flink execute the > reduceGroup operator > Workaround：using ByteArray instead of Text in reduceGroup operator > The kylin binary including the flink core jar is the root cause, we need to > change the flink related dependence scope to provided. > > The program finished with the following exception: > org.apache.flink.client.program.ProgramInvocationException: The main method > caused an error: error execute > org.apache.kylin.engine.flink.FlinkCubingMerge. Root cause: Job failed. > (JobID: 7a40a92561cfa972e3c5b975db993fce) > at > org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:593) > at > org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438) > at > org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274) > at > org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746) > at > org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273) > at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205) > at > org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1010) > at > org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1083) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) > at > org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) > at > org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1083) > Caused by: java.lang.RuntimeException: error execute > org.apache.kylin.engine.flink.FlinkCubingMerge. Root cause: Job failed. > (JobID: 7a40a92561cfa972e3c5b975db993fce) > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42) > at org.apache.kylin.common.util.FlinkEntry.main(FlinkEntry.java:46) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:576) > ... 12 more > Caused by: org.apache.flink.client.program.ProgramInvocationException: Job > failed. (JobID: 7a40a92561cfa972e3c5b975db993fce) > at > org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:262) > at > org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:338) > at > org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:326) > at > org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:62) > at > org.apache.kylin.engine.flink.FlinkCubingMerge.execute(FlinkCubingMerge.java:235) > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37) > ... 18 more > Caused by: org.apache.flink.runtime.client.JobExecutionException: Job > execution failed. > at > org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146) > at > org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:259) > ... 23 more > Caused by: java.lang.LinkageError: loader constraint violation: when > resolving method >
[jira] [Commented] (KYLIN-4396) File Descriptor Leakage in MR Build Engine
[
https://issues.apache.org/jira/browse/KYLIN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049246#comment-17049246
]
Shao Feng Shi commented on KYLIN-4396:
--
Good catching!
> File Descriptor Leakage in MR Build Engine
> --
>
> Key: KYLIN-4396
> URL: https://issues.apache.org/jira/browse/KYLIN-4396
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v3.0.0, v3.0.1
>Reporter: Xiaoxiang Yu
>Priority: Major
> Fix For: v3.1.0
>
>
> After runing for one or two days, Job Server failed to response to any http
> requests, but the process looks good, without high CPU pressure etc. In
> _kylin.out_, we find error message indicate that process has hit the limit of
> *FD.*
> After checking detail, we find many fds are not released after a long period,
> most of them are socket in state, that is *CLOSE_WAIT*.
> !https://user-images.githubusercontent.com/14030549/72678560-0fc7e580-3ae2-11ea-85ea-5dcb40e97bd7.png!
>
> !https://user-images.githubusercontent.com/14030549/72678638-c9bf5180-3ae2-11ea-880c-7f8e34d17a8c.png!
>
>
> {code:java}
> 一月 19, 2020 5:13:51 下午 org.apache.catalina.core.StandardWrapperValve invoke
> 严重: Servlet.service() for servlet [default] in context with path [/kylin]
> threw exception java.io.FileNotFoundException:
> /root/xiaoxiang.yu/deploy/kylin-all/tomcat/webapps/kylin/js/scripts.min.20191209061232.js
> (打开的文件过多) at java.io.FileInputStream.open0(Native Method) at
> java.io.FileInputStream.open(FileInputStream.java:195) at
> java.io.FileInputStream.(FileInputStream.java:138) at
> org.apache.naming.resources.FileDirContext$FileResource.streamContent(FileDirContext.java:1008)
> at
> org.apache.catalina.servlets.DefaultServlet.copy(DefaultServlet.java:2084) at
> org.apache.catalina.servlets.DefaultServlet.serveResource(DefaultServlet.java:1023)
> at
> org.apache.catalina.servlets.DefaultServlet.doGet(DefaultServlet.java:453) at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:624) at
> org.apache.catalina.servlets.DefaultServlet.service(DefaultServlet.java:433)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
>
[GitHub] [kylin] shaofengshi closed pull request #1131: KYLIN-4387 Flink cubing merge step failed
shaofengshi closed pull request #1131: KYLIN-4387 Flink cubing merge step failed URL: https://github.com/apache/kylin/pull/1131 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (KYLIN-4389) Change Flink jar dependence from compile to provided
[ https://issues.apache.org/jira/browse/KYLIN-4389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi resolved KYLIN-4389. -- Resolution: Fixed > Change Flink jar dependence from compile to provided > - > > Key: KYLIN-4389 > URL: https://issues.apache.org/jira/browse/KYLIN-4389 > Project: Kylin > Issue Type: Sub-task > Components: Flink Engine >Reporter: Harvey Yue >Assignee: Harvey Yue >Priority: Major > Fix For: v3.1.0 > > > Currently, build module engine-flink which including flink related core jar > to kylin package, blow up 70M to package, so I suggest that change flink > dependence scope > One more thing, I guess this fat jar(kylin-job-xxx.jar) cause the issue > KYLIN-4387 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4389) Change Flink jar dependence from compile to provided
[ https://issues.apache.org/jira/browse/KYLIN-4389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049235#comment-17049235 ] ASF GitHub Bot commented on KYLIN-4389: --- shaofengshi commented on pull request #1128: KYLIN-4389 Change Flink jar dependence from compile to provided URL: https://github.com/apache/kylin/pull/1128 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Change Flink jar dependence from compile to provided > - > > Key: KYLIN-4389 > URL: https://issues.apache.org/jira/browse/KYLIN-4389 > Project: Kylin > Issue Type: Sub-task > Components: Flink Engine >Reporter: Harvey Yue >Assignee: Harvey Yue >Priority: Major > Fix For: v3.1.0 > > > Currently, build module engine-flink which including flink related core jar > to kylin package, blow up 70M to package, so I suggest that change flink > dependence scope > One more thing, I guess this fat jar(kylin-job-xxx.jar) cause the issue > KYLIN-4387 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] shaofengshi closed pull request #1128: KYLIN-4389 Change Flink jar dependence from compile to provided
shaofengshi closed pull request #1128: KYLIN-4389 Change Flink jar dependence from compile to provided URL: https://github.com/apache/kylin/pull/1128 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4397) Use newLinkedHashMap in AssignmentUtil.java
[ https://issues.apache.org/jira/browse/KYLIN-4397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049234#comment-17049234 ] ASF GitHub Bot commented on KYLIN-4397: --- cpugputpu commented on pull request #1133: KYLIN-4397 Use newLinkedHashMap for deterministic order in AssignmentUtil.java URL: https://github.com/apache/kylin/pull/1133 This PR aims to solve the issue here: https://issues.apache.org/jira/browse/KYLIN-4397 The fix is to use `newLinkedHashMap` instead of `newHashMap` when initializing a Map. In this way, the non-deterministic behaviour is eliminated and the test will not suffer from the failure again. The code will be more stable. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Use newLinkedHashMap in AssignmentUtil.java > --- > > Key: KYLIN-4397 > URL: https://issues.apache.org/jira/browse/KYLIN-4397 > Project: Kylin > Issue Type: Bug >Reporter: cpugputpu >Priority: Minor > > The tests in > _org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest#reBalanceTest_ > can fail and the failure is presented as follow. > _java.lang.AssertionError: expected:<2> but was:<1>_ > _at > org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:165)_ > when executing _assertEquals(2, rsAssignment.size());_ > > The root cause of this failure is when executing _Map List>> result = assigner.reBalancePlan(rsList, cubes, > existingAssignments);_ it will iterate over a HashMap. The stack trace is > here: > java.util.HashMap$EntrySet.iterator(HashMap.java:1014) > > org.apache.kylin.stream.coordinator.assign.DefaultAssigner.reBalancePlan(DefaultAssigner.java:93) > > org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:163)_ > The specification about HashMap says that "this class makes no guarantees as > to the order of the map; in particular, it does not guarantee that the order > will remain constant over time". The documentation is here for your > reference: > [https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.htm|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html][l|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html] > The fix is to use LinkedHashMap instead of HashMap when initializing a Map. > In this way, the non-deterministic behaviour is eliminated and the test will > not suffer from the failure again. The code will be more stable. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] shaofengshi commented on issue #1128: KYLIN-4389 Change Flink jar dependence from compile to provided
shaofengshi commented on issue #1128: KYLIN-4389 Change Flink jar dependence from compile to provided URL: https://github.com/apache/kylin/pull/1128#issuecomment-593417995 New pr merged, this pr is not needed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] cpugputpu opened a new pull request #1133: KYLIN-4397 Use newLinkedHashMap for deterministic order in AssignmentUtil.java
cpugputpu opened a new pull request #1133: KYLIN-4397 Use newLinkedHashMap for deterministic order in AssignmentUtil.java URL: https://github.com/apache/kylin/pull/1133 This PR aims to solve the issue here: https://issues.apache.org/jira/browse/KYLIN-4397 The fix is to use `newLinkedHashMap` instead of `newHashMap` when initializing a Map. In this way, the non-deterministic behaviour is eliminated and the test will not suffer from the failure again. The code will be more stable. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (KYLIN-4397) Use newLinkedHashMap in AssignmentUtil.java
[ https://issues.apache.org/jira/browse/KYLIN-4397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cpugputpu updated KYLIN-4397: - Description: The tests in _org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest#reBalanceTest_ can fail and the failure is presented as follow. _java.lang.AssertionError: expected:<2> but was:<1>_ _at org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:165)_ when executing _assertEquals(2, rsAssignment.size());_ The root cause of this failure is when executing _Map>> result = assigner.reBalancePlan(rsList, cubes, existingAssignments);_ it will iterate over a HashMap. The stack trace is here: java.util.HashMap$EntrySet.iterator(HashMap.java:1014) org.apache.kylin.stream.coordinator.assign.DefaultAssigner.reBalancePlan(DefaultAssigner.java:93) org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:163)_ The specification about HashMap says that "this class makes no guarantees as to the order of the map; in particular, it does not guarantee that the order will remain constant over time". The documentation is here for your reference: [https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.htm|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html][l|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html] The fix is to use LinkedHashMap instead of HashMap when initializing a Map. In this way, the non-deterministic behaviour is eliminated and the test will not suffer from the failure again. The code will be more stable. was: The tests in _org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest#reBalanceTest_ can fail and the failure is presented as follow. _java.lang.AssertionError: expected:<2> but was:<1>_ _at org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:165)_ when executing _assertEquals(2, rsAssignment.size());_ The root cause of this failure is when executing _Map>> result = assigner.reBalancePlan(rsList, cubes, existingAssignments);_ it will iterate over a HashMap. The stack trace is here: _java.util.HashMap$EntrySet.iterator(HashMap.java:1014) org.apache.kylin.stream.coordinator.assign.DefaultAssigner.reBalancePlan(DefaultAssigner.java:93) org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:163)_ The specification about HashMap says that "this class makes no guarantees as to the order of the map; in particular, it does not guarantee that the order will remain constant over time". The documentation is here for your reference: [https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.htm|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html][l|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html] The fix is to use LinkedHashMap instead of HashMap when initializing a Map. In this way, the non-deterministic behaviour is eliminated and the test will not suffer from the failure again. The code will be more stable. > Use newLinkedHashMap in AssignmentUtil.java > --- > > Key: KYLIN-4397 > URL: https://issues.apache.org/jira/browse/KYLIN-4397 > Project: Kylin > Issue Type: Bug >Reporter: cpugputpu >Priority: Minor > > The tests in > _org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest#reBalanceTest_ > can fail and the failure is presented as follow. > _java.lang.AssertionError: expected:<2> but was:<1>_ > _at > org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:165)_ > when executing _assertEquals(2, rsAssignment.size());_ > > The root cause of this failure is when executing _Map List>> result = assigner.reBalancePlan(rsList, cubes, > existingAssignments);_ it will iterate over a HashMap. The stack trace is > here: > java.util.HashMap$EntrySet.iterator(HashMap.java:1014) > > org.apache.kylin.stream.coordinator.assign.DefaultAssigner.reBalancePlan(DefaultAssigner.java:93) > > org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:163)_ > The specification about HashMap says that "this class makes no guarantees as > to the order of the map; in particular, it does not guarantee that the order > will remain constant over time". The documentation is here for your > reference: > [https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.htm|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html][l|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html] > The fix is to use LinkedHashMap instead of HashMap when initializing a Map. > In this way, the non-deterministic behaviour is eliminated and the test will > not suffer from the failure again. The
[jira] [Updated] (KYLIN-4397) Use newLinkedHashMap in AssignmentUtil.java
[ https://issues.apache.org/jira/browse/KYLIN-4397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cpugputpu updated KYLIN-4397: - Summary: Use newLinkedHashMap in AssignmentUtil.java (was: Use newLinkedHashMap in stream-coordinator/src/main/java/org/apache/kylin/stream/coordinator/assign/AssignmentUtil.java) > Use newLinkedHashMap in AssignmentUtil.java > --- > > Key: KYLIN-4397 > URL: https://issues.apache.org/jira/browse/KYLIN-4397 > Project: Kylin > Issue Type: Bug >Reporter: cpugputpu >Priority: Minor > > The tests in > _org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest#reBalanceTest_ > can fail and the failure is presented as follow. > _java.lang.AssertionError: expected:<2> but was:<1>_ > _at > org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:165)_ > when executing _assertEquals(2, rsAssignment.size());_ > > The root cause of this failure is when executing _Map List>> result = assigner.reBalancePlan(rsList, cubes, > existingAssignments);_ it will iterate over a HashMap. The stack trace is > here: > _java.util.HashMap$EntrySet.iterator(HashMap.java:1014) > org.apache.kylin.stream.coordinator.assign.DefaultAssigner.reBalancePlan(DefaultAssigner.java:93) > org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:163)_ > The specification about HashMap says that "this class makes no guarantees as > to the order of the map; in particular, it does not guarantee that the order > will remain constant over time". The documentation is here for your > reference: > [https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.htm|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html][l|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html] > The fix is to use LinkedHashMap instead of HashMap when initializing a Map. > In this way, the non-deterministic behaviour is eliminated and the test will > not suffer from the failure again. The code will be more stable. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KYLIN-4397) Use newLinkedHashMap in stream-coordinator/src/main/java/org/apache/kylin/stream/coordinator/assign/AssignmentUtil.java
cpugputpu created KYLIN-4397: Summary: Use newLinkedHashMap in stream-coordinator/src/main/java/org/apache/kylin/stream/coordinator/assign/AssignmentUtil.java Key: KYLIN-4397 URL: https://issues.apache.org/jira/browse/KYLIN-4397 Project: Kylin Issue Type: Bug Reporter: cpugputpu The tests in _org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest#reBalanceTest_ can fail and the failure is presented as follow. _java.lang.AssertionError: expected:<2> but was:<1>_ _at org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:165)_ when executing _assertEquals(2, rsAssignment.size());_ The root cause of this failure is when executing _Map>> result = assigner.reBalancePlan(rsList, cubes, existingAssignments);_ it will iterate over a HashMap. The stack trace is here: _java.util.HashMap$EntrySet.iterator(HashMap.java:1014) org.apache.kylin.stream.coordinator.assign.DefaultAssigner.reBalancePlan(DefaultAssigner.java:93) org.apache.kylin.stream.coordinator.assign.DefaultAssignerTest.reBalanceTest(DefaultAssignerTest.java:163)_ The specification about HashMap says that "this class makes no guarantees as to the order of the map; in particular, it does not guarantee that the order will remain constant over time". The documentation is here for your reference: [https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.htm|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html][l|https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html] The fix is to use LinkedHashMap instead of HashMap when initializing a Map. In this way, the non-deterministic behaviour is eliminated and the test will not suffer from the failure again. The code will be more stable. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] shaofengshi merged pull request #1132: Upgrade tomcat vertion to 8.5.51
shaofengshi merged pull request #1132: Upgrade tomcat vertion to 8.5.51 URL: https://github.com/apache/kylin/pull/1132 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4389) Change Flink jar dependence from compile to provided
[ https://issues.apache.org/jira/browse/KYLIN-4389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049227#comment-17049227 ] ASF subversion and git services commented on KYLIN-4389: Commit 8ce3deafa1498d4f39bfb9b36e84da7627065296 in kylin's branch refs/heads/master from Harvey Yue [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=8ce3dea ] KYLIN-4389 Change Flink jar dependence from compile to provided > Change Flink jar dependence from compile to provided > - > > Key: KYLIN-4389 > URL: https://issues.apache.org/jira/browse/KYLIN-4389 > Project: Kylin > Issue Type: Sub-task > Components: Flink Engine >Reporter: Harvey Yue >Assignee: Harvey Yue >Priority: Major > Fix For: v3.1.0 > > > Currently, build module engine-flink which including flink related core jar > to kylin package, blow up 70M to package, so I suggest that change flink > dependence scope > One more thing, I guess this fat jar(kylin-job-xxx.jar) cause the issue > KYLIN-4387 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4389) Change Flink jar dependence from compile to provided
[ https://issues.apache.org/jira/browse/KYLIN-4389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049226#comment-17049226 ] ASF GitHub Bot commented on KYLIN-4389: --- shaofengshi commented on pull request #1130: KYLIN-4389 Change Flink jar dependence from compile to provided URL: https://github.com/apache/kylin/pull/1130 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Change Flink jar dependence from compile to provided > - > > Key: KYLIN-4389 > URL: https://issues.apache.org/jira/browse/KYLIN-4389 > Project: Kylin > Issue Type: Sub-task > Components: Flink Engine >Reporter: Harvey Yue >Assignee: Harvey Yue >Priority: Major > Fix For: v3.1.0 > > > Currently, build module engine-flink which including flink related core jar > to kylin package, blow up 70M to package, so I suggest that change flink > dependence scope > One more thing, I guess this fat jar(kylin-job-xxx.jar) cause the issue > KYLIN-4387 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] shaofengshi merged pull request #1130: KYLIN-4389 Change Flink jar dependence from compile to provided
shaofengshi merged pull request #1130: KYLIN-4389 Change Flink jar dependence from compile to provided URL: https://github.com/apache/kylin/pull/1130 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] codecov-io edited a comment on issue #1132: Upgrade tomcat vertion to 8.5.51
codecov-io edited a comment on issue #1132: Upgrade tomcat vertion to 8.5.51 URL: https://github.com/apache/kylin/pull/1132#issuecomment-593341757 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1132?src=pr=h1) Report > :exclamation: No coverage uploaded for pull request base (`master-hadoop3.1@47b4e3c`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/kylin/pull/1132?src=pr=tree) ```diff @@ Coverage Diff @@ ## master-hadoop3.1#1132 +/- ## === Coverage? 24.61% Complexity ? 6113 === Files ? 1416 Lines ?86102 Branches?12083 === Hits?21198 Misses ?62882 Partials? 2022 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1132?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...c/main/java/org/apache/kylin/rest/DebugTomcat.java](https://codecov.io/gh/apache/kylin/pull/1132/diff?src=pr=tree#diff-c2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9yZXN0L0RlYnVnVG9tY2F0LmphdmE=) | `0% <ø> (ø)` | `0 <0> (?)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1132?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/1132?src=pr=footer). Last update [47b4e3c...4384147](https://codecov.io/gh/apache/kylin/pull/1132?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] codecov-io commented on issue #1132: Upgrade tomcat vertion to 8.5.51
codecov-io commented on issue #1132: Upgrade tomcat vertion to 8.5.51 URL: https://github.com/apache/kylin/pull/1132#issuecomment-593341757 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1132?src=pr=h1) Report > :exclamation: No coverage uploaded for pull request base (`master-hadoop3.1@47b4e3c`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/kylin/pull/1132?src=pr=tree) ```diff @@ Coverage Diff @@ ## master-hadoop3.1#1132 +/- ## === Coverage? 24.61% Complexity ? 6113 === Files ? 1416 Lines ?86102 Branches?12083 === Hits?21197 Misses ?62883 Partials? 2022 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1132?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...c/main/java/org/apache/kylin/rest/DebugTomcat.java](https://codecov.io/gh/apache/kylin/pull/1132/diff?src=pr=tree#diff-c2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9yZXN0L0RlYnVnVG9tY2F0LmphdmE=) | `0% <ø> (ø)` | `0 <0> (?)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1132?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/1132?src=pr=footer). Last update [47b4e3c...cf5f4b0](https://codecov.io/gh/apache/kylin/pull/1132?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] RupengWang commented on issue #1132: Upgrade tomcat vertion to 8.5.51
RupengWang commented on issue #1132: Upgrade tomcat vertion to 8.5.51 URL: https://github.com/apache/kylin/pull/1132#issuecomment-593329065 Package test passed in cdh6.0 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] RupengWang opened a new pull request #1132: Upgrade tomcat vertion to 8.5.51
RupengWang opened a new pull request #1132: Upgrade tomcat vertion to 8.5.51 URL: https://github.com/apache/kylin/pull/1132 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
