[jira] [Commented] (SPARK-29322) History server is stuck reading incomplete event log file compressed with zstd

2019-10-01 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16942508#comment-16942508 ] Jungtaek Lim commented on SPARK-29322: -- For event log, we seem to still use "com.gi

[jira] [Commented] (SPARK-29322) History server is stuck reading incomplete event log file compressed with zstd

2019-10-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16942502#comment-16942502 ] Dongjoon Hyun commented on SPARK-29322: --- AFAIK, we are using Hadoop ZStandardCodec

[jira] [Commented] (SPARK-29322) History server is stuck reading incomplete event log file compressed with zstd

2019-10-01 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16942493#comment-16942493 ] Jungtaek Lim commented on SPARK-29322: -- {quote} Since this is ZSTD, you are using `

[jira] [Commented] (SPARK-29322) History server is stuck reading incomplete event log file compressed with zstd

2019-10-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16942489#comment-16942489 ] Dongjoon Hyun commented on SPARK-29322: --- Also, cc [~dbtsai] > History server is s

[jira] [Commented] (SPARK-29322) History server is stuck reading incomplete event log file compressed with zstd

2019-10-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16942488#comment-16942488 ] Dongjoon Hyun commented on SPARK-29322: --- Since this is ZSTD, you are using `hadoop

[jira] [Commented] (SPARK-29322) History server is stuck reading incomplete event log file compressed with zstd

2019-10-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16942483#comment-16942483 ] Dongjoon Hyun commented on SPARK-29322: --- Can we file and link an upstream issue Ha

[jira] [Commented] (SPARK-29322) History server is stuck reading incomplete event log file compressed with zstd

2019-10-01 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16942456#comment-16942456 ] Jungtaek Lim commented on SPARK-29322: -- Just initiated discussion on dev. mailing l

[jira] [Commented] (SPARK-29322) History server is stuck reading incomplete event log file compressed with zstd

2019-10-01 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16942446#comment-16942446 ] Jungtaek Lim commented on SPARK-29322: -- FYI, thread being stuck was finished immedi

[jira] [Commented] (SPARK-29322) History server is stuck reading incomplete event log file compressed with zstd

2019-10-01 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16942445#comment-16942445 ] Jungtaek Lim commented on SPARK-29322: -- I'll work on PR to propose removing zstd fr