[jira] [Commented] (SPARK-15343) NoClassDefFoundError when initializing Spark with YARN

[ https://issues.apache.org/jira/browse/SPARK-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15335039#comment-15335039 ] Saisai Shao commented on SPARK-15343: - If timeline is enabled, YarnClient will also p

[jira] [Commented] (SPARK-15984) WARN message "o.a.h.y.s.resourcemanager.rmapp.RMAppImpl: The specific max attempts: 0 for application: 8 is invalid" when starting application on YARN

[ https://issues.apache.org/jira/browse/SPARK-15984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15337278#comment-15337278 ] Saisai Shao commented on SPARK-15984: - Is there any problem? I guess you might set ma

[jira] [Commented] (SPARK-15984) WARN message "o.a.h.y.s.resourcemanager.rmapp.RMAppImpl: The specific max attempts: 0 for application: 8 is invalid" when starting application on YARN

[ https://issues.apache.org/jira/browse/SPARK-15984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15339136#comment-15339136 ] Saisai Shao commented on SPARK-15984: - I see. Here it is because user didn't specify

[jira] [Commented] (SPARK-16085) Spark stand-alone ui redirects to RM application master UI for yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-16085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345445#comment-15345445 ] Saisai Shao commented on SPARK-16085: - [~yeshavora], it is the expected behavior, the

[jira] [Commented] (SPARK-16146) Spark application failed by Yarn preempting

[ https://issues.apache.org/jira/browse/SPARK-16146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345942#comment-15345942 ] Saisai Shao commented on SPARK-16146: - If it is due to preemption, AM log will show t

[jira] [Created] (SPARK-16166) Correctly honor off heap memory usage in web ui and log display

Saisai Shao created SPARK-16166: --- Summary: Correctly honor off heap memory usage in web ui and log display Key: SPARK-16166 URL: https://issues.apache.org/jira/browse/SPARK-16166 Project: Spark

[jira] [Commented] (SPARK-16146) Spark application failed by Yarn preempting

[ https://issues.apache.org/jira/browse/SPARK-16146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345984#comment-15345984 ] Saisai Shao commented on SPARK-16146: - I see, that could explain why executor get los

[jira] [Created] (SPARK-4381) User should get warned when set spark.master with local in Spark Streaming

Saisai Shao created SPARK-4381: -- Summary: User should get warned when set spark.master with local in Spark Streaming Key: SPARK-4381 URL: https://issues.apache.org/jira/browse/SPARK-4381 Project: Spark

[jira] [Commented] (SPARK-4537) Add 'processing delay' and 'totalDelay' to the metrics reported by the Spark Streaming subsystem

[ https://issues.apache.org/jira/browse/SPARK-4537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14222677#comment-14222677 ] Saisai Shao commented on SPARK-4537: Hi [~gmaas], are you going to fix this issue? If

[jira] [Created] (SPARK-4595) Spark MetricsServlet is not enabled because of initialization ordering

Saisai Shao created SPARK-4595: -- Summary: Spark MetricsServlet is not enabled because of initialization ordering Key: SPARK-4595 URL: https://issues.apache.org/jira/browse/SPARK-4595 Project: Spark

[jira] [Updated] (SPARK-4595) Spark MetricsServlet is not worked because of initialization ordering

[ https://issues.apache.org/jira/browse/SPARK-4595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-4595: --- Summary: Spark MetricsServlet is not worked because of initialization ordering (was: Spark MetricsSer

[jira] [Commented] (SPARK-4595) Spark MetricsServlet is not worked because of initialization ordering

[ https://issues.apache.org/jira/browse/SPARK-4595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224177#comment-14224177 ] Saisai Shao commented on SPARK-4595: Hi Josh, I think this bug may probably be introd

[jira] [Commented] (SPARK-4002) KafkaStreamSuite "Kafka input stream" case fails on OSX

[ https://issues.apache.org/jira/browse/SPARK-4002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225516#comment-14225516 ] Saisai Shao commented on SPARK-4002: Hi TD, the Kafka unit test is quite OK in my loca

[jira] [Commented] (SPARK-4537) Add 'processing delay' and 'totalDelay' to the metrics reported by the Spark Streaming subsystem

[ https://issues.apache.org/jira/browse/SPARK-4537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225561#comment-14225561 ] Saisai Shao commented on SPARK-4537: Thanks TD, I'm going to fix this issue. > Add 'p

[jira] [Created] (SPARK-4671) Streaming block need not to replicate 2 copies when WAL is enabled

Saisai Shao created SPARK-4671: -- Summary: Streaming block need not to replicate 2 copies when WAL is enabled Key: SPARK-4671 URL: https://issues.apache.org/jira/browse/SPARK-4671 Project: Spark

[jira] [Commented] (SPARK-4707) Reliable Kafka Receiver can lose data if the block generator fails to store data

[ https://issues.apache.org/jira/browse/SPARK-4707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14232419#comment-14232419 ] Saisai Shao commented on SPARK-4707: Thanks Hari for this fix. So a basic question, ho

[jira] [Commented] (SPARK-4707) Reliable Kafka Receiver can lose data if the block generator fails to store data

[ https://issues.apache.org/jira/browse/SPARK-4707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14232451#comment-14232451 ] Saisai Shao commented on SPARK-4707: Seems the second choice is cool, looking forward

[jira] [Updated] (SPARK-4740) Netty's network bandwidth is much lower than NIO in spark-perf and Netty takes longer running time

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-4740: --- Affects Version/s: 1.2.0 > Netty's network bandwidth is much lower than NIO in spark-perf and Netty >

[jira] [Commented] (SPARK-4740) Netty's network bandwidth is much lower than NIO in spark-perf and Netty takes longer running time

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14234876#comment-14234876 ] Saisai Shao commented on SPARK-4740: We also tested with small dataset like 40GB, the

[jira] [Commented] (SPARK-4740) Netty's network bandwidth is much lower than NIO in spark-perf and Netty takes longer running time

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14234902#comment-14234902 ] Saisai Shao commented on SPARK-4740: Besides we also tested with 24 cores WSM cpu, the

[jira] [Commented] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14235036#comment-14235036 ] Saisai Shao commented on SPARK-4740: Hi [~rxin], the difference between NIO and Netty

[jira] [Commented] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14236552#comment-14236552 ] Saisai Shao commented on SPARK-4740: I will test it on my 24 cores and 12 HDDs cluster

[jira] [Commented] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14236777#comment-14236777 ] Saisai Shao commented on SPARK-4740: Hi Reynold, I just tested your patch with sort-by

[jira] [Commented] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240627#comment-14240627 ] Saisai Shao commented on SPARK-4740: Hi Aaron, would you mind giving us your system co

[jira] [Commented] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240813#comment-14240813 ] Saisai Shao commented on SPARK-4740: Thanks Aaron, we will try to use ramdisk to minim

[jira] [Commented] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14241973#comment-14241973 ] Saisai Shao commented on SPARK-4740: Hi Reynold, the code I pasted is just the example

[jira] [Comment Edited] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14241973#comment-14241973 ] Saisai Shao edited comment on SPARK-4740 at 12/11/14 1:34 AM: --

[jira] [Created] (SPARK-4847) extraStrategies cannot take effect in SQLContext

Saisai Shao created SPARK-4847: -- Summary: extraStrategies cannot take effect in SQLContext Key: SPARK-4847 URL: https://issues.apache.org/jira/browse/SPARK-4847 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3146) Improve the flexibility of Spark Streaming Kafka API to offer user the ability to process message before storing into BM

[ https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14252742#comment-14252742 ] Saisai Shao commented on SPARK-3146: Hi all, thanks a lot for your comments. My origin

[jira] [Commented] (SPARK-4960) Interceptor pattern in receivers

[ https://issues.apache.org/jira/browse/SPARK-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14258618#comment-14258618 ] Saisai Shao commented on SPARK-4960: I'd like to take a crack and provide a general so

[jira] [Commented] (SPARK-4923) Maven build should keep publishing spark-repl

[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14258666#comment-14258666 ] Saisai Shao commented on SPARK-4923: Hey [~pwendell], for some projects like Zeppelin(

[jira] [Commented] (SPARK-4960) Interceptor pattern in receivers

[ https://issues.apache.org/jira/browse/SPARK-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14258994#comment-14258994 ] Saisai Shao commented on SPARK-4960: What confused me a lot is the type inference, as

[jira] [Comment Edited] (SPARK-4960) Interceptor pattern in receivers

[ https://issues.apache.org/jira/browse/SPARK-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14258994#comment-14258994 ] Saisai Shao edited comment on SPARK-4960 at 12/26/14 8:47 AM: --

[jira] [Commented] (SPARK-4960) Interceptor pattern in receivers

[ https://issues.apache.org/jira/browse/SPARK-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14259972#comment-14259972 ] Saisai Shao commented on SPARK-4960: Hi [~tdas] and [~hshreedharan], these days I thin

[jira] [Commented] (SPARK-4960) Interceptor pattern in receivers

[ https://issues.apache.org/jira/browse/SPARK-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14259976#comment-14259976 ] Saisai Shao commented on SPARK-4960: Hi Hari, I think ReliableKafkaReciever only calle

[jira] [Commented] (SPARK-4960) Interceptor pattern in receivers

[ https://issues.apache.org/jira/browse/SPARK-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260655#comment-14260655 ] Saisai Shao commented on SPARK-4960: Hi Cody, please see the inline comments. {quote}

[jira] [Comment Edited] (SPARK-4960) Interceptor pattern in receivers

[ https://issues.apache.org/jira/browse/SPARK-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260655#comment-14260655 ] Saisai Shao edited comment on SPARK-4960 at 12/30/14 1:34 AM: --

[jira] [Created] (SPARK-4999) No need to put WAL-backed block into block manager by default

Saisai Shao created SPARK-4999: -- Summary: No need to put WAL-backed block into block manager by default Key: SPARK-4999 URL: https://issues.apache.org/jira/browse/SPARK-4999 Project: Spark Issu

[jira] [Updated] (SPARK-4999) No need to put WAL-backed block into block manager by default

[ https://issues.apache.org/jira/browse/SPARK-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-4999: --- Description: Currently WAL-backed block is read out from HDFS and put into BlockManger with storage le

[jira] [Commented] (SPARK-4999) No need to put WAL-backed block into block manager by default

[ https://issues.apache.org/jira/browse/SPARK-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260812#comment-14260812 ] Saisai Shao commented on SPARK-4999: {code} // Since storeInBlockManager = f

[jira] [Commented] (SPARK-4999) No need to put WAL-backed block into block manager by default

[ https://issues.apache.org/jira/browse/SPARK-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260890#comment-14260890 ] Saisai Shao commented on SPARK-4999: cc [~tdas], what is your opinion ? > No need to

[jira] [Created] (SPARK-5028) Add total received and processed records metrics to Streaming UI

Saisai Shao created SPARK-5028: -- Summary: Add total received and processed records metrics to Streaming UI Key: SPARK-5028 URL: https://issues.apache.org/jira/browse/SPARK-5028 Project: Spark I

[jira] [Commented] (SPARK-5001) BlockRDD removed unreasonablly in streaming

[ https://issues.apache.org/jira/browse/SPARK-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14263694#comment-14263694 ] Saisai Shao commented on SPARK-5001: Hi [~hanhg], I don't think it is a problem of Spa

[jira] [Commented] (SPARK-5001) BlockRDD removed unreasonablly in streaming

[ https://issues.apache.org/jira/browse/SPARK-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14263704#comment-14263704 ] Saisai Shao commented on SPARK-5001: Yeah, It's Spark Streaming design model's restric

[jira] [Commented] (SPARK-19802) Remote History Server

[ https://issues.apache.org/jira/browse/SPARK-19802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893580#comment-15893580 ] Saisai Shao commented on SPARK-19802: - Spark's {{ApplicationHistoryProvider}} is plug

[jira] [Commented] (SPARK-19812) YARN shuffle service fails to relocate recovery DB directories

[ https://issues.apache.org/jira/browse/SPARK-19812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900821#comment-15900821 ] Saisai Shao commented on SPARK-19812: - [~tgraves], I'm not quite sure what you mean h

[jira] [Comment Edited] (SPARK-19812) YARN shuffle service fails to relocate recovery DB directories

[ https://issues.apache.org/jira/browse/SPARK-19812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900821#comment-15900821 ] Saisai Shao edited comment on SPARK-19812 at 3/8/17 7:40 AM: -

[jira] [Comment Edited] (SPARK-19812) YARN shuffle service fails to relocate recovery DB directories

[ https://issues.apache.org/jira/browse/SPARK-19812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900821#comment-15900821 ] Saisai Shao edited comment on SPARK-19812 at 3/8/17 7:42 AM: -

[jira] [Commented] (SPARK-19143) API in Spark for distributing new delegation tokens (Improve delegation token handling in secure clusters)

[ https://issues.apache.org/jira/browse/SPARK-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15904497#comment-15904497 ] Saisai Shao commented on SPARK-19143: - Hi all, I wrote a rough design doc based on th

[jira] [Commented] (SPARK-19992) spark-submit on deployment-mode cluster

[ https://issues.apache.org/jira/browse/SPARK-19992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15929544#comment-15929544 ] Saisai Shao commented on SPARK-19992: - Are you using HDP environment, if so I guess n

[jira] [Commented] (SPARK-19992) spark-submit on deployment-mode cluster

[ https://issues.apache.org/jira/browse/SPARK-19992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15929547#comment-15929547 ] Saisai Shao commented on SPARK-19992: - Looks like I guess wrong from the url you prov

[jira] [Comment Edited] (SPARK-19992) spark-submit on deployment-mode cluster

[ https://issues.apache.org/jira/browse/SPARK-19992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15929547#comment-15929547 ] Saisai Shao edited comment on SPARK-19992 at 3/17/17 7:48 AM: -

[jira] [Created] (SPARK-19995) Using real user to connect HiveMetastore in HiveClientImpl

Saisai Shao created SPARK-19995: --- Summary: Using real user to connect HiveMetastore in HiveClientImpl Key: SPARK-19995 URL: https://issues.apache.org/jira/browse/SPARK-19995 Project: Spark Issu

[jira] [Commented] (SPARK-19941) Spark should not schedule tasks on executors on decommissioning YARN nodes

[ https://issues.apache.org/jira/browse/SPARK-19941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15932088#comment-15932088 ] Saisai Shao commented on SPARK-19941: - I think this scenario is quite similar to cont

[jira] [Commented] (SPARK-19992) spark-submit on deployment-mode cluster

[ https://issues.apache.org/jira/browse/SPARK-19992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15936241#comment-15936241 ] Saisai Shao commented on SPARK-19992: - Oh, I see. Check the code again, looks like "/

[jira] [Comment Edited] (SPARK-19992) spark-submit on deployment-mode cluster

[ https://issues.apache.org/jira/browse/SPARK-19992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15936241#comment-15936241 ] Saisai Shao edited comment on SPARK-19992 at 3/22/17 12:51 PM:

[jira] [Created] (SPARK-20059) HbaseCredentialProvider uses wrong classloader

Saisai Shao created SPARK-20059: --- Summary: HbaseCredentialProvider uses wrong classloader Key: SPARK-20059 URL: https://issues.apache.org/jira/browse/SPARK-20059 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-20058) the running application status changed from running to waiting when a master is down and it change to another standy by master

[ https://issues.apache.org/jira/browse/SPARK-20058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15936255#comment-15936255 ] Saisai Shao edited comment on SPARK-20058 at 3/22/17 1:03 PM: -

[jira] [Commented] (SPARK-20058) the running application status changed from running to waiting when a master is down and it change to another standy by master

[ https://issues.apache.org/jira/browse/SPARK-20058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15936255#comment-15936255 ] Saisai Shao commented on SPARK-20058: - Please subscribe this spark user mail list and

[jira] [Commented] (SPARK-20058) the running application status changed from running to waiting when a master is down and it change to another standy by master

[ https://issues.apache.org/jira/browse/SPARK-20058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15937841#comment-15937841 ] Saisai Shao commented on SPARK-20058: - I have a PR for this bug very very long ago (

[jira] [Comment Edited] (SPARK-20058) the running application status changed from running to waiting when a master is down and it change to another standy by master

[ https://issues.apache.org/jira/browse/SPARK-20058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15937841#comment-15937841 ] Saisai Shao edited comment on SPARK-20058 at 3/23/17 7:02 AM: -

[jira] [Commented] (SPARK-20050) Kafka 0.10 DirectStream doesn't commit last processed batch's offset when graceful shutdown

[ https://issues.apache.org/jira/browse/SPARK-20050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15937845#comment-15937845 ] Saisai Shao commented on SPARK-20050: - I think you could register a commit callback f

[jira] [Commented] (SPARK-19992) spark-submit on deployment-mode cluster

[ https://issues.apache.org/jira/browse/SPARK-19992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15939867#comment-15939867 ] Saisai Shao commented on SPARK-19992: - Sorry I cannot give you valid suggestions with

[jira] [Updated] (SPARK-20059) HbaseCredentialProvider uses wrong classloader

[ https://issues.apache.org/jira/browse/SPARK-20059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-20059: Description: {{HBaseCredentialProvider}} uses system classloader instead of child classloader, whi

[jira] [Commented] (SPARK-20059) HbaseCredentialProvider uses wrong classloader

[ https://issues.apache.org/jira/browse/SPARK-20059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15942976#comment-15942976 ] Saisai Shao commented on SPARK-20059: - [~sowen], it probably is not the same issue.

[jira] [Comment Edited] (SPARK-20059) HbaseCredentialProvider uses wrong classloader

[ https://issues.apache.org/jira/browse/SPARK-20059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15942976#comment-15942976 ] Saisai Shao edited comment on SPARK-20059 at 3/27/17 10:02 AM:

[jira] [Commented] (SPARK-19143) API in Spark for distributing new delegation tokens (Improve delegation token handling in secure clusters)

[ https://issues.apache.org/jira/browse/SPARK-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15943243#comment-15943243 ] Saisai Shao commented on SPARK-19143: - Attach a WIP branch (https://github.com/jerry

[jira] [Commented] (SPARK-19143) API in Spark for distributing new delegation tokens (Improve delegation token handling in secure clusters)

[ https://issues.apache.org/jira/browse/SPARK-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15943303#comment-15943303 ] Saisai Shao commented on SPARK-19143: - [~tgraves], can I add you as a *SPIP Shepherd*

[jira] [Commented] (SPARK-19143) API in Spark for distributing new delegation tokens (Improve delegation token handling in secure clusters)

[ https://issues.apache.org/jira/browse/SPARK-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15943289#comment-15943289 ] Saisai Shao commented on SPARK-19143: - Thanks [~tgraves], let me see how to propose a

[jira] [Commented] (SPARK-20079) Re registration of AM hangs spark cluster in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-20079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15944629#comment-15944629 ] Saisai Shao commented on SPARK-20079: - What is the specific symptom you met? I believ

[jira] [Commented] (SPARK-20128) MetricsSystem not always killed in SparkContext.stop()

[ https://issues.apache.org/jira/browse/SPARK-20128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946457#comment-15946457 ] Saisai Shao commented on SPARK-20128: - Here the exception is from MasterSource, which

[jira] [Comment Edited] (SPARK-20128) MetricsSystem not always killed in SparkContext.stop()

[ https://issues.apache.org/jira/browse/SPARK-20128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946457#comment-15946457 ] Saisai Shao edited comment on SPARK-20128 at 3/29/17 3:20 AM: -

[jira] [Commented] (SPARK-20128) MetricsSystem not always killed in SparkContext.stop()

[ https://issues.apache.org/jira/browse/SPARK-20128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946493#comment-15946493 ] Saisai Shao commented on SPARK-20128: - Sorry I cannot access the logs. What I could s

[jira] [Created] (SPARK-20172) Event log without read permission should be filtered out before actually reading it

Saisai Shao created SPARK-20172: --- Summary: Event log without read permission should be filtered out before actually reading it Key: SPARK-20172 URL: https://issues.apache.org/jira/browse/SPARK-20172 Pro

[jira] [Created] (SPARK-20239) Improve HistoryServer ACL mechanism

Saisai Shao created SPARK-20239: --- Summary: Improve HistoryServer ACL mechanism Key: SPARK-20239 URL: https://issues.apache.org/jira/browse/SPARK-20239 Project: Spark Issue Type: Bug C

[jira] [Updated] (SPARK-20239) Improve HistoryServer ACL mechanism

[ https://issues.apache.org/jira/browse/SPARK-20239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-20239: Description: Current SHS (Spark History Server) two different ACLs. * ACL of base URL, it is cont

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15962455#comment-15962455 ] Saisai Shao commented on SPARK-16742: - Hi [~mgummelt], I'm working on the design of S

[jira] [Created] (SPARK-20275) HistoryServer page shows incorrect complete date of inprogress apps

Saisai Shao created SPARK-20275: --- Summary: HistoryServer page shows incorrect complete date of inprogress apps Key: SPARK-20275 URL: https://issues.apache.org/jira/browse/SPARK-20275 Project: Spark

[jira] [Updated] (SPARK-20275) HistoryServer page shows incorrect complete date of inprogress apps

[ https://issues.apache.org/jira/browse/SPARK-20275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-20275: Attachment: screenshot-1.png > HistoryServer page shows incorrect complete date of inprogress apps

[jira] [Updated] (SPARK-20275) HistoryServer page shows incorrect complete date of inprogress apps

[ https://issues.apache.org/jira/browse/SPARK-20275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-20275: Description: The HistoryServer's incomplete page shows in-progress application's completed date as

[jira] [Commented] (SPARK-20244) Incorrect input size in UI with pyspark

[ https://issues.apache.org/jira/browse/SPARK-20244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15964422#comment-15964422 ] Saisai Shao commented on SPARK-20244: - This actually is not a UI problem, it is FileS

[jira] [Commented] (SPARK-20286) dynamicAllocation.executorIdleTimeout is ignored after unpersist

[ https://issues.apache.org/jira/browse/SPARK-20286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15965306#comment-15965306 ] Saisai Shao commented on SPARK-20286: - So the fix is that if there's no RDD get persi

[jira] [Commented] (SPARK-20286) dynamicAllocation.executorIdleTimeout is ignored after unpersist

[ https://issues.apache.org/jira/browse/SPARK-20286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15965411#comment-15965411 ] Saisai Shao commented on SPARK-20286: - bq. Maybe the best approach is to change from

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15968545#comment-15968545 ] Saisai Shao commented on SPARK-16742: - [~mgummelt], do you have a design doc of the k

[jira] [Created] (SPARK-20365) Inaccurate classpath format for AM and Containers

Saisai Shao created SPARK-20365: --- Summary: Inaccurate classpath format for AM and Containers Key: SPARK-20365 URL: https://issues.apache.org/jira/browse/SPARK-20365 Project: Spark Issue Type: B

[jira] [Updated] (SPARK-20365) Not so accurate classpath format for AM and Containers

[ https://issues.apache.org/jira/browse/SPARK-20365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-20365: Summary: Not so accurate classpath format for AM and Containers (was: Inaccurate classpath format

[jira] [Created] (SPARK-20391) Properly rename the memory related fields in ExecutorSummary REST API

Saisai Shao created SPARK-20391: --- Summary: Properly rename the memory related fields in ExecutorSummary REST API Key: SPARK-20391 URL: https://issues.apache.org/jira/browse/SPARK-20391 Project: Spark

[jira] [Commented] (SPARK-20391) Properly rename the memory related fields in ExecutorSummary REST API

[ https://issues.apache.org/jira/browse/SPARK-20391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15974685#comment-15974685 ] Saisai Shao commented on SPARK-20391: - [~irashid], would be grateful to hear your sug

[jira] [Commented] (SPARK-20391) Properly rename the memory related fields in ExecutorSummary REST API

[ https://issues.apache.org/jira/browse/SPARK-20391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15976145#comment-15976145 ] Saisai Shao commented on SPARK-20391: - Thanks [~irashid] [~tgraves] for your comments

[jira] [Commented] (SPARK-21618) http(s) not accepted in spark-submit jar uri

[ https://issues.apache.org/jira/browse/SPARK-21618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16114291#comment-16114291 ] Saisai Shao commented on SPARK-21618: - Hi Steve, I'm not quite following your comment

[jira] [Commented] (SPARK-21660) Yarn ShuffleService failed to start when the chosen directory become read-only

[ https://issues.apache.org/jira/browse/SPARK-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16121125#comment-16121125 ] Saisai Shao commented on SPARK-21660: - Will yarn NM handle this bad disk problem and

[jira] [Commented] (SPARK-21714) SparkSubmit in Yarn Client mode downloads remote files and then reuploads them again

[ https://issues.apache.org/jira/browse/SPARK-21714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16125115#comment-16125115 ] Saisai Shao commented on SPARK-21714: - I noticed this issue before and tried to fix i

[jira] [Commented] (SPARK-9104) expose network layer memory usage in shuffle part

[ https://issues.apache.org/jira/browse/SPARK-9104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16125331#comment-16125331 ] Saisai Shao commented on SPARK-9104: I think it is quite useful to get the memory deta

[jira] [Commented] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16126894#comment-16126894 ] Saisai Shao commented on SPARK-21733: - Is there any issue here? > ERROR executor.Coa

[jira] [Commented] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16126974#comment-16126974 ] Saisai Shao commented on SPARK-21733: - Should the resolution of this JIRA be "invalid

[jira] [Commented] (SPARK-21714) SparkSubmit in Yarn Client mode downloads remote files and then reuploads them again

[ https://issues.apache.org/jira/browse/SPARK-21714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16127199#comment-16127199 ] Saisai Shao commented on SPARK-21714: - Let me take a crack on this if no one is worki

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16130078#comment-16130078 ] Saisai Shao commented on SPARK-21752: - For configurations like "spark.jars.packages"

[jira] [Commented] (SPARK-21798) No config to replace deprecated SPARK_CLASSPATH config for launching daemons like History Server

[ https://issues.apache.org/jira/browse/SPARK-21798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136196#comment-16136196 ] Saisai Shao commented on SPARK-21798: - I think this one could be used, looks like the

[jira] [Commented] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136326#comment-16136326 ] Saisai Shao commented on SPARK-21733: - This is because executor is killed by NM with

[jira] [Comment Edited] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136326#comment-16136326 ] Saisai Shao edited comment on SPARK-21733 at 8/22/17 5:41 AM: -

<    6   7   8   9   10   11   12   13   >