[jira] [Updated] (LIVY-408) Upgrade Netty version to avoid some security issues

[ https://issues.apache.org/jira/browse/LIVY-408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-408: - Affects Version/s: 0.5.0 0.4.1 > Upgrade Netty version to avoid some security issues

[jira] [Updated] (LIVY-408) Upgrade Netty version to avoid some security issues

[ https://issues.apache.org/jira/browse/LIVY-408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-408: - Fix Version/s: (was: 0.5.0) > Upgrade Netty version to avoid some security issues >

[jira] [Updated] (LIVY-408) Upgrade Netty version to avoid some security issues

[ https://issues.apache.org/jira/browse/LIVY-408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-408: - Priority: Minor (was: Major) > Upgrade Netty version to avoid some security issues > --

[jira] [Created] (LIVY-408) Upgrade Netty version to avoid some security issues

Saisai Shao created LIVY-408: Summary: Upgrade Netty version to avoid some security issues Key: LIVY-408 URL: https://issues.apache.org/jira/browse/LIVY-408 Project: Livy Issue Type: Improvement

[jira] [Commented] (SPARK-22151) PYTHONPATH not picked up from the spark.yarn.appMasterEnv properly

[ https://issues.apache.org/jira/browse/SPARK-22151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183581#comment-16183581 ] Saisai Shao commented on SPARK-22151: - Checking the Yarn client code, looks like ther

[jira] [Commented] (SPARK-21737) Create communication channel between arbitrary clients and the Spark AM in YARN mode

[ https://issues.apache.org/jira/browse/SPARK-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183563#comment-16183563 ] Saisai Shao commented on SPARK-21737: - [~yoonlee95], are you still working on this th

[jira] [Updated] (SPARK-22123) Add latest failure reason for task set blacklist

[ https://issues.apache.org/jira/browse/SPARK-22123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-22123: Priority: Minor (was: Major) > Add latest failure reason for task set blacklist >

[jira] [Assigned] (SPARK-22123) Add latest failure reason for task set blacklist

[ https://issues.apache.org/jira/browse/SPARK-22123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-22123: --- Assignee: zhoukang > Add latest failure reason for task set blacklist >

[jira] [Resolved] (SPARK-22123) Add latest failure reason for task set blacklist

[ https://issues.apache.org/jira/browse/SPARK-22123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-22123. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19338 [https://githu

[jira] [Resolved] (SPARK-20785) Spark should provide jump links and add (count) in the SQL web ui.

[ https://issues.apache.org/jira/browse/SPARK-20785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-20785. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19346 [https://githu

[jira] [Assigned] (SPARK-20785) Spark should provide jump links and add (count) in the SQL web ui.

[ https://issues.apache.org/jira/browse/SPARK-20785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-20785: --- Assignee: guoxiaolongzte > Spark should provide jump links and add (count) in the SQL web u

[jira] [Commented] (SPARK-22074) Task killed by other attempt task should not be resubmitted

[ https://issues.apache.org/jira/browse/SPARK-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16182062#comment-16182062 ] Saisai Shao commented on SPARK-22074: - So if I understand correctly, this happens whe

[jira] [Commented] (SPARK-22074) Task killed by other attempt task should not be resubmitted

[ https://issues.apache.org/jira/browse/SPARK-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16181924#comment-16181924 ] Saisai Shao commented on SPARK-22074: - Hey [~XuanYuan], I'm a little confused why the

[jira] [Commented] (SPARK-9103) Tracking spark's memory usage

[ https://issues.apache.org/jira/browse/SPARK-9103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16181905#comment-16181905 ] Saisai Shao commented on SPARK-9103: Hi [~irashid], thanks a lot for your response. I

[jira] [Commented] (SPARK-22074) Task killed by other attempt task should not be resubmitted

[ https://issues.apache.org/jira/browse/SPARK-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16180506#comment-16180506 ] Saisai Shao commented on SPARK-22074: - Hi [~XuanYuan], can you please help me to unde

[jira] [Resolved] (LIVY-406) Livy fail to get app id and related information when using yarn client mode

[ https://issues.apache.org/jira/browse/LIVY-406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved LIVY-406. -- Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 0.5.0 0.4.1 > Li

[jira] [Closed] (LIVY-403) Add getCompletion for Interpreter

[ https://issues.apache.org/jira/browse/LIVY-403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao closed LIVY-403. Resolution: Duplicate > Add getCompletion for Interpreter > - > >

[jira] [Commented] (SPARK-9103) Tracking spark's memory usage

[ https://issues.apache.org/jira/browse/SPARK-9103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16176073#comment-16176073 ] Saisai Shao commented on SPARK-9103: [~irashid] I would like to hear your suggestion o

[jira] [Updated] (LIVY-398) Change docs to describe how to use shared language session

[ https://issues.apache.org/jira/browse/LIVY-398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-398: - Component/s: (was: REPL) Docs > Change docs to describe how to use shared language sess

[jira] [Resolved] (LIVY-398) Change docs to describe how to use shared language session

[ https://issues.apache.org/jira/browse/LIVY-398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved LIVY-398. -- Resolution: Fixed Fix Version/s: 0.5.0 > Change docs to describe how to use shared language session

[jira] [Assigned] (LIVY-398) Change docs to describe how to use shared language session

[ https://issues.apache.org/jira/browse/LIVY-398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-398: Assignee: Saisai Shao > Change docs to describe how to use shared language session >

[jira] [Created] (LIVY-406) Livy fail to get app id and related information when using yarn client mode

Saisai Shao created LIVY-406: Summary: Livy fail to get app id and related information when using yarn client mode Key: LIVY-406 URL: https://issues.apache.org/jira/browse/LIVY-406 Project: Livy

[jira] [Assigned] (SPARK-21934) Expose Netty memory usage via Metrics System

[ https://issues.apache.org/jira/browse/SPARK-21934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-21934: --- Assignee: Saisai Shao > Expose Netty memory usage via Metrics System > -

[jira] [Resolved] (SPARK-21934) Expose Netty memory usage via Metrics System

[ https://issues.apache.org/jira/browse/SPARK-21934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-21934. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19160 [https://githu

[jira] [Assigned] (LIVY-399) Enable real test for PySpark and SparkR interpreters

[ https://issues.apache.org/jira/browse/LIVY-399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-399: Assignee: Saisai Shao > Enable real test for PySpark and SparkR interpreters > --

[jira] [Updated] (LIVY-386) Refactor Livy core module with Java

[ https://issues.apache.org/jira/browse/LIVY-386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-386: - Description: Livy core module is used by several modules like server, repl, so it requires to support differ

[jira] [Updated] (LIVY-386) Refactor Livy core module with Java

[ https://issues.apache.org/jira/browse/LIVY-386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-386: - Summary: Refactor Livy core module with Java (was: Refactor Livy code module with Java) > Refactor Livy co

[jira] [Assigned] (LIVY-386) Refactor Livy core module with Java

[ https://issues.apache.org/jira/browse/LIVY-386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-386: Assignee: Saisai Shao > Refactor Livy core module with Java > --- > >

[jira] [Updated] (LIVY-405) ERROR RSCClient: Failed to connect to context

[ https://issues.apache.org/jira/browse/LIVY-405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-405: - Issue Type: Question (was: Bug) > ERROR RSCClient: Failed to connect to context > -

[jira] [Commented] (LIVY-405) ERROR RSCClient: Failed to connect to context

[ https://issues.apache.org/jira/browse/LIVY-405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16172600#comment-16172600 ] Saisai Shao commented on LIVY-405: -- It's too vague to understand your issue here, the excep

[jira] [Updated] (LIVY-405) ERROR RSCClient: Failed to connect to context

[ https://issues.apache.org/jira/browse/LIVY-405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-405: - Priority: Minor (was: Blocker) > ERROR RSCClient: Failed to connect to context > --

[jira] [Assigned] (SPARK-22030) GraphiteSink fails to re-connect to Graphite instances behind an ELB or any other auto-scaled LB

[ https://issues.apache.org/jira/browse/SPARK-22030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-22030: --- Assignee: Alex Mikhailau > GraphiteSink fails to re-connect to Graphite instances behind an

[jira] [Updated] (SPARK-22030) GraphiteSink fails to re-connect to Graphite instances behind an ELB or any other auto-scaled LB

[ https://issues.apache.org/jira/browse/SPARK-22030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-22030: Priority: Minor (was: Critical) > GraphiteSink fails to re-connect to Graphite instances behind an

[jira] [Resolved] (SPARK-22030) GraphiteSink fails to re-connect to Graphite instances behind an ELB or any other auto-scaled LB

[ https://issues.apache.org/jira/browse/SPARK-22030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-22030. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19210 [https://githu

[jira] [Assigned] (LIVY-396) Livy does not map YARN app states correctly

[ https://issues.apache.org/jira/browse/LIVY-396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-396: Assignee: Meisam > Livy does not map YARN app states correctly >

[jira] [Assigned] (LIVY-396) Livy does not map YARN app states correctly

[ https://issues.apache.org/jira/browse/LIVY-396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-396: Assignee: (was: Meisam Fathi) > Livy does not map YARN app states correctly > ---

[jira] [Assigned] (LIVY-396) Livy does not map YARN app states correctly

[ https://issues.apache.org/jira/browse/LIVY-396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-396: Assignee: Meisam Fathi > Livy does not map YARN app states correctly > --

[jira] [Resolved] (LIVY-396) Livy does not map YARN app states correctly

[ https://issues.apache.org/jira/browse/LIVY-396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved LIVY-396. -- Resolution: Fixed Issue resolved by pull request 39 [https://github.com/apache/incubator-livy/pull/39] >

[jira] [Commented] (LIVY-403) Add getCompletion for Interpreter

[ https://issues.apache.org/jira/browse/LIVY-403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167489#comment-16167489 ] Saisai Shao commented on LIVY-403: -- [~ajbozarth], the purpose of this JIRA is to support co

[jira] [Updated] (LIVY-403) Add getCompletion for Interpreter

[ https://issues.apache.org/jira/browse/LIVY-403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-403: - Affects Version/s: 0.5.0 > Add getCompletion for Interpreter > - > >

[jira] [Updated] (LIVY-403) Add getCompletion for Interpreter

[ https://issues.apache.org/jira/browse/LIVY-403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-403: - Component/s: REPL > Add getCompletion for Interpreter > - > >

[jira] [Updated] (SPARK-21902) BlockManager.doPut will hide actually exception when exception thrown in finally block

[ https://issues.apache.org/jira/browse/SPARK-21902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-21902: Priority: Trivial (was: Major) > BlockManager.doPut will hide actually exception when exception th

[jira] [Updated] (SPARK-21902) BlockManager.doPut will hide actually exception when exception thrown in finally block

[ https://issues.apache.org/jira/browse/SPARK-21902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-21902: Issue Type: Improvement (was: Wish) > BlockManager.doPut will hide actually exception when excepti

[jira] [Assigned] (SPARK-21902) BlockManager.doPut will hide actually exception when exception thrown in finally block

[ https://issues.apache.org/jira/browse/SPARK-21902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-21902: --- Assignee: zhoukang > BlockManager.doPut will hide actually exception when exception thrown i

[jira] [Resolved] (SPARK-21902) BlockManager.doPut will hide actually exception when exception thrown in finally block

[ https://issues.apache.org/jira/browse/SPARK-21902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-21902. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19171 [https://githu

[jira] [Resolved] (SPARK-21922) When executor failed and task metrics have not send to driver,the status will always be 'RUNNING' and the duration will be 'CurrentTime - launchTime'

[ https://issues.apache.org/jira/browse/SPARK-21922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-21922. - Resolution: Fixed Assignee: zhoukang Target Version/s: 2.3.0 > When executo

[jira] [Updated] (SPARK-21922) When executor failed and task metrics have not send to driver,the status will always be 'RUNNING' and the duration will be 'CurrentTime - launchTime'

[ https://issues.apache.org/jira/browse/SPARK-21922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-21922: Fix Version/s: 2.3.0 > When executor failed and task metrics have not send to driver,the status wil

[jira] [Updated] (SPARK-21922) When executor failed and task metrics have not send to driver,the status will always be 'RUNNING' and the duration will be 'CurrentTime - launchTime'

[ https://issues.apache.org/jira/browse/SPARK-21922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-21922: Target Version/s: (was: 2.3.0) > When executor failed and task metrics have not send to driver,th

[jira] [Commented] (SPARK-21513) SQL to_json should support all column types

[ https://issues.apache.org/jira/browse/SPARK-21513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164000#comment-16164000 ] Saisai Shao commented on SPARK-21513: - [~goldmedal] is it your correct JIRA name? [~

[jira] [Assigned] (SPARK-21513) SQL to_json should support all column types

[ https://issues.apache.org/jira/browse/SPARK-21513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-21513: --- Assignee: Jia-Xuan Liu > SQL to_json should support all column types > -

[jira] [Commented] (SPARK-21943) When I use rest Api (/ applications / [app-id] / jobs / [job-id]) to view some of the jobs that are running jobs, the returned json information is missing the “descrip

[ https://issues.apache.org/jira/browse/SPARK-21943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16161008#comment-16161008 ] Saisai Shao commented on SPARK-21943: - If you're trying to report bugs, I think you s

[jira] [Commented] (SPARK-21943) When I use rest Api (/ applications / [app-id] / jobs / [job-id]) to view some of the jobs that are running jobs, the returned json information is missing the “descrip

[ https://issues.apache.org/jira/browse/SPARK-21943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160814#comment-16160814 ] Saisai Shao commented on SPARK-21943: - >From the code, it says that job description i

[jira] [Closed] (SPARK-21906) No need to runAsSparkUser to switch UserGroupInformation in YARN mode

[ https://issues.apache.org/jira/browse/SPARK-21906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao closed SPARK-21906. --- Resolution: Not A Problem > No need to runAsSparkUser to switch UserGroupInformation in YARN mode > -

[jira] [Assigned] (SPARK-20098) DataType's typeName method returns with 'StructF' in case of StructField

[ https://issues.apache.org/jira/browse/SPARK-20098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-20098: --- Assignee: Peter Szalai > DataType's typeName method returns with 'StructF' in case of Struct

[jira] [Commented] (SPARK-21942) DiskBlockManager crashing when a root local folder has been externally deleted by OS

[ https://issues.apache.org/jira/browse/SPARK-21942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158320#comment-16158320 ] Saisai Shao commented on SPARK-21942: - Personally I would like to fail fast if such t

[jira] [Commented] (SPARK-21942) DiskBlockManager crashing when a root local folder has been externally deleted by OS

[ https://issues.apache.org/jira/browse/SPARK-21942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158271#comment-16158271 ] Saisai Shao commented on SPARK-21942: - {quote} https://github.com/search?utf8=%E2%9C%

[jira] [Assigned] (SPARK-21939) Use TimeLimits instead of Timeouts

[ https://issues.apache.org/jira/browse/SPARK-21939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-21939: --- Assignee: Dongjoon Hyun > Use TimeLimits instead of Timeouts > -

[jira] [Resolved] (SPARK-21939) Use TimeLimits instead of Timeouts

[ https://issues.apache.org/jira/browse/SPARK-21939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-21939. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19150 [https://githu

[jira] [Commented] (SPARK-21942) ix DiskBlockManager crashing when a root local folder has been externally deleted by OS

[ https://issues.apache.org/jira/browse/SPARK-21942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156914#comment-16156914 ] Saisai Shao commented on SPARK-21942: - I think in a production environment, user shou

[jira] [Updated] (SPARK-9104) expose network layer memory usage

[ https://issues.apache.org/jira/browse/SPARK-9104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-9104: --- Summary: expose network layer memory usage (was: expose network layer memory usage in shuffle part)

[jira] [Updated] (SPARK-21934) Expose Netty memory usage via Metrics System

[ https://issues.apache.org/jira/browse/SPARK-21934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-21934: Issue Type: Sub-task (was: New Feature) Parent: SPARK-9103 > Expose Netty memory usage via

[jira] [Created] (SPARK-21934) Expose Netty memory usage via Metrics System

Saisai Shao created SPARK-21934: --- Summary: Expose Netty memory usage via Metrics System Key: SPARK-21934 URL: https://issues.apache.org/jira/browse/SPARK-21934 Project: Spark Issue Type: New Fe

[jira] [Commented] (SPARK-21917) Remote http(s) resources is not supported in YARN mode

[ https://issues.apache.org/jira/browse/SPARK-21917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16154744#comment-16154744 ] Saisai Shao commented on SPARK-21917: - Thanks [~vanzin], I think your suggestion is g

[jira] [Assigned] (SPARK-18061) Spark Thriftserver needs to create SPNego principal

[ https://issues.apache.org/jira/browse/SPARK-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-18061: --- Assignee: Saisai Shao > Spark Thriftserver needs to create SPNego principal > --

[jira] [Resolved] (SPARK-18061) Spark Thriftserver needs to create SPNego principal

[ https://issues.apache.org/jira/browse/SPARK-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-18061. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18628 [https://githu

[jira] [Comment Edited] (SPARK-21917) Remote http(s) resources is not supported in YARN mode

[ https://issues.apache.org/jira/browse/SPARK-21917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16153141#comment-16153141 ] Saisai Shao edited comment on SPARK-21917 at 9/5/17 8:36 AM: -

[jira] [Commented] (SPARK-21917) Remote http(s) resources is not supported in YARN mode

[ https://issues.apache.org/jira/browse/SPARK-21917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16153141#comment-16153141 ] Saisai Shao commented on SPARK-21917: - I'm inclining to choose option 1, the only ove

[jira] [Created] (SPARK-21917) Remote http(s) resources is not supported in YARN mode

Saisai Shao created SPARK-21917: --- Summary: Remote http(s) resources is not supported in YARN mode Key: SPARK-21917 URL: https://issues.apache.org/jira/browse/SPARK-21917 Project: Spark Issue Ty

[jira] [Commented] (SPARK-21888) Cannot add stuff to Client Classpath for Yarn Cluster Mode

[ https://issues.apache.org/jira/browse/SPARK-21888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150258#comment-16150258 ] Saisai Shao commented on SPARK-21888: - Jars added by "--jars" will be added to client

[jira] [Commented] (SPARK-21882) OutputMetrics doesn't count written bytes correctly in the saveAsHadoopDataset function

[ https://issues.apache.org/jira/browse/SPARK-21882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150260#comment-16150260 ] Saisai Shao commented on SPARK-21882: - Please submit the patch to Github Apache Spark

[jira] [Assigned] (SPARK-21658) Adds the default None for value in na.replace in PySpark to match

[ https://issues.apache.org/jira/browse/SPARK-21658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-21658: --- Assignee: Chin Han Yu > Adds the default None for value in na.replace in PySpark to match >

[jira] [Commented] (SPARK-21658) Adds the default None for value in na.replace in PySpark to match

[ https://issues.apache.org/jira/browse/SPARK-21658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150059#comment-16150059 ] Saisai Shao commented on SPARK-21658: - Done :). > Adds the default None for value in

[jira] [Commented] (SPARK-11574) Spark should support StatsD sink out of box

[ https://issues.apache.org/jira/browse/SPARK-11574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16148713#comment-16148713 ] Saisai Shao commented on SPARK-11574: - Thanks a lot [~srowen]! > Spark should suppor

[jira] [Commented] (SPARK-11574) Spark should support StatsD sink out of box

[ https://issues.apache.org/jira/browse/SPARK-11574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16148701#comment-16148701 ] Saisai Shao commented on SPARK-11574: - Ping [~srowen]. Hi Sean I cannot assign this J

[jira] [Commented] (SPARK-11574) Spark should support StatsD sink out of box

[ https://issues.apache.org/jira/browse/SPARK-11574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16148333#comment-16148333 ] Saisai Shao commented on SPARK-11574: - Maybe it is my permission issue, will ask PMC

[jira] [Assigned] (SPARK-17321) YARN shuffle service should use good disk from yarn.nodemanager.local-dirs

[ https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-17321: --- Assignee: Saisai Shao > YARN shuffle service should use good disk from yarn.nodemanager.loca

[jira] [Resolved] (SPARK-17321) YARN shuffle service should use good disk from yarn.nodemanager.local-dirs

[ https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-17321. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19032 [https://githu

[jira] [Commented] (SPARK-11574) Spark should support StatsD sink out of box

[ https://issues.apache.org/jira/browse/SPARK-11574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16148292#comment-16148292 ] Saisai Shao commented on SPARK-11574: - Hi Xiaofeng, is your JIRA username still avail

[jira] [Resolved] (SPARK-11574) Spark should support StatsD sink out of box

[ https://issues.apache.org/jira/browse/SPARK-11574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-11574. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 9518 [https://github

[jira] [Commented] (SPARK-21829) Enable config to permanently blacklist a list of nodes

[ https://issues.apache.org/jira/browse/SPARK-21829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144889#comment-16144889 ] Saisai Shao commented on SPARK-21829: - I understand you concern, since it is much eas

[jira] [Commented] (SPARK-21829) Enable config to permanently blacklist a list of nodes

[ https://issues.apache.org/jira/browse/SPARK-21829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16141081#comment-16141081 ] Saisai Shao commented on SPARK-21829: - Cross post the comment here. Since you're runn

[jira] [Closed] (SPARK-21819) UserGroupInformation initialization in SparkHadoopUtilwill overwrite user config

[ https://issues.apache.org/jira/browse/SPARK-21819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao closed SPARK-21819. --- Resolution: Not A Problem > UserGroupInformation initialization in SparkHadoopUtilwill overwrite use

[jira] [Commented] (SPARK-21819) UserGroupInformation initialization in SparkHadoopUtilwill overwrite user config

[ https://issues.apache.org/jira/browse/SPARK-21819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16139590#comment-16139590 ] Saisai Shao commented on SPARK-21819: - Then I think there should no issue in Spark, r

[jira] [Commented] (SPARK-17321) YARN shuffle service should use good disk from yarn.nodemanager.local-dirs

[ https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16139502#comment-16139502 ] Saisai Shao commented on SPARK-17321: - 1. if NM recovery is enabled, then yarn will p

[jira] [Commented] (SPARK-17321) YARN shuffle service should use good disk from yarn.nodemanager.local-dirs

[ https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138320#comment-16138320 ] Saisai Shao commented on SPARK-17321: - We're facing the same issue. I think YARN shuf

[jira] [Commented] (SPARK-21819) UserGroupInformation initialization in SparkHadoopUtilwill overwrite user config

[ https://issues.apache.org/jira/browse/SPARK-21819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138308#comment-16138308 ] Saisai Shao commented on SPARK-21819: - I'm not sure if Spark expose the user API to s

[jira] [Commented] (SPARK-21819) UserGroupInformation initialization in SparkHadoopUtilwill overwrite user config

[ https://issues.apache.org/jira/browse/SPARK-21819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138300#comment-16138300 ] Saisai Shao commented on SPARK-21819: - I think here because `Configuration` object cr

[jira] [Commented] (SPARK-21660) Yarn ShuffleService failed to start when the chosen directory become read-only

[ https://issues.apache.org/jira/browse/SPARK-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16137842#comment-16137842 ] Saisai Shao commented on SPARK-21660: - Yes [~hyukjin.kwon] this is a dup JIRA. > Yar

[jira] [Commented] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136454#comment-16136454 ] Saisai Shao commented on SPARK-21733: - [~1028344...@qq.com], I'm really not sure what

[jira] [Resolved] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-21733. - Resolution: Not A Problem > ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM > -

[jira] [Commented] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136334#comment-16136334 ] Saisai Shao commented on SPARK-21733: - I'm going to close this issue again because th

[jira] [Comment Edited] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136326#comment-16136326 ] Saisai Shao edited comment on SPARK-21733 at 8/22/17 5:41 AM: -

[jira] [Commented] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136326#comment-16136326 ] Saisai Shao commented on SPARK-21733: - This is because executor is killed by NM with

[jira] [Commented] (SPARK-21798) No config to replace deprecated SPARK_CLASSPATH config for launching daemons like History Server

[ https://issues.apache.org/jira/browse/SPARK-21798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136196#comment-16136196 ] Saisai Shao commented on SPARK-21798: - I think this one could be used, looks like the

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16130078#comment-16130078 ] Saisai Shao commented on SPARK-21752: - For configurations like "spark.jars.packages"

[jira] [Commented] (SPARK-21714) SparkSubmit in Yarn Client mode downloads remote files and then reuploads them again

[ https://issues.apache.org/jira/browse/SPARK-21714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16127199#comment-16127199 ] Saisai Shao commented on SPARK-21714: - Let me take a crack on this if no one is worki

[jira] [Commented] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16126974#comment-16126974 ] Saisai Shao commented on SPARK-21733: - Should the resolution of this JIRA be "invalid

[jira] [Commented] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16126894#comment-16126894 ] Saisai Shao commented on SPARK-21733: - Is there any issue here? > ERROR executor.Coa

[jira] [Commented] (SPARK-9104) expose network layer memory usage in shuffle part

[ https://issues.apache.org/jira/browse/SPARK-9104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16125331#comment-16125331 ] Saisai Shao commented on SPARK-9104: I think it is quite useful to get the memory deta

[jira] [Commented] (SPARK-21714) SparkSubmit in Yarn Client mode downloads remote files and then reuploads them again

[ https://issues.apache.org/jira/browse/SPARK-21714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16125115#comment-16125115 ] Saisai Shao commented on SPARK-21714: - I noticed this issue before and tried to fix i

<    2   3   4   5   6   7   8   9   10   11   >