[jira] [Commented] (SPARK-22357) SparkContext.binaryFiles ignore minPartitions parameter

2017-10-26 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16220160#comment-16220160 ] Saisai Shao commented on SPARK-22357: - [~WeichenXu123] would you please format

[jira] [Assigned] (SPARK-21840) Allow multiple SparkSubmit invocations in same JVM without polluting system properties

2017-10-26 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-21840: --- Assignee: Marcelo Vanzin > Allow multiple SparkSubmit invocations in same JVM with

[jira] [Resolved] (SPARK-21840) Allow multiple SparkSubmit invocations in same JVM without polluting system properties

2017-10-26 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-21840. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19519 [https

Re: Livy with DC/OS Mesos

2017-10-25 Thread Saisai Shao
Unfortunately Mesos cluster manager is not supported currently for Livy, we only tested on local and yarn mode. On Wed, Oct 25, 2017 at 8:47 PM, Junaid Nasir wrote: > livy version 0.4 > trying to create interactive session. may be it's not supported with mesos? > > > > On Wed, Oct 25, 2017 3:55

[jira] [Updated] (SPARK-22319) SparkSubmit calls getFileStatus before calling loginUserFromKeytab

2017-10-22 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-22319: Fix Version/s: 2.2.1 > SparkSubmit calls getFileStatus before calling loginUserFromKey

[jira] [Closed] (SPARK-22272) killing task may cause the executor progress hang because of the JVM bug

2017-10-22 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao closed SPARK-22272. --- > killing task may cause the executor progress hang because of the JVM

[jira] [Resolved] (SPARK-22272) killing task may cause the executor progress hang because of the JVM bug

2017-10-22 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-22272. - Resolution: Won't Fix > killing task may cause the executor progress hang because of the

[jira] [Commented] (SPARK-22272) killing task may cause the executor progress hang because of the JVM bug

2017-10-22 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16214569#comment-16214569 ] Saisai Shao commented on SPARK-22272: - Setting "spark.file.transferTo&q

[jira] [Assigned] (SPARK-22319) SparkSubmit calls getFileStatus before calling loginUserFromKeytab

2017-10-22 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-22319: --- Assignee: Steven Rand > SparkSubmit calls getFileStatus before calling loginUserFromKey

[jira] [Resolved] (SPARK-22319) SparkSubmit calls getFileStatus before calling loginUserFromKeytab

2017-10-22 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-22319. - Resolution: Fixed Fix Version/s: 2.3.0 > SparkSubmit calls getFileStatus before call

[jira] [Updated] (SPARK-22319) SparkSubmit calls getFileStatus before calling loginUserFromKeytab

2017-10-22 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-22319: Affects Version/s: 2.2.0 > SparkSubmit calls getFileStatus before calling loginUserFromKey

[jira] [Assigned] (LIVY-245) Sharing variables across Scala Jobs in RSC.

2017-10-20 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-245: Assignee: Saisai Shao (was: Alex Man) > Sharing variables across Scala Jobs in

[jira] [Resolved] (LIVY-245) Sharing variables across Scala Jobs in RSC.

2017-10-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved LIVY-245. -- Resolution: Fixed Fix Version/s: 0.5.0 Issue resolved by pull request 19 [https://github.com

[jira] [Resolved] (LIVY-411) Session cannot be started when Python or R package is missing

2017-10-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved LIVY-411. -- Resolution: Fixed Fix Version/s: 0.5.0 Issue resolved by pull request 56 [https://github.com

[jira] [Commented] (SPARK-22312) Spark job stuck with no executor due to bug in Executor Allocation Manager

2017-10-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212071#comment-16212071 ] Saisai Shao commented on SPARK-22312: - I think it is duplicated to SPARK-1

[jira] [Assigned] (LIVY-409) Improve User Experience in livy-shell

2017-10-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-409: Assignee: Eric Perry > Improve User Experience in livy-sh

[jira] [Resolved] (LIVY-409) Improve User Experience in livy-shell

2017-10-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved LIVY-409. -- Resolution: Fixed Fix Version/s: 0.5.0 Issue resolved by pull request 55 [https://github.com

[jira] [Resolved] (SPARK-22290) Starting second context in same JVM fails to get new Hive delegation token

2017-10-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-22290. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19509 [https

[jira] [Assigned] (SPARK-22290) Starting second context in same JVM fails to get new Hive delegation token

2017-10-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-22290: --- Assignee: Marcelo Vanzin > Starting second context in same JVM fails to get new H

[jira] [Commented] (AMBARI-22260) Update Spark2 log4j default settings to latest

2017-10-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/AMBARI-22260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16210628#comment-16210628 ] Saisai Shao commented on AMBARI-22260: -- Please help to review [~sumitmoh

Review Request 63138: Update Spark2 log4j default settings to latest

2017-10-18 Thread Saisai Shao
/diff/1/ Testing --- Manually verification on refresh install of HDP 2.6 and upgrade from HDP 2.5. Thanks, Saisai Shao

[jira] [Created] (AMBARI-22260) Update Spark2 log4j default settings to latest

2017-10-18 Thread Saisai Shao (JIRA)
Saisai Shao created AMBARI-22260: Summary: Update Spark2 log4j default settings to latest Key: AMBARI-22260 URL: https://issues.apache.org/jira/browse/AMBARI-22260 Project: Ambari Issue Type

[jira] [Updated] (LIVY-411) Session cannot be started when Python or R package is missing

2017-10-17 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-411: - Reporter: Yesha Vora (was: Saisai Shao) > Session cannot be started when Python or R package is miss

[jira] [Updated] (LIVY-411) Session cannot be started when Python or R package is missing

2017-10-17 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-411: - Description: In Livy 0.5.0, we supported multiple languages in one session, but it requires that the

[jira] [Created] (LIVY-411) Session cannot be started when Python or R package is missing

2017-10-17 Thread Saisai Shao (JIRA)
Saisai Shao created LIVY-411: Summary: Session cannot be started when Python or R package is missing Key: LIVY-411 URL: https://issues.apache.org/jira/browse/LIVY-411 Project: Livy Issue Type

Re: NoSuchFileException: spark-internal - when creating interactive session on remote spark

2017-10-15 Thread Saisai Shao
Would you please provide more information about how you create a Livy session? As for now, Livy only supports spark on yarn and local mode officially, we don't test on standalone cluster mode, so maybe there's some issues in it. On Mon, Oct 16, 2017 at 4:29 AM, Junaid Nasir wrote: > Hi everyone,

[jira] [Commented] (SPARK-22242) streaming job failed to restart from checkpoint

2017-10-14 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16205018#comment-16205018 ] Saisai Shao commented on SPARK-22242: - It is not resolved, but it is the same

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2017-10-12 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16202900#comment-16202900 ] Saisai Shao commented on SPARK-9: - {quote} I don't think tha

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2017-10-12 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16201950#comment-16201950 ] Saisai Shao commented on SPARK-9: - My concern is about how to maintain this

[jira] [Commented] (SPARK-22062) BlockManager does not account for memory consumed by remote fetches

2017-10-11 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16201305#comment-16201305 ] Saisai Shao commented on SPARK-22062: - Yes, there potentially has OOM problem, b

[jira] [Commented] (SPARK-22243) streaming job failed to restart from checkpoint

2017-10-11 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16200372#comment-16200372 ] Saisai Shao commented on SPARK-22243: - Yes, it is a related issue regarding to S

[jira] [Commented] (SPARK-21737) Create communication channel between arbitrary clients and the Spark AM in YARN mode

2017-10-10 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198674#comment-16198674 ] Saisai Shao commented on SPARK-21737: - I was trying to understand how S

[jira] [Assigned] (LIVY-7) Add autocompletion API

2017-10-10 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-7?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-7: -- Assignee: Pascal Pellmont > Add autocompletion API > -- > >

[jira] [Commented] (SPARK-22199) Spark Job on YARN fails with executors "Slave registration failed"

2017-10-09 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198258#comment-16198258 ] Saisai Shao commented on SPARK-22199: - Can you please list the steps to repro

[jira] [Commented] (SPARK-21737) Create communication channel between arbitrary clients and the Spark AM in YARN mode

2017-10-09 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198204#comment-16198204 ] Saisai Shao commented on SPARK-21737: - Hi [~tgraves], I'm trying to under

[jira] [Resolved] (LIVY-7) Add autocompletion API

2017-10-09 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-7?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved LIVY-7. Resolution: Fixed Fix Version/s: 0.5.0 Issue resolved by pull request 51 [https://github.com/apache

Re: Spark cassandra connector with livy

2017-10-09 Thread Saisai Shao
Please set "spark.jars.packages" to the package you wanted in batch POST protocol "conf" field. Thanks Jerry On Tue, Oct 10, 2017 at 3:05 AM, Junaid Nasir wrote: > more info regarding the problem. > when i pass {"kind": "pyspark","jars":["datastax: > spark-cassandra-connector:2.0.1-s_2.11"]} vi

[jira] [Assigned] (SPARK-22074) Task killed by other attempt task should not be resubmitted

2017-10-08 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-22074: --- Assignee: Li Yuanjian > Task killed by other attempt task should not be resubmit

[jira] [Resolved] (SPARK-22074) Task killed by other attempt task should not be resubmitted

2017-10-08 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-22074. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19287 [https

Re: Podling Report Reminder - October 2017

2017-10-03 Thread Saisai Shao
Hi Team, Here is the October's podling report, please review and sign-off. https://wiki.apache.org/incubator/October2017 Thanks Jerry On Sun, Oct 1, 2017 at 9:06 PM, wrote: > Dear podling, > > This email was sent by an automated system on behalf of the Apache > Incubator PMC. It is an initia

[jira] [Resolved] (LIVY-408) Upgrade Netty version to avoid some security issues

2017-09-28 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved LIVY-408. -- Resolution: Fixed Fix Version/s: 0.5.0 Issue resolved by pull request 53 [https://github.com

[jira] [Assigned] (SPARK-22135) metrics in spark-dispatcher not being registered properly

2017-09-27 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-22135: --- Assignee: paul mackles > metrics in spark-dispatcher not being registered prope

[jira] [Resolved] (SPARK-22135) metrics in spark-dispatcher not being registered properly

2017-09-27 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-22135. - Resolution: Fixed Fix Version/s: 2.3.0 2.2.1 Issue resolved by pull

[jira] [Updated] (LIVY-408) Upgrade Netty version to avoid some security issues

2017-09-27 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-408: - Affects Version/s: 0.5.0 0.4.1 > Upgrade Netty version to avoid some security iss

[jira] [Updated] (LIVY-408) Upgrade Netty version to avoid some security issues

2017-09-27 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-408: - Fix Version/s: (was: 0.5.0) > Upgrade Netty version to avoid some security iss

[jira] [Updated] (LIVY-408) Upgrade Netty version to avoid some security issues

2017-09-27 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-408: - Priority: Minor (was: Major) > Upgrade Netty version to avoid some security iss

[jira] [Created] (LIVY-408) Upgrade Netty version to avoid some security issues

2017-09-27 Thread Saisai Shao (JIRA)
Saisai Shao created LIVY-408: Summary: Upgrade Netty version to avoid some security issues Key: LIVY-408 URL: https://issues.apache.org/jira/browse/LIVY-408 Project: Livy Issue Type: Improvement

[jira] [Commented] (SPARK-22151) PYTHONPATH not picked up from the spark.yarn.appMasterEnv properly

2017-09-27 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183581#comment-16183581 ] Saisai Shao commented on SPARK-22151: - Checking the Yarn client code, looks

[jira] [Commented] (SPARK-21737) Create communication channel between arbitrary clients and the Spark AM in YARN mode

2017-09-27 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183563#comment-16183563 ] Saisai Shao commented on SPARK-21737: - [~yoonlee95], are you still working on

[jira] [Updated] (SPARK-22123) Add latest failure reason for task set blacklist

2017-09-27 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-22123: Priority: Minor (was: Major) > Add latest failure reason for task set blackl

[jira] [Assigned] (SPARK-22123) Add latest failure reason for task set blacklist

2017-09-27 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-22123: --- Assignee: zhoukang > Add latest failure reason for task set blackl

[jira] [Resolved] (SPARK-22123) Add latest failure reason for task set blacklist

2017-09-27 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-22123. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19338 [https

[jira] [Resolved] (SPARK-20785) Spark should provide jump links and add (count) in the SQL web ui.

2017-09-27 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-20785. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19346 [https

[jira] [Assigned] (SPARK-20785) Spark should provide jump links and add (count) in the SQL web ui.

2017-09-27 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-20785: --- Assignee: guoxiaolongzte > Spark should provide jump links and add (count) in the SQL

[jira] [Commented] (SPARK-22074) Task killed by other attempt task should not be resubmitted

2017-09-26 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16182062#comment-16182062 ] Saisai Shao commented on SPARK-22074: - So if I understand correctly, this hap

[jira] [Commented] (SPARK-22074) Task killed by other attempt task should not be resubmitted

2017-09-26 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16181924#comment-16181924 ] Saisai Shao commented on SPARK-22074: - Hey [~XuanYuan], I'm a little con

[jira] [Commented] (SPARK-9103) Tracking spark's memory usage

2017-09-26 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16181905#comment-16181905 ] Saisai Shao commented on SPARK-9103: Hi [~irashid], thanks a lot for your resp

[jira] [Commented] (SPARK-22074) Task killed by other attempt task should not be resubmitted

2017-09-26 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16180506#comment-16180506 ] Saisai Shao commented on SPARK-22074: - Hi [~XuanYuan], can you please help m

[jira] [Resolved] (LIVY-406) Livy fail to get app id and related information when using yarn client mode

2017-09-25 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved LIVY-406. -- Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 0.5.0 0.4.1

[jira] [Closed] (LIVY-403) Add getCompletion for Interpreter

2017-09-24 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao closed LIVY-403. Resolution: Duplicate > Add getCompletion for Interpre

[jira] [Commented] (SPARK-9103) Tracking spark's memory usage

2017-09-22 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16176073#comment-16176073 ] Saisai Shao commented on SPARK-9103: [~irashid] I would like to hear your sugges

[jira] [Updated] (LIVY-398) Change docs to describe how to use shared language session

2017-09-21 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-398: - Component/s: (was: REPL) Docs > Change docs to describe how to use shared langu

[jira] [Resolved] (LIVY-398) Change docs to describe how to use shared language session

2017-09-21 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved LIVY-398. -- Resolution: Fixed Fix Version/s: 0.5.0 > Change docs to describe how to use shared langu

[jira] [Assigned] (LIVY-398) Change docs to describe how to use shared language session

2017-09-21 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-398: Assignee: Saisai Shao > Change docs to describe how to use shared language sess

[jira] [Created] (LIVY-406) Livy fail to get app id and related information when using yarn client mode

2017-09-21 Thread Saisai Shao (JIRA)
Saisai Shao created LIVY-406: Summary: Livy fail to get app id and related information when using yarn client mode Key: LIVY-406 URL: https://issues.apache.org/jira/browse/LIVY-406 Project: Livy

[jira] [Assigned] (SPARK-21934) Expose Netty memory usage via Metrics System

2017-09-20 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-21934: --- Assignee: Saisai Shao > Expose Netty memory usage via Metrics Sys

[jira] [Resolved] (SPARK-21934) Expose Netty memory usage via Metrics System

2017-09-20 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-21934. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19160 [https

Re: How to get the test logs from travis

2017-09-20 Thread Saisai Shao
t a lot of the logs and > related files which then get appended to the build results. > > https://github.com/apache/zeppelin/blob/master/.travis.yml > > On Wed, Sep 20, 2017 at 8:22 PM, Saisai Shao > wrote: > > > Hi Team, > > > > Currently it is quite pai

How to get the test logs from travis

2017-09-20 Thread Saisai Shao
Hi Team, Currently it is quite painful to figure out the cause of failure on travis test. Do you know how to get test logs from travis, is there a way supported by travis to either upload environment to some places, or log on to travis to dig the files. Previously we uploaded logs to azure when t

[jira] [Assigned] (LIVY-399) Enable real test for PySpark and SparkR interpreters

2017-09-20 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-399: Assignee: Saisai Shao > Enable real test for PySpark and SparkR interpret

[jira] [Updated] (LIVY-386) Refactor Livy core module with Java

2017-09-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-386: - Description: Livy core module is used by several modules like server, repl, so it requires to support

[jira] [Updated] (LIVY-386) Refactor Livy core module with Java

2017-09-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-386: - Summary: Refactor Livy core module with Java (was: Refactor Livy code module with Java) > Refactor L

[jira] [Assigned] (LIVY-386) Refactor Livy core module with Java

2017-09-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-386: Assignee: Saisai Shao > Refactor Livy core module with J

[jira] [Updated] (LIVY-405) ERROR RSCClient: Failed to connect to context

2017-09-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-405: - Issue Type: Question (was: Bug) > ERROR RSCClient: Failed to connect to cont

[jira] [Commented] (LIVY-405) ERROR RSCClient: Failed to connect to context

2017-09-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16172600#comment-16172600 ] Saisai Shao commented on LIVY-405: -- It's too vague to understand your issue

[jira] [Updated] (LIVY-405) ERROR RSCClient: Failed to connect to context

2017-09-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-405: - Priority: Minor (was: Blocker) > ERROR RSCClient: Failed to connect to cont

[jira] [Assigned] (SPARK-22030) GraphiteSink fails to re-connect to Graphite instances behind an ELB or any other auto-scaled LB

2017-09-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-22030: --- Assignee: Alex Mikhailau > GraphiteSink fails to re-connect to Graphite instances behind

[jira] [Updated] (SPARK-22030) GraphiteSink fails to re-connect to Graphite instances behind an ELB or any other auto-scaled LB

2017-09-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-22030: Priority: Minor (was: Critical) > GraphiteSink fails to re-connect to Graphite instances beh

[jira] [Resolved] (SPARK-22030) GraphiteSink fails to re-connect to Graphite instances behind an ELB or any other auto-scaled LB

2017-09-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-22030. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19210 [https

[jira] [Assigned] (LIVY-396) Livy does not map YARN app states correctly

2017-09-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-396: Assignee: Meisam > Livy does not map YARN app states correc

[jira] [Assigned] (LIVY-396) Livy does not map YARN app states correctly

2017-09-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-396: Assignee: (was: Meisam Fathi) > Livy does not map YARN app states correc

[jira] [Assigned] (LIVY-396) Livy does not map YARN app states correctly

2017-09-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-396: Assignee: Meisam Fathi > Livy does not map YARN app states correc

[jira] [Resolved] (LIVY-396) Livy does not map YARN app states correctly

2017-09-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved LIVY-396. -- Resolution: Fixed Issue resolved by pull request 39 [https://github.com/apache/incubator-livy/pull/39

[jira] [Commented] (LIVY-403) Add getCompletion for Interpreter

2017-09-15 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167489#comment-16167489 ] Saisai Shao commented on LIVY-403: -- [~ajbozarth], the purpose of this JIRA is to sup

[jira] [Updated] (LIVY-403) Add getCompletion for Interpreter

2017-09-15 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-403: - Affects Version/s: 0.5.0 > Add getCompletion for Interpre

[jira] [Updated] (LIVY-403) Add getCompletion for Interpreter

2017-09-15 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/LIVY-403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-403: - Component/s: REPL > Add getCompletion for Interpre

[jira] [Updated] (SPARK-21902) BlockManager.doPut will hide actually exception when exception thrown in finally block

2017-09-14 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-21902: Priority: Trivial (was: Major) > BlockManager.doPut will hide actually exception when except

[jira] [Updated] (SPARK-21902) BlockManager.doPut will hide actually exception when exception thrown in finally block

2017-09-14 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-21902: Issue Type: Improvement (was: Wish) > BlockManager.doPut will hide actually exception w

[jira] [Assigned] (SPARK-21902) BlockManager.doPut will hide actually exception when exception thrown in finally block

2017-09-14 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-21902: --- Assignee: zhoukang > BlockManager.doPut will hide actually exception when exception thr

[jira] [Resolved] (SPARK-21902) BlockManager.doPut will hide actually exception when exception thrown in finally block

2017-09-14 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-21902. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19171 [https

[jira] [Resolved] (SPARK-21922) When executor failed and task metrics have not send to driver,the status will always be 'RUNNING' and the duration will be 'CurrentTime - launchTime'

2017-09-14 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-21922. - Resolution: Fixed Assignee: zhoukang Target Version/s: 2.3.0 > W

[jira] [Updated] (SPARK-21922) When executor failed and task metrics have not send to driver,the status will always be 'RUNNING' and the duration will be 'CurrentTime - launchTime'

2017-09-14 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-21922: Fix Version/s: 2.3.0 > When executor failed and task metrics have not send to driver,the sta

[jira] [Updated] (SPARK-21922) When executor failed and task metrics have not send to driver,the status will always be 'RUNNING' and the duration will be 'CurrentTime - launchTime'

2017-09-14 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-21922: Target Version/s: (was: 2.3.0) > When executor failed and task metrics have not send to dri

[jira] [Commented] (SPARK-21513) SQL to_json should support all column types

2017-09-12 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164000#comment-16164000 ] Saisai Shao commented on SPARK-21513: - [~goldmedal] is it your correct JIRA

[jira] [Assigned] (SPARK-21513) SQL to_json should support all column types

2017-09-12 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-21513: --- Assignee: Jia-Xuan Liu > SQL to_json should support all column ty

Re: user defined sessionId / URI for Livy sessions

2017-09-11 Thread Saisai Shao
I see. So based on this, we should manage a data structure in Livy Server to keep all the live sessions' name. Also regarding to session recovery, we should persist this structure to the reliable storage and recover after restart. I'm not pretty sure if it is a good feature or not. First because w

Re: user defined sessionId / URI for Livy sessions

2017-09-11 Thread Saisai Shao
I see. So based on this, we should manage a data structure in Livy Server to keep all the live sessions' name. Also regarding to session recovery, we should persist this structure to the reliable storage and recover after restart. I'm not pretty sure if it is a good feature or not. First because w

Re: user defined sessionId / URI for Livy sessions

2017-09-11 Thread Saisai Shao
If we're using session name, how do we guarantee the uniqueness of this name? Thanks Jerry On Tue, Sep 12, 2017 at 4:51 AM, Alex Bozarth wrote: > I would agree with Marcelo's comment the JIRA that this isn't a good > feature for livy, but I'll take a look at your impl if you open a PR and > see

Re: user defined sessionId / URI for Livy sessions

2017-09-11 Thread Saisai Shao
If we're using session name, how do we guarantee the uniqueness of this name? Thanks Jerry On Tue, Sep 12, 2017 at 4:51 AM, Alex Bozarth wrote: > I would agree with Marcelo's comment the JIRA that this isn't a good > feature for livy, but I'll take a look at your impl if you open a PR and > see

[jira] [Commented] (SPARK-21943) When I use rest Api (/ applications / [app-id] / jobs / [job-id]) to view some of the jobs that are running jobs, the returned json information is missing the “descrip

2017-09-11 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16161008#comment-16161008 ] Saisai Shao commented on SPARK-21943: - If you're trying to report bugs, I

<    4   5   6   7   8   9   10   11   12   13   >