[jira] [Reopened] (SPARK-23200) Reset configuration when restarting from checkpoints

[ https://issues.apache.org/jira/browse/SPARK-23200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reopened SPARK-23200: - Assignee: (was: Santiago Saavedra) > Reset configuration when restarting from checkpoints

[jira] [Resolved] (SPARK-23279) Avoid triggering distributed job for Console sink

[ https://issues.apache.org/jira/browse/SPARK-23279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-23279. - Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 2.3.0 > Avoid triggering

[jira] [Commented] (SPARK-23279) Avoid triggering distributed job for Console sink

[ https://issues.apache.org/jira/browse/SPARK-23279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346301#comment-16346301 ] Saisai Shao commented on SPARK-23279: - Issue resolved by pull request 20447

[jira] [Created] (SPARK-23279) Avoid triggering distributed job for Console sink

Saisai Shao created SPARK-23279: --- Summary: Avoid triggering distributed job for Console sink Key: SPARK-23279 URL: https://issues.apache.org/jira/browse/SPARK-23279 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-23088) History server not showing incomplete/running applications

[ https://issues.apache.org/jira/browse/SPARK-23088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-23088. - Resolution: Fixed Assignee: paul mackles Fix Version/s: 2.4.0 Issue resolved by

[jira] [Commented] (SPARK-23187) Accumulator object can not be sent from Executor to Driver

[ https://issues.apache.org/jira/browse/SPARK-23187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16342824#comment-16342824 ] Saisai Shao commented on SPARK-23187: - I think I tried, {{accumUpdates}} can be received in driver

[jira] [Created] (SPARK-23228) Able to track Python create SparkSession in JVM

Saisai Shao created SPARK-23228: --- Summary: Able to track Python create SparkSession in JVM Key: SPARK-23228 URL: https://issues.apache.org/jira/browse/SPARK-23228 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-23200) Reset configuration when restarting from checkpoints

[ https://issues.apache.org/jira/browse/SPARK-23200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-23200: --- Assignee: Anirudh Ramanathan > Reset configuration when restarting from checkpoints >

[jira] [Commented] (SPARK-23200) Reset configuration when restarting from checkpoints

[ https://issues.apache.org/jira/browse/SPARK-23200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340648#comment-16340648 ] Saisai Shao commented on SPARK-23200: - Issue resolved by pull request 20383

[jira] [Resolved] (SPARK-23200) Reset configuration when restarting from checkpoints

[ https://issues.apache.org/jira/browse/SPARK-23200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-23200. - Resolution: Fixed Fix Version/s: 2.4.0 > Reset configuration when restarting from

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340536#comment-16340536 ] Saisai Shao commented on SPARK-23206: - Would you please summarize what kind of metrics you want to

[jira] [Resolved] (SPARK-23187) Accumulator object can not be sent from Executor to Driver

[ https://issues.apache.org/jira/browse/SPARK-23187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-23187. - Resolution: Not A Problem > Accumulator object can not be sent from Executor to Driver >

[jira] [Commented] (SPARK-23187) Accumulator object can not be sent from Executor to Driver

[ https://issues.apache.org/jira/browse/SPARK-23187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340522#comment-16340522 ] Saisai Shao commented on SPARK-23187: - I'm going to close this JIRA, as there's no issue in reporting

[jira] [Commented] (SPARK-23187) Accumulator object can not be sent from Executor to Driver

[ https://issues.apache.org/jira/browse/SPARK-23187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340521#comment-16340521 ] Saisai Shao commented on SPARK-23187: - I'm going to close this JIRA, as there's no issue in reporting

[jira] [Commented] (SPARK-23187) Accumulator object can not be sent from Executor to Driver

[ https://issues.apache.org/jira/browse/SPARK-23187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338737#comment-16338737 ] Saisai Shao commented on SPARK-23187: - Actually heartbeat report is OK according to my investigation,

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338553#comment-16338553 ] Saisai Shao commented on SPARK-23206: - I think this Jira duplicates SPARK-9103. Also seems some

[jira] [Commented] (SPARK-23187) Accumulator object can not be sent from Executor to Driver

[ https://issues.apache.org/jira/browse/SPARK-23187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335428#comment-16335428 ] Saisai Shao commented on SPARK-23187: - I will try to investigate on it. > Accumulator object can not

[jira] [Updated] (SPARK-23187) Accumulator object can not be sent from Executor to Driver

[ https://issues.apache.org/jira/browse/SPARK-23187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23187: Affects Version/s: (was: 2.3.1) 2.3.0 > Accumulator object can not be

[jira] [Resolved] (SPARK-22838) Avoid unnecessary copying of data

[ https://issues.apache.org/jira/browse/SPARK-22838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-22838. - Resolution: Invalid > Avoid unnecessary copying of data > - > >

[jira] [Assigned] (SPARK-22976) Worker cleanup can remove running driver directories

[ https://issues.apache.org/jira/browse/SPARK-22976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-22976: --- Assignee: Russell Spitzer > Worker cleanup can remove running driver directories >

[jira] [Resolved] (SPARK-22976) Worker cleanup can remove running driver directories

[ https://issues.apache.org/jira/browse/SPARK-22976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-22976. - Resolution: Fixed Fix Version/s: 2.3.0 > Worker cleanup can remove running driver

[jira] [Commented] (SPARK-23097) Migrate text socket source

[ https://issues.apache.org/jira/browse/SPARK-23097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331735#comment-16331735 ] Saisai Shao commented on SPARK-23097: - Thanks, will do. > Migrate text socket source >

[jira] [Commented] (SPARK-23097) Migrate text socket source

[ https://issues.apache.org/jira/browse/SPARK-23097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331715#comment-16331715 ] Saisai Shao commented on SPARK-23097: - Hi [~joseph.torres], would you mind if I take a shot on this

[jira] [Commented] (LIVY-434) Livy 0.5-incubating Release

[ https://issues.apache.org/jira/browse/LIVY-434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331591#comment-16331591 ] Saisai Shao commented on LIVY-434: -- >From permission side I think it is OK to do it. But I'm not sure if

[jira] [Commented] (LIVY-434) Livy 0.5-incubating Release

[ https://issues.apache.org/jira/browse/LIVY-434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331580#comment-16331580 ] Saisai Shao commented on LIVY-434: -- LGTM. > Livy 0.5-incubating Release > --- > >

[jira] [Commented] (LIVY-434) Livy 0.5-incubating Release

[ https://issues.apache.org/jira/browse/LIVY-434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331559#comment-16331559 ] Saisai Shao commented on LIVY-434: -- Sure. We can test with Spark 2.3. But my concern is that 2.3 is a big

[jira] [Commented] (LIVY-434) Livy 0.5-incubating Release

[ https://issues.apache.org/jira/browse/LIVY-434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331549#comment-16331549 ] Saisai Shao commented on LIVY-434: -- IMO, I think we should have a timeline for 0.5 release, which cannot

[jira] [Commented] (SPARK-23128) A new approach to do adaptive execution in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-23128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331542#comment-16331542 ] Saisai Shao commented on SPARK-23128: - I'm wondering if we need a SPIP for such changes? > A new

[jira] [Updated] (SPARK-23147) Stage page will throw exception when there's no complete tasks

[ https://issues.apache.org/jira/browse/SPARK-23147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23147: Description: Current Stage page's task table will throw an exception when there's no complete

[jira] [Created] (SPARK-23147) Stage page will throw exception when there's no complete tasks

Saisai Shao created SPARK-23147: --- Summary: Stage page will throw exception when there's no complete tasks Key: SPARK-23147 URL: https://issues.apache.org/jira/browse/SPARK-23147 Project: Spark

[jira] [Updated] (SPARK-23147) Stage page will throw exception when there's no complete tasks

[ https://issues.apache.org/jira/browse/SPARK-23147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23147: Attachment: Screen Shot 2018-01-18 at 8.50.08 PM.png > Stage page will throw exception when

[jira] [Updated] (SPARK-23140) DataSourceV2Strategy is missing in HiveSessionStateBuilder

[ https://issues.apache.org/jira/browse/SPARK-23140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23140: Description: DataSourceV2Strategy is not added into HiveSessionStateBuilder's planner, which will

[jira] [Updated] (SPARK-23140) DataSourceV2Strategy is missing in HiveSessionStateBuilder

[ https://issues.apache.org/jira/browse/SPARK-23140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23140: Environment: (was: DataSourceV2Strategy is not added into HiveSessionStateBuilder's planner,

[jira] [Created] (SPARK-23140) DataSourceV2Strategy is missing in HiveSessionStateBuilder

Saisai Shao created SPARK-23140: --- Summary: DataSourceV2Strategy is missing in HiveSessionStateBuilder Key: SPARK-23140 URL: https://issues.apache.org/jira/browse/SPARK-23140 Project: Spark

[jira] [Commented] (SPARK-23123) Unable to run Spark Job with Hadoop NameNode Federation using ViewFS

[ https://issues.apache.org/jira/browse/SPARK-23123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328364#comment-16328364 ] Saisai Shao commented on SPARK-23123: - [~ste...@apache.org], any thought on this issue? > Unable to

[jira] [Commented] (SPARK-23123) Unable to run Spark Job with Hadoop NameNode Federation using ViewFS

[ https://issues.apache.org/jira/browse/SPARK-23123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328361#comment-16328361 ] Saisai Shao commented on SPARK-23123: - >From the stack, looks like the issue is from YARN to

[jira] [Commented] (SPARK-23123) Unable to run Spark Job with Hadoop NameNode Federation using ViewFS

[ https://issues.apache.org/jira/browse/SPARK-23123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328297#comment-16328297 ] Saisai Shao commented on SPARK-23123: - It looks like a Hadoop issue rather than a Spark issue from

[jira] [Commented] (SPARK-22958) Spark is stuck when the only one executor fails to register with driver

[ https://issues.apache.org/jira/browse/SPARK-22958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16325986#comment-16325986 ] Saisai Shao commented on SPARK-22958: - Please see this line:

[jira] [Commented] (SPARK-23029) Setting spark.shuffle.file.buffer will make the shuffle fail

[ https://issues.apache.org/jira/browse/SPARK-23029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323872#comment-16323872 ] Saisai Shao commented on SPARK-23029: - There should be no issue unless you want to fix doc issue...

[jira] [Commented] (SPARK-23029) Setting spark.shuffle.file.buffer will make the shuffle fail

[ https://issues.apache.org/jira/browse/SPARK-23029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323871#comment-16323871 ] Saisai Shao commented on SPARK-23029: - Total buffer size for one executor is {{32MB * C (core) * R

[jira] [Commented] (SPARK-21475) Change to use NIO's Files API for external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-21475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323590#comment-16323590 ] Saisai Shao commented on SPARK-21475: - [~zsxwing] is 3.0.0 the valid fix version? > Change to use

[jira] [Commented] (SPARK-22958) Spark is stuck when the only one executor fails to register with driver

[ https://issues.apache.org/jira/browse/SPARK-22958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323569#comment-16323569 ] Saisai Shao commented on SPARK-22958: - If executor is failed to register itself to driver, it will

[jira] [Resolved] (SPARK-23029) Setting spark.shuffle.file.buffer will make the shuffle fail

[ https://issues.apache.org/jira/browse/SPARK-23029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-23029. - Resolution: Not A Problem > Setting spark.shuffle.file.buffer will make the shuffle fail >

[jira] [Commented] (SPARK-23029) Setting spark.shuffle.file.buffer will make the shuffle fail

[ https://issues.apache.org/jira/browse/SPARK-23029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323496#comment-16323496 ] Saisai Shao commented on SPARK-23029: - Please see the comments in the code in

[jira] [Commented] (SPARK-22921) Merge script should prompt for assigning jiras

[ https://issues.apache.org/jira/browse/SPARK-22921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321669#comment-16321669 ] Saisai Shao commented on SPARK-22921: - Hi [~irashid], looks like the changes will throw an exception

[jira] [Resolved] (SPARK-22587) Spark job fails if fs.defaultFS and application jar are different url

[ https://issues.apache.org/jira/browse/SPARK-22587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-22587. - Resolution: Fixed > Spark job fails if fs.defaultFS and application jar are different url >

[jira] [Updated] (SPARK-22587) Spark job fails if fs.defaultFS and application jar are different url

[ https://issues.apache.org/jira/browse/SPARK-22587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-22587: Fix Version/s: 2.3.0 > Spark job fails if fs.defaultFS and application jar are different url >

[jira] [Assigned] (SPARK-22587) Spark job fails if fs.defaultFS and application jar are different url

[ https://issues.apache.org/jira/browse/SPARK-22587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-22587: --- Assignee: Mingjie Tang > Spark job fails if fs.defaultFS and application jar are different

[jira] [Commented] (SPARK-22876) spark.yarn.am.attemptFailuresValidityInterval does not work correctly

[ https://issues.apache.org/jira/browse/SPARK-22876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309363#comment-16309363 ] Saisai Shao commented on SPARK-22876: - This looks like a bug, ideally we should get number of

[jira] [Commented] (LIVY-426) Livy stops session quietly without error message

[ https://issues.apache.org/jira/browse/LIVY-426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281417#comment-16281417 ] Saisai Shao commented on LIVY-426: -- Sorry I misunderstood your issue, what I mean is that I also saw this

[jira] [Commented] (LIVY-426) Livy stops session quietly without error message

[ https://issues.apache.org/jira/browse/LIVY-426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281335#comment-16281335 ] Saisai Shao commented on LIVY-426: -- I also encountered this exception before. Do you have a fix for this?

[jira] [Assigned] (LIVY-19) Add Spark SQL support

[ https://issues.apache.org/jira/browse/LIVY-19?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-19: --- Assignee: Saisai Shao > Add Spark SQL support > - > > Key: LIVY-19

[jira] [Commented] (LIVY-104) Switch to scala 2.11

[ https://issues.apache.org/jira/browse/LIVY-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276172#comment-16276172 ] Saisai Shao commented on LIVY-104: -- Livy server itself requires Spark artifacts to run unit tests, if we

[jira] [Commented] (LIVY-421) support multi statements run at the same time

[ https://issues.apache.org/jira/browse/LIVY-421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16274174#comment-16274174 ] Saisai Shao commented on LIVY-421: -- bq. Adding concurrency for sql make sense for me as each sql statements

[jira] [Commented] (LIVY-104) Switch to scala 2.11

[ https://issues.apache.org/jira/browse/LIVY-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16274123#comment-16274123 ] Saisai Shao commented on LIVY-104: -- Hi [~ajbozarth] would you please drive this issue if you have time, I'm

[jira] [Created] (LIVY-423) Adding Scala 2.12 support

Saisai Shao created LIVY-423: Summary: Adding Scala 2.12 support Key: LIVY-423 URL: https://issues.apache.org/jira/browse/LIVY-423 Project: Livy Issue Type: New Feature Components:

[jira] [Commented] (SPARK-22393) spark-shell can't find imported types in class constructors, extends clause

[ https://issues.apache.org/jira/browse/SPARK-22393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273985#comment-16273985 ] Saisai Shao commented on SPARK-22393: - Shall we wait for this before 2.2.1 is out? > spark-shell

[jira] [Commented] (LIVY-421) support multi statements run at the same time

[ https://issues.apache.org/jira/browse/LIVY-421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266535#comment-16266535 ] Saisai Shao commented on LIVY-421: -- I agree with what Jeff mentioned, multi-threading is dangerous for

[jira] [Commented] (SPARK-2926) Add MR-style (merge-sort) SortShuffleReader for sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16265035#comment-16265035 ] Saisai Shao commented on SPARK-2926: So your saying of 12x-30x boosting is only referring to reduce

[jira] [Commented] (SPARK-22579) BlockManager.getRemoteValues and BlockManager.getRemoteBytes should be implemented using streaming

[ https://issues.apache.org/jira/browse/SPARK-22579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16265021#comment-16265021 ] Saisai Shao commented on SPARK-22579: - I think this issue should have already been fixed by

[jira] [Commented] (SPARK-22587) Spark job fails if fs.defaultFS and application jar are different url

[ https://issues.apache.org/jira/browse/SPARK-22587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264986#comment-16264986 ] Saisai Shao commented on SPARK-22587: - [~Prabhu Joseph], I think it is because the logic of comparing

[jira] [Resolved] (LIVY-299) Only the output of last line is returned

[ https://issues.apache.org/jira/browse/LIVY-299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved LIVY-299. -- Resolution: Fixed Fix Version/s: 0.5.0 > Only the output of last line is returned >

[jira] [Commented] (SPARK-2926) Add MR-style (merge-sort) SortShuffleReader for sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264904#comment-16264904 ] Saisai Shao commented on SPARK-2926: [~XuanYuan], would you please use spark-perf's micro benchmark

[jira] [Assigned] (LIVY-416) com.fasterxml.jackson.core.JsonGenerationException is sometimes thrown

[ https://issues.apache.org/jira/browse/LIVY-416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-416: Assignee: Keiji Yoshida > com.fasterxml.jackson.core.JsonGenerationException is sometimes thrown >

[jira] [Resolved] (LIVY-416) com.fasterxml.jackson.core.JsonGenerationException is sometimes thrown

[ https://issues.apache.org/jira/browse/LIVY-416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved LIVY-416. -- Resolution: Fixed Fix Version/s: 0.5.0 > com.fasterxml.jackson.core.JsonGenerationException is

[jira] [Assigned] (LIVY-418) Respect spark.pyspark.python for PythonInterpreter

[ https://issues.apache.org/jira/browse/LIVY-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-418: Assignee: Jeff Zhang > Respect spark.pyspark.python for PythonInterpreter >

[jira] [Resolved] (LIVY-418) Respect spark.pyspark.python for PythonInterpreter

[ https://issues.apache.org/jira/browse/LIVY-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved LIVY-418. -- Resolution: Fixed Fix Version/s: 0.5.0 > Respect spark.pyspark.python for PythonInterpreter >

[jira] [Assigned] (LIVY-299) Only the output of last line is returned

[ https://issues.apache.org/jira/browse/LIVY-299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-299: Assignee: Saisai Shao > Only the output of last line is returned >

[jira] [Commented] (SPARK-22528) History service and non-HDFS filesystems

[ https://issues.apache.org/jira/browse/SPARK-22528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254669#comment-16254669 ] Saisai Shao commented on SPARK-22528: - Hi [~adobe_pmackles] I think this issue might be due to the

[jira] [Assigned] (LIVY-415) Use objects and abstract classes in for Kind and SessionState.

[ https://issues.apache.org/jira/browse/LIVY-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-415: Assignee: Arun Allamsetty > Use objects and abstract classes in for Kind and SessionState. >

[jira] [Updated] (LIVY-415) Use objects and abstract classes in for Kind and SessionState.

[ https://issues.apache.org/jira/browse/LIVY-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-415: - Fix Version/s: 0.5.0 > Use objects and abstract classes in for Kind and SessionState. >

[jira] [Updated] (LIVY-415) Use objects and abstract classes in for Kind and SessionState.

[ https://issues.apache.org/jira/browse/LIVY-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated LIVY-415: - Component/s: Server REPL > Use objects and abstract classes in for Kind and SessionState.

[jira] [Commented] (SPARK-9104) expose network layer memory usage

[ https://issues.apache.org/jira/browse/SPARK-9104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16250675#comment-16250675 ] Saisai Shao commented on SPARK-9104: [~vsr] I think SPARK-21934 already exposed Netty shuffle metrics

[jira] [Created] (LIVY-414) Support setting environment variables during session creation

Saisai Shao created LIVY-414: Summary: Support setting environment variables during session creation Key: LIVY-414 URL: https://issues.apache.org/jira/browse/LIVY-414 Project: Livy Issue Type:

[jira] [Assigned] (LIVY-412) CreateSession should be reject when too many child process "spark-submit" is running

[ https://issues.apache.org/jira/browse/LIVY-412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-412: Assignee: Hong Shen > CreateSession should be reject when too many child process "spark-submit" is

[jira] [Resolved] (LIVY-412) CreateSession should be reject when too many child process "spark-submit" is running

[ https://issues.apache.org/jira/browse/LIVY-412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved LIVY-412. -- Resolution: Fixed Fix Version/s: 0.5.0 Issue resolved by pull request 58

[jira] [Commented] (SPARK-22426) Spark AM launching containers on node where External spark shuffle service failed to initialize

[ https://issues.apache.org/jira/browse/SPARK-22426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16236939#comment-16236939 ] Saisai Shao commented on SPARK-22426: - This kind of scenario was handled in SPARK-13669 regarding to

[jira] [Commented] (SPARK-22405) Enrich the event information and add new event of ExternalCatalogEvent

[ https://issues.apache.org/jira/browse/SPARK-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235042#comment-16235042 ] Saisai Shao commented on SPARK-22405: - Thanks [~hvanhovell] for your comment, let me propose a

[jira] [Assigned] (SPARK-22172) Worker hangs when the external shuffle service port is already in use

[ https://issues.apache.org/jira/browse/SPARK-22172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-22172: --- Assignee: Devaraj K > Worker hangs when the external shuffle service port is already in use

[jira] [Resolved] (SPARK-22172) Worker hangs when the external shuffle service port is already in use

[ https://issues.apache.org/jira/browse/SPARK-22172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-22172. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19396

[jira] [Commented] (SPARK-22405) Enrich the event information and add new event of ExternalCatalogEvent

[ https://issues.apache.org/jira/browse/SPARK-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233569#comment-16233569 ] Saisai Shao commented on SPARK-22405: - Thanks [~hvanhovell] for your comments. bq. When implementing

[jira] [Commented] (SPARK-22405) Enrich the event information and add new event of ExternalCatalogEvent

[ https://issues.apache.org/jira/browse/SPARK-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226724#comment-16226724 ] Saisai Shao commented on SPARK-22405: - Thanks [~cloud_fan] for your comments. Using fake

[jira] [Commented] (SPARK-22405) Enrich the event information and add new event of ExternalCatalogEvent

[ https://issues.apache.org/jira/browse/SPARK-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226474#comment-16226474 ] Saisai Shao commented on SPARK-22405: - CC [~smilegator], what's your opinion about this proposal?

[jira] [Commented] (SPARK-22405) Enrich the event information and add new event of ExternalCatalogEvent

[ https://issues.apache.org/jira/browse/SPARK-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226468#comment-16226468 ] Saisai Shao commented on SPARK-22405: - This is the WIP branch

[jira] [Created] (SPARK-22405) Enrich the event information of ExternalCatalogEvent

Saisai Shao created SPARK-22405: --- Summary: Enrich the event information of ExternalCatalogEvent Key: SPARK-22405 URL: https://issues.apache.org/jira/browse/SPARK-22405 Project: Spark Issue

[jira] [Updated] (SPARK-22405) Enrich the event information and add new event of ExternalCatalogEvent

[ https://issues.apache.org/jira/browse/SPARK-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-22405: Summary: Enrich the event information and add new event of ExternalCatalogEvent (was: Enrich the

[jira] [Commented] (SPARK-22357) SparkContext.binaryFiles ignore minPartitions parameter

[ https://issues.apache.org/jira/browse/SPARK-22357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220431#comment-16220431 ] Saisai Shao commented on SPARK-22357: - Yes, I know this parameter is ignored, but I'm not sure is it

[jira] [Commented] (SPARK-22357) SparkContext.binaryFiles ignore minPartitions parameter

[ https://issues.apache.org/jira/browse/SPARK-22357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220395#comment-16220395 ] Saisai Shao commented on SPARK-22357: - bq. In our specific case this was a problem, because we

[jira] [Commented] (SPARK-22357) SparkContext.binaryFiles ignore minPartitions parameter

[ https://issues.apache.org/jira/browse/SPARK-22357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220160#comment-16220160 ] Saisai Shao commented on SPARK-22357: - [~WeichenXu123] would you please format the code in JIRA

[jira] [Assigned] (SPARK-21840) Allow multiple SparkSubmit invocations in same JVM without polluting system properties

[ https://issues.apache.org/jira/browse/SPARK-21840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-21840: --- Assignee: Marcelo Vanzin > Allow multiple SparkSubmit invocations in same JVM without

[jira] [Resolved] (SPARK-21840) Allow multiple SparkSubmit invocations in same JVM without polluting system properties

[ https://issues.apache.org/jira/browse/SPARK-21840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-21840. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19519

[jira] [Updated] (SPARK-22319) SparkSubmit calls getFileStatus before calling loginUserFromKeytab

[ https://issues.apache.org/jira/browse/SPARK-22319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-22319: Fix Version/s: 2.2.1 > SparkSubmit calls getFileStatus before calling loginUserFromKeytab >

[jira] [Closed] (SPARK-22272) killing task may cause the executor progress hang because of the JVM bug

[ https://issues.apache.org/jira/browse/SPARK-22272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao closed SPARK-22272. --- > killing task may cause the executor progress hang because of the JVM bug >

[jira] [Resolved] (SPARK-22272) killing task may cause the executor progress hang because of the JVM bug

[ https://issues.apache.org/jira/browse/SPARK-22272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-22272. - Resolution: Won't Fix > killing task may cause the executor progress hang because of the JVM bug

[jira] [Commented] (SPARK-22272) killing task may cause the executor progress hang because of the JVM bug

[ https://issues.apache.org/jira/browse/SPARK-22272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16214569#comment-16214569 ] Saisai Shao commented on SPARK-22272: - Setting "spark.file.transferTo" to "false" will potentially

[jira] [Assigned] (SPARK-22319) SparkSubmit calls getFileStatus before calling loginUserFromKeytab

[ https://issues.apache.org/jira/browse/SPARK-22319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-22319: --- Assignee: Steven Rand > SparkSubmit calls getFileStatus before calling loginUserFromKeytab

[jira] [Resolved] (SPARK-22319) SparkSubmit calls getFileStatus before calling loginUserFromKeytab

[ https://issues.apache.org/jira/browse/SPARK-22319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-22319. - Resolution: Fixed Fix Version/s: 2.3.0 > SparkSubmit calls getFileStatus before calling

[jira] [Updated] (SPARK-22319) SparkSubmit calls getFileStatus before calling loginUserFromKeytab

[ https://issues.apache.org/jira/browse/SPARK-22319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-22319: Affects Version/s: 2.2.0 > SparkSubmit calls getFileStatus before calling loginUserFromKeytab >

[jira] [Assigned] (LIVY-245) Sharing variables across Scala Jobs in RSC.

[ https://issues.apache.org/jira/browse/LIVY-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned LIVY-245: Assignee: Saisai Shao (was: Alex Man) > Sharing variables across Scala Jobs in RSC. >

[jira] [Resolved] (LIVY-245) Sharing variables across Scala Jobs in RSC.

[ https://issues.apache.org/jira/browse/LIVY-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved LIVY-245. -- Resolution: Fixed Fix Version/s: 0.5.0 Issue resolved by pull request 19

<    1   2   3   4   5   6   7   8   9   10   >