[jira] [Resolved] (SPARK-2652) Turning default configurations for PySpark

[ https://issues.apache.org/jira/browse/SPARK-2652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-2652. -- Resolution: Fixed Fix Version/s: (was: 1.1.0) 1.1.1

[jira] [Closed] (SPARK-3990) kryo.KryoException caused by ALS.trainImplicit in pyspark

[ https://issues.apache.org/jira/browse/SPARK-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-3990. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 This is fixed by reverting th

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182511#comment-14182511 ] Patrick Wendell commented on SPARK-3561: Hey [~ozhurakousky] - adding an @Experime

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182520#comment-14182520 ] Patrick Wendell commented on SPARK-3561: One other thing - if projects really do w

[jira] [Created] (SPARK-4074) No exception for drop nonexistent table

Michael Armbrust created SPARK-4074: --- Summary: No exception for drop nonexistent table Key: SPARK-4074 URL: https://issues.apache.org/jira/browse/SPARK-4074 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-4074) No exception for drop nonexistent table

[ https://issues.apache.org/jira/browse/SPARK-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182527#comment-14182527 ] Apache Spark commented on SPARK-4074: - User 'marmbrus' has created a pull request for

[jira] [Commented] (SPARK-2377) Create a Python API for Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182580#comment-14182580 ] Prabeesh K commented on SPARK-2377: --- Hi [~tdas], I wish start on Python API MQTT Stream

[jira] [Commented] (SPARK-3900) ApplicationMaster's shutdown hook fails and IllegalStateException is thrown.

[ https://issues.apache.org/jira/browse/SPARK-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182597#comment-14182597 ] Apache Spark commented on SPARK-3900: - User 'sarutak' has created a pull request for t

[jira] [Commented] (SPARK-2377) Create a Python API for Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182622#comment-14182622 ] Kenichi Takagiwa commented on SPARK-2377: - Hi [~prabeeshk] [~davies] is working f

[jira] [Created] (SPARK-4075) Jar url validation is not enough for Jar file

Kousuke Saruta created SPARK-4075: - Summary: Jar url validation is not enough for Jar file Key: SPARK-4075 URL: https://issues.apache.org/jira/browse/SPARK-4075 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-4075) Jar url validation is not enough for Jar file

[ https://issues.apache.org/jira/browse/SPARK-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182650#comment-14182650 ] Apache Spark commented on SPARK-4075: - User 'sarutak' has created a pull request for t

[jira] [Commented] (SPARK-4066) Make whether maven builds fails on scalastyle violation configurable

[ https://issues.apache.org/jira/browse/SPARK-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182658#comment-14182658 ] Sean Owen commented on SPARK-4066: -- Does this work? -Dscalastyle.failOnViolation was alre

[jira] [Updated] (SPARK-4038) Outlier Detection Algorithm for MLlib

[ https://issues.apache.org/jira/browse/SPARK-4038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Trivedi updated SPARK-4038: Affects Version/s: (was: 1.2.0) > Outlier Detection Algorithm for MLlib > --

[jira] [Updated] (SPARK-4038) Outlier Detection Algorithm for MLlib

[ https://issues.apache.org/jira/browse/SPARK-4038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Trivedi updated SPARK-4038: Priority: Minor (was: Major) > Outlier Detection Algorithm for MLlib >

[jira] [Commented] (SPARK-3814) Support for Bitwise AND(&), OR(|) ,XOR(^), NOT(~) in Spark HQL and SQL

[ https://issues.apache.org/jira/browse/SPARK-3814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182729#comment-14182729 ] Apache Spark commented on SPARK-3814: - User 'ravipesala' has created a pull request fo

[jira] [Commented] (SPARK-3483) Special chars in column names

[ https://issues.apache.org/jira/browse/SPARK-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182746#comment-14182746 ] Apache Spark commented on SPARK-3483: - User 'ravipesala' has created a pull request fo

[jira] [Commented] (SPARK-4022) Replace colt dependency (LGPL) with commons-math

[ https://issues.apache.org/jira/browse/SPARK-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182781#comment-14182781 ] Sean Owen commented on SPARK-4022: -- [~mengxr] [~josephkb] Great, most of this is resolved

[jira] [Commented] (SPARK-4022) Replace colt dependency (LGPL) with commons-math

[ https://issues.apache.org/jira/browse/SPARK-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182785#comment-14182785 ] Apache Spark commented on SPARK-4022: - User 'srowen' has created a pull request for th

[jira] [Resolved] (SPARK-3900) ApplicationMaster's shutdown hook fails and IllegalStateException is thrown.

[ https://issues.apache.org/jira/browse/SPARK-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3900. -- Resolution: Fixed Fix Version/s: 1.2.0 > ApplicationMaster's shutdown hook fails and Ille

[jira] [Commented] (SPARK-4064) If we create a lot of big broadcast variables, Spark has great possibility to hang

[ https://issues.apache.org/jira/browse/SPARK-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182796#comment-14182796 ] Apache Spark commented on SPARK-4064: - User 'witgo' has created a pull request for thi

[jira] [Created] (SPARK-4076) Parameter expansion in spark-config is wrong

Kousuke Saruta created SPARK-4076: - Summary: Parameter expansion in spark-config is wrong Key: SPARK-4076 URL: https://issues.apache.org/jira/browse/SPARK-4076 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-4076) Parameter expansion in spark-config is wrong

[ https://issues.apache.org/jira/browse/SPARK-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182833#comment-14182833 ] Apache Spark commented on SPARK-4076: - User 'sarutak' has created a pull request for t

[jira] [Created] (SPARK-4077) A broken string timestamp value can Spark SQL return wrong values for valid string timestamp values

Yin Huai created SPARK-4077: --- Summary: A broken string timestamp value can Spark SQL return wrong values for valid string timestamp values Key: SPARK-4077 URL: https://issues.apache.org/jira/browse/SPARK-4077

[jira] [Created] (SPARK-4078) New FsPermission instance w/o FsPermission.createImmutable in eventlog

Jie Huang created SPARK-4078: Summary: New FsPermission instance w/o FsPermission.createImmutable in eventlog Key: SPARK-4078 URL: https://issues.apache.org/jira/browse/SPARK-4078 Project: Spark

[jira] [Commented] (SPARK-4078) New FsPermission instance w/o FsPermission.createImmutable in eventlog

[ https://issues.apache.org/jira/browse/SPARK-4078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182894#comment-14182894 ] Apache Spark commented on SPARK-4078: - User 'GraceH' has created a pull request for th

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182956#comment-14182956 ] Dan Osipov commented on SPARK-3821: --- I'd like to take this on - this is needed for a lau

[jira] [Updated] (SPARK-4047) Generate runtime warning for naive implementation examples for algorithms implemented in MLlib/graphx

[ https://issues.apache.org/jira/browse/SPARK-4047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varadharajan updated SPARK-4047: Description: Based on SPARK-2434, we're generating runtime warnings to denote that the example impl

[jira] [Updated] (SPARK-4047) Generate runtime warning for naive implementation examples for algorithms implemented in MLlib/graphx

[ https://issues.apache.org/jira/browse/SPARK-4047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varadharajan updated SPARK-4047: Description: Based on SPARK-2434, we're generating runtime warnings to denote that the example impl

[jira] [Commented] (SPARK-3851) Support for reading parquet files with different but compatible schema

[ https://issues.apache.org/jira/browse/SPARK-3851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182980#comment-14182980 ] Gary Malouf commented on SPARK-3851: This is the type of issue that can bite users lon

[jira] [Commented] (SPARK-4066) Make whether maven builds fails on scalastyle violation configurable

[ https://issues.apache.org/jira/browse/SPARK-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183016#comment-14183016 ] Ted Yu commented on SPARK-4066: --- bq. -Dscalastyle.failOnViolation was already a built-in way

[jira] [Commented] (SPARK-1473) Feature selection for high dimensional datasets

[ https://issues.apache.org/jira/browse/SPARK-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183075#comment-14183075 ] Gavin Brown commented on SPARK-1473: Sorry, one extra point I didn't notice before

[jira] [Created] (SPARK-4079) Snappy bundled with Spark does not work on older Linux distributions

Marcelo Vanzin created SPARK-4079: - Summary: Snappy bundled with Spark does not work on older Linux distributions Key: SPARK-4079 URL: https://issues.apache.org/jira/browse/SPARK-4079 Project: Spark

[jira] [Resolved] (SPARK-4051) Rows in python should support conversion to dictionary

[ https://issues.apache.org/jira/browse/SPARK-4051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-4051. --- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2896 [https://github.com/

[jira] [Resolved] (SPARK-4050) Caching of temporary tables with projects fail when the final query projects fewer columns

[ https://issues.apache.org/jira/browse/SPARK-4050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4050. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2912 [https:/

[jira] [Resolved] (SPARK-2706) Enable Spark to support Hive 0.13

[ https://issues.apache.org/jira/browse/SPARK-2706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2706. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2241 [https:/

[jira] [Commented] (SPARK-3573) Dataset

[ https://issues.apache.org/jira/browse/SPARK-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183173#comment-14183173 ] Sandy Ryza commented on SPARK-3573: --- Is this still targeted for 1.2? > Dataset > --

[jira] [Commented] (SPARK-1856) Standardize MLlib interfaces

[ https://issues.apache.org/jira/browse/SPARK-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183174#comment-14183174 ] Sandy Ryza commented on SPARK-1856: --- Is this work still targeted for 1.2? > Standardize

[jira] [Created] (SPARK-4080) "IOException: unexpected exception type" while deserializing tasks

Josh Rosen created SPARK-4080: - Summary: "IOException: unexpected exception type" while deserializing tasks Key: SPARK-4080 URL: https://issues.apache.org/jira/browse/SPARK-4080 Project: Spark I

[jira] [Commented] (SPARK-4066) Make whether maven builds fails on scalastyle violation configurable

[ https://issues.apache.org/jira/browse/SPARK-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183235#comment-14183235 ] Patrick Wendell commented on SPARK-4066: [~srowen] I don't see a good argument for

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183244#comment-14183244 ] Marcelo Vanzin commented on SPARK-3561: --- bq. the best way would be to just extend Sp

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183245#comment-14183245 ] Nicholas Chammas commented on SPARK-3821: - Hey [~danospv], I'm currently in the mi

[jira] [Resolved] (SPARK-4026) Write ahead log management

[ https://issues.apache.org/jira/browse/SPARK-4026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-4026. -- Resolution: Fixed Fix Version/s: 1.2.0 > Write ahead log management > ---

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183276#comment-14183276 ] Dan Osipov commented on SPARK-3821: --- OK, great! > Could you elaborate on your use case?

[jira] [Commented] (SPARK-4027) HDFS backed Block RDD

[ https://issues.apache.org/jira/browse/SPARK-4027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183282#comment-14183282 ] Apache Spark commented on SPARK-4027: - User 'tdas' has created a pull request for this

[jira] [Commented] (SPARK-3369) Java mapPartitions Iterator->Iterable is inconsistent with Scala's Iterator->Iterator

[ https://issues.apache.org/jira/browse/SPARK-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183301#comment-14183301 ] Juliet Hougland commented on SPARK-3369: The guaruntee of semantic versioning is t

[jira] [Commented] (SPARK-4079) Snappy bundled with Spark does not work on older Linux distributions

[ https://issues.apache.org/jira/browse/SPARK-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183317#comment-14183317 ] Patrick Wendell commented on SPARK-4079: What about just catching the exception an

[jira] [Commented] (SPARK-4080) "IOException: unexpected exception type" while deserializing tasks

[ https://issues.apache.org/jira/browse/SPARK-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183323#comment-14183323 ] Apache Spark commented on SPARK-4080: - User 'JoshRosen' has created a pull request for

[jira] [Commented] (SPARK-4066) Make whether maven builds fails on scalastyle violation configurable

[ https://issues.apache.org/jira/browse/SPARK-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183340#comment-14183340 ] Sean Owen commented on SPARK-4066: -- Yeah, it's another flag to add to the build, but it's

[jira] [Updated] (SPARK-4026) Write ahead log management

[ https://issues.apache.org/jira/browse/SPARK-4026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-4026: - Assignee: Hari Shreedharan (was: Tathagata Das) > Write ahead log management > --

[jira] [Updated] (SPARK-4027) HDFS backed Block RDD

[ https://issues.apache.org/jira/browse/SPARK-4027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-4027: - Assignee: Hari Shreedharan (was: Tathagata Das) > HDFS backed Block RDD > - >

[jira] [Closed] (SPARK-2713) Executors of same application in same host should only download files & jars once

[ https://issues.apache.org/jira/browse/SPARK-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-2713. Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Zhihui Target Version/s: 1.2.0

[jira] [Closed] (SPARK-4076) Parameter expansion in spark-config is wrong

[ https://issues.apache.org/jira/browse/SPARK-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-4076. Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Kousuke Saruta > Parameter expansion in spa

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183390#comment-14183390 ] Nicholas Chammas commented on SPARK-3821: - Going for something like EMR's CLI is p

[jira] [Commented] (SPARK-4066) Make whether maven builds fails on scalastyle violation configurable

[ https://issues.apache.org/jira/browse/SPARK-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183399#comment-14183399 ] Patrick Wendell commented on SPARK-4066: That was actually my thought originally -

[jira] [Closed] (SPARK-4075) Jar url validation is not enough for Jar file

[ https://issues.apache.org/jira/browse/SPARK-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-4075. Resolution: Fixed Fix Version/s: 1.1.1 Assignee: Kousuke Saruta Target Version/s

[jira] [Closed] (SPARK-4013) Do not run multiple actor systems on each executor

[ https://issues.apache.org/jira/browse/SPARK-4013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-4013. Resolution: Fixed Fix Version/s: 1.2.0 > Do not run multiple actor systems on each executor > ---

[jira] [Resolved] (SPARK-3053) Reconcile spark.files.userClassPathFirst with spark.yarn.user.classpath.first

[ https://issues.apache.org/jira/browse/SPARK-3053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kostas Sakellis resolved SPARK-3053. Resolution: Duplicate > Reconcile spark.files.userClassPathFirst with spark.yarn.user.classp

[jira] [Created] (SPARK-4081) Categorical feature indexing

Joseph K. Bradley created SPARK-4081: Summary: Categorical feature indexing Key: SPARK-4081 URL: https://issues.apache.org/jira/browse/SPARK-4081 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-4067) refactor ExecutorUncaughtExceptionHandler as a general one as it is used like this

[ https://issues.apache.org/jira/browse/SPARK-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4067: - Component/s: Spark Core > refactor ExecutorUncaughtExceptionHandler as a general one as it is used like >

[jira] [Closed] (SPARK-4067) refactor ExecutorUncaughtExceptionHandler as a general one as it is used like this

[ https://issues.apache.org/jira/browse/SPARK-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-4067. Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Nan Zhu Target Version/s: 1.2.0

[jira] [Commented] (SPARK-2706) Enable Spark to support Hive 0.13

[ https://issues.apache.org/jira/browse/SPARK-2706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183452#comment-14183452 ] Zhan Zhang commented on SPARK-2706: --- I just check the trunk, the change is already there

[jira] [Updated] (SPARK-4006) Spark Driver crashes whenever an Executor is registered twice

[ https://issues.apache.org/jira/browse/SPARK-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4006: - Fix Version/s: 1.1.1 > Spark Driver crashes whenever an Executor is registered twice > ---

[jira] [Created] (SPARK-4082) Show Waiting/Queued Stages in Spark UI

Pat McDonough created SPARK-4082: Summary: Show Waiting/Queued Stages in Spark UI Key: SPARK-4082 URL: https://issues.apache.org/jira/browse/SPARK-4082 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-3573) Dataset

[ https://issues.apache.org/jira/browse/SPARK-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183488#comment-14183488 ] Xiangrui Meng commented on SPARK-3573: -- Yes, both the metadata PR and the UDT PR are

[jira] [Created] (SPARK-4083) Remove all unnecessary broadcasts

Davies Liu created SPARK-4083: - Summary: Remove all unnecessary broadcasts Key: SPARK-4083 URL: https://issues.apache.org/jira/browse/SPARK-4083 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-2706) Enable Spark to support Hive 0.13

[ https://issues.apache.org/jira/browse/SPARK-2706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183493#comment-14183493 ] Zhan Zhang commented on SPARK-2706: --- Michael, Please ignore my last email. I thought the

[jira] [Assigned] (SPARK-4084) Reuse sort key in Sorter

[ https://issues.apache.org/jira/browse/SPARK-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-4084: Assignee: Xiangrui Meng > Reuse sort key in Sorter > > >

[jira] [Created] (SPARK-4084) Reuse sort key in Sorter

Xiangrui Meng created SPARK-4084: Summary: Reuse sort key in Sorter Key: SPARK-4084 URL: https://issues.apache.org/jira/browse/SPARK-4084 Project: Spark Issue Type: Improvement Comp

[jira] [Commented] (SPARK-4082) Show Waiting/Queued Stages in Spark UI

[ https://issues.apache.org/jira/browse/SPARK-4082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183552#comment-14183552 ] Apache Spark commented on SPARK-4082: - User 'davies' has created a pull request for th

[jira] [Created] (SPARK-4085) Job will fail if a shuffle file that's read locally gets deleted

Kay Ousterhout created SPARK-4085: - Summary: Job will fail if a shuffle file that's read locally gets deleted Key: SPARK-4085 URL: https://issues.apache.org/jira/browse/SPARK-4085 Project: Spark

[jira] [Updated] (SPARK-4085) Job will fail if a shuffle file that's read locally gets deleted

[ https://issues.apache.org/jira/browse/SPARK-4085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-4085: -- Description: This commit: https://github.com/apache/spark/commit/665e71d14debb8a7fc1547c614867a

[jira] [Updated] (SPARK-4085) Job will fail if a shuffle file that's read locally gets deleted

[ https://issues.apache.org/jira/browse/SPARK-4085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-4085: -- Description: This commit: https://github.com/apache/spark/commit/665e71d14debb8a7fc1547c614867a

[jira] [Updated] (SPARK-4085) Job will fail if a shuffle file that's read locally gets deleted

[ https://issues.apache.org/jira/browse/SPARK-4085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-4085: -- Priority: Critical (was: Major) Upgraded to critical on [~pwendell]'s request > Job will fail

[jira] [Created] (SPARK-4086) Fold-style aggregation for VertexRDD

Ankur Dave created SPARK-4086: - Summary: Fold-style aggregation for VertexRDD Key: SPARK-4086 URL: https://issues.apache.org/jira/browse/SPARK-4086 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-4086) Fold-style aggregation for VertexRDD

[ https://issues.apache.org/jira/browse/SPARK-4086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-4086: -- Description: VertexRDD currently supports creations and joins only through a reduce-style interface wher

[jira] [Commented] (SPARK-4082) Show Waiting/Queued Stages in Spark UI

[ https://issues.apache.org/jira/browse/SPARK-4082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183572#comment-14183572 ] Pat McDonough commented on SPARK-4082: -- [~davies] - I think PR#2935 was incorrectly l

[jira] [Resolved] (SPARK-4080) "IOException: unexpected exception type" while deserializing tasks

[ https://issues.apache.org/jira/browse/SPARK-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-4080. --- Resolution: Fixed Fix Version/s: 1.1.1 1.2.0 Issue resolved by pull request

[jira] [Commented] (SPARK-4084) Reuse sort key in Sorter

[ https://issues.apache.org/jira/browse/SPARK-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183643#comment-14183643 ] Apache Spark commented on SPARK-4084: - User 'mengxr' has created a pull request for th

[jira] [Updated] (SPARK-4084) Reuse sort key in Sorter

[ https://issues.apache.org/jira/browse/SPARK-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4084: - Target Version/s: 1.2.0 > Reuse sort key in Sorter > > >

[jira] [Created] (SPARK-4087) Only use broadcast for large tasks

Davies Liu created SPARK-4087: - Summary: Only use broadcast for large tasks Key: SPARK-4087 URL: https://issues.apache.org/jira/browse/SPARK-4087 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-4087) Only use broadcast for large tasks

[ https://issues.apache.org/jira/browse/SPARK-4087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183676#comment-14183676 ] Apache Spark commented on SPARK-4087: - User 'davies' has created a pull request for th

[jira] [Resolved] (SPARK-4083) Remove all unnecessary broadcasts

[ https://issues.apache.org/jira/browse/SPARK-4083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-4083. --- Resolution: Duplicate > Remove all unnecessary broadcasts > - > >

[jira] [Commented] (SPARK-2585) Remove special handling of Hadoop JobConf

[ https://issues.apache.org/jira/browse/SPARK-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183722#comment-14183722 ] Apache Spark commented on SPARK-2585: - User 'davies' has created a pull request for th

[jira] [Resolved] (SPARK-4056) Upgrade snappy-java to 1.1.1.5

[ https://issues.apache.org/jira/browse/SPARK-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-4056. --- Resolution: Fixed Fix Version/s: 1.1.1 1.2.0 Issue resolved by pull request

[jira] [Reopened] (SPARK-2585) Remove special handling of Hadoop JobConf

[ https://issues.apache.org/jira/browse/SPARK-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reopened SPARK-2585: --- simplify the code about serialize configuration > Remove special handling of Hadoop JobConf > ---

[jira] [Updated] (SPARK-3789) Python bindings for GraphX

[ https://issues.apache.org/jira/browse/SPARK-3789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3789: -- Assignee: Kushal Datta > Python bindings for GraphX > -- > > Key

[jira] [Commented] (SPARK-4030) `destroy` method in Broadcast should be public

[ https://issues.apache.org/jira/browse/SPARK-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183874#comment-14183874 ] Josh Rosen commented on SPARK-4030: --- This is similar in spirit to SPARK-3885, which is a

[jira] [Commented] (SPARK-4028) ReceivedBlockHandler interface to abstract the functionality of storage of received data

[ https://issues.apache.org/jira/browse/SPARK-4028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183908#comment-14183908 ] Apache Spark commented on SPARK-4028: - User 'tdas' has created a pull request for this