[jira] [Created] (SPARK-3829) Make Spark logo image on the header of HistoryPage as a link to HistoryPage's page #1

Kousuke Saruta created SPARK-3829: - Summary: Make Spark logo image on the header of HistoryPage as a link to HistoryPage's page #1 Key: SPARK-3829 URL: https://issues.apache.org/jira/browse/SPARK-3829

[jira] [Commented] (SPARK-3829) Make Spark logo image on the header of HistoryPage as a link to HistoryPage's page #1

[ https://issues.apache.org/jira/browse/SPARK-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161566#comment-14161566 ] Apache Spark commented on SPARK-3829: - User 'sarutak' has created a pull request for t

[jira] [Commented] (SPARK-3270) Spark API for Application Extensions

[ https://issues.apache.org/jira/browse/SPARK-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161585#comment-14161585 ] Apache Spark commented on SPARK-3270: - User 'mmalohlava' has created a pull request fo

[jira] [Commented] (SPARK-3797) Run the shuffle service inside the YARN NodeManager as an AuxiliaryService

[ https://issues.apache.org/jira/browse/SPARK-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161636#comment-14161636 ] Sandy Ryza commented on SPARK-3797: --- Not necessarily opposed to this, but wanted to brin

[jira] [Commented] (SPARK-3797) Run the shuffle service inside the YARN NodeManager as an AuxiliaryService

[ https://issues.apache.org/jira/browse/SPARK-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161703#comment-14161703 ] Reynold Xin commented on SPARK-3797: What if we support using AuxiliaryService to run

[jira] [Commented] (SPARK-3434) Distributed block matrix

[ https://issues.apache.org/jira/browse/SPARK-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161762#comment-14161762 ] Ghousia Taj commented on SPARK-3434: Hi There, We at Impetus Infotech, are also worki

[jira] [Created] (SPARK-3830) Implement genetic algorithms in MLLib

Egor Pakhomov created SPARK-3830: Summary: Implement genetic algorithms in MLLib Key: SPARK-3830 URL: https://issues.apache.org/jira/browse/SPARK-3830 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-3831) Filter rule Improvement and bool expression optimization.

Kousuke Saruta created SPARK-3831: - Summary: Filter rule Improvement and bool expression optimization. Key: SPARK-3831 URL: https://issues.apache.org/jira/browse/SPARK-3831 Project: Spark Iss

[jira] [Updated] (SPARK-3831) Filter rule Improvement and bool expression optimization.

[ https://issues.apache.org/jira/browse/SPARK-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3831: -- Description: If we write the filter which is always FALSE like {code} SELECT * from person WHER

[jira] [Commented] (SPARK-3831) Filter rule Improvement and bool expression optimization.

[ https://issues.apache.org/jira/browse/SPARK-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161847#comment-14161847 ] Apache Spark commented on SPARK-3831: - User 'sarutak' has created a pull request for t

[jira] [Commented] (SPARK-3797) Run the shuffle service inside the YARN NodeManager as an AuxiliaryService

[ https://issues.apache.org/jira/browse/SPARK-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161891#comment-14161891 ] Thomas Graves commented on SPARK-3797: -- Nice write up Sandy. Just to point out on

[jira] [Commented] (SPARK-3630) Identify cause of Kryo+Snappy PARSING_ERROR

[ https://issues.apache.org/jira/browse/SPARK-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161900#comment-14161900 ] DB Tsai commented on SPARK-3630: We also see similar issue when we perform map -> reduceBy

[jira] [Comment Edited] (SPARK-3630) Identify cause of Kryo+Snappy PARSING_ERROR

[ https://issues.apache.org/jira/browse/SPARK-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161900#comment-14161900 ] DB Tsai edited comment on SPARK-3630 at 10/7/14 2:07 PM: - We also

[jira] [Comment Edited] (SPARK-3630) Identify cause of Kryo+Snappy PARSING_ERROR

[ https://issues.apache.org/jira/browse/SPARK-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161900#comment-14161900 ] DB Tsai edited comment on SPARK-3630 at 10/7/14 2:08 PM: - We also

[jira] [Comment Edited] (SPARK-3630) Identify cause of Kryo+Snappy PARSING_ERROR

[ https://issues.apache.org/jira/browse/SPARK-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161900#comment-14161900 ] DB Tsai edited comment on SPARK-3630 at 10/7/14 2:07 PM: - We also

[jira] [Updated] (SPARK-3831) Filter rule Improvement and bool expression optimization.

[ https://issues.apache.org/jira/browse/SPARK-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-3831: Component/s: SQL > Filter rule Improvement and bool expression optimization. > -

[jira] [Created] (SPARK-3832) Upgrade Breeze dependency to 0.10

DB Tsai created SPARK-3832: -- Summary: Upgrade Breeze dependency to 0.10 Key: SPARK-3832 URL: https://issues.apache.org/jira/browse/SPARK-3832 Project: Spark Issue Type: Task Components: ML

[jira] [Commented] (SPARK-3832) Upgrade Breeze dependency to 0.10

[ https://issues.apache.org/jira/browse/SPARK-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161929#comment-14161929 ] Apache Spark commented on SPARK-3832: - User 'dbtsai' has created a pull request for th

[jira] [Resolved] (SPARK-3627) spark on yarn reports success even though job fails

[ https://issues.apache.org/jira/browse/SPARK-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3627. -- Resolution: Fixed Fix Version/s: 1.2.0 > spark on yarn reports success even though job fa

[jira] [Updated] (SPARK-3833) Allow Spark SQL SchemaRDDs to be merged

[ https://issues.apache.org/jira/browse/SPARK-3833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Wood updated SPARK-3833: -- Summary: Allow Spark SQL SchemaRDDs to be merged (was: Allow Spark SQL SchemaRDDs to me berged) > Allo

[jira] [Created] (SPARK-3833) Allow Spark SQL SchemaRDDs to me berged

Chris Wood created SPARK-3833: - Summary: Allow Spark SQL SchemaRDDs to me berged Key: SPARK-3833 URL: https://issues.apache.org/jira/browse/SPARK-3833 Project: Spark Issue Type: Wish Co

[jira] [Commented] (SPARK-3803) ArrayIndexOutOfBoundsException found in executing computePrincipalComponents

[ https://issues.apache.org/jira/browse/SPARK-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161982#comment-14161982 ] Masaru Dobashi commented on SPARK-3803: --- Thank you for your comments. I agree with t

[jira] [Comment Edited] (SPARK-3803) ArrayIndexOutOfBoundsException found in executing computePrincipalComponents

[ https://issues.apache.org/jira/browse/SPARK-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161982#comment-14161982 ] Masaru Dobashi edited comment on SPARK-3803 at 10/7/14 3:15 PM:

[jira] [Comment Edited] (SPARK-3803) ArrayIndexOutOfBoundsException found in executing computePrincipalComponents

[ https://issues.apache.org/jira/browse/SPARK-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161982#comment-14161982 ] Masaru Dobashi edited comment on SPARK-3803 at 10/7/14 3:15 PM:

[jira] [Comment Edited] (SPARK-3803) ArrayIndexOutOfBoundsException found in executing computePrincipalComponents

[ https://issues.apache.org/jira/browse/SPARK-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161982#comment-14161982 ] Masaru Dobashi edited comment on SPARK-3803 at 10/7/14 3:14 PM:

[jira] [Updated] (SPARK-3731) RDD caching stops working in pyspark after some time

[ https://issues.apache.org/jira/browse/SPARK-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-3731: -- Target Version/s: 1.1.1, 1.2.0, 1.0.3 (was: 1.1.1, 1.2.0) > RDD caching stops working in pyspark after

[jira] [Updated] (SPARK-3731) RDD caching stops working in pyspark after some time

[ https://issues.apache.org/jira/browse/SPARK-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-3731: -- Affects Version/s: 1.0.0 > RDD caching stops working in pyspark after some time > --

[jira] [Updated] (SPARK-3731) RDD caching stops working in pyspark after some time

[ https://issues.apache.org/jira/browse/SPARK-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-3731: -- Affects Version/s: (was: 1.0.0) 1.0.2 > RDD caching stops working in pyspark

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162013#comment-14162013 ] Oleg Zhurakousky commented on SPARK-3561: - Patrick, your point about confusion wit

[jira] [Commented] (SPARK-3420) Using Sphinx to generate API docs for PySpark

[ https://issues.apache.org/jira/browse/SPARK-3420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162098#comment-14162098 ] Davies Liu commented on SPARK-3420: --- Both of them, I will retire epydoc in PR #2624, af

[jira] [Commented] (SPARK-3434) Distributed block matrix

[ https://issues.apache.org/jira/browse/SPARK-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162156#comment-14162156 ] Xiangrui Meng commented on SPARK-3434: -- [~shivaram] and [~ConcreteVitamin] Any update

[jira] [Updated] (SPARK-1297) Upgrade HBase dependency to 0.98.0

[ https://issues.apache.org/jira/browse/SPARK-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-1297: -- Attachment: spark-1297-v6.txt Patch v6 uses 0.98.5 hbase release. > Upgrade HBase dependency to 0.98.0 > --

[jira] [Commented] (SPARK-3819) Jenkins should compile Spark against multiple versions of Hadoop

[ https://issues.apache.org/jira/browse/SPARK-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162195#comment-14162195 ] Matt Cheah commented on SPARK-3819: --- Can you elaborate as to why it is not feasible to b

[jira] [Commented] (SPARK-3797) Run the shuffle service inside the YARN NodeManager as an AuxiliaryService

[ https://issues.apache.org/jira/browse/SPARK-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162254#comment-14162254 ] Andrew Or commented on SPARK-3797: -- Thanks for detailing the considerations Sandy. I agre

[jira] [Resolved] (SPARK-3827) Very long RDD names are not rendered properly in web UI

[ https://issues.apache.org/jira/browse/SPARK-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3827. --- Resolution: Fixed Fix Version/s: 1.1.1 Issue resolved by pull request 2687 [https://github.com/

[jira] [Resolved] (SPARK-2915) Storage summary table UI glitch when using sparkSQL

[ https://issues.apache.org/jira/browse/SPARK-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2915. --- Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Assignee: Hossein Falak

[jira] [Resolved] (SPARK-3297) [Spark SQL][UI] SchemaRDD toString with many columns messes up Storage tab display

[ https://issues.apache.org/jira/browse/SPARK-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3297. --- Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Assignee: Hossein Falak

[jira] [Updated] (SPARK-3808) PySpark fails to start in Windows

[ https://issues.apache.org/jira/browse/SPARK-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3808: - Affects Version/s: (was: 1.1.0) 1.2.0 > PySpark fails to start in Windows > ---

[jira] [Closed] (SPARK-3808) PySpark fails to start in Windows

[ https://issues.apache.org/jira/browse/SPARK-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3808. Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Masayoshi TSUZUKI Target Versio

[jira] [Commented] (SPARK-3761) Class anonfun$1 not found exception / sbt 13.x / Scala 2.10.4

[ https://issues.apache.org/jira/browse/SPARK-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162307#comment-14162307 ] Igor Tkachenko commented on SPARK-3761: --- After I've added line sc.addJar("") it work

[jira] [Closed] (SPARK-3761) Class anonfun$1 not found exception / sbt 13.x / Scala 2.10.4

[ https://issues.apache.org/jira/browse/SPARK-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Tkachenko closed SPARK-3761. - Resolution: Fixed > Class anonfun$1 not found exception / sbt 13.x / Scala 2.10.4 > --

[jira] [Commented] (SPARK-3808) PySpark fails to start in Windows

[ https://issues.apache.org/jira/browse/SPARK-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162310#comment-14162310 ] Andrew Or commented on SPARK-3808: -- Hey [~tsudukim] can you verify that pyspark, spark-sh

[jira] [Closed] (SPARK-3732) Yarn Client: Add option to NOT System.exit() at end of main()

[ https://issues.apache.org/jira/browse/SPARK-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sotos Matzanas closed SPARK-3732. - Resolution: Won't Fix > Yarn Client: Add option to NOT System.exit() at end of main() > --

[jira] [Resolved] (SPARK-3762) clear all SparkEnv references after stop

[ https://issues.apache.org/jira/browse/SPARK-3762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-3762. -- Resolution: Fixed Fix Version/s: 1.2.0 > clear all SparkEnv references after stop > -

[jira] [Updated] (SPARK-3731) RDD caching stops working in pyspark after some time

[ https://issues.apache.org/jira/browse/SPARK-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3731: -- Affects Version/s: (was: 1.0.2) 1.2.0 > RDD caching stops working in pyspark

[jira] [Updated] (SPARK-3731) RDD caching stops working in pyspark after some time

[ https://issues.apache.org/jira/browse/SPARK-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3731: -- Target Version/s: 1.1.1, 1.2.0 (was: 1.1.1, 1.2.0, 1.0.3) > RDD caching stops working in pyspark after

[jira] [Updated] (SPARK-3731) RDD caching stops working in pyspark after some time

[ https://issues.apache.org/jira/browse/SPARK-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3731: -- Affects Version/s: 1.0.2 > RDD caching stops working in pyspark after some time > --

[jira] [Commented] (SPARK-3797) Run the shuffle service inside the YARN NodeManager as an AuxiliaryService

[ https://issues.apache.org/jira/browse/SPARK-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162351#comment-14162351 ] Patrick Wendell commented on SPARK-3797: For the dependencies issue - the plan is

[jira] [Updated] (SPARK-3765) Add test information to sbt build docs

[ https://issues.apache.org/jira/browse/SPARK-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3765: --- Fix Version/s: 1.2.0 > Add test information to sbt build docs > --

[jira] [Resolved] (SPARK-3765) Add test information to sbt build docs

[ https://issues.apache.org/jira/browse/SPARK-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3765. Resolution: Fixed Assignee: wangfei This was resolved by: https://github.com/apache/sp

[jira] [Commented] (SPARK-3828) Spark returns inconsistent results when building with different Hadoop version

[ https://issues.apache.org/jira/browse/SPARK-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162374#comment-14162374 ] Liquan Pei commented on SPARK-3828: --- It seems that this is a bug in LineRecordReader. Fo

[jira] [Comment Edited] (SPARK-3828) Spark returns inconsistent results when building with different Hadoop version

[ https://issues.apache.org/jira/browse/SPARK-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162374#comment-14162374 ] Liquan Pei edited comment on SPARK-3828 at 10/7/14 7:33 PM: It

[jira] [Resolved] (SPARK-2582) Make Block Manager Master pluggable

[ https://issues.apache.org/jira/browse/SPARK-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2582. Resolution: Won't Fix I closed this PR a long time ago - but didn't close the asscicated JIR

[jira] [Created] (SPARK-3834) Backticks not correctly handled in subquery aliases

Michael Armbrust created SPARK-3834: --- Summary: Backticks not correctly handled in subquery aliases Key: SPARK-3834 URL: https://issues.apache.org/jira/browse/SPARK-3834 Project: Spark Issue

[jira] [Created] (SPARK-3835) Spark applications that are killed should show up as "KILLED" or "CANCELLED" in the Spark UI

Matt Cheah created SPARK-3835: - Summary: Spark applications that are killed should show up as "KILLED" or "CANCELLED" in the Spark UI Key: SPARK-3835 URL: https://issues.apache.org/jira/browse/SPARK-3835

[jira] [Commented] (SPARK-3505) Augmenting SparkStreaming updateStateByKey API with timestamp

[ https://issues.apache.org/jira/browse/SPARK-3505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162412#comment-14162412 ] Apache Spark commented on SPARK-3505: - User 'xiliu82' has created a pull request for t

[jira] [Commented] (SPARK-2960) Spark executables fail to start via symlinks

[ https://issues.apache.org/jira/browse/SPARK-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162408#comment-14162408 ] Apache Spark commented on SPARK-2960: - User 'roji' has created a pull request for this

[jira] [Commented] (SPARK-2017) web ui stage page becomes unresponsive when the number of tasks is large

[ https://issues.apache.org/jira/browse/SPARK-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162404#comment-14162404 ] Apache Spark commented on SPARK-2017: - User 'carlosfuertes' has created a pull request

[jira] [Commented] (SPARK-2803) add Kafka stream feature for fetch messages from specified starting offset position

[ https://issues.apache.org/jira/browse/SPARK-2803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162402#comment-14162402 ] Apache Spark commented on SPARK-2803: - User 'pengyanhong' has created a pull request f

[jira] [Commented] (SPARK-2805) update akka to version 2.3

[ https://issues.apache.org/jira/browse/SPARK-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162406#comment-14162406 ] Apache Spark commented on SPARK-2805: - User 'avati' has created a pull request for thi

[jira] [Commented] (SPARK-3251) Clarify learning interfaces

[ https://issues.apache.org/jira/browse/SPARK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162410#comment-14162410 ] Apache Spark commented on SPARK-3251: - User 'BigCrunsh' has created a pull request for

[jira] [Commented] (SPARK-3809) make HiveThriftServer2Suite work correctly

[ https://issues.apache.org/jira/browse/SPARK-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162418#comment-14162418 ] Apache Spark commented on SPARK-3809: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-3338) Respect user setting of spark.submit.pyFiles

[ https://issues.apache.org/jira/browse/SPARK-3338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162411#comment-14162411 ] Apache Spark commented on SPARK-3338: - User 'andrewor14' has created a pull request fo

[jira] [Commented] (SPARK-2016) rdd in-memory storage UI becomes unresponsive when the number of RDD partitions is large

[ https://issues.apache.org/jira/browse/SPARK-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162405#comment-14162405 ] Apache Spark commented on SPARK-2016: - User 'carlosfuertes' has created a pull request

[jira] [Commented] (SPARK-3812) Adapt maven build to publish effective pom.

[ https://issues.apache.org/jira/browse/SPARK-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162417#comment-14162417 ] Apache Spark commented on SPARK-3812: - User 'ScrapCodes' has created a pull request fo

[jira] [Commented] (SPARK-3166) Custom serialisers can't be shipped in application jars

[ https://issues.apache.org/jira/browse/SPARK-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162409#comment-14162409 ] Apache Spark commented on SPARK-3166: - User 'GrahamDennis' has created a pull request

[jira] [Commented] (SPARK-2489) Unsupported parquet datatype optional fixed_len_byte_array

[ https://issues.apache.org/jira/browse/SPARK-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162407#comment-14162407 ] Apache Spark commented on SPARK-2489: - User 'joesu' has created a pull request for thi

[jira] [Commented] (SPARK-3580) Add Consistent Method To Get Number of RDD Partitions Across Different Languages

[ https://issues.apache.org/jira/browse/SPARK-3580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162413#comment-14162413 ] Apache Spark commented on SPARK-3580: - User 'patmcdonough' has created a pull request

[jira] [Commented] (SPARK-3790) CosineSimilarity via DIMSUM example

[ https://issues.apache.org/jira/browse/SPARK-3790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162415#comment-14162415 ] Apache Spark commented on SPARK-3790: - User 'rezazadeh' has created a pull request for

[jira] [Commented] (SPARK-2759) The ability to read binary files into Spark

[ https://issues.apache.org/jira/browse/SPARK-2759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162403#comment-14162403 ] Apache Spark commented on SPARK-2759: - User 'kmader' has created a pull request for th

[jira] [Commented] (SPARK-3816) Add configureOutputJobPropertiesForStorageHandler to JobConf in SparkHadoopWriter

[ https://issues.apache.org/jira/browse/SPARK-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162420#comment-14162420 ] Apache Spark commented on SPARK-3816: - User 'alexliu68' has created a pull request for

[jira] [Closed] (SPARK-3825) Log more information when unrolling a block fails

[ https://issues.apache.org/jira/browse/SPARK-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3825. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 > Log more information when unrolling

[jira] [Reopened] (SPARK-3828) Spark returns inconsistent results when building with different Hadoop version

[ https://issues.apache.org/jira/browse/SPARK-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reopened SPARK-3828: -- > Spark returns inconsistent results when building with different Hadoop > version > -

[jira] [Commented] (SPARK-3828) Spark returns inconsistent results when building with different Hadoop version

[ https://issues.apache.org/jira/browse/SPARK-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162439#comment-14162439 ] Xiangrui Meng commented on SPARK-3828: -- I re-opened this because it may be a serious

[jira] [Resolved] (SPARK-3731) RDD caching stops working in pyspark after some time

[ https://issues.apache.org/jira/browse/SPARK-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3731. --- Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Fixed by Davies' PR, which I ba

[jira] [Created] (SPARK-3836) Spark REPL optionally propagate internal exceptions

Ahir Reddy created SPARK-3836: - Summary: Spark REPL optionally propagate internal exceptions Key: SPARK-3836 URL: https://issues.apache.org/jira/browse/SPARK-3836 Project: Spark Issue Type: Impr

[jira] [Commented] (SPARK-3836) Spark REPL optionally propagate internal exceptions

[ https://issues.apache.org/jira/browse/SPARK-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162470#comment-14162470 ] Apache Spark commented on SPARK-3836: - User 'ahirreddy' has created a pull request for

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162514#comment-14162514 ] Marcelo Vanzin commented on SPARK-3174: --- Hi Andrew, thanks for writing this up. My

[jira] [Commented] (SPARK-3461) Support external groupByKey using repartitionAndSortWithinPartitions

[ https://issues.apache.org/jira/browse/SPARK-3461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162547#comment-14162547 ] Davies Liu commented on SPARK-3461: --- [~pwendell] I will start to work on this after merg

[jira] [Created] (SPARK-3837) Warn when YARN is killing containers for exceeding memory limits

Sandy Ryza created SPARK-3837: - Summary: Warn when YARN is killing containers for exceeding memory limits Key: SPARK-3837 URL: https://issues.apache.org/jira/browse/SPARK-3837 Project: Spark Iss

[jira] [Commented] (SPARK-3637) NPE in ShuffleMapTask

[ https://issues.apache.org/jira/browse/SPARK-3637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162600#comment-14162600 ] Steven Lewis commented on SPARK-3637: - I see the same thing running a Java Map reduce

[jira] [Commented] (SPARK-2321) Design a proper progress reporting & event listener API

[ https://issues.apache.org/jira/browse/SPARK-2321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162614#comment-14162614 ] Apache Spark commented on SPARK-2321: - User 'JoshRosen' has created a pull request for

[jira] [Commented] (SPARK-2321) Design a proper progress reporting & event listener API

[ https://issues.apache.org/jira/browse/SPARK-2321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162615#comment-14162615 ] Josh Rosen commented on SPARK-2321: --- I've opened a WIP pull request in order to discuss

[jira] [Updated] (SPARK-3682) Add helpful warnings to the UI

[ https://issues.apache.org/jira/browse/SPARK-3682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-3682: -- Attachment: SPARK-3682Design.pdf Posting an initial design > Add helpful warnings to the UI > -

[jira] [Commented] (SPARK-3785) Support off-loading computations to a GPU

[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162662#comment-14162662 ] Reza Farivar commented on SPARK-3785: - I thought to add that the project Sumatra might

[jira] [Comment Edited] (SPARK-3785) Support off-loading computations to a GPU

[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162662#comment-14162662 ] Reza Farivar edited comment on SPARK-3785 at 10/7/14 10:13 PM: -

[jira] [Comment Edited] (SPARK-3785) Support off-loading computations to a GPU

[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162662#comment-14162662 ] Reza Farivar edited comment on SPARK-3785 at 10/7/14 10:12 PM: -

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162663#comment-14162663 ] Nicholas Chammas commented on SPARK-3821: - [~shivaram] / [~pwendell]: # In a Spark

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162685#comment-14162685 ] Sandy Ryza commented on SPARK-3174: --- bq. Maybe it makes sense to just call it `spark.dy

[jira] [Created] (SPARK-3838) Python code example for Word2Vec in user guide

Xiangrui Meng created SPARK-3838: Summary: Python code example for Word2Vec in user guide Key: SPARK-3838 URL: https://issues.apache.org/jira/browse/SPARK-3838 Project: Spark Issue Type: Sub-

[jira] [Created] (SPARK-3839) Reimplement HashOuterJoin to construct hash table of only one relation

Liquan Pei created SPARK-3839: - Summary: Reimplement HashOuterJoin to construct hash table of only one relation Key: SPARK-3839 URL: https://issues.apache.org/jira/browse/SPARK-3839 Project: Spark

[jira] [Updated] (SPARK-3661) spark.*.memory is ignored in cluster mode

[ https://issues.apache.org/jira/browse/SPARK-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3661: - Summary: spark.*.memory is ignored in cluster mode (was: spark.driver.memory is ignored in cluster mode)

[jira] [Updated] (SPARK-3661) spark.*.memory is ignored in cluster mode

[ https://issues.apache.org/jira/browse/SPARK-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3661: - Description: This is related to https://issues.apache.org/jira/browse/SPARK-3653, but for the config. Note

[jira] [Updated] (SPARK-3661) spark.*.memory is ignored in cluster mode

[ https://issues.apache.org/jira/browse/SPARK-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3661: - Description: This is related to https://issues.apache.org/jira/browse/SPARK-3653, but for the config. Note

[jira] [Commented] (SPARK-3661) spark.*.memory is ignored in cluster mode

[ https://issues.apache.org/jira/browse/SPARK-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162721#comment-14162721 ] Apache Spark commented on SPARK-3661: - User 'andrewor14' has created a pull request fo

[jira] [Created] (SPARK-3840) Spark EC2 templates fail when variables are missing

Allan Douglas R. de Oliveira created SPARK-3840: --- Summary: Spark EC2 templates fail when variables are missing Key: SPARK-3840 URL: https://issues.apache.org/jira/browse/SPARK-3840 Projec

[jira] [Updated] (SPARK-3777) Display "Executor ID" for Tasks in Stage page

[ https://issues.apache.org/jira/browse/SPARK-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3777: - Assignee: Shixiong Zhu > Display "Executor ID" for Tasks in Stage page > -

[jira] [Closed] (SPARK-3777) Display "Executor ID" for Tasks in Stage page

[ https://issues.apache.org/jira/browse/SPARK-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3777. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Target Version/s: 1.1.1,

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162755#comment-14162755 ] Shivaram Venkataraman commented on SPARK-3821: -- 1. Yes - the same stuff is in

[jira] [Commented] (SPARK-3654) Implement all extended HiveQL statements/commands with a separate parser combinator

[ https://issues.apache.org/jira/browse/SPARK-3654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162757#comment-14162757 ] Apache Spark commented on SPARK-3654: - User 'liancheng' has created a pull request for

  1   2   >