[jira] [Commented] (SPARK-4873) WriteAheadLogBasedBlockHandler improvement

[ https://issues.apache.org/jira/browse/SPARK-4873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14249716#comment-14249716 ] Apache Spark commented on SPARK-4873: - User 'zsxwing' has created a pull request for

[jira] [Commented] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14249752#comment-14249752 ] Alister Lee commented on SPARK-3533: The stackoverflow question has a good answer (I

[jira] [Commented] (SPARK-1812) Support cross-building with Scala 2.11

[ https://issues.apache.org/jira/browse/SPARK-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14250023#comment-14250023 ] Tobias Bertelsen commented on SPARK-1812: - [~schmmd] Looks like [~pwendell]

[jira] [Comment Edited] (SPARK-1812) Support cross-building with Scala 2.11

[ https://issues.apache.org/jira/browse/SPARK-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14250023#comment-14250023 ] Tobias Bertelsen edited comment on SPARK-1812 at 12/17/14 3:54 PM:

[jira] [Commented] (SPARK-4417) New API: sample RDD to fixed number of items

[ https://issues.apache.org/jira/browse/SPARK-4417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14250260#comment-14250260 ] Apache Spark commented on SPARK-4417: - User 'ilganeli' has created a pull request for

[jira] [Created] (SPARK-4874) Report number of records read/written in a task

Kostas Sakellis created SPARK-4874: -- Summary: Report number of records read/written in a task Key: SPARK-4874 URL: https://issues.apache.org/jira/browse/SPARK-4874 Project: Spark Issue

[jira] [Commented] (SPARK-4874) Report number of records read/written in a task

[ https://issues.apache.org/jira/browse/SPARK-4874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14250304#comment-14250304 ] Kostas Sakellis commented on SPARK-4874: I'm working on this. Report number of

[jira] [Commented] (SPARK-3698) Case sensitive check in spark sql is incompleted.

[ https://issues.apache.org/jira/browse/SPARK-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14250323#comment-14250323 ] Apache Spark commented on SPARK-3698: - User 'marmbrus' has created a pull request for

[jira] [Created] (SPARK-4875) Separate Transformer, Estimator params

Joseph K. Bradley created SPARK-4875: Summary: Separate Transformer, Estimator params Key: SPARK-4875 URL: https://issues.apache.org/jira/browse/SPARK-4875 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-4595) Spark MetricsServlet is not worked because of initialization ordering

[ https://issues.apache.org/jira/browse/SPARK-4595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-4595. --- Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Assignee: Saisai Shao

[jira] [Resolved] (SPARK-4625) Support Sort By in both DSL SimpleSQLParser

[ https://issues.apache.org/jira/browse/SPARK-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4625. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3481

[jira] [Commented] (SPARK-4595) Spark MetricsServlet is not worked because of initialization ordering

[ https://issues.apache.org/jira/browse/SPARK-4595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14250432#comment-14250432 ] Josh Rosen commented on SPARK-4595: --- It looks like this bug was introduced in

[jira] [Resolved] (SPARK-4750) Dynamic allocation - we need to synchronize kills

[ https://issues.apache.org/jira/browse/SPARK-4750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-4750. --- Resolution: Fixed Fix Version/s: 1.2.1 I've backported this into {{branch-1.2}} for inclusion

[jira] [Updated] (SPARK-4750) Dynamic allocation - we need to synchronize kills

[ https://issues.apache.org/jira/browse/SPARK-4750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4750: -- Labels: (was: backport-needed) Dynamic allocation - we need to synchronize kills

[jira] [Updated] (SPARK-3926) result of JavaRDD collectAsMap() is not serializable

[ https://issues.apache.org/jira/browse/SPARK-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3926: -- Labels: (was: backport-needed) result of JavaRDD collectAsMap() is not serializable

[jira] [Resolved] (SPARK-3926) result of JavaRDD collectAsMap() is not serializable

[ https://issues.apache.org/jira/browse/SPARK-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3926. --- Resolution: Fixed Fix Version/s: (was: 1.1.2) 1.2.1 I've merged this

[jira] [Updated] (SPARK-4691) Restructure a few lines in shuffle code

[ https://issues.apache.org/jira/browse/SPARK-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4691: -- Labels: (was: backport-needed) Restructure a few lines in shuffle code

[jira] [Resolved] (SPARK-4691) Restructure a few lines in shuffle code

[ https://issues.apache.org/jira/browse/SPARK-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-4691. --- Resolution: Fixed Fix Version/s: 1.2.1 I've merged this into branch-1.2 for inclusion in

[jira] [Updated] (SPARK-4714) BlockManager should check whether blocks have already been removed Checking block is null or not after having gotten info.lock in remove block method

[ https://issues.apache.org/jira/browse/SPARK-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4714: -- Labels: (was: backport-needed) BlockManager should check whether blocks have already been removed

[jira] [Created] (SPARK-4877) userClassPathFirst doesn't handle user classes inheriting from parent

Stephen Haberman created SPARK-4877: --- Summary: userClassPathFirst doesn't handle user classes inheriting from parent Key: SPARK-4877 URL: https://issues.apache.org/jira/browse/SPARK-4877 Project:

[jira] [Resolved] (SPARK-4772) Accumulators leak memory, both temporarily and permanently

[ https://issues.apache.org/jira/browse/SPARK-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-4772. --- Resolution: Fixed Fix Version/s: 1.2.1 Accumulators leak memory, both temporarily and

[jira] [Updated] (SPARK-4772) Accumulators leak memory, both temporarily and permanently

[ https://issues.apache.org/jira/browse/SPARK-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4772: -- Labels: accumulators (was: accumulators backport-needed) Accumulators leak memory, both temporarily

[jira] [Commented] (SPARK-4772) Accumulators leak memory, both temporarily and permanently

[ https://issues.apache.org/jira/browse/SPARK-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14250460#comment-14250460 ] Josh Rosen commented on SPARK-4772: --- I've merged this into {{branch-1.2}}, so this will

[jira] [Resolved] (SPARK-4841) Batch serializer bug in PySpark's RDD.zip

[ https://issues.apache.org/jira/browse/SPARK-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-4841. --- Resolution: Fixed Fix Version/s: 1.2.1 Target Version/s: (was: 1.2.1) I've merged

[jira] [Updated] (SPARK-4876) An exception thrown when accessing a Spark SQL table using a JDBC driver from a standalone app.

[ https://issues.apache.org/jira/browse/SPARK-4876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonid Mikhailov updated SPARK-4876: Description: I am running Spark version 1.1.1 (built it on Mac using: mvn -Pyarn

[jira] [Updated] (SPARK-3698) Case sensitive check in spark sql is incompleted.

[ https://issues.apache.org/jira/browse/SPARK-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-3698: Assignee: Michael Armbrust Case sensitive check in spark sql is incompleted.

[jira] [Resolved] (SPARK-4493) Don't pushdown Eq, NotEq, Lt, LtEq, Gt and GtEq predicates with nulls for Parquet

[ https://issues.apache.org/jira/browse/SPARK-4493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4493. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3367

[jira] [Updated] (SPARK-4493) Don't pushdown Eq, NotEq, Lt, LtEq, Gt and GtEq predicates with nulls for Parquet

[ https://issues.apache.org/jira/browse/SPARK-4493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-4493: Assignee: Cheng Lian Don't pushdown Eq, NotEq, Lt, LtEq, Gt and GtEq predicates with nulls

[jira] [Resolved] (SPARK-4755) SQRT(negative value) should return null

[ https://issues.apache.org/jira/browse/SPARK-4755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4755. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3616

[jira] [Commented] (SPARK-4790) Flaky test in ReceivedBlockTrackerSuite: block addition, block to batch allocation, and cleanup with write ahead log

[ https://issues.apache.org/jira/browse/SPARK-4790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14250525#comment-14250525 ] Hari Shreedharan commented on SPARK-4790: - Ah, so this is the issue:

[jira] [Commented] (SPARK-4790) Flaky test in ReceivedBlockTrackerSuite: block addition, block to batch allocation, and cleanup with write ahead log

[ https://issues.apache.org/jira/browse/SPARK-4790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14250610#comment-14250610 ] Apache Spark commented on SPARK-4790: - User 'harishreedharan' has created a pull

[jira] [Commented] (SPARK-4296) Throw Expression not in GROUP BY when using same expression in group by clause and select clause

[ https://issues.apache.org/jira/browse/SPARK-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14250611#comment-14250611 ] David Ross commented on SPARK-4296: --- I can still reproduce this issue. The test case

[jira] [Resolved] (SPARK-3739) Too many splits for small source file in table scanning

[ https://issues.apache.org/jira/browse/SPARK-3739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3739. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 2589

[jira] [Commented] (SPARK-4296) Throw Expression not in GROUP BY when using same expression in group by clause and select clause

[ https://issues.apache.org/jira/browse/SPARK-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14250654#comment-14250654 ] Michael Armbrust commented on SPARK-4296: - David, is that using Spark 1.2? Throw

[jira] [Resolved] (SPARK-4821) pyspark.mllib.rand docs not generated correctly

[ https://issues.apache.org/jira/browse/SPARK-4821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-4821. -- Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Issue resolved by pull

[jira] [Updated] (SPARK-4821) pyspark.mllib.rand docs not generated correctly

[ https://issues.apache.org/jira/browse/SPARK-4821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4821: - Target Version/s: 1.3.0, 1.2.1 (was: 1.2.0) pyspark.mllib.rand docs not generated correctly

[jira] [Commented] (SPARK-4296) Throw Expression not in GROUP BY when using same expression in group by clause and select clause

[ https://issues.apache.org/jira/browse/SPARK-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14250674#comment-14250674 ] David Ross commented on SPARK-4296: --- Hi Michael, We are trunk: {{1.3.0-SNAPSHOT}}, as of

[jira] [Reopened] (SPARK-4296) Throw Expression not in GROUP BY when using same expression in group by clause and select clause

[ https://issues.apache.org/jira/browse/SPARK-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reopened SPARK-4296: - Assignee: Cheng Lian Throw Expression not in GROUP BY when using same expression in

[jira] [Closed] (SPARK-4875) Separate Transformer, Estimator params

[ https://issues.apache.org/jira/browse/SPARK-4875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley closed SPARK-4875. Resolution: Duplicate Separate Transformer, Estimator params

[jira] [Created] (SPARK-4878) driverPropsFetcher causes spurious Akka disassociate errors

Stephen Haberman created SPARK-4878: --- Summary: driverPropsFetcher causes spurious Akka disassociate errors Key: SPARK-4878 URL: https://issues.apache.org/jira/browse/SPARK-4878 Project: Spark

[jira] [Commented] (SPARK-4878) driverPropsFetcher causes spurious Akka disassociate errors

[ https://issues.apache.org/jira/browse/SPARK-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14250711#comment-14250711 ] Stephen Haberman commented on SPARK-4878: - Here are some of the log messages in

[jira] [Resolved] (SPARK-4856) Null empty string should not be considered as StringType at begining in Json schema inferring

[ https://issues.apache.org/jira/browse/SPARK-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4856. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3708

[jira] [Updated] (SPARK-4856) Null empty string should not be considered as StringType at begining in Json schema inferring

[ https://issues.apache.org/jira/browse/SPARK-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-4856: Assignee: Cheng Hao Null empty string should not be considered as StringType at begining

[jira] [Commented] (SPARK-4069) [SPARK-YARN] ApplicationMaster should release all executors' containers before unregistering itself from Yarn RM

[ https://issues.apache.org/jira/browse/SPARK-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14250762#comment-14250762 ] David McWhorter commented on SPARK-4069: Seeing the same behavior, a spark

[jira] [Resolved] (SPARK-3891) Support Hive Percentile UDAF with array of percentile values

[ https://issues.apache.org/jira/browse/SPARK-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3891. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 2802

[jira] [Updated] (SPARK-4854) Custom UDTF with Lateral View throws ClassNotFound exception in Spark SQL CLI

[ https://issues.apache.org/jira/browse/SPARK-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-4854: Target Version/s: 1.3.0 Custom UDTF with Lateral View throws ClassNotFound exception in

[jira] [Updated] (SPARK-4006) Spark Driver crashes whenever an Executor is registered twice

[ https://issues.apache.org/jira/browse/SPARK-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4006: - Fix Version/s: 1.0.3 Spark Driver crashes whenever an Executor is registered twice

[jira] [Updated] (SPARK-4874) Report number of records read/written in a task

[ https://issues.apache.org/jira/browse/SPARK-4874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-4874: -- Assignee: Kostas Sakellis Report number of records read/written in a task

[jira] [Commented] (SPARK-4140) Document the dynamic allocation feature

[ https://issues.apache.org/jira/browse/SPARK-4140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251305#comment-14251305 ] Apache Spark commented on SPARK-4140: - User 'andrewor14' has created a pull request

[jira] [Commented] (SPARK-2087) Clean Multi-user semantics for thrift JDBC/ODBC server.

[ https://issues.apache.org/jira/browse/SPARK-2087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251343#comment-14251343 ] guowei commented on SPARK-2087: --- I'm not sure that a full SQLContext per session is a good