[jira] [Commented] (SPARK-2356) Exception: Could not locate executable null\bin\winutils.exe in the Hadoop

[ https://issues.apache.org/jira/browse/SPARK-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096673#comment-14096673 ] Tarek Nabil commented on SPARK-2356: There is a nice workaround documented here: http

[jira] [Updated] (SPARK-3031) Create JsonSerializable and move JSON serialization from JsonProtocol into each class

[ https://issues.apache.org/jira/browse/SPARK-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3031: --- Component/s: Spark Core > Create JsonSerializable and move JSON serialization from JsonProtocol into

[jira] [Created] (SPARK-3031) Create JsonSerializable and move JSON serialization from JsonProtocol into each class

Reynold Xin created SPARK-3031: -- Summary: Create JsonSerializable and move JSON serialization from JsonProtocol into each class Key: SPARK-3031 URL: https://issues.apache.org/jira/browse/SPARK-3031 Proje

[jira] [Updated] (SPARK-3029) Disable local execution of Spark jobs by default

[ https://issues.apache.org/jira/browse/SPARK-3029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3029: --- Priority: Blocker (was: Major) Target Version/s: 1.1.0 > Disable local execution of Spar

[jira] [Commented] (SPARK-3031) Create JsonSerializable and move JSON serialization from JsonProtocol into each class

[ https://issues.apache.org/jira/browse/SPARK-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096664#comment-14096664 ] Reynold Xin commented on SPARK-3031: cc [~andrewor14] [~adav] > Create JsonSerializab

[jira] [Resolved] (SPARK-1170) Add histogram() to PySpark

[ https://issues.apache.org/jira/browse/SPARK-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan Kumar resolved SPARK-1170. -- Resolution: Duplicate Davies is working on this. > Add histogram() to PySpark > --

[jira] [Commented] (SPARK-3019) Pluggable block transfer (data plane communication) interface

[ https://issues.apache.org/jira/browse/SPARK-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096658#comment-14096658 ] Reynold Xin commented on SPARK-3019: Possibly, although I think MapR FS is more optimi

[jira] [Created] (SPARK-3030) reuse python worker

Davies Liu created SPARK-3030: - Summary: reuse python worker Key: SPARK-3030 URL: https://issues.apache.org/jira/browse/SPARK-3030 Project: Spark Issue Type: Improvement Components: PyS

[jira] [Commented] (SPARK-3019) Pluggable block transfer (data plane communication) interface

[ https://issues.apache.org/jira/browse/SPARK-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096654#comment-14096654 ] Hari Shreedharan commented on SPARK-3019: - Why specifically MapR FS? You could use

[jira] [Created] (SPARK-3029) Disable local execution of Spark jobs by default

Aaron Davidson created SPARK-3029: - Summary: Disable local execution of Spark jobs by default Key: SPARK-3029 URL: https://issues.apache.org/jira/browse/SPARK-3029 Project: Spark Issue Type:

[jira] [Updated] (SPARK-2456) Scheduler refactoring

[ https://issues.apache.org/jira/browse/SPARK-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2456: --- Component/s: Spark Core > Scheduler refactoring > - > > Key: SPAR

[jira] [Commented] (SPARK-3028) sparkEventToJson should support SparkListenerExecutorMetricsUpdate

[ https://issues.apache.org/jira/browse/SPARK-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096645#comment-14096645 ] Reynold Xin commented on SPARK-3028: [~sandyr] [~sandyryza] Would you be able to do th

[jira] [Created] (SPARK-3028) sparkEventToJson should support SparkListenerExecutorMetricsUpdate

Reynold Xin created SPARK-3028: -- Summary: sparkEventToJson should support SparkListenerExecutorMetricsUpdate Key: SPARK-3028 URL: https://issues.apache.org/jira/browse/SPARK-3028 Project: Spark

[jira] [Commented] (SPARK-2356) Exception: Could not locate executable null\bin\winutils.exe in the Hadoop

[ https://issues.apache.org/jira/browse/SPARK-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096635#comment-14096635 ] Tarek Nabil commented on SPARK-2356: Yes, but the whole point is that you should do no

[jira] [Commented] (SPARK-2356) Exception: Could not locate executable null\bin\winutils.exe in the Hadoop

[ https://issues.apache.org/jira/browse/SPARK-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096636#comment-14096636 ] Kostiantyn Kudriavtsev commented on SPARK-2356: --- Guoqiang, Spark works not e

[jira] [Updated] (SPARK-3027) Tighten visibility and provide Java friendly callback API in TaskContext

[ https://issues.apache.org/jira/browse/SPARK-3027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3027: --- Summary: Tighten visibility and provide Java friendly callback API in TaskContext (was: Tighten the

[jira] [Commented] (SPARK-3027) Tighten visibility and provide Java friendly callback API in TaskContext

[ https://issues.apache.org/jira/browse/SPARK-3027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096632#comment-14096632 ] Apache Spark commented on SPARK-3027: - User 'rxin' has created a pull request for this

[jira] [Created] (SPARK-3027) Tighten the visibility of fields in TaskContext and provide Java friendly callback API

Reynold Xin created SPARK-3027: -- Summary: Tighten the visibility of fields in TaskContext and provide Java friendly callback API Key: SPARK-3027 URL: https://issues.apache.org/jira/browse/SPARK-3027 Proj

[jira] [Commented] (SPARK-2926) Add MR-style (merge-sort) SortShuffleReader for sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096608#comment-14096608 ] Saisai Shao commented on SPARK-2926: Hi Matei, I just uploaded a Spark shuffle perfo

[jira] [Comment Edited] (SPARK-3005) Spark with Mesos fine-grained mode throws UnsupportedOperationException in MesosSchedulerBackend.killTask()

[ https://issues.apache.org/jira/browse/SPARK-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096539#comment-14096539 ] Xu Zhongxing edited comment on SPARK-3005 at 8/14/14 5:57 AM: --

[jira] [Updated] (SPARK-2926) Add MR-style (merge-sort) SortShuffleReader for sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-2926: --- Attachment: Spark Shuffle Test Report.pdf > Add MR-style (merge-sort) SortShuffleReader for sort-base

[jira] [Commented] (SPARK-2356) Exception: Could not locate executable null\bin\winutils.exe in the Hadoop

[ https://issues.apache.org/jira/browse/SPARK-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096601#comment-14096601 ] Guoqiang Li commented on SPARK-2356: This should be problems caused by not set HADOOP_

[jira] [Commented] (SPARK-3025) Allow JDBC clients to set a fair scheduler pool

[ https://issues.apache.org/jira/browse/SPARK-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096580#comment-14096580 ] Apache Spark commented on SPARK-3025: - User 'pwendell' has created a pull request for

[jira] [Updated] (SPARK-2356) Exception: Could not locate executable null\bin\winutils.exe in the Hadoop

[ https://issues.apache.org/jira/browse/SPARK-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2356: --- Description: I'm trying to run some transformation on Spark, it works fine on cluster (YARN, linux m

[jira] [Updated] (SPARK-3005) Spark with Mesos fine-grained mode throws UnsupportedOperationException in MesosSchedulerBackend.killTask()

[ https://issues.apache.org/jira/browse/SPARK-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3005: --- Description: I am using Spark, Mesos, spark-cassandra-connector to do some work on a cassandra clust

[jira] [Issue Comment Deleted] (SPARK-3019) Pluggable block transfer (data plane communication) interface

[ https://issues.apache.org/jira/browse/SPARK-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3019: --- Comment: was deleted (was: Design Doc - draft 1) > Pluggable block transfer (data plane communicatio

[jira] [Updated] (SPARK-3019) Pluggable block transfer (data plane communication) interface

[ https://issues.apache.org/jira/browse/SPARK-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3019: --- Attachment: (was: PluggableBlockTransferServiceProposalforSpark.pdf) > Pluggable block transfer (

[jira] [Updated] (SPARK-3019) Pluggable block transfer (data plane communication) interface

[ https://issues.apache.org/jira/browse/SPARK-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3019: --- Attachment: PluggableBlockTransferServiceProposalforSpark - draft 1.pdf Added first draft > Pluggabl

[jira] [Updated] (SPARK-3019) Pluggable block transfer (data plane communication) interface

[ https://issues.apache.org/jira/browse/SPARK-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3019: --- Description: The attached design doc proposes a standard interface for block transferring, which wil

[jira] [Updated] (SPARK-3019) Pluggable block transfer (data plane communication) interface

[ https://issues.apache.org/jira/browse/SPARK-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3019: --- Attachment: PluggableBlockTransferServiceProposalforSpark.pdf Design Doc - draft 1 > Pluggable block

[jira] [Created] (SPARK-3026) Provide a good error message if JDBC server is used but Spark is not compiled with -Pthriftserver

Patrick Wendell created SPARK-3026: -- Summary: Provide a good error message if JDBC server is used but Spark is not compiled with -Pthriftserver Key: SPARK-3026 URL: https://issues.apache.org/jira/browse/SPARK-302

[jira] [Updated] (SPARK-3026) Provide a good error message if JDBC server is used but Spark is not compiled with -Pthriftserver

[ https://issues.apache.org/jira/browse/SPARK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3026: --- Priority: Critical (was: Major) > Provide a good error message if JDBC server is used but Sp

[jira] [Created] (SPARK-3025) Allow JDBC clients to set a fair scheduler pool

Patrick Wendell created SPARK-3025: -- Summary: Allow JDBC clients to set a fair scheduler pool Key: SPARK-3025 URL: https://issues.apache.org/jira/browse/SPARK-3025 Project: Spark Issue Type:

[jira] [Commented] (SPARK-3024) CLI interface to Driver

[ https://issues.apache.org/jira/browse/SPARK-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096547#comment-14096547 ] Jeff Hammerbacher commented on SPARK-3024: -- It would be nice to be able to list t

[jira] [Commented] (SPARK-3005) Spark with Mesos fine-grained mode throws UnsupportedOperationException in MesosSchedulerBackend.killTask()

[ https://issues.apache.org/jira/browse/SPARK-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096539#comment-14096539 ] Xu Zhongxing commented on SPARK-3005: - Could adding an empty killTask method to Mesos

[jira] [Commented] (SPARK-3024) CLI interface to Driver

[ https://issues.apache.org/jira/browse/SPARK-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096541#comment-14096541 ] Patrick Wendell commented on SPARK-3024: Hey Jeff - mind giving a bit more color o

[jira] [Updated] (SPARK-3023) SIGINT to driver with yarn-client should release containers on the cluster

[ https://issues.apache.org/jira/browse/SPARK-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Hammerbacher updated SPARK-3023: - Issue Type: Improvement (was: Bug) > SIGINT to driver with yarn-client should release co

[jira] [Created] (SPARK-3024) CLI interface to Driver

Jeff Hammerbacher created SPARK-3024: Summary: CLI interface to Driver Key: SPARK-3024 URL: https://issues.apache.org/jira/browse/SPARK-3024 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-3023) SIGINT to driver with yarn-client should release containers on the cluster

Jeff Hammerbacher created SPARK-3023: Summary: SIGINT to driver with yarn-client should release containers on the cluster Key: SPARK-3023 URL: https://issues.apache.org/jira/browse/SPARK-3023 Proj

[jira] [Updated] (SPARK-3005) Spark with Mesos fine-grained mode throws UnsupportedOperationException in MesosSchedulerBackend.killTask()

[ https://issues.apache.org/jira/browse/SPARK-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] OuyangJin updated SPARK-3005: - Attachment: SPARK-3005_1.diff a quick fix for fine grained killTask > Spark with Mesos fine-grained mod

[jira] [Commented] (SPARK-3022) FindBinsForLevel in decision tree should call findBin only once for each feature

[ https://issues.apache.org/jira/browse/SPARK-3022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096526#comment-14096526 ] Qiping Li commented on SPARK-3022: -- What's more, there's no need to store `feature2bins`

[jira] [Updated] (SPARK-3022) FindBinsForLevel in decision tree should call findBin only once for each feature

[ https://issues.apache.org/jira/browse/SPARK-3022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiping Li updated SPARK-3022: - Description: `findbinsForLevel` is applied to every `LabeledPoint` to find bins for all nodes at a given

[jira] [Created] (SPARK-3022) FindBinsForLevel in decision tree should call findBin only once for each feature

Qiping Li created SPARK-3022: Summary: FindBinsForLevel in decision tree should call findBin only once for each feature Key: SPARK-3022 URL: https://issues.apache.org/jira/browse/SPARK-3022 Project: Spark

[jira] [Commented] (SPARK-2554) CountDistinct and SumDistinct should do partial aggregation

[ https://issues.apache.org/jira/browse/SPARK-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096486#comment-14096486 ] Apache Spark commented on SPARK-2554: - User 'marmbrus' has created a pull request for

[jira] [Updated] (SPARK-2625) Fix ShuffleReadMetrics for NettyBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-2625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2625: --- Component/s: Spark Core > Fix ShuffleReadMetrics for NettyBlockFetcherIterator >

[jira] [Commented] (SPARK-2356) Exception: Could not locate executable null\bin\winutils.exe in the Hadoop

[ https://issues.apache.org/jira/browse/SPARK-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096450#comment-14096450 ] Tarek Nabil commented on SPARK-2356: Are there any workarounds for this issue? I'm not

[jira] [Assigned] (SPARK-2625) Fix ShuffleReadMetrics for NettyBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-2625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-2625: -- Assignee: Reynold Xin > Fix ShuffleReadMetrics for NettyBlockFetcherIterator >

[jira] [Commented] (SPARK-2625) Fix ShuffleReadMetrics for NettyBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-2625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096449#comment-14096449 ] Reynold Xin commented on SPARK-2625: This will be resolved as part of https://github.c

[jira] [Resolved] (SPARK-3020) Print completed indices rather than tasks in web UI

[ https://issues.apache.org/jira/browse/SPARK-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-3020. Resolution: Fixed Fix Version/s: 1.1.0 > Print completed indices rather than tasks in web UI

[jira] [Resolved] (SPARK-2986) setting properties seems not effective

[ https://issues.apache.org/jira/browse/SPARK-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2986. - Resolution: Fixed Fix Version/s: 1.1.0 > setting properties seems not effective >

[jira] [Resolved] (SPARK-2970) spark-sql script ends with IOException when EventLogging is enabled

[ https://issues.apache.org/jira/browse/SPARK-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2970. - Resolution: Fixed Fix Version/s: 1.1.0 > spark-sql script ends with IOException wh

[jira] [Resolved] (SPARK-2935) Failure with push down of conjunctive parquet predicates

[ https://issues.apache.org/jira/browse/SPARK-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2935. - Resolution: Fixed Fix Version/s: 1.1.0 > Failure with push down of conjunctive par

[jira] [Resolved] (SPARK-2994) Support for Hive UDFs that take arrays of structs as arguments

[ https://issues.apache.org/jira/browse/SPARK-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2994. - Resolution: Fixed Fix Version/s: 1.1.0 > Support for Hive UDFs that take arrays of

[jira] [Created] (SPARK-3021) Job remains in Active Stages after failing

Michael Armbrust created SPARK-3021: --- Summary: Job remains in Active Stages after failing Key: SPARK-3021 URL: https://issues.apache.org/jira/browse/SPARK-3021 Project: Spark Issue Type: Bu

[jira] [Commented] (SPARK-3014) Log a more informative messages in a couple failure scenarios

[ https://issues.apache.org/jira/browse/SPARK-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096330#comment-14096330 ] Apache Spark commented on SPARK-3014: - User 'sryza' has created a pull request for thi

[jira] [Commented] (SPARK-3020) Print completed indices rather than tasks in web UI

[ https://issues.apache.org/jira/browse/SPARK-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096320#comment-14096320 ] Apache Spark commented on SPARK-3020: - User 'pwendell' has created a pull request for

[jira] [Resolved] (SPARK-3013) Doctest of inferSchema in Spark SQL Python API fails

[ https://issues.apache.org/jira/browse/SPARK-3013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3013. - Resolution: Fixed Fix Version/s: 1.1.0 > Doctest of inferSchema in Spark SQL Pytho

[jira] [Resolved] (SPARK-2963) The description about building to use HiveServer and CLI is incomplete

[ https://issues.apache.org/jira/browse/SPARK-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2963. - Resolution: Fixed Fix Version/s: 1.1.0 > The description about building to use Hiv

[jira] [Resolved] (SPARK-2817) add "show create table" support

[ https://issues.apache.org/jira/browse/SPARK-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2817. - Resolution: Fixed Fix Version/s: 1.1.0 > add "show create table" support > --

[jira] [Resolved] (SPARK-3004) HiveThriftServer2 throws exception when the result set contains NULL

[ https://issues.apache.org/jira/browse/SPARK-3004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3004. - Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Cheng Lian > HiveThriftS

[jira] [Created] (SPARK-3020) Print completed indices rather than tasks in web UI

Patrick Wendell created SPARK-3020: -- Summary: Print completed indices rather than tasks in web UI Key: SPARK-3020 URL: https://issues.apache.org/jira/browse/SPARK-3020 Project: Spark Issue T

[jira] [Commented] (SPARK-2907) Use mutable.HashMap to represent Model in Word2Vec

[ https://issues.apache.org/jira/browse/SPARK-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096301#comment-14096301 ] Apache Spark commented on SPARK-2907: - User 'Ishiihara' has created a pull request for

[jira] [Created] (SPARK-3019) Pluggable block transfer (data plane communication) interface

Reynold Xin created SPARK-3019: -- Summary: Pluggable block transfer (data plane communication) interface Key: SPARK-3019 URL: https://issues.apache.org/jira/browse/SPARK-3019 Project: Spark Issu

[jira] [Updated] (SPARK-3018) Release all BlockFetcherIterator upon task completion/failure

[ https://issues.apache.org/jira/browse/SPARK-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3018: --- Description: BlockFetcherIterator retains ReferenceCountedBuffers returned by client.fetchBlocks. Tho

[jira] [Commented] (SPARK-3015) Removing broadcast in quick successions causes Akka timeout

[ https://issues.apache.org/jira/browse/SPARK-3015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096251#comment-14096251 ] Apache Spark commented on SPARK-3015: - User 'andrewor14' has created a pull request fo

[jira] [Created] (SPARK-3018) Release all BlockFetcherIterator upon task completion/failure

Reynold Xin created SPARK-3018: -- Summary: Release all BlockFetcherIterator upon task completion/failure Key: SPARK-3018 URL: https://issues.apache.org/jira/browse/SPARK-3018 Project: Spark Issu

[jira] [Created] (SPARK-3017) Implement unit/integration tests for connection failures

Reynold Xin created SPARK-3017: -- Summary: Implement unit/integration tests for connection failures Key: SPARK-3017 URL: https://issues.apache.org/jira/browse/SPARK-3017 Project: Spark Issue Type

[jira] [Created] (SPARK-3016) Client should be able to put blocks in addition to fetch blocks

Reynold Xin created SPARK-3016: -- Summary: Client should be able to put blocks in addition to fetch blocks Key: SPARK-3016 URL: https://issues.apache.org/jira/browse/SPARK-3016 Project: Spark Is

[jira] [Resolved] (SPARK-2983) improve performance of sortByKey()

[ https://issues.apache.org/jira/browse/SPARK-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-2983. -- Resolution: Fixed Fix Version/s: 1.1.0 > improve performance of sortByKey() > --

[jira] [Created] (SPARK-3015) Removing broadcast in quick successions causes Akka timeout

Andrew Or created SPARK-3015: Summary: Removing broadcast in quick successions causes Akka timeout Key: SPARK-3015 URL: https://issues.apache.org/jira/browse/SPARK-3015 Project: Spark Issue Type

[jira] [Updated] (SPARK-3014) Log a more informative messages in a couple failure scenarios

[ https://issues.apache.org/jira/browse/SPARK-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-3014: -- Description: This is what shows up currently when the user code fails to initialize a SparkContext whe

[jira] [Updated] (SPARK-3014) Log a more informative messages in a couple failure scenarios

[ https://issues.apache.org/jira/browse/SPARK-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-3014: -- Summary: Log a more informative messages in a couple failure scenarios (was: Log a more informative me

[jira] [Created] (SPARK-3014) Log a more informative message when yarn-cluster app fails because SparkContext wasn't initialized

Sandy Ryza created SPARK-3014: - Summary: Log a more informative message when yarn-cluster app fails because SparkContext wasn't initialized Key: SPARK-3014 URL: https://issues.apache.org/jira/browse/SPARK-3014

[jira] [Updated] (SPARK-1297) Upgrade HBase dependency to 0.98.0

[ https://issues.apache.org/jira/browse/SPARK-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1297: --- Assignee: Ted Yu > Upgrade HBase dependency to 0.98.0 > -- > >

[jira] [Updated] (SPARK-1391) BlockManager cannot transfer blocks larger than 2G in size

[ https://issues.apache.org/jira/browse/SPARK-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1391: --- Assignee: (was: Min Zhou) > BlockManager cannot transfer blocks larger than 2G in size >

[jira] [Updated] (SPARK-1391) BlockManager cannot transfer blocks larger than 2G in size

[ https://issues.apache.org/jira/browse/SPARK-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1391: --- Issue Type: Improvement (was: Bug) > BlockManager cannot transfer blocks larger than 2G in size > --

[jira] [Updated] (SPARK-2846) Add configureInputJobPropertiesForStorageHandler to initialization of job conf

[ https://issues.apache.org/jira/browse/SPARK-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated SPARK-2846: Summary: Add configureInputJobPropertiesForStorageHandler to initialization of job conf (was: Spark SQL hi

[jira] [Commented] (SPARK-3013) Doctest of inferSchema in Spark SQL Python API fails

[ https://issues.apache.org/jira/browse/SPARK-3013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095942#comment-14095942 ] Apache Spark commented on SPARK-3013: - User 'davies' has created a pull request for th

[jira] [Commented] (SPARK-2846) Spark SQL hive implementation bypass StorageHandler which breaks any customized StorageHandler

[ https://issues.apache.org/jira/browse/SPARK-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095925#comment-14095925 ] Apache Spark commented on SPARK-2846: - User 'alexliu68' has created a pull request for

[jira] [Updated] (SPARK-2969) Make ScalaReflection be able to handle ArrayType.containsNull and MapType.valueContainsNull.

[ https://issues.apache.org/jira/browse/SPARK-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2969: Priority: Critical (was: Major) > Make ScalaReflection be able to handle ArrayType.contain

[jira] [Commented] (SPARK-2846) Spark SQL hive implementation bypass StorageHandler which breaks any customized StorageHandler

[ https://issues.apache.org/jira/browse/SPARK-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095879#comment-14095879 ] Alex Liu commented on SPARK-2846: - pull @ https://github.com/apache/spark/pull/1927 > Spa

[jira] [Commented] (SPARK-1442) Add Window function support

[ https://issues.apache.org/jira/browse/SPARK-1442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095861#comment-14095861 ] Adam Nowak commented on SPARK-1442: --- Does the Spark SQLContext support windowing functio

[jira] [Updated] (SPARK-3013) Doctest of inferSchema in Spark SQL Python API fails

[ https://issues.apache.org/jira/browse/SPARK-3013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-3013: Assignee: Davies Liu > Doctest of inferSchema in Spark SQL Python API fails > -

[jira] [Commented] (SPARK-2140) yarn stable client doesn't properly handle MEMORY_OVERHEAD for AM

[ https://issues.apache.org/jira/browse/SPARK-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095721#comment-14095721 ] Thomas Graves commented on SPARK-2140: -- ah it seems things have changed. Its now actu

[jira] [Created] (SPARK-3013) Doctest of inferSchema in Spark SQL Python API fails

Cheng Lian created SPARK-3013: - Summary: Doctest of inferSchema in Spark SQL Python API fails Key: SPARK-3013 URL: https://issues.apache.org/jira/browse/SPARK-3013 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-3012) Standardized Distance Functions between two Vectors for MLlib

Yu Ishikawa created SPARK-3012: -- Summary: Standardized Distance Functions between two Vectors for MLlib Key: SPARK-3012 URL: https://issues.apache.org/jira/browse/SPARK-3012 Project: Spark Issu

[jira] [Commented] (SPARK-3011) _temporary directory should be filtered out by sqlContext.parquetFile

[ https://issues.apache.org/jira/browse/SPARK-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095640#comment-14095640 ] Joseph Su commented on SPARK-3011: -- Pull request is here: https://github.com/apache/spark

[jira] [Commented] (SPARK-3011) _temporary directory should be filtered out by sqlContext.parquetFile

[ https://issues.apache.org/jira/browse/SPARK-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095641#comment-14095641 ] Apache Spark commented on SPARK-3011: - User 'joesu' has created a pull request for thi

[jira] [Commented] (SPARK-3011) _temporary directory should be filtered out by sqlContext.parquetFile

[ https://issues.apache.org/jira/browse/SPARK-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095636#comment-14095636 ] Sean Owen commented on SPARK-3011: -- Duplicate, or very closely related: https://issues.a

[jira] [Created] (SPARK-3011) _temporary directory should be filtered out by sqlContext.parquetFile

Joseph Su created SPARK-3011: Summary: _temporary directory should be filtered out by sqlContext.parquetFile Key: SPARK-3011 URL: https://issues.apache.org/jira/browse/SPARK-3011 Project: Spark

[jira] [Commented] (SPARK-3010) fix redundant conditional

[ https://issues.apache.org/jira/browse/SPARK-3010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095587#comment-14095587 ] Apache Spark commented on SPARK-3010: - User 'scwf' has created a pull request for this

[jira] [Comment Edited] (SPARK-2426) Quadratic Minimization for MLlib ALS

[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095232#comment-14095232 ] Debasish Das edited comment on SPARK-2426 at 8/13/14 3:31 PM: --

[jira] [Created] (SPARK-3010) fix redundant conditional

wangfei created SPARK-3010: -- Summary: fix redundant conditional Key: SPARK-3010 URL: https://issues.apache.org/jira/browse/SPARK-3010 Project: Spark Issue Type: Improvement Components: Spa

[jira] [Updated] (SPARK-2426) Quadratic Minimization for MLlib ALS

[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Debasish Das updated SPARK-2426: Description: Current ALS supports least squares and nonnegative least squares. I presented ADMM an

[jira] [Commented] (SPARK-3009) ApplicationInfo doesn't get initialised after deserialisation during recovery

[ https://issues.apache.org/jira/browse/SPARK-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095483#comment-14095483 ] Jacek Lewandowski commented on SPARK-3009: -- [~andrewor14] could you review it ple

[jira] [Commented] (SPARK-3009) ApplicationInfo doesn't get initialised after deserialisation during recovery

[ https://issues.apache.org/jira/browse/SPARK-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095479#comment-14095479 ] Apache Spark commented on SPARK-3009: - User 'jacek-lewandowski' has created a pull req

[jira] [Commented] (SPARK-3003) FailedStage could not be cancelled by DAGScheduler when cancelJob or cancelStage

[ https://issues.apache.org/jira/browse/SPARK-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095474#comment-14095474 ] Apache Spark commented on SPARK-3003: - User 'YanTangZhai' has created a pull request f

[jira] [Created] (SPARK-3009) ApplicationInfo doesn't get initialised after deserialisation during recovery

Jacek Lewandowski created SPARK-3009: Summary: ApplicationInfo doesn't get initialised after deserialisation during recovery Key: SPARK-3009 URL: https://issues.apache.org/jira/browse/SPARK-3009 P

[jira] [Created] (SPARK-3008) PySpark fails due to zipimport not able to load the assembly jar (/usr/bin/python: No module named pyspark)

Jai Kumar Singh created SPARK-3008: -- Summary: PySpark fails due to zipimport not able to load the assembly jar (/usr/bin/python: No module named pyspark) Key: SPARK-3008 URL: https://issues.apache.org/jira/brows

[jira] [Issue Comment Deleted] (SPARK-2204) Scheduler for Mesos in fine-grained mode launches tasks on wrong executors

[ https://issues.apache.org/jira/browse/SPARK-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Zhongxing updated SPARK-2204: Comment: was deleted (was: I encountered this issue again when I use Spark 1.0.2, Mesos 0.18.1, sp

  1   2   >