[jira] [Commented] (SPARK-2468) Netty-based block server / client module

[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201754#comment-14201754 ] Lianhui Wang commented on SPARK-2468: - [~adav] when i set the spark config "spark.shuf

[jira] [Commented] (SPARK-4294) The same function should have the same realization.

[ https://issues.apache.org/jira/browse/SPARK-4294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201740#comment-14201740 ] Apache Spark commented on SPARK-4294: - User 'watermen' has created a pull request for

[jira] [Commented] (SPARK-4294) The same function should have the same realization.

[ https://issues.apache.org/jira/browse/SPARK-4294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201728#comment-14201728 ] Apache Spark commented on SPARK-4294: - User 'watermen' has created a pull request for

[jira] [Created] (SPARK-4294) The same function should have the same realization.

Yadong Qi created SPARK-4294: Summary: The same function should have the same realization. Key: SPARK-4294 URL: https://issues.apache.org/jira/browse/SPARK-4294 Project: Spark Issue Type: Improve

[jira] [Commented] (SPARK-2468) Netty-based block server / client module

[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201684#comment-14201684 ] zzc commented on SPARK-2468: @Aaron Davidson, error still occur by setting the spark config "

[jira] [Comment Edited] (SPARK-4232) Truncate table not works when specific the table from non-current database session

[ https://issues.apache.org/jira/browse/SPARK-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14198430#comment-14198430 ] shengli edited comment on SPARK-4232 at 11/7/14 6:31 AM: - There is

[jira] [Comment Edited] (SPARK-4232) Truncate table not works when specific the table from non-current database session

[ https://issues.apache.org/jira/browse/SPARK-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14198430#comment-14198430 ] shengli edited comment on SPARK-4232 at 11/7/14 6:31 AM: - There is

[jira] [Commented] (SPARK-4293) Make Cast be able to handle complex types.

[ https://issues.apache.org/jira/browse/SPARK-4293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201670#comment-14201670 ] Apache Spark commented on SPARK-4293: - User 'ueshin' has created a pull request for th

[jira] [Created] (SPARK-4293) Make Cast be able to handle complex types.

Takuya Ueshin created SPARK-4293: Summary: Make Cast be able to handle complex types. Key: SPARK-4293 URL: https://issues.apache.org/jira/browse/SPARK-4293 Project: Spark Issue Type: Improvem

[jira] [Comment Edited] (SPARK-4257) Spark master can only be accessed by hostname

[ https://issues.apache.org/jira/browse/SPARK-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201658#comment-14201658 ] jiezhou edited comment on SPARK-4257 at 11/7/14 5:56 AM: - In start

[jira] [Commented] (SPARK-4257) Spark master can only be accessed by hostname

[ https://issues.apache.org/jira/browse/SPARK-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201658#comment-14201658 ] jiezhou commented on SPARK-4257: In start-master.sh, the SPARK_MASTER_IP's default value i

[jira] [Updated] (SPARK-4204) Utils.exceptionString only return the information for the outermost exception

[ https://issues.apache.org/jira/browse/SPARK-4204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4204: - Assignee: Shixiong Zhu > Utils.exceptionString only return the information for the outermost exception > -

[jira] [Closed] (SPARK-4204) Utils.exceptionString only return the information for the outermost exception

[ https://issues.apache.org/jira/browse/SPARK-4204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-4204. Resolution: Fixed Fix Version/s: 1.2.0 Target Version/s: 1.2.0 > Utils.exceptionString only

[jira] [Comment Edited] (SPARK-4290) Provide an equivalent functionality of distributed cache as MR does

[ https://issues.apache.org/jira/browse/SPARK-4290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201621#comment-14201621 ] Xuefu Zhang edited comment on SPARK-4290 at 11/7/14 5:37 AM: -

[jira] [Commented] (SPARK-4292) incorrect result set in JDBC/ODBC

[ https://issues.apache.org/jira/browse/SPARK-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201626#comment-14201626 ] Apache Spark commented on SPARK-4292: - User 'scwf' has created a pull request for this

[jira] [Created] (SPARK-4292) incorrect result set in JDBC/ODBC

wangfei created SPARK-4292: -- Summary: incorrect result set in JDBC/ODBC Key: SPARK-4292 URL: https://issues.apache.org/jira/browse/SPARK-4292 Project: Spark Issue Type: Bug Components: SQL

[jira] [Commented] (SPARK-4290) Provide an equivalent functionality of distributed cache as MR does

[ https://issues.apache.org/jira/browse/SPARK-4290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201621#comment-14201621 ] Xuefu Zhang commented on SPARK-4290: Yes, SparkContext#addFile() seems to be what we n

[jira] [Commented] (SPARK-4290) Provide an equivalent functionality of distributed cache as MR does

[ https://issues.apache.org/jira/browse/SPARK-4290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201614#comment-14201614 ] Reynold Xin commented on SPARK-4290: We might be able to provide an alternative broadc

[jira] [Commented] (SPARK-4291) Drop "Code" from network module names

[ https://issues.apache.org/jira/browse/SPARK-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201613#comment-14201613 ] Apache Spark commented on SPARK-4291: - User 'andrewor14' has created a pull request fo

[jira] [Commented] (SPARK-4281) Yarn shuffle service jars need to include dependencies

[ https://issues.apache.org/jira/browse/SPARK-4281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201611#comment-14201611 ] Apache Spark commented on SPARK-4281: - User 'andrewor14' has created a pull request fo

[jira] [Assigned] (SPARK-4281) Yarn shuffle service jars need to include dependencies

[ https://issues.apache.org/jira/browse/SPARK-4281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reassigned SPARK-4281: Assignee: Andrew Or (was: Patrick Wendell) > Yarn shuffle service jars need to include dependencie

[jira] [Commented] (SPARK-4187) External shuffle service should not use Java serializer

[ https://issues.apache.org/jira/browse/SPARK-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201596#comment-14201596 ] Apache Spark commented on SPARK-4187: - User 'aarondav' has created a pull request for

[jira] [Updated] (SPARK-3920) Add option to support aggregation using treeAggregate in decision tree

[ https://issues.apache.org/jira/browse/SPARK-3920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3920: - Fix Version/s: (was: 1.2.0) > Add option to support aggregation using treeAggregate in decisio

[jira] [Resolved] (SPARK-4242) Add SASL to external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-4242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson resolved SPARK-4242. --- Resolution: Fixed Fix Version/s: 1.2.0 > Add SASL to external shuffle service > ---

[jira] [Resolved] (SPARK-4236) External shuffle service must cleanup its shuffle files

[ https://issues.apache.org/jira/browse/SPARK-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson resolved SPARK-4236. --- Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Aaron Davidson > External shu

[jira] [Updated] (SPARK-4291) Drop "Code" from network module names

[ https://issues.apache.org/jira/browse/SPARK-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4291: - Description: In maven, the network modules have the suffix "Code", which is inconsistent with other modul

[jira] [Created] (SPARK-4291) Drop "Code" from network module names

Andrew Or created SPARK-4291: Summary: Drop "Code" from network module names Key: SPARK-4291 URL: https://issues.apache.org/jira/browse/SPARK-4291 Project: Spark Issue Type: Bug Compone

[jira] [Commented] (SPARK-4290) Provide an equivalent functionality of distributed cache as MR does

[ https://issues.apache.org/jira/browse/SPARK-4290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201572#comment-14201572 ] Sandy Ryza commented on SPARK-4290: --- If you call SparkContext#addFile, the file will be

[jira] [Commented] (SPARK-4290) Provide an equivalent functionality of distributed cache as MR does

[ https://issues.apache.org/jira/browse/SPARK-4290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201561#comment-14201561 ] Xuefu Zhang commented on SPARK-4290: Hi [~rxin], from the documentation of above java

[jira] [Commented] (SPARK-4290) Provide an equivalent functionality of distributed cache as MR does

[ https://issues.apache.org/jira/browse/SPARK-4290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201509#comment-14201509 ] Reynold Xin commented on SPARK-4290: Ok maybe I was misinterpreting how distributed ca

[jira] [Commented] (SPARK-4290) Provide an equivalent functionality of distributed cache as MR does

[ https://issues.apache.org/jira/browse/SPARK-4290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201496#comment-14201496 ] Xuefu Zhang commented on SPARK-4290: Hi [~rxin], by "out of box", do you mean org.apa

[jira] [Commented] (SPARK-2468) Netty-based block server / client module

[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201487#comment-14201487 ] zzc commented on SPARK-2468: YES, I see it, and start to compile and test. > Netty-based bloc

[jira] [Commented] (SPARK-2468) Netty-based block server / client module

[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201483#comment-14201483 ] Reynold Xin commented on SPARK-2468: It's been merged. > Netty-based block server / c

[jira] [Commented] (SPARK-2468) Netty-based block server / client module

[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201482#comment-14201482 ] zzc commented on SPARK-2468: Thank you. > Netty-based block server / client module >

[jira] [Closed] (SPARK-4195) retry to fetch blocks's result when fetchfailed's reason is connection timeout

[ https://issues.apache.org/jira/browse/SPARK-4195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-4195. -- Resolution: Duplicate Assignee: Aaron Davidson > retry to fetch blocks's result when fetchfailed's

[jira] [Resolved] (SPARK-4188) Shuffle fetches should be retried at a lower level

[ https://issues.apache.org/jira/browse/SPARK-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-4188. Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Aaron Davidson > Shuffle fetches sh

[jira] [Commented] (SPARK-4287) Add TTL-based cleanup in external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-4287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201462#comment-14201462 ] Andrew Or commented on SPARK-4287: -- Yeah that might be better, though you'll need to inve

[jira] [Commented] (SPARK-4290) Provide an equivalent functionality of distributed cache as MR does

[ https://issues.apache.org/jira/browse/SPARK-4290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201433#comment-14201433 ] Reynold Xin commented on SPARK-4290: Distributed cache should already work out of the

[jira] [Commented] (SPARK-2468) Netty-based block server / client module

[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201432#comment-14201432 ] Aaron Davidson commented on SPARK-2468: --- [~zzcclp] I believe it is close to merging.

[jira] [Commented] (SPARK-4287) Add TTL-based cleanup in external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-4287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201428#comment-14201428 ] Aaron Davidson commented on SPARK-4287: --- I think the right solution here might be to

[jira] [Commented] (SPARK-4290) Provide an equivalent functionality of distributed cache as MR does

[ https://issues.apache.org/jira/browse/SPARK-4290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201421#comment-14201421 ] Xuefu Zhang commented on SPARK-4290: CC: [~sandyr], [~rxin] > Provide an equivalent

[jira] [Created] (SPARK-4290) Provide an equivalent functionality of distributed cache as MR does

Xuefu Zhang created SPARK-4290: -- Summary: Provide an equivalent functionality of distributed cache as MR does Key: SPARK-4290 URL: https://issues.apache.org/jira/browse/SPARK-4290 Project: Spark

[jira] [Created] (SPARK-4289) Creating an instance of Hadoop Job fails in the Spark shell when toString() is called on the instance.

Corey J. Nolet created SPARK-4289: - Summary: Creating an instance of Hadoop Job fails in the Spark shell when toString() is called on the instance. Key: SPARK-4289 URL: https://issues.apache.org/jira/browse/SPARK-

[jira] [Created] (SPARK-4288) Add Sparse Autoencoder algorithm to MLlib

Guoqiang Li created SPARK-4288: -- Summary: Add Sparse Autoencoder algorithm to MLlib Key: SPARK-4288 URL: https://issues.apache.org/jira/browse/SPARK-4288 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-4287) Add TTL-based cleanup in external shuffle service

Andrew Or created SPARK-4287: Summary: Add TTL-based cleanup in external shuffle service Key: SPARK-4287 URL: https://issues.apache.org/jira/browse/SPARK-4287 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2468) Netty-based block server / client module

[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201378#comment-14201378 ] zzc commented on SPARK-2468: @Aaron Davidson, Thank you for your recommendation. By the way,

[jira] [Updated] (SPARK-4277) Support external shuffle service on Worker

[ https://issues.apache.org/jira/browse/SPARK-4277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4277: - Affects Version/s: 1.2.0 > Support external shuffle service on Worker > --

[jira] [Closed] (SPARK-4277) Support external shuffle service on Worker

[ https://issues.apache.org/jira/browse/SPARK-4277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-4277. Resolution: Fixed Fix Version/s: 1.2.0 > Support external shuffle service on Worker > ---

[jira] [Updated] (SPARK-3066) Support recommendAll in matrix factorization model

[ https://issues.apache.org/jira/browse/SPARK-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3066: - Target Version/s: (was: 1.2.0) > Support recommendAll in matrix factorization model > --

[jira] [Updated] (SPARK-4285) Transpose RDD[Vector] to column store for ML

[ https://issues.apache.org/jira/browse/SPARK-4285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4285: - Assignee: Joseph K. Bradley > Transpose RDD[Vector] to column store for ML > -

[jira] [Updated] (SPARK-4285) Transpose RDD[Vector] to column store for ML

[ https://issues.apache.org/jira/browse/SPARK-4285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-4285: - Issue Type: Sub-task (was: New Feature) Parent: SPARK-3717 > Transpose RDD[Vector

[jira] [Updated] (SPARK-4286) Support External Shuffle Service with Mesos integration

[ https://issues.apache.org/jira/browse/SPARK-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated SPARK-4286: Component/s: Mesos > Support External Shuffle Service with Mesos integration > -

[jira] [Commented] (SPARK-4286) Support External Shuffle Service with Mesos integration

[ https://issues.apache.org/jira/browse/SPARK-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201312#comment-14201312 ] Timothy Chen commented on SPARK-4286: - Please assign to me, thanks. > Support Externa

[jira] [Created] (SPARK-4285) Transpose RDD[Vector] to column store for ML

Joseph K. Bradley created SPARK-4285: Summary: Transpose RDD[Vector] to column store for ML Key: SPARK-4285 URL: https://issues.apache.org/jira/browse/SPARK-4285 Project: Spark Issue Type

[jira] [Created] (SPARK-4286) Support External Shuffle Service with Mesos integration

Timothy Chen created SPARK-4286: --- Summary: Support External Shuffle Service with Mesos integration Key: SPARK-4286 URL: https://issues.apache.org/jira/browse/SPARK-4286 Project: Spark Issue Typ

[jira] [Created] (SPARK-4284) BinaryClassificationMetrics precision-recall method names should correspond to return types

Joseph K. Bradley created SPARK-4284: Summary: BinaryClassificationMetrics precision-recall method names should correspond to return types Key: SPARK-4284 URL: https://issues.apache.org/jira/browse/SPARK-4284

[jira] [Commented] (SPARK-3797) Run the shuffle service inside the YARN NodeManager as an AuxiliaryService

[ https://issues.apache.org/jira/browse/SPARK-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201077#comment-14201077 ] Apache Spark commented on SPARK-3797: - User 'andrewor14' has created a pull request fo

[jira] [Updated] (SPARK-4283) Spark source code does not correctly import into eclipse

[ https://issues.apache.org/jira/browse/SPARK-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Yang updated SPARK-4283: - Attachment: spark_eclipse.diff patch for all the pom files > Spark source code does not correctly import

[jira] [Created] (SPARK-4283) Spark source code does not correctly import into eclipse

Yang Yang created SPARK-4283: Summary: Spark source code does not correctly import into eclipse Key: SPARK-4283 URL: https://issues.apache.org/jira/browse/SPARK-4283 Project: Spark Issue Type: Bu

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201042#comment-14201042 ] Patrick Wendell commented on SPARK-3561: Hey [~ozhurakousky] - as I said before, I

[jira] [Commented] (SPARK-4282) Stopping flag in YarnClientSchedulerBackend should be volatile

[ https://issues.apache.org/jira/browse/SPARK-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201032#comment-14201032 ] Apache Spark commented on SPARK-4282: - User 'sarutak' has created a pull request for t

[jira] [Created] (SPARK-4282) Stopping flag in YarnClientSchedulerBackend should be volatile

Kousuke Saruta created SPARK-4282: - Summary: Stopping flag in YarnClientSchedulerBackend should be volatile Key: SPARK-4282 URL: https://issues.apache.org/jira/browse/SPARK-4282 Project: Spark

[jira] [Updated] (SPARK-4148) PySpark's sample uses the same seed for all partitions

[ https://issues.apache.org/jira/browse/SPARK-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4148: - Fix Version/s: 1.2.0 > PySpark's sample uses the same seed for all partitions > --

[jira] [Reopened] (SPARK-4148) PySpark's sample uses the same seed for all partitions

[ https://issues.apache.org/jira/browse/SPARK-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reopened SPARK-4148: -- reopen this issue because branch-1.0 is not fixed > PySpark's sample uses the same seed for all par

[jira] [Created] (SPARK-4281) Yarn shuffle service jars need to include dependencies

Andrew Or created SPARK-4281: Summary: Yarn shuffle service jars need to include dependencies Key: SPARK-4281 URL: https://issues.apache.org/jira/browse/SPARK-4281 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-4280) In dynamic allocation, add option to never kill executors with cached blocks

[ https://issues.apache.org/jira/browse/SPARK-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200801#comment-14200801 ] Sandy Ryza commented on SPARK-4280: --- My thinking was that it would just be based on whet

[jira] [Commented] (SPARK-4280) In dynamic allocation, add option to never kill executors with cached blocks

[ https://issues.apache.org/jira/browse/SPARK-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200785#comment-14200785 ] Andrew Or commented on SPARK-4280: -- Hey [~sandyr] does this require the application to ex

[jira] [Commented] (SPARK-4133) PARSING_ERROR(2) when upgrading issues from 1.0.2 to 1.1.0

[ https://issues.apache.org/jira/browse/SPARK-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200762#comment-14200762 ] Josh Rosen commented on SPARK-4133: --- We can't remove that StreamingContext constructor s

[jira] [Created] (SPARK-4280) In dynamic allocation, add option to never kill executors with cached blocks

Sandy Ryza created SPARK-4280: - Summary: In dynamic allocation, add option to never kill executors with cached blocks Key: SPARK-4280 URL: https://issues.apache.org/jira/browse/SPARK-4280 Project: Spark

[jira] [Created] (SPARK-4279) Implementing TinkerPop on top of GraphX

Brennon York created SPARK-4279: --- Summary: Implementing TinkerPop on top of GraphX Key: SPARK-4279 URL: https://issues.apache.org/jira/browse/SPARK-4279 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-677) PySpark should not collect results through local filesystem

[ https://issues.apache.org/jira/browse/SPARK-677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200686#comment-14200686 ] Josh Rosen commented on SPARK-677: -- No, it's still an issue in 1.2.0: {code} def collect

[jira] [Created] (SPARK-4278) SparkSQL job failing with java.lang.ClassCastException

Parviz Deyhim created SPARK-4278: Summary: SparkSQL job failing with java.lang.ClassCastException Key: SPARK-4278 URL: https://issues.apache.org/jira/browse/SPARK-4278 Project: Spark Issue Ty

[jira] [Resolved] (SPARK-4173) EdgePartitionBuilder uses wrong value for first clustered index

[ https://issues.apache.org/jira/browse/SPARK-4173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-4173. --- Resolution: Fixed Fix Version/s: 1.0.3 1.1.1 1.2.0 Issue

[jira] [Commented] (SPARK-2447) Add common solution for sending upsert actions to HBase (put, deletes, and increment)

[ https://issues.apache.org/jira/browse/SPARK-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200646#comment-14200646 ] Ted Malaska commented on SPARK-2447: I totally understand. I also don't know the answ

[jira] [Resolved] (SPARK-4249) A problem of EdgePartitionBuilder in Graphx

[ https://issues.apache.org/jira/browse/SPARK-4249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-4249. --- Resolution: Fixed Fix Version/s: 1.0.3 1.1.1 1.2.0 Issue

[jira] [Commented] (SPARK-2447) Add common solution for sending upsert actions to HBase (put, deletes, and increment)

[ https://issues.apache.org/jira/browse/SPARK-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200634#comment-14200634 ] Patrick Wendell commented on SPARK-2447: Hey All, I have a question about this -

[jira] [Commented] (SPARK-4277) Support external shuffle service on Worker

[ https://issues.apache.org/jira/browse/SPARK-4277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200631#comment-14200631 ] Apache Spark commented on SPARK-4277: - User 'aarondav' has created a pull request for

[jira] [Resolved] (SPARK-4264) SQL HashJoin induces "refCnt = 0" error in ShuffleBlockFetcherIterator

[ https://issues.apache.org/jira/browse/SPARK-4264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson resolved SPARK-4264. --- Resolution: Fixed > SQL HashJoin induces "refCnt = 0" error in ShuffleBlockFetcherIterator > -

[jira] [Created] (SPARK-4277) Support external shuffle service on Worker

Aaron Davidson created SPARK-4277: - Summary: Support external shuffle service on Worker Key: SPARK-4277 URL: https://issues.apache.org/jira/browse/SPARK-4277 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2447) Add common solution for sending upsert actions to HBase (put, deletes, and increment)

[ https://issues.apache.org/jira/browse/SPARK-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200602#comment-14200602 ] Ted Malaska commented on SPARK-2447: Cool thanks. Connect me at ted.mala...@cloudera.

[jira] [Commented] (SPARK-2447) Add common solution for sending upsert actions to HBase (put, deletes, and increment)

[ https://issues.apache.org/jira/browse/SPARK-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200593#comment-14200593 ] Norman He commented on SPARK-2447: -- Yes, I would like to help. Let me start with facade w

[jira] [Commented] (SPARK-4276) Spark streaming requires at least two working thread

[ https://issues.apache.org/jira/browse/SPARK-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200571#comment-14200571 ] varun sharma commented on SPARK-4276: - [~srowen] Yeah right. I spent lot of time to fi

[jira] [Commented] (SPARK-2447) Add common solution for sending upsert actions to HBase (put, deletes, and increment)

[ https://issues.apache.org/jira/browse/SPARK-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200554#comment-14200554 ] Ted Malaska commented on SPARK-2447: Hey Norman, Totally agree. TD and I talked abou

[jira] [Updated] (SPARK-2348) In Windows having a enviorinment variable named 'classpath' gives error

[ https://issues.apache.org/jira/browse/SPARK-2348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-2348: - Priority: Critical (was: Major) > In Windows having a enviorinment variable named 'classpath' giv

[jira] [Closed] (SPARK-2237) Add ZLIBCompressionCodec code

[ https://issues.apache.org/jira/browse/SPARK-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia closed SPARK-2237. Resolution: Won't Fix > Add ZLIBCompressionCodec code > - > >

[jira] [Commented] (SPARK-4276) Spark streaming requires at least two working thread

[ https://issues.apache.org/jira/browse/SPARK-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200540#comment-14200540 ] Apache Spark commented on SPARK-4276: - User 'svar29' has created a pull request for th

[jira] [Commented] (SPARK-4276) Spark streaming requires at least two working thread

[ https://issues.apache.org/jira/browse/SPARK-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200539#comment-14200539 ] Sean Owen commented on SPARK-4276: -- This is basically the same concern addressed already

[jira] [Commented] (SPARK-2447) Add common solution for sending upsert actions to HBase (put, deletes, and increment)

[ https://issues.apache.org/jira/browse/SPARK-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200538#comment-14200538 ] Norman He commented on SPARK-2447: -- Hi Ted and Tathagaat Das, Would the spark/sparkstrea

[jira] [Resolved] (SPARK-1185) In Spark Programming Guide, "Master URLs" should mention yarn-client

[ https://issues.apache.org/jira/browse/SPARK-1185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-1185. -- Resolution: Fixed > In Spark Programming Guide, "Master URLs" should mention yarn-client > -

[jira] [Resolved] (SPARK-1023) Remove Thread.sleep(5000) from TaskSchedulerImpl

[ https://issues.apache.org/jira/browse/SPARK-1023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-1023. -- Resolution: Fixed > Remove Thread.sleep(5000) from TaskSchedulerImpl > -

[jira] [Closed] (SPARK-1000) Crash when running SparkPi example with local-cluster

[ https://issues.apache.org/jira/browse/SPARK-1000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia closed SPARK-1000. Resolution: Cannot Reproduce > Crash when running SparkPi example with local-cluster > -

[jira] [Commented] (SPARK-993) Don't reuse Writable objects in HadoopRDDs by default

[ https://issues.apache.org/jira/browse/SPARK-993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200531#comment-14200531 ] Matei Zaharia commented on SPARK-993: - Arun, you'd see this issue if you do collect() o

[jira] [Commented] (SPARK-4238) Perform network-level retry of shuffle file fetches

[ https://issues.apache.org/jira/browse/SPARK-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200526#comment-14200526 ] Aaron Davidson commented on SPARK-4238: --- Oh, whoops. > Perform network-level retry

[jira] [Resolved] (SPARK-993) Don't reuse Writable objects in HadoopRDDs by default

[ https://issues.apache.org/jira/browse/SPARK-993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-993. - Resolution: Won't Fix We investigated this for 1.0 but found that many InputFormats behave wrongly

[jira] [Updated] (SPARK-4188) Shuffle fetches should be retried at a lower level

[ https://issues.apache.org/jira/browse/SPARK-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson updated SPARK-4188: -- Description: During periods of high network (or GC) load, it is not uncommon that IOExceptions

[jira] [Closed] (SPARK-4238) Perform network-level retry of shuffle file fetches

[ https://issues.apache.org/jira/browse/SPARK-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson closed SPARK-4238. - Resolution: Duplicate > Perform network-level retry of shuffle file fetches >

[jira] [Commented] (SPARK-4188) Shuffle fetches should be retried at a lower level

[ https://issues.apache.org/jira/browse/SPARK-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200527#comment-14200527 ] Apache Spark commented on SPARK-4188: - User 'aarondav' has created a pull request for

[jira] [Resolved] (SPARK-798) AMI: ami-530f7a3a and Mesos

[ https://issues.apache.org/jira/browse/SPARK-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-798. - Resolution: Won't Fix This is now about a pretty old AMI, so I'll close it. New versions of Spark u

[jira] [Commented] (SPARK-2468) Netty-based block server / client module

[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200519#comment-14200519 ] Aaron Davidson commented on SPARK-2468: --- [~zzcclp] Use of epoll mode is highly depen

[jira] [Resolved] (SPARK-681) Optimize hashtables used in Spark

[ https://issues.apache.org/jira/browse/SPARK-681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-681. - Resolution: Fixed > Optimize hashtables used in Spark > - > >

  1   2   >