[jira] [Resolved] (SPARK-5366) check for mode of private key file

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5366. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 4162

[jira] [Resolved] (SPARK-5656) NegativeArraySizeException in EigenValueDecomposition.symmetricEigs for large n and/or large k

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5656. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 4433

[jira] [Resolved] (SPARK-5672) Don't return `ERROR 500` when have missing args

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5672. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4239

[jira] [Updated] (SPARK-5672) Don't return `ERROR 500` when have missing args

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5672: - Priority: Minor (was: Major) Assignee: Kirill A. Korinskiy Don't return `ERROR 500` when have

[jira] [Commented] (SPARK-5673) Implement Streaming wrapper for all linear methos

2015-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311163#comment-14311163 ] Apache Spark commented on SPARK-5673: - User 'catap' has created a pull request for

[jira] [Updated] (SPARK-5673) Implement Streaming wrapper for all linear methos

2015-02-08 Thread Kirill A. Korinskiy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill A. Korinskiy updated SPARK-5673: --- Component/s: MLlib Implement Streaming wrapper for all linear methos

[jira] [Commented] (SPARK-3431) Parallelize Scala/Java test execution

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311270#comment-14311270 ] Sean Owen commented on SPARK-3431: -- Haven't tried anything recently, no. Parallelize

[jira] [Commented] (SPARK-5625) Spark binaries do not incude Spark Core

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311272#comment-14311272 ] Sean Owen commented on SPARK-5625: -- I think you may be running into problems with older

[jira] [Updated] (SPARK-761) Print a nicer error message when incompatible Spark binaries try to talk

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-761: Description: Not sure what component this falls under, or if this is still an issue. Patrick Wendell / Matei

[jira] [Updated] (SPARK-5140) Two RDDs which are scheduled concurrently should be able to wait on parent in all cases

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5140: - Component/s: Spark Core Two RDDs which are scheduled concurrently should be able to wait on parent in

[jira] [Updated] (SPARK-5334) NullPointerException when getting files from S3 (hadoop 2.3+)

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5334: - Component/s: Input/Output Related to, or resolved by, SPARK-5671? NullPointerException when getting

[jira] [Updated] (SPARK-4229) Create hadoop configuration in a consistent way

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4229: - Component/s: Spark Core Create hadoop configuration in a consistent way

[jira] [Updated] (SPARK-4540) Improve Executor ID Logging

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4540: - Component/s: Spark Core Improve Executor ID Logging --- Key:

[jira] [Updated] (SPARK-4321) Make Kryo serialization work for closures

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4321: - Component/s: Spark Core Make Kryo serialization work for closures

[jira] [Updated] (SPARK-4563) Allow spark driver to bind to different ip then advertise ip

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4563: - Component/s: Deploy Allow spark driver to bind to different ip then advertise ip

[jira] [Updated] (SPARK-5412) Cannot bind Master to a specific hostname as per the documentation

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5412: - Component/s: Deploy Cannot bind Master to a specific hostname as per the documentation

[jira] [Updated] (SPARK-5360) For CoGroupedRDD, rdds for narrow dependencies and shuffle handles are included twice in serialized task

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5360: - Component/s: Spark Core For CoGroupedRDD, rdds for narrow dependencies and shuffle handles are

[jira] [Updated] (SPARK-5423) ExternalAppendOnlyMap won't delete temp spilled file if some exception happens during using it

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5423: - Component/s: Shuffle ExternalAppendOnlyMap won't delete temp spilled file if some exception happens

[jira] [Updated] (SPARK-3132) Avoid serialization for Array[Byte] in TorrentBroadcast

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-3132: - Component/s: Spark Core Avoid serialization for Array[Byte] in TorrentBroadcast

[jira] [Updated] (SPARK-1866) Closure cleaner does not null shadowed fields when outer scope is referenced

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-1866: - Component/s: Spark Core Closure cleaner does not null shadowed fields when outer scope is referenced

[jira] [Resolved] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3039. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4315

[jira] [Commented] (SPARK-5668) spark_ec2.py region parameter could be either mandatory or its value displayed

2015-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311264#comment-14311264 ] Apache Spark commented on SPARK-5668: - User 'MiguelPeralvo' has created a pull request

[jira] [Updated] (SPARK-4440) Enhance the job progress API to expose more information

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4440: - Component/s: Spark Core Enhance the job progress API to expose more information

[jira] [Updated] (SPARK-761) Print a nicer error message when incompatible Spark binaries try to talk

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-761: Component/s: Spark Core Description: I don't know how possible this is, as incompatibilities manifest in

[jira] [Updated] (SPARK-672) Executor gets stuck in a zombie state after running out of memory

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-672: Component/s: Spark Core Executor gets stuck in a zombie state after running out of memory

[jira] [Updated] (SPARK-761) Print a nicer error message when incompatible Spark binaries try to talk

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-761: Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) Print a nicer error message when

[jira] [Resolved] (SPARK-672) Executor gets stuck in a zombie state after running out of memory

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-672. - Resolution: Duplicate The right-er answer is to fail for lack of memory faster, per SPARK-1989. Executor

[jira] [Updated] (SPARK-704) ConnectionManager sometimes cannot detect loss of sending connections

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-704: Component/s: Spark Core ConnectionManager sometimes cannot detect loss of sending connections

[jira] [Updated] (SPARK-5065) BroadCast can still work after sc had been stopped.

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5065: - Component/s: Spark Core Priority: Minor (was: Major) BroadCast can still work after sc had been

[jira] [Updated] (SPARK-5332) Efficient way to deal with ExecutorLost

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5332: - Component/s: Spark Core Priority: Minor (was: Major) Efficient way to deal with ExecutorLost

[jira] [Resolved] (SPARK-4895) Support a shared RDD store among different Spark contexts

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4895. -- Resolution: Duplicate Since I don't see additional work here, and this covers almost exactly the same

[jira] [Updated] (SPARK-4681) Turn on host level blacklisting by default

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4681: - Component/s: Scheduler Turn on host level blacklisting by default

[jira] [Updated] (SPARK-5311) EventLoggingListener throws exception if log directory does not exist

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5311: - Component/s: Spark Core EventLoggingListener throws exception if log directory does not exist

[jira] [Updated] (SPARK-4783) System.exit() calls in SparkContext disrupt applications embedding Spark

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4783: - Component/s: Spark Core System.exit() calls in SparkContext disrupt applications embedding Spark

[jira] [Updated] (SPARK-4723) To abort the stages which have attempted some times

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4723: - Component/s: Scheduler To abort the stages which have attempted some times

[jira] [Updated] (SPARK-1346) Backport SPARK-1210 into 0.9 branch

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-1346: - Component/s: Spark Core Backport SPARK-1210 into 0.9 branch ---

[jira] [Updated] (SPARK-2958) FileClientHandler should not be shared in the pipeline

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-2958: - Component/s: Spark Core FileClientHandler should not be shared in the pipeline

[jira] [Updated] (SPARK-839) Bug in how failed executors are removed by ID from standalone cluster

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-839: Component/s: Spark Core Bug in how failed executors are removed by ID from standalone cluster

[jira] [Resolved] (SPARK-3965) Spark assembly for hadoop2 contains avro-mapred for hadoop1

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3965. -- Resolution: Duplicate Spark assembly for hadoop2 contains avro-mapred for hadoop1

[jira] [Commented] (SPARK-5334) NullPointerException when getting files from S3 (hadoop 2.3+)

2015-02-08 Thread Kevin (Sangwoo) Kim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311324#comment-14311324 ] Kevin (Sangwoo) Kim commented on SPARK-5334: [~srowen] Oh thanks! I'll test

[jira] [Updated] (SPARK-5225) Support coalesed Input Metrics from different sources

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5225: - Component/s: Spark Core Support coalesed Input Metrics from different sources

[jira] [Updated] (SPARK-2666) when task is FetchFailed cancel running tasks of failedStage

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-2666: - Component/s: Spark Core when task is FetchFailed cancel running tasks of failedStage

[jira] [Updated] (SPARK-4087) Only use broadcast for large tasks

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4087: - Component/s: Spark Core Only use broadcast for large tasks --

[jira] [Updated] (SPARK-4059) spark-master/spark-worker may use SPARK_MASTER_IP/STANDALONE_SPARK_MASTER_HOST

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4059: - Priority: Minor (was: Major) It's not clear what this means? spark-master/spark-worker may use

[jira] [Updated] (SPARK-4059) spark-master/spark-worker may use SPARK_MASTER_IP/STANDALONE_SPARK_MASTER_HOST

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4059: - Component/s: Deploy spark-master/spark-worker may use SPARK_MASTER_IP/STANDALONE_SPARK_MASTER_HOST

[jira] [Updated] (SPARK-4539) History Server counts incomplete applications against the retainedApplications total, fails to show eligible completed applications

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4539: - Component/s: Spark Core History Server counts incomplete applications against the retainedApplications

[jira] [Updated] (SPARK-4560) Lambda deserialization error

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4560: - Component/s: Spark Core Lambda deserialization error

[jira] [Updated] (SPARK-5594) SparkException: Failed to get broadcast (TorrentBroadcast)

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5594: - Component/s: Spark Core SparkException: Failed to get broadcast (TorrentBroadcast)

[jira] [Updated] (SPARK-2319) Number of tasks on executors become negative after executor failures

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-2319: - Component/s: Web UI Number of tasks on executors become negative after executor failures

[jira] [Updated] (SPARK-2520) the executor is thrown java.io.StreamCorruptedException

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-2520: - Component/s: Shuffle the executor is thrown java.io.StreamCorruptedException

[jira] [Updated] (SPARK-5377) Dynamically add jar into Spark Driver's classpath.

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5377: - Component/s: Spark Core Dynamically add jar into Spark Driver's classpath.

[jira] [Updated] (SPARK-5113) Audit and document use of hostnames and IP addresses in Spark

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5113: - Component/s: Spark Core Audit and document use of hostnames and IP addresses in Spark

[jira] [Updated] (SPARK-2913) Spark's log4j.properties should always appear ahead of Hadoop's on classpath

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-2913: - Component/s: Deploy Spark's log4j.properties should always appear ahead of Hadoop's on classpath

[jira] [Updated] (SPARK-4351) Record cacheable RDD reads and display RDD miss rates

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4351: - Component/s: Spark Core Record cacheable RDD reads and display RDD miss rates

[jira] [Updated] (SPARK-4605) Proposed Contribution: Spark Kernel to enable interactive Spark applications

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4605: - Component/s: Project Infra Proposed Contribution: Spark Kernel to enable interactive Spark applications

[jira] [Updated] (SPARK-5581) When writing sorted map output file, avoid open / close between each partition

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5581: - Component/s: Shuffle When writing sorted map output file, avoid open / close between each partition

[jira] [Updated] (SPARK-5607) NullPointerException in objenesis

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5607: - Component/s: Tests NullPointerException in objenesis -

[jira] [Updated] (SPARK-5654) Integrate SparkR into Apache Spark

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5654: - Component/s: Project Infra Integrate SparkR into Apache Spark --

[jira] [Updated] (SPARK-5319) Choosing partition size instead of count

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5319: - Component/s: Spark Core Choosing partition size instead of count

[jira] [Resolved] (SPARK-5340) Spark startup in local mode should not always create HTTP file server

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5340. -- Resolution: Won't Fix Per PR discussion, WontFix. Spark startup in local mode should not always

[jira] [Updated] (SPARK-5616) Add examples for PySpark API

2015-02-08 Thread dongxu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dongxu updated SPARK-5616: -- Description: PySpark API examples are less than Spark scala API. For example: 1.Broadcast: how to use

[jira] [Commented] (SPARK-5625) Spark binaries do not incude Spark Core

2015-02-08 Thread DeepakVohra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311447#comment-14311447 ] DeepakVohra commented on SPARK-5625: Extracting/opening with WinZip is only to verify

[jira] [Comment Edited] (SPARK-5625) Spark binaries do not incude Spark Core

2015-02-08 Thread DeepakVohra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311447#comment-14311447 ] DeepakVohra edited comment on SPARK-5625 at 2/8/15 5:56 PM:

[jira] [Commented] (SPARK-761) Print a nicer error message when incompatible Spark binaries try to talk

2015-02-08 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311475#comment-14311475 ] Patrick Wendell commented on SPARK-761: --- I think the main thing to catch would be

[jira] [Updated] (SPARK-4687) SparkContext#addFile doesn't keep file folder information

2015-02-08 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4687: --- Component/s: Spark Core SparkContext#addFile doesn't keep file folder information

[jira] [Updated] (SPARK-5299) Is http://www.apache.org/dist/spark/KEYS out of date?

2015-02-08 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5299: --- Component/s: (was: Deploy) Build Is

[jira] [Updated] (SPARK-3033) [Hive] java.math.BigDecimal cannot be cast to org.apache.hadoop.hive.common.type.HiveDecimal

2015-02-08 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3033: --- Component/s: (was: Spark Core) [Hive] java.math.BigDecimal cannot be cast to

[jira] [Updated] (SPARK-761) Print a nicer error message when incompatible Spark binaries try to talk

2015-02-08 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-761: -- Labels: starter (was: ) Print a nicer error message when incompatible Spark binaries try to

[jira] [Updated] (SPARK-761) Print a nicer error message when incompatible Spark binaries try to talk

2015-02-08 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-761: -- Description: As a starter task, it would be good to audit the current behavior for different

[jira] [Commented] (SPARK-761) Print a nicer error message when incompatible Spark binaries try to talk

2015-02-08 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311485#comment-14311485 ] Andrew Ash commented on SPARK-761: -- Another thing could be a basic check for version

[jira] [Commented] (SPARK-761) Print a nicer error message when incompatible Spark binaries try to talk

2015-02-08 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311490#comment-14311490 ] Patrick Wendell commented on SPARK-761: --- [~aash] right now we don't explicitly encode

[jira] [Commented] (SPARK-3242) Spark 1.0.2 ec2 scripts creates clusters with Spark 1.0.1 installed by default

2015-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311328#comment-14311328 ] Apache Spark commented on SPARK-3242: - User 'srowen' has created a pull request for

[jira] [Updated] (SPARK-3033) [Hive] java.math.BigDecimal cannot be cast to org.apache.hadoop.hive.common.type.HiveDecimal

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-3033: - Priority: Major (was: Blocker) [Hive] java.math.BigDecimal cannot be cast to

[jira] [Commented] (SPARK-5625) Spark binaries do not incude Spark Core

2015-02-08 Thread DeepakVohra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311348#comment-14311348 ] DeepakVohra commented on SPARK-5625: The other jars in the Spark binaries lib

[jira] [Updated] (SPARK-4869) The variable names in IF statement of Spark SQL doesn't resolve to its value.

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4869: - Component/s: (was: Spark Core) SQL Priority: Major (was: Blocker) The

[jira] [Updated] (SPARK-5139) select table_alias.* with joins and selecting column names from inner queries not supported

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5139: - Priority: Major (was: Blocker) Issue Type: Improvement (was: Bug) select table_alias.* with

[jira] [Commented] (SPARK-5625) Spark binaries do not incude Spark Core

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311378#comment-14311378 ] Sean Owen commented on SPARK-5625: -- As I've said, the assembly is a JAR file. You do not

[jira] [Commented] (SPARK-5625) Spark binaries do not incude Spark Core

2015-02-08 Thread DeepakVohra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311347#comment-14311347 ] DeepakVohra commented on SPARK-5625: WinZIp version is the latest 18.5. Spark

[jira] [Comment Edited] (SPARK-5625) Spark binaries do not incude Spark Core

2015-02-08 Thread DeepakVohra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311348#comment-14311348 ] DeepakVohra edited comment on SPARK-5625 at 2/8/15 3:21 PM:

[jira] [Updated] (SPARK-5659) Flaky Test: org.apache.spark.streaming.ReceiverSuite.block

2015-02-08 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5659: --- Component/s: Tests Flaky Test: org.apache.spark.streaming.ReceiverSuite.block

[jira] [Commented] (SPARK-5625) Spark binaries do not incude Spark Core

2015-02-08 Thread DeepakVohra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311500#comment-14311500 ] DeepakVohra commented on SPARK-5625: On re-test Spark classes get found in Spark

[jira] [Commented] (SPARK-5631) Server IPC version 7 cannot communicate with client version 4

2015-02-08 Thread DeepakVohra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311506#comment-14311506 ] DeepakVohra commented on SPARK-5631: This means you have mismatched Hadoop versions,

[jira] [Commented] (SPARK-5021) GaussianMixtureEM should be faster for SparseVector input

2015-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311530#comment-14311530 ] Apache Spark commented on SPARK-5021: - User 'MechCoder' has created a pull request for

[jira] [Commented] (SPARK-5631) Server IPC version 7 cannot communicate with client version 4

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311531#comment-14311531 ] Sean Owen commented on SPARK-5631: -- So, one problem is that the {{cdh4}} binary is

[jira] [Commented] (SPARK-5625) Spark binaries do not incude Spark Core

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311533#comment-14311533 ] Sean Owen commented on SPARK-5625: -- You asked this in a separate issue and it is

[jira] [Commented] (SPARK-5021) GaussianMixtureEM should be faster for SparseVector input

2015-02-08 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311532#comment-14311532 ] Manoj Kumar commented on SPARK-5021: I have created a working pull request. Let us

[jira] [Updated] (SPARK-5273) Improve documentation examples for LinearRegression

2015-02-08 Thread Dev Lakhani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dev Lakhani updated SPARK-5273: --- Affects Version/s: (was: 1.2.0) Improve documentation examples for LinearRegression

[jira] [Updated] (SPARK-5273) Improve documentation examples for LinearRegression

2015-02-08 Thread Dev Lakhani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dev Lakhani updated SPARK-5273: --- Affects Version/s: 1.2.0 Improve documentation examples for LinearRegression

[jira] [Comment Edited] (SPARK-4550) In sort-based shuffle, store map outputs in serialized form

2015-02-08 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14310130#comment-14310130 ] Sandy Ryza edited comment on SPARK-4550 at 2/8/15 9:07 PM: --- I

[jira] [Commented] (SPARK-4588) Add API for feature attributes

2015-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311586#comment-14311586 ] Apache Spark commented on SPARK-4588: - User 'srowen' has created a pull request for

[jira] [Created] (SPARK-5674) Spark Job Explain Plan Proof of Concept

2015-02-08 Thread Kostas Sakellis (JIRA)
Kostas Sakellis created SPARK-5674: -- Summary: Spark Job Explain Plan Proof of Concept Key: SPARK-5674 URL: https://issues.apache.org/jira/browse/SPARK-5674 Project: Spark Issue Type:

[jira] [Commented] (SPARK-5635) Allow users to run .scala files directly from spark-submit

2015-02-08 Thread Grant Henke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311588#comment-14311588 ] Grant Henke commented on SPARK-5635: The method I listed I thought to be a workaround

[jira] [Commented] (SPARK-5625) Spark binaries do not incude Spark Core

2015-02-08 Thread DeepakVohra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311596#comment-14311596 ] DeepakVohra commented on SPARK-5625: Thanks Sean. Spark binaries do not incude Spark

[jira] [Commented] (SPARK-5631) Server IPC version 7 cannot communicate with client version 4

2015-02-08 Thread DeepakVohra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311597#comment-14311597 ] DeepakVohra commented on SPARK-5631: Thanks for the clarification. Server IPC

[jira] [Comment Edited] (SPARK-5631) Server IPC version 7 cannot communicate with client version 4

2015-02-08 Thread DeepakVohra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311597#comment-14311597 ] DeepakVohra edited comment on SPARK-5631 at 2/8/15 10:38 PM: -

[jira] [Commented] (SPARK-2958) FileClientHandler should not be shared in the pipeline

2015-02-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311602#comment-14311602 ] Reynold Xin commented on SPARK-2958: cc [~adav] this is no longer a problem in the new

[jira] [Updated] (SPARK-3991) Not Serializable , Nullpinter Exceptions in SQL server mode

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-3991: - Priority: Major (was: Blocker) Downgrading until it's clear what the issue is. There are several items

[jira] [Updated] (SPARK-3034) [HIve] java.sql.Date cannot be cast to java.sql.Timestamp

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-3034: - Component/s: (was: Spark Core) Priority: Major (was: Blocker) Can you provide steps to

[jira] [Resolved] (SPARK-2998) scala.collection.mutable.HashSet cannot be cast to scala.collection.mutable.BitSet

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2998. -- Resolution: Duplicate scala.collection.mutable.HashSet cannot be cast to

[jira] [Resolved] (SPARK-4840) Incorrect documentation of master url on Running Spark on Mesos page

2015-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4840. -- Resolution: Not a Problem OK, if that doesn't prove to be the answer, reopen with more info.

  1   2   >