[jira] [Commented] (SPARK-4819) Remove Guava's Optional from public API

[ https://issues.apache.org/jira/browse/SPARK-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311610#comment-14311610 ] Sean Owen commented on SPARK-4819: -- Yeah there are several things we can't do until Spark

[jira] [Resolved] (SPARK-2072) Streaming not processing a file with particular number of entries

[ https://issues.apache.org/jira/browse/SPARK-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2072. -- Resolution: Cannot Reproduce Streaming not processing a file with particular number of entries

[jira] [Updated] (SPARK-4598) Paginate stage page to avoid OOM with 100,000 tasks

[ https://issues.apache.org/jira/browse/SPARK-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4598: - Component/s: (was: Spark Core) Web UI Priority: Major (was: Critical)

[jira] [Resolved] (SPARK-3171) Don't print meaningless information of SelectionKey

[ https://issues.apache.org/jira/browse/SPARK-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3171. -- Resolution: Won't Fix Per PR discussion, sounds like WontFix due to risk of log messages generating

[jira] [Updated] (SPARK-3171) Don't print meaningless information of SelectionKey

[ https://issues.apache.org/jira/browse/SPARK-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-3171: - Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) Don't print meaningless

[jira] [Commented] (SPARK-4544) Spark JVM Metrics doesn't have context.

[ https://issues.apache.org/jira/browse/SPARK-4544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311613#comment-14311613 ] Sean Owen commented on SPARK-4544: -- Can you clarify what you mean? are you asking whether

[jira] [Resolved] (SPARK-755) Kryo serialization failing - MLbase

[ https://issues.apache.org/jira/browse/SPARK-755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-755. - Resolution: Cannot Reproduce Kryo serialization failing - MLbase ---

[jira] [Commented] (SPARK-4311) ContainerLauncher setting up executor -- invalid Xms settings (-Xms0m -Xmx0m)

[ https://issues.apache.org/jira/browse/SPARK-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311625#comment-14311625 ] Sean Owen commented on SPARK-4311: -- Is this still reproducible? ContainerLauncher

[jira] [Resolved] (SPARK-3482) Allow symlinking to scripts (spark-shell, spark-submit, ...)

[ https://issues.apache.org/jira/browse/SPARK-3482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3482. -- Resolution: Duplicate Allow symlinking to scripts (spark-shell, spark-submit, ...)

[jira] [Resolved] (SPARK-5665) Update netlib-java documentation

[ https://issues.apache.org/jira/browse/SPARK-5665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-5665. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4448

[jira] [Updated] (SPARK-5665) Update netlib-java documentation

[ https://issues.apache.org/jira/browse/SPARK-5665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-5665: - Assignee: Sam Halliday Update netlib-java documentation

[jira] [Commented] (SPARK-839) Bug in how failed executors are removed by ID from standalone cluster

[ https://issues.apache.org/jira/browse/SPARK-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311734#comment-14311734 ] Mark Hamstra commented on SPARK-839: Fixed long ago. Bug in how failed executors are

[jira] [Closed] (SPARK-839) Bug in how failed executors are removed by ID from standalone cluster

[ https://issues.apache.org/jira/browse/SPARK-839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Hamstra closed SPARK-839. -- Resolution: Fixed Bug in how failed executors are removed by ID from standalone cluster

[jira] [Reopened] (SPARK-5613) YarnClientSchedulerBackend fails to get application report when yarn restarts

[ https://issues.apache.org/jira/browse/SPARK-5613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kashish Jain reopened SPARK-5613: - The fix has been cherry-picked to 1.2 and master branch but not 1.3. YarnClientSchedulerBackend

[jira] [Created] (SPARK-5679) Flaky tests in InputOutputMetricsSuite: input metrics with interleaved reads and .input metrics with mixed read method

Patrick Wendell created SPARK-5679: -- Summary: Flaky tests in InputOutputMetricsSuite: input metrics with interleaved reads and .input metrics with mixed read method Key: SPARK-5679 URL:

[jira] [Updated] (SPARK-5679) Flaky tests in InputOutputMetricsSuite: input metrics with interleaved reads and input metrics with mixed read method

[ https://issues.apache.org/jira/browse/SPARK-5679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5679: --- Description: Please audit these and see if there are any assumptions with respect to File IO

[jira] [Updated] (SPARK-5679) Flaky tests in InputOutputMetricsSuite: input metrics with interleaved reads and input metrics with mixed read method

[ https://issues.apache.org/jira/browse/SPARK-5679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5679: --- Summary: Flaky tests in InputOutputMetricsSuite: input metrics with interleaved reads and

[jira] [Created] (SPARK-5680) Sum function on all null values, should return zero

Venkata Ramana G created SPARK-5680: --- Summary: Sum function on all null values, should return zero Key: SPARK-5680 URL: https://issues.apache.org/jira/browse/SPARK-5680 Project: Spark

[jira] [Commented] (SPARK-5680) Sum function on all null values, should return zero

[ https://issues.apache.org/jira/browse/SPARK-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311892#comment-14311892 ] Apache Spark commented on SPARK-5680: - User 'gvramana' has created a pull request for

[jira] [Resolved] (SPARK-5539) User guide for LDA

[ https://issues.apache.org/jira/browse/SPARK-5539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-5539. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4465

[jira] [Created] (SPARK-5681) Calling graceful stop() immediately after start() on StreamingContext should not get stuck indefinitely

Liang-Chi Hsieh created SPARK-5681: -- Summary: Calling graceful stop() immediately after start() on StreamingContext should not get stuck indefinitely Key: SPARK-5681 URL:

[jira] [Updated] (SPARK-5658) Finalize DDL and write support APIs

[ https://issues.apache.org/jira/browse/SPARK-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-5658: Issue Type: Sub-task (was: Improvement) Parent: SPARK-5180 Finalize DDL and write support APIs

[jira] [Resolved] (SPARK-5643) Add a show method to print the content of a DataFrame in columnar format

[ https://issues.apache.org/jira/browse/SPARK-5643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-5643. Resolution: Fixed Fix Version/s: 1.3.0 Add a show method to print the content of a

[jira] [Updated] (SPARK-5675) XyzType companion object should subclass XyzType

[ https://issues.apache.org/jira/browse/SPARK-5675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5675: --- Summary: XyzType companion object should subclass XyzType (was: StringType case object should be

[jira] [Updated] (SPARK-5675) XyzType companion object should subclass XyzType

[ https://issues.apache.org/jira/browse/SPARK-5675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5675: --- Description: Otherwise, the following will always be false in Java: {code} dataType instanceof

[jira] [Created] (SPARK-5682) Reuse hadoop encrypted shuffle algorithm to enable spark encrypted shuffle

liyunzhang_intel created SPARK-5682: --- Summary: Reuse hadoop encrypted shuffle algorithm to enable spark encrypted shuffle Key: SPARK-5682 URL: https://issues.apache.org/jira/browse/SPARK-5682

[jira] [Commented] (SPARK-5681) Calling graceful stop() immediately after start() on StreamingContext should not get stuck indefinitely

[ https://issues.apache.org/jira/browse/SPARK-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311904#comment-14311904 ] Apache Spark commented on SPARK-5681: - User 'viirya' has created a pull request for

[jira] [Created] (SPARK-5676) License missing from spark-ec2 repo

Florian Verhein created SPARK-5676: -- Summary: License missing from spark-ec2 repo Key: SPARK-5676 URL: https://issues.apache.org/jira/browse/SPARK-5676 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4896) Don't redundantly copy executor dependencies in Utils.fetchFile

[ https://issues.apache.org/jira/browse/SPARK-4896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4896: --- Component/s: Spark Core Don't redundantly copy executor dependencies in Utils.fetchFile

[jira] [Updated] (SPARK-5355) SparkConf is not thread-safe

[ https://issues.apache.org/jira/browse/SPARK-5355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5355: --- Component/s: Spark Core SparkConf is not thread-safe

[jira] [Closed] (SPARK-3587) Spark SQL can't support lead() over() window function

[ https://issues.apache.org/jira/browse/SPARK-3587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-3587. -- Resolution: Duplicate Assignee: (was: William Benton) Spark SQL can't support lead() over()

[jira] [Updated] (SPARK-5677) Python DataFrame API remaining tasks

[ https://issues.apache.org/jira/browse/SPARK-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5677: --- Assignee: Davies Liu Python DataFrame API remaining tasks

[jira] [Created] (SPARK-5677) Python DataFrame API remaining tasks

Reynold Xin created SPARK-5677: -- Summary: Python DataFrame API remaining tasks Key: SPARK-5677 URL: https://issues.apache.org/jira/browse/SPARK-5677 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-5539) User guide for LDA

[ https://issues.apache.org/jira/browse/SPARK-5539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311845#comment-14311845 ] Apache Spark commented on SPARK-5539: - User 'mengxr' has created a pull request for

[jira] [Created] (SPARK-5678) DataFrame.to_pandas

Reynold Xin created SPARK-5678: -- Summary: DataFrame.to_pandas Key: SPARK-5678 URL: https://issues.apache.org/jira/browse/SPARK-5678 Project: Spark Issue Type: Sub-task Components:

[jira] [Closed] (SPARK-2958) FileClientHandler should not be shared in the pipeline

[ https://issues.apache.org/jira/browse/SPARK-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson closed SPARK-2958. - Resolution: Done FileClientHandler should not be shared in the pipeline

[jira] [Commented] (SPARK-2958) FileClientHandler should not be shared in the pipeline

[ https://issues.apache.org/jira/browse/SPARK-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311796#comment-14311796 ] Aaron Davidson commented on SPARK-2958: --- This class no longer exists (I think we

[jira] [Resolved] (SPARK-5660) Local Matrix apply should be public

[ https://issues.apache.org/jira/browse/SPARK-5660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-5660. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4462

[jira] [Resolved] (SPARK-4405) Matrices.* construction methods should check for rows x cols overflow

[ https://issues.apache.org/jira/browse/SPARK-4405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-4405. -- Resolution: Fixed Fix Version/s: 1.3.0 Matrices.* construction methods should check for

[jira] [Updated] (SPARK-4405) Matrices.* construction methods should check for rows x cols overflow

[ https://issues.apache.org/jira/browse/SPARK-4405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4405: - Assignee: Sean Owen Matrices.* construction methods should check for rows x cols overflow

[jira] [Updated] (SPARK-4405) Matrices.* construction methods should check for rows x cols overflow

[ https://issues.apache.org/jira/browse/SPARK-4405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4405: - Target Version/s: 1.3.0 (was: 1.2.0) Matrices.* construction methods should check for rows x

[jira] [Updated] (SPARK-5289) Backport publishing of repl, yarn into branch-1.2

[ https://issues.apache.org/jira/browse/SPARK-5289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5289: --- Component/s: Build Backport publishing of repl, yarn into branch-1.2

[jira] [Updated] (SPARK-5254) Update the user guide to make clear that spark.mllib is not being deprecated

[ https://issues.apache.org/jira/browse/SPARK-5254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5254: --- Component/s: MLlib Update the user guide to make clear that spark.mllib is not being

[jira] [Updated] (SPARK-5308) MD5 / SHA1 hash format doesn't match standard Maven output

[ https://issues.apache.org/jira/browse/SPARK-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5308: --- Fix Version/s: (was: 1.2.1) MD5 / SHA1 hash format doesn't match standard Maven output

[jira] [Updated] (SPARK-5308) MD5 / SHA1 hash format doesn't match standard Maven output

[ https://issues.apache.org/jira/browse/SPARK-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5308: --- Fix Version/s: 1.2.1 MD5 / SHA1 hash format doesn't match standard Maven output

[jira] [Updated] (SPARK-2444) Make spark.yarn.executor.memoryOverhead a first class citizen

[ https://issues.apache.org/jira/browse/SPARK-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-2444: - Priority: Minor (was: Major) Fix Version/s: 1.2.0 Make spark.yarn.executor.memoryOverhead a

[jira] [Updated] (SPARK-2444) Make spark.yarn.executor.memoryOverhead a first class citizen

[ https://issues.apache.org/jira/browse/SPARK-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-2444: - Assignee: Nishkam Ravi PS the commit was

[jira] [Resolved] (SPARK-2444) Make spark.yarn.executor.memoryOverhead a first class citizen

[ https://issues.apache.org/jira/browse/SPARK-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2444. -- Resolution: Fixed This looks like it was in the docs from 1.2.0 Make

[jira] [Commented] (SPARK-5420) Cross-langauge load/store functions for creating and saving DataFrames

[ https://issues.apache.org/jira/browse/SPARK-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311710#comment-14311710 ] Yin Huai commented on SPARK-5420: - APIs will be finalized by the work of SPARK-5658.

[jira] [Commented] (SPARK-4617) Fix spark.yarn.applicationMaster.waitTries doc

[ https://issues.apache.org/jira/browse/SPARK-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311765#comment-14311765 ] Sandy Ryza commented on SPARK-4617: --- This got fixed by SPARK-3779. Closing now. Fix

[jira] [Commented] (SPARK-5472) Add support for reading from and writing to a JDBC database

[ https://issues.apache.org/jira/browse/SPARK-5472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311766#comment-14311766 ] Apache Spark commented on SPARK-5472: - User 'hunglin' has created a pull request for

[jira] [Resolved] (SPARK-4617) Fix spark.yarn.applicationMaster.waitTries doc

[ https://issues.apache.org/jira/browse/SPARK-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved SPARK-4617. --- Resolution: Not a Problem Fix spark.yarn.applicationMaster.waitTries doc

[jira] [Resolved] (SPARK-4290) Provide an equivalent functionality of distributed cache as MR does

[ https://issues.apache.org/jira/browse/SPARK-4290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4290. -- Resolution: Not a Problem It sounds like SparkContext.addFile is enough for this use case, and if not,

[jira] [Resolved] (SPARK-5598) Model import/export for ALS

[ https://issues.apache.org/jira/browse/SPARK-5598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-5598. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4422

[jira] [Created] (SPARK-5675) StringType case object should be subclass of StringType class

Reynold Xin created SPARK-5675: -- Summary: StringType case object should be subclass of StringType class Key: SPARK-5675 URL: https://issues.apache.org/jira/browse/SPARK-5675 Project: Spark

[jira] [Commented] (SPARK-4405) Matrices.* construction methods should check for rows x cols overflow

[ https://issues.apache.org/jira/browse/SPARK-4405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311621#comment-14311621 ] Apache Spark commented on SPARK-4405: - User 'srowen' has created a pull request for

[jira] [Commented] (SPARK-5660) Local Matrix apply should be public

[ https://issues.apache.org/jira/browse/SPARK-5660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311668#comment-14311668 ] Apache Spark commented on SPARK-5660: - User 'mengxr' has created a pull request for

[jira] [Commented] (SPARK-4503) The history server is not compatible with HDFS HA

[ https://issues.apache.org/jira/browse/SPARK-4503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311667#comment-14311667 ] Sanghyun Yun commented on SPARK-4503: - I think you need to add following attributes in

[jira] [Comment Edited] (SPARK-4503) The history server is not compatible with HDFS HA

[ https://issues.apache.org/jira/browse/SPARK-4503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311667#comment-14311667 ] Sanghyun Yun edited comment on SPARK-4503 at 2/9/15 12:25 AM: --

[jira] [Commented] (SPARK-4833) spark-shell.cmd not starting again after a while in windows 8.1

[ https://issues.apache.org/jira/browse/SPARK-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311715#comment-14311715 ] Sean Owen commented on SPARK-4833: -- Is this still happening, and I wonder if it's the

[jira] [Updated] (SPARK-4833) spark-shell.cmd not starting again after a while in windows 8.1

[ https://issues.apache.org/jira/browse/SPARK-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4833: - Component/s: Windows The There are no more files message turns up hits about a corrupted Windows system.

[jira] [Resolved] (SPARK-5468) Remove Python LocalHiveContext

[ https://issues.apache.org/jira/browse/SPARK-5468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-5468. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Reynold Xin Remove Python

[jira] [Updated] (SPARK-2208) local metrics tests can fail on fast machines

[ https://issues.apache.org/jira/browse/SPARK-2208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-2208: - Component/s: (was: Spark Core) Tests Priority: Minor (was: Major) local

[jira] [Issue Comment Deleted] (SPARK-5420) Cross-langauge load/store functions for creating and saving DataFrames

[ https://issues.apache.org/jira/browse/SPARK-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-5420: Comment: was deleted (was: I am copying the summary of write related interfaces from

[jira] [Issue Comment Deleted] (SPARK-5420) Cross-langauge load/store functions for creating and saving DataFrames

[ https://issues.apache.org/jira/browse/SPARK-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-5420: Comment: was deleted (was: h3. End user APIs added to SQLContext (load related) h4. Load data through a

[jira] [Issue Comment Deleted] (SPARK-5501) Write support for the data source API

[ https://issues.apache.org/jira/browse/SPARK-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-5501: Comment: was deleted (was: h3. End user APIs added to DataFrame (write related) h4. Save a DataFrame as a

<    1   2