[jira] [Resolved] (SPARK-1779) Warning when spark.storage.memoryFraction is not between 0 and 1

2014-08-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1779. Resolution: Fixed Fixed via: https://github.com/apache/spark/pull/714 Warning when

[jira] [Created] (SPARK-2859) Update url of Kryo project in tuning.md

2014-08-05 Thread Guancheng Chen (JIRA)
Guancheng Chen created SPARK-2859: - Summary: Update url of Kryo project in tuning.md Key: SPARK-2859 URL: https://issues.apache.org/jira/browse/SPARK-2859 Project: Spark Issue Type:

[jira] [Commented] (SPARK-2862) DoubleRDDFunctions.histogram() throws exception for some inputs

2014-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086182#comment-14086182 ] Apache Spark commented on SPARK-2862: - User 'nrchandan' has created a pull request for

[jira] [Updated] (SPARK-2862) DoubleRDDFunctions.histogram() throws exception for some inputs

2014-08-05 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-2862: --- Affects Version/s: 1.0.1 DoubleRDDFunctions.histogram() throws exception for some inputs

[jira] [Updated] (SPARK-2861) Doc comment of DoubleRDDFunctions.histogram is incorrect

2014-08-05 Thread Chandan Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan Kumar updated SPARK-2861: - Description: The documentation comment of histogram method of DoubleRDDFunctions class in

[jira] [Created] (SPARK-2863) Emulate Hive type coercion in native reimplementations of Hive UDFs

2014-08-05 Thread William Benton (JIRA)
William Benton created SPARK-2863: - Summary: Emulate Hive type coercion in native reimplementations of Hive UDFs Key: SPARK-2863 URL: https://issues.apache.org/jira/browse/SPARK-2863 Project: Spark

[jira] [Commented] (SPARK-2636) no where to get job identifier while submit spark job through spark API

2014-08-05 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086494#comment-14086494 ] Marcelo Vanzin commented on SPARK-2636: --- (BTW, just checked SPARK-2321, so if you

[jira] [Created] (SPARK-2864) fix random seed in Word2Vec

2014-08-05 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-2864: Summary: fix random seed in Word2Vec Key: SPARK-2864 URL: https://issues.apache.org/jira/browse/SPARK-2864 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2622) Add Jenkins build numbers to SparkQA messages

2014-08-05 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086513#comment-14086513 ] Xiangrui Meng commented on SPARK-2622: -- The build number is included in the SparkQA

[jira] [Closed] (SPARK-2622) Add Jenkins build numbers to SparkQA messages

2014-08-05 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-2622. Resolution: Fixed Add Jenkins build numbers to SparkQA messages

[jira] [Resolved] (SPARK-1890) add modify acls to the web UI for the kill button

2014-08-05 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1890. -- Resolution: Fixed add modify acls to the web UI for the kill button

[jira] [Commented] (SPARK-2854) Finalize _acceptable_types in pyspark.sql

2014-08-05 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086560#comment-14086560 ] Yin Huai commented on SPARK-2854: - Since we have already do conversions for ByteType and

[jira] [Created] (SPARK-2865) Potential deadlock: tasks could hang forever waiting to fetch a remote block even though most tasks finish

2014-08-05 Thread Zongheng Yang (JIRA)
Zongheng Yang created SPARK-2865: Summary: Potential deadlock: tasks could hang forever waiting to fetch a remote block even though most tasks finish Key: SPARK-2865 URL:

[jira] [Updated] (SPARK-2865) Potential deadlock: tasks could hang forever waiting to fetch a remote block even though most tasks finish

2014-08-05 Thread Zongheng Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zongheng Yang updated SPARK-2865: - Description: In the application I tested, most of the tasks out of 128 tasks could finish, but

[jira] [Updated] (SPARK-2865) Potential deadlock: tasks could hang forever waiting to fetch a remote block even though most tasks finish

2014-08-05 Thread Zongheng Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zongheng Yang updated SPARK-2865: - Description: In the application I tested, most of the tasks out of 128 tasks could finish, but

[jira] [Resolved] (SPARK-2860) Resolving CASE WHEN throws None.get exception

2014-08-05 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2860. - Resolution: Fixed Fix Version/s: 1.1.0 Resolving CASE WHEN throws None.get

[jira] [Created] (SPARK-2866) ORDER BY attributes must appear in SELECT clause

2014-08-05 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-2866: --- Summary: ORDER BY attributes must appear in SELECT clause Key: SPARK-2866 URL: https://issues.apache.org/jira/browse/SPARK-2866 Project: Spark Issue

[jira] [Resolved] (SPARK-2859) Update url of Kryo project in related docs

2014-08-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2859. Resolution: Fixed Fix Version/s: 1.1.0 1.0.3 Issue resolved by

[jira] [Updated] (SPARK-2534) Avoid pulling in the entire RDD or PairRDDFunctions in various operators

2014-08-05 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-2534: - Component/s: Spark Core Avoid pulling in the entire RDD or PairRDDFunctions in various

[jira] [Created] (SPARK-2867) saveAsHadoopFile() in PairRDDFunction.scala should allow use other OutputCommiter class

2014-08-05 Thread Joseph Su (JIRA)
Joseph Su created SPARK-2867: Summary: saveAsHadoopFile() in PairRDDFunction.scala should allow use other OutputCommiter class Key: SPARK-2867 URL: https://issues.apache.org/jira/browse/SPARK-2867

[jira] [Updated] (SPARK-1977) mutable.BitSet in ALS not serializable with KryoSerializer

2014-08-05 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-1977: - Fix Version/s: (was: 1.0.1) 1.0.2 mutable.BitSet in ALS not serializable

[jira] [Commented] (SPARK-1834) NoSuchMethodError when invoking JavaPairRDD.reduce() in Java

2014-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086649#comment-14086649 ] Sean Owen commented on SPARK-1834: -- Ah, you're right:

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

2014-08-05 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086657#comment-14086657 ] Zhan Zhang commented on SPARK-1537: --- I am also interested in it and trying to integrate

[jira] [Commented] (SPARK-2848) Shade Guava in Spark deliverables

2014-08-05 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086670#comment-14086670 ] Marcelo Vanzin commented on SPARK-2848: --- Question for others ([~pwendell], [~sowen],

[jira] [Updated] (SPARK-2699) Improve compatibility with parquet file/table

2014-08-05 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2699: Target Version/s: 1.2.0 (was: 1.1.0) Improve compatibility with parquet file/table

[jira] [Updated] (SPARK-2380) Support displaying accumulator contents in the web UI

2014-08-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2380: --- Fix Version/s: 1.1.0 Support displaying accumulator contents in the web UI

[jira] [Resolved] (SPARK-2380) Support displaying accumulator contents in the web UI

2014-08-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2380. Resolution: Fixed Resolved by: https://github.com/apache/spark/pull/1309 Support

[jira] [Created] (SPARK-2868) Support named accumulators in Python

2014-08-05 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-2868: -- Summary: Support named accumulators in Python Key: SPARK-2868 URL: https://issues.apache.org/jira/browse/SPARK-2868 Project: Spark Issue Type: New

[jira] [Assigned] (SPARK-2583) ConnectionManager cannot distinguish whether error occurred or not

2014-08-05 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-2583: - Assignee: Josh Rosen (was: Kousuke Saruta) ConnectionManager cannot distinguish whether error

[jira] [Commented] (SPARK-1856) Standardize MLlib interfaces

2014-08-05 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086697#comment-14086697 ] Xiangrui Meng commented on SPARK-1856: -- Yes, MLI and MLbase are research projects at

[jira] [Resolved] (SPARK-1680) Clean up use of setExecutorEnvs in SparkConf

2014-08-05 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1680. -- Resolution: Fixed Clean up use of setExecutorEnvs in SparkConf

[jira] [Updated] (SPARK-2863) Emulate Hive type coercion in native reimplementations of Hive functions

2014-08-05 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2863: Assignee: William Benton Emulate Hive type coercion in native reimplementations of Hive

[jira] [Updated] (SPARK-2844) Existing JVM Hive Context not correctly used in Python Hive Context

2014-08-05 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2844: Priority: Major (was: Minor) Target Version/s: 1.1.0 Existing JVM Hive

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

2014-08-05 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086808#comment-14086808 ] Zhan Zhang commented on SPARK-1537: --- Do you mind sharing your thoughts, design document

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

2014-08-05 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086817#comment-14086817 ] Marcelo Vanzin commented on SPARK-1537: --- Currently busy with other more urgent

[jira] [Commented] (SPARK-2854) Finalize _acceptable_types in pyspark.sql

2014-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086877#comment-14086877 ] Apache Spark commented on SPARK-2854: - User 'yhuai' has created a pull request for

[jira] [Updated] (SPARK-2650) Wrong initial sizes for in-memory column buffers

2014-08-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-2650: -- Target Version/s: 1.2.0 (was: 1.1.0) Wrong initial sizes for in-memory column buffers

[jira] [Commented] (SPARK-2866) ORDER BY attributes must appear in SELECT clause

2014-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087077#comment-14087077 ] Apache Spark commented on SPARK-2866: - User 'marmbrus' has created a pull request for

[jira] [Commented] (SPARK-2636) no where to get job identifier while submit spark job through spark API

2014-08-05 Thread Chengxiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087102#comment-14087102 ] Chengxiang Li commented on SPARK-2636: -- {quote} There are two ways I think. One is

[jira] [Created] (SPARK-2872) Fix conflict between code and doc in YarnClientSchedulerBackend

2014-08-05 Thread Zhihui (JIRA)
Zhihui created SPARK-2872: - Summary: Fix conflict between code and doc in YarnClientSchedulerBackend Key: SPARK-2872 URL: https://issues.apache.org/jira/browse/SPARK-2872 Project: Spark Issue Type:

[jira] [Commented] (SPARK-2872) Fix conflict between code and doc in YarnClientSchedulerBackend

2014-08-05 Thread Zhihui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087120#comment-14087120 ] Zhihui commented on SPARK-2872: --- PR https://github.com/apache/spark/pull/1684 Fix conflict

[jira] [Commented] (SPARK-2848) Shade Guava in Spark deliverables

2014-08-05 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087179#comment-14087179 ] Marcelo Vanzin commented on SPARK-2848: --- Nevermind the question, I got code mostly

[jira] [Created] (SPARK-2874) Spark SQL related scripts don't show complete usage message

2014-08-05 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-2874: - Summary: Spark SQL related scripts don't show complete usage message Key: SPARK-2874 URL: https://issues.apache.org/jira/browse/SPARK-2874 Project: Spark Issue