[jira] [Commented] (SPARK-2786) Python correlations

2014-07-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081978#comment-14081978 ] Apache Spark commented on SPARK-2786: - User 'dorx' has created a pull request for this

[jira] [Resolved] (SPARK-2648) Randomize order of executors when fetching shuffle blocks

2014-07-31 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2648. Resolution: Not a Problem It turns out we already randomized these, just in a different par

[jira] [Created] (SPARK-2786) Python correlations

2014-07-31 Thread Doris Xin (JIRA)
Doris Xin created SPARK-2786: Summary: Python correlations Key: SPARK-2786 URL: https://issues.apache.org/jira/browse/SPARK-2786 Project: Spark Issue Type: Sub-task Reporter: Doris Xi

[jira] [Resolved] (SPARK-2738) Remove redundant imports in BlockManagerSuite

2014-07-31 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2738. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1642 [https://

[jira] [Updated] (SPARK-2738) Remove redundant imports in BlockManagerSuite

2014-07-31 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2738: --- Assignee: Sandy Ryza > Remove redundant imports in BlockManagerSuite > --

[jira] [Resolved] (SPARK-2576) slave node throws NoClassDefFoundError $line11.$read$ when executing a Spark QL query on HDFS CSV file

2014-07-31 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2576. - Resolution: Fixed Fix Version/s: 1.1.0 > slave node throws NoClassDefFoundError $l

[jira] [Resolved] (SPARK-2632) Importing a method of class in Spark REPL causes the REPL to pulls in unnecessary stuff.

2014-07-31 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2632. - Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Prashant Sharma > Import

[jira] [Resolved] (SPARK-2702) Upgrade Tachyon dependency to 0.5.0

2014-07-31 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2702. Resolution: Fixed Issue resolved by pull request 1651 [https://github.com/apache/spark/pull

[jira] [Commented] (SPARK-2780) Create a StreamingContext.setLocalProperty for setting local property of jobs launched by streaming

2014-07-31 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081947#comment-14081947 ] Saisai Shao commented on SPARK-2780: Hi TD, I think the fair scheduler setting can be

[jira] [Updated] (SPARK-2692) Decision Tree API update

2014-07-31 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2692: - Target Version/s: 1.2.0 (was: 1.1.0) > Decision Tree API update > > >

[jira] [Commented] (SPARK-2201) Improve FlumeInputDStream's stability and make it scalable

2014-07-31 Thread sunsc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081935#comment-14081935 ] sunsc commented on SPARK-2201: -- The problem of the original implementation is that the config

[jira] [Updated] (SPARK-2309) Generalize the binary logistic regression into multinomial logistic regression

2014-07-31 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2309: - Target Version/s: 1.2.0 (was: 1.1.0) > Generalize the binary logistic regression into multinomia

[jira] [Created] (SPARK-2785) Avoid asserts for unimplemented hive features

2014-07-31 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-2785: --- Summary: Avoid asserts for unimplemented hive features Key: SPARK-2785 URL: https://issues.apache.org/jira/browse/SPARK-2785 Project: Spark Issue Type:

[jira] [Commented] (SPARK-2179) Public API for DataTypes and Schema

2014-07-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081903#comment-14081903 ] Apache Spark commented on SPARK-2179: - User 'yhuai' has created a pull request for thi

[jira] [Created] (SPARK-2784) Make language configurable using SQLConf instead of hql/sql functions

2014-07-31 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-2784: --- Summary: Make language configurable using SQLConf instead of hql/sql functions Key: SPARK-2784 URL: https://issues.apache.org/jira/browse/SPARK-2784 Project: Sp

[jira] [Created] (SPARK-2783) Basic support for analyze in HiveContext

2014-07-31 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-2783: --- Summary: Basic support for analyze in HiveContext Key: SPARK-2783 URL: https://issues.apache.org/jira/browse/SPARK-2783 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-2767) SparkSQL CLI doens't output error message if query failed.

2014-07-31 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2767: Target Version/s: 1.1.0 > SparkSQL CLI doens't output error message if query failed. >

[jira] [Updated] (SPARK-2220) Fix remaining Hive Commands

2014-07-31 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2220: Target Version/s: 1.2.0 (was: 1.1.0) > Fix remaining Hive Commands > -

[jira] [Resolved] (SPARK-2779) asInstanceOf[Map[...]] should use scala.collection.Map instead of scala.collection.immutable.Map

2014-07-31 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2779. - Resolution: Fixed Fix Version/s: 1.1.0 > asInstanceOf[Map[...]] should use scala.c

[jira] [Resolved] (SPARK-2782) Spearman correlation computes wrong ranks when numPartitions > RDD size

2014-07-31 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-2782. -- Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1710 [https://gith

[jira] [Resolved] (SPARK-2777) ALS factors should be persist in memory and disk

2014-07-31 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-2777. -- Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1700 [https://gith

[jira] [Resolved] (SPARK-2766) ScalaReflectionSuite throw an llegalArgumentException in JDK 6

2014-07-31 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2766. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1683 [https://

[jira] [Resolved] (SPARK-2756) Decision Tree bugs

2014-07-31 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-2756. -- Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1673 [https://gith

[jira] [Resolved] (SPARK-2724) Python version of Random RDD without support for arbitrary distribution

2014-07-31 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-2724. -- Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1628 [https://gith

[jira] [Commented] (SPARK-1842) update scala-logging-slf4j to version 2.1.2

2014-07-31 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081866#comment-14081866 ] Guoqiang Li commented on SPARK-1842: [~avati] related work: https://github.com/apache

[jira] [Resolved] (SPARK-2145) Add lower bound on sampling rate to guarantee sampling performance

2014-07-31 Thread Doris Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doris Xin resolved SPARK-2145. -- Resolution: Fixed Resolved with PR https://github.com/apache/spark/pull/1025 > Add lower bound on samp

[jira] [Commented] (SPARK-1812) Support cross-building with Scala 2.11

2014-07-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081862#comment-14081862 ] Apache Spark commented on SPARK-1812: - User 'avati' has created a pull request for thi

[jira] [Commented] (SPARK-1842) update scala-logging-slf4j to version 2.1.2

2014-07-31 Thread Anand Avati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081863#comment-14081863 ] Anand Avati commented on SPARK-1842: Posted https://github.com/apache/spark/pull/1701

[jira] [Commented] (SPARK-2782) Spearman correlation computes wrong ranks when numPartitions > RDD size

2014-07-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081861#comment-14081861 ] Apache Spark commented on SPARK-2782: - User 'dorx' has created a pull request for this

[jira] [Created] (SPARK-2782) Spearman correlation computes wrong ranks when numPartitions > RDD size

2014-07-31 Thread Doris Xin (JIRA)
Doris Xin created SPARK-2782: Summary: Spearman correlation computes wrong ranks when numPartitions > RDD size Key: SPARK-2782 URL: https://issues.apache.org/jira/browse/SPARK-2782 Project: Spark

[jira] [Commented] (SPARK-1812) Support cross-building with Scala 2.11

2014-07-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081855#comment-14081855 ] Apache Spark commented on SPARK-1812: - User 'avati' has created a pull request for thi

[jira] [Commented] (SPARK-1812) Support cross-building with Scala 2.11

2014-07-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081856#comment-14081856 ] Apache Spark commented on SPARK-1812: - User 'avati' has created a pull request for thi

[jira] [Commented] (SPARK-2711) Create a ShuffleMemoryManager that allocates across spilling collections in the same task

2014-07-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081852#comment-14081852 ] Apache Spark commented on SPARK-2711: - User 'mateiz' has created a pull request for th

[jira] [Updated] (SPARK-2711) Create a ShuffleMemoryManager that allocates across spilling collections in the same task

2014-07-31 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-2711: - Target Version/s: 1.1.0 > Create a ShuffleMemoryManager that allocates across spilling collection

[jira] [Resolved] (SPARK-2436) Apply size-based optimization to planning BroadcastNestedLoopJoin

2014-07-31 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2436. - Resolution: Fixed Fix Version/s: 1.1.0 > Apply size-based optimization to planning

[jira] [Resolved] (SPARK-2531) Make BroadcastNestedLoopJoin take into account a BuildSide

2014-07-31 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2531. - Resolution: Fixed Fix Version/s: 1.1.0 Target Version/s: 1.1.0 (was: 1.1

[jira] [Updated] (SPARK-2781) Analyzer should check resolution of LogicalPlans

2014-07-31 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2781: Target Version/s: 1.2.0 > Analyzer should check resolution of LogicalPlans > --

[jira] [Updated] (SPARK-2781) Analyzer should check resolution of LogicalPlans

2014-07-31 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2781: Fix Version/s: (was: 1.0.1) (was: 1.1.0) > Analyzer should check

[jira] [Updated] (SPARK-2781) Analyzer should check resolution of LogicalPlans

2014-07-31 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2781: Affects Version/s: 1.1.0 1.0.1 > Analyzer should check resolution of

[jira] [Commented] (SPARK-2781) Analyzer should check resolution of LogicalPlans

2014-07-31 Thread Aaron Staple (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081833#comment-14081833 ] Aaron Staple commented on SPARK-2781: - No problem, I think the current validation chec

[jira] [Reopened] (SPARK-2781) Analyzer should check resolution of LogicalPlans

2014-07-31 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reopened SPARK-2781: - I'm sorry... I thought this was stale and did not read it carefully. Reopening. > Analyzer s

[jira] [Commented] (SPARK-2781) Analyzer should check resolution of LogicalPlans

2014-07-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081828#comment-14081828 ] Apache Spark commented on SPARK-2781: - User 'staple' has created a pull request for th

[jira] [Resolved] (SPARK-2781) Analyzer should check resolution of LogicalPlans

2014-07-31 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2781. - Resolution: Fixed Fix Version/s: 1.1.0 1.0.1 Assignee:

[jira] [Created] (SPARK-2781) Analyzer should check resolution of LogicalPlans

2014-07-31 Thread Aaron Staple (JIRA)
Aaron Staple created SPARK-2781: --- Summary: Analyzer should check resolution of LogicalPlans Key: SPARK-2781 URL: https://issues.apache.org/jira/browse/SPARK-2781 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-2780) Create a StreamingContext.setLocalProperty for setting local property of jobs launched by streaming

2014-07-31 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-2780: Summary: Create a StreamingContext.setLocalProperty for setting local property of jobs launched by streaming Key: SPARK-2780 URL: https://issues.apache.org/jira/browse/SPARK-2780

[jira] [Commented] (SPARK-2779) asInstanceOf[Map[...]] should use scala.collection.Map instead of scala.collection.immutable.Map

2014-07-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081732#comment-14081732 ] Apache Spark commented on SPARK-2779: - User 'yhuai' has created a pull request for thi

[jira] [Created] (SPARK-2779) asInstanceOf[Map[...]] should use scala.collection.Map instead of scala.collection.immutable.Map

2014-07-31 Thread Yin Huai (JIRA)
Yin Huai created SPARK-2779: --- Summary: asInstanceOf[Map[...]] should use scala.collection.Map instead of scala.collection.immutable.Map Key: SPARK-2779 URL: https://issues.apache.org/jira/browse/SPARK-2779

[jira] [Commented] (SPARK-1812) Support cross-building with Scala 2.11

2014-07-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081671#comment-14081671 ] Apache Spark commented on SPARK-1812: - User 'avati' has created a pull request for thi

[jira] [Commented] (SPARK-695) Exponential recursion in getPreferredLocations

2014-07-31 Thread Aaron Staple (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081669#comment-14081669 ] Aaron Staple commented on SPARK-695: Progress has been made on a PR here: https://githu

[jira] [Commented] (SPARK-1812) Support cross-building with Scala 2.11

2014-07-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081657#comment-14081657 ] Apache Spark commented on SPARK-1812: - User 'avati' has created a pull request for thi

[jira] [Commented] (SPARK-1812) Support cross-building with Scala 2.11

2014-07-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081650#comment-14081650 ] Apache Spark commented on SPARK-1812: - User 'avati' has created a pull request for thi

[jira] [Commented] (SPARK-1812) Support cross-building with Scala 2.11

2014-07-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081642#comment-14081642 ] Apache Spark commented on SPARK-1812: - User 'avati' has created a pull request for thi

[jira] [Resolved] (SPARK-2771) GenerateMIMAIgnore fails scalastyle check due to long line

2014-07-31 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved SPARK-2771. --- Resolution: Fixed > GenerateMIMAIgnore fails scalastyle check due to long line >

[jira] [Commented] (SPARK-2282) PySpark crashes if too many tasks complete quickly

2014-07-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081616#comment-14081616 ] Josh Rosen commented on SPARK-2282: --- Merged the improved fix from https://github.com/apa

[jira] [Updated] (SPARK-2282) PySpark crashes if too many tasks complete quickly

2014-07-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-2282: -- Fix Version/s: 1.1.0 > PySpark crashes if too many tasks complete quickly > ---

[jira] [Comment Edited] (SPARK-2282) PySpark crashes if too many tasks complete quickly

2014-07-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081616#comment-14081616 ] Josh Rosen edited comment on SPARK-2282 at 7/31/14 10:37 PM: -

[jira] [Commented] (SPARK-2017) web ui stage page becomes unresponsive when the number of tasks is large

2014-07-31 Thread Carlos Fuertes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081605#comment-14081605 ] Carlos Fuertes commented on SPARK-2017: --- I did not realize that the tasks all have t

[jira] [Commented] (SPARK-2017) web ui stage page becomes unresponsive when the number of tasks is large

2014-07-31 Thread Carlos Fuertes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081598#comment-14081598 ] Carlos Fuertes commented on SPARK-2017: --- I have done some tests with the solution wh

[jira] [Updated] (SPARK-2096) Correctly parse dot notations for accessing an array of structs

2014-07-31 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2096: Target Version/s: 1.2.0 (was: 1.1.0) > Correctly parse dot notations for accessing an arra

[jira] [Resolved] (SPARK-2740) In JavaPairRdd, allow user to specify ascending and numPartitions for sortByKey

2014-07-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2740. --- Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Rui Li > In JavaPairRdd, allow user

[jira] [Updated] (SPARK-2063) Creating a SchemaRDD via sql() does not correctly resolve nested types

2014-07-31 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2063: Target Version/s: 1.2.0 (was: 1.1.0) > Creating a SchemaRDD via sql() does not correctly r

[jira] [Commented] (SPARK-2447) Add common solution for sending upsert actions to HBase (put, deletes, and increment)

2014-07-31 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081524#comment-14081524 ] Tathagata Das commented on SPARK-2447: -- Exactly!! That's why I feel that both have it

[jira] [Commented] (SPARK-2447) Add common solution for sending upsert actions to HBase (put, deletes, and increment)

2014-07-31 Thread Ted Malaska (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081514#comment-14081514 ] Ted Malaska commented on SPARK-2447: Tell me if I'm wrong but the core offering of 112

[jira] [Created] (SPARK-2778) Add unit tests for Yarn integration

2014-07-31 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-2778: - Summary: Add unit tests for Yarn integration Key: SPARK-2778 URL: https://issues.apache.org/jira/browse/SPARK-2778 Project: Spark Issue Type: Test

[jira] [Commented] (SPARK-1131) Better document the --args option for yarn-standalone mode

2014-07-31 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081497#comment-14081497 ] Marcelo Vanzin commented on SPARK-1131: --- This is probably obsolete now with spark-su

[jira] [Commented] (SPARK-1576) Passing of JAVA_OPTS to YARN on command line

2014-07-31 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081490#comment-14081490 ] Marcelo Vanzin commented on SPARK-1576: --- With Sandy's recent patch (https://github.c

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

2014-07-31 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081465#comment-14081465 ] Marcelo Vanzin commented on SPARK-1537: --- I'm working on this but this all sort of de

[jira] [Updated] (SPARK-2272) Feature scaling which standardizes the range of independent variables or features of data.

2014-07-31 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2272: - Assignee: DB Tsai > Feature scaling which standardizes the range of independent variables or > f

[jira] [Closed] (SPARK-2776) Add normalizeByCol method to mllib.util.MLUtils

2014-07-31 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-2776. Resolution: Duplicate > Add normalizeByCol method to mllib.util.MLUtils > -

[jira] [Commented] (SPARK-2777) ALS factors should be persist in memory and disk

2014-07-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081401#comment-14081401 ] Apache Spark commented on SPARK-2777: - User 'mengxr' has created a pull request for th

[jira] [Updated] (SPARK-2756) Decision Tree bugs

2014-07-31 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-2756: - Description: 3 bugs: Bug 1: Indexing is inconsistent for aggregate calculations for unor

[jira] [Created] (SPARK-2777) ALS factors should be persist in memory and disk

2014-07-31 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-2777: Summary: ALS factors should be persist in memory and disk Key: SPARK-2777 URL: https://issues.apache.org/jira/browse/SPARK-2777 Project: Spark Issue Type: Im

[jira] [Commented] (SPARK-2468) zero-copy shuffle network communication

2014-07-31 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081357#comment-14081357 ] Reynold Xin commented on SPARK-2468: It's something I'd like to prototype for 1.2. Do

[jira] [Resolved] (SPARK-2511) Add TF-IDF featurizer

2014-07-31 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-2511. -- Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1671 [https://gith

[jira] [Resolved] (SPARK-2646) log4j initialization not quite compatible with log4j 2.x

2014-07-31 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2646. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1547 [https://

[jira] [Commented] (SPARK-2678) `Spark-submit` overrides user application options

2014-07-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081302#comment-14081302 ] Apache Spark commented on SPARK-2678: - User 'liancheng' has created a pull request for

[jira] [Resolved] (SPARK-2749) Spark SQL Java tests aren't compiling in Jenkins' Maven builds; missing junit:junit dep

2014-07-31 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2749. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1690 [https://

[jira] [Commented] (SPARK-2774) Set preferred locations for reduce tasks

2014-07-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081300#comment-14081300 ] Josh Rosen commented on SPARK-2774: --- I found two research papers discussing locality-awa

[jira] [Updated] (SPARK-2749) Spark SQL Java tests aren't compiling in Jenkins' Maven builds; missing junit:junit dep

2014-07-31 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2749: --- Assignee: Sean Owen > Spark SQL Java tests aren't compiling in Jenkins' Maven builds; missing

[jira] [Resolved] (SPARK-2772) Spark Project SQL fails compilation

2014-07-31 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved SPARK-2772. --- Resolution: Not a Problem 'mvn install' solves the issue. > Spark Project SQL fails compilation > --

[jira] [Created] (SPARK-2776) Add normalizeByCol method to mllib.util.MLUtils

2014-07-31 Thread Andres Perez (JIRA)
Andres Perez created SPARK-2776: --- Summary: Add normalizeByCol method to mllib.util.MLUtils Key: SPARK-2776 URL: https://issues.apache.org/jira/browse/SPARK-2776 Project: Spark Issue Type: New F

[jira] [Resolved] (SPARK-2664) Deal with `--conf` options in spark-submit that relate to flags

2014-07-31 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2664. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1665 [https://

[jira] [Created] (SPARK-2775) HiveContext does not support dots in column names.

2014-07-31 Thread Yin Huai (JIRA)
Yin Huai created SPARK-2775: --- Summary: HiveContext does not support dots in column names. Key: SPARK-2775 URL: https://issues.apache.org/jira/browse/SPARK-2775 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2774) Set preferred locations for reduce tasks

2014-07-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081256#comment-14081256 ] Apache Spark commented on SPARK-2774: - User 'shivaram' has created a pull request for

[jira] [Resolved] (SPARK-2028) Let users of HadoopRDD access the partition InputSplits

2014-07-31 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-2028. -- Resolution: Fixed Fix Version/s: 1.1.0 > Let users of HadoopRDD access the partition Inp

[jira] [Created] (SPARK-2774) Set preferred locations for reduce tasks

2014-07-31 Thread Shivaram Venkataraman (JIRA)
Shivaram Venkataraman created SPARK-2774: Summary: Set preferred locations for reduce tasks Key: SPARK-2774 URL: https://issues.apache.org/jira/browse/SPARK-2774 Project: Spark Issue

[jira] [Resolved] (SPARK-2397) Get rid of LocalHiveContext

2014-07-31 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2397. - Resolution: Fixed Fix Version/s: 1.1.0 > Get rid of LocalHiveContext > ---

[jira] [Resolved] (SPARK-2743) Parquet has issues with capital letters and case insensitivity

2014-07-31 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2743. - Resolution: Fixed Fix Version/s: 1.1.0 > Parquet has issues with capital letters a

[jira] [Commented] (SPARK-2744) The configuration "spark.history.retainedApplications" is invalid

2014-07-31 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081191#comment-14081191 ] Marcelo Vanzin commented on SPARK-2744: --- Sort of. The docs say: {quote} The number

[jira] [Commented] (SPARK-2773) Shuffle:use growth rate to predict if need to spill

2014-07-31 Thread uncleGen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081161#comment-14081161 ] uncleGen commented on SPARK-2773: - here is my improvement: https://github.com/apache/spark

[jira] [Commented] (SPARK-2773) Shuffle:use growth rate to predict if need to spill

2014-07-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081163#comment-14081163 ] Apache Spark commented on SPARK-2773: - User 'uncleGen' has created a pull request for

[jira] [Commented] (SPARK-2762) SparkILoop leaks memory in multi-repl configurations

2014-07-31 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081154#comment-14081154 ] Matei Zaharia commented on SPARK-2762: -- PR: https://github.com/apache/spark/pull/1674

[jira] [Created] (SPARK-2773) Shuffle:use growth rate to predict if need to spill

2014-07-31 Thread uncleGen (JIRA)
uncleGen created SPARK-2773: --- Summary: Shuffle:use growth rate to predict if need to spill Key: SPARK-2773 URL: https://issues.apache.org/jira/browse/SPARK-2773 Project: Spark Issue Type: Improveme

[jira] [Resolved] (SPARK-2762) SparkILoop leaks memory in multi-repl configurations

2014-07-31 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-2762. -- Resolution: Fixed Fix Version/s: 1.1.0 > SparkILoop leaks memory in multi-repl configura

[jira] [Updated] (SPARK-2762) SparkILoop leaks memory in multi-repl configurations

2014-07-31 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-2762: - Assignee: Timothy Hunter > SparkILoop leaks memory in multi-repl configurations > ---

[jira] [Created] (SPARK-2772) Spark Project SQL fails compilation

2014-07-31 Thread Ted Yu (JIRA)
Ted Yu created SPARK-2772: - Summary: Spark Project SQL fails compilation Key: SPARK-2772 URL: https://issues.apache.org/jira/browse/SPARK-2772 Project: Spark Issue Type: Bug Reporter: Ted

[jira] [Updated] (SPARK-2771) GenerateMIMAIgnore fails scalastyle check due to long line

2014-07-31 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-2771: -- Attachment: spark-2771-v1.txt Patch v1 shortens line 118 to 100 chars wide. > GenerateMIMAIgnore fails scalast

[jira] [Commented] (SPARK-2771) GenerateMIMAIgnore fails scalastyle check due to long line

2014-07-31 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081050#comment-14081050 ] Sean Owen commented on SPARK-2771: -- Already on it :) https://github.com/apache/spark/pull

[jira] [Created] (SPARK-2771) GenerateMIMAIgnore fails scalastyle check due to long line

2014-07-31 Thread Ted Yu (JIRA)
Ted Yu created SPARK-2771: - Summary: GenerateMIMAIgnore fails scalastyle check due to long line Key: SPARK-2771 URL: https://issues.apache.org/jira/browse/SPARK-2771 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-2770) Rename spark-ganglia-lgpl to ganglia-lgpl

2014-07-31 Thread Chris Fregly (JIRA)
Chris Fregly created SPARK-2770: --- Summary: Rename spark-ganglia-lgpl to ganglia-lgpl Key: SPARK-2770 URL: https://issues.apache.org/jira/browse/SPARK-2770 Project: Spark Issue Type: Improvement

  1   2   >