[jira] [Commented] (SPARK-6054) SQL UDF returning object of case class; regression from 1.2.0

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377410#comment-14377410 ] Apache Spark commented on SPARK-6054: - User 'marmbrus' has created a pull request for

[jira] [Comment Edited] (SPARK-6479) Create off-heap block storage API (internal)

2015-03-24 Thread Henry Saputra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377386#comment-14377386 ] Henry Saputra edited comment on SPARK-6479 at 3/24/15 7:02 AM: -

[jira] [Assigned] (SPARK-6458) Bad error message for invalid data sources

2015-03-24 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-6458: --- Assignee: Michael Armbrust > Bad error message for invalid data sources > ---

[jira] [Commented] (SPARK-3306) Addition of external resource dependency in executors

2015-03-24 Thread Yan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377414#comment-14377414 ] Yan commented on SPARK-3306: The "external resource" primarily will serve the purpose of reuse

[jira] [Commented] (SPARK-3306) Addition of external resource dependency in executors

2015-03-24 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377416#comment-14377416 ] Reynold Xin commented on SPARK-3306: Sorry I still don't get it. Can't you just use a

[jira] [Updated] (SPARK-6487) Add sequential pattern mining algorithm to Spark MLlib

2015-03-24 Thread Zhang JiaJin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhang JiaJin updated SPARK-6487: Description: [~mengxr] [~zhangyouhua] Sequential pattern mining is an important branch in the patter

[jira] [Commented] (SPARK-6458) Bad error message for invalid data sources

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377419#comment-14377419 ] Apache Spark commented on SPARK-6458: - User 'marmbrus' has created a pull request for

[jira] [Updated] (SPARK-6492) SparkContext.stop() can deadlock when DAGSchedulerEventProcessLoop dies

2015-03-24 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-6492: -- Summary: SparkContext.stop() can deadlock when DAGSchedulerEventProcessLoop dies (was: Spark can freeze

[jira] [Created] (SPARK-6492) Spark can freeze / deadlock when DAGSchedulerEventProcessLoop dies

2015-03-24 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-6492: - Summary: Spark can freeze / deadlock when DAGSchedulerEventProcessLoop dies Key: SPARK-6492 URL: https://issues.apache.org/jira/browse/SPARK-6492 Project: Spark I

[jira] [Commented] (SPARK-5763) Sort-based Groupby and Join to resolve skewed data

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377430#comment-14377430 ] Apache Spark commented on SPARK-5763: - User 'lianhuiwang' has created a pull request f

[jira] [Assigned] (SPARK-6376) Subqueries are thrown away too early in dataframes

2015-03-24 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-6376: --- Assignee: Michael Armbrust > Subqueries are thrown away too early in dataframes > ---

[jira] [Commented] (SPARK-6376) Subqueries are thrown away too early in dataframes

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377458#comment-14377458 ] Apache Spark commented on SPARK-6376: - User 'marmbrus' has created a pull request for

[jira] [Commented] (SPARK-6437) SQL ExternalSort should use CompletionIterator to clean up temp files

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377462#comment-14377462 ] Apache Spark commented on SPARK-6437: - User 'marmbrus' has created a pull request for

[jira] [Commented] (SPARK-6428) Add to style checker "public method must have explicit type defined"

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377464#comment-14377464 ] Apache Spark commented on SPARK-6428: - User 'rxin' has created a pull request for this

[jira] [Resolved] (SPARK-6452) CheckAnalysis should throw when the Aggregate node contains missing input attribute(s)

2015-03-24 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6452. - Resolution: Fixed Fix Version/s: 1.4.0 1.3.1 Issue resolved by p

[jira] [Updated] (SPARK-6409) It is not necessary that avoid old inteface of hive, because this will make some UDAF can not work.

2015-03-24 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6409: Assignee: DoingDone9 > It is not necessary that avoid old inteface of hive, because this wil

[jira] [Commented] (SPARK-6465) GenericRowWithSchema: KryoException: Class cannot be created (missing no-arg constructor):

2015-03-24 Thread Earthson Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377490#comment-14377490 ] Earthson Lu commented on SPARK-6465: I'm confused. https://github.com/apache/spark/bl

[jira] [Commented] (SPARK-6483) Spark SQL udf(ScalaUdf) is very slow

2015-03-24 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377494#comment-14377494 ] Cheng Hao commented on SPARK-6483: -- Can you re-run those 2 queries without GROUP BY? >

[jira] [Created] (SPARK-6493) Support numeric(a,b) in the parser

2015-03-24 Thread DoingDone9 (JIRA)
DoingDone9 created SPARK-6493: - Summary: Support numeric(a,b) in the parser Key: SPARK-6493 URL: https://issues.apache.org/jira/browse/SPARK-6493 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-6493) Support numeric(a,b) in the parser

2015-03-24 Thread DoingDone9 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DoingDone9 updated SPARK-6493: -- Description: support sql like that : select cast(20.12 as numeric(4,2)) from src limit1; > Support num

[jira] [Commented] (SPARK-6459) Warn when Column API is constructing trivially true equality

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377502#comment-14377502 ] Apache Spark commented on SPARK-6459: - User 'marmbrus' has created a pull request for

[jira] [Assigned] (SPARK-6459) Warn when Column API is constructing trivially true equality

2015-03-24 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-6459: --- Assignee: Michael Armbrust > Warn when Column API is constructing trivially true equa

[jira] [Commented] (SPARK-6469) The YARN driver in yarn-client mode will not use the local directories configured for YARN

2015-03-24 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377505#comment-14377505 ] Christophe PRÉAUD commented on SPARK-6469: -- Sure, I will take care of this. > Th

[jira] [Created] (SPARK-6494) rdd polymorphic method zipPartitions refactor

2015-03-24 Thread sjk (JIRA)
sjk created SPARK-6494: -- Summary: rdd polymorphic method zipPartitions refactor Key: SPARK-6494 URL: https://issues.apache.org/jira/browse/SPARK-6494 Project: Spark Issue Type: Improvement R

[jira] [Commented] (SPARK-6494) rdd polymorphic method zipPartitions refactor

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377520#comment-14377520 ] Apache Spark commented on SPARK-6494: - User 'shijinkui' has created a pull request for

[jira] [Updated] (SPARK-6495) DataFrame#insertInto method should support insert rows with sub-columns

2015-03-24 Thread Chaozhong Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaozhong Yang updated SPARK-6495: -- Description: The original table's schema is like this: ``` root |-- a: string (nullable = true)

[jira] [Created] (SPARK-6495) DataFrame#insertInto method should support insert rows with sub-columns

2015-03-24 Thread Chaozhong Yang (JIRA)
Chaozhong Yang created SPARK-6495: - Summary: DataFrame#insertInto method should support insert rows with sub-columns Key: SPARK-6495 URL: https://issues.apache.org/jira/browse/SPARK-6495 Project: Spar

[jira] [Updated] (SPARK-6495) DataFrame#insertInto method should support insert rows with sub-columns

2015-03-24 Thread Chaozhong Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaozhong Yang updated SPARK-6495: -- Description: The original table's schema is like this: |-- a: string (nullable = true) |-- b:

[jira] [Commented] (SPARK-6456) Spark Sql throwing exception on large partitioned data

2015-03-24 Thread pankaj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377569#comment-14377569 ] pankaj commented on SPARK-6456: --- It was the issue of large number of partition. actually the

[jira] [Commented] (SPARK-6383) Few examples on Dataframe operation give compiler errors

2015-03-24 Thread Tijo Thomas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377571#comment-14377571 ] Tijo Thomas commented on SPARK-6383: The Assignee: for this issues appeared as "Unassi

[jira] [Closed] (SPARK-6456) Spark Sql throwing exception on large partitioned data

2015-03-24 Thread pankaj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pankaj closed SPARK-6456. - Resolution: Fixed It was the issue of large number of partition. actually the number was too high. i removed old

[jira] [Resolved] (SPARK-6494) rdd polymorphic method zipPartitions refactor

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6494. -- Resolution: Won't Fix Please see PR comments. The changes that are intended as in the PR are problemati

[jira] [Resolved] (SPARK-6297) EventLog permissions are always set to 770 which causes problems

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6297. -- Resolution: Not a Problem Provisionally closing this as not a problem unless there is more indication t

[jira] [Commented] (SPARK-6469) The YARN driver in yarn-client mode will not use the local directories configured for YARN

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377599#comment-14377599 ] Apache Spark commented on SPARK-6469: - User 'preaudc' has created a pull request for t

[jira] [Updated] (SPARK-6482) Remove synchronization of Hive Native commands

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6482: - Component/s: SQL [~dyross] let's assign Components > Remove synchronization of Hive Native commands > ---

[jira] [Updated] (SPARK-6491) Spark will put the current working dir to the CLASSPATH

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6491: - Component/s: Spark Submit [~marsishandsome] please assign a Component to JIRAs > Spark will put the curre

[jira] [Commented] (SPARK-6480) histogram() bucket function is wrong in some simple edge cases

2015-03-24 Thread Frank Rosner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377611#comment-14377611 ] Frank Rosner commented on SPARK-6480: - Thanks for picking it up [~srowen]! > histogra

[jira] [Updated] (SPARK-4814) Enable assertions in SBT, Maven tests / AssertionError from Hive's LazyBinaryInteger

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4814: - Labels: (was: backport-needed) > Enable assertions in SBT, Maven tests / AssertionError from Hive's > L

[jira] [Resolved] (SPARK-4814) Enable assertions in SBT, Maven tests / AssertionError from Hive's LazyBinaryInteger

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4814. -- Resolution: Fixed Target Version/s: (was: 1.0.3) Provisionally deciding that it's not worth

[jira] [Updated] (SPARK-6383) Few examples on Dataframe operation give compiler errors

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6383: - Assignee: Tijo Thomas > Few examples on Dataframe operation give compiler errors > --

[jira] [Resolved] (SPARK-6477) Run MIMA tests before the Spark test suite

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6477. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5145 [https://github.com/ap

[jira] [Updated] (SPARK-6477) Run MIMA tests before the Spark test suite

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6477: - Assignee: Brennon York > Run MIMA tests before the Spark test suite >

[jira] [Resolved] (SPARK-6449) Driver OOM results in reported application result SUCCESS

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6449. -- Resolution: Duplicate Fix Version/s: (was: 1.3.0) > Driver OOM results in reported applicatio

[jira] [Reopened] (SPARK-6449) Driver OOM results in reported application result SUCCESS

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-6449: -- (Just doing this to link to SPARK-6018 as a Duplicate which will link it in the other issue too) > Driver

[jira] [Resolved] (SPARK-5368) Spark should support NAT (via akka improvements)

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5368. -- Resolution: Duplicate Fix Version/s: (was: 1.2.2) Target Version/s: (was: 1.2.2)

[jira] [Updated] (SPARK-6493) Support numeric(a,b) in the sqlContext

2015-03-24 Thread DoingDone9 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DoingDone9 updated SPARK-6493: -- Description: support sql like that : select cast(20.12 as numeric(4,2)) from src limit 1; was: suppo

[jira] [Updated] (SPARK-6493) Support numeric(a,b) in the sqlContext

2015-03-24 Thread DoingDone9 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DoingDone9 updated SPARK-6493: -- Summary: Support numeric(a,b) in the sqlContext (was: Support numeric(a,b) in the parser) > Support nu

[jira] [Commented] (SPARK-6484) Ganglia metrics xml reporter doesn't escape correctly

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377692#comment-14377692 ] Sean Owen commented on SPARK-6484: -- [~joshrosen] see https://github.com/apache/spark/pull

[jira] [Created] (SPARK-6496) Multinomial Logistic Regression failed when initialWeights is not null

2015-03-24 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-6496: -- Summary: Multinomial Logistic Regression failed when initialWeights is not null Key: SPARK-6496 URL: https://issues.apache.org/jira/browse/SPARK-6496 Project: Spark

[jira] [Updated] (SPARK-6496) Multinomial Logistic Regression failed when initialWeights is not null

2015-03-24 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-6496: --- Description: This bug is easy to reproduce, when use Multinomial Logistic Regression to train multicl

[jira] [Commented] (SPARK-6496) Multinomial Logistic Regression failed when initialWeights is not null

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377713#comment-14377713 ] Sean Owen commented on SPARK-6496: -- The problem is numFeatures = -1, not initialWeights.

[jira] [Commented] (SPARK-6493) Support numeric(a,b) in the sqlContext

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377736#comment-14377736 ] Apache Spark commented on SPARK-6493: - User 'DoingDone9' has created a pull request fo

[jira] [Created] (SPARK-6497) Class is not registered: scala.reflect.ManifestFactory$$anon$9

2015-03-24 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-6497: - Summary: Class is not registered: scala.reflect.ManifestFactory$$anon$9 Key: SPARK-6497 URL: https://issues.apache.org/jira/browse/SPARK-6497 Project: Spark

[jira] [Commented] (SPARK-6496) Multinomial Logistic Regression failed when initialWeights is not null

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377740#comment-14377740 ] Apache Spark commented on SPARK-6496: - User 'yanboliang' has created a pull request fo

[jira] [Commented] (SPARK-6483) Spark SQL udf(ScalaUdf) is very slow

2015-03-24 Thread zzc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377746#comment-14377746 ] zzc commented on SPARK-6483: I test it later. > Spark SQL udf(ScalaUdf) is very slow > --

[jira] [Commented] (SPARK-6497) Class is not registered: scala.reflect.ManifestFactory$$anon$9

2015-03-24 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377749#comment-14377749 ] Daniel Darabos commented on SPARK-6497: --- By the way, have you considered running the

[jira] [Updated] (SPARK-6499) pyspark: printSchema command on a dataframe hangs

2015-03-24 Thread cynepia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cynepia updated SPARK-6499: --- Summary: pyspark: printSchema command on a dataframe hangs (was: pyspark dataframe filter does not work as ex

[jira] [Created] (SPARK-6499) pyspark dataframe filter does not work as expected

2015-03-24 Thread cynepia (JIRA)
cynepia created SPARK-6499: -- Summary: pyspark dataframe filter does not work as expected Key: SPARK-6499 URL: https://issues.apache.org/jira/browse/SPARK-6499 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-6499) pyspark: printSchema command on a dataframe hangs

2015-03-24 Thread cynepia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cynepia updated SPARK-6499: --- Attachment: airports.json pyspark.txt > pyspark: printSchema command on a dataframe hangs > --

[jira] [Commented] (SPARK-6496) Multinomial Logistic Regression failed when initialWeights is not null

2015-03-24 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377759#comment-14377759 ] Yanbo Liang commented on SPARK-6496: [~srowen] I have address this issue at github. >

[jira] [Commented] (SPARK-4922) Support dynamic allocation for coarse-grained Mesos

2015-03-24 Thread Hans van den Bogert (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377760#comment-14377760 ] Hans van den Bogert commented on SPARK-4922: What were/are the reasons for not

[jira] [Comment Edited] (SPARK-6496) Multinomial Logistic Regression failed when initialWeights is not null

2015-03-24 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377759#comment-14377759 ] Yanbo Liang edited comment on SPARK-6496 at 3/24/15 12:08 PM: --

[jira] [Commented] (SPARK-5763) Sort-based Groupby and Join to resolve skewed data

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377763#comment-14377763 ] Apache Spark commented on SPARK-5763: - User 'lianhuiwang' has created a pull request f

[jira] [Created] (SPARK-6500) Scala code example in README.md does not compile

2015-03-24 Thread Nick (JIRA)
Nick created SPARK-6500: --- Summary: Scala code example in README.md does not compile Key: SPARK-6500 URL: https://issues.apache.org/jira/browse/SPARK-6500 Project: Spark Issue Type: Bug Repo

[jira] [Resolved] (SPARK-6500) Scala code example in README.md does not compile

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6500. -- Resolution: Invalid You're reading the PySpark example :) https://github.com/apache/spark/blob/master/RE

[jira] [Updated] (SPARK-6500) Scala code example in README.md does not compile

2015-03-24 Thread Nick (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick updated SPARK-6500: Description: I just downloaded and installed Spark 1.3. Inside README.md there is this example {code} And run th

[jira] [Closed] (SPARK-6500) Scala code example in README.md does not compile

2015-03-24 Thread Nick (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick closed SPARK-6500. --- > Scala code example in README.md does not compile > > >

[jira] [Commented] (SPARK-5173) support python application running on yarn cluster mode

2015-03-24 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377829#comment-14377829 ] Thomas Graves commented on SPARK-5173: -- [~andrewor14] This pull request has went in,

[jira] [Commented] (SPARK-2429) Hierarchical Implementation of KMeans

2015-03-24 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377840#comment-14377840 ] Yu Ishikawa commented on SPARK-2429: [~freeman-lab], [~mengxr], [~josephkb], [~rnowlin

[jira] [Updated] (SPARK-6387) HTTP mode of HiveThriftServer2 doesn't work when built with Hive 0.12.0

2015-03-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6387: -- Issue Type: Sub-task (was: Bug) Parent: SPARK-6109 > HTTP mode of HiveThriftServer2 doesn't wor

[jira] [Created] (SPARK-6501) Blacklist Hive 0.13.1 specific tests when compiled against Hive 0.12.0

2015-03-24 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-6501: - Summary: Blacklist Hive 0.13.1 specific tests when compiled against Hive 0.12.0 Key: SPARK-6501 URL: https://issues.apache.org/jira/browse/SPARK-6501 Project: Spark

[jira] [Created] (SPARK-6502) HiveThriftServer2 fails to inspect underlying Hive version when compiled against Hive 0.12.0

2015-03-24 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-6502: - Summary: HiveThriftServer2 fails to inspect underlying Hive version when compiled against Hive 0.12.0 Key: SPARK-6502 URL: https://issues.apache.org/jira/browse/SPARK-6502

[jira] [Commented] (SPARK-5479) PySpark on yarn mode need to support non-local python files

2015-03-24 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377881#comment-14377881 ] Thomas Graves commented on SPARK-5479: -- Was this fixed by https://github.com/apache/s

[jira] [Resolved] (SPARK-6473) Launcher lib shouldn't try to figure out Scala version when not in dev mode

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6473. -- Resolution: Fixed Issue resolved by pull request 5143 [https://github.com/apache/spark/pull/5143] > Lau

[jira] [Commented] (SPARK-5162) Python yarn-cluster mode

2015-03-24 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377887#comment-14377887 ] Thomas Graves commented on SPARK-5162: -- So is there anything left on this jira to do?

[jira] [Updated] (SPARK-6473) Launcher lib shouldn't try to figure out Scala version when not in dev mode

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6473: - Component/s: (was: Spark Core) Spark Submit Priority: Minor (was: Major) > La

[jira] [Created] (SPARK-6503) Create Jenkins builder for testing Spark SQL with Hive 0.12.0

2015-03-24 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-6503: - Summary: Create Jenkins builder for testing Spark SQL with Hive 0.12.0 Key: SPARK-6503 URL: https://issues.apache.org/jira/browse/SPARK-6503 Project: Spark Issue

[jira] [Commented] (SPARK-6109) Unit tests fail when compiled against Hive 0.12.0

2015-03-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377897#comment-14377897 ] Cheng Lian commented on SPARK-6109: --- I had created [PR #4851|https://github.com/apache/s

[jira] [Created] (SPARK-6504) Cannot read Parquet files generated from different versions at once

2015-03-24 Thread Marius Soutier (JIRA)
Marius Soutier created SPARK-6504: - Summary: Cannot read Parquet files generated from different versions at once Key: SPARK-6504 URL: https://issues.apache.org/jira/browse/SPARK-6504 Project: Spark

[jira] [Created] (SPARK-6505) Remove the reflection call in HiveFunctionWrapper

2015-03-24 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-6505: - Summary: Remove the reflection call in HiveFunctionWrapper Key: SPARK-6505 URL: https://issues.apache.org/jira/browse/SPARK-6505 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6505) Remove the reflection call in HiveFunctionWrapper

2015-03-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377915#comment-14377915 ] Cheng Lian commented on SPARK-6505: --- Here is [a WiP simpler fix for SPARK-4785|https://

[jira] [Created] (SPARK-6506) python support yarn cluster mode requires SPARK_HOME to be set

2015-03-24 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-6506: Summary: python support yarn cluster mode requires SPARK_HOME to be set Key: SPARK-6506 URL: https://issues.apache.org/jira/browse/SPARK-6506 Project: Spark

[jira] [Created] (SPARK-6507) Create separate Hive Driver instance for each SQL query in HiveThriftServer2

2015-03-24 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-6507: - Summary: Create separate Hive Driver instance for each SQL query in HiveThriftServer2 Key: SPARK-6507 URL: https://issues.apache.org/jira/browse/SPARK-6507 Project: Spark

[jira] [Created] (SPARK-6508) error handling issue running python in yarn cluster mode

2015-03-24 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-6508: Summary: error handling issue running python in yarn cluster mode Key: SPARK-6508 URL: https://issues.apache.org/jira/browse/SPARK-6508 Project: Spark Issue

[jira] [Comment Edited] (SPARK-2394) Make it easier to read LZO-compressed files from EC2 clusters

2015-03-24 Thread Theodore Vasiloudis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375766#comment-14375766 ] Theodore Vasiloudis edited comment on SPARK-2394 at 3/24/15 3:09 PM: ---

[jira] [Comment Edited] (SPARK-2394) Make it easier to read LZO-compressed files from EC2 clusters

2015-03-24 Thread Theodore Vasiloudis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375766#comment-14375766 ] Theodore Vasiloudis edited comment on SPARK-2394 at 3/24/15 3:08 PM: ---

[jira] [Updated] (SPARK-6469) Improving documentation on YARN local directories usage

2015-03-24 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christophe Préaud updated SPARK-6469: - Summary: Improving documentation on YARN local directories usage (was: The YARN driver in

[jira] [Commented] (SPARK-6479) Create off-heap block storage API (internal)

2015-03-24 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378011#comment-14378011 ] Sandy Ryza commented on SPARK-6479: --- I believe he means wrapping Spark's call-outs to Ta

[jira] [Comment Edited] (SPARK-2426) Quadratic Minimization for MLlib ALS

2015-03-24 Thread Debasish Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377357#comment-14377357 ] Debasish Das edited comment on SPARK-2426 at 3/24/15 3:23 PM: --

[jira] [Comment Edited] (SPARK-2426) Quadratic Minimization for MLlib ALS

2015-03-24 Thread Debasish Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377357#comment-14377357 ] Debasish Das edited comment on SPARK-2426 at 3/24/15 3:23 PM: --

[jira] [Commented] (SPARK-5173) support python application running on yarn cluster mode

2015-03-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378035#comment-14378035 ] Andrew Or commented on SPARK-5173: -- It appears not. I just closed it. > support python a

[jira] [Closed] (SPARK-5173) support python application running on yarn cluster mode

2015-03-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-5173. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Lianhui Wang > support python application r

[jira] [Commented] (SPARK-6495) DataFrame#insertInto method should support insert rows with sub-columns

2015-03-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378051#comment-14378051 ] Cheng Lian commented on SPARK-6495: --- Inserting a subset of columns in the original schem

[jira] [Commented] (SPARK-6323) Large rank matrix factorization with Nonlinear loss and constraints

2015-03-24 Thread Debasish Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378062#comment-14378062 ] Debasish Das commented on SPARK-6323: - I did some more reading and realized that even

[jira] [Commented] (SPARK-1473) Feature selection for high dimensional datasets

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378069#comment-14378069 ] Apache Spark commented on SPARK-1473: - User 'sramirez' has created a pull request for

[jira] [Commented] (SPARK-3306) Addition of external resource dependency in executors

2015-03-24 Thread Yan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378073#comment-14378073 ] Yan commented on SPARK-3306: If by "global singleton object", you meant it to be in the Execut

[jira] [Commented] (SPARK-1303) Added discretization capability to MLlib.

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378070#comment-14378070 ] Apache Spark commented on SPARK-1303: - User 'sramirez' has created a pull request for

[jira] [Commented] (SPARK-6373) Add SSL/TLS for the Netty based BlockTransferService

2015-03-24 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378084#comment-14378084 ] Aaron Davidson commented on SPARK-6373: --- >From my brief look at your work, it seemed

[jira] [Resolved] (SPARK-5559) Flaky test: o.a.s.streaming.flume.FlumeStreamSuite

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5559. -- Resolution: Fixed Fix Version/s: 1.4.0 1.3.1 1.2.2 > Flaky

[jira] [Updated] (SPARK-5559) Flaky test: o.a.s.streaming.flume.FlumeStreamSuite

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5559: - Component/s: (was: Project Infra) Tests Priority: Major (was

  1   2   3   >