[jira] [Commented] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294785#comment-15294785 ] Davies Liu commented on SPARK-15285: [~kiszk] Go ahead, don't know why I can't assign

[jira] [Updated] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15285: --- Assignee: (was: Wenchen Fan) > Generated SpecificSafeProjection.apply method grows beyond 64 KB >

[jira] [Resolved] (SPARK-15101) Audit: ml.clustering and ml.recommendation

2016-05-20 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang resolved SPARK-15101. Resolution: Fixed Fix Version/s: 2.0.0 > Audit: ml.clustering and ml.recommendation > --

[jira] [Commented] (SPARK-15344) Unable to set default log level for PySpark

2016-05-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-15344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294777#comment-15294777 ] Maciej BryƄski commented on SPARK-15344: [~felixcheung] Marcelo is right. Spark-s

[jira] [Resolved] (SPARK-15205) Codegen can compile the same source code more than twice

2016-05-20 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-15205. Resolution: Fixed Assignee: Kousuke Saruta Fix Version/s: 2.0.0

[jira] [Commented] (SPARK-15205) Codegen can compile the same source code more than twice

2016-05-20 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294776#comment-15294776 ] Kousuke Saruta commented on SPARK-15205: As a result, this issue was resolved for

[jira] [Updated] (SPARK-15165) Codegen can break because toCommentSafeString is not actually safe

2016-05-20 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-15165: --- Fix Version/s: 1.6.2 > Codegen can break because toCommentSafeString is not actually safe > -

[jira] [Updated] (SPARK-15461) modify python test script using default version 2.7

2016-05-20 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-15461: --- Priority: Major (was: Critical) > modify python test script using default version 2.7 >

[jira] [Updated] (SPARK-15461) modify python test script using default version 2.7

2016-05-20 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-15461: --- Description: To Spark 2.0, the python test script do not support python 2.6, so need to update the d

[jira] [Commented] (SPARK-15329) When start spark with yarn: spark.SparkContext: Error initializing SparkContext.

2016-05-20 Thread Vijay Parmar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294753#comment-15294753 ] Vijay Parmar commented on SPARK-15329: -- Presumably, it should have worked or have sh

[jira] [Commented] (SPARK-15101) Audit: ml.clustering and ml.recommendation

2016-05-20 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294737#comment-15294737 ] yuhao yang commented on SPARK-15101: Resolve the issue here as all sub tasks are fini

[jira] [Assigned] (SPARK-15461) modify python test script using default version 2.7

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15461: Assignee: (was: Apache Spark) > modify python test script using default version 2.7 >

[jira] [Commented] (SPARK-15461) modify python test script using default version 2.7

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294732#comment-15294732 ] Apache Spark commented on SPARK-15461: -- User 'WeichenXu123' has created a pull reque

[jira] [Assigned] (SPARK-15461) modify python test script using default version 2.7

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15461: Assignee: Apache Spark > modify python test script using default version 2.7 > ---

[jira] [Updated] (SPARK-15461) modify python test script using default version 2.7

2016-05-20 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-15461: --- Component/s: Tests PySpark > modify python test script using default version 2.7 > -

[jira] [Created] (SPARK-15460) Issue Exceptions from Thrift Server and Spark-SQL Cli when Users Inputting hive.metastore.warehouse.dir

2016-05-20 Thread Xiao Li (JIRA)
Xiao Li created SPARK-15460: --- Summary: Issue Exceptions from Thrift Server and Spark-SQL Cli when Users Inputting hive.metastore.warehouse.dir Key: SPARK-15460 URL: https://issues.apache.org/jira/browse/SPARK-15460

[jira] [Commented] (SPARK-15460) Issue Exceptions from Thrift Server and Spark-SQL Cli when Users Inputting hive.metastore.warehouse.dir

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294730#comment-15294730 ] Apache Spark commented on SPARK-15460: -- User 'gatorsmile' has created a pull request

[jira] [Created] (SPARK-15461) modify python test script using default version 2.7

2016-05-20 Thread Weichen Xu (JIRA)
Weichen Xu created SPARK-15461: -- Summary: modify python test script using default version 2.7 Key: SPARK-15461 URL: https://issues.apache.org/jira/browse/SPARK-15461 Project: Spark Issue Type: B

[jira] [Assigned] (SPARK-15460) Issue Exceptions from Thrift Server and Spark-SQL Cli when Users Inputting hive.metastore.warehouse.dir

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15460: Assignee: (was: Apache Spark) > Issue Exceptions from Thrift Server and Spark-SQL Cli

[jira] [Updated] (SPARK-15460) Issue Exceptions from Thrift Server and Spark-SQL Cli when Users Inputting hive.metastore.warehouse.dir

2016-05-20 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-15460: Issue Type: Improvement (was: Bug) > Issue Exceptions from Thrift Server and Spark-SQL Cli when Users Inpu

[jira] [Assigned] (SPARK-15460) Issue Exceptions from Thrift Server and Spark-SQL Cli when Users Inputting hive.metastore.warehouse.dir

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15460: Assignee: Apache Spark > Issue Exceptions from Thrift Server and Spark-SQL Cli when Users

[jira] [Closed] (SPARK-15320) Spark-SQL Cli Ignores Parameter hive.metastore.warehouse.dir

2016-05-20 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li closed SPARK-15320. --- Resolution: Not A Problem > Spark-SQL Cli Ignores Parameter hive.metastore.warehouse.dir > -

[jira] [Commented] (SPARK-15098) Audit: ml.classification

2016-05-20 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294719#comment-15294719 ] yuhao yang commented on SPARK-15098: I've made a pass and found no notable changes re

[jira] [Resolved] (SPARK-15437) Failed to create HiveContext in SparkR

2016-05-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15437. - Resolution: Fixed Assignee: Reynold Xin Fix Version/s: 2.0.0 > Failed to create H

[jira] [Resolved] (SPARK-15424) Revert SPARK-14807 Create a hivecontext-compatibility module

2016-05-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15424. - Resolution: Fixed > Revert SPARK-14807 Create a hivecontext-compatibility module > --

[jira] [Commented] (SPARK-15459) Make Range logical and physical explain consistent

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294686#comment-15294686 ] Apache Spark commented on SPARK-15459: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-15459) Make Range logical and physical explain consistent

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15459: Assignee: Reynold Xin (was: Apache Spark) > Make Range logical and physical explain consi

[jira] [Assigned] (SPARK-15459) Make Range logical and physical explain consistent

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15459: Assignee: Apache Spark (was: Reynold Xin) > Make Range logical and physical explain consi

[jira] [Created] (SPARK-15459) Make Range logical and physical explain consistent

2016-05-20 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-15459: --- Summary: Make Range logical and physical explain consistent Key: SPARK-15459 URL: https://issues.apache.org/jira/browse/SPARK-15459 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-15429) When `spark.streaming.concurrentJobs > 1`, PIDRateEstimator cannot estimate the receiving rate accurately.

2016-05-20 Thread Albert Cheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15292908#comment-15292908 ] Albert Cheng edited comment on SPARK-15429 at 5/21/16 2:58 AM:

[jira] [Commented] (SPARK-15329) When start spark with yarn: spark.SparkContext: Error initializing SparkContext.

2016-05-20 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294653#comment-15294653 ] Saisai Shao commented on SPARK-15329: - {code} 2016-05-15 00:06:08,368 WARN org.apach

[jira] [Updated] (SPARK-15423) why it is very slow to clean resources in Spark-2.0.0-preview

2016-05-20 Thread zszhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zszhong updated SPARK-15423: Summary: why it is very slow to clean resources in Spark-2.0.0-preview (was: why it is very slow to clean

[jira] [Commented] (SPARK-15423) why it is very slow to clean resources in Spark

2016-05-20 Thread zszhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294637#comment-15294637 ] zszhong commented on SPARK-15423: - I've also downloaded spark-1.6.1 to run the same code

[jira] [Commented] (SPARK-15453) Sort Merge Join to use bucketing metadata to optimize query plan

2016-05-20 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294627#comment-15294627 ] Tejas Patil commented on SPARK-15453: - [~rxin] Yes. I updated the jira title. If we a

[jira] [Comment Edited] (SPARK-15453) Sort Merge Join to use bucketing metadata to optimize query plan

2016-05-20 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294627#comment-15294627 ] Tejas Patil edited comment on SPARK-15453 at 5/21/16 1:42 AM: -

[jira] [Assigned] (SPARK-15458) Disable schema inference for streaming datasets on file streams

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15458: Assignee: Apache Spark (was: Tathagata Das) > Disable schema inference for streaming data

[jira] [Commented] (SPARK-15458) Disable schema inference for streaming datasets on file streams

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294623#comment-15294623 ] Apache Spark commented on SPARK-15458: -- User 'tdas' has created a pull request for t

[jira] [Assigned] (SPARK-15458) Disable schema inference for streaming datasets on file streams

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15458: Assignee: Tathagata Das (was: Apache Spark) > Disable schema inference for streaming data

[jira] [Created] (SPARK-15458) Disable schema inference for streaming datasets on file streams

2016-05-20 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-15458: - Summary: Disable schema inference for streaming datasets on file streams Key: SPARK-15458 URL: https://issues.apache.org/jira/browse/SPARK-15458 Project: Spark

[jira] [Commented] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-20 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294591#comment-15294591 ] Joseph K. Bradley commented on SPARK-15457: --- By the way, I plan to deprecate sp

[jira] [Commented] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-20 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294590#comment-15294590 ] Joseph K. Bradley commented on SPARK-15457: --- OK I have a WIP one for the SGD is

[jira] [Commented] (SPARK-7159) Support multiclass logistic regression in spark.ml

2016-05-20 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294588#comment-15294588 ] DB Tsai commented on SPARK-7159: Hello [~sethah], I think we will make it as separate Sof

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-05-20 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294580#comment-15294580 ] Liwei Lin commented on SPARK-15406: --- Hi [~c...@koeninger.org], any plan on this? Thanks

[jira] [Resolved] (SPARK-15456) PySpark Shell fails to create SparkContext if HiveConf not found

2016-05-20 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15456. --- Resolution: Fixed Assignee: Bryan Cutler Fix Version/s: 2.0.0 Target Vers

[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR

2016-05-20 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294522#comment-15294522 ] Miao Wang commented on SPARK-15439: --- Reproduced. Now I am analyzing the reason. > Fail

[jira] [Updated] (SPARK-15273) YarnSparkHadoopUtil#getOutOfMemoryErrorArgument should respect OnOutOfMemoryError parameter given by user

2016-05-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15273: -- Assignee: Ted Yu > YarnSparkHadoopUtil#getOutOfMemoryErrorArgument should respect > OnOutOfMemoryError

[jira] [Resolved] (SPARK-15273) YarnSparkHadoopUtil#getOutOfMemoryErrorArgument should respect OnOutOfMemoryError parameter given by user

2016-05-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15273. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13057 [https://github.co

[jira] [Commented] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294495#comment-15294495 ] Sean Owen commented on SPARK-15457: --- Yeah I have a PR brewing that will fix some of the

[jira] [Commented] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294494#comment-15294494 ] Kazuaki Ishizaki commented on SPARK-15285: -- I can take it today if they are busy

[jira] [Assigned] (SPARK-15456) PySpark Shell fails to create SparkContext if HiveConf not found

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15456: Assignee: Apache Spark > PySpark Shell fails to create SparkContext if HiveConf not found

[jira] [Assigned] (SPARK-15456) PySpark Shell fails to create SparkContext if HiveConf not found

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15456: Assignee: (was: Apache Spark) > PySpark Shell fails to create SparkContext if HiveConf

[jira] [Commented] (SPARK-15456) PySpark Shell fails to create SparkContext if HiveConf not found

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294462#comment-15294462 ] Apache Spark commented on SPARK-15456: -- User 'BryanCutler' has created a pull reques

[jira] [Updated] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-20 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15457: -- Description: Several classes and methods have been deprecated and are creating lots of

[jira] [Commented] (SPARK-15455) For IsolatedClientLoader, we need to provide a conf to disable sharing Hadoop classes

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294454#comment-15294454 ] Apache Spark commented on SPARK-15455: -- User 'yhuai' has created a pull request for

[jira] [Created] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-20 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-15457: - Summary: Eliminate MLlib 2.0 build warnings from deprecations Key: SPARK-15457 URL: https://issues.apache.org/jira/browse/SPARK-15457 Project: Spark

[jira] [Created] (SPARK-15456) PySpark Shell fails to create SparkContext if HiveConf not found

2016-05-20 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-15456: Summary: PySpark Shell fails to create SparkContext if HiveConf not found Key: SPARK-15456 URL: https://issues.apache.org/jira/browse/SPARK-15456 Project: Spark

[jira] [Commented] (SPARK-15456) PySpark Shell fails to create SparkContext if HiveConf not found

2016-05-20 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294446#comment-15294446 ] Bryan Cutler commented on SPARK-15456: -- I can submit a fix for this > PySpark Shell

[jira] [Commented] (SPARK-15327) Catalyst code generation fails with complex data structure

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294437#comment-15294437 ] Apache Spark commented on SPARK-15327: -- User 'davies' has created a pull request for

[jira] [Assigned] (SPARK-15327) Catalyst code generation fails with complex data structure

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15327: Assignee: Apache Spark (was: Davies Liu) > Catalyst code generation fails with complex da

[jira] [Assigned] (SPARK-15327) Catalyst code generation fails with complex data structure

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15327: Assignee: Davies Liu (was: Apache Spark) > Catalyst code generation fails with complex da

[jira] [Commented] (SPARK-15451) Spark PR builder should fail if code doesn't compile against JDK 7

2016-05-20 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294426#comment-15294426 ] Marcelo Vanzin commented on SPARK-15451: I think there's still some value even if

[jira] [Updated] (SPARK-15449) Wrong Data Format - Documentation Issue

2016-05-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15449: -- Target Version/s: (was: 1.6.1) Fix Version/s: (was: 1.6.1) [~wangmiao1981] the problem is

[jira] [Resolved] (SPARK-15078) Add all TPCDS 1.4 benchmark queries for SparkSQL

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15078. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13188 [https://github.

[jira] [Comment Edited] (SPARK-7159) Support multiclass logistic regression in spark.ml

2016-05-20 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294372#comment-15294372 ] Seth Hendrickson edited comment on SPARK-7159 at 5/20/16 10:17 PM: -

[jira] [Resolved] (SPARK-15446) catalyst using BigInteger.longValueExact that not supporting java 7 and compile error

2016-05-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15446. --- Resolution: Duplicate > catalyst using BigInteger.longValueExact that not supporting java 7 and > co

[jira] [Reopened] (SPARK-15446) catalyst using BigInteger.longValueExact that not supporting java 7 and compile error

2016-05-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-15446: --- [~WeichenXu123] yes please search JIRA first, and "Fixed" is not the correct resolution > catalyst usin

[jira] [Commented] (SPARK-7159) Support multiclass logistic regression in spark.ml

2016-05-20 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294372#comment-15294372 ] Seth Hendrickson commented on SPARK-7159: - [~dbtsai][~josephkb] I'd like to take t

[jira] [Commented] (SPARK-15451) Spark PR builder should fail if code doesn't compile against JDK 7

2016-05-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294370#comment-15294370 ] Sean Owen commented on SPARK-15451: --- That's fine with me, but so is just going ahead an

[jira] [Resolved] (SPARK-15454) HadoopFsRelation should filter out files starting with _

2016-05-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15454. - Resolution: Fixed Fix Version/s: 2.0.0 > HadoopFsRelation should filter out files starting

[jira] [Assigned] (SPARK-15327) Catalyst code generation fails with complex data structure

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15327: -- Assignee: Davies Liu > Catalyst code generation fails with complex data structure > --

[jira] [Created] (SPARK-15455) For IsolatedClientLoader, we need to provide a conf to disable sharing Hadoop classes

2016-05-20 Thread Yin Huai (JIRA)
Yin Huai created SPARK-15455: Summary: For IsolatedClientLoader, we need to provide a conf to disable sharing Hadoop classes Key: SPARK-15455 URL: https://issues.apache.org/jira/browse/SPARK-15455 Project

[jira] [Commented] (SPARK-8426) Add blacklist mechanism for YARN container allocation

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294242#comment-15294242 ] Apache Spark commented on SPARK-8426: - User 'squito' has created a pull request for th

[jira] [Commented] (SPARK-15449) Wrong Data Format - Documentation Issue

2016-05-20 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294237#comment-15294237 ] Miao Wang commented on SPARK-15449: --- This example doesn't require libsvm format as it h

[jira] [Commented] (SPARK-11827) Support java.math.BigInteger in Type-Inference utilities for POJOs

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294219#comment-15294219 ] Apache Spark commented on SPARK-11827: -- User 'ted-yu' has created a pull request for

[jira] [Assigned] (SPARK-15405) YARN uploading the same __spark_conf__.zip twice

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15405: Assignee: Apache Spark > YARN uploading the same __spark_conf__.zip twice > --

[jira] [Commented] (SPARK-15405) YARN uploading the same __spark_conf__.zip twice

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294212#comment-15294212 ] Apache Spark commented on SPARK-15405: -- User 'vanzin' has created a pull request for

[jira] [Assigned] (SPARK-15405) YARN uploading the same __spark_conf__.zip twice

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15405: Assignee: (was: Apache Spark) > YARN uploading the same __spark_conf__.zip twice > ---

[jira] [Commented] (SPARK-4563) Allow spark driver to bind to different ip then advertise ip

2016-05-20 Thread Miles Crawford (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294181#comment-15294181 ] Miles Crawford commented on SPARK-4563: --- Any chance we could boost the priority of t

[jira] [Commented] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294178#comment-15294178 ] Davies Liu commented on SPARK-15285: cc [~cloud_fan] > Generated SpecificSafeProject

[jira] [Updated] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15285: --- Assignee: Wenchen Fan > Generated SpecificSafeProjection.apply method grows beyond 64 KB > --

[jira] [Updated] (SPARK-15453) Sort Merge Join to use bucketing metadata to optimize query plan

2016-05-20 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tejas Patil updated SPARK-15453: Summary: Sort Merge Join to use bucketing metadata to optimize query plan (was: Improve join plann

[jira] [Commented] (SPARK-14331) Exceptions saving to parquetFile after join from dataframes in master

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294168#comment-15294168 ] Davies Liu commented on SPARK-14331: Could you post the full stacktrace? This excepti

[jira] [Commented] (SPARK-15453) Improve join planning for bucketed / sorted tables

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294165#comment-15294165 ] Apache Spark commented on SPARK-15453: -- User 'tejasapatil' has created a pull reques

[jira] [Assigned] (SPARK-15453) Improve join planning for bucketed / sorted tables

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15453: Assignee: Apache Spark > Improve join planning for bucketed / sorted tables >

[jira] [Assigned] (SPARK-15453) Improve join planning for bucketed / sorted tables

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15453: Assignee: (was: Apache Spark) > Improve join planning for bucketed / sorted tables > -

[jira] [Commented] (SPARK-15165) Codegen can break because toCommentSafeString is not actually safe

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294144#comment-15294144 ] Apache Spark commented on SPARK-15165: -- User 'sarutak' has created a pull request fo

[jira] [Commented] (SPARK-15205) Codegen can compile the same source code more than twice

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294145#comment-15294145 ] Apache Spark commented on SPARK-15205: -- User 'sarutak' has created a pull request fo

[jira] [Closed] (SPARK-15448) Flaky test:pyspark.ml.tests.DefaultValuesTests.test_java_params

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-15448. -- Resolution: Duplicate Fix Version/s: 2.0.0 > Flaky test:pyspark.ml.tests.DefaultValuesTests.test

[jira] [Assigned] (SPARK-14031) Dataframe to csv IO, system performance enters high CPU state and write operation takes 1 hour to complete

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14031: Assignee: Apache Spark (was: Davies Liu) > Dataframe to csv IO, system performance enters

[jira] [Assigned] (SPARK-14031) Dataframe to csv IO, system performance enters high CPU state and write operation takes 1 hour to complete

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14031: Assignee: Davies Liu (was: Apache Spark) > Dataframe to csv IO, system performance enters

[jira] [Commented] (SPARK-14031) Dataframe to csv IO, system performance enters high CPU state and write operation takes 1 hour to complete

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294127#comment-15294127 ] Apache Spark commented on SPARK-14031: -- User 'davies' has created a pull request for

[jira] [Resolved] (SPARK-15438) Improve the explain of whole-stage codegen

2016-05-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15438. - Resolution: Fixed Fix Version/s: 2.0.0 > Improve the explain of whole-stage codegen >

[jira] [Commented] (SPARK-15447) Performance test for ALS in Spark 2.0

2016-05-20 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294116#comment-15294116 ] Nick Pentreath commented on SPARK-15447: [~mengxr] yes will aim to run some tests

[jira] [Commented] (SPARK-15294) Add pivot functionality to SparkR

2016-05-20 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294091#comment-15294091 ] Felix Cheung commented on SPARK-15294: -- Feel free to ping me if you need any help!

[jira] [Assigned] (SPARK-15442) PySpark QuantileDiscretizer missing "relativeError" param

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15442: Assignee: Apache Spark (was: Nick Pentreath) > PySpark QuantileDiscretizer missing "relat

[jira] [Assigned] (SPARK-15442) PySpark QuantileDiscretizer missing "relativeError" param

2016-05-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15442: Assignee: Nick Pentreath (was: Apache Spark) > PySpark QuantileDiscretizer missing "relat

[jira] [Assigned] (SPARK-15442) PySpark QuantileDiscretizer missing "relativeError" param

2016-05-20 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath reassigned SPARK-15442: -- Assignee: Nick Pentreath > PySpark QuantileDiscretizer missing "relativeError" param >

[jira] [Assigned] (SPARK-14031) Dataframe to csv IO, system performance enters high CPU state and write operation takes 1 hour to complete

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-14031: -- Assignee: Davies Liu > Dataframe to csv IO, system performance enters high CPU state and write

[jira] [Commented] (SPARK-15453) Improve join planning for bucketed / sorted tables

2016-05-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294053#comment-15294053 ] Reynold Xin commented on SPARK-15453: - [~tejasp] there are multiple issues here right

[jira] [Updated] (SPARK-15453) Improve join planning for bucketed / sorted tables

2016-05-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15453: Summary: Improve join planning for bucketed / sorted tables (was: Support for SMB Join) > Improve

  1   2   3   >