[jira] [Created] (SPARK-3844) Truncate appName in WebUI if it is too long

2014-10-08 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-3844: Summary: Truncate appName in WebUI if it is too long Key: SPARK-3844 URL: https://issues.apache.org/jira/browse/SPARK-3844 Project: Spark Issue Type: Improve

[jira] [Updated] (SPARK-3844) Truncate appName in WebUI if it is too long

2014-10-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3844: - Attachment: long-title.png > Truncate appName in WebUI if it is too long > ---

[jira] [Commented] (SPARK-3844) Truncate appName in WebUI if it is too long

2014-10-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163215#comment-14163215 ] Apache Spark commented on SPARK-3844: - User 'mengxr' has created a pull request for th

[jira] [Commented] (SPARK-3559) appendReadColumnIDs and appendReadColumnNames introduce unnecessary columns in the lists of needed column ids and column names stored in hiveConf

2014-10-08 Thread Venkata Ramana G (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163218#comment-14163218 ] Venkata Ramana G commented on SPARK-3559: - As same hiveConf is used across queries

[jira] [Commented] (SPARK-3158) Avoid 1 extra aggregation for DecisionTree training

2014-10-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163223#comment-14163223 ] Apache Spark commented on SPARK-3158: - User 'chouqin' has created a pull request for t

[jira] [Commented] (SPARK-3630) Identify cause of Kryo+Snappy PARSING_ERROR

2014-10-08 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163225#comment-14163225 ] Adrian Wang commented on SPARK-3630: I ran into similar issue when using SQL, and got

[jira] [Created] (SPARK-3845) SQLContext(...) should inherit configurations from SparkContext

2014-10-08 Thread Jianshi Huang (JIRA)
Jianshi Huang created SPARK-3845: Summary: SQLContext(...) should inherit configurations from SparkContext Key: SPARK-3845 URL: https://issues.apache.org/jira/browse/SPARK-3845 Project: Spark

[jira] [Created] (SPARK-3846) KryoException when doing joins in SparkSQL

2014-10-08 Thread Jianshi Huang (JIRA)
Jianshi Huang created SPARK-3846: Summary: KryoException when doing joins in SparkSQL Key: SPARK-3846 URL: https://issues.apache.org/jira/browse/SPARK-3846 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-3846) KryoException when doing joins in SparkSQL

2014-10-08 Thread Jianshi Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianshi Huang updated SPARK-3846: - Description: The error is reproducible when I join two tables manually. The error message is

[jira] [Closed] (SPARK-2890) Spark SQL should allow SELECT with duplicated columns

2014-10-08 Thread Jianshi Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianshi Huang closed SPARK-2890. > Spark SQL should allow SELECT with duplicated columns > --

[jira] [Updated] (SPARK-3846) KryoException when doing joins in SparkSQL

2014-10-08 Thread Jianshi Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianshi Huang updated SPARK-3846: - Description: The error is reproducible when I join two tables manually. The error message is like

[jira] [Updated] (SPARK-2429) Hierarchical Implementation of KMeans

2014-10-08 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Ishikawa updated SPARK-2429: --- Attachment: The Result of Benchmarking a Hierarchical Clustering.pdf Hi [~rnowling], I'm sorry for th

[jira] [Comment Edited] (SPARK-2429) Hierarchical Implementation of KMeans

2014-10-08 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163318#comment-14163318 ] Yu Ishikawa edited comment on SPARK-2429 at 10/8/14 10:52 AM: --

[jira] [Commented] (SPARK-3785) Support off-loading computations to a GPU

2014-10-08 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163326#comment-14163326 ] Mridul Muralidharan commented on SPARK-3785: [~sowen] We had prototyped a solu

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

2014-10-08 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163351#comment-14163351 ] Mridul Muralidharan commented on SPARK-3561: [~pwendell] If I understood the p

[jira] [Commented] (SPARK-3845) SQLContext(...) should inherit configurations from SparkContext

2014-10-08 Thread Venkata Ramana G (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163367#comment-14163367 ] Venkata Ramana G commented on SPARK-3845: - As I understand that is the way, it wor

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

2014-10-08 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163376#comment-14163376 ] Mridul Muralidharan commented on SPARK-3561: [~ozhurakousky] I think the disco

[jira] [Commented] (SPARK-3630) Identify cause of Kryo+Snappy PARSING_ERROR

2014-10-08 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163391#comment-14163391 ] DB Tsai commented on SPARK-3630: I think there are some issues in the shuffle manger with

[jira] [Created] (SPARK-3847) Enum.hashCode is only consistent within the same JVM

2014-10-08 Thread Nathan Bijnens (JIRA)
Nathan Bijnens created SPARK-3847: - Summary: Enum.hashCode is only consistent within the same JVM Key: SPARK-3847 URL: https://issues.apache.org/jira/browse/SPARK-3847 Project: Spark Issue Ty

[jira] [Commented] (SPARK-3814) Bitwise & does not work in Hive

2014-10-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163406#comment-14163406 ] Apache Spark commented on SPARK-3814: - User 'ravipesala' has created a pull request fo

[jira] [Commented] (SPARK-3847) Enum.hashCode is only consistent within the same JVM

2014-10-08 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163407#comment-14163407 ] Mridul Muralidharan commented on SPARK-3847: Wow, nice bug ! This is unexpecte

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2014-10-08 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163423#comment-14163423 ] DB Tsai commented on SPARK-1239: +1, we run into this issue as well. > Don't fetch all ma

[jira] [Commented] (SPARK-3630) Identify cause of Kryo+Snappy PARSING_ERROR

2014-10-08 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163458#comment-14163458 ] DB Tsai commented on SPARK-3630: I think there is something else going wrong in the curren

[jira] [Commented] (SPARK-1720) use LD_LIBRARY_PATH instead of -Djava.library.path

2014-10-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163483#comment-14163483 ] Apache Spark commented on SPARK-1720: - User 'witgo' has created a pull request for thi

[jira] [Commented] (SPARK-1719) spark.executor.extraLibraryPath isn't applied on yarn

2014-10-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163484#comment-14163484 ] Apache Spark commented on SPARK-1719: - User 'witgo' has created a pull request for thi

[jira] [Resolved] (SPARK-3837) Warn when YARN is killing containers for exceeding memory limits

2014-10-08 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3837. -- Resolution: Duplicate duplicate of SPARK-3780 > Warn when YARN is killing containers for exceed

[jira] [Updated] (SPARK-3780) YarnAllocator should look at the container completed diagnostic message

2014-10-08 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3780: - Issue Type: Improvement (was: Bug) > YarnAllocator should look at the container completed diagnos

[jira] [Resolved] (SPARK-3788) Yarn dist cache code is not friendly to HDFS HA, Federation

2014-10-08 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3788. -- Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 > Yarn dist cache code is

[jira] [Commented] (SPARK-3121) Wrong implementation of implicit bytesWritableConverter

2014-10-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163523#comment-14163523 ] Apache Spark commented on SPARK-3121: - User 'james64' has created a pull request for t

[jira] [Commented] (SPARK-3559) appendReadColumnIDs and appendReadColumnNames introduce unnecessary columns in the lists of needed column ids and column names stored in hiveConf

2014-10-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163543#comment-14163543 ] Apache Spark commented on SPARK-3559: - User 'gvramana' has created a pull request for

[jira] [Commented] (SPARK-3121) Wrong implementation of implicit bytesWritableConverter

2014-10-08 Thread Jakub Dubovsky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163554#comment-14163554 ] Jakub Dubovsky commented on SPARK-3121: --- I am working on an issue > Wrong implement

[jira] [Commented] (SPARK-3781) code style format

2014-10-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163569#comment-14163569 ] Apache Spark commented on SPARK-3781: - User 'shijinkui' has created a pull request for

[jira] [Created] (SPARK-3848) yarn alpha doesn't build on master

2014-10-08 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-3848: Summary: yarn alpha doesn't build on master Key: SPARK-3848 URL: https://issues.apache.org/jira/browse/SPARK-3848 Project: Spark Issue Type: Bug Co

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

2014-10-08 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163618#comment-14163618 ] Nicholas Chammas commented on SPARK-3561: - {quote} Obviously this does not work in

[jira] [Created] (SPARK-3849) Automate remaining Scala style rules

2014-10-08 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3849: --- Summary: Automate remaining Scala style rules Key: SPARK-3849 URL: https://issues.apache.org/jira/browse/SPARK-3849 Project: Spark Issue Type: Improvem

[jira] [Commented] (SPARK-3848) yarn alpha doesn't build on master

2014-10-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163637#comment-14163637 ] Apache Spark commented on SPARK-3848: - User 'sarutak' has created a pull request for t

[jira] [Created] (SPARK-3850) Scala style: Disallow trailing spaces

2014-10-08 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3850: --- Summary: Scala style: Disallow trailing spaces Key: SPARK-3850 URL: https://issues.apache.org/jira/browse/SPARK-3850 Project: Spark Issue Type: Sub-tas

[jira] [Updated] (SPARK-3848) yarn alpha doesn't build on master

2014-10-08 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3848: - Assignee: Kousuke Saruta (was: Thomas Graves) > yarn alpha doesn't build on master >

[jira] [Commented] (SPARK-3785) Support off-loading computations to a GPU

2014-10-08 Thread RJ Nowling (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163712#comment-14163712 ] RJ Nowling commented on SPARK-3785: --- Part of my graduate work involved implementing phys

[jira] [Commented] (SPARK-3630) Identify cause of Kryo+Snappy PARSING_ERROR

2014-10-08 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163754#comment-14163754 ] Adrian Wang commented on SPARK-3630: So we can limit this issue to some bug in shuffle

[jira] [Resolved] (SPARK-3848) yarn alpha doesn't build on master

2014-10-08 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3848. -- Resolution: Fixed Fix Version/s: 1.2.0 > yarn alpha doesn't build on master > ---

[jira] [Resolved] (SPARK-3420) Using Sphinx to generate API docs for PySpark

2014-10-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3420. --- Resolution: Fixed Fix Version/s: 1.2.0 I think this is resolved by https://github.com/apache/sp

[jira] [Reopened] (SPARK-3412) Add Missing Types for Row API

2014-10-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reopened SPARK-3412: --- It looks like that other PR accidentally mentioned this JIRA, so the merge script automatically closed th

[jira] [Updated] (SPARK-3158) Avoid 1 extra aggregation for DecisionTree training

2014-10-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3158: - Assignee: Qiping Li > Avoid 1 extra aggregation for DecisionTree training > --

[jira] [Commented] (SPARK-3633) Fetches failure observed after SPARK-2711

2014-10-08 Thread Chen Song (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163859#comment-14163859 ] Chen Song commented on SPARK-3633: -- Looks like we have addressed fetch failure caused by

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

2014-10-08 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164019#comment-14164019 ] Patrick Wendell commented on SPARK-3561: I wanted to wait for a more complete desi

[jira] [Commented] (SPARK-3594) try more rows during inferSchema

2014-10-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164029#comment-14164029 ] Apache Spark commented on SPARK-3594: - User 'davies' has created a pull request for th

[jira] [Created] (SPARK-3851) Support for reading parquet files with different but compatible schema

2014-10-08 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-3851: --- Summary: Support for reading parquet files with different but compatible schema Key: SPARK-3851 URL: https://issues.apache.org/jira/browse/SPARK-3851 Project: S

[jira] [Created] (SPARK-3852) Document spark.driver.extra* configs

2014-10-08 Thread Andrew Or (JIRA)
Andrew Or created SPARK-3852: Summary: Document spark.driver.extra* configs Key: SPARK-3852 URL: https://issues.apache.org/jira/browse/SPARK-3852 Project: Spark Issue Type: Bug Componen

[jira] [Created] (SPARK-3853) JsonRDD does not support converting fields to type Timestamp

2014-10-08 Thread Michael Timper (JIRA)
Michael Timper created SPARK-3853: - Summary: JsonRDD does not support converting fields to type Timestamp Key: SPARK-3853 URL: https://issues.apache.org/jira/browse/SPARK-3853 Project: Spark

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

2014-10-08 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164095#comment-14164095 ] Mridul Muralidharan commented on SPARK-3561: I agree with [~pwendell] that it

[jira] [Commented] (SPARK-3853) JsonRDD does not support converting fields to type Timestamp

2014-10-08 Thread Michael Timper (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164098#comment-14164098 ] Michael Timper commented on SPARK-3853: --- I have a fix, I'll submit a pull request sh

[jira] [Updated] (SPARK-2759) The ability to read binary files into Spark

2014-10-08 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-2759: - Target Version/s: 1.2.0 > The ability to read binary files into Spark > --

[jira] [Updated] (SPARK-2759) The ability to read binary files into Spark

2014-10-08 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-2759: - Assignee: Kevin Mader > The ability to read binary files into Spark >

[jira] [Created] (SPARK-3854) Scala style: require spaces before `{`

2014-10-08 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-3854: - Summary: Scala style: require spaces before `{` Key: SPARK-3854 URL: https://issues.apache.org/jira/browse/SPARK-3854 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-3851) Support for reading parquet files with different but compatible schema

2014-10-08 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164191#comment-14164191 ] Cody Koeninger commented on SPARK-3851: --- So I have a couple of questions 1. Does i

[jira] [Resolved] (SPARK-3841) Pretty-print Params case classes for tests

2014-10-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3841. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2700 [https://githu

[jira] [Created] (SPARK-3855) Binding Exception when running PythonUDFs

2014-10-08 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-3855: --- Summary: Binding Exception when running PythonUDFs Key: SPARK-3855 URL: https://issues.apache.org/jira/browse/SPARK-3855 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3855) Binding Exception when running PythonUDFs

2014-10-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164210#comment-14164210 ] Apache Spark commented on SPARK-3855: - User 'marmbrus' has created a pull request for

[jira] [Commented] (SPARK-3720) support ORC in spark sql

2014-10-08 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164222#comment-14164222 ] Zhan Zhang commented on SPARK-3720: --- There is another gira spark-2883 opened on 06/Aug/1

[jira] [Commented] (SPARK-3720) support ORC in spark sql

2014-10-08 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164224#comment-14164224 ] Zhan Zhang commented on SPARK-3720: --- By the way, I don't think the current approach can

[jira] [Comment Edited] (SPARK-3720) support ORC in spark sql

2014-10-08 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164222#comment-14164222 ] Zhan Zhang edited comment on SPARK-3720 at 10/8/14 9:45 PM: Th

[jira] [Comment Edited] (SPARK-3720) support ORC in spark sql

2014-10-08 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164224#comment-14164224 ] Zhan Zhang edited comment on SPARK-3720 at 10/8/14 9:47 PM: By

[jira] [Commented] (SPARK-3847) Enum.hashCode is only consistent within the same JVM

2014-10-08 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164246#comment-14164246 ] Patrick Wendell commented on SPARK-3847: We might be able to reflect on the key ty

[jira] [Created] (SPARK-3856) Clean deprecated usage after breeze 0.10 upgrade

2014-10-08 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-3856: Summary: Clean deprecated usage after breeze 0.10 upgrade Key: SPARK-3856 URL: https://issues.apache.org/jira/browse/SPARK-3856 Project: Spark Issue Type: Im

[jira] [Commented] (SPARK-3856) Clean deprecated usage after breeze 0.10 upgrade

2014-10-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164256#comment-14164256 ] Apache Spark commented on SPARK-3856: - User 'mengxr' has created a pull request for th

[jira] [Updated] (SPARK-3838) Python code example for Word2Vec in user guide

2014-10-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3838: - Assignee: Liquan Pei > Python code example for Word2Vec in user guide > --

[jira] [Updated] (SPARK-3439) Add Canopy Clustering Algorithm

2014-10-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3439: - Target Version/s: (was: 1.2.0) > Add Canopy Clustering Algorithm > -

[jira] [Updated] (SPARK-3855) Binding Exception when running PythonUDFs

2014-10-08 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-3855: -- Description: {code} from pyspark import * from pyspark.sql import * sc = SparkContext() sqlContext = SQL

[jira] [Resolved] (SPARK-3336) [Spark SQL] In pyspark, cannot group by field on UDF

2014-10-08 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-3336. --- Resolution: Duplicate duplicated to https://issues.apache.org/jira/browse/SPARK-3855 > [Spark SQL] In

[jira] [Resolved] (SPARK-3843) Cleanup scalastyle.txt at the end of running dev/scalastyle

2014-10-08 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3843. Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Kousuke Saruta

[jira] [Commented] (SPARK-3847) Enum.hashCode is only consistent within the same JVM

2014-10-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164280#comment-14164280 ] Josh Rosen commented on SPARK-3847: --- Java arrays' hashCodes have a similar problem: they

[jira] [Commented] (SPARK-3637) NPE in ShuffleMapTask

2014-10-08 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164286#comment-14164286 ] Patrick Wendell commented on SPARK-3637: Can you provide a small example that repr

[jira] [Created] (SPARK-3857) Create a join package for various join operators

2014-10-08 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-3857: -- Summary: Create a join package for various join operators Key: SPARK-3857 URL: https://issues.apache.org/jira/browse/SPARK-3857 Project: Spark Issue Type: Task

[jira] [Updated] (SPARK-3637) NPE in ShuffleMapTask

2014-10-08 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3637: --- Component/s: Spark Core > NPE in ShuffleMapTask > - > > Ke

[jira] [Commented] (SPARK-3857) Create a join package for various join operators

2014-10-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164291#comment-14164291 ] Apache Spark commented on SPARK-3857: - User 'rxin' has created a pull request for this

[jira] [Commented] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

2014-10-08 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164342#comment-14164342 ] Davies Liu commented on SPARK-2870: --- [~nchammas] There is something different from sc.in

[jira] [Commented] (SPARK-3853) JsonRDD does not support converting fields to type Timestamp

2014-10-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164344#comment-14164344 ] Apache Spark commented on SPARK-3853: - User 'mtimper' has created a pull request for t

[jira] [Created] (SPARK-3858) SchemaRDD.generate ignores alias argument

2014-10-08 Thread Nathan Howell (JIRA)
Nathan Howell created SPARK-3858: Summary: SchemaRDD.generate ignores alias argument Key: SPARK-3858 URL: https://issues.apache.org/jira/browse/SPARK-3858 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-3859) Use consistent config names for duration (with units!)

2014-10-08 Thread Andrew Or (JIRA)
Andrew Or created SPARK-3859: Summary: Use consistent config names for duration (with units!) Key: SPARK-3859 URL: https://issues.apache.org/jira/browse/SPARK-3859 Project: Spark Issue Type: Impr

[jira] [Updated] (SPARK-3859) Use consistent config names for duration (with units!)

2014-10-08 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3859: - Description: There are many configs in Spark that refer to some unit of time. However, from the first glan

[jira] [Commented] (SPARK-3858) SchemaRDD.generate ignores alias argument

2014-10-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164376#comment-14164376 ] Apache Spark commented on SPARK-3858: - User 'NathanHowell' has created a pull request

[jira] [Created] (SPARK-3860) Improve dimension joins

2014-10-08 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-3860: -- Summary: Improve dimension joins Key: SPARK-3860 URL: https://issues.apache.org/jira/browse/SPARK-3860 Project: Spark Issue Type: Improvement Component

[jira] [Created] (SPARK-3861) Avoid rebuilding hash tables on each partition

2014-10-08 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-3861: -- Summary: Avoid rebuilding hash tables on each partition Key: SPARK-3861 URL: https://issues.apache.org/jira/browse/SPARK-3861 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-3862) MultiWayBroadcastInnerHashJoin

2014-10-08 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-3862: -- Summary: MultiWayBroadcastInnerHashJoin Key: SPARK-3862 URL: https://issues.apache.org/jira/browse/SPARK-3862 Project: Spark Issue Type: Sub-task Compo

[jira] [Commented] (SPARK-3861) Avoid rebuilding hash tables on each partition

2014-10-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164418#comment-14164418 ] Apache Spark commented on SPARK-3861: - User 'rxin' has created a pull request for this

[jira] [Created] (SPARK-3864) Specialize join for tables with unique integer keys

2014-10-08 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-3864: -- Summary: Specialize join for tables with unique integer keys Key: SPARK-3864 URL: https://issues.apache.org/jira/browse/SPARK-3864 Project: Spark Issue Type: Sub

[jira] [Created] (SPARK-3863) Cache broadcasted tables and reuse them across queries

2014-10-08 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-3863: -- Summary: Cache broadcasted tables and reuse them across queries Key: SPARK-3863 URL: https://issues.apache.org/jira/browse/SPARK-3863 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-3831) Filter rule Improvement and bool expression optimization.

2014-10-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3831. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2692 [https:/

[jira] [Resolved] (SPARK-3713) Use JSON to serialize DataType

2014-10-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3713. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2563 [https:/

[jira] [Created] (SPARK-3865) Dimension table broadcast shouldn't be eager

2014-10-08 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-3865: -- Summary: Dimension table broadcast shouldn't be eager Key: SPARK-3865 URL: https://issues.apache.org/jira/browse/SPARK-3865 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-3866) Clean up python/run-tests problems

2014-10-08 Thread cocoatomo (JIRA)
cocoatomo created SPARK-3866: Summary: Clean up python/run-tests problems Key: SPARK-3866 URL: https://issues.apache.org/jira/browse/SPARK-3866 Project: Spark Issue Type: Bug Components

[jira] [Updated] (SPARK-3866) Clean up python/run-tests problems

2014-10-08 Thread cocoatomo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cocoatomo updated SPARK-3866: - Description: This issue is a overhaul issue to remove problems encountered when I run ./python/run-tests

[jira] [Updated] (SPARK-3866) Clean up python/run-tests problems

2014-10-08 Thread cocoatomo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cocoatomo updated SPARK-3866: - Environment: Mac OS X 10.9.5, Python 2.7.8, IPython 2.2.0, Java 1.8.0_20 (was: Mac OS X 10.9.5, Python 2.

[jira] [Commented] (SPARK-3819) Jenkins should compile Spark against multiple versions of Hadoop

2014-10-08 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164453#comment-14164453 ] Patrick Wendell commented on SPARK-3819: Sorry I should have been more clear. We a

[jira] [Updated] (SPARK-3866) Clean up python/run-tests problems

2014-10-08 Thread cocoatomo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cocoatomo updated SPARK-3866: - Description: This issue is a overhaul issue to remove problems encountered when I run ./python/run-tests

[jira] [Updated] (SPARK-3866) Clean up python/run-tests problems

2014-10-08 Thread cocoatomo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cocoatomo updated SPARK-3866: - Description: This issue is a overhaul issue to remove problems encountered when I run ./python/run-tests

[jira] [Commented] (SPARK-3839) Reimplement HashOuterJoin to construct hash table of only one relation

2014-10-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164459#comment-14164459 ] Apache Spark commented on SPARK-3839: - User 'Ishiihara' has created a pull request for

[jira] [Updated] (SPARK-3866) Clean up python/run-tests problems

2014-10-08 Thread cocoatomo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cocoatomo updated SPARK-3866: - Attachment: unit-tests.log An output from ./python/run-tests > Clean up python/run-tests problems > -

[jira] [Comment Edited] (SPARK-2429) Hierarchical Implementation of KMeans

2014-10-08 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163318#comment-14163318 ] Yu Ishikawa edited comment on SPARK-2429 at 10/9/14 12:29 AM: --

  1   2   >