[jira] [Updated] (SPARK-13591) Remove Back-ticks in Attribute/Alias Names

2016-02-29 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-13591: Description: When calling .sql, back-ticks are automatically added. When using .sql as AttributeReference/

[jira] [Created] (SPARK-13591) Remove Back-ticks in Attribute/Alias Names

2016-02-29 Thread Xiao Li (JIRA)
Xiao Li created SPARK-13591: --- Summary: Remove Back-ticks in Attribute/Alias Names Key: SPARK-13591 URL: https://issues.apache.org/jira/browse/SPARK-13591 Project: Spark Issue Type: Bug Co

[jira] [Resolved] (SPARK-13550) Add java example for ml.clustering.BisectingKMeans

2016-02-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-13550. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11428 [https://g

[jira] [Updated] (SPARK-13550) Add java example for ml.clustering.BisectingKMeans

2016-02-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13550: -- Assignee: zhengruifeng > Add java example for ml.clustering.BisectingKMeans > -

[jira] [Commented] (SPARK-13581) LibSVM throws MatchError

2016-02-29 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173383#comment-15173383 ] Jeff Zhang commented on SPARK-13581: I suspect it is issue in the code generation. Be

[jira] [Created] (SPARK-13590) Document the behavior of spark.ml logistic regression when there are constant features

2016-02-29 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-13590: - Summary: Document the behavior of spark.ml logistic regression when there are constant features Key: SPARK-13590 URL: https://issues.apache.org/jira/browse/SPARK-13590

[jira] [Resolved] (SPARK-13029) Logistic regression returns inaccurate results when there is a column with identical value, and fit_intercept=false

2016-02-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-13029. --- Resolution: Won't Fix As discussed on the PR page, we decided to keep the current behavior, w

[jira] [Updated] (SPARK-13551) Fix fix wrong comment and remove meanless lines in mllib.JavaBisectingKMeansExample

2016-02-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13551: -- Target Version/s: 2.0.0 > Fix fix wrong comment and remove meanless lines in > mllib.JavaBisec

[jira] [Updated] (SPARK-13551) Fix fix wrong comment and remove meanless lines in mllib.JavaBisectingKMeansExample

2016-02-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13551: -- Assignee: zhengruifeng > Fix fix wrong comment and remove meanless lines in > mllib.JavaBisect

[jira] [Resolved] (SPARK-13551) Fix fix wrong comment and remove meanless lines in mllib.JavaBisectingKMeansExample

2016-02-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-13551. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11429 [https://g

[jira] [Commented] (SPARK-13546) GBT with many trees consistently giving java.lang.StackOverflowError

2016-02-29 Thread Glen Maisey (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173261#comment-15173261 ] Glen Maisey commented on SPARK-13546: - As a workaround I've thrown checkpointing on.

[jira] [Commented] (SPARK-13117) WebUI should use the local ip not 0.0.0.0

2016-02-29 Thread Jay Panicker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173254#comment-15173254 ] Jay Panicker commented on SPARK-13117: -- Sorry, [~devaraj.k] , I did not check the f

[jira] [Updated] (SPARK-13588) Unable to map Parquet file to Hive Table using HiveContext

2016-02-29 Thread Akshat Thakar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akshat Thakar updated SPARK-13588: -- Summary: Unable to map Parquet file to Hive Table using HiveContext (was: Unable to Map Parque

[jira] [Updated] (SPARK-13588) Unable to Map Parquet file to Hive Table using HiveContext

2016-02-29 Thread Akshat Thakar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akshat Thakar updated SPARK-13588: -- Description: I am trying to map existing Parquet file with external table using Pyspark script

[jira] [Updated] (SPARK-13588) Unable to Map Parquet file to Hive Table using HiveContext

2016-02-29 Thread Akshat Thakar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akshat Thakar updated SPARK-13588: -- Description: I am trying to map existing Parquet file with external table using Pyspark script

[jira] [Commented] (SPARK-13589) Flaky test: ParquetHadoopFsRelationSuite.test all data types - ByteType

2016-02-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173249#comment-15173249 ] Cheng Lian commented on SPARK-13589: [~nongli] Is SPARK-13533 related to this one? >

[jira] [Created] (SPARK-13589) Flaky test: ParquetHadoopFsRelationSuite.test all data types - ByteType

2016-02-29 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-13589: -- Summary: Flaky test: ParquetHadoopFsRelationSuite.test all data types - ByteType Key: SPARK-13589 URL: https://issues.apache.org/jira/browse/SPARK-13589 Project: Spark

[jira] [Created] (SPARK-13588) Unable to Map Parquet file to Hive Table using HiveContext

2016-02-29 Thread Akshat Thakar (JIRA)
Akshat Thakar created SPARK-13588: - Summary: Unable to Map Parquet file to Hive Table using HiveContext Key: SPARK-13588 URL: https://issues.apache.org/jira/browse/SPARK-13588 Project: Spark

[jira] [Comment Edited] (SPARK-13587) Support virtualenv in PySpark

2016-02-29 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173228#comment-15173228 ] Jeff Zhang edited comment on SPARK-13587 at 3/1/16 5:04 AM: I

[jira] [Commented] (SPARK-13587) Support virtualenv in PySpark

2016-02-29 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173228#comment-15173228 ] Jeff Zhang commented on SPARK-13587: I have implemented POC for this features. Here's

[jira] [Comment Edited] (SPARK-13587) Support virtualenv in PySpark

2016-02-29 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173228#comment-15173228 ] Jeff Zhang edited comment on SPARK-13587 at 3/1/16 5:02 AM: I

[jira] [Commented] (SPARK-12720) SQL generation support for cube, rollup, and grouping set

2016-02-29 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173226#comment-15173226 ] Xiao Li commented on SPARK-12720: - [~yhuai] I did a few tries. Unfortunately, its Expand

[jira] [Created] (SPARK-13587) Support virtualenv in PySpark

2016-02-29 Thread Jeff Zhang (JIRA)
Jeff Zhang created SPARK-13587: -- Summary: Support virtualenv in PySpark Key: SPARK-13587 URL: https://issues.apache.org/jira/browse/SPARK-13587 Project: Spark Issue Type: Improvement C

[jira] [Commented] (SPARK-13117) WebUI should use the local ip not 0.0.0.0

2016-02-29 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173224#comment-15173224 ] Devaraj K commented on SPARK-13117: --- I think we can start the Jetty server with the def

[jira] [Commented] (SPARK-13117) WebUI should use the local ip not 0.0.0.0

2016-02-29 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173218#comment-15173218 ] Devaraj K commented on SPARK-13117: --- [~jaypanicker], the proposed PR does the same but

[jira] [Comment Edited] (SPARK-13117) WebUI should use the local ip not 0.0.0.0

2016-02-29 Thread Jay Panicker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173214#comment-15173214 ] Jay Panicker edited comment on SPARK-13117 at 3/1/16 4:40 AM: -

[jira] [Comment Edited] (SPARK-13117) WebUI should use the local ip not 0.0.0.0

2016-02-29 Thread Jay Panicker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173214#comment-15173214 ] Jay Panicker edited comment on SPARK-13117 at 3/1/16 4:40 AM: -

[jira] [Commented] (SPARK-13117) WebUI should use the local ip not 0.0.0.0

2016-02-29 Thread Jay Panicker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173214#comment-15173214 ] Jay Panicker commented on SPARK-13117: -- On systems with multiple interfaces, abilit

[jira] [Comment Edited] (SPARK-6160) ChiSqSelector should keep test statistic info

2016-02-29 Thread Gayathri Murali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173163#comment-15173163 ] Gayathri Murali edited comment on SPARK-6160 at 3/1/16 3:57 AM:

[jira] [Comment Edited] (SPARK-6160) ChiSqSelector should keep test statistic info

2016-02-29 Thread Gayathri Murali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173163#comment-15173163 ] Gayathri Murali edited comment on SPARK-6160 at 3/1/16 3:53 AM:

[jira] [Updated] (SPARK-13586) add config to skip generate down time batch when restart StreamingContext

2016-02-29 Thread jeanlyn (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jeanlyn updated SPARK-13586: Priority: Minor (was: Major) > add config to skip generate down time batch when restart StreamingContext >

[jira] [Assigned] (SPARK-13586) add config to skip generate down time batch when restart StreamingContext

2016-02-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13586: Assignee: Apache Spark > add config to skip generate down time batch when restart Streamin

[jira] [Assigned] (SPARK-13586) add config to skip generate down time batch when restart StreamingContext

2016-02-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13586: Assignee: (was: Apache Spark) > add config to skip generate down time batch when resta

[jira] [Commented] (SPARK-13586) add config to skip generate down time batch when restart StreamingContext

2016-02-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173177#comment-15173177 ] Apache Spark commented on SPARK-13586: -- User 'jeanlyn' has created a pull request fo

[jira] [Created] (SPARK-13586) add config to skip generate down time batch when restart StreamingContext

2016-02-29 Thread jeanlyn (JIRA)
jeanlyn created SPARK-13586: --- Summary: add config to skip generate down time batch when restart StreamingContext Key: SPARK-13586 URL: https://issues.apache.org/jira/browse/SPARK-13586 Project: Spark

[jira] [Commented] (SPARK-6160) ChiSqSelector should keep test statistic info

2016-02-29 Thread Gayathri Murali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173163#comment-15173163 ] Gayathri Murali commented on SPARK-6160: [~josephkb] Should the test statistics re

[jira] [Commented] (SPARK-12719) SQL generation support for generators (including UDTF)

2016-02-29 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173140#comment-15173140 ] Xiao Li commented on SPARK-12719: - Actually, I involved my teammate to work on this. The

[jira] [Commented] (SPARK-12721) SQL generation support for script transformation

2016-02-29 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173138#comment-15173138 ] Xiao Li commented on SPARK-12721: - Sorry, this is delayed until Spark-13535 is resolved.

[jira] [Commented] (SPARK-12720) SQL generation support for cube, rollup, and grouping set

2016-02-29 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173133#comment-15173133 ] Xiao Li commented on SPARK-12720: - Yeah. This is what I hope. Let me use this PR to do a

[jira] [Updated] (SPARK-13580) Driver makes no progress when Executor's akka thread exits due to OOM.

2016-02-29 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated SPARK-13580: --- Summary: Driver makes no progress when Executor's akka thread exits due to OOM. (was: Driver makes n

[jira] [Commented] (SPARK-12720) SQL generation support for cube, rollup, and grouping set

2016-02-29 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173070#comment-15173070 ] Yin Huai commented on SPARK-12720: -- [~smilegator] Will the approach of handling expand i

[jira] [Updated] (SPARK-13581) LibSVM throws MatchError

2016-02-29 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky updated SPARK-13581: -- Description: When running an action on a DataFrame obtained by reading from a libsvm file a Ma

[jira] [Commented] (SPARK-13581) LibSVM throws MatchError

2016-02-29 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173058#comment-15173058 ] Jakob Odersky commented on SPARK-13581: --- It's in spark "data/mllib/sample_libsvm_da

[jira] [Commented] (SPARK-13581) LibSVM throws MatchError

2016-02-29 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173052#comment-15173052 ] Jeff Zhang commented on SPARK-13581: [~jodersky] Can you attach the data file ? I gue

[jira] [Commented] (SPARK-13583) Support `UnusedImports` Java checkstyle rule

2016-02-29 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173040#comment-15173040 ] Dongjoon Hyun commented on SPARK-13583: --- I changed the content of this issue since

[jira] [Updated] (SPARK-13583) Support `UnusedImports` Java checkstyle rule

2016-02-29 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-13583: -- Description: After SPARK-6990, `dev/lint-java` keeps Java code healthy and helps PR review by

[jira] [Created] (SPARK-13585) addPyFile behavior change between 1.6 and before

2016-02-29 Thread Santhosh Gorantla Ramakrishna (JIRA)
Santhosh Gorantla Ramakrishna created SPARK-13585: - Summary: addPyFile behavior change between 1.6 and before Key: SPARK-13585 URL: https://issues.apache.org/jira/browse/SPARK-13585 Pro

[jira] [Comment Edited] (SPARK-13580) Driver makes no progress after failed to remove broadcast on Executor

2016-02-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172786#comment-15172786 ] Shixiong Zhu edited comment on SPARK-13580 at 3/1/16 12:06 AM:

[jira] [Closed] (SPARK-13580) Driver makes no progress after failed to remove broadcast on Executor

2016-02-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu closed SPARK-13580. Resolution: Not A Bug > Driver makes no progress after failed to remove broadcast on Executor > ---

[jira] [Assigned] (SPARK-13584) ContinuousQueryManagerSuite floods the logs with garbage

2016-02-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13584: Assignee: (was: Apache Spark) > ContinuousQueryManagerSuite floods the logs with garba

[jira] [Commented] (SPARK-13584) ContinuousQueryManagerSuite floods the logs with garbage

2016-02-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172910#comment-15172910 ] Apache Spark commented on SPARK-13584: -- User 'zsxwing' has created a pull request fo

[jira] [Assigned] (SPARK-13584) ContinuousQueryManagerSuite floods the logs with garbage

2016-02-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13584: Assignee: Apache Spark > ContinuousQueryManagerSuite floods the logs with garbage > --

[jira] [Created] (SPARK-13584) ContinuousQueryManagerSuite floods the logs with garbage

2016-02-29 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-13584: Summary: ContinuousQueryManagerSuite floods the logs with garbage Key: SPARK-13584 URL: https://issues.apache.org/jira/browse/SPARK-13584 Project: Spark Issu

[jira] [Assigned] (SPARK-13583) Enforce `UnusedImports` Java checkstyle rule

2016-02-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13583: Assignee: (was: Apache Spark) > Enforce `UnusedImports` Java checkstyle rule > ---

[jira] [Commented] (SPARK-13583) Enforce `UnusedImports` Java checkstyle rule

2016-02-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172882#comment-15172882 ] Apache Spark commented on SPARK-13583: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Comment Edited] (SPARK-13337) DataFrame join-on-columns function should support null-safe equal

2016-02-29 Thread Zhong Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172881#comment-15172881 ] Zhong Wang edited comment on SPARK-13337 at 2/29/16 11:40 PM: -

[jira] [Assigned] (SPARK-13583) Enforce `UnusedImports` Java checkstyle rule

2016-02-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13583: Assignee: Apache Spark > Enforce `UnusedImports` Java checkstyle rule > --

[jira] [Commented] (SPARK-13337) DataFrame join-on-columns function should support null-safe equal

2016-02-29 Thread Zhong Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172881#comment-15172881 ] Zhong Wang commented on SPARK-13337: It doesn't help in my case, because it doesn't s

[jira] [Created] (SPARK-13583) Enforce `UnusedImports` Java checkstyle rule

2016-02-29 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-13583: - Summary: Enforce `UnusedImports` Java checkstyle rule Key: SPARK-13583 URL: https://issues.apache.org/jira/browse/SPARK-13583 Project: Spark Issue Type: Ta

[jira] [Assigned] (SPARK-13582) Improve performance of parquet reader with dictionary encoding

2016-02-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13582: Assignee: Apache Spark (was: Davies Liu) > Improve performance of parquet reader with dic

[jira] [Assigned] (SPARK-13582) Improve performance of parquet reader with dictionary encoding

2016-02-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13582: Assignee: Davies Liu (was: Apache Spark) > Improve performance of parquet reader with dic

[jira] [Commented] (SPARK-13582) Improve performance of parquet reader with dictionary encoding

2016-02-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172841#comment-15172841 ] Apache Spark commented on SPARK-13582: -- User 'davies' has created a pull request for

[jira] [Created] (SPARK-13582) Improve performance of parquet reader with dictionary encoding

2016-02-29 Thread Davies Liu (JIRA)
Davies Liu created SPARK-13582: -- Summary: Improve performance of parquet reader with dictionary encoding Key: SPARK-13582 URL: https://issues.apache.org/jira/browse/SPARK-13582 Project: Spark I

[jira] [Issue Comment Deleted] (SPARK-13571) Track current database in SQL/HiveContext

2016-02-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13571: -- Comment: was deleted (was: User 'andrewor14' has created a pull request for this issue: https://github.

[jira] [Commented] (SPARK-13571) Track current database in SQL/HiveContext

2016-02-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172825#comment-15172825 ] Apache Spark commented on SPARK-13571: -- User 'andrewor14' has created a pull request

[jira] [Updated] (SPARK-13581) LibSVM throws MatchError

2016-02-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13581: -- Assignee: Jeff Zhang I guess the input it's hoping to treat as a vector type is just a double from the

[jira] [Commented] (SPARK-13580) Driver makes no progress after failed to remove broadcast on Executor

2016-02-29 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172810#comment-15172810 ] Liyin Tang commented on SPARK-13580: Thanks [~zsxwing] for the investigation! That's

[jira] [Commented] (SPARK-13337) DataFrame join-on-columns function should support null-safe equal

2016-02-29 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172796#comment-15172796 ] Xiao Li commented on SPARK-13337: - Sorry, I do not get your point. Join-using-columns doe

[jira] [Commented] (SPARK-13580) Driver makes no progress after failed to remove broadcast on Executor

2016-02-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172786#comment-15172786 ] Shixiong Zhu commented on SPARK-13580: -- It hapens in an Akka thread. You can add {{-

[jira] [Created] (SPARK-13581) LibSVM throws MatchError

2016-02-29 Thread Jakob Odersky (JIRA)
Jakob Odersky created SPARK-13581: - Summary: LibSVM throws MatchError Key: SPARK-13581 URL: https://issues.apache.org/jira/browse/SPARK-13581 Project: Spark Issue Type: Bug Componen

[jira] [Commented] (SPARK-13430) Expose ml summary function in PySpark for classification and regression models

2016-02-29 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172775#comment-15172775 ] Bryan Cutler commented on SPARK-13430: -- I can work on adding this > Expose ml summa

[jira] [Commented] (SPARK-12817) Remove CacheManager and replace it with new BlockManager.getOrElseUpdate method

2016-02-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172773#comment-15172773 ] Apache Spark commented on SPARK-12817: -- User 'JoshRosen' has created a pull request

[jira] [Updated] (SPARK-12817) Remove CacheManager and replace it with new BlockManager.getOrElseUpdate method

2016-02-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-12817: --- Description: CacheManager directly calls MemoryStore.unrollSafely() and has its own logic for handli

[jira] [Updated] (SPARK-12817) Remove CacheManager and replace it with new BlockManager.getOrElseUpdate method

2016-02-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-12817: --- Summary: Remove CacheManager and replace it with new BlockManager.getOrElseUpdate method (was: Simpl

[jira] [Comment Edited] (SPARK-13337) DataFrame join-on-columns function should support null-safe equal

2016-02-29 Thread Zhong Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172709#comment-15172709 ] Zhong Wang edited comment on SPARK-13337 at 2/29/16 10:05 PM: -

[jira] [Commented] (SPARK-13337) DataFrame join-on-columns function should support null-safe equal

2016-02-29 Thread Zhong Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172709#comment-15172709 ] Zhong Wang commented on SPARK-13337: For an outer join, it is difficult to eliminate

[jira] [Commented] (SPARK-13580) Driver makes no progress after failed to remove broadcast on Executor

2016-02-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172689#comment-15172689 ] Shixiong Zhu commented on SPARK-13580: -- OOM in the executor side: Exception: java.l

[jira] [Commented] (SPARK-13580) Driver makes no progress after failed to remove broadcast on Executor

2016-02-29 Thread Jingwei Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172672#comment-15172672 ] Jingwei Lu commented on SPARK-13580: Attached the executor log for #11.[~zsxwing] >

[jira] [Updated] (SPARK-13580) Driver makes no progress after failed to remove broadcast on Executor

2016-02-29 Thread Jingwei Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingwei Lu updated SPARK-13580: --- Attachment: stderrfiltered.txt.gz > Driver makes no progress after failed to remove broadcast on Exec

[jira] [Comment Edited] (SPARK-13580) Driver makes no progress after failed to remove broadcast on Executor

2016-02-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172658#comment-15172658 ] Shixiong Zhu edited comment on SPARK-13580 at 2/29/16 9:38 PM:

[jira] [Commented] (SPARK-13580) Driver makes no progress after failed to remove broadcast on Executor

2016-02-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172658#comment-15172658 ] Shixiong Zhu commented on SPARK-13580: -- Could you post the executor log? > Driver m

[jira] [Commented] (SPARK-10063) Remove DirectParquetOutputCommitter

2016-02-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172654#comment-15172654 ] Steve Loughran commented on SPARK-10063: sorry! HADOOP-9565 > Remove DirectParqu

[jira] [Updated] (SPARK-13580) Driver makes no progress after failed to remove broadcast on Executor

2016-02-29 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated SPARK-13580: --- Attachment: executor_jstack driver_log.txt driver_jstack.txt > Driver

[jira] [Created] (SPARK-13580) Driver makes no progress after failed to remove broadcast on Executor

2016-02-29 Thread Liyin Tang (JIRA)
Liyin Tang created SPARK-13580: -- Summary: Driver makes no progress after failed to remove broadcast on Executor Key: SPARK-13580 URL: https://issues.apache.org/jira/browse/SPARK-13580 Project: Spark

[jira] [Commented] (SPARK-13255) Integrate vectorized parquet scan with whole stage codegen.

2016-02-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172632#comment-15172632 ] Apache Spark commented on SPARK-13255: -- User 'nongli' has created a pull request for

[jira] [Resolved] (SPARK-13478) Fetching delegation tokens for Hive fails when using proxy users

2016-02-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-13478. Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.0.0 > Fetching

[jira] [Updated] (SPARK-13123) Add wholestage codegen for sort

2016-02-29 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-13123: - Assignee: Sameer Agarwal (was: Nong Li) > Add wholestage codegen for sort >

[jira] [Created] (SPARK-13579) Stop building assemblies for Spark

2016-02-29 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-13579: -- Summary: Stop building assemblies for Spark Key: SPARK-13579 URL: https://issues.apache.org/jira/browse/SPARK-13579 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-13123) Add wholestage codegen for sort

2016-02-29 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-13123. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11359 [https://github.com/

[jira] [Created] (SPARK-13578) Make launcher lib and user scripts handle jar directories instead of single assembly file

2016-02-29 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-13578: -- Summary: Make launcher lib and user scripts handle jar directories instead of single assembly file Key: SPARK-13578 URL: https://issues.apache.org/jira/browse/SPARK-13578

[jira] [Created] (SPARK-13577) Allow YARN to handle multiple jars, archive when uploading Spark dependencies

2016-02-29 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-13577: -- Summary: Allow YARN to handle multiple jars, archive when uploading Spark dependencies Key: SPARK-13577 URL: https://issues.apache.org/jira/browse/SPARK-13577 Pro

[jira] [Updated] (SPARK-13575) Remove streaming backends' assemblies

2016-02-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-13575: --- Component/s: (was: YARN) (was: Spark Core) Streaming

[jira] [Created] (SPARK-13576) Make examples jar not be an assembly

2016-02-29 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-13576: -- Summary: Make examples jar not be an assembly Key: SPARK-13576 URL: https://issues.apache.org/jira/browse/SPARK-13576 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-13575) Remove streaming backends' assemblies

2016-02-29 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-13575: -- Summary: Remove streaming backends' assemblies Key: SPARK-13575 URL: https://issues.apache.org/jira/browse/SPARK-13575 Project: Spark Issue Type: Sub-tas

[jira] [Updated] (SPARK-12941) Spark-SQL JDBC Oracle dialect fails to map string datatypes to Oracle VARCHAR datatype

2016-02-29 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-12941: - Fix Version/s: 1.5.3 1.4.2 > Spark-SQL JDBC Oracle dialect fails to map string datatyp

[jira] [Resolved] (SPARK-7253) Add example of belief propagation with GraphX

2016-02-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-7253. -- Resolution: Workaround An example was provided using the GraphFrames API (https://github.com/gr

[jira] [Resolved] (SPARK-3665) Java API for GraphX

2016-02-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3665. -- Resolution: Workaround GraphFrames (https://github.com/graphframes/graphframes) wraps GraphX al

[jira] [Resolved] (SPARK-3789) [GRAPHX] Python bindings for GraphX

2016-02-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3789. -- Resolution: Workaround GraphFrames (https://github.com/graphframes/graphframes) wraps GraphX al

[jira] [Resolved] (SPARK-7256) Add Graph abstraction which uses DataFrame

2016-02-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-7256. -- Resolution: Later GraphFrames (https://github.com/graphframes/graphframes) implemented this idea

[jira] [Commented] (SPARK-13574) Improve parquet dictionary decoding for strings

2016-02-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172518#comment-15172518 ] Apache Spark commented on SPARK-13574: -- User 'nongli' has created a pull request for

  1   2   3   >