[jira] [Commented] (SPARK-17396) Threads number keep increasing when query on external CSV partitioned table

2016-09-05 Thread pin_zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464602#comment-15464602 ] pin_zhang commented on SPARK-17396: --- 1.Thousand of thread created look like ForkJoinPool-20-worker-9"

[jira] [Assigned] (SPARK-17387) Creating SparkContext() from python without spark-submit ignores user conf

2016-09-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17387: Assignee: (was: Apache Spark) > Creating SparkContext() from python without

[jira] [Commented] (SPARK-17387) Creating SparkContext() from python without spark-submit ignores user conf

2016-09-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464579#comment-15464579 ] Apache Spark commented on SPARK-17387: -- User 'zjffdu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17387) Creating SparkContext() from python without spark-submit ignores user conf

2016-09-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17387: Assignee: Apache Spark > Creating SparkContext() from python without spark-submit ignores

[jira] [Commented] (SPARK-17379) Upgrade netty-all to 4.0.41.Final (4.1.5-Final not compatible)

2016-09-05 Thread Adam Roberts (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464539#comment-15464539 ] Adam Roberts commented on SPARK-17379: -- Title modified to at least mention 4.1.5, no new test

[jira] [Commented] (SPARK-17396) Threads number keep increasing when query on external CSV partitioned table

2016-09-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464525#comment-15464525 ] Sean Owen commented on SPARK-17396: --- Can you be more specific? what pool, and what problem do you

[jira] [Updated] (SPARK-17379) Upgrade netty-all to 4.0.41.Final (4.1.5-Final not compatible)

2016-09-05 Thread Adam Roberts (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Roberts updated SPARK-17379: - Summary: Upgrade netty-all to 4.0.41.Final (4.1.5-Final not compatible) (was: Upgrade netty-all

[jira] [Updated] (SPARK-17379) Upgrade netty-all to 4.0.41.Final (4.1.5-Final not compatible)

2016-09-05 Thread Adam Roberts (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Roberts updated SPARK-17379: - Priority: Trivial (was: Minor) > Upgrade netty-all to 4.0.41.Final (4.1.5-Final not compatible)

[jira] [Commented] (SPARK-17397) what to do when awaitTermination() throws?

2016-09-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464519#comment-15464519 ] Sean Owen commented on SPARK-17397: --- Generally questions go to u...@spark.apache.org, but this might

[jira] [Updated] (SPARK-17378) Upgrade snappy-java to 1.1.2.6

2016-09-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17378: -- Assignee: Adam Roberts Priority: Trivial (was: Minor) > Upgrade snappy-java to 1.1.2.6 >

[jira] [Assigned] (SPARK-17378) Upgrade snappy-java to 1.1.2.6

2016-09-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17378: Assignee: Apache Spark > Upgrade snappy-java to 1.1.2.6 > --

[jira] [Assigned] (SPARK-17378) Upgrade snappy-java to 1.1.2.6

2016-09-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17378: Assignee: (was: Apache Spark) > Upgrade snappy-java to 1.1.2.6 >

[jira] [Commented] (SPARK-17378) Upgrade snappy-java to 1.1.2.6

2016-09-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464497#comment-15464497 ] Apache Spark commented on SPARK-17378: -- User 'a-roberts' has created a pull request for this issue:

[jira] [Updated] (SPARK-17378) Upgrade snappy-java to 1.1.2.6

2016-09-05 Thread Adam Roberts (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Roberts updated SPARK-17378: - Description: We should upgrade the snappy-java version to pick up the following fix we see at

[jira] [Commented] (SPARK-4502) Spark SQL reads unneccesary nested fields from Parquet

2016-09-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464492#comment-15464492 ] Apache Spark commented on SPARK-4502: - User 'xuanyuanking' has created a pull request for this issue:

[jira] [Commented] (SPARK-13721) Add support for LATERAL VIEW OUTER explode()

2016-09-05 Thread Ewan Leith (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464427#comment-15464427 ] Ewan Leith commented on SPARK-13721: Assuming Don's use case is the same as ours, we have to do odd

[jira] [Comment Edited] (SPARK-17380) Spark streaming with a multi shard Kinesis freezes after several days (memory/resource leak?)

2016-09-05 Thread Xeto (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15463014#comment-15463014 ] Xeto edited comment on SPARK-17380 at 9/5/16 8:10 AM: -- I simplified the POC on Spark

[jira] [Comment Edited] (SPARK-17400) MinMaxScaler.transform() outputs DenseVector by default, which causes poor performance

2016-09-05 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464360#comment-15464360 ] Nick Pentreath edited comment on SPARK-17400 at 9/5/16 7:42 AM: Can you

[jira] [Commented] (SPARK-17400) MinMaxScaler.transform() outputs DenseVector by default, which causes poor performance

2016-09-05 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464360#comment-15464360 ] Nick Pentreath commented on SPARK-17400: Can you comment more on the performance issue - are you

[jira] [Updated] (SPARK-17400) MinMaxScaler.transform() outputs DenseVector by default, which causes poor performance

2016-09-05 Thread Frank Dai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frank Dai updated SPARK-17400: -- Description: MinMaxScaler.transform() outputs DenseVector by default, which will cause poor

[jira] [Updated] (SPARK-17400) MinMaxScaler.transform() outputs DenseVector by default, which causes poor performance

2016-09-05 Thread Frank Dai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frank Dai updated SPARK-17400: -- Description: MinMaxScaler.transform() outputs DenseVector by default, which will cause poor

[jira] [Created] (SPARK-17400) MinMaxScaler.transform() outputs DenseVector by default, which causes poor performance

2016-09-05 Thread Frank Dai (JIRA)
Frank Dai created SPARK-17400: - Summary: MinMaxScaler.transform() outputs DenseVector by default, which causes poor performance Key: SPARK-17400 URL: https://issues.apache.org/jira/browse/SPARK-17400

[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows

2016-09-05 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464250#comment-15464250 ] Hyukjin Kwon commented on SPARK-17339: -- Yeap, I totally agree. Thank you both! Will submit a PR

[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows

2016-09-05 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464247#comment-15464247 ] Kousuke Saruta commented on SPARK-17339: [~hyukjin.kwon] Go ahead and submit a PR. Thanks! > Fix

[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows

2016-09-05 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464244#comment-15464244 ] Shivaram Venkataraman commented on SPARK-17339: --- Thanks [~hyukjin.kwon] -- It will be great

[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows

2016-09-05 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464236#comment-15464236 ] Hyukjin Kwon commented on SPARK-17339: -- [~sarutak] [~shivaram] Please cc me if any of you submit a

[jira] [Created] (SPARK-17399) Update specific data in mySql Table from Spark

2016-09-05 Thread Farman Ali (JIRA)
Farman Ali created SPARK-17399: -- Summary: Update specific data in mySql Table from Spark Key: SPARK-17399 URL: https://issues.apache.org/jira/browse/SPARK-17399 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows

2016-09-05 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464193#comment-15464193 ] Kousuke Saruta commented on SPARK-17339: [~shivaram] I think more discussion is needed for the

<    1   2