[jira] [Assigned] (SPARK-13855) Spark 1.6.1 artifacts not found in S3 bucket / direct download

2016-03-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell reassigned SPARK-13855: --- Assignee: Patrick Wendell (was: Michael Armbrust) > Spark 1.6.1 artifacts not found

[jira] [Created] (SPARK-13928) Move org.apache.spark.Logging into org.apache.spark.internal.Logging

2016-03-15 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-13928: --- Summary: Move org.apache.spark.Logging into org.apache.spark.internal.Logging Key: SPARK-13928 URL: https://issues.apache.org/jira/browse/SPARK-13928 Project: Spark

[jira] [Updated] (SPARK-13927) Add row/column iterator to local matrix

2016-03-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13927: -- Summary: Add row/column iterator to local matrix (was: Add row/column iterator to matrix) > A

[jira] [Updated] (SPARK-13927) Add row/column iterator to local matrices

2016-03-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13927: -- Summary: Add row/column iterator to local matrices (was: Add row/column iterator to local matr

[jira] [Created] (SPARK-13927) Add row/column iterator to matrix

2016-03-15 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-13927: - Summary: Add row/column iterator to matrix Key: SPARK-13927 URL: https://issues.apache.org/jira/browse/SPARK-13927 Project: Spark Issue Type: New Feature

[jira] [Assigned] (SPARK-13764) Parse modes in JSON data source

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13764: Assignee: (was: Apache Spark) > Parse modes in JSON data source >

[jira] [Commented] (SPARK-13764) Parse modes in JSON data source

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196887#comment-15196887 ] Apache Spark commented on SPARK-13764: -- User 'HyukjinKwon' has created a pull reques

[jira] [Assigned] (SPARK-13764) Parse modes in JSON data source

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13764: Assignee: Apache Spark > Parse modes in JSON data source > ---

[jira] [Comment Edited] (SPARK-13857) Feature parity for ALS ML with MLLIB

2016-03-15 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15195696#comment-15195696 ] Nick Pentreath edited comment on SPARK-13857 at 3/16/16 6:45 AM: --

[jira] [Comment Edited] (SPARK-13857) Feature parity for ALS ML with MLLIB

2016-03-15 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15195702#comment-15195702 ] Nick Pentreath edited comment on SPARK-13857 at 3/16/16 6:42 AM: --

[jira] [Comment Edited] (SPARK-13857) Feature parity for ALS ML with MLLIB

2016-03-15 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15195696#comment-15195696 ] Nick Pentreath edited comment on SPARK-13857 at 3/16/16 6:42 AM: --

[jira] [Comment Edited] (SPARK-13857) Feature parity for ALS ML with MLLIB

2016-03-15 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15195696#comment-15195696 ] Nick Pentreath edited comment on SPARK-13857 at 3/16/16 6:41 AM: --

[jira] [Comment Edited] (SPARK-13857) Feature parity for ALS ML with MLLIB

2016-03-15 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15195696#comment-15195696 ] Nick Pentreath edited comment on SPARK-13857 at 3/16/16 6:38 AM: --

[jira] [Comment Edited] (SPARK-13857) Feature parity for ALS ML with MLLIB

2016-03-15 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15195696#comment-15195696 ] Nick Pentreath edited comment on SPARK-13857 at 3/16/16 6:38 AM: --

[jira] [Resolved] (SPARK-13899) Produce InternalRow instead of external Row

2016-03-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13899. - Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.0.0 > Produce InternalR

[jira] [Assigned] (SPARK-13926) Automatically use Kryo serializer when shuffling RDDs with simple types

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13926: Assignee: Apache Spark (was: Josh Rosen) > Automatically use Kryo serializer when shuffli

[jira] [Assigned] (SPARK-13926) Automatically use Kryo serializer when shuffling RDDs with simple types

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13926: Assignee: Josh Rosen (was: Apache Spark) > Automatically use Kryo serializer when shuffli

[jira] [Commented] (SPARK-13926) Automatically use Kryo serializer when shuffling RDDs with simple types

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196857#comment-15196857 ] Apache Spark commented on SPARK-13926: -- User 'JoshRosen' has created a pull request

[jira] [Resolved] (SPARK-13920) MIMA checks should apply to @Experimental and @DeveloperAPI APIs

2016-03-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13920. - Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.0.0 > MIMA checks shou

[jira] [Created] (SPARK-13926) Automatically use Kryo serializer when it is known to be safe

2016-03-15 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-13926: -- Summary: Automatically use Kryo serializer when it is known to be safe Key: SPARK-13926 URL: https://issues.apache.org/jira/browse/SPARK-13926 Project: Spark Is

[jira] [Updated] (SPARK-13926) Automatically use Kryo serializer when shuffling RDDs with simple types

2016-03-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-13926: --- Summary: Automatically use Kryo serializer when shuffling RDDs with simple types (was: Automatically

[jira] [Comment Edited] (SPARK-13877) Consider removing Kafka modules from Spark / Spark Streaming

2016-03-15 Thread Hari Shreedharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196805#comment-15196805 ] Hari Shreedharan edited comment on SPARK-13877 at 3/16/16 5:49 AM:

[jira] [Commented] (SPARK-13877) Consider removing Kafka modules from Spark / Spark Streaming

2016-03-15 Thread Hari Shreedharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196805#comment-15196805 ] Hari Shreedharan commented on SPARK-13877: -- [~c...@koeninger.org] - Sure. I agre

[jira] [Created] (SPARK-13925) Expose R-like summary statistics in SparkR::glm for more family and link functions

2016-03-15 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-13925: - Summary: Expose R-like summary statistics in SparkR::glm for more family and link functions Key: SPARK-13925 URL: https://issues.apache.org/jira/browse/SPARK-13925

[jira] [Updated] (SPARK-13925) Expose R-like summary statistics in SparkR::glm for more family and link functions

2016-03-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13925: -- Priority: Critical (was: Major) > Expose R-like summary statistics in SparkR::glm for more fam

[jira] [Resolved] (SPARK-9837) Provide R-like summary statistics for GLMs via iteratively reweighted least squares

2016-03-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-9837. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11694 [https://gith

[jira] [Resolved] (SPARK-13917) Generate code for broadcast left semi join

2016-03-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13917. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11742 [https://github.

[jira] [Updated] (SPARK-13903) Modify output nullability with constraints for Join and Filter operators

2016-03-15 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-13903: Summary: Modify output nullability with constraints for Join and Filter operators (was: Mo

[jira] [Updated] (SPARK-13903) Modify output nullability with constraints for Join and Filter operators

2016-03-15 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-13903: Description: With constraints and optimization, we can make sure some outputs of a Join (or

[jira] [Commented] (SPARK-13641) getModelFeatures of ml.api.r.SparkRWrapper cannot (always) reveal the original column names

2016-03-15 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196765#comment-15196765 ] Xusen Yin commented on SPARK-13641: --- [~muralidh] I gonna close this JIRA since I find t

[jira] [Comment Edited] (SPARK-13641) getModelFeatures of ml.api.r.SparkRWrapper cannot (always) reveal the original column names

2016-03-15 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196765#comment-15196765 ] Xusen Yin edited comment on SPARK-13641 at 3/16/16 5:00 AM: I

[jira] [Assigned] (SPARK-13924) officially support multi-insert

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13924: Assignee: Apache Spark > officially support multi-insert > ---

[jira] [Assigned] (SPARK-13924) officially support multi-insert

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13924: Assignee: (was: Apache Spark) > officially support multi-insert >

[jira] [Commented] (SPARK-13924) officially support multi-insert

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196760#comment-15196760 ] Apache Spark commented on SPARK-13924: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-13316) "SparkException: DStream has not been initialized" when restoring StreamingContext from checkpoint and the dstream is created afterwards

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196758#comment-15196758 ] Apache Spark commented on SPARK-13316: -- User 'mwws' has created a pull request for t

[jira] [Assigned] (SPARK-13316) "SparkException: DStream has not been initialized" when restoring StreamingContext from checkpoint and the dstream is created afterwards

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13316: Assignee: Apache Spark > "SparkException: DStream has not been initialized" when restoring

[jira] [Assigned] (SPARK-13316) "SparkException: DStream has not been initialized" when restoring StreamingContext from checkpoint and the dstream is created afterwards

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13316: Assignee: (was: Apache Spark) > "SparkException: DStream has not been initialized" whe

[jira] [Commented] (SPARK-13900) Spark SQL queries with OR condition is not optimized properly

2016-03-15 Thread Ashok kumar Rajendran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196756#comment-15196756 ] Ashok kumar Rajendran commented on SPARK-13900: --- Along with dimensions cond

[jira] [Commented] (SPARK-3308) Ability to read JSON Arrays as tables

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196754#comment-15196754 ] Apache Spark commented on SPARK-3308: - User 'HyukjinKwon' has created a pull request f

[jira] [Commented] (SPARK-13900) Spark SQL queries with OR condition is not optimized properly

2016-03-15 Thread Ashok kumar Rajendran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196753#comment-15196753 ] Ashok kumar Rajendran commented on SPARK-13900: --- Explain plan for query wit

[jira] [Commented] (SPARK-13900) Spark SQL queries with OR condition is not optimized properly

2016-03-15 Thread Ashok kumar Rajendran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196752#comment-15196752 ] Ashok kumar Rajendran commented on SPARK-13900: --- Hi Xio Li, Thanks for loo

[jira] [Comment Edited] (SPARK-13821) TPC-DS Query 20 fails to compile

2016-03-15 Thread Dilip Biswal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196464#comment-15196464 ] Dilip Biswal edited comment on SPARK-13821 at 3/16/16 4:41 AM:

[jira] [Created] (SPARK-13924) officially support multi-insert

2016-03-15 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-13924: --- Summary: officially support multi-insert Key: SPARK-13924 URL: https://issues.apache.org/jira/browse/SPARK-13924 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-13920) MIMA checks should apply to @Experimental and @DeveloperAPI APIs

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13920: Assignee: Apache Spark > MIMA checks should apply to @Experimental and @DeveloperAPI APIs

[jira] [Commented] (SPARK-13920) MIMA checks should apply to @Experimental and @DeveloperAPI APIs

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196743#comment-15196743 ] Apache Spark commented on SPARK-13920: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Assigned] (SPARK-13920) MIMA checks should apply to @Experimental and @DeveloperAPI APIs

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13920: Assignee: (was: Apache Spark) > MIMA checks should apply to @Experimental and @Develop

[jira] [Resolved] (SPARK-13918) merge SortMergeJoin and SortMergeOuterJoin

2016-03-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13918. - Resolution: Fixed Fix Version/s: 2.0.0 > merge SortMergeJoin and SortMergeOuterJoin >

[jira] [Commented] (SPARK-13764) Parse modes in JSON data source

2016-03-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196665#comment-15196665 ] Hyukjin Kwon commented on SPARK-13764: -- The issue SPARK-3308 is related with support

[jira] [Comment Edited] (SPARK-13843) Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages

2016-03-15 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196661#comment-15196661 ] Liwei Lin edited comment on SPARK-13843 at 3/16/16 2:50 AM: h

[jira] [Assigned] (SPARK-13923) Implement SessionCatalog to manage temp functions and tables

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13923: Assignee: Andrew Or (was: Apache Spark) > Implement SessionCatalog to manage temp functio

[jira] [Commented] (SPARK-13923) Implement SessionCatalog to manage temp functions and tables

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196664#comment-15196664 ] Apache Spark commented on SPARK-13923: -- User 'andrewor14' has created a pull request

[jira] [Assigned] (SPARK-13923) Implement SessionCatalog to manage temp functions and tables

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13923: Assignee: Apache Spark (was: Andrew Or) > Implement SessionCatalog to manage temp functio

[jira] [Commented] (SPARK-13843) Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages

2016-03-15 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196661#comment-15196661 ] Liwei Lin commented on SPARK-13843: --- hi [~zsxwing], we didn't move streaming-kinesis (w

[jira] [Created] (SPARK-13923) Implement SessionCatalog to manage temp functions and tables

2016-03-15 Thread Andrew Or (JIRA)
Andrew Or created SPARK-13923: - Summary: Implement SessionCatalog to manage temp functions and tables Key: SPARK-13923 URL: https://issues.apache.org/jira/browse/SPARK-13923 Project: Spark Issue

[jira] [Comment Edited] (SPARK-13920) MIMA checks should apply to @Experimental and @DeveloperAPI APIs

2016-03-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196626#comment-15196626 ] Dongjoon Hyun edited comment on SPARK-13920 at 3/16/16 2:16 AM: ---

[jira] [Commented] (SPARK-13920) MIMA checks should apply to @Experimental and @DeveloperAPI APIs

2016-03-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196626#comment-15196626 ] Dongjoon Hyun commented on SPARK-13920: --- Oh, [~joshrosen]. I totally agree with th

[jira] [Commented] (SPARK-13915) Allow bin/spark-submit to be called via symbolic link

2016-03-15 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196590#comment-15196590 ] Saisai Shao commented on SPARK-13915: - I suppose it should be worked if you set {{SPA

[jira] [Commented] (SPARK-13872) Memory leak in SortMergeOuterJoin

2016-03-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196585#comment-15196585 ] Yin Huai commented on SPARK-13872: -- Also cc [~davies] > Memory leak in SortMergeOuterJo

[jira] [Commented] (SPARK-13914) Add functionality to back up spark event logs

2016-03-15 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196580#comment-15196580 ] Saisai Shao commented on SPARK-13914: - Agree with [~srowen], this feature is quite us

[jira] [Assigned] (SPARK-13922) Filter rows with null attributes in parquet vectorized reader

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13922: Assignee: (was: Apache Spark) > Filter rows with null attributes in parquet vectorized

[jira] [Commented] (SPARK-13922) Filter rows with null attributes in parquet vectorized reader

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196577#comment-15196577 ] Apache Spark commented on SPARK-13922: -- User 'sameeragarwal' has created a pull requ

[jira] [Assigned] (SPARK-13922) Filter rows with null attributes in parquet vectorized reader

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13922: Assignee: Apache Spark > Filter rows with null attributes in parquet vectorized reader > -

[jira] [Created] (SPARK-13922) Filter rows with null attributes in parquet vectorized reader

2016-03-15 Thread Sameer Agarwal (JIRA)
Sameer Agarwal created SPARK-13922: -- Summary: Filter rows with null attributes in parquet vectorized reader Key: SPARK-13922 URL: https://issues.apache.org/jira/browse/SPARK-13922 Project: Spark

[jira] [Assigned] (SPARK-13921) Store serialized blocks as multiple chunks in MemoryStore

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13921: Assignee: Josh Rosen (was: Apache Spark) > Store serialized blocks as multiple chunks in

[jira] [Commented] (SPARK-13921) Store serialized blocks as multiple chunks in MemoryStore

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196567#comment-15196567 ] Apache Spark commented on SPARK-13921: -- User 'JoshRosen' has created a pull request

[jira] [Assigned] (SPARK-13921) Store serialized blocks as multiple chunks in MemoryStore

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13921: Assignee: Apache Spark (was: Josh Rosen) > Store serialized blocks as multiple chunks in

[jira] [Created] (SPARK-13921) Store serialized blocks as multiple chunks in MemoryStore

2016-03-15 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-13921: -- Summary: Store serialized blocks as multiple chunks in MemoryStore Key: SPARK-13921 URL: https://issues.apache.org/jira/browse/SPARK-13921 Project: Spark Issue T

[jira] [Commented] (SPARK-13872) Memory leak in SortMergeOuterJoin

2016-03-15 Thread Mark Hamstra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196561#comment-15196561 ] Mark Hamstra commented on SPARK-13872: -- [~joshrosen] > Memory leak in SortMergeOute

[jira] [Comment Edited] (SPARK-13832) TPC-DS Query 36 fails with Parser error

2016-03-15 Thread Xin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196377#comment-15196377 ] Xin Wu edited comment on SPARK-13832 at 3/16/16 12:39 AM: -- Tryin

[jira] [Assigned] (SPARK-2208) local metrics tests can fail on fast machines

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-2208: --- Assignee: Apache Spark > local metrics tests can fail on fast machines >

[jira] [Commented] (SPARK-2208) local metrics tests can fail on fast machines

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196556#comment-15196556 ] Apache Spark commented on SPARK-2208: - User 'joan38' has created a pull request for th

[jira] [Assigned] (SPARK-2208) local metrics tests can fail on fast machines

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-2208: --- Assignee: (was: Apache Spark) > local metrics tests can fail on fast machines > -

[jira] [Commented] (SPARK-13920) MIMA checks should apply to @Experimental and @DeveloperAPI APIs

2016-03-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196548#comment-15196548 ] Josh Rosen commented on SPARK-13920: I am not planning to work on this, so this task

[jira] [Created] (SPARK-13920) MIMA checks should apply to @Experimental and @DeveloperAPI APIs

2016-03-15 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-13920: -- Summary: MIMA checks should apply to @Experimental and @DeveloperAPI APIs Key: SPARK-13920 URL: https://issues.apache.org/jira/browse/SPARK-13920 Project: Spark

[jira] [Commented] (SPARK-13602) o.a.s.deploy.worker.DriverRunner may leak the driver processes

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196537#comment-15196537 ] Apache Spark commented on SPARK-13602: -- User 'BryanCutler' has created a pull reques

[jira] [Assigned] (SPARK-13602) o.a.s.deploy.worker.DriverRunner may leak the driver processes

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13602: Assignee: (was: Apache Spark) > o.a.s.deploy.worker.DriverRunner may leak the driver p

[jira] [Assigned] (SPARK-13602) o.a.s.deploy.worker.DriverRunner may leak the driver processes

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13602: Assignee: Apache Spark > o.a.s.deploy.worker.DriverRunner may leak the driver processes >

[jira] [Assigned] (SPARK-13919) Resolving the Conflicts of ColumnPruning and PushPredicateThroughProject

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13919: Assignee: Apache Spark > Resolving the Conflicts of ColumnPruning and PushPredicateThrough

[jira] [Commented] (SPARK-13919) Resolving the Conflicts of ColumnPruning and PushPredicateThroughProject

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196535#comment-15196535 ] Apache Spark commented on SPARK-13919: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-13919) Resolving the Conflicts of ColumnPruning and PushPredicateThroughProject

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13919: Assignee: (was: Apache Spark) > Resolving the Conflicts of ColumnPruning and PushPredi

[jira] [Reopened] (SPARK-13804) Spark SQL's DataFrame.count() Major Divergent (Non-Linear) Performance Slowdown going from 4million rows to 16+ million rows

2016-03-15 Thread Michael Nguyen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Nguyen reopened SPARK-13804: I posted this issue to user@ at http://apache-spark-user-list.1001560.n3.nabble.com/DataFrame-

[jira] [Updated] (SPARK-13919) Resolving the Conflicts of ColumnPruning and PushPredicateThroughProject

2016-03-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-13919: Description: Now, {{ColumnPruning}} and {{PushPredicateThroughProject}} reverse each other's effect. Altho

[jira] [Created] (SPARK-13919) Resolving the Conflicts of ColumnPruning and PushPredicateThroughProject

2016-03-15 Thread Xiao Li (JIRA)
Xiao Li created SPARK-13919: --- Summary: Resolving the Conflicts of ColumnPruning and PushPredicateThroughProject Key: SPARK-13919 URL: https://issues.apache.org/jira/browse/SPARK-13919 Project: Spark

[jira] [Commented] (SPARK-8489) Add regression tests for SPARK-8470

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196517#comment-15196517 ] Apache Spark commented on SPARK-8489: - User 'dongjoon-hyun' has created a pull request

[jira] [Commented] (SPARK-12653) Re-enable test "SPARK-8489: MissingRequirementError during reflection"

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196516#comment-15196516 ] Apache Spark commented on SPARK-12653: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Commented] (SPARK-13118) Support for classes defined in package objects

2016-03-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196487#comment-15196487 ] Reynold Xin commented on SPARK-13118: - The JIRA is the same. > Support for classes

[jira] [Commented] (SPARK-13118) Support for classes defined in package objects

2016-03-15 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196463#comment-15196463 ] Jakob Odersky commented on SPARK-13118: --- Should I remove the JIRA ID from my existi

[jira] [Commented] (SPARK-13821) TPC-DS Query 20 fails to compile

2016-03-15 Thread Dilip Biswal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196464#comment-15196464 ] Dilip Biswal commented on SPARK-13821: -- [~roycecil] Just tried the original query no

[jira] [Commented] (SPARK-13820) TPC-DS Query 10 fails to compile

2016-03-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196430#comment-15196430 ] Davies Liu commented on SPARK-13820: Unfortunately, subquery is not target for 2.0.

[jira] [Commented] (SPARK-13118) Support for classes defined in package objects

2016-03-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196427#comment-15196427 ] Reynold Xin commented on SPARK-13118: - I'm moving this out to its own ticket. I thi

[jira] [Updated] (SPARK-13118) Support for classes defined in package objects

2016-03-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-13118: Issue Type: Improvement (was: Sub-task) Parent: (was: SPARK-) > Support for classe

[jira] [Resolved] (SPARK-9999) Dataset API on top of Catalyst/DataFrame

2016-03-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-. Resolution: Fixed Fix Version/s: 2.0.0 > Dataset API on top of Catalyst/DataFrame > -

[jira] [Commented] (SPARK-13820) TPC-DS Query 10 fails to compile

2016-03-15 Thread Suresh Thalamati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196419#comment-15196419 ] Suresh Thalamati commented on SPARK-13820: -- This query contains correlated subqu

[jira] [Comment Edited] (SPARK-13832) TPC-DS Query 36 fails with Parser error

2016-03-15 Thread Xin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196377#comment-15196377 ] Xin Wu edited comment on SPARK-13832 at 3/15/16 10:43 PM: -- Tryin

[jira] [Comment Edited] (SPARK-13832) TPC-DS Query 36 fails with Parser error

2016-03-15 Thread Xin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196377#comment-15196377 ] Xin Wu edited comment on SPARK-13832 at 3/15/16 10:44 PM: -- Tryin

[jira] [Comment Edited] (SPARK-13832) TPC-DS Query 36 fails with Parser error

2016-03-15 Thread Xin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196377#comment-15196377 ] Xin Wu edited comment on SPARK-13832 at 3/15/16 10:41 PM: -- Tryin

[jira] [Commented] (SPARK-13918) merge SortMergeJoin and SortMergeOuterJoin

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196376#comment-15196376 ] Apache Spark commented on SPARK-13918: -- User 'davies' has created a pull request for

[jira] [Assigned] (SPARK-13918) merge SortMergeJoin and SortMergeOuterJoin

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13918: Assignee: Davies Liu (was: Apache Spark) > merge SortMergeJoin and SortMergeOuterJoin > -

[jira] [Assigned] (SPARK-13918) merge SortMergeJoin and SortMergeOuterJoin

2016-03-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13918: Assignee: Apache Spark (was: Davies Liu) > merge SortMergeJoin and SortMergeOuterJoin > -

[jira] [Commented] (SPARK-13832) TPC-DS Query 36 fails with Parser error

2016-03-15 Thread Xin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196377#comment-15196377 ] Xin Wu commented on SPARK-13832: Trying this query in Spark 2.0 and I needed to change gr

  1   2   3   4   >