[jira] [Created] (SPARK-13713) Replace ANTLR3 SQL parser by a ANTLR4 SQL parser

2016-03-06 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-13713: - Summary: Replace ANTLR3 SQL parser by a ANTLR4 SQL parser Key: SPARK-13713 URL: https://issues.apache.org/jira/browse/SPARK-13713 Project: Spark

[jira] [Commented] (SPARK-13711) Apache Spark driver stopping JVM when master not available

2016-03-06 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182681#comment-15182681 ] Shixiong Zhu commented on SPARK-13711: -- Sorry, I misread the description. Did you run in the client

[jira] [Commented] (SPARK-12718) SQL generation support for window functions

2016-03-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182676#comment-15182676 ] Wenchen Fan commented on SPARK-12718: - Hi [~smilegator], It seems that I underestimate the difficulty

[jira] [Commented] (SPARK-13711) Apache Spark driver stopping JVM when master not available

2016-03-06 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182674#comment-15182674 ] Shixiong Zhu commented on SPARK-13711: -- This is the correct behavior. Driver needs to talk with

[jira] [Assigned] (SPARK-12718) SQL generation support for window functions

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12718: Assignee: Xiao Li (was: Apache Spark) > SQL generation support for window functions >

[jira] [Assigned] (SPARK-12718) SQL generation support for window functions

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12718: Assignee: Apache Spark (was: Xiao Li) > SQL generation support for window functions >

[jira] [Commented] (SPARK-12718) SQL generation support for window functions

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182670#comment-15182670 ] Apache Spark commented on SPARK-12718: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13712) Add OneVsOne to ML

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13712: Assignee: (was: Apache Spark) > Add OneVsOne to ML > -- > >

[jira] [Commented] (SPARK-13712) Add OneVsOne to ML

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182660#comment-15182660 ] Apache Spark commented on SPARK-13712: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-13712) Add OneVsOne to ML

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13712: Assignee: Apache Spark > Add OneVsOne to ML > -- > > Key:

[jira] [Created] (SPARK-13712) Add OneVsOne to ML

2016-03-06 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-13712: Summary: Add OneVsOne to ML Key: SPARK-13712 URL: https://issues.apache.org/jira/browse/SPARK-13712 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-12718) SQL generation support for window functions

2016-03-06 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182644#comment-15182644 ] Xiao Li commented on SPARK-12718: - So far, SQL generation support for Window functions can work well.

[jira] [Created] (SPARK-13711) Apache Spark driver stopping JVM when master not available

2016-03-06 Thread Era (JIRA)
Era created SPARK-13711: --- Summary: Apache Spark driver stopping JVM when master not available Key: SPARK-13711 URL: https://issues.apache.org/jira/browse/SPARK-13711 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-12718) SQL generation support for window functions

2016-03-06 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182584#comment-15182584 ] Xiao Li commented on SPARK-12718: - Sure, Thanks! > SQL generation support for window functions >

[jira] [Commented] (SPARK-12718) SQL generation support for window functions

2016-03-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182583#comment-15182583 ] Wenchen Fan commented on SPARK-12718: - Then finish it, we can consolidate them later. > SQL

[jira] [Commented] (SPARK-12718) SQL generation support for window functions

2016-03-06 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182578#comment-15182578 ] Xiao Li commented on SPARK-12718: - Hi, [~cloud_fan] Yeah. Almost done. Just let me know if I should

[jira] [Commented] (SPARK-13710) Spark shell shows ERROR when launching on Windows

2016-03-06 Thread Masayoshi TSUZUKI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182565#comment-15182565 ] Masayoshi TSUZUKI commented on SPARK-13710: --- It shows the similar ERROR message and stacktrace

[jira] [Created] (SPARK-13710) Spark shell shows ERROR when launching on Windows

2016-03-06 Thread Masayoshi TSUZUKI (JIRA)
Masayoshi TSUZUKI created SPARK-13710: - Summary: Spark shell shows ERROR when launching on Windows Key: SPARK-13710 URL: https://issues.apache.org/jira/browse/SPARK-13710 Project: Spark

[jira] [Commented] (SPARK-13600) Incorrect number of buckets in QuantileDiscretizer

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182512#comment-15182512 ] Apache Spark commented on SPARK-13600: -- User 'oliverpierson' has created a pull request for this

[jira] [Assigned] (SPARK-13600) Incorrect number of buckets in QuantileDiscretizer

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13600: Assignee: Oliver Pierson (was: Apache Spark) > Incorrect number of buckets in

[jira] [Assigned] (SPARK-13600) Incorrect number of buckets in QuantileDiscretizer

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13600: Assignee: Apache Spark (was: Oliver Pierson) > Incorrect number of buckets in

[jira] [Commented] (SPARK-12718) SQL generation support for window functions

2016-03-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182496#comment-15182496 ] Wenchen Fan commented on SPARK-12718: - Hi, [~xiaol], are you still working on it? I was working on it

[jira] [Updated] (SPARK-13709) Spark unable to decode Avro when partitioned

2016-03-06 Thread Chris Miller (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Miller updated SPARK-13709: - Description: There is a problem decoding Avro data with SparkSQL when partitioned. The schema

[jira] [Updated] (SPARK-13709) Spark unable to decode Avro when partitioned

2016-03-06 Thread Chris Miller (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Miller updated SPARK-13709: - Description: There is a problem decoding Avro data with SparkSQL when partitioned. The schema

[jira] [Created] (SPARK-13709) Spark unable to decode Avro when partitioned

2016-03-06 Thread Chris Miller (JIRA)
Chris Miller created SPARK-13709: Summary: Spark unable to decode Avro when partitioned Key: SPARK-13709 URL: https://issues.apache.org/jira/browse/SPARK-13709 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-12243) PySpark tests are slow in Jenkins

2016-03-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182482#comment-15182482 ] Dongjoon Hyun commented on SPARK-12243: --- Here is the real Jenkins test time. {code} Tests passed in

[jira] [Comment Edited] (SPARK-5928) Remote Shuffle Blocks cannot be more than 2 GB

2016-03-06 Thread Taro L. Saito (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182472#comment-15182472 ] Taro L. Saito edited comment on SPARK-5928 at 3/7/16 2:24 AM: -- FYI. I created

[jira] [Commented] (SPARK-8884) 1-sample Anderson-Darling Goodness-of-Fit test

2016-03-06 Thread Jose Cambronero (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182476#comment-15182476 ] Jose Cambronero commented on SPARK-8884: [~yuhaoyan] please do! I unfortunately got really busy

[jira] [Commented] (SPARK-13034) PySpark ml.classification support export/import

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182474#comment-15182474 ] Apache Spark commented on SPARK-13034: -- User 'GayathriMurali' has created a pull request for this

[jira] [Assigned] (SPARK-13034) PySpark ml.classification support export/import

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13034: Assignee: (was: Apache Spark) > PySpark ml.classification support export/import >

[jira] [Assigned] (SPARK-13034) PySpark ml.classification support export/import

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13034: Assignee: Apache Spark > PySpark ml.classification support export/import >

[jira] [Commented] (SPARK-5928) Remote Shuffle Blocks cannot be more than 2 GB

2016-03-06 Thread Taro L. Saito (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182472#comment-15182472 ] Taro L. Saito commented on SPARK-5928: -- FYI. I created LArray library that can handle data larger

[jira] [Comment Edited] (SPARK-12243) PySpark tests are slow in Jenkins

2016-03-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182460#comment-15182460 ] Dongjoon Hyun edited comment on SPARK-12243 at 3/7/16 1:49 AM: --- According

[jira] [Commented] (SPARK-12243) PySpark tests are slow in Jenkins

2016-03-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182460#comment-15182460 ] Dongjoon Hyun commented on SPARK-12243: --- According to the log, the total time of all tests are

[jira] [Assigned] (SPARK-12243) PySpark tests are slow in Jenkins

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12243: Assignee: (was: Apache Spark) > PySpark tests are slow in Jenkins >

[jira] [Commented] (SPARK-12243) PySpark tests are slow in Jenkins

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182455#comment-15182455 ] Apache Spark commented on SPARK-12243: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-12243) PySpark tests are slow in Jenkins

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12243: Assignee: Apache Spark > PySpark tests are slow in Jenkins >

[jira] [Commented] (SPARK-12718) SQL generation support for window functions

2016-03-06 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182445#comment-15182445 ] Xiao Li commented on SPARK-12718: - In Window Spec, the possible inputs are: 1. partition by + order by,

[jira] [Commented] (SPARK-12243) PySpark tests are slow in Jenkins

2016-03-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182442#comment-15182442 ] Dongjoon Hyun commented on SPARK-12243: --- Hi, [~joshrosen]. According to the recent [Running

[jira] [Commented] (SPARK-12718) SQL generation support for window functions

2016-03-06 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182441#comment-15182441 ] Xiao Li commented on SPARK-12718: - If users use cluster by clauses, or DISTRIBUTE BY + SORT BY clauses,

[jira] [Commented] (SPARK-12718) SQL generation support for window functions

2016-03-06 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182439#comment-15182439 ] Xiao Li commented on SPARK-12718: - Will not add extra subquery here. Trying to rebuild the original

[jira] [Commented] (SPARK-6162) Handle missing values in GBM

2016-03-06 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182434#comment-15182434 ] Joseph K. Bradley commented on SPARK-6162: -- I agree this will be nice to add someday, but it's

[jira] [Closed] (SPARK-12731) PySpark docstring cleanup

2016-03-06 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley closed SPARK-12731. - Resolution: Won't Fix > PySpark docstring cleanup > - > >

[jira] [Commented] (SPARK-12731) PySpark docstring cleanup

2016-03-06 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182432#comment-15182432 ] Joseph K. Bradley commented on SPARK-12731: --- Alright, it sounds like the consensus is that we

[jira] [Assigned] (SPARK-13667) Support for specifying custom date format for date and timestamp types

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13667: Assignee: Apache Spark > Support for specifying custom date format for date and timestamp

[jira] [Assigned] (SPARK-13667) Support for specifying custom date format for date and timestamp types

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13667: Assignee: (was: Apache Spark) > Support for specifying custom date format for date

[jira] [Commented] (SPARK-13667) Support for specifying custom date format for date and timestamp types

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182431#comment-15182431 ] Apache Spark commented on SPARK-13667: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-8884) 1-sample Anderson-Darling Goodness-of-Fit test

2016-03-06 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182420#comment-15182420 ] yuhao yang commented on SPARK-8884: --- Hi [~josepablocam]. Do you mind if I continue to work on this? I

[jira] [Assigned] (SPARK-12566) GLM model family, link function support in SparkR:::glm

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12566: Assignee: Apache Spark (was: yuhao yang) > GLM model family, link function support in

[jira] [Assigned] (SPARK-12566) GLM model family, link function support in SparkR:::glm

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12566: Assignee: yuhao yang (was: Apache Spark) > GLM model family, link function support in

[jira] [Commented] (SPARK-12566) GLM model family, link function support in SparkR:::glm

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182401#comment-15182401 ] Apache Spark commented on SPARK-12566: -- User 'hhbyyh' has created a pull request for this issue:

[jira] [Commented] (SPARK-12566) GLM model family, link function support in SparkR:::glm

2016-03-06 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182384#comment-15182384 ] yuhao yang commented on SPARK-12566: Since we already have a glm in SparkR which is based on

[jira] [Commented] (SPARK-13496) Optimizing count distinct changes the resulting column name

2016-03-06 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182332#comment-15182332 ] Ryan Blue commented on SPARK-13496: --- I wouldn't say this is a duplicate, though I'm fine with

[jira] [Reopened] (SPARK-13620) Avoid reverse DNS lookup for 0.0.0.0 on startup

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-13620: --- > Avoid reverse DNS lookup for 0.0.0.0 on startup > --- > >

[jira] [Updated] (SPARK-13609) Support Column Pruning for MapPartitions

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13609: -- Assignee: Xiao Li > Support Column Pruning for MapPartitions >

[jira] [Updated] (SPARK-13647) also check if numeric value is within allowed range in _verify_type

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13647: -- Assignee: Wenchen Fan > also check if numeric value is within allowed range in _verify_type >

[jira] [Resolved] (SPARK-13620) Avoid reverse DNS lookup for 0.0.0.0 on startup

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13620. --- Resolution: Not A Problem Fix Version/s: (was: 2.0.0) > Avoid reverse DNS lookup for

[jira] [Updated] (SPARK-13598) Remove LeftSemiJoinBNL

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13598: -- Assignee: Davies Liu > Remove LeftSemiJoinBNL > -- > > Key:

[jira] [Updated] (SPARK-13574) Improve parquet dictionary decoding for strings

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13574: -- Assignee: Nong Li > Improve parquet dictionary decoding for strings >

[jira] [Updated] (SPARK-13630) Add optimizer rule to collapse sorts

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13630: -- Priority: Minor (was: Major) Fix Version/s: (was: 2.0.0) > Add optimizer rule to

[jira] [Updated] (SPARK-13255) Integrate vectorized parquet scan with whole stage codegen.

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13255: -- Assignee: Nong Li > Integrate vectorized parquet scan with whole stage codegen. >

[jira] [Updated] (SPARK-13685) Rename catalog.Catalog to ExternalCatalog

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13685: -- Fix Version/s: (was: 2.0.0) > Rename catalog.Catalog to ExternalCatalog >

[jira] [Updated] (SPARK-13677) Support Tree-Based Feature Transformation for mllib

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13677: -- Component/s: MLlib > Support Tree-Based Feature Transformation for mllib >

[jira] [Updated] (SPARK-13668) Reorder filter/join predicates to short-circuit isNotNull checks

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13668: -- Component/s: SQL > Reorder filter/join predicates to short-circuit isNotNull checks >

[jira] [Updated] (SPARK-13668) Reorder filter/join predicates to short-circuit isNotNull checks

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13668: -- Priority: Minor (was: Major) > Reorder filter/join predicates to short-circuit isNotNull checks >

[jira] [Updated] (SPARK-13702) Use diamond operator for generic instance creation in Java code

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13702: -- Component/s: Examples > Use diamond operator for generic instance creation in Java code >

[jira] [Updated] (SPARK-13648) org.apache.spark.sql.hive.client.VersionsSuite fails NoClassDefFoundError on IBM JDK

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13648: -- Priority: Minor (was: Major) Component/s: SQL Summary:

[jira] [Updated] (SPARK-13606) Error from python worker: /usr/local/bin/python2.7: undefined symbol: _PyCodec_LookupTextEncoding

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13606: -- Component/s: PySpark > Error from python worker: /usr/local/bin/python2.7: undefined symbol: >

[jira] [Updated] (SPARK-13679) Pyspark job fails with Oozie

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13679: -- Target Version/s: (was: 1.6.0) Priority: Minor (was: Major) > Pyspark job fails with

[jira] [Updated] (SPARK-13692) Fix trivial Coverity/Checkstyle defects

2016-03-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-13692: -- Description: This issue fixes the following potential bugs and Java coding style detected by

[jira] [Resolved] (SPARK-13438) Remove by default dash from output paths

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13438. --- Resolution: Won't Fix > Remove by default dash from output paths >

[jira] [Updated] (SPARK-13688) Add option to use dynamic allocation even if spark.executor.instances is set.

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13688: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) > Add option to use dynamic

[jira] [Resolved] (SPARK-13688) Add option to use dynamic allocation even if spark.executor.instances is set.

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13688. --- Resolution: Won't Fix > Add option to use dynamic allocation even if spark.executor.instances is

[jira] [Commented] (SPARK-13599) Groovy-all ends up in spark-assembly if hive profile set

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182299#comment-15182299 ] Sean Owen commented on SPARK-13599: --- [~rxin] do you object to me back-porting this to 1.6? I didn't see

[jira] [Resolved] (SPARK-13496) Optimizing count distinct changes the resulting column name

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13496. --- Resolution: Duplicate Provisionally labeling this a duplicate then > Optimizing count distinct

[jira] [Updated] (SPARK-13705) UpdateStateByKey Operation documentation incorrectly refers to StatefulNetworkWordCount

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13705: -- Target Version/s: (was: 1.6.0) Priority: Trivial (was: Minor) Fix Version/s:

[jira] [Resolved] (SPARK-13230) HashMap.merged not working properly with Spark

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13230. --- Resolution: Not A Problem OK, considering this not a problem (in Spark) for now, with the noted

[jira] [Resolved] (SPARK-13700) Rdd.mapAsync(): Easily mix Spark and asynchroneous transformation

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13700. --- Resolution: Not A Problem I think this might be best to float on a list first, since I don't think

[jira] [Resolved] (SPARK-13703) Remove obsolete scala-2.10 source files

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13703. --- Resolution: Not A Problem Yea, for the moment 2.10 support has not been dropped. Dropping it would

[jira] [Resolved] (SPARK-13701) MLlib ALS fails on arm64 (java.lang.UnsatisfiedLinkError: org.jblas.NativeBlas.dgemm))

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13701. --- Resolution: Not A Problem Yeah, I don't think Spark works on arm64 because jblas does not. The thing

[jira] [Resolved] (SPARK-13708) Null pointer Exception while starting spark shell

2016-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13708. --- Resolution: Invalid Please ask questions on u...@spark.apache.org

[jira] [Created] (SPARK-13708) Null pointer Exception while starting spark shell

2016-03-06 Thread Sowmya Dureddy (JIRA)
Sowmya Dureddy created SPARK-13708: -- Summary: Null pointer Exception while starting spark shell Key: SPARK-13708 URL: https://issues.apache.org/jira/browse/SPARK-13708 Project: Spark Issue

[jira] [Comment Edited] (SPARK-10548) Concurrent execution in SQL does not work

2016-03-06 Thread nicerobot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182265#comment-15182265 ] nicerobot edited comment on SPARK-10548 at 3/6/16 6:47 PM: --- I might be

[jira] [Commented] (SPARK-10548) Concurrent execution in SQL does not work

2016-03-06 Thread nicerobot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182265#comment-15182265 ] nicerobot commented on SPARK-10548: --- I might be misunderstanding the solution but i'm not clear how the

[jira] [Commented] (SPARK-8480) Add setName for Dataframe

2016-03-06 Thread Neelesh Srinivas Salian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182262#comment-15182262 ] Neelesh Srinivas Salian commented on SPARK-8480: Looking into this. Will post a PR for

[jira] [Comment Edited] (SPARK-13707) Streaming UI tab misleading for window operations

2016-03-06 Thread Jatin Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182259#comment-15182259 ] Jatin Kumar edited comment on SPARK-13707 at 3/6/16 6:29 PM: - The records

[jira] [Updated] (SPARK-13707) Streaming UI tab misleading for window operations

2016-03-06 Thread Jatin Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jatin Kumar updated SPARK-13707: Attachment: Screen Shot 2016-03-06 at 11.09.55 pm.png The records shown in image are of the 2 sec

[jira] [Commented] (SPARK-13707) Streaming UI tab misleading for window operations

2016-03-06 Thread Jatin Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182257#comment-15182257 ] Jatin Kumar commented on SPARK-13707: - Ideally all 2 sec batches should be linked to the final 120

[jira] [Updated] (SPARK-13707) Streaming UI tab misleading for window operations

2016-03-06 Thread Jatin Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jatin Kumar updated SPARK-13707: Description: 'Streaming' tab on spark UI is misleading when the job has a window operation which

[jira] [Created] (SPARK-13707) Streaming UI tab misleading for window operations

2016-03-06 Thread Jatin Kumar (JIRA)
Jatin Kumar created SPARK-13707: --- Summary: Streaming UI tab misleading for window operations Key: SPARK-13707 URL: https://issues.apache.org/jira/browse/SPARK-13707 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-13697) TransformFunctionSerializer.loads doesn't restore the function's module name if it's '__main__'

2016-03-06 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13697. Resolution: Fixed Fix Version/s: 1.4.2 1.6.1 1.5.3

[jira] [Issue Comment Deleted] (SPARK-12313) getPartitionsByFilter doesnt handle predicates on all / multiple Partition Columns

2016-03-06 Thread Harsh Gupta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh Gupta updated SPARK-12313: Comment: was deleted (was: [~liancheng] Should this issue occur after your PR

[jira] [Commented] (SPARK-12313) getPartitionsByFilter doesnt handle predicates on all / multiple Partition Columns

2016-03-06 Thread Harsh Gupta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182221#comment-15182221 ] Harsh Gupta commented on SPARK-12313: - Hi [~lian cheng] . Should this issue occur even after your PR

[jira] [Commented] (SPARK-12313) getPartitionsByFilter doesnt handle predicates on all / multiple Partition Columns

2016-03-06 Thread Harsh Gupta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182220#comment-15182220 ] Harsh Gupta commented on SPARK-12313: - [~liancheng] Should this issue occur after your PR

[jira] [Commented] (SPARK-13701) MLlib ALS fails on arm64 (java.lang.UnsatisfiedLinkError: org.jblas.NativeBlas.dgemm))

2016-03-06 Thread Santiago M. Mola (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182214#comment-15182214 ] Santiago M. Mola commented on SPARK-13701: -- Installed gfortran. Now it fails on NLSSuite, then

[jira] [Updated] (SPARK-13706) Python Example for Train Validation Split Missing

2016-03-06 Thread Jeremy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy updated SPARK-13706: --- Description: An example of how to use TrainValidationSplit in pyspark needs to be added. Should be

[jira] [Commented] (SPARK-13706) Python Example for Train Validation Split Missing

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182212#comment-15182212 ] Apache Spark commented on SPARK-13706: -- User 'JeremyNixon' has created a pull request for this

[jira] [Assigned] (SPARK-13706) Python Example for Train Validation Split Missing

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13706: Assignee: (was: Apache Spark) > Python Example for Train Validation Split Missing >

[jira] [Assigned] (SPARK-13706) Python Example for Train Validation Split Missing

2016-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13706: Assignee: Apache Spark > Python Example for Train Validation Split Missing >

[jira] [Created] (SPARK-13706) Python Example for Train Validation Split Missing

2016-03-06 Thread Jeremy (JIRA)
Jeremy created SPARK-13706: -- Summary: Python Example for Train Validation Split Missing Key: SPARK-13706 URL: https://issues.apache.org/jira/browse/SPARK-13706 Project: Spark Issue Type: Bug

  1   2   >