[jira] [Commented] (SPARK-18872) New test cases for EXISTS subquery

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852674#comment-15852674 ] Apache Spark commented on SPARK-18872: -- User 'dilipbiswal' has created a pull request for this

[jira] [Assigned] (SPARK-19446) Remove unused findTightestCommonType in TypeCoercion

2017-02-03 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-19446: --- Assignee: Hyukjin Kwon > Remove unused findTightestCommonType in TypeCoercion >

[jira] [Resolved] (SPARK-19446) Remove unused findTightestCommonType in TypeCoercion

2017-02-03 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-19446. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16786

[jira] [Commented] (SPARK-19428) Ability to select first row of groupby

2017-02-03 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852582#comment-15852582 ] Takeshi Yamamuro commented on SPARK-19428: -- Thanks for the explanation!

[jira] [Commented] (SPARK-19353) Support binary I/O in PipedRDD

2017-02-03 Thread Sergei Lebedev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852498#comment-15852498 ] Sergei Lebedev commented on SPARK-19353: For reference: we have a fully backward-compatible

[jira] [Assigned] (SPARK-13619) Jobs page UI shows wrong number of failed tasks

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13619: Assignee: Apache Spark > Jobs page UI shows wrong number of failed tasks >

[jira] [Commented] (SPARK-13619) Jobs page UI shows wrong number of failed tasks

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852470#comment-15852470 ] Apache Spark commented on SPARK-13619: -- User 'devaraj-kavali' has created a pull request for this

[jira] [Assigned] (SPARK-13619) Jobs page UI shows wrong number of failed tasks

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13619: Assignee: (was: Apache Spark) > Jobs page UI shows wrong number of failed tasks >

[jira] [Commented] (SPARK-19456) Add LinearSVC R API

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852450#comment-15852450 ] Apache Spark commented on SPARK-19456: -- User 'wangmiao1981' has created a pull request for this

[jira] [Commented] (SPARK-19386) Bisecting k-means in SparkR documentation

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852449#comment-15852449 ] Apache Spark commented on SPARK-19386: -- User 'actuaryzhang' has created a pull request for this

[jira] [Assigned] (SPARK-19456) Add LinearSVC R API

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19456: Assignee: (was: Apache Spark) > Add LinearSVC R API > --- > >

[jira] [Assigned] (SPARK-19456) Add LinearSVC R API

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19456: Assignee: Apache Spark > Add LinearSVC R API > --- > >

[jira] [Created] (SPARK-19456) Add LinearSVC R API

2017-02-03 Thread Miao Wang (JIRA)
Miao Wang created SPARK-19456: - Summary: Add LinearSVC R API Key: SPARK-19456 URL: https://issues.apache.org/jira/browse/SPARK-19456 Project: Spark Issue Type: New Feature Components:

[jira] [Commented] (SPARK-18873) New test cases for scalar subquery

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852439#comment-15852439 ] Apache Spark commented on SPARK-18873: -- User 'nsyca' has created a pull request for this issue:

[jira] [Updated] (SPARK-19418) Dataset generated java code fails to compile as java.lang.Long does not accept UTF8String in constructor

2017-02-03 Thread Suresh Avadhanula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Avadhanula updated SPARK-19418: -- Priority: Minor (was: Major) > Dataset generated java code fails to compile as

[jira] [Commented] (SPARK-19418) Dataset generated java code fails to compile as java.lang.Long does not accept UTF8String in constructor

2017-02-03 Thread Suresh Avadhanula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852416#comment-15852416 ] Suresh Avadhanula commented on SPARK-19418: --- Figure out the "workaround". Solution is based on

[jira] [Commented] (SPARK-19455) Add option for case-insensitive Parquet field resolution

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852242#comment-15852242 ] Apache Spark commented on SPARK-19455: -- User 'budde' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19455) Add option for case-insensitive Parquet field resolution

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19455: Assignee: (was: Apache Spark) > Add option for case-insensitive Parquet field

[jira] [Assigned] (SPARK-19455) Add option for case-insensitive Parquet field resolution

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19455: Assignee: Apache Spark > Add option for case-insensitive Parquet field resolution >

[jira] [Created] (SPARK-19455) Add option for case-insensitive Parquet field resolution

2017-02-03 Thread Adam Budde (JIRA)
Adam Budde created SPARK-19455: -- Summary: Add option for case-insensitive Parquet field resolution Key: SPARK-19455 URL: https://issues.apache.org/jira/browse/SPARK-19455 Project: Spark Issue

[jira] [Commented] (SPARK-10063) Remove DirectParquetOutputCommitter

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852170#comment-15852170 ] Apache Spark commented on SPARK-10063: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17161) Add PySpark-ML JavaWrapper convenience function to create py4j JavaArrays

2017-02-03 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk reassigned SPARK-17161: --- Assignee: Bryan Cutler Affects Version/s: 2.2.0 > Add PySpark-ML JavaWrapper

[jira] [Commented] (SPARK-19409) Upgrade Parquet to 1.8.2

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852090#comment-15852090 ] Apache Spark commented on SPARK-19409: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-19326) Speculated task attempts do not get launched in few scenarios

2017-02-03 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852085#comment-15852085 ] Kay Ousterhout commented on SPARK-19326: I see that makes sense; thanks for the additional

[jira] [Assigned] (SPARK-19452) Fix bug in the name assignment method in SparkR

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19452: Assignee: (was: Apache Spark) > Fix bug in the name assignment method in SparkR >

[jira] [Assigned] (SPARK-19452) Fix bug in the name assignment method in SparkR

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19452: Assignee: Apache Spark > Fix bug in the name assignment method in SparkR >

[jira] [Commented] (SPARK-19452) Fix bug in the name assignment method in SparkR

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852069#comment-15852069 ] Apache Spark commented on SPARK-19452: -- User 'actuaryzhang' has created a pull request for this

[jira] [Assigned] (SPARK-19386) Bisecting k-means in SparkR documentation

2017-02-03 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-19386: Assignee: Krishna Kalyan (was: Miao Wang) > Bisecting k-means in SparkR documentation >

[jira] [Resolved] (SPARK-19386) Bisecting k-means in SparkR documentation

2017-02-03 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-19386. -- Resolution: Fixed Fix Version/s: 2.2.0 > Bisecting k-means in SparkR documentation >

[jira] [Assigned] (SPARK-19454) Improve DataFrame.replace API

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19454: Assignee: (was: Apache Spark) > Improve DataFrame.replace API >

[jira] [Assigned] (SPARK-19454) Improve DataFrame.replace API

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19454: Assignee: Apache Spark > Improve DataFrame.replace API > - >

[jira] [Commented] (SPARK-19454) Improve DataFrame.replace API

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852046#comment-15852046 ] Apache Spark commented on SPARK-19454: -- User 'zero323' has created a pull request for this issue:

[jira] [Created] (SPARK-19454) Improve DataFrame.replace API

2017-02-03 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-19454: -- Summary: Improve DataFrame.replace API Key: SPARK-19454 URL: https://issues.apache.org/jira/browse/SPARK-19454 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-19453) Correct DataFrame.replace docs

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19453: Assignee: Apache Spark > Correct DataFrame.replace docs > --

[jira] [Assigned] (SPARK-19453) Correct DataFrame.replace docs

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19453: Assignee: (was: Apache Spark) > Correct DataFrame.replace docs >

[jira] [Assigned] (SPARK-19453) Correct DataFrame.replace docs

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19453: Assignee: Apache Spark > Correct DataFrame.replace docs > --

[jira] [Commented] (SPARK-19453) Correct DataFrame.replace docs

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852017#comment-15852017 ] Apache Spark commented on SPARK-19453: -- User 'zero323' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19453) Correct DataFrame.replace docs

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19453: Assignee: (was: Apache Spark) > Correct DataFrame.replace docs >

[jira] [Updated] (SPARK-19453) Correct DataFrame.replace docs

2017-02-03 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-19453: --- Summary: Correct DataFrame.replace docs (was: Correct Column.replace docs) >

[jira] [Updated] (SPARK-19453) Correct Column.replace docs

2017-02-03 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-19453: --- Summary: Correct Column.replace docs (was: Correct ) > Correct Column.replace docs

[jira] [Created] (SPARK-19453) Correct

2017-02-03 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-19453: -- Summary: Correct Key: SPARK-19453 URL: https://issues.apache.org/jira/browse/SPARK-19453 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-19452) Fix bug in the name assignment method in SparkR

2017-02-03 Thread Wayne Zhang (JIRA)
Wayne Zhang created SPARK-19452: --- Summary: Fix bug in the name assignment method in SparkR Key: SPARK-19452 URL: https://issues.apache.org/jira/browse/SPARK-19452 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2017-02-03 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-18539. Resolution: Fixed Assignee: Dongjoon Hyun Target Version/s: 2.2.0 > Cannot

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2017-02-03 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851965#comment-15851965 ] Cheng Lian commented on SPARK-18539: SPARK-19409 upgrades parquet-mr to 1.8.2 and fixed this issue.

[jira] [Commented] (SPARK-19409) Upgrade Parquet to 1.8.2

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851959#comment-15851959 ] Apache Spark commented on SPARK-19409: -- User 'liancheng' has created a pull request for this issue:

[jira] [Commented] (SPARK-17213) Parquet String Pushdown for Non-Eq Comparisons Broken

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851960#comment-15851960 ] Apache Spark commented on SPARK-17213: -- User 'liancheng' has created a pull request for this issue:

[jira] [Updated] (SPARK-19451) Long values in Window function

2017-02-03 Thread Julien Champ (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Champ updated SPARK-19451: - Description: Hi there, there seems to be a major limitation in spark window functions and

[jira] [Commented] (SPARK-19233) Inconsistent Behaviour of Spark Streaming Checkpoint

2017-02-03 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851787#comment-15851787 ] Nan Zhu commented on SPARK-19233: - ping > Inconsistent Behaviour of Spark Streaming Checkpoint >

[jira] [Assigned] (SPARK-19450) Replace askWithRetry with askSync.

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19450: Assignee: (was: Apache Spark) > Replace askWithRetry with askSync. >

[jira] [Assigned] (SPARK-19450) Replace askWithRetry with askSync.

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19450: Assignee: Apache Spark > Replace askWithRetry with askSync. >

[jira] [Commented] (SPARK-19280) Failed Recovery from checkpoint caused by the multi-threads issue in Spark Streaming scheduler

2017-02-03 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851786#comment-15851786 ] Nan Zhu commented on SPARK-19280: - ping > Failed Recovery from checkpoint caused by the multi-threads

[jira] [Commented] (SPARK-19450) Replace askWithRetry with askSync.

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851785#comment-15851785 ] Apache Spark commented on SPARK-19450: -- User 'jinxing64' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-19428) Ability to select first row of groupby

2017-02-03 Thread Luke Miner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851784#comment-15851784 ] Luke Miner edited comment on SPARK-19428 at 2/3/17 5:33 PM: Couple of things.

[jira] [Commented] (SPARK-19428) Ability to select first row of groupby

2017-02-03 Thread Luke Miner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851784#comment-15851784 ] Luke Miner commented on SPARK-19428: Couple of things. Sometimes I just want a random row from each

[jira] [Created] (SPARK-19451) Long values in Window function

2017-02-03 Thread Julien Champ (JIRA)
Julien Champ created SPARK-19451: Summary: Long values in Window function Key: SPARK-19451 URL: https://issues.apache.org/jira/browse/SPARK-19451 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-19450) Replace askWithRetry with askSync.

2017-02-03 Thread jin xing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jin xing updated SPARK-19450: - Description: *askSync* is already added in *RpcEndpointRef* (see SPARK-19347 and

[jira] [Created] (SPARK-19450) Replace askWithRetry with askSync.

2017-02-03 Thread jin xing (JIRA)
jin xing created SPARK-19450: Summary: Replace askWithRetry with askSync. Key: SPARK-19450 URL: https://issues.apache.org/jira/browse/SPARK-19450 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-19448) unify some duplication function in MetaStoreRelation

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851631#comment-15851631 ] Apache Spark commented on SPARK-19448: -- User 'windpiger' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19448) unify some duplication function in MetaStoreRelation

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19448: Assignee: (was: Apache Spark) > unify some duplication function in MetaStoreRelation

[jira] [Assigned] (SPARK-19448) unify some duplication function in MetaStoreRelation

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19448: Assignee: Apache Spark > unify some duplication function in MetaStoreRelation >

[jira] [Commented] (SPARK-19449) Inconsistent results between ml package RandomForestClassificationModel and mllib package RandomForestModel

2017-02-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851593#comment-15851593 ] Sean Owen commented on SPARK-19449: --- I don't think this can be made fully deterministic even when

[jira] [Assigned] (SPARK-19444) Tokenizer example does not compile without extra imports

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19444: Assignee: Apache Spark > Tokenizer example does not compile without extra imports >

[jira] [Commented] (SPARK-19444) Tokenizer example does not compile without extra imports

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851589#comment-15851589 ] Apache Spark commented on SPARK-19444: -- User 'anshbansal' has created a pull request for this issue:

[jira] [Commented] (SPARK-19444) Tokenizer example does not compile without extra imports

2017-02-03 Thread Aseem Bansal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851588#comment-15851588 ] Aseem Bansal commented on SPARK-19444: -- https://github.com/apache/spark/pull/16789 > Tokenizer

[jira] [Assigned] (SPARK-19444) Tokenizer example does not compile without extra imports

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19444: Assignee: (was: Apache Spark) > Tokenizer example does not compile without extra

[jira] [Commented] (SPARK-19449) Inconsistent results between ml package RandomForestClassificationModel and mllib package RandomForestModel

2017-02-03 Thread Aseem Bansal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851576#comment-15851576 ] Aseem Bansal commented on SPARK-19449: -- Isn't the decision tree debug string print it as a series of

[jira] [Commented] (SPARK-19449) Inconsistent results between ml package RandomForestClassificationModel and mllib package RandomForestModel

2017-02-03 Thread Aseem Bansal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851568#comment-15851568 ] Aseem Bansal commented on SPARK-19449: -- [~srowen] I removed some extra code. The part where I did

[jira] [Updated] (SPARK-19449) Inconsistent results between ml package RandomForestClassificationModel and mllib package RandomForestModel

2017-02-03 Thread Aseem Bansal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aseem Bansal updated SPARK-19449: - Description: I worked on some code to convert ml package RandomForestClassificationModel to

[jira] [Updated] (SPARK-19449) Inconsistent results between ml package RandomForestClassificationModel and mllib package RandomForestModel

2017-02-03 Thread Aseem Bansal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aseem Bansal updated SPARK-19449: - Description: I worked on some code to convert ml package RandomForestClassificationModel to

[jira] [Updated] (SPARK-18874) First phase: Deferring the correlated predicate pull up to Optimizer phase

2017-02-03 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nattavut Sutyanyong updated SPARK-18874: Attachment: SPARK-18874-3.pdf Design document version 1.1 dated February 3, 2017.

[jira] [Commented] (SPARK-18874) First phase: Deferring the correlated predicate pull up to Optimizer phase

2017-02-03 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851550#comment-15851550 ] Nattavut Sutyanyong commented on SPARK-18874: - I have published a design document as a

[jira] [Updated] (SPARK-19449) Inconsistent results between ml package RandomForestClassificationModel and mllib package RandomForestModel

2017-02-03 Thread Aseem Bansal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aseem Bansal updated SPARK-19449: - Description: I worked on some code to convert ml package RandomForestClassificationModel to

[jira] [Commented] (SPARK-19449) Inconsistent results between ml package RandomForestClassificationModel and mllib package RandomForestModel

2017-02-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851540#comment-15851540 ] Sean Owen commented on SPARK-19449: --- Can you boil this down? this is a lot of code to look at. I would

[jira] [Assigned] (SPARK-16742) Kerberos support for Spark on Mesos

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16742: Assignee: Apache Spark > Kerberos support for Spark on Mesos >

[jira] [Assigned] (SPARK-16742) Kerberos support for Spark on Mesos

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16742: Assignee: (was: Apache Spark) > Kerberos support for Spark on Mesos >

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

2017-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851539#comment-15851539 ] Apache Spark commented on SPARK-16742: -- User 'arinconstrio' has created a pull request for this

[jira] [Created] (SPARK-19449) Inconsistent results between ml package RandomForestClassificationModel and mllib package RandomForestModel

2017-02-03 Thread Aseem Bansal (JIRA)
Aseem Bansal created SPARK-19449: Summary: Inconsistent results between ml package RandomForestClassificationModel and mllib package RandomForestModel Key: SPARK-19449 URL:

[jira] [Assigned] (SPARK-19244) Sort MemoryConsumers according to their memory usage when spilling

2017-02-03 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-19244: --- Assignee: Liang-Chi Hsieh > Sort MemoryConsumers according to their memory

[jira] [Resolved] (SPARK-19244) Sort MemoryConsumers according to their memory usage when spilling

2017-02-03 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-19244. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request

[jira] [Commented] (SPARK-19444) Tokenizer example does not compile without extra imports

2017-02-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851515#comment-15851515 ] Sean Owen commented on SPARK-19444: --- You're right, I think this may be a copy-and-paste problem. I

[jira] [Resolved] (SPARK-16043) Prepare GenericArrayData implementation specialized for a primitive array

2017-02-03 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-16043. -- Resolution: Fixed Fix Version/s: 2.2.0 > Prepare GenericArrayData

[jira] [Resolved] (SPARK-16042) Eliminate nullcheck code at projection for an array type

2017-02-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16042. --- Resolution: Duplicate Subsumed by another issue according to PR > Eliminate nullcheck code at

[jira] [Resolved] (SPARK-16094) Support HashAggregateExec for non-partial aggregates

2017-02-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16094. --- Resolution: Won't Fix > Support HashAggregateExec for non-partial aggregates >

[jira] [Resolved] (SPARK-16041) Disallow Duplicate Columns in `partitionBy`, `bucketBy` and `sortBy`

2017-02-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16041. --- Resolution: Duplicate This was apparently subsumed by another issue. > Disallow Duplicate Columns

[jira] [Closed] (SPARK-16200) Rename AggregateFunction#supportsPartial

2017-02-03 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro closed SPARK-16200. Resolution: Won't Fix > Rename AggregateFunction#supportsPartial >

[jira] [Comment Edited] (SPARK-16200) Rename AggregateFunction#supportsPartial

2017-02-03 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851501#comment-15851501 ] Takeshi Yamamuro edited comment on SPARK-16200 at 2/3/17 1:54 PM: -- okay,

[jira] [Commented] (SPARK-16200) Rename AggregateFunction#supportsPartial

2017-02-03 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851501#comment-15851501 ] Takeshi Yamamuro commented on SPARK-16200: -- okay, thanks for letting me know! It's okay to set

[jira] [Commented] (SPARK-16043) Prepare GenericArrayData implementation specialized for a primitive array

2017-02-03 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851500#comment-15851500 ] Takeshi Yamamuro commented on SPARK-16043: -- I think the issue this ticket describes has been

[jira] [Closed] (SPARK-15180) Support subexpression elimination in Fliter

2017-02-03 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh closed SPARK-15180. --- Resolution: Won't Fix > Support subexpression elimination in Fliter >

[jira] [Commented] (SPARK-15180) Support subexpression elimination in Fliter

2017-02-03 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851491#comment-15851491 ] Liang-Chi Hsieh commented on SPARK-15180: - [~hyukjin.kwon] Yes. I resolved this. Thanks! >

[jira] [Commented] (SPARK-15911) Remove additional Project to be consistent with SQL when insert into table

2017-02-03 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851488#comment-15851488 ] Liang-Chi Hsieh commented on SPARK-15911: - [~hyukjin.kwon] Thanks! > Remove additional Project

[jira] [Resolved] (SPARK-17161) Add PySpark-ML JavaWrapper convenience function to create py4j JavaArrays

2017-02-03 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk resolved SPARK-17161. - Resolution: Fixed Fix Version/s: 2.2.0 > Add PySpark-ML JavaWrapper convenience function to

[jira] [Commented] (SPARK-16200) Rename AggregateFunction#supportsPartial

2017-02-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851474#comment-15851474 ] Hyukjin Kwon commented on SPARK-16200: -- (maybe it seems good to double-check this one too per

[jira] [Created] (SPARK-19448) unify some duplication function in MetaStoreRelation

2017-02-03 Thread Song Jun (JIRA)
Song Jun created SPARK-19448: Summary: unify some duplication function in MetaStoreRelation Key: SPARK-19448 URL: https://issues.apache.org/jira/browse/SPARK-19448 Project: Spark Issue Type:

[jira] [Commented] (SPARK-16094) Support HashAggregateExec for non-partial aggregates

2017-02-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851473#comment-15851473 ] Hyukjin Kwon commented on SPARK-16094: -- [~maropu], I just happened to see this JIRA. Maybe would

[jira] [Commented] (SPARK-19372) Code generation for Filter predicate including many OR conditions exceeds JVM method size limit

2017-02-03 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851472#comment-15851472 ] Kazuaki Ishizaki commented on SPARK-19372: -- I was able to reproduce this. I am thinking how to

[jira] [Commented] (SPARK-16043) Prepare GenericArrayData implementation specialized for a primitive array

2017-02-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851471#comment-15851471 ] Hyukjin Kwon commented on SPARK-16043: -- (maybe would be great if this one is checked too per

[jira] [Commented] (SPARK-16042) Eliminate nullcheck code at projection for an array type

2017-02-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851469#comment-15851469 ] Hyukjin Kwon commented on SPARK-16042: -- [~kiszk], would this JIRA maybe be resolvable per

[jira] [Commented] (SPARK-16041) Disallow Duplicate Columns in `partitionBy`, `bucketBy` and `sortBy`

2017-02-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851468#comment-15851468 ] Hyukjin Kwon commented on SPARK-16041: -- [~smilegator], I just happened to see this JIRA just while

[jira] [Resolved] (SPARK-15911) Remove additional Project to be consistent with SQL when insert into table

2017-02-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15911. -- Resolution: Duplicate I am resolving this per

  1   2   >