[jira] [Assigned] (SPARK-14504) Enable Oracle docker integration tests

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14504: Assignee: (was: Apache Spark) > Enable Oracle docker integration tests >

[jira] [Assigned] (SPARK-14504) Enable Oracle docker integration tests

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14504: Assignee: Apache Spark > Enable Oracle docker integration tests >

[jira] [Commented] (SPARK-14504) Enable Oracle docker integration tests

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233356#comment-15233356 ] Apache Spark commented on SPARK-14504: -- User 'lresende' has created a pull request for this issue:

[jira] [Created] (SPARK-14504) Enable Oracle docker integration tests

2016-04-08 Thread Luciano Resende (JIRA)
Luciano Resende created SPARK-14504: --- Summary: Enable Oracle docker integration tests Key: SPARK-14504 URL: https://issues.apache.org/jira/browse/SPARK-14504 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-13419) SubquerySuite should use checkAnswer rather than ScalaTest's assertResult

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1526#comment-1526 ] Apache Spark commented on SPARK-13419: -- User 'lresende' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13419) SubquerySuite should use checkAnswer rather than ScalaTest's assertResult

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13419: Assignee: (was: Apache Spark) > SubquerySuite should use checkAnswer rather than

[jira] [Assigned] (SPARK-13419) SubquerySuite should use checkAnswer rather than ScalaTest's assertResult

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13419: Assignee: Apache Spark > SubquerySuite should use checkAnswer rather than ScalaTest's

[jira] [Assigned] (SPARK-14480) Simplify CSV parsing process with a better performance

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14480: Assignee: Apache Spark > Simplify CSV parsing process with a better performance >

[jira] [Assigned] (SPARK-14480) Simplify CSV parsing process with a better performance

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14480: Assignee: (was: Apache Spark) > Simplify CSV parsing process with a better

[jira] [Commented] (SPARK-14480) Simplify CSV parsing process with a better performance

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1524#comment-1524 ] Apache Spark commented on SPARK-14480: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Resolved] (SPARK-14498) Various cleanups for ML documentation

2016-04-08 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-14498. - Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12266

[jira] [Commented] (SPARK-11368) Spark shouldn't scan all partitions when using Python UDF and filter over partitioned column is given

2016-04-08 Thread Yan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233231#comment-15233231 ] Yan commented on SPARK-11368: - The issue seems to be gone with the latest master code (for 2.0):

[jira] [Commented] (SPARK-14437) Spark using Netty RPC gets wrong address in some setups

2016-04-08 Thread Kevin Hogeland (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233211#comment-15233211 ] Kevin Hogeland commented on SPARK-14437: Will do soon. Can we get this fix applied to 1.6? >

[jira] [Resolved] (SPARK-14454) Better exception handling while marking tasks as failed

2016-04-08 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14454. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12234

[jira] [Commented] (SPARK-14437) Spark using Netty RPC gets wrong address in some setups

2016-04-08 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233196#comment-15233196 ] Shixiong Zhu commented on SPARK-14437: -- [~hogeland] could you open a new ticket for this issue? I

[jira] [Resolved] (SPARK-14437) Spark using Netty RPC gets wrong address in some setups

2016-04-08 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-14437. -- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.0.0 > Spark using

[jira] [Commented] (SPARK-928) Add support for Unsafe-based serializer in Kryo 2.22

2016-04-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233162#comment-15233162 ] Josh Rosen commented on SPARK-928: -- We've now upgraded to Kryo 3.0.0 (in SPARK-11416), so it would be

[jira] [Commented] (SPARK-7708) Incorrect task serialization with Kryo closure serializer

2016-04-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233159#comment-15233159 ] Josh Rosen commented on SPARK-7708: --- I just upgraded to Kryo 3.0.0 in SPARK-11416, so if someone want to

[jira] [Resolved] (SPARK-11416) Upgrade kryo package to version 3.0

2016-04-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-11416. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12076

[jira] [Commented] (SPARK-13944) Separate out local linear algebra as a standalone module without Spark dependency

2016-04-08 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233142#comment-15233142 ] DB Tsai commented on SPARK-13944: - There will be no converting back and forth in 2.0. Basically, `ml`

[jira] [Commented] (SPARK-14502) Add optimization for Non-Nullable Binary Comparison Simplification

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233109#comment-15233109 ] Apache Spark commented on SPARK-14502: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-14479) GLM predict type should be link or response?

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14479: -- Target Version/s: 2.0.0 > GLM predict type should be link or response? >

[jira] [Commented] (SPARK-4591) Algorithm/model parity in spark.ml (Scala)

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233104#comment-15233104 ] Joseph K. Bradley commented on SPARK-4591: -- Note: I am leaving this task targeted at 2.0 to bring

[jira] [Commented] (SPARK-14479) GLM predict type should be link or response?

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233117#comment-15233117 ] Joseph K. Bradley commented on SPARK-14479: --- +[~dbtsai] I'm ambivalent here. I'd assume new

[jira] [Created] (SPARK-14503) spark.ml API for FPGrowth

2016-04-08 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-14503: - Summary: spark.ml API for FPGrowth Key: SPARK-14503 URL: https://issues.apache.org/jira/browse/SPARK-14503 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-14502) Add optimization for Non-Nullable Binary Comparison Simplification

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14502: Assignee: (was: Apache Spark) > Add optimization for Non-Nullable Binary Comparison

[jira] [Assigned] (SPARK-14502) Add optimization for Non-Nullable Binary Comparison Simplification

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14502: Assignee: Apache Spark > Add optimization for Non-Nullable Binary Comparison

[jira] [Commented] (SPARK-10793) Make spark's use/subclassing of hive more maintainable

2016-04-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233108#comment-15233108 ] Josh Rosen commented on SPARK-10793: The Kryo part of this seems to be addressed by my reintroduction

[jira] [Updated] (SPARK-4591) Algorithm/model parity in spark.ml (Scala)

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-4591: - Description: This is an umbrella JIRA for porting spark.mllib implementations to use the

[jira] [Created] (SPARK-14502) Add optimization for Non-Nullable Binary Comparison Simplification

2016-04-08 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-14502: - Summary: Add optimization for Non-Nullable Binary Comparison Simplification Key: SPARK-14502 URL: https://issues.apache.org/jira/browse/SPARK-14502 Project: Spark

[jira] [Created] (SPARK-14501) spark.ml parity for fpm - frequent items

2016-04-08 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-14501: - Summary: spark.ml parity for fpm - frequent items Key: SPARK-14501 URL: https://issues.apache.org/jira/browse/SPARK-14501 Project: Spark Issue

[jira] [Created] (SPARK-14500) Accept Dataset[_] instead of DataFrame in MLlib APIs

2016-04-08 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-14500: - Summary: Accept Dataset[_] instead of DataFrame in MLlib APIs Key: SPARK-14500 URL: https://issues.apache.org/jira/browse/SPARK-14500 Project: Spark Issue

[jira] [Comment Edited] (SPARK-6617) Word2Vec is nondeterministic

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233025#comment-15233025 ] Joseph K. Bradley edited comment on SPARK-6617 at 4/8/16 10:32 PM: ---

[jira] [Updated] (SPARK-14087) PySpark ML JavaModel does not properly own params after being fit

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14087: -- Target Version/s: 2.0.0 > PySpark ML JavaModel does not properly own params after

[jira] [Created] (SPARK-14499) Add tests to make sure drop partitions of an external table will not delete data

2016-04-08 Thread Yin Huai (JIRA)
Yin Huai created SPARK-14499: Summary: Add tests to make sure drop partitions of an external table will not delete data Key: SPARK-14499 URL: https://issues.apache.org/jira/browse/SPARK-14499 Project:

[jira] [Commented] (SPARK-6617) Word2Vec is nondeterministic

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233025#comment-15233025 ] Joseph K. Bradley commented on SPARK-6617: -- [~mengxr] Does this still need to stay open? Is

[jira] [Assigned] (SPARK-14498) Various cleanups for ML documentation

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14498: Assignee: Joseph K. Bradley (was: Apache Spark) > Various cleanups for ML documentation

[jira] [Assigned] (SPARK-14498) Various cleanups for ML documentation

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14498: Assignee: Apache Spark (was: Joseph K. Bradley) > Various cleanups for ML documentation

[jira] [Commented] (SPARK-14498) Various cleanups for ML documentation

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232991#comment-15232991 ] Apache Spark commented on SPARK-14498: -- User 'jkbradley' has created a pull request for this issue:

[jira] [Created] (SPARK-14498) Various cleanups for ML documentation

2016-04-08 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-14498: - Summary: Various cleanups for ML documentation Key: SPARK-14498 URL: https://issues.apache.org/jira/browse/SPARK-14498 Project: Spark Issue Type:

[jira] [Commented] (SPARK-14477) Allow custom mirrors for downloading artifacts in build/mvn

2016-04-08 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232939#comment-15232939 ] Mark Grover commented on SPARK-14477: - Thanks Marcelo for committing! Much appreciated! Ah, I didn't

[jira] [Resolved] (SPARK-14435) Shade Kryo in our custom Hive 1.2.1 fork

2016-04-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-14435. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12215

[jira] [Updated] (SPARK-14394) Generate AggregateHashMap class during TungstenAggregate codegen

2016-04-08 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-14394: - Assignee: Sameer Agarwal > Generate AggregateHashMap class during TungstenAggregate codegen >

[jira] [Resolved] (SPARK-14394) Generate AggregateHashMap class during TungstenAggregate codegen

2016-04-08 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-14394. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12161

[jira] [Commented] (SPARK-10086) Flaky StreamingKMeans test in PySpark

2016-04-08 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232888#comment-15232888 ] Bryan Cutler commented on SPARK-10086: -- The changes to the test I proposed earlier are still valid,

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-04-08 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232885#comment-15232885 ] Mark Grover commented on SPARK-12177: - Thanks Cody. I agree about the separate subproject and I will

[jira] [Updated] (SPARK-13089) spark.ml Naive Bayes user guide

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-13089: -- Shepherd: Joseph K. Bradley Assignee: yuhao yang Target

[jira] [Assigned] (SPARK-14497) Use top instead of sortBy() to get top N frequent words as dict in ConutVectorizer

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14497: Assignee: Apache Spark > Use top instead of sortBy() to get top N frequent words as dict

[jira] [Assigned] (SPARK-14497) Use top instead of sortBy() to get top N frequent words as dict in ConutVectorizer

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14497: Assignee: (was: Apache Spark) > Use top instead of sortBy() to get top N frequent

[jira] [Commented] (SPARK-14497) Use top instead of sortBy() to get top N frequent words as dict in ConutVectorizer

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232881#comment-15232881 ] Apache Spark commented on SPARK-14497: -- User 'lionelfeng' has created a pull request for this issue:

[jira] [Updated] (SPARK-14497) Use top instead of sortBy() to get top N frequent words as dict in ConutVectorizer

2016-04-08 Thread Feng Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Wang updated SPARK-14497: -- Description: It's not necessary to sort the whole rdd to get top n frequent words. // Sort

[jira] [Updated] (SPARK-14497) Use top instead of sortBy() to get top N frequent words as dict in ConutVectorizer

2016-04-08 Thread Feng Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Wang updated SPARK-14497: -- Description: It's not necessary to sort the whole rdd to get top n frequent words. // Sort

[jira] [Created] (SPARK-14497) Use top instead of sortBy() to get top N frequent words as dict in ConutVectorizer

2016-04-08 Thread Feng Wang (JIRA)
Feng Wang created SPARK-14497: - Summary: Use top instead of sortBy() to get top N frequent words as dict in ConutVectorizer Key: SPARK-14497 URL: https://issues.apache.org/jira/browse/SPARK-14497

[jira] [Commented] (SPARK-14496) some typos in the java doc while browsing the codes

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232851#comment-15232851 ] Apache Spark commented on SPARK-14496: -- User 'bomeng' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14496) some typos in the java doc while browsing the codes

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14496: Assignee: (was: Apache Spark) > some typos in the java doc while browsing the codes >

[jira] [Assigned] (SPARK-14496) some typos in the java doc while browsing the codes

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14496: Assignee: Apache Spark > some typos in the java doc while browsing the codes >

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-04-08 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232828#comment-15232828 ] Cody Koeninger commented on SPARK-12177: Ok, since SPARK-13877 has been rejected and we're

[jira] [Created] (SPARK-14496) some typos in the java doc while browsing the codes

2016-04-08 Thread Bo Meng (JIRA)
Bo Meng created SPARK-14496: --- Summary: some typos in the java doc while browsing the codes Key: SPARK-14496 URL: https://issues.apache.org/jira/browse/SPARK-14496 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-12810) PySpark CrossValidatorModel should support avgMetrics

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-12810: -- Labels: starter (was: ) > PySpark CrossValidatorModel should support avgMetrics >

[jira] [Commented] (SPARK-10086) Flaky StreamingKMeans test in PySpark

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232777#comment-15232777 ] Joseph K. Bradley commented on SPARK-10086: --- I'm removing the target version. This is an

[jira] [Updated] (SPARK-10086) Flaky StreamingKMeans test in PySpark

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-10086: -- Target Version/s: (was: 2.0.0) > Flaky StreamingKMeans test in PySpark >

[jira] [Updated] (SPARK-14222) Cross-publish jackson-module-scala for Scala 2.12

2016-04-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-14222: --- Summary: Cross-publish jackson-module-scala for Scala 2.12 (was: Remove jackson-module-scala

[jira] [Updated] (SPARK-14439) Cross-publish json4s-jackson for Scala 2.12 or remove json4s dependency

2016-04-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-14439: --- Summary: Cross-publish json4s-jackson for Scala 2.12 or remove json4s dependency (was:

[jira] [Commented] (SPARK-14438) Cross-publish Breeze for Scala 2.12

2016-04-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232765#comment-15232765 ] Josh Rosen commented on SPARK-14438: If we upgrade to the latest version of Breeze, this is going to

[jira] [Updated] (SPARK-13783) Model export/import for spark.ml: GBTs

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-13783: -- Assignee: Yanbo Liang > Model export/import for spark.ml: GBTs >

[jira] [Resolved] (SPARK-14448) Improvements to ColumnVector

2016-04-08 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14448. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12225

[jira] [Commented] (SPARK-11502) Word2VecSuite needs appropriate checks

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232737#comment-15232737 ] Joseph K. Bradley commented on SPARK-11502: --- [~yuhaoyan] I just came across this JIRA again.

[jira] [Commented] (SPARK-14495) Distinct aggregation cannot be used in the having clause

2016-04-08 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232717#comment-15232717 ] Yin Huai commented on SPARK-14495: -- The current workaround is to use a subquery and then use where. >

[jira] [Updated] (SPARK-14495) Distinct aggregation cannot be used in the having clause

2016-04-08 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-14495: - Target Version/s: 1.6.2, 2.0.0 (was: 2.0.0) > Distinct aggregation cannot be used in the having clause

[jira] [Created] (SPARK-14495) Distinct aggregation cannot be used in the having clause

2016-04-08 Thread Yin Huai (JIRA)
Yin Huai created SPARK-14495: Summary: Distinct aggregation cannot be used in the having clause Key: SPARK-14495 URL: https://issues.apache.org/jira/browse/SPARK-14495 Project: Spark Issue Type:

[jira] [Updated] (SPARK-13448) Document MLlib behavior changes in Spark 2.0

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-13448: -- Description: This JIRA keeps a list of MLlib behavior changes in Spark 2.0. So we can

[jira] [Assigned] (SPARK-14298) LDA should support disable checkpoint

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14298: Assignee: Apache Spark (was: Yanbo Liang) > LDA should support disable checkpoint >

[jira] [Assigned] (SPARK-14298) LDA should support disable checkpoint

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14298: Assignee: Yanbo Liang (was: Apache Spark) > LDA should support disable checkpoint >

[jira] [Reopened] (SPARK-14298) LDA should support disable checkpoint

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reopened SPARK-14298: --- Reopening for backports, pending unit test PR > LDA should support disable checkpoint >

[jira] [Resolved] (SPARK-14298) LDA should support disable checkpoint

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-14298. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12089

[jira] [Commented] (SPARK-14401) Merge our sbt-pom-reader changes upstream

2016-04-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232664#comment-15232664 ] Josh Rosen commented on SPARK-14401: I saw that change; it seems pretty complicated, so I think it's

[jira] [Commented] (SPARK-14437) Spark using Netty RPC gets wrong address in some setups

2016-04-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232636#comment-15232636 ] Marcelo Vanzin commented on SPARK-14437: Not without trying to figure out what code might be

[jira] [Commented] (SPARK-14437) Spark using Netty RPC gets wrong address in some setups

2016-04-08 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232628#comment-15232628 ] Shixiong Zhu commented on SPARK-14437: -- Thanks a lot for your tests, [~hogeland]. [~vanzin] Any

[jira] [Commented] (SPARK-14209) Application failure during preemption.

2016-04-08 Thread Miles Crawford (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232598#comment-15232598 ] Miles Crawford commented on SPARK-14209: So, I have launched the exact same simple application

[jira] [Assigned] (SPARK-12569) DecisionTreeRegressor: provide variance of prediction: Python API

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-12569: - Assignee: Joseph K. Bradley > DecisionTreeRegressor: provide variance of

[jira] [Updated] (SPARK-12569) DecisionTreeRegressor: provide variance of prediction: Python API

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-12569: -- Assignee: Miao Wang (was: Joseph K. Bradley) > DecisionTreeRegressor: provide

[jira] [Resolved] (SPARK-12569) DecisionTreeRegressor: provide variance of prediction: Python API

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-12569. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12116

[jira] [Resolved] (SPARK-14373) PySpark ml RandomForestClassifier, Regressor support export/import

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-14373. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12238

[jira] [Updated] (SPARK-14373) PySpark ml RandomForestClassifier, Regressor support export/import

2016-04-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14373: -- Assignee: Kai Jiang > PySpark ml RandomForestClassifier, Regressor support

[jira] [Commented] (SPARK-14494) Fix the race conditions in MemoryStream and MemorySink

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232534#comment-15232534 ] Apache Spark commented on SPARK-14494: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14494) Fix the race conditions in MemoryStream and MemorySink

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14494: Assignee: Apache Spark (was: Shixiong Zhu) > Fix the race conditions in MemoryStream and

[jira] [Assigned] (SPARK-14494) Fix the race conditions in MemoryStream and MemorySink

2016-04-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14494: Assignee: Shixiong Zhu (was: Apache Spark) > Fix the race conditions in MemoryStream and

[jira] [Created] (SPARK-14494) Fix the race conditions in MemoryStream and MemorySink

2016-04-08 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-14494: Summary: Fix the race conditions in MemoryStream and MemorySink Key: SPARK-14494 URL: https://issues.apache.org/jira/browse/SPARK-14494 Project: Spark Issue

[jira] [Resolved] (SPARK-14477) Allow custom mirrors for downloading artifacts in build/mvn

2016-04-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-14477. Resolution: Fixed Assignee: Mark Grover Fix Version/s: 2.0.0 > Allow

[jira] [Commented] (SPARK-14478) Should StandardScaler use biased variance to scale?

2016-04-08 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232468#comment-15232468 ] Yanbo Liang commented on SPARK-14478: - Should we add a param that control whether use biased or

[jira] [Commented] (SPARK-14433) PySpark ml GaussianMixture

2016-04-08 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232465#comment-15232465 ] Miao Wang commented on SPARK-14433: --- Thanks! I started to learn the usage and will begin coding later

[jira] [Commented] (SPARK-14475) Propagate user-defined context from driver to executors

2016-04-08 Thread Eric Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232455#comment-15232455 ] Eric Liang commented on SPARK-14475: I think the main difference is that this is transparent to the

[jira] [Created] (SPARK-14493) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." should always be used with a user defined path

2016-04-08 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-14493: -- Summary: "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." should always be used with a user defined path Key: SPARK-14493 URL: https://issues.apache.org/jira/browse/SPARK-14493

[jira] [Commented] (SPARK-14488) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." creates persisted table

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232422#comment-15232422 ] Cheng Lian commented on SPARK-14488: Yea, that's why I came to this DDL command, because this command

[jira] [Comment Edited] (SPARK-14488) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." creates persisted table

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232414#comment-15232414 ] Cheng Lian edited comment on SPARK-14488 at 4/8/16 4:17 PM: Discussed with

[jira] [Commented] (SPARK-14488) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." creates persisted table

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232414#comment-15232414 ] Cheng Lian commented on SPARK-14488: Discussed with [~yhuai] offline, and here's the summary:

[jira] [Commented] (SPARK-14488) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." creates persisted table

2016-04-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232402#comment-15232402 ] Herman van Hovell commented on SPARK-14488: --- {{CreateTempTableUsingAsSelect}} should be planned

[jira] [Commented] (SPARK-14488) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." creates persisted table

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232401#comment-15232401 ] Cheng Lian commented on SPARK-14488: Ah, sorry, the logical plan class {{CreateTableUsingAsSelect}}

[jira] [Updated] (SPARK-14488) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." creates persisted table

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14488: --- Description: The following Spark shell snippet reproduces this bug: {code} sqlContext range 10

[jira] [Updated] (SPARK-14488) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." creates persisted table

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14488: --- Description: The following Spark shell snippet reproduces this bug: {code} sqlContext range 10

  1   2   >