[jira] [Commented] (SPARK-6567) Large linear model parallelism via a join and reduceByKey

2017-02-24 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882182#comment-15882182 ] Nick Pentreath commented on SPARK-6567: --- This JIRA has been around for a while without any movement.

[jira] [Commented] (SPARK-3434) Distributed block matrix

2017-02-24 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882179#comment-15882179 ] Nick Pentreath commented on SPARK-3434: --- This JIRA only has SPARK-3976 open. There was an old PR for

[jira] [Commented] (SPARK-2336) Approximate k-NN Models for MLLib

2017-02-24 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882187#comment-15882187 ] Nick Pentreath commented on SPARK-2336: --- I think it's safe to say that this now lives in a Spark

[jira] [Closed] (SPARK-10041) Proposal of Parameter Server Interface for Spark

2017-02-24 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath closed SPARK-10041. -- Resolution: Won't Fix > Proposal of Parameter Server Interface for Spark >

[jira] [Closed] (SPARK-10041) Proposal of Parameter Server Interface for Spark

2017-02-24 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath closed SPARK-10041. -- Resolution: Won't Fix > Proposal of Parameter Server Interface for Spark >

[jira] [Reopened] (SPARK-10041) Proposal of Parameter Server Interface for Spark

2017-02-24 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath reopened SPARK-10041: > Proposal of Parameter Server Interface for Spark >

[jira] [Commented] (SPARK-10041) Proposal of Parameter Server Interface for Spark

2017-02-24 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882198#comment-15882198 ] Nick Pentreath commented on SPARK-10041: I think it is safe to say this is not going to be part

[jira] [Created] (SPARK-19724) create table for hive tables with an existed default location should throw an exception

2017-02-24 Thread Song Jun (JIRA)
Song Jun created SPARK-19724: Summary: create table for hive tables with an existed default location should throw an exception Key: SPARK-19724 URL: https://issues.apache.org/jira/browse/SPARK-19724

[jira] [Created] (SPARK-19725) different parquet dependency in spark2.x and Hive2.x cause failure of HoS when using parquet file format

2017-02-24 Thread KaiXu (JIRA)
KaiXu created SPARK-19725: - Summary: different parquet dependency in spark2.x and Hive2.x cause failure of HoS when using parquet file format Key: SPARK-19725 URL: https://issues.apache.org/jira/browse/SPARK-19725

[jira] [Updated] (SPARK-19711) Bug in gapply function

2017-02-24 Thread Luis Felipe Sant Ana (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luis Felipe Sant Ana updated SPARK-19711: - Attachment: (was: resume.R) > Bug in gapply function >

[jira] [Updated] (SPARK-19711) Bug in gapply function

2017-02-24 Thread Luis Felipe Sant Ana (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luis Felipe Sant Ana updated SPARK-19711: - Attachment: resume.R > Bug in gapply function > -- > >

[jira] [Comment Edited] (SPARK-19711) Bug in gapply function

2017-02-24 Thread Luis Felipe Sant Ana (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882387#comment-15882387 ] Luis Felipe Sant Ana edited comment on SPARK-19711 at 2/24/17 10:18 AM:

[jira] [Commented] (SPARK-19711) Bug in gapply function

2017-02-24 Thread Luis Felipe Sant Ana (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882387#comment-15882387 ] Luis Felipe Sant Ana commented on SPARK-19711: -- Hello Felix, I uploaded two files, a CSV

[jira] [Created] (SPARK-19726) Faild to insert null timestamp value to mysql using spark jdbc

2017-02-24 Thread AnfengYuan (JIRA)
AnfengYuan created SPARK-19726: -- Summary: Faild to insert null timestamp value to mysql using spark jdbc Key: SPARK-19726 URL: https://issues.apache.org/jira/browse/SPARK-19726 Project: Spark

[jira] [Updated] (SPARK-19724) create managed table for hive tables with an existed default location should throw an exception

2017-02-24 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-19724: - Summary: create managed table for hive tables with an existed default location should throw an exception

[jira] [Assigned] (SPARK-19724) create managed table for hive tables with an existed default location should throw an exception

2017-02-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19724: Assignee: (was: Apache Spark) > create managed table for hive tables with an existed

[jira] [Updated] (SPARK-19711) Bug in gapply function

2017-02-24 Thread Luis Felipe Sant Ana (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luis Felipe Sant Ana updated SPARK-19711: - Attachment: resume.R > Bug in gapply function > -- > >

[jira] [Commented] (SPARK-19724) create managed table for hive tables with an existed default location should throw an exception

2017-02-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882339#comment-15882339 ] Apache Spark commented on SPARK-19724: -- User 'windpiger' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19724) create managed table for hive tables with an existed default location should throw an exception

2017-02-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19724: Assignee: Apache Spark > create managed table for hive tables with an existed default

[jira] [Updated] (SPARK-19711) Bug in gapply function

2017-02-24 Thread Luis Felipe Sant Ana (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luis Felipe Sant Ana updated SPARK-19711: - Attachment: mv_demand_20170221.csv > Bug in gapply function >

[jira] [Commented] (SPARK-19699) createOrReplaceTable does not always replace an existing table of the same name

2017-02-24 Thread Genmao Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882385#comment-15882385 ] Genmao Yu commented on SPARK-19699: --- Good catch! Maybe we can add {{rdd.id}} or something else.

[jira] [Comment Edited] (SPARK-19699) createOrReplaceTable does not always replace an existing table of the same name

2017-02-24 Thread Genmao Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882385#comment-15882385 ] Genmao Yu edited comment on SPARK-19699 at 2/24/17 10:16 AM: - Good catch!

[jira] [Commented] (SPARK-19711) Bug in gapply function

2017-02-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882204#comment-15882204 ] Felix Cheung commented on SPARK-19711: -- Hmm, if you have a way to dump the data.frame in your UDF so

[jira] [Commented] (SPARK-19714) Bucketizer Bug Regarding Handling Unbucketed Inputs

2017-02-24 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882216#comment-15882216 ] Nick Pentreath commented on SPARK-19714: I agree that the parameter naming is perhaps misleading.

[jira] [Updated] (SPARK-19711) Bug in gapply function

2017-02-24 Thread Luis Felipe Sant Ana (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luis Felipe Sant Ana updated SPARK-19711: - Attachment: resume.R > Bug in gapply function > -- > >

[jira] [Commented] (SPARK-19714) Bucketizer Bug Regarding Handling Unbucketed Inputs

2017-02-24 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882224#comment-15882224 ] Nick Pentreath commented on SPARK-19714: Another alternative is that we do expand the "invalid"

[jira] [Updated] (SPARK-19691) Calculating percentile of decimal column fails with ClassCastException

2017-02-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-19691: -- Fix Version/s: 2.1.1 > Calculating percentile of decimal column fails with

[jira] [Issue Comment Deleted] (SPARK-19699) createOrReplaceTable does not always replace an existing table of the same name

2017-02-24 Thread Genmao Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Genmao Yu updated SPARK-19699: -- Comment: was deleted (was: Good catch! Maybe we can add {{rdd.id}} or something else. [~cloud_fan]

[jira] [Comment Edited] (SPARK-15678) Not use cache on appends and overwrites

2017-02-24 Thread Gen TANG (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877735#comment-15877735 ] Gen TANG edited comment on SPARK-15678 at 2/24/17 10:44 AM: Hi, All It

[jira] [Comment Edited] (SPARK-15678) Not use cache on appends and overwrites

2017-02-24 Thread Gen TANG (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877735#comment-15877735 ] Gen TANG edited comment on SPARK-15678 at 2/24/17 10:44 AM: Hi, All It

[jira] [Commented] (SPARK-18813) MLlib 2.2 Roadmap

2017-02-24 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882206#comment-15882206 ] Nick Pentreath commented on SPARK-18813: FYI I've started going through a few of the top Watched

[jira] [Comment Edited] (SPARK-19714) Bucketizer Bug Regarding Handling Unbucketed Inputs

2017-02-24 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882216#comment-15882216 ] Nick Pentreath edited comment on SPARK-19714 at 2/24/17 8:35 AM: - I agree

[jira] [Commented] (SPARK-19725) different parquet dependency in spark2.x and Hive2.x cause failure of HoS when using parquet file format

2017-02-24 Thread KaiXu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882311#comment-15882311 ] KaiXu commented on SPARK-19725: --- using parquet-provided profile can workaround this issue, but it's better

[jira] [Updated] (SPARK-19711) Bug in gapply function

2017-02-24 Thread Luis Felipe Sant Ana (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luis Felipe Sant Ana updated SPARK-19711: - Attachment: (was: resume.R) > Bug in gapply function >

[jira] [Created] (SPARK-19727) Spark SQL round function modifies original column

2017-02-24 Thread JIRA
SÅ‚awomir Bogutyn created SPARK-19727: Summary: Spark SQL round function modifies original column Key: SPARK-19727 URL: https://issues.apache.org/jira/browse/SPARK-19727 Project: Spark

[jira] [Comment Edited] (SPARK-19711) Bug in gapply function

2017-02-24 Thread Luis Felipe Sant Ana (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882475#comment-15882475 ] Luis Felipe Sant Ana edited comment on SPARK-19711 at 2/24/17 11:24 AM:

[jira] [Comment Edited] (SPARK-19711) Bug in gapply function

2017-02-24 Thread Luis Felipe Sant Ana (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882475#comment-15882475 ] Luis Felipe Sant Ana edited comment on SPARK-19711 at 2/24/17 11:23 AM:

[jira] [Commented] (SPARK-19711) Bug in gapply function

2017-02-24 Thread Luis Felipe Sant Ana (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882475#comment-15882475 ] Luis Felipe Sant Ana commented on SPARK-19711: -- The problem seems to be in using the string

[jira] [Updated] (SPARK-19724) create managed table for hive tables with an existed default location should throw an exception

2017-02-24 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-19724: - Description: This JIRA is a follow up work after

[jira] [Commented] (SPARK-17147) Spark Streaming Kafka 0.10 Consumer Can't Handle Non-consecutive Offsets (i.e. Log Compaction)

2017-02-24 Thread Dean Wampler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882778#comment-15882778 ] Dean Wampler commented on SPARK-17147: -- We're interested in this enhancement. Anyone know if and one

[jira] [Created] (SPARK-19728) PythonUDF with multiple parents shouldn't be pushed down when used as a predicat

2017-02-24 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-19728: -- Summary: PythonUDF with multiple parents shouldn't be pushed down when used as a predicat Key: SPARK-19728 URL: https://issues.apache.org/jira/browse/SPARK-19728

[jira] [Updated] (SPARK-19728) PythonUDF with multiple parents shouldn't be pushed down when used as a predicate

2017-02-24 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-19728: --- Summary: PythonUDF with multiple parents shouldn't be pushed down when used as a

[jira] [Updated] (SPARK-19724) create a managed table with an existed default location should throw an exception

2017-02-24 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-19724: - Summary: create a managed table with an existed default location should throw an exception (was: create

[jira] [Updated] (SPARK-19730) Predicate Subqueries do not push results of subqueries to data source

2017-02-24 Thread Shawn Lavelle (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Lavelle updated SPARK-19730: -- Description: When a SparkSQL query contains a subquery in the where clause, such as a

[jira] [Commented] (SPARK-19711) Bug in gapply function

2017-02-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883337#comment-15883337 ] Felix Cheung commented on SPARK-19711: -- Thanks I'll look into this shortly. > Bug in gapply

[jira] [Resolved] (SPARK-19725) different parquet dependency in spark2.x and Hive2.x cause failure of HoS when using parquet file format

2017-02-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19725. --- Resolution: Not A Problem Hive 2 isn't supported, is it? Spark is already on Parquet 1.8. >

[jira] [Commented] (SPARK-13446) Spark need to support reading data from Hive 2.0.0 metastore

2017-02-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883943#comment-15883943 ] Apache Spark commented on SPARK-13446: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13446) Spark need to support reading data from Hive 2.0.0 metastore

2017-02-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13446: Assignee: Apache Spark > Spark need to support reading data from Hive 2.0.0 metastore >

[jira] [Assigned] (SPARK-13446) Spark need to support reading data from Hive 2.0.0 metastore

2017-02-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13446: Assignee: (was: Apache Spark) > Spark need to support reading data from Hive 2.0.0

[jira] [Commented] (SPARK-17495) Hive hash implementation

2017-02-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883951#comment-15883951 ] Apache Spark commented on SPARK-17495: -- User 'tejasapatil' has created a pull request for this

[jira] [Commented] (SPARK-19734) OneHotEncoder __init__ uses dropLast but doc strings all say includeFirst

2017-02-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883953#comment-15883953 ] Sean Owen commented on SPARK-19734: --- Agreed, feel free to open a PR to fix it. > OneHotEncoder

[jira] [Commented] (SPARK-14079) Limit the number of queries on SQL UI

2017-02-24 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883966#comment-15883966 ] Hyukjin Kwon commented on SPARK-14079: -- [~shixi...@databricks.com], I am just curious if this JIRA

[jira] [Comment Edited] (SPARK-14079) Limit the number of queries on SQL UI

2017-02-24 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883966#comment-15883966 ] Hyukjin Kwon edited comment on SPARK-14079 at 2/25/17 2:47 AM: --- [~zsxwing],

[jira] [Created] (SPARK-19734) OneHotEncoder __init__ uses dropLast but doc strings all say includeFirst

2017-02-24 Thread Corey (JIRA)
Corey created SPARK-19734: - Summary: OneHotEncoder __init__ uses dropLast but doc strings all say includeFirst Key: SPARK-19734 URL: https://issues.apache.org/jira/browse/SPARK-19734 Project: Spark

[jira] [Commented] (SPARK-19715) Option to Strip Paths in FileSource

2017-02-24 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882995#comment-15882995 ] Steve Loughran commented on SPARK-19715: This is a silly question, but has the situation " a

[jira] [Resolved] (SPARK-19161) Improving UDF Docstrings

2017-02-24 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk resolved SPARK-19161. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16534

[jira] [Commented] (SPARK-19161) Improving UDF Docstrings

2017-02-24 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883013#comment-15883013 ] holdenk commented on SPARK-19161: - Thanks for working on this [~zero323], having better docs for UDFs

[jira] [Assigned] (SPARK-19161) Improving UDF Docstrings

2017-02-24 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk reassigned SPARK-19161: --- Assignee: Maciej Szymkiewicz > Improving UDF Docstrings > > >

[jira] [Commented] (SPARK-19714) Bucketizer Bug Regarding Handling Unbucketed Inputs

2017-02-24 Thread Bill Chambers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883125#comment-15883125 ] Bill Chambers commented on SPARK-19714: --- The thing is QuantileDiscretizer and Bucketizer do

[jira] [Resolved] (SPARK-19038) Can't find keytab file when using Hive catalog

2017-02-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-19038. Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 2.2.0

[jira] [Commented] (SPARK-15678) Not use cache on appends and overwrites

2017-02-24 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883035#comment-15883035 ] Kazuaki Ishizaki commented on SPARK-15678: -- Sorry for being late to reply. According to the

[jira] [Comment Edited] (SPARK-19714) Bucketizer Bug Regarding Handling Unbucketed Inputs

2017-02-24 Thread Bill Chambers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883125#comment-15883125 ] Bill Chambers edited comment on SPARK-19714 at 2/24/17 5:15 PM: The thing

[jira] [Resolved] (SPARK-19707) Improve the invalid path check for sc.addJar

2017-02-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-19707. Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 2.2.0

[jira] [Commented] (SPARK-19351) Support for obtaining file splits from underlying InputFormat

2017-02-24 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883177#comment-15883177 ] Reynold Xin commented on SPARK-19351: - Approach 1 should be supported today. I actually think our

[jira] [Resolved] (SPARK-17495) Hive hash implementation

2017-02-24 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17495. - Resolution: Fixed Fix Version/s: 2.2.0 > Hive hash implementation >

[jira] [Reopened] (SPARK-17495) Hive hash implementation

2017-02-24 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tejas Patil reopened SPARK-17495: - Re-opening. This is not done yet as there are few datatypes that need to be handled and making

[jira] [Commented] (SPARK-17495) Hive hash implementation

2017-02-24 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883203#comment-15883203 ] Tejas Patil commented on SPARK-17495: - [~rxin] : No probs. Any opinion about my comment from

[jira] [Created] (SPARK-19730) Predicate Subqueries do not push results of subqueries to data source

2017-02-24 Thread Shawn Lavelle (JIRA)
Shawn Lavelle created SPARK-19730: - Summary: Predicate Subqueries do not push results of subqueries to data source Key: SPARK-19730 URL: https://issues.apache.org/jira/browse/SPARK-19730 Project:

[jira] [Updated] (SPARK-19730) Predicate Subqueries do not push results of subqueries to data source

2017-02-24 Thread Shawn Lavelle (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Lavelle updated SPARK-19730: -- Description: When a SparkSQL query contains a subquery in the where clause, such as a

[jira] [Commented] (SPARK-17495) Hive hash implementation

2017-02-24 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883189#comment-15883189 ] Reynold Xin commented on SPARK-17495: - Ah yes. I kept doing it ... :) > Hive hash implementation >

[jira] [Updated] (SPARK-19572) Allow to disable hive in sparkR shell

2017-02-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19572: - Target Version/s: (was: 2.1.1) > Allow to disable hive in sparkR shell >

[jira] [Updated] (SPARK-19730) Predicate Subqueries do not push results of subqueries to data source

2017-02-24 Thread Shawn Lavelle (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Lavelle updated SPARK-19730: -- Description: When a SparkSQL query contains a subquery in the where clause, such as a

[jira] [Assigned] (SPARK-17495) Hive hash implementation

2017-02-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17495: Assignee: Apache Spark (was: Tejas Patil) > Hive hash implementation >

[jira] [Assigned] (SPARK-17495) Hive hash implementation

2017-02-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17495: Assignee: Tejas Patil (was: Apache Spark) > Hive hash implementation >

[jira] [Resolved] (SPARK-2336) Approximate k-NN Models for MLLib

2017-02-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2336. -- Resolution: Won't Fix > Approximate k-NN Models for MLLib > - > >

[jira] [Assigned] (SPARK-17078) show estimated stats when doing explain

2017-02-24 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-17078: --- Assignee: Zhenhua Wang > show estimated stats when doing explain >

[jira] [Resolved] (SPARK-17078) show estimated stats when doing explain

2017-02-24 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-17078. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16594

[jira] [Updated] (SPARK-19730) Predicate Subqueries do not push results of subqueries to data source

2017-02-24 Thread Shawn Lavelle (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Lavelle updated SPARK-19730: -- Description: When a SparkSQL query contains a subquery in the where clause, such as a

[jira] [Commented] (SPARK-14409) Investigate adding a RankingEvaluator to ML

2017-02-24 Thread Yong Tang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883417#comment-15883417 ] Yong Tang commented on SPARK-14409: --- Thanks [~mlnick] for the reminder. I will take a look and update

[jira] [Commented] (SPARK-14409) Investigate adding a RankingEvaluator to ML

2017-02-24 Thread Roberto Mirizzi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883431#comment-15883431 ] Roberto Mirizzi commented on SPARK-14409: - [~mlnick] my implementation was conceptually close to

[jira] [Comment Edited] (SPARK-17495) Hive hash implementation

2017-02-24 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883203#comment-15883203 ] Tejas Patil edited comment on SPARK-17495 at 2/24/17 5:57 PM: -- [~rxin] : No

[jira] [Updated] (SPARK-19730) Predicate Subqueries do not push results of subqueries to data source

2017-02-24 Thread Shawn Lavelle (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Lavelle updated SPARK-19730: -- Description: When a SparkSQL query contains a subquery in the where clause, such as a

[jira] [Created] (SPARK-19729) Strange behaviour with reading csv with schema into dataframe

2017-02-24 Thread Mazen Melouk (JIRA)
Mazen Melouk created SPARK-19729: Summary: Strange behaviour with reading csv with schema into dataframe Key: SPARK-19729 URL: https://issues.apache.org/jira/browse/SPARK-19729 Project: Spark

[jira] [Commented] (SPARK-19698) Race condition in stale attempt task completion vs current attempt task completion when task is doing persistent state changes

2017-02-24 Thread Charles Allen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883331#comment-15883331 ] Charles Allen commented on SPARK-19698: --- [~mridulm80] is there documentation somewhere that

[jira] [Closed] (SPARK-19560) Improve tests for when DAGScheduler learns of "successful" ShuffleMapTask from a failed executor

2017-02-24 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout closed SPARK-19560. -- Resolution: Fixed Target Version/s: 2.2.0 > Improve tests for when DAGScheduler

[jira] [Closed] (SPARK-4681) Turn on executor level blacklisting by default

2017-02-24 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout closed SPARK-4681. - Resolution: Duplicate This was for the old blacklisting mechanism. The linked JIRAs introduce a

[jira] [Commented] (SPARK-17495) Hive hash implementation

2017-02-24 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883394#comment-15883394 ] Reynold Xin commented on SPARK-17495: - Let me put some thoughts here Please let me know if I

[jira] [Updated] (SPARK-19730) Predicate Subqueries do not push results of subqueries to data source

2017-02-24 Thread Shawn Lavelle (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Lavelle updated SPARK-19730: -- Description: When a SparkSQL query contains a subquery in the where clause, such as a

[jira] [Updated] (SPARK-19730) Predicate Subqueries do not push results of subqueries to data source

2017-02-24 Thread Shawn Lavelle (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Lavelle updated SPARK-19730: -- Description: When a SparkSQL query contains a subquery in the where clause, such as a

[jira] [Updated] (SPARK-19730) Predicate Subqueries do not push results of subqueries to data source

2017-02-24 Thread Shawn Lavelle (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Lavelle updated SPARK-19730: -- Description: When a SparkSQL query contains a subquery in the where clause, such as a

[jira] [Created] (SPARK-19731) IN Operator should support arrays

2017-02-24 Thread Shawn Lavelle (JIRA)
Shawn Lavelle created SPARK-19731: - Summary: IN Operator should support arrays Key: SPARK-19731 URL: https://issues.apache.org/jira/browse/SPARK-19731 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-19597) ExecutorSuite should have test for tasks that are not deserialiazable

2017-02-24 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-19597. Resolution: Fixed Fix Version/s: 2.2.0 > ExecutorSuite should have test for tasks

[jira] [Updated] (SPARK-14503) spark.ml Scala API for FPGrowth

2017-02-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14503: -- Shepherd: Joseph K. Bradley (was: Nick Pentreath) > spark.ml Scala API for FPGrowth >

[jira] [Comment Edited] (SPARK-19732) DataFrame.fillna() does not work for bools in PySpark

2017-02-24 Thread Len Frodgers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883522#comment-15883522 ] Len Frodgers edited comment on SPARK-19732 at 2/24/17 9:12 PM: --- Actually

[jira] [Commented] (SPARK-19732) DataFrame.fillna() does not work for bools in PySpark

2017-02-24 Thread Len Frodgers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883522#comment-15883522 ] Len Frodgers commented on SPARK-19732: -- Actually there's another anomaly: Spark (and pyspark)

[jira] [Commented] (SPARK-19715) Option to Strip Paths in FileSource

2017-02-24 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883642#comment-15883642 ] Michael Armbrust commented on SPARK-19715: -- This isn't a hypothetical. A user of structured

[jira] [Created] (SPARK-19732) DataFrame.fillna() does not work for bools in PySpark

2017-02-24 Thread Len Frodgers (JIRA)
Len Frodgers created SPARK-19732: Summary: DataFrame.fillna() does not work for bools in PySpark Key: SPARK-19732 URL: https://issues.apache.org/jira/browse/SPARK-19732 Project: Spark Issue

[jira] [Created] (SPARK-19735) Remove HOLD_DDLTIME from Catalog APIs

2017-02-24 Thread Xiao Li (JIRA)
Xiao Li created SPARK-19735: --- Summary: Remove HOLD_DDLTIME from Catalog APIs Key: SPARK-19735 URL: https://issues.apache.org/jira/browse/SPARK-19735 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-19735) Remove HOLD_DDLTIME from Catalog APIs

2017-02-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884012#comment-15884012 ] Apache Spark commented on SPARK-19735: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19735) Remove HOLD_DDLTIME from Catalog APIs

2017-02-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19735: Assignee: Xiao Li (was: Apache Spark) > Remove HOLD_DDLTIME from Catalog APIs >

  1   2   >