[jira] [Updated] (SPARK-19674) Ignore driver accumulator updates don't belong to the execution when merging all accumulator updates

2017-02-23 Thread Carson Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carson Wang updated SPARK-19674: Summary: Ignore driver accumulator updates don't belong to the execution when merging all accumulat

[jira] [Updated] (SPARK-19674) Ignore driver accumulator updates don't belong to the execution when merging all accumulator updates

2017-02-23 Thread Carson Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carson Wang updated SPARK-19674: Description: In SQLListener.getExecutionMetrics, driver accumulator updates don't belong to the exe

[jira] [Updated] (SPARK-18832) Spark SQL: thiriftserver unable to run a registered Hive UDTF

2017-02-23 Thread Lokesh Yadav (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Yadav updated SPARK-18832: - Environment: HDP: 2.5 Spark: 2.0.0 Description: Spark Thriftserver is unable to run a Hive

[jira] [Commented] (SPARK-18832) Spark SQL: thiriftserver unable to run a registered Hive UDTF

2017-02-23 Thread Lokesh Yadav (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880107#comment-15880107 ] Lokesh Yadav commented on SPARK-18832: -- Hi Dongjoon Actually the interface I was usi

[jira] [Issue Comment Deleted] (SPARK-18832) Spark SQL: thiriftserver unable to run a registered Hive UDTF

2017-02-23 Thread Lokesh Yadav (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Yadav updated SPARK-18832: - Comment: was deleted (was: This is the code for the sample UDTF that I am using to test: {{ pac

[jira] [Commented] (SPARK-18832) Spark SQL: thiriftserver unable to run a registered Hive UDTF

2017-02-23 Thread Lokesh Yadav (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880108#comment-15880108 ] Lokesh Yadav commented on SPARK-18832: -- This is the code for the sample UDTF that I

[jira] [Updated] (SPARK-18832) Spark SQL: thiriftserver unable to run a registered Hive UDTF

2017-02-23 Thread Lokesh Yadav (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Yadav updated SPARK-18832: - Attachment: SampleUDTF.java > Spark SQL: thiriftserver unable to run a registered Hive UDTF > ---

[jira] [Updated] (SPARK-18832) Spark SQL: Thriftserver unable to run a registered Hive UDTF

2017-02-23 Thread Lokesh Yadav (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Yadav updated SPARK-18832: - Summary: Spark SQL: Thriftserver unable to run a registered Hive UDTF (was: Spark SQL: thiriftse

[jira] [Commented] (SPARK-18832) Spark SQL: thiriftserver unable to run a registered Hive UDTF

2017-02-23 Thread Lokesh Yadav (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880110#comment-15880110 ] Lokesh Yadav commented on SPARK-18832: -- I have also attached the code for the Sample

[jira] [Updated] (SPARK-18832) Spark SQL: Thriftserver unable to run a registered Hive UDTF

2017-02-23 Thread Lokesh Yadav (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Yadav updated SPARK-18832: - Description: Spark Thriftserver is unable to run a HiveUDTF. It throws the error that it is unabl

[jira] [Updated] (SPARK-18832) Spark SQL: Thriftserver unable to run a registered Hive UDTF

2017-02-23 Thread Lokesh Yadav (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Yadav updated SPARK-18832: - Description: Spark Thriftserver is unable to run a HiveUDTF. It throws the error that it is unabl

[jira] [Commented] (SPARK-19668) Multiple NGram sizes

2017-02-23 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880131#comment-15880131 ] Nick Pentreath commented on SPARK-19668: The simplest will be to keep the existin

[jira] [Created] (SPARK-19707) Improve the invalid path handling for sc.addJar

2017-02-23 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-19707: --- Summary: Improve the invalid path handling for sc.addJar Key: SPARK-19707 URL: https://issues.apache.org/jira/browse/SPARK-19707 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-19707) Improve the invalid path check for sc.addJar

2017-02-23 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-19707: Summary: Improve the invalid path check for sc.addJar (was: Improve the invalid path handling for

[jira] [Commented] (SPARK-19707) Improve the invalid path check for sc.addJar

2017-02-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880175#comment-15880175 ] Apache Spark commented on SPARK-19707: -- User 'jerryshao' has created a pull request

[jira] [Assigned] (SPARK-19707) Improve the invalid path check for sc.addJar

2017-02-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19707: Assignee: (was: Apache Spark) > Improve the invalid path check for sc.addJar > ---

[jira] [Assigned] (SPARK-19707) Improve the invalid path check for sc.addJar

2017-02-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19707: Assignee: Apache Spark > Improve the invalid path check for sc.addJar > --

[jira] [Created] (SPARK-19708) delete jar unable

2017-02-23 Thread backdoor_sunlight (JIRA)
backdoor_sunlight created SPARK-19708: - Summary: delete jar unable Key: SPARK-19708 URL: https://issues.apache.org/jira/browse/SPARK-19708 Project: Spark Issue Type: Bug Compone

[jira] [Created] (SPARK-19709) CSV datasource fails to read empty file

2017-02-23 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-19709: Summary: CSV datasource fails to read empty file Key: SPARK-19709 URL: https://issues.apache.org/jira/browse/SPARK-19709 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-19709) CSV datasource fails to read empty file

2017-02-23 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880241#comment-15880241 ] Hyukjin Kwon commented on SPARK-19709: -- Let me fix this soon. > CSV datasource fail

[jira] [Commented] (SPARK-18113) Sending AskPermissionToCommitOutput failed, driver enter into task deadloop

2017-02-23 Thread xukun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880279#comment-15880279 ] xukun commented on SPARK-18113: --- [~jinxing6042[~jinxing6...@126.com] When appmaster get com

[jira] [Commented] (SPARK-14409) Investigate adding a RankingEvaluator to ML

2017-02-23 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880312#comment-15880312 ] Nick Pentreath commented on SPARK-14409: [~danilo.ascione] Yes, your solution is

[jira] [Commented] (SPARK-14409) Investigate adding a RankingEvaluator to ML

2017-02-23 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880324#comment-15880324 ] Nick Pentreath commented on SPARK-14409: [~roberto.mirizzi] If using the current

[jira] [Updated] (SPARK-18832) Spark SQL: Thriftserver unable to run a registered Hive UDTF

2017-02-23 Thread Lokesh Yadav (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Yadav updated SPARK-18832: - Description: Spark Thriftserver is unable to run a HiveUDTF. It throws the error that it is unabl

[jira] [Created] (SPARK-19710) Test Failures in SQLQueryTests on big endian platforms

2017-02-23 Thread Pete Robbins (JIRA)
Pete Robbins created SPARK-19710: Summary: Test Failures in SQLQueryTests on big endian platforms Key: SPARK-19710 URL: https://issues.apache.org/jira/browse/SPARK-19710 Project: Spark Issue

[jira] [Commented] (SPARK-8480) Add setName for Dataframe

2017-02-23 Thread Emlyn Corrin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880338#comment-15880338 ] Emlyn Corrin commented on SPARK-8480: - If anyone just wants a way to identify the RDDs

[jira] [Assigned] (SPARK-19710) Test Failures in SQLQueryTests on big endian platforms

2017-02-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19710: Assignee: (was: Apache Spark) > Test Failures in SQLQueryTests on big endian platforms

[jira] [Assigned] (SPARK-19710) Test Failures in SQLQueryTests on big endian platforms

2017-02-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19710: Assignee: Apache Spark > Test Failures in SQLQueryTests on big endian platforms >

[jira] [Commented] (SPARK-19710) Test Failures in SQLQueryTests on big endian platforms

2017-02-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880342#comment-15880342 ] Apache Spark commented on SPARK-19710: -- User 'robbinspg' has created a pull request

[jira] [Comment Edited] (SPARK-18113) Sending AskPermissionToCommitOutput failed, driver enter into task deadloop

2017-02-23 Thread xukun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880279#comment-15880279 ] xukun edited comment on SPARK-18113 at 2/23/17 12:34 PM: - [~jinxi

[jira] [Comment Edited] (SPARK-18113) Sending AskPermissionToCommitOutput failed, driver enter into task deadloop

2017-02-23 Thread xukun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880279#comment-15880279 ] xukun edited comment on SPARK-18113 at 2/23/17 12:35 PM: - [~jinxi

[jira] [Commented] (SPARK-18813) MLlib 2.2 Roadmap

2017-02-23 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880387#comment-15880387 ] Nick Pentreath commented on SPARK-18813: Thanks for this Joseph and everyone for

[jira] [Resolved] (SPARK-19708) delete jar unable

2017-02-23 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-19708. -- Resolution: Duplicate > delete jar unable > - > > Key: SPARK-19

[jira] [Commented] (SPARK-6072) Enable hash joins for null-safe equality predicates

2017-02-23 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880456#comment-15880456 ] Hyukjin Kwon commented on SPARK-6072: - Hi [~dimazhiyanov], could you confirm ^ please?

[jira] [Commented] (SPARK-6678) select count(DISTINCT C_UID) from parquetdir may be can optimize

2017-02-23 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880463#comment-15880463 ] Hyukjin Kwon commented on SPARK-6678: - gentle ping [~cnstar9988] > select count(DISTI

[jira] [Resolved] (SPARK-9275) IsolatedClientLoader could not load shared JNI libraries

2017-02-23 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-9275. - Resolution: Duplicate > IsolatedClientLoader could not load shared JNI libraries > ---

[jira] [Commented] (SPARK-18359) Let user specify locale in CSV parsing

2017-02-23 Thread Arkadiusz Bicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880544#comment-15880544 ] Arkadiusz Bicz commented on SPARK-18359: Can interface to specify locale looks li

[jira] [Resolved] (SPARK-12051) Can't register UDF from Hive thrift server

2017-02-23 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12051. -- Resolution: Duplicate I am resolving this as a duplicate per your comment in https://issues.ap

[jira] [Updated] (SPARK-11784) Support Timestamp filter pushdown in Parquet datasource

2017-02-23 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-11784: - Summary: Support Timestamp filter pushdown in Parquet datasource (was: enable Timestamp filter

[jira] [Comment Edited] (SPARK-12264) Add a typeTag or scalaTypeTag method to DataType

2017-02-23 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15813946#comment-15813946 ] Hyukjin Kwon edited comment on SPARK-12264 at 2/23/17 3:09 PM:

[jira] [Commented] (SPARK-19634) Feature parity for descriptive statistics in MLlib

2017-02-23 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880623#comment-15880623 ] Nick Pentreath commented on SPARK-19634: Thanks [~timhunter]. In terms of perfor

[jira] [Commented] (SPARK-19634) Feature parity for descriptive statistics in MLlib

2017-02-23 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880628#comment-15880628 ] Nick Pentreath commented on SPARK-19634: Ah I see it was discussed in the design

[jira] [Updated] (SPARK-19711) Bug in gapply function

2017-02-23 Thread Luis Felipe Sant Ana (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luis Felipe Sant Ana updated SPARK-19711: - Description: I have a dataframe in SparkR like CNPJPID DATA

[jira] [Created] (SPARK-19711) Bug in gapply function

2017-02-23 Thread Luis Felipe Sant Ana (JIRA)
Luis Felipe Sant Ana created SPARK-19711: Summary: Bug in gapply function Key: SPARK-19711 URL: https://issues.apache.org/jira/browse/SPARK-19711 Project: Spark Issue Type: Bug

[jira] [Issue Comment Deleted] (SPARK-12890) Spark SQL query related to only partition fields should not scan the whole data.

2017-02-23 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-12890: - Comment: was deleted (was: [~rxin] Could you confirm if this is an issue?) > Spark SQL query rel

[jira] [Issue Comment Deleted] (SPARK-12890) Spark SQL query related to only partition fields should not scan the whole data.

2017-02-23 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-12890: - Comment: was deleted (was: Actually I don't still understand what is an issue here. This might no

[jira] [Closed] (SPARK-16951) Alternative implementation of NOT IN to Anti-join

2017-02-23 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nattavut Sutyanyong closed SPARK-16951. --- Resolution: Won't Fix > Alternative implementation of NOT IN to Anti-join > -

[jira] [Commented] (SPARK-16951) Alternative implementation of NOT IN to Anti-join

2017-02-23 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880694#comment-15880694 ] Nattavut Sutyanyong commented on SPARK-16951: - I am going to close off this J

[jira] [Created] (SPARK-19712) EXISTS and Left Semi join do not produce the same plan

2017-02-23 Thread Nattavut Sutyanyong (JIRA)
Nattavut Sutyanyong created SPARK-19712: --- Summary: EXISTS and Left Semi join do not produce the same plan Key: SPARK-19712 URL: https://issues.apache.org/jira/browse/SPARK-19712 Project: Spark

[jira] [Updated] (SPARK-19712) EXISTS and Left Semi join do not produce the same plan

2017-02-23 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nattavut Sutyanyong updated SPARK-19712: Description: This problem was found during the development of SPARK-18874. The EXI

[jira] [Commented] (SPARK-19712) EXISTS and Left Semi join do not produce the same plan

2017-02-23 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880747#comment-15880747 ] Nattavut Sutyanyong commented on SPARK-19712: - This is because Optimizer trea

[jira] [Commented] (SPARK-19712) EXISTS and Left Semi join do not produce the same plan

2017-02-23 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880769#comment-15880769 ] Nattavut Sutyanyong commented on SPARK-19712: - Note there is a related discus

[jira] [Closed] (SPARK-5226) Add DBSCAN Clustering Algorithm to MLlib

2017-02-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-5226. Resolution: Won't Fix > Add DBSCAN Clustering Algorithm to MLlib > -

[jira] [Commented] (SPARK-5226) Add DBSCAN Clustering Algorithm to MLlib

2017-02-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880786#comment-15880786 ] Xiangrui Meng commented on SPARK-5226: -- I closed this ticket as "Won't Do" due to DBS

[jira] [Created] (SPARK-19713) saveAsTable

2017-02-23 Thread Balaram R Gadiraju (JIRA)
Balaram R Gadiraju created SPARK-19713: -- Summary: saveAsTable Key: SPARK-19713 URL: https://issues.apache.org/jira/browse/SPARK-19713 Project: Spark Issue Type: Bug Components:

[jira] [Resolved] (SPARK-19691) Calculating percentile of decimal column fails with ClassCastException

2017-02-23 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19691. --- Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.2.0 > C

[jira] [Created] (SPARK-19714) Bucketizer Bug Regarding Handling Unbucketed Inputs

2017-02-23 Thread Bill Chambers (JIRA)
Bill Chambers created SPARK-19714: - Summary: Bucketizer Bug Regarding Handling Unbucketed Inputs Key: SPARK-19714 URL: https://issues.apache.org/jira/browse/SPARK-19714 Project: Spark Issue T

[jira] [Commented] (SPARK-19711) Bug in gapply function

2017-02-23 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880980#comment-15880980 ] Felix Cheung commented on SPARK-19711: -- This could happen when the return data does

[jira] [Created] (SPARK-19715) Option to Strip Paths in FileSource

2017-02-23 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-19715: Summary: Option to Strip Paths in FileSource Key: SPARK-19715 URL: https://issues.apache.org/jira/browse/SPARK-19715 Project: Spark Issue Type: New F

[jira] [Resolved] (SPARK-19682) Issue warning (or error) when subset method "[[" takes vector index

2017-02-23 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-19682. -- Resolution: Fixed Assignee: Wayne Zhang Fix Version/s: 2.2.0

[jira] [Resolved] (SPARK-19497) dropDuplicates with watermark

2017-02-23 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-19497. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16970 [https://g

[jira] [Updated] (SPARK-18924) Improve collect/createDataFrame performance in SparkR

2017-02-23 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18924: -- Target Version/s: (was: 2.2.0) > Improve collect/createDataFrame performance in Spark

[jira] [Updated] (SPARK-17822) JVMObjectTracker.objMap may leak JVM objects

2017-02-23 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-17822: -- Target Version/s: 2.1.0, 2.0.3 (was: 2.0.3, 2.1.1, 2.2.0) > JVMObjectTracker.objMap ma

[jira] [Updated] (SPARK-18812) Clarify "Spark ML"

2017-02-23 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18812: -- Target Version/s: 2.1.0 (was: 2.1.1, 2.2.0) > Clarify "Spark ML" > --

[jira] [Updated] (SPARK-13786) Pyspark ml.tuning support export/import

2017-02-23 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-13786: -- Target Version/s: 2.3.0 (was: 2.2.0) > Pyspark ml.tuning support export/import > -

[jira] [Commented] (SPARK-18822) Support ML Pipeline in SparkR

2017-02-23 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881098#comment-15881098 ] Joseph K. Bradley commented on SPARK-18822: --- How's this going? Just checking i

[jira] [Commented] (SPARK-15571) Pipeline unit test improvements for 2.2

2017-02-23 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881102#comment-15881102 ] Joseph K. Bradley commented on SPARK-15571: --- [~rowanv] Thanks, and sorry for th

[jira] [Updated] (SPARK-15571) Pipeline unit test improvements for 2.3

2017-02-23 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15571: -- Target Version/s: 2.3.0 (was: 2.2.0) > Pipeline unit test improvements for 2.3 > -

[jira] [Updated] (SPARK-15571) Pipeline unit test improvements for 2.3

2017-02-23 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15571: -- Summary: Pipeline unit test improvements for 2.3 (was: Pipeline unit test improvements

[jira] [Updated] (SPARK-18592) Move DT/RF/GBT Param setter methods to subclasses

2017-02-23 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18592: -- Target Version/s: 2.1.0 (was: 2.1.0, 2.2.0) > Move DT/RF/GBT Param setter methods to s

[jira] [Updated] (SPARK-18618) SparkR GLM model predict should support type as a argument

2017-02-23 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18618: -- Labels: (was: 2.2.0) > SparkR GLM model predict should support type as a argument > -

[jira] [Commented] (SPARK-19711) Bug in gapply function

2017-02-23 Thread Luis Felipe Sant Ana (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881108#comment-15881108 ] Luis Felipe Sant Ana commented on SPARK-19711: -- Hi Felix, I have removed the

[jira] [Comment Edited] (SPARK-19711) Bug in gapply function

2017-02-23 Thread Luis Felipe Sant Ana (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881108#comment-15881108 ] Luis Felipe Sant Ana edited comment on SPARK-19711 at 2/23/17 7:45 PM:

[jira] [Comment Edited] (SPARK-19711) Bug in gapply function

2017-02-23 Thread Luis Felipe Sant Ana (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881108#comment-15881108 ] Luis Felipe Sant Ana edited comment on SPARK-19711 at 2/23/17 7:47 PM:

[jira] [Resolved] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

2017-02-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18699. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16928 [https://githu

[jira] [Assigned] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

2017-02-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-18699: --- Assignee: Takeshi Yamamuro > Spark CSV parsing types other than String throws exception when

[jira] [Commented] (SPARK-19459) ORC tables cannot be read when they contain char/varchar columns

2017-02-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881176#comment-15881176 ] Apache Spark commented on SPARK-19459: -- User 'hvanhovell' has created a pull request

[jira] [Updated] (SPARK-18966) NOT IN subquery with correlated expressions may return incorrect result

2017-02-23 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nattavut Sutyanyong updated SPARK-18966: Issue Type: Sub-task (was: Bug) Parent: SPARK-18455 > NOT IN subquery with

[jira] [Resolved] (SPARK-19706) add Column.contains in pyspark

2017-02-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19706. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17036 [https://githu

[jira] [Updated] (SPARK-19716) Dataset should allow by-name resolution for struct type elements in array

2017-02-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-19716: Description: if we have a DataFrame with schema {{}}, and convert it to Dataset with {{case class

[jira] [Created] (SPARK-19716) Dataset should allow by-name resolution for struct type elements in array

2017-02-23 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-19716: --- Summary: Dataset should allow by-name resolution for struct type elements in array Key: SPARK-19716 URL: https://issues.apache.org/jira/browse/SPARK-19716 Project: Spar

[jira] [Resolved] (SPARK-19684) Move info about running specific tests to developer website

2017-02-23 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-19684. Resolution: Fixed Fix Version/s: 2.2.0 > Move info about running specific tests to d

[jira] [Updated] (SPARK-19716) Dataset should allow by-name resolution for struct type elements in array

2017-02-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-19716: Description: if we have a DataFrame with schema {{a: int, b: int, c: int}}, and convert it to Data

[jira] [Created] (SPARK-19717) Expanding Spark ML under Different Namespace

2017-02-23 Thread Shouheng Yi (JIRA)
Shouheng Yi created SPARK-19717: --- Summary: Expanding Spark ML under Different Namespace Key: SPARK-19717 URL: https://issues.apache.org/jira/browse/SPARK-19717 Project: Spark Issue Type: Wish

[jira] [Updated] (SPARK-19716) Dataset should allow by-name resolution for struct type elements in array

2017-02-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-19716: Description: if we have a DataFrame with schema {{a: int, b: int, c: int}}, and convert it to Data

[jira] [Updated] (SPARK-19716) Dataset should allow by-name resolution for struct type elements in array

2017-02-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-19716: Description: if we have a DataFrame with schema {{a: int, b: int, c: int}}, and convert it to Data

[jira] [Created] (SPARK-19718) Fix flaky test: org.apache.spark.sql.kafka010.KafkaSourceStressForDontFailOnDataLossSuite: stress test for failOnDataLoss=false

2017-02-23 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-19718: Summary: Fix flaky test: org.apache.spark.sql.kafka010.KafkaSourceStressForDontFailOnDataLossSuite: stress test for failOnDataLoss=false Key: SPARK-19718 URL: https://issues.apac

[jira] [Commented] (SPARK-19717) Expanding Spark ML under Different Namespace

2017-02-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881331#comment-15881331 ] Sean Owen commented on SPARK-19717: --- I don't know that this should be a JIRA. What are

[jira] [Closed] (SPARK-19717) Expanding Spark ML under Different Namespace

2017-02-23 Thread Shouheng Yi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shouheng Yi closed SPARK-19717. --- Resolution: Fixed Duplicated issue https://issues.apache.org/jira/browse/SPARK-19498 > Expanding Spa

[jira] [Reopened] (SPARK-19717) Expanding Spark ML under Different Namespace

2017-02-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-19717: --- > Expanding Spark ML under Different Namespace > > >

[jira] [Resolved] (SPARK-19717) Expanding Spark ML under Different Namespace

2017-02-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19717. --- Resolution: Duplicate > Expanding Spark ML under Different Namespace > --

[jira] [Created] (SPARK-19719) Structured Streaming write to Kafka

2017-02-23 Thread Tyson Condie (JIRA)
Tyson Condie created SPARK-19719: Summary: Structured Streaming write to Kafka Key: SPARK-19719 URL: https://issues.apache.org/jira/browse/SPARK-19719 Project: Spark Issue Type: New Feature

[jira] [Assigned] (SPARK-19719) Structured Streaming write to Kafka

2017-02-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19719: Assignee: (was: Apache Spark) > Structured Streaming write to Kafka >

[jira] [Assigned] (SPARK-19719) Structured Streaming write to Kafka

2017-02-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19719: Assignee: Apache Spark > Structured Streaming write to Kafka > ---

[jira] [Commented] (SPARK-19719) Structured Streaming write to Kafka

2017-02-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881346#comment-15881346 ] Apache Spark commented on SPARK-19719: -- User 'tcondie' has created a pull request fo

[jira] [Commented] (SPARK-19698) Race condition in stale attempt task completion vs current attempt task completion when task is doing persistent state changes

2017-02-23 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881358#comment-15881358 ] Kay Ousterhout commented on SPARK-19698: I think this is the same issue as SPARK-

[jira] [Comment Edited] (SPARK-19698) Race condition in stale attempt task completion vs current attempt task completion when task is doing persistent state changes

2017-02-23 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881358#comment-15881358 ] Kay Ousterhout edited comment on SPARK-19698 at 2/23/17 9:57 PM: --

[jira] [Updated] (SPARK-19718) Fix flaky test: org.apache.spark.sql.kafka010.KafkaSourceStressForDontFailOnDataLossSuite: stress test for failOnDataLoss=false

2017-02-23 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19718: - Description: SPARK-19617 changed HDFSMetadataLog to enable interrupts when using the local file

[jira] [Commented] (SPARK-19263) DAGScheduler should avoid sending conflicting task set.

2017-02-23 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881406#comment-15881406 ] Kay Ousterhout commented on SPARK-19263: Just noting that this was fixed by https

[jira] [Resolved] (SPARK-14658) when executor lost DagScheduer may submit one stage twice even if the first running taskset for this stage is not finished

2017-02-23 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-14658. Resolution: Duplicate I'm fairly sure this duplicates SPARK-19263, as Mark mentioned on the

  1   2   >