[jira] [Commented] (SPARK-19354) Killed tasks are getting marked as FAILED

2017-01-24 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837346#comment-15837346 ] Devaraj K commented on SPARK-19354: --- Thanks [~uncleGen] for the comment. Here the error occurred during

[jira] [Updated] (SPARK-19359) partition path created by Hive should be deleted after rename a partition with upper-case

2017-01-24 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jun updated SPARK-19359: - Issue Type: Improvement (was: Bug) > partition path created by Hive should be deleted after rename a

[jira] [Assigned] (SPARK-19359) partition path created by Hive should be deleted after rename a partition with upper-case

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19359: Assignee: Apache Spark > partition path created by Hive should be deleted after rename a

[jira] [Assigned] (SPARK-19359) partition path created by Hive should be deleted after rename a partition with upper-case

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19359: Assignee: (was: Apache Spark) > partition path created by Hive should be deleted

[jira] [Commented] (SPARK-19359) partition path created by Hive should be deleted after rename a partition with upper-case

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837343#comment-15837343 ] Apache Spark commented on SPARK-19359: -- User 'windpiger' has created a pull request for this issue:

[jira] [Created] (SPARK-19359) partition path created by Hive should be deleted after rename a partition with upper-case

2017-01-24 Thread Song Jun (JIRA)
Song Jun created SPARK-19359: Summary: partition path created by Hive should be deleted after rename a partition with upper-case Key: SPARK-19359 URL: https://issues.apache.org/jira/browse/SPARK-19359

[jira] [Commented] (SPARK-19147) netty throw NPE

2017-01-24 Thread Genmao Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837333#comment-15837333 ] Genmao Yu commented on SPARK-19147: --- After dig into code, this issue may occurs when executor is

[jira] [Comment Edited] (SPARK-19147) netty throw NPE

2017-01-24 Thread Genmao Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837333#comment-15837333 ] Genmao Yu edited comment on SPARK-19147 at 1/25/17 7:39 AM: After dig into

[jira] [Commented] (SPARK-18710) Add offset to GeneralizedLinearRegression models

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837306#comment-15837306 ] Apache Spark commented on SPARK-18710: -- User 'actuaryzhang' has created a pull request for this

[jira] [Assigned] (SPARK-18710) Add offset to GeneralizedLinearRegression models

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18710: Assignee: Apache Spark (was: Wayne Zhang) > Add offset to GeneralizedLinearRegression

[jira] [Assigned] (SPARK-18710) Add offset to GeneralizedLinearRegression models

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18710: Assignee: Wayne Zhang (was: Apache Spark) > Add offset to GeneralizedLinearRegression

[jira] [Commented] (SPARK-19256) Hive bucketing support

2017-01-24 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837302#comment-15837302 ] Tejas Patil commented on SPARK-19256: - BTW: In its current state, Spark writes data to hive bucketed

[jira] [Commented] (SPARK-19122) Unnecessary shuffle+sort added if join predicates ordering differ from bucketing and sorting order

2017-01-24 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837300#comment-15837300 ] Tejas Patil commented on SPARK-19122: - [~hvanhovell] : ping !! If you are busy, can you suggest

[jira] [Reopened] (SPARK-18710) Add offset to GeneralizedLinearRegression models

2017-01-24 Thread Wayne Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wayne Zhang reopened SPARK-18710: - > Add offset to GeneralizedLinearRegression models >

[jira] [Commented] (SPARK-19256) Hive bucketing support

2017-01-24 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837294#comment-15837294 ] Tejas Patil commented on SPARK-19256: - [~cloud_fan] [~hvanhovell] : Are you guys ok with the proposal

[jira] [Commented] (SPARK-16046) Add Aggregations Section to Spark SQL programming guide

2017-01-24 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837259#comment-15837259 ] Xiao Li commented on SPARK-16046: - I updated the JIRA description based on what the PR did. Please open a

[jira] [Updated] (SPARK-16046) Add Aggregations Section to Spark SQL programming guide

2017-01-24 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-16046: Assignee: Anton Okolnychyi > Add Aggregations Section to Spark SQL programming guide >

[jira] [Updated] (SPARK-16046) Add examples of aggregates

2017-01-24 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-16046: Description: - A separate subsection for Aggregations under “Getting Started” in the Spark SQL

[jira] [Updated] (SPARK-16046) Add Aggregations Section to Spark SQL programming guide

2017-01-24 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-16046: Summary: Add Aggregations Section to Spark SQL programming guide (was: Add examples of aggregates) > Add

[jira] [Updated] (SPARK-16046) Add examples of aggregates

2017-01-24 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-16046: Summary: Add examples of aggregates (was: Add Spark SQL Dataset Tutorial) > Add examples of aggregates >

[jira] [Resolved] (SPARK-16046) Add Spark SQL Dataset Tutorial

2017-01-24 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-16046. - Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 Issue resolved by pull request

[jira] [Commented] (SPARK-10924) Failed to update accumulators for ShuffleMapTask: Broken pipe

2017-01-24 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837249#comment-15837249 ] Hyukjin Kwon commented on SPARK-10924: -- [~ptallada], Would this be possible to provide a

[jira] [Commented] (SPARK-19358) LiveListenerBus shall log the event name when dropping them due to a fully filled queue

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837139#comment-15837139 ] Apache Spark commented on SPARK-19358: -- User 'CodingCat' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19358) LiveListenerBus shall log the event name when dropping them due to a fully filled queue

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19358: Assignee: (was: Apache Spark) > LiveListenerBus shall log the event name when

[jira] [Assigned] (SPARK-19358) LiveListenerBus shall log the event name when dropping them due to a fully filled queue

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19358: Assignee: Apache Spark > LiveListenerBus shall log the event name when dropping them due

[jira] [Created] (SPARK-19358) LiveListenerBus shall log the event name when dropping them due to a fully filled queue

2017-01-24 Thread Nan Zhu (JIRA)
Nan Zhu created SPARK-19358: --- Summary: LiveListenerBus shall log the event name when dropping them due to a fully filled queue Key: SPARK-19358 URL: https://issues.apache.org/jira/browse/SPARK-19358

[jira] [Assigned] (SPARK-19350) Cardinality estimation of Limit and Sample

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19350: Assignee: Apache Spark > Cardinality estimation of Limit and Sample >

[jira] [Updated] (SPARK-19350) Cardinality estimation of Limit and Sample

2017-01-24 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-19350: - Description: Currently, LocalLimit/GlobalLimit/Sample propagates the same row count and column

[jira] [Assigned] (SPARK-19350) Cardinality estimation of Limit and Sample

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19350: Assignee: (was: Apache Spark) > Cardinality estimation of Limit and Sample >

[jira] [Commented] (SPARK-19350) Cardinality estimation of Limit and Sample

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837081#comment-15837081 ] Apache Spark commented on SPARK-19350: -- User 'wzhfy' has created a pull request for this issue:

[jira] [Commented] (SPARK-19354) Killed tasks are getting marked as FAILED

2017-01-24 Thread Genmao Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837048#comment-15837048 ] Genmao Yu commented on SPARK-19354: --- IMHO, the killed tasks will be failed finally, so there is no

[jira] [Commented] (SPARK-10141) Number of tasks on executors still become negative after failures

2017-01-24 Thread Genmao Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837043#comment-15837043 ] Genmao Yu commented on SPARK-10141: --- I think this is fix in https://github.com/apache/spark/pull/14969,

[jira] [Commented] (SPARK-19356) Number of active tasks is negative even when there is no failed executor

2017-01-24 Thread Genmao Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837041#comment-15837041 ] Genmao Yu commented on SPARK-19356: --- I think this is fix in https://github.com/apache/spark/pull/14969,

[jira] [Issue Comment Deleted] (SPARK-18563) mapWithState: initialState should have a timeout setting per record

2017-01-24 Thread Genmao Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Genmao Yu updated SPARK-18563: -- Comment: was deleted (was: I do not know is there any plan to add new feature to DStreams? Maybe, we

[jira] [Closed] (SPARK-19343) Do once optimistic checkpoint before stop

2017-01-24 Thread Genmao Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Genmao Yu closed SPARK-19343. - Resolution: Won't Fix > Do once optimistic checkpoint before stop >

[jira] [Updated] (SPARK-19357) Parallel Model Evaluation for ML Tuning

2017-01-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-19357: - Summary: Parallel Model Evaluation for ML Tuning (was: Parallel Model Evaluation for ML

[jira] [Commented] (SPARK-19357) Parallel Model Evaluation for ML Pipeline Tuning

2017-01-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836977#comment-15836977 ] Bryan Cutler commented on SPARK-19357: -- I'm working on this > Parallel Model Evaluation for ML

[jira] [Created] (SPARK-19357) Parallel Model Evaluation for ML Pipeline Tuning

2017-01-24 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-19357: Summary: Parallel Model Evaluation for ML Pipeline Tuning Key: SPARK-19357 URL: https://issues.apache.org/jira/browse/SPARK-19357 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19071) Optimizations for ML Pipeline Tuning

2017-01-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836958#comment-15836958 ] Bryan Cutler commented on SPARK-19071: -- [~cyp] thanks for your interest. I agree with Nick that

[jira] [Commented] (SPARK-19071) Optimizations for ML Pipeline Tuning

2017-01-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836957#comment-15836957 ] Bryan Cutler commented on SPARK-19071: -- Thanks for the comments [~josephkb] and [~mlnick]. I'll

[jira] [Updated] (SPARK-19350) Cardinality estimation of Limit and Sample

2017-01-24 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-19350: - Summary: Cardinality estimation of Limit and Sample (was: Improve cardinality estimation of

[jira] [Assigned] (SPARK-19277) YARN topology script configuration needs to be localized by Spark

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19277: Assignee: (was: Apache Spark) > YARN topology script configuration needs to be

[jira] [Assigned] (SPARK-19277) YARN topology script configuration needs to be localized by Spark

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19277: Assignee: Apache Spark > YARN topology script configuration needs to be localized by

[jira] [Commented] (SPARK-19277) YARN topology script configuration needs to be localized by Spark

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836951#comment-15836951 ] Apache Spark commented on SPARK-19277: -- User 'vanzin' has created a pull request for this issue:

[jira] [Updated] (SPARK-19356) Number of active tasks is negative even when there is no failed executor

2017-01-24 Thread Lan Jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lan Jiang updated SPARK-19356: -- Attachment: Screen Shot 2017-01-24 at 4.39.09 PM.png > Number of active tasks is negative even when

[jira] [Created] (SPARK-19356) Number of active tasks is negative even when there is no failed executor

2017-01-24 Thread Lan Jiang (JIRA)
Lan Jiang created SPARK-19356: - Summary: Number of active tasks is negative even when there is no failed executor Key: SPARK-19356 URL: https://issues.apache.org/jira/browse/SPARK-19356 Project: Spark

[jira] [Resolved] (SPARK-19330) Also show tooltip for successful batches

2017-01-24 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19330. -- Resolution: Fixed Assignee: Liwei Lin Fix Version/s: 2.2.0

[jira] [Assigned] (SPARK-19355) Use map output statistices to improve global limit's parallelism

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19355: Assignee: Apache Spark > Use map output statistices to improve global limit's parallelism

[jira] [Created] (SPARK-19355) Use map output statistices to improve global limit's parallelism

2017-01-24 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-19355: --- Summary: Use map output statistices to improve global limit's parallelism Key: SPARK-19355 URL: https://issues.apache.org/jira/browse/SPARK-19355 Project:

[jira] [Assigned] (SPARK-19355) Use map output statistices to improve global limit's parallelism

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19355: Assignee: (was: Apache Spark) > Use map output statistices to improve global limit's

[jira] [Commented] (SPARK-19355) Use map output statistices to improve global limit's parallelism

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836924#comment-15836924 ] Apache Spark commented on SPARK-19355: -- User 'viirya' has created a pull request for this issue:

[jira] [Updated] (SPARK-13534) Implement Apache Arrow serializer for Spark DataFrame for use in DataFrame.toPandas

2017-01-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-13534: - Attachment: benchmark.py Script for benchmarks > Implement Apache Arrow serializer for Spark

[jira] [Created] (SPARK-19354) Killed tasks are getting marked as FAILED

2017-01-24 Thread Devaraj K (JIRA)
Devaraj K created SPARK-19354: - Summary: Killed tasks are getting marked as FAILED Key: SPARK-19354 URL: https://issues.apache.org/jira/browse/SPARK-19354 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-17436) dataframe.write sometimes does not keep sorting

2017-01-24 Thread Jason Moore (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836847#comment-15836847 ] Jason Moore commented on SPARK-17436: - Ahh, I think you are correct. The issue on the write seems to

[jira] [Commented] (SPARK-19334) Fix the code injection vulnerability related to Generator functions.

2017-01-24 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836841#comment-15836841 ] Kousuke Saruta commented on SPARK-19334: [~hvanhovell] Thanks. The affects version was also

[jira] [Updated] (SPARK-19334) Fix the code injection vulnerability related to Generator functions.

2017-01-24 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-19334: --- Affects Version/s: (was: 2.1.0) 2.2.0 > Fix the code injection

[jira] [Updated] (SPARK-19334) Fix the code injection vulnerability related to Generator functions.

2017-01-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-19334: -- Assignee: Kousuke Saruta > Fix the code injection vulnerability related to Generator

[jira] [Commented] (SPARK-19334) Fix the code injection vulnerability related to Generator functions.

2017-01-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836770#comment-15836770 ] Herman van Hovell commented on SPARK-19334: --- I have set the target version to 2.2, since we

[jira] [Comment Edited] (SPARK-19334) Fix the code injection vulnerability related to Generator functions.

2017-01-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836770#comment-15836770 ] Herman van Hovell edited comment on SPARK-19334 at 1/24/17 10:37 PM: -

[jira] [Resolved] (SPARK-19334) Fix the code injection vulnerability related to Generator functions.

2017-01-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19334. --- Resolution: Fixed Fix Version/s: 2.2.0 Target Version/s: 2.2.0

[jira] [Resolved] (SPARK-19017) NOT IN subquery with more than one column may return incorrect results

2017-01-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19017. --- Resolution: Fixed Assignee: Nattavut Sutyanyong Fix Version/s: 2.2.0

[jira] [Comment Edited] (SPARK-19352) Sorting issues on relatively big datasets

2017-01-24 Thread Ivan Gozali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836653#comment-15836653 ] Ivan Gozali edited comment on SPARK-19352 at 1/24/17 9:22 PM: -- Does this

[jira] [Commented] (SPARK-19352) Sorting issues on relatively big datasets

2017-01-24 Thread Ivan Gozali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836653#comment-15836653 ] Ivan Gozali commented on SPARK-19352: - Does this mean that {{Dataset.write.partitionBy()}} performs a

[jira] [Created] (SPARK-19353) Support binary I/O in PipedRDD

2017-01-24 Thread Sergei Lebedev (JIRA)
Sergei Lebedev created SPARK-19353: -- Summary: Support binary I/O in PipedRDD Key: SPARK-19353 URL: https://issues.apache.org/jira/browse/SPARK-19353 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-19340) Opening a file in CSV format will result in an exception if the filename contains special characters

2017-01-24 Thread Reza Safi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836629#comment-15836629 ] Reza Safi edited comment on SPARK-19340 at 1/24/17 9:03 PM: [~jayadevan.m]

[jira] [Commented] (SPARK-19340) Opening a file in CSV format will result in an exception if the filename contains special characters

2017-01-24 Thread Reza Safi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836629#comment-15836629 ] Reza Safi commented on SPARK-19340: --- [~jayadevan.m] You can have does file names in hadoop. In fact the

[jira] [Commented] (SPARK-19352) Sorting issues on relatively big datasets

2017-01-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836626#comment-15836626 ] Sean Owen commented on SPARK-19352: --- You repartition by userID after sorting -- is that not probably

[jira] [Updated] (SPARK-19340) Opening a file in CSV format will result in an exception if the filename contains special characters

2017-01-24 Thread Reza Safi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated SPARK-19340: -- Description: If you want to open a file that its name is like {noformat} "*{*}*.*" {noformat} or

[jira] [Created] (SPARK-19352) Sorting issues on relatively big datasets

2017-01-24 Thread Ivan Gozali (JIRA)
Ivan Gozali created SPARK-19352: --- Summary: Sorting issues on relatively big datasets Key: SPARK-19352 URL: https://issues.apache.org/jira/browse/SPARK-19352 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-15573) Backwards-compatible persistence for spark.ml

2017-01-24 Thread Asher Krim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836604#comment-15836604 ] Asher Krim edited comment on SPARK-15573 at 1/24/17 8:41 PM: - Thanks for your

[jira] [Commented] (SPARK-15573) Backwards-compatible persistence for spark.ml

2017-01-24 Thread Asher Krim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836604#comment-15836604 ] Asher Krim commented on SPARK-15573: Thanks for your comment [~jkbradley] What I mean by "coupling"

[jira] [Commented] (SPARK-11471) Improve the way that we plan shuffled join

2017-01-24 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836547#comment-15836547 ] Andrew Ash commented on SPARK-11471: [~yhuai] I'm interested in helping make progress on this -- it's

[jira] [Commented] (SPARK-19336) LinearSVC Python API

2017-01-24 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836523#comment-15836523 ] Miao Wang commented on SPARK-19336: --- [~mlnick] Thanks! > LinearSVC Python API > >

[jira] [Updated] (SPARK-19340) Opening a file in CSV format will result in an exception if the filename contains special characters

2017-01-24 Thread Reza Safi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated SPARK-19340: -- Priority: Minor (was: Major) > Opening a file in CSV format will result in an exception if the

[jira] [Commented] (SPARK-19336) LinearSVC Python API

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836515#comment-15836515 ] Apache Spark commented on SPARK-19336: -- User 'wangmiao1981' has created a pull request for this

[jira] [Assigned] (SPARK-19336) LinearSVC Python API

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19336: Assignee: Miao Wang (was: Apache Spark) > LinearSVC Python API > >

[jira] [Assigned] (SPARK-19336) LinearSVC Python API

2017-01-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19336: Assignee: Apache Spark (was: Miao Wang) > LinearSVC Python API > >

[jira] [Updated] (SPARK-19340) Opening a file in CSV format will result in an exception if the filename contains special characters

2017-01-24 Thread Reza Safi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated SPARK-19340: -- Description: If you want to open a file that its name is like {noformat} "*{*}*.*" {noformat} or

[jira] [Commented] (SPARK-19340) Opening a file in CSV format will result in an exception if the filename contains special characters

2017-01-24 Thread Reza Safi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836489#comment-15836489 ] Reza Safi commented on SPARK-19340: --- [~hyukjin.kwon] I updated the description and the way to

[jira] [Updated] (SPARK-19340) Opening a file in CSV format will result in an exception if the filename contains special characters

2017-01-24 Thread Reza Safi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated SPARK-19340: -- Description: If you want to open a file that its name is like {noformat} "*{*}*.*" {noformat} or

[jira] [Updated] (SPARK-17913) Filter/join expressions can return incorrect results when comparing strings to longs

2017-01-24 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-17913: Assignee: Wenchen Fan > Filter/join expressions can return incorrect results when comparing strings > to

[jira] [Resolved] (SPARK-18036) Decision Trees do not handle edge cases

2017-01-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-18036. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16377

[jira] [Resolved] (SPARK-17913) Filter/join expressions can return incorrect results when comparing strings to longs

2017-01-24 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-17913. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 15880

[jira] [Updated] (SPARK-18036) Decision Trees do not handle edge cases

2017-01-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18036: -- Assignee: Ilya Matiach > Decision Trees do not handle edge cases >

[jira] [Resolved] (SPARK-19139) AES-based authentication mechanism for Spark

2017-01-24 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19139. -- Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.2.0 > AES-based

[jira] [Resolved] (SPARK-10651) Flaky test: BroadcastSuite

2017-01-24 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-10651. -- Resolution: Fixed Fix Version/s: 2.2.0 The root cause of the failure is SPARK-17755,

[jira] [Commented] (SPARK-12339) NullPointerException on stage kill from web UI

2017-01-24 Thread Shashank Mandil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836463#comment-15836463 ] Shashank Mandil commented on SPARK-12339: - [~andrewor14] Just checked this also affects 1.6

[jira] [Commented] (SPARK-17248) Add native Scala enum support to Dataset Encoders

2017-01-24 Thread Leif Warner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836374#comment-15836374 ] Leif Warner commented on SPARK-17248: - The spark documentation itself says "Consider using numeric