[jira] [Commented] (SPARK-25232) Support Full-Text Search in Spark SQL

2018-08-25 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592802#comment-16592802 ] Liang-Chi Hsieh commented on SPARK-25232: - This looks to me more like a specific datasource that

[jira] [Commented] (SPARK-23836) Support returning StructType to the level support in GroupedMap Arrow's "scalar" UDFS (or similar)

2018-08-25 Thread Andrew Malone Melo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592751#comment-16592751 ] Andrew Malone Melo commented on SPARK-23836: [~bryanc] - I think it could be either that or

[jira] [Resolved] (SPARK-25199) InferSchema "all Strings" if one of many CSVs is empty

2018-08-25 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk resolved SPARK-25199. Resolution: Cannot Reproduce > InferSchema "all Strings" if one of many CSVs is empty >

[jira] [Commented] (SPARK-25199) InferSchema "all Strings" if one of many CSVs is empty

2018-08-25 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592704#comment-16592704 ] Maxim Gekk commented on SPARK-25199: I wasn't able to reproduce the issue on the current master:

[jira] [Created] (SPARK-25242) Suggestion to make sql config setting fluent

2018-08-25 Thread Florence Hope (JIRA)
Florence Hope created SPARK-25242: - Summary: Suggestion to make sql config setting fluent Key: SPARK-25242 URL: https://issues.apache.org/jira/browse/SPARK-25242 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-25241) Configurable empty values when reading/writing CSV files

2018-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25241: Assignee: (was: Apache Spark) > Configurable empty values when reading/writing CSV

[jira] [Commented] (SPARK-25241) Configurable empty values when reading/writing CSV files

2018-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592667#comment-16592667 ] Apache Spark commented on SPARK-25241: -- User 'mmolimar' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25241) Configurable empty values when reading/writing CSV files

2018-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25241: Assignee: Apache Spark > Configurable empty values when reading/writing CSV files >

[jira] [Updated] (SPARK-25241) Configurable empty values when reading/writing CSV files

2018-08-25 Thread Mario Molina (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mario Molina updated SPARK-25241: - Component/s: (was: Input/Output) SQL > Configurable empty values when

[jira] [Created] (SPARK-25241) Configurable empty values when reading/writing CSV files

2018-08-25 Thread Mario Molina (JIRA)
Mario Molina created SPARK-25241: Summary: Configurable empty values when reading/writing CSV files Key: SPARK-25241 URL: https://issues.apache.org/jira/browse/SPARK-25241 Project: Spark

[jira] [Commented] (SPARK-25165) Cannot parse Hive Struct

2018-08-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592633#comment-16592633 ] Hyukjin Kwon commented on SPARK-25165: -- Mind if I ask how you created a Hive table? > Cannot parse

[jira] [Commented] (SPARK-25232) Support Full-Text Search in Spark SQL

2018-08-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592624#comment-16592624 ] Hyukjin Kwon commented on SPARK-25232: -- cc [~viirya] FYI. As far as I remember you know this bit as

[jira] [Commented] (SPARK-25232) Support Full-Text Search in Spark SQL

2018-08-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592623#comment-16592623 ] Hyukjin Kwon commented on SPARK-25232: -- Yea, this is different with RLIKE. It requires inverted

[jira] [Commented] (SPARK-25230) Upper behavior incorrect for string contains "ß"

2018-08-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592622#comment-16592622 ] Hyukjin Kwon commented on SPARK-25230: -- I think this is because we set the locale to ROOT to avoid

[jira] [Commented] (SPARK-25227) Extend functionality of to_json

2018-08-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592619#comment-16592619 ] Hyukjin Kwon commented on SPARK-25227: -- Also, let's fix the JIRA title to be more specific while we

[jira] [Commented] (SPARK-25227) Extend functionality of to_json

2018-08-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592618#comment-16592618 ] Hyukjin Kwon commented on SPARK-25227: -- Can you post a reproducer against the current master?

[jira] [Commented] (SPARK-25226) Extend functionality of from_json

2018-08-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592617#comment-16592617 ] Hyukjin Kwon commented on SPARK-25226: -- Can you post a reproducer against the current master? >

[jira] [Commented] (SPARK-25225) Add support for "List"-Type columns

2018-08-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592614#comment-16592614 ] Hyukjin Kwon commented on SPARK-25225: -- Can't you just manually cast to string first? > Add

[jira] [Commented] (SPARK-25193) insert overwrite doesn't throw exception when drop old data fails

2018-08-25 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592609#comment-16592609 ] Marco Gaido commented on SPARK-25193: - Well, this I think is HIVE-12505. So it would need to be

[jira] [Resolved] (SPARK-24688) Clarify comments about LabeledPoint as (label, feature) pair rather than (feature, label)

2018-08-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24688. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21665

[jira] [Assigned] (SPARK-24688) Clarify comments about LabeledPoint as (label, feature) pair rather than (feature, label)

2018-08-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-24688: - Assignee: Weizhe Huang > Clarify comments about LabeledPoint as (label, feature) pair rather

[jira] [Updated] (SPARK-25240) A deadlock in ALTER TABLE RECOVER PARTITIONS

2018-08-25 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-25240: --- Summary: A deadlock in ALTER TABLE RECOVER PARTITIONS (was: Dead-lock in ALTER TABLE RECOVER

[jira] [Updated] (SPARK-24688) Clarify comments about LabeledPoint as (label, feature) pair rather than (feature, label)

2018-08-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-24688: -- Summary: Clarify comments about LabeledPoint as (label, feature) pair rather than (feature, label)

[jira] [Commented] (SPARK-25240) Dead-lock in ALTER TABLE RECOVER PARTITIONS

2018-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592590#comment-16592590 ] Apache Spark commented on SPARK-25240: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25240) Dead-lock in ALTER TABLE RECOVER PARTITIONS

2018-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25240: Assignee: (was: Apache Spark) > Dead-lock in ALTER TABLE RECOVER PARTITIONS >

[jira] [Assigned] (SPARK-25240) Dead-lock in ALTER TABLE RECOVER PARTITIONS

2018-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25240: Assignee: Apache Spark > Dead-lock in ALTER TABLE RECOVER PARTITIONS >

[jira] [Created] (SPARK-25240) Dead-lock in ALTER TABLE RECOVER PARTITIONS

2018-08-25 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-25240: -- Summary: Dead-lock in ALTER TABLE RECOVER PARTITIONS Key: SPARK-25240 URL: https://issues.apache.org/jira/browse/SPARK-25240 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-25239) Spark Streaming for Kafka should allow uniform batch size per partition for streaming RDD

2018-08-25 Thread Sidhavratha Kumar (JIRA)
Sidhavratha Kumar created SPARK-25239: - Summary: Spark Streaming for Kafka should allow uniform batch size per partition for streaming RDD Key: SPARK-25239 URL:

[jira] [Assigned] (SPARK-25238) Lint-Python: Upgrading to the current version of pycodestyle fails

2018-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25238: Assignee: (was: Apache Spark) > Lint-Python: Upgrading to the current version of

[jira] [Assigned] (SPARK-25238) Lint-Python: Upgrading to the current version of pycodestyle fails

2018-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25238: Assignee: Apache Spark > Lint-Python: Upgrading to the current version of pycodestyle

[jira] [Commented] (SPARK-25238) Lint-Python: Upgrading to the current version of pycodestyle fails

2018-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592537#comment-16592537 ] Apache Spark commented on SPARK-25238: -- User 'cclauss' has created a pull request for this issue:

[jira] [Issue Comment Deleted] (SPARK-17368) Scala value classes create encoder problems and break at runtime

2018-08-25 Thread Minh Thai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Minh Thai updated SPARK-17368: -- Comment: was deleted (was: [~jodersky] I know that this is an old ticket but I still want to give

[jira] [Updated] (SPARK-25238) Lint-Python: Upgrading to the current version of pycodestyle fails

2018-08-25 Thread cclauss (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cclauss updated SPARK-25238: Description: See https://github.com/apache/spark/pull/22231

[jira] [Updated] (SPARK-25238) Lint-Python: Upgrading to the current version of pycodestyle fails

2018-08-25 Thread cclauss (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cclauss updated SPARK-25238: Description: See https://github.com/apache/spark/pull/22231

[jira] [Updated] (SPARK-25237) FileScanRdd's inputMetrics is wrong when select the datasource table with limit

2018-08-25 Thread du (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] du updated SPARK-25237: --- Description: In FileScanRdd, we will update inputMetrics's bytesRead using updateBytesRead  every 1000 rows or

[jira] [Assigned] (SPARK-25237) FileScanRdd's inputMetrics is wrong when select the datasource table with limit

2018-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25237: Assignee: Apache Spark > FileScanRdd's inputMetrics is wrong when select the datasource

[jira] [Commented] (SPARK-25237) FileScanRdd's inputMetrics is wrong when select the datasource table with limit

2018-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592502#comment-16592502 ] Apache Spark commented on SPARK-25237: -- User 'dujunling' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25237) FileScanRdd's inputMetrics is wrong when select the datasource table with limit

2018-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25237: Assignee: (was: Apache Spark) > FileScanRdd's inputMetrics is wrong when select the

[jira] [Created] (SPARK-25237) FileScanRdd's inputMetrics is wrong when select the datasource table with limit

2018-08-25 Thread du (JIRA)
du created SPARK-25237: -- Summary: FileScanRdd's inputMetrics is wrong when select the datasource table with limit Key: SPARK-25237 URL: https://issues.apache.org/jira/browse/SPARK-25237 Project: Spark

[jira] [Created] (SPARK-25238) Lint-Python: Upgrading to the current version of pycodestyle fails

2018-08-25 Thread cclauss (JIRA)
cclauss created SPARK-25238: --- Summary: Lint-Python: Upgrading to the current version of pycodestyle fails Key: SPARK-25238 URL: https://issues.apache.org/jira/browse/SPARK-25238 Project: Spark