[jira] [Assigned] (SPARK-28130) Pretty messages not being printed for skipped PySpark tests when xmlrunner is available

2019-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28130: Assignee: Apache Spark (was: Hyukjin Kwon) > Pretty messages not being printed for

[jira] [Assigned] (SPARK-28130) Pretty messages not being printed for skipped PySpark tests when xmlrunner is available

2019-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28130: Assignee: Hyukjin Kwon (was: Apache Spark) > Pretty messages not being printed for

[jira] [Updated] (SPARK-28130) Pretty messages not being printed for skipped PySpark tests when xmlrunner is available

2019-06-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-28130: - Component/s: (was: Project Infra) > Pretty messages not being printed for skipped PySpark

[jira] [Comment Edited] (SPARK-24634) Add a new metric regarding number of rows later than watermark

2019-06-20 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869179#comment-16869179 ] Jungtaek Lim edited comment on SPARK-24634 at 6/21/19 4:38 AM: --- I'm now

[jira] [Commented] (SPARK-24634) Add a new metric regarding number of rows later than watermark

2019-06-20 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869179#comment-16869179 ] Jungtaek Lim commented on SPARK-24634: -- I'm now revisiting this issue to alleviate SPARK-28074 -

[jira] [Assigned] (SPARK-28130) Pretty messages not being printed for skipped PySpark tests when xmlrunner is available

2019-06-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28130: Assignee: Hyukjin Kwon > Pretty messages not being printed for skipped PySpark tests

[jira] [Updated] (SPARK-28130) Pretty messages not being printed for skipped PySpark tests when xmlrunner is available

2019-06-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-28130: - Description: When {{xmlrunner}} is installed, seems {{xmlrunner}} does not respect

[jira] [Created] (SPARK-28130) Pretty messages not being printed for skipped PySpark tests when xmlrunner is available

2019-06-20 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-28130: Summary: Pretty messages not being printed for skipped PySpark tests when xmlrunner is available Key: SPARK-28130 URL: https://issues.apache.org/jira/browse/SPARK-28130

[jira] [Assigned] (SPARK-28120) RocksDB state storage

2019-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28120: Assignee: (was: Apache Spark) > RocksDB state storage > - > >

[jira] [Commented] (SPARK-28120) RocksDB state storage

2019-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869136#comment-16869136 ] Apache Spark commented on SPARK-28120: -- User 'itsvikramagr' has created a pull request for this

[jira] [Assigned] (SPARK-28120) RocksDB state storage

2019-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28120: Assignee: Apache Spark > RocksDB state storage > - > >

[jira] [Commented] (SPARK-28120) RocksDB state storage

2019-06-20 Thread Vikram Agrawal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869134#comment-16869134 ] Vikram Agrawal commented on SPARK-28120: I have raised a PR for it -

[jira] [Resolved] (SPARK-28127) Micro optimization on TreeNode's mapChildren method

2019-06-20 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-28127. - Resolution: Fixed Assignee: Yesheng Ma Fix Version/s: 3.0.0 > Micro optimization on

[jira] [Commented] (SPARK-28129) Add float8.sql

2019-06-20 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869113#comment-16869113 ] Yuming Wang commented on SPARK-28129: - I'm working on. > Add float8.sql > -- > >

[jira] [Created] (SPARK-28129) Add float8.sql

2019-06-20 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28129: --- Summary: Add float8.sql Key: SPARK-28129 URL: https://issues.apache.org/jira/browse/SPARK-28129 Project: Spark Issue Type: Sub-task Components: SQL

[jira] [Issue Comment Deleted] (SPARK-28071) Add strings.sql

2019-06-20 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28071: Comment: was deleted (was: I'm working on.) > Add strings.sql > --- > >

[jira] [Assigned] (SPARK-28128) Pandas Grouped UDFs should skip over empty partitions

2019-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28128: Assignee: (was: Apache Spark) > Pandas Grouped UDFs should skip over empty

[jira] [Assigned] (SPARK-28128) Pandas Grouped UDFs should skip over empty partitions

2019-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28128: Assignee: Apache Spark > Pandas Grouped UDFs should skip over empty partitions >

[jira] [Created] (SPARK-28128) Pandas Grouped UDFs should skip over empty partitions

2019-06-20 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-28128: Summary: Pandas Grouped UDFs should skip over empty partitions Key: SPARK-28128 URL: https://issues.apache.org/jira/browse/SPARK-28128 Project: Spark Issue

[jira] [Commented] (SPARK-28109) TRIM(type trimStr FROM str) returns incorrect result

2019-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869079#comment-16869079 ] Apache Spark commented on SPARK-28109: -- User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-28109) TRIM(type trimStr FROM str) returns incorrect result

2019-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869076#comment-16869076 ] Apache Spark commented on SPARK-28109: -- User 'wangyum' has created a pull request for this issue:

[jira] [Resolved] (SPARK-28118) Add `spark.eventLog.compression.codec` configuration

2019-06-20 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-28118. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24921

[jira] [Assigned] (SPARK-28127) Micro optimization on TreeNode's mapChildren method

2019-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28127: Assignee: (was: Apache Spark) > Micro optimization on TreeNode's mapChildren method

[jira] [Assigned] (SPARK-28127) Micro optimization on TreeNode's mapChildren method

2019-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28127: Assignee: Apache Spark > Micro optimization on TreeNode's mapChildren method >

[jira] [Created] (SPARK-28127) Micro optimization on TreeNode's mapChildren method

2019-06-20 Thread Yesheng Ma (JIRA)
Yesheng Ma created SPARK-28127: -- Summary: Micro optimization on TreeNode's mapChildren method Key: SPARK-28127 URL: https://issues.apache.org/jira/browse/SPARK-28127 Project: Spark Issue Type:

[jira] [Commented] (SPARK-23160) Add window.sql

2019-06-20 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868936#comment-16868936 ] Xiao Li commented on SPARK-23160: - Thanks! [~DylanGuedes] > Add window.sql > -- > >

[jira] [Commented] (SPARK-27946) Hive DDL to Spark DDL conversion USING "show create table"

2019-06-20 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868935#comment-16868935 ] Xiao Li commented on SPARK-27946: - Thanks! It is not urgent.  > Hive DDL to Spark DDL conversion USING

[jira] [Commented] (SPARK-26038) Decimal toScalaBigInt/toJavaBigInteger not work for decimals not fitting in long

2019-06-20 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868934#comment-16868934 ] Xiao Li commented on SPARK-26038: - Yes. It sounds good to me. Feel free to submit a backport PR.  >

[jira] [Updated] (SPARK-26038) Decimal toScalaBigInt/toJavaBigInteger not work for decimals not fitting in long

2019-06-20 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-26038: --- Labels: correctness (was: ) We just independently rediscovered this bug. I'm adding the

[jira] [Comment Edited] (SPARK-26038) Decimal toScalaBigInt/toJavaBigInteger not work for decimals not fitting in long

2019-06-20 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868926#comment-16868926 ] Josh Rosen edited comment on SPARK-26038 at 6/20/19 8:47 PM: - We just

[jira] [Resolved] (SPARK-28115) Fix flaky test: SparkContextSuite.test resource scheduling under local-cluster mode

2019-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-28115. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24917

[jira] [Commented] (SPARK-26836) Columns get switched in Spark SQL using Avro backed Hive table if schema evolves

2019-06-20 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1686#comment-1686 ] Dongjoon Hyun commented on SPARK-26836: --- cc [~dbtsai] > Columns get switched in Spark SQL using

[jira] [Commented] (SPARK-28109) TRIM(type trimStr FROM str) returns incorrect result

2019-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868875#comment-16868875 ] Apache Spark commented on SPARK-28109: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-28126) Support TRIM(trimStr FROM str) syntax

2019-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28126: Assignee: (was: Apache Spark) > Support TRIM(trimStr FROM str) syntax >

[jira] [Resolved] (SPARK-28111) Upgrade `xbean-asm7-shaded` to 4.14

2019-06-20 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28111. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 3.0.0 This is

[jira] [Commented] (SPARK-28126) Support TRIM(trimStr FROM str) syntax

2019-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868873#comment-16868873 ] Apache Spark commented on SPARK-28126: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-28126) Support TRIM(trimStr FROM str) syntax

2019-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28126: Assignee: Apache Spark > Support TRIM(trimStr FROM str) syntax >

[jira] [Created] (SPARK-28126) Support TRIM(trimStr FROM str) syntax

2019-06-20 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-28126: - Summary: Support TRIM(trimStr FROM str) syntax Key: SPARK-28126 URL: https://issues.apache.org/jira/browse/SPARK-28126 Project: Spark Issue Type:

[jira] [Created] (SPARK-28125) dataframes created by randomSplit have overlapping rows

2019-06-20 Thread Zachary (JIRA)
Zachary created SPARK-28125: --- Summary: dataframes created by randomSplit have overlapping rows Key: SPARK-28125 URL: https://issues.apache.org/jira/browse/SPARK-28125 Project: Spark Issue Type:

[jira] [Commented] (SPARK-28079) CSV fails to detect corrupt record unless "columnNameOfCorruptRecord" is manually added to the schema

2019-06-20 Thread F Jimenez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868582#comment-16868582 ] F Jimenez commented on SPARK-28079: --- When an explicit user-defined schema is not passed in, one will

[jira] [Commented] (SPARK-28079) CSV fails to detect corrupt record unless "columnNameOfCorruptRecord" is manually added to the schema

2019-06-20 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868577#comment-16868577 ] Liang-Chi Hsieh commented on SPARK-28079: - {{columnNameOfCorruptRecord}} currently applied only

[jira] [Assigned] (SPARK-28071) Add strings.sql

2019-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28071: Assignee: Apache Spark > Add strings.sql > --- > > Key:

[jira] [Assigned] (SPARK-28071) Add strings.sql

2019-06-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28071: Assignee: (was: Apache Spark) > Add strings.sql > --- > >

[jira] [Comment Edited] (SPARK-24961) sort operation causes out of memory

2019-06-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-24961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868522#comment-16868522 ] Juha Iso-Sipilä edited comment on SPARK-24961 at 6/20/19 1:15 PM: -- I am

[jira] [Commented] (SPARK-24961) sort operation causes out of memory

2019-06-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-24961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868522#comment-16868522 ] Juha Iso-Sipilä commented on SPARK-24961: - I am using Spark 2.4.3 (pyspark) and I observe the

[jira] [Updated] (SPARK-28124) Faster S3 file source with SQS

2019-06-20 Thread Abhishek Dixit (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Dixit updated SPARK-28124: --- Description: Using FileStreamSource to read files from a S3 bucket has problems both in

[jira] [Updated] (SPARK-28124) Faster S3 file source with SQS

2019-06-20 Thread Abhishek Dixit (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Dixit updated SPARK-28124: --- Description: Using FileStreamSource to read files from a S3 bucket has problems both in

[jira] [Updated] (SPARK-28124) Faster S3 file source with SQS

2019-06-20 Thread Abhishek Dixit (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Dixit updated SPARK-28124: --- Description: Using FileStreamSource to read files from a S3 bucket has problems both in

[jira] [Updated] (SPARK-28124) Faster S3 file source with SQS

2019-06-20 Thread Abhishek Dixit (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Dixit updated SPARK-28124: --- Description: Using FileStreamSource to read files from a s3 bucket has problems both in

[jira] [Updated] (SPARK-28124) Faster S3 file source with SQS

2019-06-20 Thread Abhishek Dixit (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Dixit updated SPARK-28124: --- Description: Using FileStreamSource to read files from a s3 bucket has problems both in

[jira] [Updated] (SPARK-28124) Faster S3 file source with SQS

2019-06-20 Thread Abhishek Dixit (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Dixit updated SPARK-28124: --- Description: Using FileStreamSource to read files from a s3 bucket has problems both in

[jira] [Updated] (SPARK-28124) Faster S3 file source with SQS

2019-06-20 Thread Abhishek Dixit (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Dixit updated SPARK-28124: --- Description: Using FileStreamSource to read files from a s3 bucket has problems both in

[jira] [Updated] (SPARK-28124) Faster S3 file source with SQS

2019-06-20 Thread Abhishek Dixit (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Dixit updated SPARK-28124: --- Description: Using FileStreamSource to read files from a s3 bucket has problems both in

[jira] [Created] (SPARK-28124) Faster S3 file source with SQS

2019-06-20 Thread Abhishek Dixit (JIRA)
Abhishek Dixit created SPARK-28124: -- Summary: Faster S3 file source with SQS Key: SPARK-28124 URL: https://issues.apache.org/jira/browse/SPARK-28124 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-28123) String Functions: Add support btrim

2019-06-20 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28123: --- Summary: String Functions: Add support btrim Key: SPARK-28123 URL: https://issues.apache.org/jira/browse/SPARK-28123 Project: Spark Issue Type: Sub-task

[jira] [Comment Edited] (SPARK-26836) Columns get switched in Spark SQL using Avro backed Hive table if schema evolves

2019-06-20 Thread david ravet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868414#comment-16868414 ] david ravet edited comment on SPARK-26836 at 6/20/19 10:05 AM: --- Hi,   We

[jira] [Commented] (SPARK-26836) Columns get switched in Spark SQL using Avro backed Hive table if schema evolves

2019-06-20 Thread david ravet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868414#comment-16868414 ] david ravet commented on SPARK-26836: - Hi,   We encounter the same issue with Spark 2.2.0 when

[jira] [Commented] (SPARK-13809) State Store: A new framework for state management for computing Streaming Aggregates

2019-06-20 Thread Vikram Agrawal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868403#comment-16868403 ] Vikram Agrawal commented on SPARK-13809: [~skonto] I have raised SPARK-28120 for rocksdb

[jira] [Created] (SPARK-28122) Binary String Functions: SHA functions

2019-06-20 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28122: --- Summary: Binary String Functions: SHA functions Key: SPARK-28122 URL: https://issues.apache.org/jira/browse/SPARK-28122 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-28121) String Functions: decode can not accept 'escape' as charset

2019-06-20 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28121: --- Summary: String Functions: decode can not accept 'escape' as charset Key: SPARK-28121 URL: https://issues.apache.org/jira/browse/SPARK-28121 Project: Spark

[jira] [Commented] (SPARK-28079) CSV fails to detect corrupt record unless "columnNameOfCorruptRecord" is manually added to the schema

2019-06-20 Thread F Jimenez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868367#comment-16868367 ] F Jimenez commented on SPARK-28079: --- I can confirm there is no change using 2.4.4 built from latest

[jira] [Updated] (SPARK-27952) String Functions: regexp_replace is not compatible

2019-06-20 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27952: Summary: String Functions: regexp_replace is not compatible (was: Built-in function: lag) >

[jira] [Updated] (SPARK-27952) Built-in function: lag

2019-06-20 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27952: Description: PostgreSQL's {{regexp_replace}} has the syntax: {{regexp_replace(source, pattern,

[jira] [Commented] (SPARK-28079) CSV fails to detect corrupt record unless "columnNameOfCorruptRecord" is manually added to the schema

2019-06-20 Thread F Jimenez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868345#comment-16868345 ] F Jimenez commented on SPARK-28079: --- No, or not yet ;). I'm trying to build 2.4.4 from the branch to

[jira] [Created] (SPARK-28120) RocksDB state storage

2019-06-20 Thread Vikram Agrawal (JIRA)
Vikram Agrawal created SPARK-28120: -- Summary: RocksDB state storage Key: SPARK-28120 URL: https://issues.apache.org/jira/browse/SPARK-28120 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-28079) CSV fails to detect corrupt record unless "columnNameOfCorruptRecord" is manually added to the schema

2019-06-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868341#comment-16868341 ] Hyukjin Kwon commented on SPARK-28079: -- Did you check the doc added by the fix of SPARK-28058

[jira] [Commented] (SPARK-28079) CSV fails to detect corrupt record unless "columnNameOfCorruptRecord" is manually added to the schema

2019-06-20 Thread F Jimenez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868327#comment-16868327 ] F Jimenez commented on SPARK-28079: --- Hi, sorry for the late response I'll paste the relevant part in

[jira] [Commented] (SPARK-28094) Multiple left joins or aggregations in one query produce incorrect results

2019-06-20 Thread Joe Ammann (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868313#comment-16868313 ] Joe Ammann commented on SPARK-28094: Thanks for those clarification. I think (after reading your

[jira] [Created] (SPARK-28119) Cannot read environment variable inside custom property file

2019-06-20 Thread JIRA
MINET J-Sébastien created SPARK-28119: - Summary: Cannot read environment variable inside custom property file Key: SPARK-28119 URL: https://issues.apache.org/jira/browse/SPARK-28119 Project:

[jira] [Comment Edited] (SPARK-28083) ANSI SQL: LIKE predicate: ESCAPE clause

2019-06-20 Thread jiaan.geng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868291#comment-16868291 ] jiaan.geng edited comment on SPARK-28083 at 6/20/19 6:39 AM: - I'm working

[jira] [Commented] (SPARK-28083) ANSI SQL: LIKE predicate: ESCAPE clause

2019-06-20 Thread jiaan.geng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868291#comment-16868291 ] jiaan.geng commented on SPARK-28083: Spark SQL already supported > ANSI SQL: LIKE predicate: ESCAPE