[jira] [Updated] (SPARK-30704) Use jekyll-redirect-from 0.15.0 instead of the latest

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30704: -- Description: We use Ruby 2.3 in our release docker image. The latest version of

[jira] [Created] (SPARK-30704) Use jekyll-redirect-from 0.15.0 instead of the latest

2020-02-01 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-30704: - Summary: Use jekyll-redirect-from 0.15.0 instead of the latest Key: SPARK-30704 URL: https://issues.apache.org/jira/browse/SPARK-30704 Project: Spark

[jira] [Resolved] (SPARK-27686) Update migration guide

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27686. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27161

[jira] [Assigned] (SPARK-27686) Update migration guide

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-27686: - Assignee: Yuming Wang > Update migration guide > --- > >

[jira] [Commented] (SPARK-30703) Add a documentation page for ANSI mode

2020-02-01 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028253#comment-17028253 ] Takeshi Yamamuro commented on SPARK-30703: -- Yea, sure, I will this week ;) > Add a

[jira] [Commented] (SPARK-30703) Add a documentation page for ANSI mode

2020-02-01 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028251#comment-17028251 ] Xiao Li commented on SPARK-30703: - [~maropu] Could you help this? > Add a documentation page for ANSI

[jira] [Created] (SPARK-30703) Add a documentation page for ANSI mode

2020-02-01 Thread Xiao Li (Jira)
Xiao Li created SPARK-30703: --- Summary: Add a documentation page for ANSI mode Key: SPARK-30703 URL: https://issues.apache.org/jira/browse/SPARK-30703 Project: Spark Issue Type: Documentation

[jira] [Updated] (SPARK-19842) Informational Referential Integrity Constraints Support in Spark

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-19842: -- Target Version/s: (was: 3.0.0) > Informational Referential Integrity Constraints Support in

[jira] [Commented] (SPARK-19842) Informational Referential Integrity Constraints Support in Spark

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028234#comment-17028234 ] Dongjoon Hyun commented on SPARK-19842: --- I removed the `Target Version : 3.0.0` because we created

[jira] [Updated] (SPARK-20964) Make some keywords reserved along with the ANSI/SQL standard

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-20964: -- Target Version/s: (was: 3.0.0) > Make some keywords reserved along with the ANSI/SQL

[jira] [Commented] (SPARK-20964) Make some keywords reserved along with the ANSI/SQL standard

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028233#comment-17028233 ] Dongjoon Hyun commented on SPARK-20964: --- Hi, [~maropu]. I removed the target version first. Please

[jira] [Updated] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-22231: -- Target Version/s: 3.1.0 (was: 3.0.0) > Support of map, filter, withColumn, dropColumn in

[jira] [Updated] (SPARK-24625) put all the backward compatible behavior change configs under spark.sql.legacy.*

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24625: -- Target Version/s: (was: 3.0.0) > put all the backward compatible behavior change configs

[jira] [Commented] (SPARK-24625) put all the backward compatible behavior change configs under spark.sql.legacy.*

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028232#comment-17028232 ] Dongjoon Hyun commented on SPARK-24625: --- For this, I'll remove the target version first. Please

[jira] [Updated] (SPARK-24941) Add RDDBarrier.coalesce() function

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24941: -- Target Version/s: 3.1.0 (was: 3.0.0) > Add RDDBarrier.coalesce() function >

[jira] [Commented] (SPARK-25531) new write APIs for data source v2

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028231#comment-17028231 ] Dongjoon Hyun commented on SPARK-25531: --- I moved this to 3.1.0 for now. If we can resolve this

[jira] [Updated] (SPARK-25531) new write APIs for data source v2

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25531: -- Target Version/s: 3.1.0 (was: 3.0.0) > new write APIs for data source v2 >

[jira] [Updated] (SPARK-24942) Improve cluster resource management with jobs containing barrier stage

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24942: -- Target Version/s: 3.1.0 (was: 3.0.0) > Improve cluster resource management with jobs

[jira] [Updated] (SPARK-25383) Image data source supports sample pushdown

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25383: -- Target Version/s: 3.1.0 (was: 3.0.0) > Image data source supports sample pushdown >

[jira] [Commented] (SPARK-25531) new write APIs for data source v2

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028230#comment-17028230 ] Dongjoon Hyun commented on SPARK-25531: --- Hi, [~cloud_fan]. Could you resolve this issue or adjust

[jira] [Updated] (SPARK-26425) Add more constraint checks in file streaming source to avoid checkpoint corruption

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26425: -- Target Version/s: 3.1.0 (was: 3.0.0) > Add more constraint checks in file streaming source

[jira] [Updated] (SPARK-25752) Add trait to easily whitelist logical operators that produce named output from CleanupAliases

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25752: -- Target Version/s: 3.1.0 (was: 3.0.0) > Add trait to easily whitelist logical operators that

[jira] [Resolved] (SPARK-27471) Reorganize public v2 catalog API

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27471. --- Assignee: Ryan Blue Resolution: Done > Reorganize public v2 catalog API >

[jira] [Commented] (SPARK-27471) Reorganize public v2 catalog API

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028229#comment-17028229 ] Dongjoon Hyun commented on SPARK-27471: --- I believe we've done what we need at 3.0.0. For the rest

[jira] [Updated] (SPARK-27780) Shuffle server & client should be versioned to enable smoother upgrade

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27780: -- Target Version/s: 3.1.0 (was: 3.0.0) > Shuffle server & client should be versioned to enable

[jira] [Updated] (SPARK-27936) Support local dependency uploading from --py-files

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27936: -- Target Version/s: 3.1.0 > Support local dependency uploading from --py-files >

[jira] [Updated] (SPARK-28629) Capture the missing rules in HiveSessionStateBuilder

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28629: -- Target Version/s: 3.1.0 (was: 3.0.0) > Capture the missing rules in HiveSessionStateBuilder

[jira] [Updated] (SPARK-28717) Update SQL ALTER TABLE RENAME to use TableCatalog API

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28717: -- Target Version/s: (was: 3.0.0) > Update SQL ALTER TABLE RENAME to use TableCatalog API >

[jira] [Updated] (SPARK-27936) Support local dependency uploading from --py-files

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27936: -- Target Version/s: (was: 3.0.0) > Support local dependency uploading from --py-files >

[jira] [Resolved] (SPARK-30097) Adding support for core writers

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30097. --- Resolution: Won't Do > Adding support for core writers >

[jira] [Updated] (SPARK-30097) Adding support for core writers

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30097: -- Target Version/s: (was: 3.0.0) > Adding support for core writers >

[jira] [Updated] (SPARK-30186) support Dynamic Partition Pruning in Adaptive Execution

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30186: -- Fix Version/s: (was: 3.0.0) > support Dynamic Partition Pruning in Adaptive Execution >

[jira] [Updated] (SPARK-30186) support Dynamic Partition Pruning in Adaptive Execution

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30186: -- Target Version/s: (was: 3.0.0) > support Dynamic Partition Pruning in Adaptive Execution >

[jira] [Closed] (SPARK-30097) Adding support for core writers

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-30097. - > Adding support for core writers > > > Key:

[jira] [Updated] (SPARK-30334) Add metadata around semi-structured columns to Spark

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30334: -- Target Version/s: 3.1.0 (was: 3.0.0) > Add metadata around semi-structured columns to Spark

[jira] [Updated] (SPARK-30324) Simplify API for JSON access in DataFrames/SQL

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30324: -- Target Version/s: 3.1.0 (was: 3.0.0) > Simplify API for JSON access in DataFrames/SQL >

[jira] [Updated] (SPARK-30567) setDelegateCatalog should be called if catalog has implemented CatalogExtension

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30567: -- Target Version/s: (was: 3.1.0) > setDelegateCatalog should be called if catalog has

[jira] [Updated] (SPARK-30667) Support simple all gather in barrier task context

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30667: -- Target Version/s: 3.1.0 (was: 3.0.0) > Support simple all gather in barrier task context >

[jira] [Updated] (SPARK-30567) setDelegateCatalog should be called if catalog has implemented CatalogExtension

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30567: -- Fix Version/s: (was: 3.0.0) > setDelegateCatalog should be called if catalog has

[jira] [Updated] (SPARK-30567) setDelegateCatalog should be called if catalog has implemented CatalogExtension

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30567: -- Target Version/s: 3.1.0 (was: 3.0.0) > setDelegateCatalog should be called if catalog has

[jira] [Updated] (SPARK-24941) Add RDDBarrier.coalesce() function

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24941: -- Target Version/s: 3.1.0 > Add RDDBarrier.coalesce() function >

[jira] [Updated] (SPARK-30334) Add metadata around semi-structured columns to Spark

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30334: -- Target Version/s: 3.1.0 > Add metadata around semi-structured columns to Spark >

[jira] [Updated] (SPARK-20964) Make some keywords reserved along with the ANSI/SQL standard

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-20964: -- Target Version/s: 3.1.0 > Make some keywords reserved along with the ANSI/SQL standard >

[jira] [Updated] (SPARK-30186) support Dynamic Partition Pruning in Adaptive Execution

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30186: -- Target Version/s: 3.1.0 > support Dynamic Partition Pruning in Adaptive Execution >

[jira] [Updated] (SPARK-25531) new write APIs for data source v2

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25531: -- Target Version/s: 3.1.0 > new write APIs for data source v2 >

[jira] [Updated] (SPARK-24942) Improve cluster resource management with jobs containing barrier stage

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24942: -- Target Version/s: 3.1.0 > Improve cluster resource management with jobs containing barrier

[jira] [Updated] (SPARK-30667) Support simple all gather in barrier task context

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30667: -- Target Version/s: 3.1.0 > Support simple all gather in barrier task context >

[jira] [Updated] (SPARK-27780) Shuffle server & client should be versioned to enable smoother upgrade

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27780: -- Target Version/s: 3.1.0 > Shuffle server & client should be versioned to enable smoother

[jira] [Updated] (SPARK-26425) Add more constraint checks in file streaming source to avoid checkpoint corruption

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26425: -- Target Version/s: 3.1.0 > Add more constraint checks in file streaming source to avoid

[jira] [Updated] (SPARK-30567) setDelegateCatalog should be called if catalog has implemented CatalogExtension

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30567: -- Target Version/s: 3.1.0 > setDelegateCatalog should be called if catalog has implemented >

[jira] [Updated] (SPARK-30324) Simplify API for JSON access in DataFrames/SQL

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30324: -- Target Version/s: 3.1.0 > Simplify API for JSON access in DataFrames/SQL >

[jira] [Updated] (SPARK-30097) Adding support for core writers

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30097: -- Target Version/s: 3.1.0 > Adding support for core writers >

[jira] [Updated] (SPARK-24625) put all the backward compatible behavior change configs under spark.sql.legacy.*

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24625: -- Target Version/s: 3.1.0 > put all the backward compatible behavior change configs under >

[jira] [Updated] (SPARK-19842) Informational Referential Integrity Constraints Support in Spark

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-19842: -- Target Version/s: 3.1.0 > Informational Referential Integrity Constraints Support in Spark >

[jira] [Updated] (SPARK-27471) Reorganize public v2 catalog API

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27471: -- Target Version/s: 3.1.0 > Reorganize public v2 catalog API >

[jira] [Updated] (SPARK-28629) Capture the missing rules in HiveSessionStateBuilder

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28629: -- Target Version/s: 3.1.0 > Capture the missing rules in HiveSessionStateBuilder >

[jira] [Updated] (SPARK-28717) Update SQL ALTER TABLE RENAME to use TableCatalog API

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28717: -- Target Version/s: 3.1.0 > Update SQL ALTER TABLE RENAME to use TableCatalog API >

[jira] [Updated] (SPARK-25383) Image data source supports sample pushdown

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25383: -- Target Version/s: 3.1.0 > Image data source supports sample pushdown >

[jira] [Updated] (SPARK-25752) Add trait to easily whitelist logical operators that produce named output from CleanupAliases

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25752: -- Target Version/s: 3.1.0 > Add trait to easily whitelist logical operators that produce named

[jira] [Updated] (SPARK-27936) Support local dependency uploading from --py-files

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27936: -- Target Version/s: 3.1.0 > Support local dependency uploading from --py-files >

[jira] [Updated] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-02-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-22231: -- Target Version/s: 3.1.0 > Support of map, filter, withColumn, dropColumn in nested list of

[jira] [Commented] (SPARK-30657) Streaming limit after streaming dropDuplicates can throw error

2020-02-01 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028223#comment-17028223 ] Shixiong Zhu commented on SPARK-30657: -- [~tdas] Make sense. Agreed that the risk is high but the

[jira] [Comment Edited] (SPARK-30701) SQL test running on Windows: hadoop chgrp warnings

2020-02-01 Thread Guram Savinov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028026#comment-17028026 ] Guram Savinov edited comment on SPARK-30701 at 2/1/20 9:38 AM: --- So the

[jira] [Updated] (SPARK-30701) SQL test running on Windows: hadoop chgrp warnings

2020-02-01 Thread Guram Savinov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guram Savinov updated SPARK-30701: -- Environment: Windows 10 Winutils 2.7.1: 

[jira] [Created] (SPARK-30702) Support subexpression elimination in whole stage codegen

2020-02-01 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-30702: --- Summary: Support subexpression elimination in whole stage codegen Key: SPARK-30702 URL: https://issues.apache.org/jira/browse/SPARK-30702 Project: Spark Issue

[jira] [Commented] (SPARK-30701) SQL test running on Windows: hadoop chgrp warnings

2020-02-01 Thread Guram Savinov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028026#comment-17028026 ] Guram Savinov commented on SPARK-30701: --- So the problem is: backslash character isn't included to

[jira] [Updated] (SPARK-30701) SQL test running on Windows: hadoop chgrp warnings

2020-02-01 Thread Guram Savinov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guram Savinov updated SPARK-30701: -- Attachment: HadoopGroupTest.java > SQL test running on Windows: hadoop chgrp warnings >

[jira] [Updated] (SPARK-30701) SQL test running on Windows: hadoop chgrp warnings

2020-02-01 Thread Guram Savinov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guram Savinov updated SPARK-30701: -- Description: Running SparkSQL local embedded unit tests on Win10, using winutils. Got

[jira] [Updated] (SPARK-30701) SQL test running on Windows: hadoop chgrp warnings

2020-02-01 Thread Guram Savinov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guram Savinov updated SPARK-30701: -- Description: Running SparkSQL local embedded unit tests on Win10, using winutils. Got

[jira] [Updated] (SPARK-30701) SQL test running on Windows: hadoop chgrp warnings

2020-02-01 Thread Guram Savinov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guram Savinov updated SPARK-30701: -- Description: Running SparkSQL local embedded unit tests on Win10, using winutils. Got

[jira] [Updated] (SPARK-30701) SQL test running on Windows: hadoop chgrp warnings

2020-02-01 Thread Guram Savinov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guram Savinov updated SPARK-30701: -- Description: Running SparkSQL local embedded unit tests on Win10, using winutils. Got

[jira] [Updated] (SPARK-30701) SQL test running on Windows: hadoop chgrp warnings

2020-02-01 Thread Guram Savinov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guram Savinov updated SPARK-30701: -- Environment: Windows 10 Winutils 2.7.1: 

[jira] [Updated] (SPARK-30701) SQL test running on Windows: hadoop chgrp warnings

2020-02-01 Thread Guram Savinov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guram Savinov updated SPARK-30701: -- Description: Running SparkSQL embedded unit tests on Win10, using winutils. Got warnings

[jira] [Updated] (SPARK-30701) SQL test running on Windows: hadoop chgrp warnings

2020-02-01 Thread Guram Savinov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guram Savinov updated SPARK-30701: -- Description: Running SparkSQL unit tests on Win10, using winutils. Got warnings about

[jira] [Updated] (SPARK-30701) SQL test running on Windows: hadoop chgrp warnings

2020-02-01 Thread Guram Savinov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guram Savinov updated SPARK-30701: -- Description: {code:java} -chgrp: 'TEST\Domain users' does not match expected pattern for

[jira] [Updated] (SPARK-30701) SQL test running on Windows: hadoop chgrp warnings

2020-02-01 Thread Guram Savinov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guram Savinov updated SPARK-30701: -- Affects Version/s: (was: 2.3.0) 2.4.4 > SQL test running on

[jira] [Updated] (SPARK-30701) SQL test running on Windows: hadoop chgrp warnings

2020-02-01 Thread Guram Savinov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guram Savinov updated SPARK-30701: -- Labels: WIndows hive unit-test (was: bulk-closed) > SQL test running on Windows: hadoop

[jira] [Updated] (SPARK-30701) SQL test running on Windows: hadoop chgrp warnings

2020-02-01 Thread Guram Savinov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guram Savinov updated SPARK-30701: -- Component/s: (was: SparkR) > SQL test running on Windows: hadoop chgrp warnings >

[jira] [Created] (SPARK-30701) SQL test running on Windows: hadoop chgrp warnings

2020-02-01 Thread Guram Savinov (Jira)
Guram Savinov created SPARK-30701: - Summary: SQL test running on Windows: hadoop chgrp warnings Key: SPARK-30701 URL: https://issues.apache.org/jira/browse/SPARK-30701 Project: Spark Issue