[jira] [Resolved] (SPARK-32467) Avoid encoding URL twice on https redirect

2020-07-31 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-32467. Resolution: Fixed The issue is resolved in https://github.com/apache/spark/pull/29271 >

[jira] [Updated] (SPARK-32467) Avoid encoding URL twice on https redirect

2020-07-31 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-32467: --- Affects Version/s: 3.0.1 > Avoid encoding URL twice on https redirect >

[jira] [Updated] (SPARK-32119) ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes with --jars

2020-07-31 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-32119: --- Description: ExecutorPlugin can't work with Standalone Cluster and Kubernetes when a jar

[jira] [Updated] (SPARK-32119) ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes with --jars

2020-07-31 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-32119: --- Summary: ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes with --jars

[jira] [Comment Edited] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

2020-07-31 Thread JinxinTang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169219#comment-17169219 ] JinxinTang edited comment on SPARK-32500 at 8/1/20, 2:24 AM: - Hi,

[jira] [Commented] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

2020-07-31 Thread JinxinTang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169219#comment-17169219 ] JinxinTang commented on SPARK-32500: Hi, [~abhishekd0907], I have test the code in master,

[jira] [Updated] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

2020-07-31 Thread JinxinTang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-32500: --- Attachment: image-2020-08-01-10-21-51-246.png > Query and Batch Id not set for Structured Streaming

[jira] [Resolved] (SPARK-32514) Pyspark: Issue using sql query in foreachBatch sink

2020-07-31 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-32514. -- Resolution: Not A Problem Python doesn't allow abbreviating () with no param, whereas Scala

[jira] [Commented] (SPARK-32402) Implement ALTER TABLE in JDBC Table Catalog

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169209#comment-17169209 ] Apache Spark commented on SPARK-32402: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32402) Implement ALTER TABLE in JDBC Table Catalog

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32402: Assignee: Apache Spark > Implement ALTER TABLE in JDBC Table Catalog >

[jira] [Assigned] (SPARK-32402) Implement ALTER TABLE in JDBC Table Catalog

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32402: Assignee: (was: Apache Spark) > Implement ALTER TABLE in JDBC Table Catalog >

[jira] [Commented] (SPARK-32402) Implement ALTER TABLE in JDBC Table Catalog

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169208#comment-17169208 ] Apache Spark commented on SPARK-32402: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Created] (SPARK-32514) Pyspark: Issue using sql query in foreachBatch sink

2020-07-31 Thread Muru (Jira)
Muru created SPARK-32514: Summary: Pyspark: Issue using sql query in foreachBatch sink Key: SPARK-32514 URL: https://issues.apache.org/jira/browse/SPARK-32514 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-32018) Fix UnsafeRow set overflowed decimal

2020-07-31 Thread Sunitha Kambhampati (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169132#comment-17169132 ] Sunitha Kambhampati commented on SPARK-32018: - [@cloud-fan|https://github.com/cloud-fan], I

[jira] [Updated] (SPARK-32511) Add dropFields method to Column class

2020-07-31 Thread fqaiser94 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fqaiser94 updated SPARK-32511: -- Description: Based on the discussions in the parent ticket (SPARK-22231), add a new  {{dropFields}} 

[jira] [Updated] (SPARK-32511) Add dropFields method to Column class

2020-07-31 Thread fqaiser94 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fqaiser94 updated SPARK-32511: -- Description: Based on the discussions in the parent ticket (SPARK-22231), add a new  {{dropFields}} 

[jira] [Updated] (SPARK-32511) Add dropFields method to Column class

2020-07-31 Thread fqaiser94 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fqaiser94 updated SPARK-32511: -- Description: Based on the discussions in the parent ticket (SPARK-22231), add a new  {{dropFields}} 

[jira] [Updated] (SPARK-32511) Add dropFields method to Column class

2020-07-31 Thread fqaiser94 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fqaiser94 updated SPARK-32511: -- Description: Based on the discussions in the parent ticket (SPARK-22231), add a new  {{dropFields}} 

[jira] [Updated] (SPARK-32511) Add dropFields method to Column class

2020-07-31 Thread fqaiser94 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fqaiser94 updated SPARK-32511: -- Description: Based on the discussions in the parent ticket (SPARK-22231) and following on, add a new 

[jira] [Updated] (SPARK-32511) Add dropFields method to Column class

2020-07-31 Thread fqaiser94 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fqaiser94 updated SPARK-32511: -- Description: Based on the extensive discussions in the parent ticket, it was determined we should

[jira] [Updated] (SPARK-32511) Add dropFields method to Column class

2020-07-31 Thread fqaiser94 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fqaiser94 updated SPARK-32511: -- Description: Based on the discussions in the parent ticket (SPARK-22241) and following on, add a new 

[jira] [Updated] (SPARK-32511) Add dropFields method to Column class

2020-07-31 Thread fqaiser94 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fqaiser94 updated SPARK-32511: -- Description: Based on the discussions in the parent ticket, add a new {{dropFields}} method to the 

[jira] [Updated] (SPARK-32511) Add dropFields method to Column class

2020-07-31 Thread fqaiser94 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fqaiser94 updated SPARK-32511: -- Description: Based on the discussions in the parent ticket,  Added a new {{dropFields}} method to

[jira] [Commented] (SPARK-32511) Add dropFields method to Column class

2020-07-31 Thread Rohit Mishra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169087#comment-17169087 ] Rohit Mishra commented on SPARK-32511: -- [~fqaiser94], Can you please add a description? > Add

[jira] [Commented] (SPARK-32513) Rename classes/files with the Jdbc prefix to JDBC

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169081#comment-17169081 ] Apache Spark commented on SPARK-32513: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32513) Rename classes/files with the Jdbc prefix to JDBC

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32513: Assignee: Apache Spark > Rename classes/files with the Jdbc prefix to JDBC >

[jira] [Commented] (SPARK-32513) Rename classes/files with the Jdbc prefix to JDBC

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169082#comment-17169082 ] Apache Spark commented on SPARK-32513: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32513) Rename classes/files with the Jdbc prefix to JDBC

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32513: Assignee: (was: Apache Spark) > Rename classes/files with the Jdbc prefix to JDBC >

[jira] [Created] (SPARK-32513) Rename classes/files with the Jdbc prefix to JDBC

2020-07-31 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-32513: -- Summary: Rename classes/files with the Jdbc prefix to JDBC Key: SPARK-32513 URL: https://issues.apache.org/jira/browse/SPARK-32513 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32405) Apply table options while creating tables in JDBC Table Catalog

2020-07-31 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169050#comment-17169050 ] Maxim Gekk commented on SPARK-32405: properties passed to createTable are ignored, see 

[jira] [Commented] (SPARK-31470) Introduce SORTED BY clause in CREATE TABLE statement

2020-07-31 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169042#comment-17169042 ] Cheng Su commented on SPARK-31470: -- [~yumwang] - thanks for creating this jira! Would like to know more

[jira] [Commented] (SPARK-32405) Apply table options while creating tables in JDBC Table Catalog

2020-07-31 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169041#comment-17169041 ] Huaxin Gao commented on SPARK-32405: [~maxgekk] Hi Max, could you please explain what these table

[jira] [Updated] (SPARK-32332) AQE doesn't adequately allow for Columnar Processing extension

2020-07-31 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-32332: -- Fix Version/s: 3.0.1 > AQE doesn't adequately allow for Columnar Processing extension >

[jira] [Updated] (SPARK-32512) Add basic partition command for datasourcev2

2020-07-31 Thread Jackey Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackey Lee updated SPARK-32512: --- Description: This Jira is trying to add basic partition command API, `AlterTableAddPartitionExec`

[jira] [Assigned] (SPARK-32511) Add dropFields method to Column class

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32511: Assignee: (was: Apache Spark) > Add dropFields method to Column class >

[jira] [Assigned] (SPARK-32511) Add dropFields method to Column class

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32511: Assignee: Apache Spark > Add dropFields method to Column class >

[jira] [Commented] (SPARK-32511) Add dropFields method to Column class

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168966#comment-17168966 ] Apache Spark commented on SPARK-32511: -- User 'fqaiser94' has created a pull request for this issue:

[jira] [Created] (SPARK-32512) Add basic partition command for datasourcev2

2020-07-31 Thread Jackey Lee (Jira)
Jackey Lee created SPARK-32512: -- Summary: Add basic partition command for datasourcev2 Key: SPARK-32512 URL: https://issues.apache.org/jira/browse/SPARK-32512 Project: Spark Issue Type:

[jira] [Created] (SPARK-32511) Add dropFields method to Column class

2020-07-31 Thread fqaiser94 (Jira)
fqaiser94 created SPARK-32511: - Summary: Add dropFields method to Column class Key: SPARK-32511 URL: https://issues.apache.org/jira/browse/SPARK-32511 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-32083) Unnecessary tasks are launched when input is empty with AQE

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168946#comment-17168946 ] Apache Spark commented on SPARK-32083: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-32083) Unnecessary tasks are launched when input is empty with AQE

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168945#comment-17168945 ] Apache Spark commented on SPARK-32083: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-22947) SPIP: as-of join in Spark SQL

2020-07-31 Thread Xuyan Xiao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168904#comment-17168904 ] Xuyan Xiao commented on SPARK-22947: [~icexelloss] Any update on this issue? > SPIP: as-of join in

[jira] [Commented] (SPARK-32495) Update jackson versions from 2.4.6 and so on(2.4.x)

2020-07-31 Thread SHOBHIT SHUKLA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168868#comment-17168868 ] SHOBHIT SHUKLA commented on SPARK-32495: [~prashant] Here is the link where Jackson databind

[jira] [Comment Edited] (SPARK-32495) Update jackson versions from 2.4.6 and so on(2.4.x)

2020-07-31 Thread SHOBHIT SHUKLA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168853#comment-17168853 ] SHOBHIT SHUKLA edited comment on SPARK-32495 at 7/31/20, 1:15 PM: --

[jira] [Updated] (SPARK-32495) Update jackson versions from 2.4.6 and so on(2.4.x)

2020-07-31 Thread SHOBHIT SHUKLA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SHOBHIT SHUKLA updated SPARK-32495: --- Description: As a vulnerability for Fasterxml Jackson version 2.6.7.3 is affected by

[jira] [Updated] (SPARK-32495) Update jackson versions from 2.4.6 and so on(2.4.x)

2020-07-31 Thread SHOBHIT SHUKLA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SHOBHIT SHUKLA updated SPARK-32495: --- Description: As a vulnerability for Fasterxml Jackson version 2.6.7.3 is affected by

[jira] [Updated] (SPARK-32495) Update jackson versions from 2.4.6 and so on(2.4.x)

2020-07-31 Thread SHOBHIT SHUKLA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SHOBHIT SHUKLA updated SPARK-32495: --- Description: As a vulnerability for Fasterxml Jackson version 2.6.7.3 is affected by

[jira] [Commented] (SPARK-32509) Unused DPP Filter causes issue in canonicalization and prevents reuse exchange

2020-07-31 Thread Rohit Mishra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168856#comment-17168856 ] Rohit Mishra commented on SPARK-32509: -- [~prakharjain09], Meanwhile you are working on the pull

[jira] [Commented] (SPARK-32495) Update jackson versions from 2.4.6 and so on(2.4.x)

2020-07-31 Thread SHOBHIT SHUKLA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168853#comment-17168853 ] SHOBHIT SHUKLA commented on SPARK-32495: The NVD databases lists CVE-2017-15095 and

[jira] [Commented] (SPARK-32507) Main Page

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168850#comment-17168850 ] Apache Spark commented on SPARK-32507: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-32507) Main Page

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32507: Assignee: Hyukjin Kwon (was: Apache Spark) > Main Page > - > >

[jira] [Assigned] (SPARK-32507) Main Page

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32507: Assignee: Apache Spark (was: Hyukjin Kwon) > Main Page > - > >

[jira] [Commented] (SPARK-32507) Main Page

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168848#comment-17168848 ] Apache Spark commented on SPARK-32507: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-32480) Support insert overwrite to move the data to trash

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168833#comment-17168833 ] Apache Spark commented on SPARK-32480: -- User 'Udbhav30' has created a pull request for this issue:

[jira] [Commented] (SPARK-32509) Unused DPP Filter causes issue in canonicalization and prevents reuse exchange

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168831#comment-17168831 ] Apache Spark commented on SPARK-32509: -- User 'prakharjain09' has created a pull request for this

[jira] [Commented] (SPARK-32480) Support insert overwrite to move the data to trash

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168829#comment-17168829 ] Apache Spark commented on SPARK-32480: -- User 'Udbhav30' has created a pull request for this issue:

[jira] [Commented] (SPARK-32510) JDBC doesn't check duplicate column names in nested structures

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168830#comment-17168830 ] Apache Spark commented on SPARK-32510: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32509) Unused DPP Filter causes issue in canonicalization and prevents reuse exchange

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32509: Assignee: Apache Spark > Unused DPP Filter causes issue in canonicalization and prevents

[jira] [Assigned] (SPARK-32480) Support insert overwrite to move the data to trash

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32480: Assignee: Apache Spark > Support insert overwrite to move the data to trash >

[jira] [Assigned] (SPARK-32509) Unused DPP Filter causes issue in canonicalization and prevents reuse exchange

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32509: Assignee: (was: Apache Spark) > Unused DPP Filter causes issue in canonicalization

[jira] [Assigned] (SPARK-32510) JDBC doesn't check duplicate column names in nested structures

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32510: Assignee: Apache Spark > JDBC doesn't check duplicate column names in nested structures

[jira] [Assigned] (SPARK-32480) Support insert overwrite to move the data to trash

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32480: Assignee: (was: Apache Spark) > Support insert overwrite to move the data to trash

[jira] [Assigned] (SPARK-32510) JDBC doesn't check duplicate column names in nested structures

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32510: Assignee: (was: Apache Spark) > JDBC doesn't check duplicate column names in nested

[jira] [Commented] (SPARK-32495) Update jackson versions from 2.4.6 and so on(2.4.x)

2020-07-31 Thread Prashant Sharma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168825#comment-17168825 ] Prashant Sharma commented on SPARK-32495: - Furthermore, according to

[jira] [Created] (SPARK-32510) JDBC doesn't check duplicate column names in nested structures

2020-07-31 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-32510: -- Summary: JDBC doesn't check duplicate column names in nested structures Key: SPARK-32510 URL: https://issues.apache.org/jira/browse/SPARK-32510 Project: Spark

[jira] [Created] (SPARK-32509) Unused DPP Filter causes issue in canonicalization and prevents reuse exchange

2020-07-31 Thread Prakhar Jain (Jira)
Prakhar Jain created SPARK-32509: Summary: Unused DPP Filter causes issue in canonicalization and prevents reuse exchange Key: SPARK-32509 URL: https://issues.apache.org/jira/browse/SPARK-32509

[jira] [Assigned] (SPARK-32508) Disallow empty part col values in partition spec before static partition writing

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32508: Assignee: (was: Apache Spark) > Disallow empty part col values in partition spec

[jira] [Assigned] (SPARK-32508) Disallow empty part col values in partition spec before static partition writing

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32508: Assignee: Apache Spark > Disallow empty part col values in partition spec before static

[jira] [Commented] (SPARK-32508) Disallow empty part col values in partition spec before static partition writing

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168630#comment-17168630 ] Apache Spark commented on SPARK-32508: -- User 'cxzl25' has created a pull request for this issue:

[jira] [Created] (SPARK-32508) Disallow empty part col values in partition spec before static partition writing

2020-07-31 Thread dzcxzl (Jira)
dzcxzl created SPARK-32508: -- Summary: Disallow empty part col values in partition spec before static partition writing Key: SPARK-32508 URL: https://issues.apache.org/jira/browse/SPARK-32508 Project: Spark

[jira] [Commented] (SPARK-32495) Update jackson versions from 2.4.6 and so on(2.4.x)

2020-07-31 Thread Prashant Sharma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168628#comment-17168628 ] Prashant Sharma commented on SPARK-32495: - Is this not specific to, jackson-databind? And the

[jira] [Commented] (SPARK-32268) Bloom Filter Join

2020-07-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168582#comment-17168582 ] Yuming Wang commented on SPARK-32268: - {code:scala} import org.apache.spark.benchmark.Benchmark

[jira] [Commented] (SPARK-31894) Introduce UnsafeRow format validation for streaming state store

2020-07-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168564#comment-17168564 ] Apache Spark commented on SPARK-31894: -- User 'xuanyuanking' has created a pull request for this

[jira] [Updated] (SPARK-32507) Main Page

2020-07-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32507: - Description: We should make a main package to overview PySpark properly. See the demo example:

[jira] [Updated] (SPARK-32507) Main Page

2020-07-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32507: - Description: We should make a main package to overview PySpark properly. See the demo example:

[jira] [Created] (SPARK-32507) Main Page

2020-07-31 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-32507: Summary: Main Page Key: SPARK-32507 URL: https://issues.apache.org/jira/browse/SPARK-32507 Project: Spark Issue Type: Sub-task Components:

[jira] [Commented] (SPARK-32506) flaky test: pyspark.mllib.tests.test_streaming_algorithms.StreamingLinearRegressionWithTests

2020-07-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168525#comment-17168525 ] Wenchen Fan commented on SPARK-32506: - cc [~ruifengz] [~weichenxu123] > flaky test: >

[jira] [Created] (SPARK-32506) flaky test: pyspark.mllib.tests.test_streaming_algorithms.StreamingLinearRegressionWithTests

2020-07-31 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-32506: --- Summary: flaky test: pyspark.mllib.tests.test_streaming_algorithms.StreamingLinearRegressionWithTests Key: SPARK-32506 URL: https://issues.apache.org/jira/browse/SPARK-32506

[jira] [Updated] (SPARK-32456) Check the Distinct by assuming it as Aggregate for Structured Streaming

2020-07-31 Thread Yuanjian Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanjian Li updated SPARK-32456: Description: We want to fix 2 things here: 1. Give better error message for Distinct related

[jira] [Updated] (SPARK-32456) Check the Distinct by assuming it as Aggregate for Structured Streaming

2020-07-31 Thread Yuanjian Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanjian Li updated SPARK-32456: Summary: Check the Distinct by assuming it as Aggregate for Structured Streaming (was: Give

[jira] [Commented] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

2020-07-31 Thread Abhishek Dixit (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168423#comment-17168423 ] Abhishek Dixit commented on SPARK-32500: [~rohitmishr1484] I used the following example to

[jira] [Updated] (SPARK-32505) Only codegen build side separately for shuffled hash join

2020-07-31 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-32505: - Parent: SPARK-32461 Issue Type: Sub-task (was: Improvement) > Only codegen build side

[jira] [Created] (SPARK-32505) Only codegen build side separately for shuffled hash join

2020-07-31 Thread Cheng Su (Jira)
Cheng Su created SPARK-32505: Summary: Only codegen build side separately for shuffled hash join Key: SPARK-32505 URL: https://issues.apache.org/jira/browse/SPARK-32505 Project: Spark Issue