[jira] [Commented] (SPARK-36808) Upgrade Kafka to 2.8.1

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492420#comment-17492420 ] Apache Spark commented on SPARK-36808: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-36808) Upgrade Kafka to 2.8.1

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492417#comment-17492417 ] Apache Spark commented on SPARK-36808: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-36808) Upgrade Kafka to 2.8.1

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492413#comment-17492413 ] Apache Spark commented on SPARK-36808: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-36808) Upgrade Kafka to 2.8.1

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492414#comment-17492414 ] Apache Spark commented on SPARK-36808: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-36808) Upgrade Kafka to 2.8.1

2022-02-14 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492408#comment-17492408 ] Kousuke Saruta commented on SPARK-36808: [~dongjoon] Sure, I'll do it. > Upgrade Kafka to 2.8.1

[jira] [Commented] (SPARK-37707) Allow store assignment between TimestampNTZ and Date/Timestamp

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492405#comment-17492405 ] Apache Spark commented on SPARK-37707: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-38213) support Metrics information report to kafkaSink.

2022-02-14 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492391#comment-17492391 ] Senthil Kumar commented on SPARK-38213: --- Working on this > support Metrics information report to

[jira] [Created] (SPARK-38213) support Metrics information report to kafkaSink.

2022-02-14 Thread YuanGuanhu (Jira)
YuanGuanhu created SPARK-38213: -- Summary: support Metrics information report to kafkaSink. Key: SPARK-38213 URL: https://issues.apache.org/jira/browse/SPARK-38213 Project: Spark Issue Type: New

[jira] [Assigned] (SPARK-38212) Streaming dropDuplicates should remove out-of-watermark keys

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38212: Assignee: (was: Apache Spark) > Streaming dropDuplicates should remove

[jira] [Commented] (SPARK-38212) Streaming dropDuplicates should remove out-of-watermark keys

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492386#comment-17492386 ] Apache Spark commented on SPARK-38212: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38212) Streaming dropDuplicates should remove out-of-watermark keys

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38212: Assignee: Apache Spark > Streaming dropDuplicates should remove out-of-watermark keys >

[jira] [Created] (SPARK-38212) Streaming dropDuplicates should remove out-of-watermark keys

2022-02-14 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-38212: --- Summary: Streaming dropDuplicates should remove out-of-watermark keys Key: SPARK-38212 URL: https://issues.apache.org/jira/browse/SPARK-38212 Project: Spark

[jira] [Commented] (SPARK-36808) Upgrade Kafka to 2.8.1

2022-02-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492372#comment-17492372 ] Dongjoon Hyun commented on SPARK-36808: --- +1 for backporting. Could you make a PR, please,

[jira] [Resolved] (SPARK-38203) Fix SQLInsertTestSuite and SchemaPruningSuite under ANSI mode

2022-02-14 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-38203. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35511

[jira] [Resolved] (SPARK-38145) Address the 'pyspark' tagged tests when "spark.sql.ansi.enabled" is True

2022-02-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38145. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35454

[jira] [Assigned] (SPARK-38145) Address the 'pyspark' tagged tests when "spark.sql.ansi.enabled" is True

2022-02-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38145: Assignee: Haejoon Lee > Address the 'pyspark' tagged tests when "spark.sql.ansi.enabled"

[jira] [Commented] (SPARK-37867) Compile aggregate functions of build-in JDBC dialect

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492351#comment-17492351 ] Apache Spark commented on SPARK-37867: -- User 'beliefer' has created a pull request for this issue:

[jira] [Commented] (SPARK-37867) Compile aggregate functions of build-in JDBC dialect

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492350#comment-17492350 ] Apache Spark commented on SPARK-37867: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38207) Add bucketed scan behavior change to migration guide

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38207: Assignee: Apache Spark > Add bucketed scan behavior change to migration guide >

[jira] [Commented] (SPARK-38207) Add bucketed scan behavior change to migration guide

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492346#comment-17492346 ] Apache Spark commented on SPARK-38207: -- User 'manuzhang' has created a pull request for this issue:

[jira] [Resolved] (SPARK-38211) Add SQL migration guide on restoring loose upcast from string

2022-02-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38211. -- Fix Version/s: 3.3.0 3.0.4 3.2.2

[jira] [Assigned] (SPARK-38207) Add bucketed scan behavior change to migration guide

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38207: Assignee: (was: Apache Spark) > Add bucketed scan behavior change to migration guide

[jira] [Assigned] (SPARK-38211) Add SQL migration guide on restoring loose upcast from string

2022-02-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38211: Assignee: Manu Zhang > Add SQL migration guide on restoring loose upcast from string >

[jira] [Resolved] (SPARK-38183) Show warning when creating pandas-on-Spark session under ANSI mode.

2022-02-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38183. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35488

[jira] [Assigned] (SPARK-38183) Show warning when creating pandas-on-Spark session under ANSI mode.

2022-02-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38183: Assignee: Haejoon Lee > Show warning when creating pandas-on-Spark session under ANSI

[jira] [Commented] (SPARK-38211) Add SQL migration guide on restoring loose upcast from string

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492325#comment-17492325 ] Apache Spark commented on SPARK-38211: -- User 'manuzhang' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38211) Add SQL migration guide on restoring loose upcast from string

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38211: Assignee: (was: Apache Spark) > Add SQL migration guide on restoring loose upcast

[jira] [Assigned] (SPARK-38211) Add SQL migration guide on restoring loose upcast from string

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38211: Assignee: Apache Spark > Add SQL migration guide on restoring loose upcast from string >

[jira] [Commented] (SPARK-38211) Add SQL migration guide on restoring loose upcast from string

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492324#comment-17492324 ] Apache Spark commented on SPARK-38211: -- User 'manuzhang' has created a pull request for this issue:

[jira] [Updated] (SPARK-38211) Add SQL migration guide on restoring loose upcast from string

2022-02-14 Thread Manu Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang updated SPARK-38211: --- Summary: Add SQL migration guide on restoring loose upcast from string (was: Add SQL migration

[jira] [Created] (SPARK-38211) Add SQL migration guide on preserving loose upcast

2022-02-14 Thread Manu Zhang (Jira)
Manu Zhang created SPARK-38211: -- Summary: Add SQL migration guide on preserving loose upcast Key: SPARK-38211 URL: https://issues.apache.org/jira/browse/SPARK-38211 Project: Spark Issue Type:

[jira] [Commented] (SPARK-35173) Support columns batch adding in PySpark.dataframe

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492320#comment-17492320 ] Apache Spark commented on SPARK-35173: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-38121) Use SparkSession instead of SQLContext inside PySpark

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492310#comment-17492310 ] Apache Spark commented on SPARK-38121: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-38121) Use SparkSession instead of SQLContext inside PySpark

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492309#comment-17492309 ] Apache Spark commented on SPARK-38121: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Resolved] (SPARK-38121) Use SparkSession instead of SQLContext inside PySpark

2022-02-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38121. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35410

[jira] [Assigned] (SPARK-38121) Use SparkSession instead of SQLContext inside PySpark

2022-02-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38121: Assignee: Hyukjin Kwon > Use SparkSession instead of SQLContext inside PySpark >

[jira] [Resolved] (SPARK-35173) Support columns batch adding in PySpark.dataframe

2022-02-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35173. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 32431

[jira] [Assigned] (SPARK-35173) Support columns batch adding in PySpark.dataframe

2022-02-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35173: Assignee: Yikun Jiang > Support columns batch adding in PySpark.dataframe >

[jira] [Commented] (SPARK-38210) Spark documentation build README is stale

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492254#comment-17492254 ] Apache Spark commented on SPARK-38210: -- User 'khalidmammadov' has created a pull request for this

[jira] [Assigned] (SPARK-38210) Spark documentation build README is stale

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38210: Assignee: (was: Apache Spark) > Spark documentation build README is stale >

[jira] [Commented] (SPARK-38210) Spark documentation build README is stale

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492253#comment-17492253 ] Apache Spark commented on SPARK-38210: -- User 'khalidmammadov' has created a pull request for this

[jira] [Assigned] (SPARK-38210) Spark documentation build README is stale

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38210: Assignee: Apache Spark > Spark documentation build README is stale >

[jira] [Updated] (SPARK-38183) Show warning when creating pandas-on-Spark session under ANSI mode.

2022-02-14 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-38183: Description: Since pandas API on Spark follows the behavior of pandas, not SQL, some unexpected

[jira] [Created] (SPARK-38210) Spark documentation build README is stale

2022-02-14 Thread Khalid Mammadov (Jira)
Khalid Mammadov created SPARK-38210: --- Summary: Spark documentation build README is stale Key: SPARK-38210 URL: https://issues.apache.org/jira/browse/SPARK-38210 Project: Spark Issue Type:

[jira] [Commented] (SPARK-36808) Upgrade Kafka to 2.8.1

2022-02-14 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492248#comment-17492248 ] Ismael Juma commented on SPARK-36808: - I think it would be good to backport this to the stable

[jira] [Commented] (SPARK-33913) Upgrade Kafka to 2.8.0

2022-02-14 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492247#comment-17492247 ] Ismael Juma commented on SPARK-33913: - Sounds good, I had not seen the other ticket. > Upgrade

[jira] [Commented] (SPARK-33913) Upgrade Kafka to 2.8.0

2022-02-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492242#comment-17492242 ] Dongjoon Hyun commented on SPARK-33913: --- Actually, you had better ping on SPARK-36808 (Upgrade

[jira] [Updated] (SPARK-38144) Remove unused `spark.storage.safetyFraction` config

2022-02-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38144: -- Fix Version/s: 3.1.4 (was: 3.1.3) > Remove unused

[jira] [Updated] (SPARK-38030) Query with cast containing non-nullable columns fails with AQE on Spark 3.1.1

2022-02-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38030: -- Fix Version/s: 3.1.4 (was: 3.1.3) > Query with cast containing

[jira] [Updated] (SPARK-38120) HiveExternalCatalog.listPartitions is failing when partition column name is upper case and dot in partition value

2022-02-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38120: -- Fix Version/s: 3.1.4 (was: 3.1.3) > HiveExternalCatalog.listPartitions

[jira] [Updated] (SPARK-38151) Handle `Pacific/Kanton` in DateTimeUtilsSuite

2022-02-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38151: -- Fix Version/s: 3.1.4 (was: 3.1.3) > Handle `Pacific/Kanton` in

[jira] [Updated] (SPARK-38056) Structured streaming not working in history server when using LevelDB

2022-02-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38056: -- Fix Version/s: 3.1.4 (was: 3.1.3) > Structured streaming not working

[jira] [Resolved] (SPARK-38208) 'Column' object is not callable

2022-02-14 Thread Joyce Arruda Recacho (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joyce Arruda Recacho resolved SPARK-38208. -- Fix Version/s: 3.1.2 Resolution: Fixed > 'Column' object is not

[jira] [Comment Edited] (SPARK-38208) 'Column' object is not callable

2022-02-14 Thread Joyce Arruda Recacho (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492114#comment-17492114 ] Joyce Arruda Recacho edited comment on SPARK-38208 at 2/14/22, 5:20 PM:

[jira] [Commented] (SPARK-38208) 'Column' object is not callable

2022-02-14 Thread Joyce Arruda Recacho (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492114#comment-17492114 ] Joyce Arruda Recacho commented on SPARK-38208: -- the correct is 'otherwise' and not

[jira] [Updated] (SPARK-38208) 'Column' object is not callable

2022-02-14 Thread Joyce Arruda Recacho (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joyce Arruda Recacho updated SPARK-38208: - Affects Version/s: 3.1.2 > 'Column' object is not callable >

[jira] [Updated] (SPARK-38198) Fix `QueryExecution.debug#toFile` use the passed in `maxFields` when `explainMode` is `CodegenMode`

2022-02-14 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-38198: - Fix Version/s: 3.2.2 > Fix `QueryExecution.debug#toFile` use the passed in `maxFields` when >

[jira] [Created] (SPARK-38209) Selectively include EXTERNAL TABLE source files via REGEX

2022-02-14 Thread melin (Jira)
melin created SPARK-38209: - Summary: Selectively include EXTERNAL TABLE source files via REGEX Key: SPARK-38209 URL: https://issues.apache.org/jira/browse/SPARK-38209 Project: Spark Issue Type:

[jira] [Created] (SPARK-38208) 'Column' object is not callable

2022-02-14 Thread Joyce Arruda Recacho (Jira)
Joyce Arruda Recacho created SPARK-38208: Summary: 'Column' object is not callable Key: SPARK-38208 URL: https://issues.apache.org/jira/browse/SPARK-38208 Project: Spark Issue Type:

[jira] [Commented] (SPARK-38198) Fix `QueryExecution.debug#toFile` use the passed in `maxFields` when `explainMode` is `CodegenMode`

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491980#comment-17491980 ] Apache Spark commented on SPARK-38198: -- User 'LuciferYang' has created a pull request for this

[jira] [Commented] (SPARK-38198) Fix `QueryExecution.debug#toFile` use the passed in `maxFields` when `explainMode` is `CodegenMode`

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491979#comment-17491979 ] Apache Spark commented on SPARK-38198: -- User 'LuciferYang' has created a pull request for this

[jira] [Resolved] (SPARK-38097) Improve the error for pivoting of unsupported value types

2022-02-14 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-38097. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35433

[jira] [Assigned] (SPARK-38097) Improve the error for pivoting of unsupported value types

2022-02-14 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-38097: Assignee: Yuto Akutsu > Improve the error for pivoting of unsupported value types >

[jira] [Commented] (SPARK-38027) Undefined link function causing error in GLM that uses Tweedie family

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491956#comment-17491956 ] Apache Spark commented on SPARK-38027: -- User 'manuzhang' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38027) Undefined link function causing error in GLM that uses Tweedie family

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38027: Assignee: Apache Spark > Undefined link function causing error in GLM that uses Tweedie

[jira] [Assigned] (SPARK-38027) Undefined link function causing error in GLM that uses Tweedie family

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38027: Assignee: (was: Apache Spark) > Undefined link function causing error in GLM that

[jira] [Created] (SPARK-38207) Add bucketed scan behavior change to migration guide

2022-02-14 Thread Manu Zhang (Jira)
Manu Zhang created SPARK-38207: -- Summary: Add bucketed scan behavior change to migration guide Key: SPARK-38207 URL: https://issues.apache.org/jira/browse/SPARK-38207 Project: Spark Issue Type:

[jira] [Commented] (SPARK-38033) The structured streaming processing cannot be started because the commitId and offsetId are inconsistent

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491946#comment-17491946 ] Apache Spark commented on SPARK-38033: -- User 'LLiu' has created a pull request for this issue:

[jira] [Commented] (SPARK-38033) The structured streaming processing cannot be started because the commitId and offsetId are inconsistent

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491944#comment-17491944 ] Apache Spark commented on SPARK-38033: -- User 'LLiu' has created a pull request for this issue:

[jira] [Created] (SPARK-38206) Relax the requirement of data type comparison for keys in stream-stream join

2022-02-14 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-38206: Summary: Relax the requirement of data type comparison for keys in stream-stream join Key: SPARK-38206 URL: https://issues.apache.org/jira/browse/SPARK-38206

[jira] [Created] (SPARK-38205) The columns in state schema should be relaxed to be nullable

2022-02-14 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-38205: Summary: The columns in state schema should be relaxed to be nullable Key: SPARK-38205 URL: https://issues.apache.org/jira/browse/SPARK-38205 Project: Spark

[jira] [Commented] (SPARK-38124) Revive HashClusteredDistribution and apply to stream-stream join

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491933#comment-17491933 ] Apache Spark commented on SPARK-38124: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Commented] (SPARK-38204) All state operators are at a risk of inconsistency between state partitioning and operator partitioning

2022-02-14 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491927#comment-17491927 ] Jungtaek Lim commented on SPARK-38204: -- Since this is not a regression, we could change the

[jira] [Created] (SPARK-38204) All state operators are at a risk of inconsistency between state partitioning and operator partitioning

2022-02-14 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-38204: Summary: All state operators are at a risk of inconsistency between state partitioning and operator partitioning Key: SPARK-38204 URL:

[jira] [Updated] (SPARK-38204) All state operators are at a risk of inconsistency between state partitioning and operator partitioning

2022-02-14 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-38204: - Labels: correctness (was: ) > All state operators are at a risk of inconsistency between state

[jira] [Assigned] (SPARK-38198) Fix `QueryExecution.debug#toFile` use the passed in `maxFields` when `explainMode` is `CodegenMode`

2022-02-14 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-38198: Assignee: Yang Jie > Fix `QueryExecution.debug#toFile` use the passed in `maxFields` when >

[jira] [Resolved] (SPARK-38198) Fix `QueryExecution.debug#toFile` use the passed in `maxFields` when `explainMode` is `CodegenMode`

2022-02-14 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-38198. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35506

[jira] [Commented] (SPARK-38203) Fix SQLInsertTestSuite and SchemaPruningSuite under ANSI mode

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491920#comment-17491920 ] Apache Spark commented on SPARK-38203: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-38203) Fix SQLInsertTestSuite and SchemaPruningSuite under ANSI mode

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38203: Assignee: Gengliang Wang (was: Apache Spark) > Fix SQLInsertTestSuite and

[jira] [Assigned] (SPARK-38203) Fix SQLInsertTestSuite and SchemaPruningSuite under ANSI mode

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38203: Assignee: Apache Spark (was: Gengliang Wang) > Fix SQLInsertTestSuite and

[jira] [Commented] (SPARK-38203) Fix SQLInsertTestSuite and SchemaPruningSuite under ANSI mode

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491918#comment-17491918 ] Apache Spark commented on SPARK-38203: -- User 'gengliangwang' has created a pull request for this

[jira] [Created] (SPARK-38203) Fix SQLInsertTestSuite and SchemaPruningSuite under ANSI mode

2022-02-14 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-38203: -- Summary: Fix SQLInsertTestSuite and SchemaPruningSuite under ANSI mode Key: SPARK-38203 URL: https://issues.apache.org/jira/browse/SPARK-38203 Project: Spark

[jira] [Updated] (SPARK-38154) Set up a new GA job to run tests with ANSI mode

2022-02-14 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-38154: --- Epic Link: SPARK-35030 > Set up a new GA job to run tests with ANSI mode >

[jira] [Commented] (SPARK-28329) SELECT INTO syntax

2022-02-14 Thread gabrywu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491884#comment-17491884 ] gabrywu commented on SPARK-28329: - [~smilegator] Is there a plan to support that? select into a scalar

[jira] [Assigned] (SPARK-38202) Invalid URL in SparkContext.addedJars will constantly fails Executor.run()

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38202: Assignee: Apache Spark > Invalid URL in SparkContext.addedJars will constantly fails

[jira] [Assigned] (SPARK-38202) Invalid URL in SparkContext.addedJars will constantly fails Executor.run()

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38202: Assignee: (was: Apache Spark) > Invalid URL in SparkContext.addedJars will

[jira] [Commented] (SPARK-38202) Invalid URL in SparkContext.addedJars will constantly fails Executor.run()

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491873#comment-17491873 ] Apache Spark commented on SPARK-38202: -- User 'bozhang2820' has created a pull request for this

[jira] [Commented] (SPARK-38201) Fix KubernetesUtils#uploadFileToHadoopCompatibleFS use passed in `delSrc` and `overwrite`

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491867#comment-17491867 ] Apache Spark commented on SPARK-38201: -- User 'LuciferYang' has created a pull request for this

[jira] [Commented] (SPARK-22605) OutputMetrics empty for DataFrame writes

2022-02-14 Thread zoli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491869#comment-17491869 ] zoli commented on SPARK-22605: -- Not solved, still an issue in Spark v3.1.2 for structured streaming. >

[jira] [Created] (SPARK-38202) Invalid URL in SparkContext.addedJars will constantly fails Executor.run()

2022-02-14 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-38202: Summary: Invalid URL in SparkContext.addedJars will constantly fails Executor.run() Key: SPARK-38202 URL: https://issues.apache.org/jira/browse/SPARK-38202 Project: Spark

[jira] [Assigned] (SPARK-38201) Fix KubernetesUtils#uploadFileToHadoopCompatibleFS use passed in `delSrc` and `overwrite`

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38201: Assignee: (was: Apache Spark) > Fix KubernetesUtils#uploadFileToHadoopCompatibleFS

[jira] [Assigned] (SPARK-38201) Fix KubernetesUtils#uploadFileToHadoopCompatibleFS use passed in `delSrc` and `overwrite`

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38201: Assignee: Apache Spark > Fix KubernetesUtils#uploadFileToHadoopCompatibleFS use passed

[jira] [Commented] (SPARK-38201) Fix KubernetesUtils#uploadFileToHadoopCompatibleFS use passed in `delSrc` and `overwrite`

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491866#comment-17491866 ] Apache Spark commented on SPARK-38201: -- User 'LuciferYang' has created a pull request for this

[jira] [Created] (SPARK-38201) Fix KubernetesUtils#uploadFileToHadoopCompatibleFS use passed in `delSrc` and `overwrite`

2022-02-14 Thread Yang Jie (Jira)
Yang Jie created SPARK-38201: Summary: Fix KubernetesUtils#uploadFileToHadoopCompatibleFS use passed in `delSrc` and `overwrite` Key: SPARK-38201 URL: https://issues.apache.org/jira/browse/SPARK-38201

[jira] [Commented] (SPARK-36488) "Invalid usage of '*' in expression" error due to the feature of 'quotedRegexColumnNames' in some scenarios.

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491859#comment-17491859 ] Apache Spark commented on SPARK-36488: -- User 'LuciferYang' has created a pull request for this

[jira] [Commented] (SPARK-36488) "Invalid usage of '*' in expression" error due to the feature of 'quotedRegexColumnNames' in some scenarios.

2022-02-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491858#comment-17491858 ] Apache Spark commented on SPARK-36488: -- User 'LuciferYang' has created a pull request for this

[jira] [Updated] (SPARK-38195) Add the TIMESTAMPADD() function

2022-02-14 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-38195: - Summary: Add the TIMESTAMPADD() function (was: Add the timestampadd() function) > Add the

[jira] [Commented] (SPARK-38200) [SQL] Spark JDBC Savemode Supports replace

2022-02-14 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491853#comment-17491853 ] melin commented on SPARK-38200: --- [~beliefer]  > [SQL] Spark JDBC Savemode Supports replace >

[jira] [Updated] (SPARK-38200) [SQL] Spark JDBC Savemode Supports replace

2022-02-14 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-38200: -- Description: When writing data into a relational database, data duplication needs to be considered. Both

[jira] [Created] (SPARK-38200) [SQL] Spark JDBC Savemode Supports replace

2022-02-14 Thread melin (Jira)
melin created SPARK-38200: - Summary: [SQL] Spark JDBC Savemode Supports replace Key: SPARK-38200 URL: https://issues.apache.org/jira/browse/SPARK-38200 Project: Spark Issue Type: Improvement

  1   2   >