[jira] [Resolved] (SPARK-36547) Downgrade scala-maven-plugin to 4.3.0

[ https://issues.apache.org/jira/browse/SPARK-36547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-36547. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33791

[jira] [Commented] (SPARK-35083) Support remote scheduler pool file

[ https://issues.apache.org/jira/browse/SPARK-35083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401980#comment-17401980 ] Apache Spark commented on SPARK-35083: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-36549) Add taskStatus support multiple value to doc

[ https://issues.apache.org/jira/browse/SPARK-36549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36549: Assignee: (was: Apache Spark) > Add taskStatus support multiple value to doc >

[jira] [Commented] (SPARK-36549) Add taskStatus support multiple value to doc

[ https://issues.apache.org/jira/browse/SPARK-36549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401973#comment-17401973 ] Apache Spark commented on SPARK-36549: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-36549) Add taskStatus support multiple value to doc

[ https://issues.apache.org/jira/browse/SPARK-36549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36549: Assignee: Apache Spark > Add taskStatus support multiple value to doc >

[jira] [Updated] (SPARK-36549) Add taskStatus support multiple value to doc

[ https://issues.apache.org/jira/browse/SPARK-36549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36549: -- Summary: Add taskStatus support multiple value to doc (was: Improve doc about [SPARK-34092][SQL]

[jira] [Created] (SPARK-36549) Improve doc about [SPARK-34092][SQL] Support Stage level restful api filter task details by task status

angerszhu created SPARK-36549: - Summary: Improve doc about [SPARK-34092][SQL] Support Stage level restful api filter task details by task status Key: SPARK-36549 URL: https://issues.apache.org/jira/browse/SPARK-36549

[jira] [Commented] (SPARK-35933) PartitionFilters and pushFilters not applied to window functions

[ https://issues.apache.org/jira/browse/SPARK-35933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401895#comment-17401895 ] Pablo Langa Blanco commented on SPARK-35933: can we close this ticket? > PartitionFilters

[jira] [Comment Edited] (SPARK-32985) Decouple bucket filter pruning and bucket table scan

[ https://issues.apache.org/jira/browse/SPARK-32985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401893#comment-17401893 ] Gabe Church edited comment on SPARK-32985 at 8/19/21, 11:02 PM: This is

[jira] [Comment Edited] (SPARK-32985) Decouple bucket filter pruning and bucket table scan

[ https://issues.apache.org/jira/browse/SPARK-32985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401893#comment-17401893 ] Gabe Church edited comment on SPARK-32985 at 8/19/21, 11:02 PM: This is

[jira] [Comment Edited] (SPARK-32985) Decouple bucket filter pruning and bucket table scan

[ https://issues.apache.org/jira/browse/SPARK-32985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401893#comment-17401893 ] Gabe Church edited comment on SPARK-32985 at 8/19/21, 11:01 PM: This is

[jira] [Commented] (SPARK-32985) Decouple bucket filter pruning and bucket table scan

[ https://issues.apache.org/jira/browse/SPARK-32985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401893#comment-17401893 ] Gabe Church commented on SPARK-32985: - This is a very important addition, thank you! Many cases

[jira] [Commented] (SPARK-35312) Introduce new Option in Kafka source to specify minimum number of records to read per trigger

[ https://issues.apache.org/jira/browse/SPARK-35312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401871#comment-17401871 ] Apache Spark commented on SPARK-35312: -- User 'xuanyuanking' has created a pull request for this

[jira] [Commented] (SPARK-35312) Introduce new Option in Kafka source to specify minimum number of records to read per trigger

[ https://issues.apache.org/jira/browse/SPARK-35312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401867#comment-17401867 ] Apache Spark commented on SPARK-35312: -- User 'xuanyuanking' has created a pull request for this

[jira] [Updated] (SPARK-36548) Throwing NoClassDefFoundError for Logging$class

[ https://issues.apache.org/jira/browse/SPARK-36548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sadagopan kalyanaraman updated SPARK-36548: --- Description: I'm getting NoClassDefFoundError. is it removed in 3.1.1?  

[jira] [Created] (SPARK-36548) Throwing NoClassDefFoundError for Logging$class

sadagopan kalyanaraman created SPARK-36548: -- Summary: Throwing NoClassDefFoundError for Logging$class Key: SPARK-36548 URL: https://issues.apache.org/jira/browse/SPARK-36548 Project: Spark

[jira] [Assigned] (SPARK-36547) Downgrade scala-maven-plugin to 4.3.0

[ https://issues.apache.org/jira/browse/SPARK-36547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36547: Assignee: Apache Spark (was: Gengliang Wang) > Downgrade scala-maven-plugin to 4.3.0 >

[jira] [Assigned] (SPARK-36547) Downgrade scala-maven-plugin to 4.3.0

[ https://issues.apache.org/jira/browse/SPARK-36547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36547: Assignee: Gengliang Wang (was: Apache Spark) > Downgrade scala-maven-plugin to 4.3.0 >

[jira] [Commented] (SPARK-36547) Downgrade scala-maven-plugin to 4.3.0

[ https://issues.apache.org/jira/browse/SPARK-36547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401809#comment-17401809 ] Apache Spark commented on SPARK-36547: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-36547) Downgrade scala-maven-plugin to 4.3.0

[ https://issues.apache.org/jira/browse/SPARK-36547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401808#comment-17401808 ] Apache Spark commented on SPARK-36547: -- User 'gengliangwang' has created a pull request for this

[jira] [Created] (SPARK-36547) Downgrade scala-maven-plugin to 4.3.0

Gengliang Wang created SPARK-36547: -- Summary: Downgrade scala-maven-plugin to 4.3.0 Key: SPARK-36547 URL: https://issues.apache.org/jira/browse/SPARK-36547 Project: Spark Issue Type:

[jira] [Updated] (SPARK-34305) Unify v1 and v2 ALTER TABLE .. SET SERDE tests

[ https://issues.apache.org/jira/browse/SPARK-34305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-34305: --- Fix Version/s: (was: 3.2.0) 3.3.0 > Unify v1 and v2 ALTER TABLE ..

[jira] [Updated] (SPARK-34332) Unify v1 and v2 ALTER TABLE .. SET LOCATION tests

[ https://issues.apache.org/jira/browse/SPARK-34332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-34332: --- Fix Version/s: (was: 3.2.0) 3.3.0 > Unify v1 and v2 ALTER TABLE ..

[jira] [Updated] (SPARK-36082) when the right side is small enough to use SingleColumn Null Aware Anti Join

[ https://issues.apache.org/jira/browse/SPARK-36082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-36082: --- Fix Version/s: (was: 3.2.0) 3.3.0 > when the right side is small

[jira] [Resolved] (SPARK-34884) Improve dynamic partition pruning evaluation

[ https://issues.apache.org/jira/browse/SPARK-34884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-34884. Resolution: Fixed > Improve dynamic partition pruning evaluation >

[jira] [Commented] (SPARK-35083) Support remote scheduler pool file

[ https://issues.apache.org/jira/browse/SPARK-35083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401686#comment-17401686 ] Apache Spark commented on SPARK-35083: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-35083) Support remote scheduler pool file

[ https://issues.apache.org/jira/browse/SPARK-35083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401685#comment-17401685 ] Apache Spark commented on SPARK-35083: -- User 'gengliangwang' has created a pull request for this

[jira] [Updated] (SPARK-36546) Make unionByName null-filling behavior work with array of struct columns

[ https://issues.apache.org/jira/browse/SPARK-36546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishal Dhavale updated SPARK-36546: --- Description: Currently, unionByName workes with two DataFrames with slightly different

[jira] [Updated] (SPARK-36546) Make unionByName null-filling behavior work with array of struct columns

[ https://issues.apache.org/jira/browse/SPARK-36546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishal Dhavale updated SPARK-36546: --- Description: Currently, unionByName workes with two DataFrames with slightly different

[jira] [Updated] (SPARK-36545) sparkstreaming input rate exceed spark.streaming.kafka.maxRatePerPartition

[ https://issues.apache.org/jira/browse/SPARK-36545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] handong updated SPARK-36545: Description:   i have a spark streaming application with kafka .  Here are the parameters: kafka

[jira] [Updated] (SPARK-36545) sparkstreaming input rate exceed spark.streaming.kafka.maxRatePerPartition

[ https://issues.apache.org/jira/browse/SPARK-36545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] handong updated SPARK-36545: Description: !kafka.png!   was: !kafka.png! i have a spark streaming application with kafka .  Here

[jira] [Updated] (SPARK-36545) sparkstreaming input rate exceed spark.streaming.kafka.maxRatePerPartition

[ https://issues.apache.org/jira/browse/SPARK-36545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] handong updated SPARK-36545: Description: !kafka.png! i have a spark streaming application with kafka .  Here are the parameters:

[jira] [Updated] (SPARK-36545) sparkstreaming input rate exceed spark.streaming.kafka.maxRatePerPartition

[ https://issues.apache.org/jira/browse/SPARK-36545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] handong updated SPARK-36545: Attachment: kafka.png > sparkstreaming input rate exceed spark.streaming.kafka.maxRatePerPartition >

[jira] [Updated] (SPARK-36546) Make unionByName null-filling behavior work with array of struct columns

[ https://issues.apache.org/jira/browse/SPARK-36546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishal Dhavale updated SPARK-36546: --- Description: Currently, unionByName workes with two DataFrames with slightly different

[jira] [Updated] (SPARK-36545) sparkstreaming input rate exceed spark.streaming.kafka.maxRatePerPartition

[ https://issues.apache.org/jira/browse/SPARK-36545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] handong updated SPARK-36545: Component/s: YARN Spark Submit Description: i have a spark streaming application

[jira] [Updated] (SPARK-36546) Make unionByName null-filling behavior work with array of struct columns

[ https://issues.apache.org/jira/browse/SPARK-36546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishal Dhavale updated SPARK-36546: --- Description: Currently, unionByName workes with two DataFrames with slightly different

[jira] [Created] (SPARK-36546) Make unionByName null-filling behavior work with array of struct columns

Vishal Dhavale created SPARK-36546: -- Summary: Make unionByName null-filling behavior work with array of struct columns Key: SPARK-36546 URL: https://issues.apache.org/jira/browse/SPARK-36546

[jira] [Created] (SPARK-36545) sparkstreaming input rate exceed spark.streaming.kafka.maxRatePerPartition

handong created SPARK-36545: --- Summary: sparkstreaming input rate exceed spark.streaming.kafka.maxRatePerPartition Key: SPARK-36545 URL: https://issues.apache.org/jira/browse/SPARK-36545 Project: Spark

[jira] [Commented] (SPARK-36428) the 'seconds' parameter of 'make_timestamp' should accept integer type

[ https://issues.apache.org/jira/browse/SPARK-36428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401598#comment-17401598 ] Apache Spark commented on SPARK-36428: -- User 'beliefer' has created a pull request for this issue:

[jira] [Commented] (SPARK-36428) the 'seconds' parameter of 'make_timestamp' should accept integer type

[ https://issues.apache.org/jira/browse/SPARK-36428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401596#comment-17401596 ] Apache Spark commented on SPARK-36428: -- User 'beliefer' has created a pull request for this issue:

[jira] [Resolved] (SPARK-36538) Environment variables part in config doc isn't properly documented.

[ https://issues.apache.org/jira/browse/SPARK-36538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuto Akutsu resolved SPARK-36538. - Resolution: Invalid > Environment variables part in config doc isn't properly documented. >

[jira] [Assigned] (SPARK-36541) Images used in pyspark documentation still use Koalas

[ https://issues.apache.org/jira/browse/SPARK-36541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36541: Assignee: Apache Spark > Images used in pyspark documentation still use Koalas >

[jira] [Assigned] (SPARK-36541) Images used in pyspark documentation still use Koalas

[ https://issues.apache.org/jira/browse/SPARK-36541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36541: Assignee: (was: Apache Spark) > Images used in pyspark documentation still use

[jira] [Commented] (SPARK-36541) Images used in pyspark documentation still use Koalas

[ https://issues.apache.org/jira/browse/SPARK-36541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401567#comment-17401567 ] Apache Spark commented on SPARK-36541: -- User 'yoda-mon' has created a pull request for this issue:

[jira] [Updated] (SPARK-36444) Remove OptimizeSubqueries from batch of PartitionPruning

[ https://issues.apache.org/jira/browse/SPARK-36444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-36444: Affects Version/s: 3.0.3 3.1.2 > Remove OptimizeSubqueries from batch of

[jira] [Updated] (SPARK-36444) Remove OptimizeSubqueries from batch of PartitionPruning

[ https://issues.apache.org/jira/browse/SPARK-36444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-36444: Affects Version/s: (was: 3.3.0) 3.2.0 > Remove OptimizeSubqueries from

[jira] [Resolved] (SPARK-36444) Remove OptimizeSubqueries from batch of PartitionPruning

[ https://issues.apache.org/jira/browse/SPARK-36444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-36444. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33664

[jira] [Assigned] (SPARK-36444) Remove OptimizeSubqueries from batch of PartitionPruning

[ https://issues.apache.org/jira/browse/SPARK-36444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-36444: --- Assignee: Yuming Wang > Remove OptimizeSubqueries from batch of PartitionPruning >

[jira] [Comment Edited] (SPARK-30873) Handling Node Decommissioning for Yarn cluster manger in Spark

[ https://issues.apache.org/jira/browse/SPARK-30873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401514#comment-17401514 ] Saurabh Chawla edited comment on SPARK-30873 at 8/19/21, 7:56 AM: --

[jira] [Commented] (SPARK-35083) Support remote scheduler pool file

[ https://issues.apache.org/jira/browse/SPARK-35083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401517#comment-17401517 ] Apache Spark commented on SPARK-35083: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-30873) Handling Node Decommissioning for Yarn cluster manger in Spark

[ https://issues.apache.org/jira/browse/SPARK-30873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401514#comment-17401514 ] Saurabh Chawla edited comment on SPARK-30873 at 8/19/21, 7:46 AM: --

[jira] [Comment Edited] (SPARK-30873) Handling Node Decommissioning for Yarn cluster manger in Spark

[ https://issues.apache.org/jira/browse/SPARK-30873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401514#comment-17401514 ] Saurabh Chawla edited comment on SPARK-30873 at 8/19/21, 7:35 AM: --

[jira] [Commented] (SPARK-30873) Handling Node Decommissioning for Yarn cluster manger in Spark

[ https://issues.apache.org/jira/browse/SPARK-30873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401514#comment-17401514 ] Saurabh Chawla commented on SPARK-30873: [~kings129] - The backport of the 

[jira] [Resolved] (SPARK-36519) Store the RocksDB format in the checkpoint for a streaming query

[ https://issues.apache.org/jira/browse/SPARK-36519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-36519. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33749

[jira] [Resolved] (SPARK-36536) Split the JSON/CSV option of datetime format to in read and in write

[ https://issues.apache.org/jira/browse/SPARK-36536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-36536. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33769