[jira] [Updated] (SPARK-48421) SPJ: Add documentation

[ https://issues.apache.org/jira/browse/SPARK-48421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48421: --- Labels: pull-request-available (was: ) > SPJ: Add documentation > -- >

[jira] [Created] (SPARK-48421) SPJ: Add documentation

Szehon Ho created SPARK-48421: - Summary: SPJ: Add documentation Key: SPARK-48421 URL: https://issues.apache.org/jira/browse/SPARK-48421 Project: Spark Issue Type: Documentation Componen

[jira] [Updated] (SPARK-48420) Upgrade netty to `4.1.110.Final`

[ https://issues.apache.org/jira/browse/SPARK-48420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48420: --- Labels: pull-request-available (was: ) > Upgrade netty to `4.1.110.Final` > ---

[jira] [Created] (SPARK-48420) Upgrade netty to `4.1.110.Final`

BingKun Pan created SPARK-48420: --- Summary: Upgrade netty to `4.1.110.Final` Key: SPARK-48420 URL: https://issues.apache.org/jira/browse/SPARK-48420 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-48419) Foldable propagation replace foldable column should use origin column

[ https://issues.apache.org/jira/browse/SPARK-48419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48419: --- Labels: pull-request-available (was: ) > Foldable propagation replace foldable column shoul

[jira] [Updated] (SPARK-48419) Foldable propagation replace foldable column should use origin column

[ https://issues.apache.org/jira/browse/SPARK-48419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KnightChess updated SPARK-48419: Summary: Foldable propagation replace foldable column should use origin column (was: Foldable pro

[jira] [Created] (SPARK-48419) Foldable propagation change output schema

KnightChess created SPARK-48419: --- Summary: Foldable propagation change output schema Key: SPARK-48419 URL: https://issues.apache.org/jira/browse/SPARK-48419 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-48418) Spark structured streaming: Add microbatch timestamp to foreachBatch method

[ https://issues.apache.org/jira/browse/SPARK-48418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anil Dasari updated SPARK-48418: Description: We are on Spark 3.x and using Spark dstream + kafka and planning to use structured s

[jira] [Created] (SPARK-48418) Spark structured streaming: Add microbatch timestamp to foreachBatch method

Anil Dasari created SPARK-48418: --- Summary: Spark structured streaming: Add microbatch timestamp to foreachBatch method Key: SPARK-48418 URL: https://issues.apache.org/jira/browse/SPARK-48418 Project: Sp

[jira] [Updated] (SPARK-47041) PushDownUtils uses FileScanBuilder instead of SupportsPushDownCatalystFilters trait

[ https://issues.apache.org/jira/browse/SPARK-47041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47041: --- Labels: pull-request-available (was: ) > PushDownUtils uses FileScanBuilder instead of Supp

[jira] [Updated] (SPARK-48416) Support related nested WITH expression

[ https://issues.apache.org/jira/browse/SPARK-48416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48416: --- Labels: pull-request-available (was: ) > Support related nested WITH expression > -

[jira] [Resolved] (SPARK-48394) Cleanup mapIdToMapIndex on mapoutput unregister

[ https://issues.apache.org/jira/browse/SPARK-48394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48394. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46706 [https://

[jira] [Assigned] (SPARK-48394) Cleanup mapIdToMapIndex on mapoutput unregister

[ https://issues.apache.org/jira/browse/SPARK-48394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48394: - Assignee: wuyi > Cleanup mapIdToMapIndex on mapoutput unregister >

[jira] [Resolved] (SPARK-48407) Teradata: Document Type Conversion rules between Spark SQL and teradata

[ https://issues.apache.org/jira/browse/SPARK-48407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48407. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46728 [https://

[jira] [Resolved] (SPARK-48325) Always specify messages in ExecutorRunner.killProcess

[ https://issues.apache.org/jira/browse/SPARK-48325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48325. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46641 [https://

[jira] [Commented] (SPARK-48417) Filesystems do not load with spark.jars.packages configuration

[ https://issues.apache.org/jira/browse/SPARK-48417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17849399#comment-17849399 ] Ravi Dalal commented on SPARK-48417: For anyone facing this issue, use following con

[jira] [Closed] (SPARK-48417) Filesystems do not load with spark.jars.packages configuration

[ https://issues.apache.org/jira/browse/SPARK-48417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Dalal closed SPARK-48417. -- > Filesystems do not load with spark.jars.packages configuration > ---

[jira] [Resolved] (SPARK-48417) Filesystems do not load with spark.jars.packages configuration

[ https://issues.apache.org/jira/browse/SPARK-48417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Dalal resolved SPARK-48417. Resolution: Not A Problem Apologies. We missed a configuration parameter. Found it after creating

[jira] [Updated] (SPARK-48417) Filesystems do not load with spark.jars.packages configuration

[ https://issues.apache.org/jira/browse/SPARK-48417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Dalal updated SPARK-48417: --- Attachment: pyspark_mleap.py pyspark_spark_jar_package_config_logs.txt

[jira] [Created] (SPARK-48417) Filesystems do not load with spark.jars.packages configuration

Ravi Dalal created SPARK-48417: -- Summary: Filesystems do not load with spark.jars.packages configuration Key: SPARK-48417 URL: https://issues.apache.org/jira/browse/SPARK-48417 Project: Spark I

[jira] [Updated] (SPARK-48411) Add E2E test for DropDuplicateWithinWatermark

[ https://issues.apache.org/jira/browse/SPARK-48411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48411: --- Labels: pull-request-available (was: ) > Add E2E test for DropDuplicateWithinWatermark > --

[jira] [Updated] (SPARK-45373) Minimizing calls to HiveMetaStore layer for getting partitions, when tables are repeated

[ https://issues.apache.org/jira/browse/SPARK-45373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-45373: - Priority: Major (was: Minor) > Minimizing calls to HiveMetaStore layer for getting partitions, when tables >

[jira] [Commented] (SPARK-48411) Add E2E test for DropDuplicateWithinWatermark

[ https://issues.apache.org/jira/browse/SPARK-48411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17849356#comment-17849356 ] Yuchen Liu commented on SPARK-48411: I will work on this. > Add E2E test for DropDu

[jira] [Commented] (SPARK-48411) Add E2E test for DropDuplicateWithinWatermark

[ https://issues.apache.org/jira/browse/SPARK-48411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17849348#comment-17849348 ] Wei Liu commented on SPARK-48411: - Sorry I tagged the wrong Yuchen > Add E2E test f

[jira] [Updated] (SPARK-48338) Sql Scripting support for Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-48338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Milicevic updated SPARK-48338: Attachment: [Design Doc] Sql Scripting - OSS.pdf > Sql Scripting support for Spark SQL > -

[jira] [Updated] (SPARK-48343) Interpreter support

[ https://issues.apache.org/jira/browse/SPARK-48343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Milicevic updated SPARK-48343: Description: Implement interpreter for SQL scripting: * Interpreter * Interpreter testin

[jira] [Created] (SPARK-48416) Support related nested WITH expression

Mingliang Zhu created SPARK-48416: - Summary: Support related nested WITH expression Key: SPARK-48416 URL: https://issues.apache.org/jira/browse/SPARK-48416 Project: Spark Issue Type: Improvem

[jira] [Commented] (SPARK-48388) Fix SET behavior for scripts

[ https://issues.apache.org/jira/browse/SPARK-48388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17849301#comment-17849301 ] David Milicevic commented on SPARK-48388: - Already have ready changes. I will wo

[jira] [Updated] (SPARK-48415) TypeName support parameterized datatypes

[ https://issues.apache.org/jira/browse/SPARK-48415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48415: --- Labels: pull-request-available (was: ) > TypeName support parameterized datatypes > ---

[jira] [Created] (SPARK-48415) TypeName support parameterized datatypes

Ruifeng Zheng created SPARK-48415: - Summary: TypeName support parameterized datatypes Key: SPARK-48415 URL: https://issues.apache.org/jira/browse/SPARK-48415 Project: Spark Issue Type: New Fe

[jira] [Commented] (SPARK-21720) Filter predicate with many conditions throw stackoverflow error

[ https://issues.apache.org/jira/browse/SPARK-21720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17849267#comment-17849267 ] Abhishek Singh commented on SPARK-21720: Hi, I m using spark version 2.4 I m get

[jira] [Updated] (SPARK-48414) Fix breaking change in python's `fromJson`

[ https://issues.apache.org/jira/browse/SPARK-48414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48414: --- Labels: pull-request-available (was: ) > Fix breaking change in python's `fromJson` > -

[jira] [Resolved] (SPARK-48168) Add bitwise shifting operators support

[ https://issues.apache.org/jira/browse/SPARK-48168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You resolved SPARK-48168. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46440 [https://github.c

[jira] [Resolved] (SPARK-48412) Refactor data type json parse

[ https://issues.apache.org/jira/browse/SPARK-48412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48412. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46733 [https://

[jira] [Resolved] (SPARK-48409) Upgrade MySQL & Postgres & mariadb docker image version

[ https://issues.apache.org/jira/browse/SPARK-48409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-48409. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46704 [https://github.com

[jira] [Assigned] (SPARK-48409) Upgrade MySQL & Postgres & mariadb docker image version

[ https://issues.apache.org/jira/browse/SPARK-48409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-48409: Assignee: BingKun Pan > Upgrade MySQL & Postgres & mariadb docker image version > ---

[jira] [Updated] (SPARK-48409) Upgrade MySQL & Postgres & mariadb docker image version

[ https://issues.apache.org/jira/browse/SPARK-48409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48409: --- Labels: pull-request-available (was: ) > Upgrade MySQL & Postgres & mariadb docker image ve

[jira] [Resolved] (SPARK-48384) Exclude `io.netty:netty-tcnative-boringssl-static` from `zookeeper`

[ https://issues.apache.org/jira/browse/SPARK-48384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-48384. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46695 [https://github.com

[jira] [Assigned] (SPARK-48384) Exclude `io.netty:netty-tcnative-boringssl-static` from `zookeeper`

[ https://issues.apache.org/jira/browse/SPARK-48384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-48384: Assignee: BingKun Pan > Exclude `io.netty:netty-tcnative-boringssl-static` from `zookeeper` > ---

[jira] [Updated] (SPARK-48413) ALTER COLUMN with collation

[ https://issues.apache.org/jira/browse/SPARK-48413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48413: --- Labels: pull-request-available (was: ) > ALTER COLUMN with collation >

[jira] [Updated] (SPARK-48412) Refactor data type json parse

[ https://issues.apache.org/jira/browse/SPARK-48412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48412: --- Labels: pull-request-available (was: ) > Refactor data type json parse > --

[jira] [Updated] (SPARK-48413) ALTER COLUMN with collation

[ https://issues.apache.org/jira/browse/SPARK-48413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikola Mandic updated SPARK-48413: -- Epic Link: SPARK-46830 > ALTER COLUMN with collation > --- > >

[jira] [Created] (SPARK-48413) ALTER COLUMN with collation

Nikola Mandic created SPARK-48413: - Summary: ALTER COLUMN with collation Key: SPARK-48413 URL: https://issues.apache.org/jira/browse/SPARK-48413 Project: Spark Issue Type: Task Comp

[jira] [Created] (SPARK-48412) Refactor data type json parse

Ruifeng Zheng created SPARK-48412: - Summary: Refactor data type json parse Key: SPARK-48412 URL: https://issues.apache.org/jira/browse/SPARK-48412 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-48410) Fix InitCap expression

[ https://issues.apache.org/jira/browse/SPARK-48410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48410: --- Labels: pull-request-available (was: ) > Fix InitCap expression > -- >

[jira] [Created] (SPARK-48411) Add E2E test for DropDuplicateWithinWatermark

Wei Liu created SPARK-48411: --- Summary: Add E2E test for DropDuplicateWithinWatermark Key: SPARK-48411 URL: https://issues.apache.org/jira/browse/SPARK-48411 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-48411) Add E2E test for DropDuplicateWithinWatermark

[ https://issues.apache.org/jira/browse/SPARK-48411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17849202#comment-17849202 ] Wei Liu commented on SPARK-48411: - [~liuyuchen777] is going to work on this > Add E2E t

[jira] [Updated] (SPARK-47257) Assign error classes to ALTER COLUMN errors

[ https://issues.apache.org/jira/browse/SPARK-47257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47257: --- Labels: pull-request-available starter (was: starter) > Assign error classes to ALTER COLUM

[jira] [Updated] (SPARK-48403) Fix Upper & Lower expressions for UTF8_BINARY_LCASE

[ https://issues.apache.org/jira/browse/SPARK-48403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-48403: - Summary: Fix Upper & Lower expressions for UTF8_BINARY_LCASE (was: Fix Upper & Lower expression

[jira] [Created] (SPARK-48410) Fix InitCap expression

Uroš Bojanić created SPARK-48410: Summary: Fix InitCap expression Key: SPARK-48410 URL: https://issues.apache.org/jira/browse/SPARK-48410 Project: Spark Issue Type: Sub-task Compone

[jira] [Updated] (SPARK-48403) Fix Upper & Lower expressions for UTF8_BINARY_LCASE)

[ https://issues.apache.org/jira/browse/SPARK-48403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-48403: - Summary: Fix Upper & Lower expressions for UTF8_BINARY_LCASE) (was: Fix Upper, Lower, InitCap f

[jira] [Created] (SPARK-48409) Upgrade MySQL & Postgres & mariadb docker image version

BingKun Pan created SPARK-48409: --- Summary: Upgrade MySQL & Postgres & mariadb docker image version Key: SPARK-48409 URL: https://issues.apache.org/jira/browse/SPARK-48409 Project: Spark Issue T