[jira] [Assigned] (SPARK-48580) Add consistency check and fallback for mapIds in push-merged block meta

2024-06-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48580: -- Assignee: Apache Spark > Add consistency check and fallback for mapIds in p

[jira] [Assigned] (SPARK-48582) Bump `braces` from 3.0.2 to 3.0.3 in /ui-test

2024-06-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48582: -- Assignee: Apache Spark > Bump `braces` from 3.0.2 to 3.0.3 in /ui-t

[jira] [Updated] (SPARK-48580) Add consistency check and fallback for mapIds in push-merged block meta

2024-06-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48580: --- Labels: pull-request-available (was: ) > Add consistency check and fallback for map

[jira] [Updated] (SPARK-48580) Add consistency check and fallback for mapIds in push-merged block meta

2024-06-11 Thread gaoyajun02 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gaoyajun02 updated SPARK-48580: --- Parent: SPARK-33235 Issue Type: Sub-task (was: Bug) > Add consistency check and fallb

[jira] [Updated] (SPARK-48580) Add consistency check and fallback for mapIds in push-merged block meta

2024-06-11 Thread gaoyajun02 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gaoyajun02 updated SPARK-48580: --- Summary: Add consistency check and fallback for mapIds in push-merged block meta (was: MergedBlock

[jira] [Resolved] (SPARK-48551) Perf improvement for escapePathName

2024-06-11 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-48551. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46894 [https

[jira] [Updated] (SPARK-48583) Replace deprecated `FileUtils#writeStringToFile`

2024-06-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48583: --- Labels: pull-request-available (was: ) > Replace deprecated `FileUtils#writeStringToF

[jira] [Created] (SPARK-48583) Replace deprecated `FileUtils#writeStringToFile`

2024-06-11 Thread Wei Guo (Jira)
Wei Guo created SPARK-48583: --- Summary: Replace deprecated `FileUtils#writeStringToFile` Key: SPARK-48583 URL: https://issues.apache.org/jira/browse/SPARK-48583 Project: Spark Issue Type

[jira] [Comment Edited] (SPARK-48562) Writing to JDBC Temporary View Failed

2024-06-11 Thread Junqing Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853926#comment-17853926 ] Junqing Li edited comment on SPARK-48562 at 6/11/24 7:31 AM: - [~cloud_fan

[jira] [Comment Edited] (SPARK-48562) Writing to JDBC Temporary View Failed

2024-06-11 Thread Junqing Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853926#comment-17853926 ] Junqing Li edited comment on SPARK-48562 at 6/11/24 7:10 AM: - [~cloud_fan

[jira] [Commented] (SPARK-48562) Writing to JDBC Temporary View Failed

2024-06-11 Thread Junqing Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853926#comment-17853926 ] Junqing Li commented on SPARK-48562: [~cloud_fan] Maybe we need to move *{color:#172b4d

[jira] [Updated] (SPARK-48562) Writing to JDBC Temporary View Failed

2024-06-11 Thread Junqing Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junqing Li updated SPARK-48562: --- Description: When creating a JDBC temporary view, *ApplyCharTypePadding* would add a Project

[jira] [Updated] (SPARK-48578) Add new expressions for UTF8 string validation

2024-06-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48578: --- Labels: pull-request-available (was: ) > Add new expressions for UTF8 string validat

[jira] [Updated] (SPARK-48577) Replace invalid byte sequences in UTF8Strings

2024-06-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48577: --- Labels: pull-request-available (was: ) > Replace invalid byte sequences in UTF8Stri

[jira] [Updated] (SPARK-48582) Bump `braces` from 3.0.2 to 3.0.3 in /ui-test

2024-06-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48582: --- Labels: pull-request-available (was: ) > Bump `braces` from 3.0.2 to 3.0.3 in /ui-t

[jira] [Created] (SPARK-48582) Bump `braces` from 3.0.2 to 3.0.3 in /ui-test

2024-06-10 Thread Yang Jie (Jira)
Yang Jie created SPARK-48582: Summary: Bump `braces` from 3.0.2 to 3.0.3 in /ui-test Key: SPARK-48582 URL: https://issues.apache.org/jira/browse/SPARK-48582 Project: Spark Issue Type

[jira] [Updated] (SPARK-48581) Upgrade dropwizard metrics to 4.2.26

2024-06-10 Thread Wei Guo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Guo updated SPARK-48581: Summary: Upgrade dropwizard metrics to 4.2.26 (was: Upgrade dropwizard metrics 4.2.26) > Upgr

[jira] [Updated] (SPARK-48581) Upgrade dropwizard metrics 4.2.26

2024-06-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48581: --- Labels: pull-request-available (was: ) > Upgrade dropwizard metrics 4.2

[jira] [Created] (SPARK-48581) Upgrade dropwizard metrics 4.2.26

2024-06-10 Thread Wei Guo (Jira)
Wei Guo created SPARK-48581: --- Summary: Upgrade dropwizard metrics 4.2.26 Key: SPARK-48581 URL: https://issues.apache.org/jira/browse/SPARK-48581 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-48565) Fix thread dump display in UI

2024-06-10 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-48565. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46916 [https

[jira] [Assigned] (SPARK-48565) Fix thread dump display in UI

2024-06-10 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-48565: Assignee: Cheng Pan > Fix thread dump display in

[jira] [Resolved] (SPARK-48563) Upgrade pickle to 1.5

2024-06-10 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-48563. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46913 [https

[jira] [Updated] (SPARK-48580) MergedBlock read by reduce have missing chunks, leading to inconsistent shuffle data

2024-06-10 Thread gaoyajun02 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gaoyajun02 updated SPARK-48580: --- Description: When push-based shuffle enabled, 0.03% of the spark application in our cluster

[jira] [Updated] (SPARK-48580) MergedBlock read by reduce have missing chunks, leading to inconsistent shuffle data

2024-06-10 Thread gaoyajun02 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gaoyajun02 updated SPARK-48580: --- Description: When push-based shuffle enabled, 0.03% of the spark application in our cluster

[jira] [Updated] (SPARK-48580) MergedBlock read by reduce have missing chunks, leading to inconsistent shuffle data

2024-06-10 Thread gaoyajun02 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gaoyajun02 updated SPARK-48580: --- Attachment: (was: image-2024-06-11-10-19-22-284.png) > MergedBlock read by reduce have miss

[jira] [Updated] (SPARK-48580) MergedBlock read by reduce have missing chunks, leading to inconsistent shuffle data

2024-06-10 Thread gaoyajun02 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gaoyajun02 updated SPARK-48580: --- Attachment: image-2024-06-11-10-19-57-227.png > MergedBlock read by reduce have missing chu

[jira] [Updated] (SPARK-48580) MergedBlock read by reduce have missing chunks, leading to inconsistent shuffle data

2024-06-10 Thread gaoyajun02 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gaoyajun02 updated SPARK-48580: --- Attachment: image-2024-06-11-10-19-22-284.png > MergedBlock read by reduce have missing chu

[jira] [Updated] (SPARK-48580) MergedBlock read by reduce have missing chunks, leading to inconsistent shuffle data

2024-06-10 Thread gaoyajun02 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gaoyajun02 updated SPARK-48580: --- Summary: MergedBlock read by reduce have missing chunks, leading to inconsistent shuffle data

[jira] [Created] (SPARK-48580) The merge blocks read by reduce have missing chunks, leading to inconsistent shuffle data

2024-06-10 Thread gaoyajun02 (Jira)
gaoyajun02 created SPARK-48580: -- Summary: The merge blocks read by reduce have missing chunks, leading to inconsistent shuffle data Key: SPARK-48580 URL: https://issues.apache.org/jira/browse/SPARK-48580

[jira] [Updated] (SPARK-48480) StreamingQueryListener thread should not be interruptable

2024-06-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48480: --- Labels: pull-request-available (was: ) > StreamingQueryListener thread sho

[jira] [Resolved] (SPARK-48569) Connect - StreamingQuery.name should return null when not specified

2024-06-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48569. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46920 [https

[jira] [Assigned] (SPARK-48410) Fix InitCap expression

2024-06-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48410: --- Assignee: Uroš Bojanić > Fix InitCap express

[jira] [Resolved] (SPARK-48410) Fix InitCap expression

2024-06-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48410. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46732 [https

[jira] [Resolved] (SPARK-48403) Fix Upper & Lower expressions for UTF8_BINARY_LCASE

2024-06-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48403. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46720 [https

[jira] [Resolved] (SPARK-48564) Propagate cached schema in set operations

2024-06-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48564. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46915 [https

[jira] [Assigned] (SPARK-48564) Propagate cached schema in set operations

2024-06-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48564: Assignee: Ruifeng Zheng > Propagate cached schema in set operati

[jira] [Assigned] (SPARK-48342) [M0] Parser support

2024-06-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48342: -- Assignee: (was: Apache Spark) > [M0] Parser supp

[jira] [Assigned] (SPARK-48342) [M0] Parser support

2024-06-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48342: -- Assignee: Apache Spark > [M0] Parser supp

[jira] [Updated] (SPARK-48579) [M1] Merge DatabricksSqlParser with SparkSqlParser

2024-06-10 Thread David Milicevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Milicevic updated SPARK-48579: Description: OSS has single parser for SQL statements - SparkSqlParser. However

[jira] [Updated] (SPARK-48579) [M1] Merge DatabricksSqlParser with SparkSqlParser

2024-06-10 Thread David Milicevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Milicevic updated SPARK-48579: Description: OSS has single parser for SQL statements - SparkSqlParser. However

[jira] [Updated] (SPARK-48579) [M1] Merge DatabricksSqlParser with SparkSqlParser

2024-06-10 Thread David Milicevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Milicevic updated SPARK-48579: Description: OSS has single parser for SQL statements - SparkSqlParser. However

[jira] [Created] (SPARK-48579) [M1] Merge DatabricksSqlParser with SparkSqlParser

2024-06-10 Thread David Milicevic (Jira)
David Milicevic created SPARK-48579: --- Summary: [M1] Merge DatabricksSqlParser with SparkSqlParser Key: SPARK-48579 URL: https://issues.apache.org/jira/browse/SPARK-48579 Project: Spark

[jira] [Commented] (SPARK-48311) Nested pythonUDF in groupBy and aggregate result in Binding Exception

2024-06-10 Thread Sumit Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853582#comment-17853582 ] Sumit Singh commented on SPARK-48311: - I have created the PR based on details explained in details

[jira] [Updated] (SPARK-48311) Nested pythonUDF in groupBy and aggregate result in Binding Exception

2024-06-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48311: --- Labels: pull-request-available (was: ) > Nested pythonUDF in groupBy and aggregate res

[jira] [Updated] (SPARK-48576) Rename UTF8_BINARY_LCASE to UTF8_LCASE

2024-06-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48576: --- Labels: pull-request-available (was: ) > Rename UTF8_BINARY_LCASE to UTF8_LC

[jira] [Updated] (SPARK-48280) Improve collation testing surface area using expression walking

2024-06-10 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-48280: -- Summary: Improve collation testing surface area using expression walking (was: Add

[jira] [Created] (SPARK-48578) Add new expressions for UTF8 string validation

2024-06-09 Thread Jira
Uroš Bojanić created SPARK-48578: Summary: Add new expressions for UTF8 string validation Key: SPARK-48578 URL: https://issues.apache.org/jira/browse/SPARK-48578 Project: Spark Issue Type

[jira] [Created] (SPARK-48577) Replace invalid byte sequences in UTF8Strings

2024-06-09 Thread Jira
Uroš Bojanić created SPARK-48577: Summary: Replace invalid byte sequences in UTF8Strings Key: SPARK-48577 URL: https://issues.apache.org/jira/browse/SPARK-48577 Project: Spark Issue Type

[jira] [Created] (SPARK-48576) Rename UTF8_BINARY_LCASE to UTF8_LCASE

2024-06-09 Thread Jira
Uroš Bojanić created SPARK-48576: Summary: Rename UTF8_BINARY_LCASE to UTF8_LCASE Key: SPARK-48576 URL: https://issues.apache.org/jira/browse/SPARK-48576 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-48560) Make StreamingQueryListener.spark settable

2024-06-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48560. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46909 [https

[jira] [Updated] (SPARK-48575) spark.history.fs.update.interval calling too many directory pollings when spark log dir contains many sparkEvent apps

2024-06-09 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-48575: Attachment: EventLogFileReaders.patch > spark.history.fs.update.interval calling too m

[jira] [Updated] (SPARK-48575) spark.history.fs.update.interval calling too many directory pollings when spark log dir contains many sparkEvent apps

2024-06-09 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-48575: Attachment: (was: EventLogFileReaders.patch) > spark.history.fs.update.inter

[jira] [Assigned] (SPARK-48012) SPJ: Support Transfrom Expressions for One Side Shuffle

2024-06-09 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-48012: Assignee: Szehon Ho > SPJ: Support Transfrom Expressions for One Side Shuf

[jira] [Resolved] (SPARK-48012) SPJ: Support Transfrom Expressions for One Side Shuffle

2024-06-09 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-48012. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46255 [https

[jira] [Commented] (SPARK-48463) MLLib function unable to handle nested data

2024-06-08 Thread someshwar kale (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853459#comment-17853459 ] someshwar kale commented on SPARK-48463: Currently there is no way to handle backtick(`) spark

[jira] [Updated] (SPARK-48575) spark.history.fs.update.interval calling too many directory pollings when spark log dir contains many sparkEvent apps

2024-06-08 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-48575: Attachment: EventLogFileReaders.patch > spark.history.fs.update.interval calling too m

[jira] [Updated] (SPARK-48575) spark.history.fs.update.interval calling too many directory pollings when spark log dir contains many sparkEvent apps

2024-06-08 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-48575: Description: In the case of a sparkLog dir containing "lot" of spark eventLog

[jira] [Updated] (SPARK-48575) spark.history.fs.update.interval calling too many directory pollings when spark log dir contains many sparkEvent apps

2024-06-08 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-48575: Description: In the case of a sparkLog dir containing "lot" of spark eventLog

[jira] [Created] (SPARK-48575) spark.history.fs.update.interval calling too many directory pollings when spark log dir contains many sparkEvent apps

2024-06-08 Thread Arnaud Nauwynck (Jira)
://issues.apache.org/jira/browse/SPARK-48575 Project: Spark Issue Type: Improvement Components: Spark Core Affects Versions: 3.4.3, 3.5.1, 3.3.3, 4.0.0 Reporter: Arnaud Nauwynck In the case of a sparkLog dir containing "lot" of spark eventLogs sub-dirs

[jira] [Commented] (SPARK-39247) Support returning a table or set of rows in CREATE FUNCTION

2024-06-08 Thread Jonathan Boarman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853410#comment-17853410 ] Jonathan Boarman commented on SPARK-39247: -- Being able to create a *UDTF* (user-defined table

[jira] [Commented] (SPARK-45201) NoClassDefFoundError: InternalFutureFailureAccess when compiling Spark 3.5.0

2024-06-08 Thread Sebastian Daberdaku (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853387#comment-17853387 ] Sebastian Daberdaku commented on SPARK-45201: - For version 3.5.1 this patch can be used

[jira] [Updated] (SPARK-45201) NoClassDefFoundError: InternalFutureFailureAccess when compiling Spark 3.5.0

2024-06-08 Thread Sebastian Daberdaku (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastian Daberdaku updated SPARK-45201: Attachment: spark-3.5.1.patch > NoClassDefFoundError: InternalFutureFailureAcc

[jira] [Updated] (SPARK-45201) NoClassDefFoundError: InternalFutureFailureAccess when compiling Spark 3.5.0

2024-06-08 Thread Sebastian Daberdaku (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastian Daberdaku updated SPARK-45201: Affects Version/s: 3.5.1 > NoClassDefFoundError: InternalFutureFailureAccess w

[jira] [Updated] (SPARK-48574) Add support for StructTypes with collations

2024-06-08 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-48574: -- Description: While adding expression walker it was noticed that StructType support

[jira] [Created] (SPARK-48574) Add support for StructTypes with collations

2024-06-08 Thread Mihailo Milosevic (Jira)
Mihailo Milosevic created SPARK-48574: - Summary: Add support for StructTypes with collations Key: SPARK-48574 URL: https://issues.apache.org/jira/browse/SPARK-48574 Project: Spark Issue

[jira] [Updated] (SPARK-48572) Fix DateSub, DateAdd, WindowTime, TimeWindow and SessionWindow expressions

2024-06-08 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-48572: -- Description: While adding Expression Walker testing, these expression were found

[jira] [Updated] (SPARK-48572) Fix DateSub, DateAdd, WindowTime, TimeWindow and SessionWindow expressions

2024-06-08 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-48572: -- Summary: Fix DateSub, DateAdd, WindowTime, TimeWindow and SessionWindow expressions

[jira] [Created] (SPARK-48572) Fix DateSub and DateAdd expression implicit casting

2024-06-08 Thread Mihailo Milosevic (Jira)
Mihailo Milosevic created SPARK-48572: - Summary: Fix DateSub and DateAdd expression implicit casting Key: SPARK-48572 URL: https://issues.apache.org/jira/browse/SPARK-48572 Project: Spark

[jira] [Updated] (SPARK-48571) Reduce the number of accesses to S3 object storage

2024-06-08 Thread Oliver Caballero Alvarez (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Caballero Alvarez updated SPARK-48571: - Description: If we access a Spark table on an object storage file

[jira] [Updated] (SPARK-48571) Reduce the number of accesses to S3 object storage

2024-06-08 Thread Oliver Caballero Alvarez (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Caballero Alvarez updated SPARK-48571: - Description: If we access a Spark table on an object storage file

[jira] [Created] (SPARK-48571) Reduce the number of accesses to S3 object storage

2024-06-08 Thread Oliver Caballero Alvarez (Jira)
Oliver Caballero Alvarez created SPARK-48571: Summary: Reduce the number of accesses to S3 object storage Key: SPARK-48571 URL: https://issues.apache.org/jira/browse/SPARK-48571 Project

[jira] [Updated] (SPARK-48571) Reduce the number of accesses to S3 object storage

2024-06-08 Thread Oliver Caballero Alvarez (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Caballero Alvarez updated SPARK-48571: - Attachment: Spark 3.5 Hadoop-aws 3.1.PNG Spark 3.2

[jira] [Commented] (SPARK-48463) MLLib function unable to handle nested data

2024-06-08 Thread someshwar kale (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853363#comment-17853363 ] someshwar kale commented on SPARK-48463: I don't think renaming 200+ (simple metadata

[jira] [Updated] (SPARK-48530) [M0] Support for local variables

2024-06-08 Thread David Milicevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Milicevic updated SPARK-48530: Description: At the moment, variables in SQL scripts are creating session variables. We

[jira] [Commented] (SPARK-48463) MLLib function unable to handle nested data

2024-06-08 Thread Chhavi Bansal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853332#comment-17853332 ] Chhavi Bansal commented on SPARK-48463: --- hi [~skale1990]  Thanks for responding. Yes, this seems

[jira] [Updated] (SPARK-48567) Pyspark StreamingQuery lastProgress and friend should return actual StreamingQueryProgress

2024-06-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48567: --- Labels: pull-request-available (was: ) > Pyspark StreamingQuery lastProgress and fri

[jira] [Updated] (SPARK-45373) Minimizing calls to HiveMetaStore layer for getting partitions, when tables are repeated

2024-06-07 Thread Asif (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-45373: - Shepherd: Peter Toth (was: Wenchen Fan) > Minimizing calls to HiveMetaStore layer for getting partiti

[jira] [Created] (SPARK-48568) Improve Performance of CodeFormatter with Java StringBuilder

2024-06-07 Thread Ahir Reddy (Jira)
Ahir Reddy created SPARK-48568: -- Summary: Improve Performance of CodeFormatter with Java StringBuilder Key: SPARK-48568 URL: https://issues.apache.org/jira/browse/SPARK-48568 Project: Spark

[jira] [Created] (SPARK-48567) Pyspark StreamingQuery lastProgress and friend should return actual StreamingQueryProgress

2024-06-07 Thread Wei Liu (Jira)
Wei Liu created SPARK-48567: --- Summary: Pyspark StreamingQuery lastProgress and friend should return actual StreamingQueryProgress Key: SPARK-48567 URL: https://issues.apache.org/jira/browse/SPARK-48567

[jira] [Updated] (SPARK-48566) [Bug] Partition indices are incorrect when UDTF analyze() uses both select and partitionColumns

2024-06-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48566: --- Labels: pull-request-available (was: ) > [Bug] Partition indices are incorrect when U

[jira] [Created] (SPARK-48566) [Bug] Partition indices are incorrect when UDTF analyze() uses both select and partitionColumns

2024-06-07 Thread Daniel (Jira)
Daniel created SPARK-48566: -- Summary: [Bug] Partition indices are incorrect when UDTF analyze() uses both select and partitionColumns Key: SPARK-48566 URL: https://issues.apache.org/jira/browse/SPARK-48566

[jira] [Updated] (SPARK-48302) Preserve nulls in map columns in PyArrow Tables

2024-06-07 Thread Ian Cook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ian Cook updated SPARK-48302: - Target Version/s: 4.0.0 > Preserve nulls in map columns in PyArrow Tab

[jira] [Updated] (SPARK-48302) Preserve nulls in map columns in PyArrow Tables

2024-06-07 Thread Ian Cook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ian Cook updated SPARK-48302: - Parent: SPARK-44111 Issue Type: Sub-task (was: Bug) > Preserve nulls in map columns in PyAr

[jira] [Updated] (SPARK-48302) Preserve nulls in map columns in PyArrow Tables

2024-06-07 Thread Ian Cook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ian Cook updated SPARK-48302: - Affects Version/s: 3.5.1 > Preserve nulls in map columns in PyArrow Tab

[jira] [Commented] (SPARK-32385) Publish a "bill of materials" (BOM) descriptor for Spark with correct versions of various dependencies

2024-06-07 Thread James Grinter (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853172#comment-17853172 ] James Grinter commented on SPARK-32385: --- Regarding the convention of projects publishing boms

[jira] [Updated] (SPARK-48565) Fix thread dump display in UI

2024-06-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48565: --- Labels: pull-request-available (was: ) > Fix thread dump display in

[jira] [Created] (SPARK-48565) Fix thread dump display in UI

2024-06-07 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-48565: - Summary: Fix thread dump display in UI Key: SPARK-48565 URL: https://issues.apache.org/jira/browse/SPARK-48565 Project: Spark Issue Type: Bug Components

[jira] [Updated] (SPARK-48564) Propagate cached schema in set operations

2024-06-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48564: --- Labels: pull-request-available (was: ) > Propagate cached schema in set operati

[jira] [Created] (SPARK-48564) Propagate cached schema in set operations

2024-06-07 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-48564: - Summary: Propagate cached schema in set operations Key: SPARK-48564 URL: https://issues.apache.org/jira/browse/SPARK-48564 Project: Spark Issue Type: Sub

[jira] [Updated] (SPARK-48563) Upgrade pickle to 1.5

2024-06-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48563: --- Labels: pull-request-available (was: ) > Upgrade pickle to

[jira] [Created] (SPARK-48563) Upgrade pickle to 1.5

2024-06-07 Thread Yang Jie (Jira)
Yang Jie created SPARK-48563: Summary: Upgrade pickle to 1.5 Key: SPARK-48563 URL: https://issues.apache.org/jira/browse/SPARK-48563 Project: Spark Issue Type: Improvement Components

[jira] [Assigned] (SPARK-48559) Fetch globalTempDatabase name directly without invoking initialization of GlobalaTempViewManager

2024-06-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48559: --- Assignee: Wenchen Fan > Fetch globalTempDatabase name directly without invok

[jira] [Resolved] (SPARK-48559) Fetch globalTempDatabase name directly without invoking initialization of GlobalaTempViewManager

2024-06-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48559. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46907 [https

[jira] [Resolved] (SPARK-48561) Throw PandasNotImplementedError for unsupported plotting functions

2024-06-07 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48561. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46911 [https

[jira] [Updated] (SPARK-48472) Enable reflect expressions with collated strings

2024-06-07 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-48472: -- Description: As a movement to collated world, we need to make sure all expressions

[jira] [Updated] (SPARK-48472) Enable reflect expressions with collated strings

2024-06-07 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-48472: -- Summary: Enable reflect expressions with collated strings (was: Expression Walker

[jira] [Updated] (SPARK-48472) Expression Walker Test

2024-06-07 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-48472: -- Parent: SPARK-46837 Issue Type: Sub-task (was: Improvement) > Express

[jira] [Updated] (SPARK-48472) Expression Walker Test

2024-06-07 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-48472: -- Epic Link: (was: SPARK-46830) > Expression Walker T

[jira] [Updated] (SPARK-48562) Writing to JDBC Temporary View Failed

2024-06-07 Thread Junqing Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junqing Li updated SPARK-48562: --- Description: When creating a JDBC temporary view, `ApplyCharTypePadding` would add a Project

[jira] [Created] (SPARK-48562) Writing to JDBC Temporary View Failed

2024-06-07 Thread Junqing Li (Jira)
Junqing Li created SPARK-48562: -- Summary: Writing to JDBC Temporary View Failed Key: SPARK-48562 URL: https://issues.apache.org/jira/browse/SPARK-48562 Project: Spark Issue Type: Bug

<    1   2   3   4   5   6   7   8   9   10   >