[jira] [Resolved] (SPARK-31402) Incorrect rebasing of BCE dates

[ https://issues.apache.org/jira/browse/SPARK-31402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31402. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28172 [https://gith

[jira] [Assigned] (SPARK-31402) Incorrect rebasing of BCE dates

[ https://issues.apache.org/jira/browse/SPARK-31402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31402: --- Assignee: Maxim Gekk > Incorrect rebasing of BCE dates > --- >

[jira] [Updated] (SPARK-31407) Fix hive/SQLQuerySuite.derived from Hive query file: drop_database_removes_partition_dirs.q

[ https://issues.apache.org/jira/browse/SPARK-31407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31407: - Description: Test "derived from Hive query file: drop_database_removes_partition_dirs.q" can fail if we run it s

[jira] [Updated] (SPARK-31407) Fix hive/SQLQuerySuite.derived from Hive query file: drop_database_removes_partition_dirs.q

[ https://issues.apache.org/jira/browse/SPARK-31407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31407: - Description: Test "derived from Hive query file: drop_database_removes_partition_dirs.q" can fail if we run it s

[jira] [Resolved] (SPARK-18886) Delay scheduling should not delay some executors indefinitely if one task is scheduled before delay timeout

[ https://issues.apache.org/jira/browse/SPARK-18886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18886. - Fix Version/s: 3.1.0 Resolution: Fixed > Delay scheduling should not delay some executors

[jira] [Assigned] (SPARK-18886) Delay scheduling should not delay some executors indefinitely if one task is scheduled before delay timeout

[ https://issues.apache.org/jira/browse/SPARK-18886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-18886: --- Assignee: Nicholas Brett Marcott > Delay scheduling should not delay some executors indefin

[jira] [Created] (SPARK-31431) CalendarInterval encoder support

Kent Yao created SPARK-31431: Summary: CalendarInterval encoder support Key: SPARK-31431 URL: https://issues.apache.org/jira/browse/SPARK-31431 Project: Spark Issue Type: Improvement Co

[jira] [Commented] (SPARK-31429) Add additional fields in ExpressionDescription for more granular category in documentation

[ https://issues.apache.org/jira/browse/SPARK-31429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082077#comment-17082077 ] Takeshi Yamamuro commented on SPARK-31429: -- Thanks for filing, Huaxin. I added

[jira] [Updated] (SPARK-31429) Add additional fields in ExpressionDescription for more granular category in documentation

[ https://issues.apache.org/jira/browse/SPARK-31429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-31429: - Description: Add additional fields in ExpressionDescription so we can have more granular

[jira] [Updated] (SPARK-31429) Add additional fields in ExpressionDescription for more granular category in documentation

[ https://issues.apache.org/jira/browse/SPARK-31429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-31429: - Description: Add additional fields in ExpressionDescription so we can have more granular

[jira] [Assigned] (SPARK-31398) Speed up reading dates in ORC

[ https://issues.apache.org/jira/browse/SPARK-31398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31398: --- Assignee: Maxim Gekk > Speed up reading dates in ORC > - > >

[jira] [Resolved] (SPARK-31398) Speed up reading dates in ORC

[ https://issues.apache.org/jira/browse/SPARK-31398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31398. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28169 [https://gith

[jira] [Updated] (SPARK-31430) Bug in the approximate quantile computation.

[ https://issues.apache.org/jira/browse/SPARK-31430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddartha Naidu updated SPARK-31430: Attachment: approx_quantile_data.csv > Bug in the approximate quantile computation. >

[jira] [Commented] (SPARK-31429) Add additional fields in ExpressionDescription for more granular category in documentation

[ https://issues.apache.org/jira/browse/SPARK-31429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082068#comment-17082068 ] Huaxin Gao commented on SPARK-31429: related Jira https://issues.apache.org/jira/bro

[jira] [Comment Edited] (SPARK-31429) Add additional fields in ExpressionDescription for more granular category in documentation

[ https://issues.apache.org/jira/browse/SPARK-31429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082064#comment-17082064 ] Hyukjin Kwon edited comment on SPARK-31429 at 4/13/20, 5:08 AM: --

[jira] [Commented] (SPARK-31429) Add additional fields in ExpressionDescription for more granular category in documentation

[ https://issues.apache.org/jira/browse/SPARK-31429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082064#comment-17082064 ] Hyukjin Kwon commented on SPARK-31429: -- [~huaxingao] can you also related JIRA link

[jira] [Resolved] (SPARK-31403) TreeNode asCode function incorrectly handles null literals

[ https://issues.apache.org/jira/browse/SPARK-31403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31403. -- Resolution: Cannot Reproduce Seems I can't reproduce from the master. Probably fixed somewhere

[jira] [Updated] (SPARK-31403) TreeNode asCode function incorrectly handles null literals

[ https://issues.apache.org/jira/browse/SPARK-31403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31403: - Component/s: (was: Spark Core) SQL > TreeNode asCode function incorrectly h

[jira] [Created] (SPARK-31430) Bug in the approximate quantile computation.

Siddartha Naidu created SPARK-31430: --- Summary: Bug in the approximate quantile computation. Key: SPARK-31430 URL: https://issues.apache.org/jira/browse/SPARK-31430 Project: Spark Issue Type

[jira] [Resolved] (SPARK-31367) add octet_length to functions

[ https://issues.apache.org/jira/browse/SPARK-31367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31367. -- Resolution: Incomplete I am leaving this resolved for now due to no feedback from the reporter

[jira] [Resolved] (SPARK-31373) Cluster tried to fetch blocks from blacklisted node of previous stage

[ https://issues.apache.org/jira/browse/SPARK-31373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31373. -- Resolution: Invalid Let's ask questions into mailing list rather then filing as an issue (see

[jira] [Updated] (SPARK-31374) Returning complex types in Pandas UDF

[ https://issues.apache.org/jira/browse/SPARK-31374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31374: - Target Version/s: (was: 3.0.0) > Returning complex types in Pandas UDF > -

[jira] [Commented] (SPARK-31375) Overwriting into dynamic partitions is appending data in pyspark

[ https://issues.apache.org/jira/browse/SPARK-31375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082057#comment-17082057 ] Hyukjin Kwon commented on SPARK-31375: -- [~Chaitanya Chaganti]can you show a self-co

[jira] [Resolved] (SPARK-31376) Non-global sort support for structured streaming

[ https://issues.apache.org/jira/browse/SPARK-31376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31376. -- Resolution: Invalid > Non-global sort support for structured streaming > -

[jira] [Resolved] (SPARK-31376) Non-global sort support for structured streaming

[ https://issues.apache.org/jira/browse/SPARK-31376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31376. -- Resolution: Incomplete > Non-global sort support for structured streaming > --

[jira] [Reopened] (SPARK-31376) Non-global sort support for structured streaming

[ https://issues.apache.org/jira/browse/SPARK-31376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-31376: -- > Non-global sort support for structured streaming > -

[jira] [Commented] (SPARK-31376) Non-global sort support for structured streaming

[ https://issues.apache.org/jira/browse/SPARK-31376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082055#comment-17082055 ] Hyukjin Kwon commented on SPARK-31376: -- I am resolving it for now - seems definitel

[jira] [Resolved] (SPARK-31386) Reading broadcast in UDF raises MemoryError when spark.executor.pyspark.memory is set

[ https://issues.apache.org/jira/browse/SPARK-31386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31386. -- Resolution: Incomplete Can you reproduce in a plain Spark cluster, not in EMR? possibly an iss

[jira] [Updated] (SPARK-31399) Closure cleaner broken in Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-31399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31399: - Summary: Closure cleaner broken in Scala 2.12 (was: closure cleaner is broken in Scala 2.12) >

[jira] [Issue Comment Deleted] (SPARK-31423) DATES and TIMESTAMPS for a certain range are off by 10 days when stored in ORC

[ https://issues.apache.org/jira/browse/SPARK-31423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-31423: --- Comment: was deleted (was: This is intentional behavior because ORC format assumes the hybrid calen

[jira] [Commented] (SPARK-31423) DATES and TIMESTAMPS for a certain range are off by 10 days when stored in ORC

[ https://issues.apache.org/jira/browse/SPARK-31423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082051#comment-17082051 ] Maxim Gekk commented on SPARK-31423: This is intentional behavior because ORC format

[jira] [Assigned] (SPARK-31383) Clean up the SQL documents in docs/sql-ref*

[ https://issues.apache.org/jira/browse/SPARK-31383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-31383: Assignee: Takeshi Yamamuro > Clean up the SQL documents in docs/sql-ref* > --

[jira] [Commented] (SPARK-31429) Add additional fields in ExpressionDescription for more granular category in documentation

[ https://issues.apache.org/jira/browse/SPARK-31429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082050#comment-17082050 ] Huaxin Gao commented on SPARK-31429: cc [~hyukjin.kwon] [~maropu] > Add additional

[jira] [Resolved] (SPARK-31383) Clean up the SQL documents in docs/sql-ref*

[ https://issues.apache.org/jira/browse/SPARK-31383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31383. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28151 [https://gi

[jira] [Created] (SPARK-31429) Add additional fields in ExpressionDescription for more granular category in documentation

Huaxin Gao created SPARK-31429: -- Summary: Add additional fields in ExpressionDescription for more granular category in documentation Key: SPARK-31429 URL: https://issues.apache.org/jira/browse/SPARK-31429

[jira] [Resolved] (SPARK-31419) Document Table-valued Function and Inline Table

[ https://issues.apache.org/jira/browse/SPARK-31419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31419. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28185 [https://gi

[jira] [Assigned] (SPARK-31419) Document Table-valued Function and Inline Table

[ https://issues.apache.org/jira/browse/SPARK-31419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-31419: Assignee: Huaxin Gao > Document Table-valued Function and Inline Table >

[jira] [Assigned] (SPARK-31319) Document UDF in SQL Reference

[ https://issues.apache.org/jira/browse/SPARK-31319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-31319: Assignee: Huaxin Gao > Document UDF in SQL Reference > - > >

[jira] [Resolved] (SPARK-31319) Document UDF in SQL Reference

[ https://issues.apache.org/jira/browse/SPARK-31319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31319. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28087 [https://gi

[jira] [Resolved] (SPARK-31413) Accessing the sequence number and partition id for records in Kinesis adapter

[ https://issues.apache.org/jira/browse/SPARK-31413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31413. -- Resolution: Invalid > Accessing the sequence number and partition id for records in Kinesis ad

[jira] [Commented] (SPARK-31413) Accessing the sequence number and partition id for records in Kinesis adapter

[ https://issues.apache.org/jira/browse/SPARK-31413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082040#comment-17082040 ] Hyukjin Kwon commented on SPARK-31413: -- Questions should go to the mailing list. Yo

[jira] [Commented] (SPARK-31423) DATES and TIMESTAMPS for a certain range are off by 10 days when stored in ORC

[ https://issues.apache.org/jira/browse/SPARK-31423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082039#comment-17082039 ] Hyukjin Kwon commented on SPARK-31423: -- cc [~maxgekk], [~cloud_fan] FYI. > DATES a

[jira] [Commented] (SPARK-31427) Spark Structure streaming read data twice per every micro-batch.

[ https://issues.apache.org/jira/browse/SPARK-31427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082036#comment-17082036 ] Jungtaek Lim commented on SPARK-31427: -- Could you please check whether using Spark

[jira] [Resolved] (SPARK-29854) lpad and rpad built in function not throw Exception for invalid len value

[ https://issues.apache.org/jira/browse/SPARK-29854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29854. -- Resolution: Cannot Reproduce > lpad and rpad built in function not throw Exception for invalid

[jira] [Resolved] (SPARK-29799) Split a kafka partition into multiple KafkaRDD partitions in the kafka external plugin for Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-29799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29799. -- Resolution: Duplicate > Split a kafka partition into multiple KafkaRDD partitions in the kafka

[jira] [Resolved] (SPARK-31414) Performance regression with new TimestampFormatter for json and csv

[ https://issues.apache.org/jira/browse/SPARK-31414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31414. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28181 [https://gith

[jira] [Assigned] (SPARK-31414) Performance regression with new TimestampFormatter for json and csv

[ https://issues.apache.org/jira/browse/SPARK-31414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31414: --- Assignee: Kent Yao > Performance regression with new TimestampFormatter for json and csv >

[jira] [Resolved] (SPARK-31330) Automatically label PRs based on the paths they touch

[ https://issues.apache.org/jira/browse/SPARK-31330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31330. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28114 [https://gi

[jira] [Assigned] (SPARK-31330) Automatically label PRs based on the paths they touch

[ https://issues.apache.org/jira/browse/SPARK-31330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-31330: Assignee: Nicholas Chammas > Automatically label PRs based on the paths they touch >

[jira] [Updated] (SPARK-31384) NPE in OptimizeSkewedJoin when there's a inputRDD of plan has 0 partition

[ https://issues.apache.org/jira/browse/SPARK-31384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-31384: Parent: SPARK-31412 Issue Type: Sub-task (was: Bug) > NPE in OptimizeSkewedJoin when there's a in

[jira] [Updated] (SPARK-31206) AQE will use the same SubqueryExec even if subqueryReuseEnabled=false

[ https://issues.apache.org/jira/browse/SPARK-31206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-31206: Parent: SPARK-31412 Issue Type: Sub-task (was: Bug) > AQE will use the same SubqueryExec even if

[jira] [Updated] (SPARK-31096) Replace `Array` with `Seq` in AQE `CustomShuffleReaderExec`

[ https://issues.apache.org/jira/browse/SPARK-31096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-31096: Parent: SPARK-31412 Issue Type: Sub-task (was: Bug) > Replace `Array` with `Seq` in AQE `CustomSh

[jira] [Created] (SPARK-31428) Document Common Table Expression in SQL Reference

Huaxin Gao created SPARK-31428: -- Summary: Document Common Table Expression in SQL Reference Key: SPARK-31428 URL: https://issues.apache.org/jira/browse/SPARK-31428 Project: Spark Issue Type: Sub

[jira] [Updated] (SPARK-31046) Make more efficient and clean up AQE update UI code

[ https://issues.apache.org/jira/browse/SPARK-31046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-31046: Parent: SPARK-31412 Issue Type: Sub-task (was: Improvement) > Make more efficient and clean up AQ

[jira] [Updated] (SPARK-31045) Add config for AQE logging level

[ https://issues.apache.org/jira/browse/SPARK-31045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-31045: Parent: SPARK-31412 Issue Type: Sub-task (was: Improvement) > Add config for AQE logging level >

[jira] [Updated] (SPARK-30999) Don't cancel a QueryStageExec when it's already finished

[ https://issues.apache.org/jira/browse/SPARK-30999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30999: Parent: SPARK-31412 Issue Type: Sub-task (was: Improvement) > Don't cancel a QueryStageExec when

[jira] [Updated] (SPARK-30922) Remove the max split config after changing the multi sub joins to multi sub partitions

[ https://issues.apache.org/jira/browse/SPARK-30922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30922: Parent: SPARK-31412 Issue Type: Sub-task (was: Bug) > Remove the max split config after changing

[jira] [Updated] (SPARK-30991) Refactor AQE readers and RDDs

[ https://issues.apache.org/jira/browse/SPARK-30991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30991: Parent: SPARK-31412 Issue Type: Sub-task (was: Bug) > Refactor AQE readers and RDDs > ---

[jira] [Updated] (SPARK-30906) Turning off AQE in CacheManager is not thread-safe

[ https://issues.apache.org/jira/browse/SPARK-30906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30906: Parent: SPARK-31412 Issue Type: Sub-task (was: Bug) > Turning off AQE in CacheManager is not thre

[jira] [Updated] (SPARK-30801) Subqueries should not be AQE-ed if main query is not

[ https://issues.apache.org/jira/browse/SPARK-30801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30801: Parent: SPARK-31412 Issue Type: Sub-task (was: Bug) > Subqueries should not be AQE-ed if main que

[jira] [Updated] (SPARK-30751) Combine the skewed readers into one in AQE skew join optimizations

[ https://issues.apache.org/jira/browse/SPARK-30751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30751: Parent: SPARK-31412 Issue Type: Sub-task (was: Bug) > Combine the skewed readers into one in AQE

[jira] [Updated] (SPARK-30719) AQE should not issue a "not supported" warning for queries being by-passed

[ https://issues.apache.org/jira/browse/SPARK-30719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30719: Parent: SPARK-31412 Issue Type: Sub-task (was: Bug) > AQE should not issue a "not supported" warn

[jira] [Updated] (SPARK-31416) Check more strictly that a field name can be used as a valid Java identifier for codegen

[ https://issues.apache.org/jira/browse/SPARK-31416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31416: -- Affects Version/s: (was: 3.1.0) 3.0.0 > Check more strictly that a

[jira] [Updated] (SPARK-31416) Check more strictly that a field name can be used as a valid Java identifier for codegen

[ https://issues.apache.org/jira/browse/SPARK-31416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31416: -- Parent: SPARK-29194 Issue Type: Sub-task (was: Improvement) > Check more strictly tha

[jira] [Resolved] (SPARK-31416) Check more strictly that a field name can be used as a valid Java identifier for codegen

[ https://issues.apache.org/jira/browse/SPARK-31416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31416. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28184 [https://

[jira] [Assigned] (SPARK-31424) Rename AdaptiveSparkPlanHelper.collectInPlanAndSubqueries to collectWithSubqueries

[ https://issues.apache.org/jira/browse/SPARK-31424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31424: - Assignee: Xiao Li > Rename AdaptiveSparkPlanHelper.collectInPlanAndSubqueries to > col

[jira] [Resolved] (SPARK-31424) Rename AdaptiveSparkPlanHelper.collectInPlanAndSubqueries to collectWithSubqueries

[ https://issues.apache.org/jira/browse/SPARK-31424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31424. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28193 [https://

[jira] [Updated] (SPARK-30571) coalesce shuffle reader with splitting shuffle fetch request fails

[ https://issues.apache.org/jira/browse/SPARK-30571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30571: Parent: SPARK-31412 Issue Type: Sub-task (was: Bug) > coalesce shuffle reader with splitting shuf

[jira] [Updated] (SPARK-30549) Fix the subquery metrics showing issue in UI When enable AQE

[ https://issues.apache.org/jira/browse/SPARK-30549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30549: Parent: SPARK-31412 Issue Type: Sub-task (was: Bug) > Fix the subquery metrics showing issue in U

[jira] [Updated] (SPARK-30403) Fix the NoSuchElementException exception when enable AQE with InSubquery use case

[ https://issues.apache.org/jira/browse/SPARK-30403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30403: Parent: SPARK-31412 Issue Type: Sub-task (was: Bug) > Fix the NoSuchElementException exception wh

[jira] [Updated] (SPARK-30407) reset the metrics info of AdaptiveSparkPlanExec plan when enable aqe

[ https://issues.apache.org/jira/browse/SPARK-30407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30407: Parent: SPARK-31412 Issue Type: Sub-task (was: Bug) > reset the metrics info of AdaptiveSparkPlan

[jira] [Updated] (SPARK-30188) Fix tests when enable Adaptive Query Execution

[ https://issues.apache.org/jira/browse/SPARK-30188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30188: Parent: SPARK-31412 Issue Type: Sub-task (was: Test) > Fix tests when enable Adaptive Query Execu

[jira] [Updated] (SPARK-30307) remove ReusedQueryStageExec

[ https://issues.apache.org/jira/browse/SPARK-30307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30307: Parent: SPARK-31412 Issue Type: Sub-task (was: Improvement) > remove ReusedQueryStageExec > -

[jira] [Updated] (SPARK-30315) Add adaptive execution context

[ https://issues.apache.org/jira/browse/SPARK-30315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30315: Parent: SPARK-31412 Issue Type: Sub-task (was: Task) > Add adaptive execution context > -

[jira] [Resolved] (SPARK-30315) Add adaptive execution context

[ https://issues.apache.org/jira/browse/SPARK-30315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-30315. - Fix Version/s: 3.0.0 Assignee: Wei Xue Resolution: Fixed > Add adaptive execution contex

[jira] [Updated] (SPARK-30291) Catch the exception when do materialize in AQE

[ https://issues.apache.org/jira/browse/SPARK-30291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30291: Parent: SPARK-31412 Issue Type: Sub-task (was: Bug) > Catch the exception when do materialize in

[jira] [Updated] (SPARK-29906) Reading of csv file fails with adaptive execution turned on

[ https://issues.apache.org/jira/browse/SPARK-29906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-29906: Parent: SPARK-31412 Issue Type: Sub-task (was: Bug) > Reading of csv file fails with adaptive exe

[jira] [Updated] (SPARK-29893) Improve the local reader performance by changing the task number from 1 to multi

[ https://issues.apache.org/jira/browse/SPARK-29893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-29893: Parent: SPARK-31412 Issue Type: Sub-task (was: Improvement) > Improve the local reader performanc

[jira] [Created] (SPARK-31427) Spark Structure streaming read data twice per every micro-batch.

Nick Hryhoriev created SPARK-31427: -- Summary: Spark Structure streaming read data twice per every micro-batch. Key: SPARK-31427 URL: https://issues.apache.org/jira/browse/SPARK-31427 Project: Spark

[jira] [Updated] (SPARK-29759) LocalShuffleReaderExec.outputPartitioning should use the corrected attributes

[ https://issues.apache.org/jira/browse/SPARK-29759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-29759: Parent: SPARK-31412 Issue Type: Sub-task (was: Improvement) > LocalShuffleReaderExec.outputPartit

[jira] [Updated] (SPARK-9853) Optimize shuffle fetch of contiguous partition IDs

[ https://issues.apache.org/jira/browse/SPARK-9853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-9853: --- Parent Issue: SPARK-31412 (was: SPARK-9850) > Optimize shuffle fetch of contiguous partition IDs > -

[jira] [Updated] (SPARK-29060) Add tree traversal helper for adaptive spark plans

[ https://issues.apache.org/jira/browse/SPARK-29060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-29060: Parent: SPARK-31412 Issue Type: Sub-task (was: Improvement) > Add tree traversal helper for adapt

[jira] [Updated] (SPARK-29002) Avoid changing SMJ to BHJ if the build side has a high ratio of empty partitions

[ https://issues.apache.org/jira/browse/SPARK-29002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-29002: Parent: SPARK-31412 Issue Type: Sub-task (was: Improvement) > Avoid changing SMJ to BHJ if the bu

[jira] [Assigned] (SPARK-31348) Document Join in SQL Reference

[ https://issues.apache.org/jira/browse/SPARK-31348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-31348: Assignee: Huaxin Gao > Document Join in SQL Reference > -- >

[jira] [Resolved] (SPARK-31348) Document Join in SQL Reference

[ https://issues.apache.org/jira/browse/SPARK-31348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31348. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28121 [https://gi

[jira] [Created] (SPARK-31426) Regression in loading/saving timestamps from/to ORC files

Maxim Gekk created SPARK-31426: -- Summary: Regression in loading/saving timestamps from/to ORC files Key: SPARK-31426 URL: https://issues.apache.org/jira/browse/SPARK-31426 Project: Spark Issue T

[jira] [Created] (SPARK-31425) UnsafeKVExternalSorter should also respect UnsafeAlignedOffset

wuyi created SPARK-31425: Summary: UnsafeKVExternalSorter should also respect UnsafeAlignedOffset Key: SPARK-31425 URL: https://issues.apache.org/jira/browse/SPARK-31425 Project: Spark Issue Type: I

[jira] [Commented] (SPARK-29854) lpad and rpad built in function not throw Exception for invalid len value

[ https://issues.apache.org/jira/browse/SPARK-29854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17081694#comment-17081694 ] Sathyaprakash Govindasamy commented on SPARK-29854: --- In Spark 3.0, you

[jira] [Commented] (SPARK-31420) Infinite timeline redraw in job details page

[ https://issues.apache.org/jira/browse/SPARK-31420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17081690#comment-17081690 ] Kousuke Saruta commented on SPARK-31420: Maybe, all the versions which use vis.j