[jira] [Created] (SPARK-36030) Support DS v2 metrics at writing path

L. C. Hsieh created SPARK-36030: --- Summary: Support DS v2 metrics at writing path Key: SPARK-36030 URL: https://issues.apache.org/jira/browse/SPARK-36030 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-35972) When replace ExtractValue in NestedColumnAliasing we should use semanticEquals

[ https://issues.apache.org/jira/browse/SPARK-35972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35972: Summary: When replace ExtractValue in NestedColumnAliasing we should use semanticEquals (was:

[jira] [Commented] (SPARK-35972) NestColumnPruning cause execute loss output

[ https://issues.apache.org/jira/browse/SPARK-35972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17375304#comment-17375304 ] L. C. Hsieh commented on SPARK-35972: - Fixed at https://github.com/apache/spark/pull/33183. >

[jira] [Resolved] (SPARK-35972) NestColumnPruning cause execute loss output

[ https://issues.apache.org/jira/browse/SPARK-35972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35972. - Resolution: Fixed > NestColumnPruning cause execute loss output >

[jira] [Updated] (SPARK-35972) NestColumnPruning cause execute loss output

[ https://issues.apache.org/jira/browse/SPARK-35972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35972: Affects Version/s: 3.2.0 > NestColumnPruning cause execute loss output >

[jira] [Updated] (SPARK-35972) NestColumnPruning cause execute loss output

[ https://issues.apache.org/jira/browse/SPARK-35972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35972: Issue Type: Bug (was: Improvement) > NestColumnPruning cause execute loss output >

[jira] [Updated] (SPARK-35972) NestColumnPruning cause execute loss output

[ https://issues.apache.org/jira/browse/SPARK-35972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35972: Fix Version/s: 3.2.0 > NestColumnPruning cause execute loss output >

[jira] [Commented] (SPARK-35972) NestColumnPruning cause execute loss output

[ https://issues.apache.org/jira/browse/SPARK-35972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17374976#comment-17374976 ] L. C. Hsieh commented on SPARK-35972: - >From the description, looks like a bug? > NestColumnPruning

[jira] [Resolved] (SPARK-35940) Refactor EquivalentExpressions to make it more efficient

[ https://issues.apache.org/jira/browse/SPARK-35940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35940. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33142

[jira] [Assigned] (SPARK-35940) Refactor EquivalentExpressions to make it more efficient

[ https://issues.apache.org/jira/browse/SPARK-35940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35940: --- Assignee: Wenchen Fan > Refactor EquivalentExpressions to make it more efficient >

[jira] [Assigned] (SPARK-35785) Cleanup support for RocksDB instance

[ https://issues.apache.org/jira/browse/SPARK-35785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35785: --- Assignee: Yuanjian Li (was: Apache Spark) > Cleanup support for RocksDB instance >

[jira] [Assigned] (SPARK-35785) Cleanup support for RocksDB instance

[ https://issues.apache.org/jira/browse/SPARK-35785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35785: --- Assignee: Apache Spark > Cleanup support for RocksDB instance >

[jira] [Resolved] (SPARK-35785) Cleanup support for RocksDB instance

[ https://issues.apache.org/jira/browse/SPARK-35785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35785. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 32933

[jira] [Assigned] (SPARK-35779) Support dynamic filtering for v2 tables

[ https://issues.apache.org/jira/browse/SPARK-35779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35779: --- Assignee: Anton Okolnychyi > Support dynamic filtering for v2 tables >

[jira] [Resolved] (SPARK-35779) Support dynamic filtering for v2 tables

[ https://issues.apache.org/jira/browse/SPARK-35779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35779. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32921

[jira] [Resolved] (SPARK-35829) Clean up evaluates subexpressions and add more flexibility to evaluate particular subexpressoin

[ https://issues.apache.org/jira/browse/SPARK-35829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35829. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32980

[jira] [Resolved] (SPARK-35784) Implementation for RocksDB instance

[ https://issues.apache.org/jira/browse/SPARK-35784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35784. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32928

[jira] [Assigned] (SPARK-35784) Implementation for RocksDB instance

[ https://issues.apache.org/jira/browse/SPARK-35784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35784: --- Assignee: Yuanjian Li > Implementation for RocksDB instance >

[jira] [Updated] (SPARK-35886) Codegen issue for decimal type

[ https://issues.apache.org/jira/browse/SPARK-35886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35886: Affects Version/s: 3.0.3 3.1.2 > Codegen issue for decimal type >

[jira] [Resolved] (SPARK-35884) EXPLAIN FORMATTED for AQE

[ https://issues.apache.org/jira/browse/SPARK-35884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35884. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33067

[jira] [Assigned] (SPARK-35884) EXPLAIN FORMATTED for AQE

[ https://issues.apache.org/jira/browse/SPARK-35884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35884: --- Assignee: Wenchen Fan > EXPLAIN FORMATTED for AQE > - > >

[jira] [Assigned] (SPARK-35290) unionByName with null filling fails for some nested structs

[ https://issues.apache.org/jira/browse/SPARK-35290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35290: --- Assignee: Adam Binford > unionByName with null filling fails for some nested structs >

[jira] [Resolved] (SPARK-35290) unionByName with null filling fails for some nested structs

[ https://issues.apache.org/jira/browse/SPARK-35290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35290. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33040

[jira] [Resolved] (SPARK-34889) Introduce MergingSessionsIterator merging elements directly which belong to the same session

[ https://issues.apache.org/jira/browse/SPARK-34889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-34889. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31987

[jira] [Assigned] (SPARK-34889) Introduce MergingSessionsIterator merging elements directly which belong to the same session

[ https://issues.apache.org/jira/browse/SPARK-34889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-34889: --- Assignee: Jungtaek Lim > Introduce MergingSessionsIterator merging elements directly which

[jira] [Commented] (SPARK-35542) Bucketizer created for multiple columns with parameters splitsArray,  inputCols and outputCols can not be loaded after saving it.

[ https://issues.apache.org/jira/browse/SPARK-35542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367829#comment-17367829 ] L. C. Hsieh commented on SPARK-35542: - So this is PySpark only issue and Scala Bucketizer is fine?

[jira] [Resolved] (SPARK-35611) Introduce the strategy on mismatched offset for start offset timestamp on Kafka data source

[ https://issues.apache.org/jira/browse/SPARK-35611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35611. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32747

[jira] [Assigned] (SPARK-35611) Introduce the strategy on mismatched offset for start offset timestamp on Kafka data source

[ https://issues.apache.org/jira/browse/SPARK-35611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35611: --- Assignee: Jungtaek Lim > Introduce the strategy on mismatched offset for start offset

[jira] [Created] (SPARK-35829) Clean up evaluates subexpressions and add more flexibility to evaluate particular subexpressoin

L. C. Hsieh created SPARK-35829: --- Summary: Clean up evaluates subexpressions and add more flexibility to evaluate particular subexpressoin Key: SPARK-35829 URL: https://issues.apache.org/jira/browse/SPARK-35829

[jira] [Resolved] (SPARK-35448) Subexpression elimination enhancements

[ https://issues.apache.org/jira/browse/SPARK-35448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35448. - Resolution: Fixed > Subexpression elimination enhancements >

[jira] [Commented] (SPARK-35752) Clean up unused code in getLocalInputVariableValues

[ https://issues.apache.org/jira/browse/SPARK-35752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17363398#comment-17363398 ] L. C. Hsieh commented on SPARK-35752: - Note that it is because if there was non-split subexpressions

[jira] [Commented] (SPARK-35752) Clean up unused code in getLocalInputVariableValues

[ https://issues.apache.org/jira/browse/SPARK-35752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362805#comment-17362805 ] L. C. Hsieh commented on SPARK-35752: - Found exceptional case. Seems invalid. > Clean up unused

[jira] [Resolved] (SPARK-35752) Clean up unused code in getLocalInputVariableValues

[ https://issues.apache.org/jira/browse/SPARK-35752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35752. - Resolution: Invalid > Clean up unused code in getLocalInputVariableValues >

[jira] [Created] (SPARK-35752) Clean up unused code in getLocalInputVariableValues

L. C. Hsieh created SPARK-35752: --- Summary: Clean up unused code in getLocalInputVariableValues Key: SPARK-35752 URL: https://issues.apache.org/jira/browse/SPARK-35752 Project: Spark Issue

[jira] [Resolved] (SPARK-35701) Contention on SQLConf.sqlConfEntries and SQLConf.staticConfKeys

[ https://issues.apache.org/jira/browse/SPARK-35701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35701. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32865

[jira] [Resolved] (SPARK-35653) [SQL] CatalystToExternalMap interpreted path fails for Map with case classes as keys or values

[ https://issues.apache.org/jira/browse/SPARK-35653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35653. - Fix Version/s: 3.0.3 3.1.3 3.2.0 Resolution: Fixed

[jira] [Assigned] (SPARK-35653) [SQL] CatalystToExternalMap interpreted path fails for Map with case classes as keys or values

[ https://issues.apache.org/jira/browse/SPARK-35653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35653: --- Assignee: Emil Ejbyfeldt > [SQL] CatalystToExternalMap interpreted path fails for Map with

[jira] [Created] (SPARK-35689) Add logging for null value retrieval for SymmetricHashJoinStateManager

L. C. Hsieh created SPARK-35689: --- Summary: Add logging for null value retrieval for SymmetricHashJoinStateManager Key: SPARK-35689 URL: https://issues.apache.org/jira/browse/SPARK-35689 Project: Spark

[jira] [Commented] (SPARK-35659) Avoid write null to StateStore

[ https://issues.apache.org/jira/browse/SPARK-35659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359470#comment-17359470 ] L. C. Hsieh commented on SPARK-35659: - The issue was resolved at

[jira] [Resolved] (SPARK-35659) Avoid write null to StateStore

[ https://issues.apache.org/jira/browse/SPARK-35659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35659. - Resolution: Fixed > Avoid write null to StateStore > -- > >

[jira] [Updated] (SPARK-35659) Avoid write null to StateStore

[ https://issues.apache.org/jira/browse/SPARK-35659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35659: Fix Version/s: 3.1.3 3.2.0 > Avoid write null to StateStore >

[jira] [Commented] (SPARK-35564) Support subexpression elimination for non-common branches of conditional expressions

[ https://issues.apache.org/jira/browse/SPARK-35564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17358285#comment-17358285 ] L. C. Hsieh commented on SPARK-35564: - If you mean a common expr in tail conditions other than the

[jira] [Assigned] (SPARK-35499) Apply black to pandas API on Spark codes.

[ https://issues.apache.org/jira/browse/SPARK-35499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35499: --- Assignee: Haejoon Lee > Apply black to pandas API on Spark codes. >

[jira] [Resolved] (SPARK-35499) Apply black to pandas API on Spark codes.

[ https://issues.apache.org/jira/browse/SPARK-35499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35499. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32779

[jira] [Updated] (SPARK-35659) Avoid write null to StateStore

[ https://issues.apache.org/jira/browse/SPARK-35659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35659: Description: According to {{get}} method doc in StateStore API, it returns non-null row if the

[jira] [Created] (SPARK-35659) Avoid write null to StateStore

L. C. Hsieh created SPARK-35659: --- Summary: Avoid write null to StateStore Key: SPARK-35659 URL: https://issues.apache.org/jira/browse/SPARK-35659 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-35564) Support subexpression elimination for non-common branches of conditional expressions

[ https://issues.apache.org/jira/browse/SPARK-35564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17358215#comment-17358215 ] L. C. Hsieh commented on SPARK-35564: - Do you mean {{CaseWhen(($"id", myUdf($"id") :: ($"id" + 1,

[jira] [Commented] (SPARK-35564) Support subexpression elimination for non-common branches of conditional expressions

[ https://issues.apache.org/jira/browse/SPARK-35564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17358209#comment-17358209 ] L. C. Hsieh commented on SPARK-35564: - For the case {{spark.range(2).select(coalesce($"id",

[jira] [Comment Edited] (SPARK-35564) Support subexpression elimination for non-common branches of conditional expressions

[ https://issues.apache.org/jira/browse/SPARK-35564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17358209#comment-17358209 ] L. C. Hsieh edited comment on SPARK-35564 at 6/6/21, 7:49 PM: -- For the case

[jira] [Commented] (SPARK-35564) Support subexpression elimination for non-common branches of conditional expressions

[ https://issues.apache.org/jira/browse/SPARK-35564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17358170#comment-17358170 ] L. C. Hsieh commented on SPARK-35564: - {{select(myUdf($"id"), coalesce($"id", myUdf($"id")))}} =>

[jira] [Created] (SPARK-35637) 2 active tasks shown in Spark UI but executor core is 1 and task cpu is 1

L. C. Hsieh created SPARK-35637: --- Summary: 2 active tasks shown in Spark UI but executor core is 1 and task cpu is 1 Key: SPARK-35637 URL: https://issues.apache.org/jira/browse/SPARK-35637 Project:

[jira] [Resolved] (SPARK-35580) Support subexpression elimination for higher order functions

[ https://issues.apache.org/jira/browse/SPARK-35580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35580. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32735

[jira] [Assigned] (SPARK-35580) Support subexpression elimination for higher order functions

[ https://issues.apache.org/jira/browse/SPARK-35580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35580: --- Assignee: L. C. Hsieh > Support subexpression elimination for higher order functions >

[jira] [Resolved] (SPARK-35560) Remove redundant subexpression evaluation in nested subexpressions

[ https://issues.apache.org/jira/browse/SPARK-35560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35560. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32699

[jira] [Commented] (SPARK-35564) Support subexpression elimination for non-common branches of conditional expressions

[ https://issues.apache.org/jira/browse/SPARK-35564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17354590#comment-17354590 ] L. C. Hsieh commented on SPARK-35564: - > I don't really think this is much of a corner case, but a

[jira] [Comment Edited] (SPARK-35564) Support subexpression elimination for non-common branches of conditional expressions

[ https://issues.apache.org/jira/browse/SPARK-35564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17354206#comment-17354206 ] L. C. Hsieh edited comment on SPARK-35564 at 5/31/21, 4:16 AM: --- Thanks

[jira] [Commented] (SPARK-35564) Support subexpression elimination for non-common branches of conditional expressions

[ https://issues.apache.org/jira/browse/SPARK-35564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17354206#comment-17354206 ] L. C. Hsieh commented on SPARK-35564: - Thanks [~hyukjin.kwon] for the ping. > Create a

[jira] [Updated] (SPARK-35566) Fix number of output rows for StateStoreRestoreExec

[ https://issues.apache.org/jira/browse/SPARK-35566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35566: Priority: Minor (was: Major) > Fix number of output rows for StateStoreRestoreExec >

[jira] [Created] (SPARK-35566) Fix number of output rows for StateStoreRestoreExec

L. C. Hsieh created SPARK-35566: --- Summary: Fix number of output rows for StateStoreRestoreExec Key: SPARK-35566 URL: https://issues.apache.org/jira/browse/SPARK-35566 Project: Spark Issue

[jira] [Created] (SPARK-35565) Add a config for ignoring metadata directory of file stream sink

L. C. Hsieh created SPARK-35565: --- Summary: Add a config for ignoring metadata directory of file stream sink Key: SPARK-35565 URL: https://issues.apache.org/jira/browse/SPARK-35565 Project: Spark

[jira] [Updated] (SPARK-35560) Remove redundant subexpression evaluation in nested subexpressions

[ https://issues.apache.org/jira/browse/SPARK-35560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35560: Parent: SPARK-35448 Issue Type: Sub-task (was: Improvement) > Remove redundant

[jira] [Created] (SPARK-35560) Remove redundant subexpression evaluation in nested subexpressions

L. C. Hsieh created SPARK-35560: --- Summary: Remove redundant subexpression evaluation in nested subexpressions Key: SPARK-35560 URL: https://issues.apache.org/jira/browse/SPARK-35560 Project: Spark

[jira] [Assigned] (SPARK-35541) Simplify OptimizeSkewedJoin

[ https://issues.apache.org/jira/browse/SPARK-35541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35541: --- Assignee: Wenchen Fan > Simplify OptimizeSkewedJoin > --- > >

[jira] [Resolved] (SPARK-35541) Simplify OptimizeSkewedJoin

[ https://issues.apache.org/jira/browse/SPARK-35541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35541. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32685

[jira] [Commented] (SPARK-33121) Spark Streaming 3.1.1 hangs on shutdown

[ https://issues.apache.org/jira/browse/SPARK-33121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17350629#comment-17350629 ] L. C. Hsieh commented on SPARK-33121: - Hmm, I cannot reproduce in branch-3.1/2.4 or master branch.

[jira] [Updated] (SPARK-35449) Should not extract common expressions from value expressions when elseValue is empty in CaseWhen

[ https://issues.apache.org/jira/browse/SPARK-35449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35449: Fix Version/s: 3.1.2 > Should not extract common expressions from value expressions when

[jira] [Commented] (SPARK-35449) Should not extract common expressions from value expressions when elseValue is empty in CaseWhen

[ https://issues.apache.org/jira/browse/SPARK-35449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17350297#comment-17350297 ] L. C. Hsieh commented on SPARK-35449: - This issue was fixed at

[jira] [Resolved] (SPARK-35449) Should not extract common expressions from value expressions when elseValue is empty in CaseWhen

[ https://issues.apache.org/jira/browse/SPARK-35449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35449. - Fix Version/s: 3.2.0 Resolution: Fixed > Should not extract common expressions from

[jira] [Assigned] (SPARK-35449) Should not extract common expressions from value expressions when elseValue is empty in CaseWhen

[ https://issues.apache.org/jira/browse/SPARK-35449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35449: --- Assignee: Adam Binford > Should not extract common expressions from value expressions when

[jira] [Updated] (SPARK-35449) Should not extract common expressions from value expressions when elseValue is empty in CaseWhen

[ https://issues.apache.org/jira/browse/SPARK-35449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35449: Affects Version/s: 3.1.1 > Should not extract common expressions from value expressions when

[jira] [Assigned] (SPARK-35381) Fix lambda variable name issues in nested DataFrame functions in R APIs

[ https://issues.apache.org/jira/browse/SPARK-35381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35381: --- Assignee: Hyukjin Kwon > Fix lambda variable name issues in nested DataFrame functions in

[jira] [Updated] (SPARK-35320) from_json cannot parse maps with timestamp as key

[ https://issues.apache.org/jira/browse/SPARK-35320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35320: Issue Type: Improvement (was: Bug) > from_json cannot parse maps with timestamp as key >

[jira] [Commented] (SPARK-35320) from_json cannot parse maps with timestamp as key

[ https://issues.apache.org/jira/browse/SPARK-35320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17350193#comment-17350193 ] L. C. Hsieh commented on SPARK-35320: - I think `from_json` documents that for MapType, StringType is

[jira] [Resolved] (SPARK-35439) Children subexpr should come first than parent subexpr in subexpression elimination

[ https://issues.apache.org/jira/browse/SPARK-35439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35439. - Fix Version/s: 3.2.0 Assignee: L. C. Hsieh (was: Apache Spark) Resolution:

[jira] [Created] (SPARK-35449) Should not extract common expressions from value expressions when elseValue is empty in CaseWhen

L. C. Hsieh created SPARK-35449: --- Summary: Should not extract common expressions from value expressions when elseValue is empty in CaseWhen Key: SPARK-35449 URL: https://issues.apache.org/jira/browse/SPARK-35449

[jira] [Commented] (SPARK-35449) Should not extract common expressions from value expressions when elseValue is empty in CaseWhen

[ https://issues.apache.org/jira/browse/SPARK-35449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17347842#comment-17347842 ] L. C. Hsieh commented on SPARK-35449: - cc [~Kimahriman] > Should not extract common expressions

[jira] [Updated] (SPARK-35410) Unused subexpressions leftover in WholeStageCodegen subexpression elimination

[ https://issues.apache.org/jira/browse/SPARK-35410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35410: Parent: SPARK-35448 Issue Type: Sub-task (was: Bug) > Unused subexpressions leftover in

[jira] [Updated] (SPARK-35439) Children subexpr should come first than parent subexpr in subexpression elimination

[ https://issues.apache.org/jira/browse/SPARK-35439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35439: Parent: SPARK-35448 Issue Type: Sub-task (was: Improvement) > Children subexpr should

[jira] [Created] (SPARK-35448) Subexpression elimination enhancements

L. C. Hsieh created SPARK-35448: --- Summary: Subexpression elimination enhancements Key: SPARK-35448 URL: https://issues.apache.org/jira/browse/SPARK-35448 Project: Spark Issue Type: Umbrella

[jira] [Updated] (SPARK-35439) Children subexpr should come first than parent subexpr in subexpression elimination

[ https://issues.apache.org/jira/browse/SPARK-35439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35439: Description: EquivalentExpressions maintains a map of equivalent expressions. It is HashMap now

[jira] [Updated] (SPARK-35439) Children subexpr should come first than parent subexpr in subexpression elimination

[ https://issues.apache.org/jira/browse/SPARK-35439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35439: Priority: Major (was: Minor) > Children subexpr should come first than parent subexpr in

[jira] [Updated] (SPARK-35439) Children subexpr should come first than parent subexpr in subexpression elimination

[ https://issues.apache.org/jira/browse/SPARK-35439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35439: Affects Version/s: (was: 3.1.1) (was: 3.0.2) > Children subexpr

[jira] [Updated] (SPARK-35439) Children subexpr should come first than parent subexpr in subexpression elimination

[ https://issues.apache.org/jira/browse/SPARK-35439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35439: Summary: Children subexpr should come first than parent subexpr in subexpression elimination

[jira] [Created] (SPARK-35439) Use LinkedHashMap as the map of equivalent expressions to preserve insertion order

L. C. Hsieh created SPARK-35439: --- Summary: Use LinkedHashMap as the map of equivalent expressions to preserve insertion order Key: SPARK-35439 URL: https://issues.apache.org/jira/browse/SPARK-35439

[jira] [Updated] (SPARK-34135) hello world

[ https://issues.apache.org/jira/browse/SPARK-34135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-34135: Fix Version/s: (was: 2.4.8) > hello world > --- > > Key: SPARK-34135

[jira] [Commented] (SPARK-35410) Unused subexpressions leftover in WholeStageCodegen subexpression elimination

[ https://issues.apache.org/jira/browse/SPARK-35410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345055#comment-17345055 ] L. C. Hsieh commented on SPARK-35410: - Thanks for reporting. I will look into this. > Unused

[jira] [Updated] (SPARK-34750) Parquet with invalid chars on column name reads double as null when a clean schema is applied

[ https://issues.apache.org/jira/browse/SPARK-34750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-34750: Fix Version/s: (was: 2.4.8) > Parquet with invalid chars on column name reads double as null

[jira] [Resolved] (SPARK-35329) Split generated switch code into pieces in ExpandExec

[ https://issues.apache.org/jira/browse/SPARK-35329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35329. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32457

[jira] [Assigned] (SPARK-35329) Split generated switch code into pieces in ExpandExec

[ https://issues.apache.org/jira/browse/SPARK-35329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35329: --- Assignee: Takeshi Yamamuro > Split generated switch code into pieces in ExpandExec >

[jira] [Created] (SPARK-35397) Replace sys.err usage with explicit exception type

L. C. Hsieh created SPARK-35397: --- Summary: Replace sys.err usage with explicit exception type Key: SPARK-35397 URL: https://issues.apache.org/jira/browse/SPARK-35397 Project: Spark Issue Type:

[jira] [Commented] (SPARK-35356) Fix issue of the createTable when externalCatalog is InMemoryCatalog

[ https://issues.apache.org/jira/browse/SPARK-35356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342938#comment-17342938 ] L. C. Hsieh commented on SPARK-35356: - Is the affect version wrong? Do you mean 3.0.2? > Fix issue

[jira] [Updated] (SPARK-35356) Fix issue of the createTable when externalCatalog is InMemoryCatalog

[ https://issues.apache.org/jira/browse/SPARK-35356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35356: Fix Version/s: (was: 3.0.0) > Fix issue of the createTable when externalCatalog is

[jira] [Commented] (SPARK-35371) Scala UDF returning string or complex type applied to array members returns wrong data

[ https://issues.apache.org/jira/browse/SPARK-35371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342937#comment-17342937 ] L. C. Hsieh commented on SPARK-35371: - Oh, I think it was fixed by SPARK-34829. > Scala UDF

[jira] [Commented] (SPARK-35371) Scala UDF returning string or complex type applied to array members returns wrong data

[ https://issues.apache.org/jira/browse/SPARK-35371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342936#comment-17342936 ] L. C. Hsieh commented on SPARK-35371: - I just ran the example in both current master branch, and

[jira] [Commented] (SPARK-34205) Add pipe API to Dataset

[ https://issues.apache.org/jira/browse/SPARK-34205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342788#comment-17342788 ] L. C. Hsieh commented on SPARK-34205: - This is basically driven by our internal customer use-case.

[jira] [Resolved] (SPARK-34205) Add pipe API to Dataset

[ https://issues.apache.org/jira/browse/SPARK-34205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-34205. - Resolution: Won't Fix > Add pipe API to Dataset > --- > >

[jira] [Created] (SPARK-35358) Set maximum Java heap used for release build

L. C. Hsieh created SPARK-35358: --- Summary: Set maximum Java heap used for release build Key: SPARK-35358 URL: https://issues.apache.org/jira/browse/SPARK-35358 Project: Spark Issue Type:

[jira] [Created] (SPARK-35347) Use MethodUtils for method looking up in Invoke and StaticInvoke

L. C. Hsieh created SPARK-35347: --- Summary: Use MethodUtils for method looking up in Invoke and StaticInvoke Key: SPARK-35347 URL: https://issues.apache.org/jira/browse/SPARK-35347 Project: Spark

[jira] [Resolved] (SPARK-35232) Nested column pruning should retain column metadata

[ https://issues.apache.org/jira/browse/SPARK-35232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35232. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32354

[jira] [Assigned] (SPARK-35232) Nested column pruning should retain column metadata

[ https://issues.apache.org/jira/browse/SPARK-35232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35232: --- Assignee: Chao Sun > Nested column pruning should retain column metadata >

<    1   2   3   4   5   6   7   8   9   10   >