[jira] [Created] (SPARK-48292) Improve stage failure reason message in OutputCommitCoordinator

L. C. Hsieh created SPARK-48292: --- Summary: Improve stage failure reason message in OutputCommitCoordinator Key: SPARK-48292 URL: https://issues.apache.org/jira/browse/SPARK-48292 Project: Spark

[jira] [Updated] (SPARK-48292) Improve stage failure reason message in OutputCommitCoordinator

[ https://issues.apache.org/jira/browse/SPARK-48292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-48292: Affects Version/s: (was: 3.5.1) (was: 3.4.3) > Improve stage

[jira] [Updated] (SPARK-48292) Improve stage failure reason message in OutputCommitCoordinator

[ https://issues.apache.org/jira/browse/SPARK-48292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-48292: Affects Version/s: 3.5.1 4.0.0 > Improve stage failure reason message in

[jira] [Commented] (SPARK-48073) StateStore schema incompatibility between 3.2 and 3.4

[ https://issues.apache.org/jira/browse/SPARK-48073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843583#comment-17843583 ] L. C. Hsieh commented on SPARK-48073: - The breaking change was introduced by

[jira] [Commented] (SPARK-48073) StateStore schema incompatibility between 3.2 and 3.4

[ https://issues.apache.org/jira/browse/SPARK-48073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842794#comment-17842794 ] L. C. Hsieh commented on SPARK-48073: - Yea, if that is only thing we can do, then I will go to make

[jira] [Commented] (SPARK-48073) StateStore schema incompatibility between 3.2 and 3.4

[ https://issues.apache.org/jira/browse/SPARK-48073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842712#comment-17842712 ] L. C. Hsieh commented on SPARK-48073: - cc [~kabhwan] > StateStore schema incompatibility between

[jira] [Created] (SPARK-48073) StateStore schema incompatibility between 3.2 and 3.4

L. C. Hsieh created SPARK-48073: --- Summary: StateStore schema incompatibility between 3.2 and 3.4 Key: SPARK-48073 URL: https://issues.apache.org/jira/browse/SPARK-48073 Project: Spark Issue

[jira] [Commented] (SPARK-44444) Enabled ANSI mode by default

[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836398#comment-17836398 ] L. C. Hsieh commented on SPARK-4: - Thank you [~dongjoon] > Enabled ANSI mode by default >

[jira] [Commented] (SPARK-44444) Enabled ANSI mode by default

[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836283#comment-17836283 ] L. C. Hsieh commented on SPARK-4: - What impact this change could bring to user applications?

[jira] [Commented] (SPARK-47773) Enhancing the Flexibility of Spark's Physical Plan to Enable Execution on Various Native Engines

[ https://issues.apache.org/jira/browse/SPARK-47773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835168#comment-17835168 ] L. C. Hsieh commented on SPARK-47773: - Can you open comment on the SPIP doc? > Enhancing the

[jira] [Assigned] (SPARK-47528) Add UserDefinedType support to DataTypeUtils.canWrite

[ https://issues.apache.org/jira/browse/SPARK-47528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-47528: --- Assignee: L. C. Hsieh > Add UserDefinedType support to DataTypeUtils.canWrite >

[jira] [Created] (SPARK-47528) Add UserDefinedType support to DataTypeUtils.canWrite

L. C. Hsieh created SPARK-47528: --- Summary: Add UserDefinedType support to DataTypeUtils.canWrite Key: SPARK-47528 URL: https://issues.apache.org/jira/browse/SPARK-47528 Project: Spark Issue

[jira] [Created] (SPARK-46791) Support Java `Set` in JavaTypeInference

L. C. Hsieh created SPARK-46791: --- Summary: Support Java `Set` in JavaTypeInference Key: SPARK-46791 URL: https://issues.apache.org/jira/browse/SPARK-46791 Project: Spark Issue Type:

[jira] [Created] (SPARK-46502) Support timestamp types in UnwrapCastInBinaryComparison

L. C. Hsieh created SPARK-46502: --- Summary: Support timestamp types in UnwrapCastInBinaryComparison Key: SPARK-46502 URL: https://issues.apache.org/jira/browse/SPARK-46502 Project: Spark Issue

[jira] [Assigned] (SPARK-46339) Directory with number name should not be treated as metadata log

[ https://issues.apache.org/jira/browse/SPARK-46339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-46339: --- Assignee: L. C. Hsieh > Directory with number name should not be treated as metadata log >

[jira] [Created] (SPARK-46339) Directory with number name should not be treated as metadata log

L. C. Hsieh created SPARK-46339: --- Summary: Directory with number name should not be treated as metadata log Key: SPARK-46339 URL: https://issues.apache.org/jira/browse/SPARK-46339 Project: Spark

[jira] [Created] (SPARK-45816) Return null when overflowing during casting from timestamp to integers

L. C. Hsieh created SPARK-45816: --- Summary: Return null when overflowing during casting from timestamp to integers Key: SPARK-45816 URL: https://issues.apache.org/jira/browse/SPARK-45816 Project: Spark

[jira] [Updated] (SPARK-45678) Cover BufferReleasingInputStream.available under tryOrFetchFailedException

[ https://issues.apache.org/jira/browse/SPARK-45678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-45678: Fix Version/s: 3.4.2 3.5.1 > Cover BufferReleasingInputStream.available under

[jira] [Updated] (SPARK-45678) Cover BufferReleasingInputStream.available under tryOrFetchFailedException

[ https://issues.apache.org/jira/browse/SPARK-45678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-45678: Issue Type: Bug (was: Improvement) > Cover BufferReleasingInputStream.available under

[jira] [Updated] (SPARK-45678) Cover BufferReleasingInputStream.available under tryOrFetchFailedException

[ https://issues.apache.org/jira/browse/SPARK-45678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-45678: Affects Version/s: 3.5.0 3.4.1 > Cover

[jira] [Created] (SPARK-45678) Cover BufferReleasingInputStream.available under tryOrFetchFailedException

L. C. Hsieh created SPARK-45678: --- Summary: Cover BufferReleasingInputStream.available under tryOrFetchFailedException Key: SPARK-45678 URL: https://issues.apache.org/jira/browse/SPARK-45678 Project:

[jira] [Created] (SPARK-45618) Remove BaseErrorHandler

L. C. Hsieh created SPARK-45618: --- Summary: Remove BaseErrorHandler Key: SPARK-45618 URL: https://issues.apache.org/jira/browse/SPARK-45618 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-45617) Upgrade Apache Commons Crypto 1.2.0

L. C. Hsieh created SPARK-45617: --- Summary: Upgrade Apache Commons Crypto 1.2.0 Key: SPARK-45617 URL: https://issues.apache.org/jira/browse/SPARK-45617 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-45473) Incorrect error message for RoundBase

L. C. Hsieh created SPARK-45473: --- Summary: Incorrect error message for RoundBase Key: SPARK-45473 URL: https://issues.apache.org/jira/browse/SPARK-45473 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-44859) Fix incorrect property name `asyncProgressCheckpointingInterval` in structured streaming doc

L. C. Hsieh created SPARK-44859: --- Summary: Fix incorrect property name `asyncProgressCheckpointingInterval` in structured streaming doc Key: SPARK-44859 URL: https://issues.apache.org/jira/browse/SPARK-44859

[jira] [Commented] (SPARK-44646) Migrate Log4j 2.x in Spark 3.4.1 to Logback

[ https://issues.apache.org/jira/browse/SPARK-44646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750895#comment-17750895 ] L. C. Hsieh commented on SPARK-44646: - I have not used it but maybe you can try

[jira] [Commented] (SPARK-41283) Feature parity: Functions API in Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-41283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691922#comment-17691922 ] L. C. Hsieh commented on SPARK-41283: - Seems all sub-tasks are resolved? Can we resolve this now? >

[jira] [Resolved] (SPARK-42222) Spark 3.3 Backport: SPARK-41344 Reading V2 datasource masks underlying error

[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-4. - Resolution: Won't Fix > Spark 3.3 Backport: SPARK-41344 Reading V2 datasource masks underlying

[jira] [Resolved] (SPARK-41695) Upgrade netty to 4.1.86.Final

[ https://issues.apache.org/jira/browse/SPARK-41695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-41695. - Resolution: Won't Fix > Upgrade netty to 4.1.86.Final > - > >

[jira] [Resolved] (SPARK-41696) Upgrade Hadoop to 3.3.4

[ https://issues.apache.org/jira/browse/SPARK-41696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-41696. - Resolution: Won't Fix > Upgrade Hadoop to 3.3.4 > --- > >

[jira] [Commented] (SPARK-42346) distinct(count colname) with UNION ALL causes query analyzer bug

[ https://issues.apache.org/jira/browse/SPARK-42346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684332#comment-17684332 ] L. C. Hsieh commented on SPARK-42346: - I will wait for this patch before cutting RC1. >

[jira] [Commented] (SPARK-41049) Nondeterministic expressions have unstable values if they are children of CodegenFallback expressions

[ https://issues.apache.org/jira/browse/SPARK-41049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655532#comment-17655532 ] L. C. Hsieh commented on SPARK-41049: - For a correctness bug, I think we should backport it, though

[jira] [Created] (SPARK-41442) Only update SQLMetric value if merging with valid metric

L. C. Hsieh created SPARK-41442: --- Summary: Only update SQLMetric value if merging with valid metric Key: SPARK-41442 URL: https://issues.apache.org/jira/browse/SPARK-41442 Project: Spark Issue

[jira] [Resolved] (SPARK-41155) Add error message to SchemaColumnConvertNotSupportedException

[ https://issues.apache.org/jira/browse/SPARK-41155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-41155. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38669

[jira] [Created] (SPARK-41155) Add error message to SchemaColumnConvertNotSupportedException

L. C. Hsieh created SPARK-41155: --- Summary: Add error message to SchemaColumnConvertNotSupportedException Key: SPARK-41155 URL: https://issues.apache.org/jira/browse/SPARK-41155 Project: Spark

[jira] [Created] (SPARK-40826) Add additional checkpoint rename file check

L. C. Hsieh created SPARK-40826: --- Summary: Add additional checkpoint rename file check Key: SPARK-40826 URL: https://issues.apache.org/jira/browse/SPARK-40826 Project: Spark Issue Type:

[jira] [Commented] (SPARK-36681) Fail to load Snappy codec

[ https://issues.apache.org/jira/browse/SPARK-36681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17615201#comment-17615201 ] L. C. Hsieh commented on SPARK-36681: - This is fixed in 3.3.0 and later, yes, by upgrading to Hadoop

[jira] [Commented] (SPARK-40508) Treat unknown partitioning as UnknownPartitioning

[ https://issues.apache.org/jira/browse/SPARK-40508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607900#comment-17607900 ] L. C. Hsieh commented on SPARK-40508: - [~csun] Seems he is already in contributor list. I just

[jira] [Assigned] (SPARK-40508) Treat unknown partitioning as UnknownPartitioning

[ https://issues.apache.org/jira/browse/SPARK-40508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-40508: --- Assignee: Ted Yu > Treat unknown partitioning as UnknownPartitioning >

[jira] [Resolved] (SPARK-40484) Upgrade log4j2 to 2.19.0

[ https://issues.apache.org/jira/browse/SPARK-40484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-40484. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37926

[jira] [Assigned] (SPARK-40484) Upgrade log4j2 to 2.19.0

[ https://issues.apache.org/jira/browse/SPARK-40484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-40484: --- Assignee: Yang Jie > Upgrade log4j2 to 2.19.0 > > >

[jira] [Resolved] (SPARK-40406) The default logging should go to stderr

[ https://issues.apache.org/jira/browse/SPARK-40406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-40406. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37854

[jira] [Assigned] (SPARK-40406) The default logging should go to stderr

[ https://issues.apache.org/jira/browse/SPARK-40406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-40406: --- Assignee: Yang Jie > The default logging should go to stderr >

[jira] [Resolved] (SPARK-40033) Nested schema pruning support through element_at

[ https://issues.apache.org/jira/browse/SPARK-40033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-40033. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37463

[jira] [Assigned] (SPARK-40033) Nested schema pruning support through element_at

[ https://issues.apache.org/jira/browse/SPARK-40033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-40033: --- Assignee: XiDuo You > Nested schema pruning support through element_at >

[jira] [Resolved] (SPARK-40234) Clean only MDC items set by Spark

[ https://issues.apache.org/jira/browse/SPARK-40234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-40234. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37680

[jira] [Assigned] (SPARK-40234) Clean only MDC items set by Spark

[ https://issues.apache.org/jira/browse/SPARK-40234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-40234: --- Assignee: L. C. Hsieh > Clean only MDC items set by Spark >

[jira] [Created] (SPARK-40234) Clean only MDC items set by Spark

L. C. Hsieh created SPARK-40234: --- Summary: Clean only MDC items set by Spark Key: SPARK-40234 URL: https://issues.apache.org/jira/browse/SPARK-40234 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-40088) Add SparkPlanWIthAQESuite

[ https://issues.apache.org/jira/browse/SPARK-40088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-40088. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37619

[jira] [Assigned] (SPARK-40088) Add SparkPlanWIthAQESuite

[ https://issues.apache.org/jira/browse/SPARK-40088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-40088: --- Assignee: Kazuyuki Tanimura > Add SparkPlanWIthAQESuite > - > >

[jira] [Commented] (SPARK-40128) Add DELTA_LENGTH_BYTE_ARRAY as a recognized standalone encoding in VectorizedColumnReader

[ https://issues.apache.org/jira/browse/SPARK-40128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581077#comment-17581077 ] L. C. Hsieh commented on SPARK-40128: - Added [~dennishuo] as Spark contributor and assigned this

[jira] [Assigned] (SPARK-40128) Add DELTA_LENGTH_BYTE_ARRAY as a recognized standalone encoding in VectorizedColumnReader

[ https://issues.apache.org/jira/browse/SPARK-40128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-40128: --- Assignee: Dennis Huo > Add DELTA_LENGTH_BYTE_ARRAY as a recognized standalone encoding in

[jira] [Resolved] (SPARK-39635) Custom driver metrics for Datasource v2

[ https://issues.apache.org/jira/browse/SPARK-39635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-39635. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37205

[jira] [Assigned] (SPARK-39635) Custom driver metrics for Datasource v2

[ https://issues.apache.org/jira/browse/SPARK-39635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-39635: --- Assignee: Karuppayya > Custom driver metrics for Datasource v2 >

[jira] [Assigned] (SPARK-39958) Add warning log when unable to load custom metric object

[ https://issues.apache.org/jira/browse/SPARK-39958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-39958: --- Assignee: L. C. Hsieh > Add warning log when unable to load custom metric object >

[jira] [Resolved] (SPARK-39958) Add warning log when unable to load custom metric object

[ https://issues.apache.org/jira/browse/SPARK-39958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-39958. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37386

[jira] [Resolved] (SPARK-39954) Upgrade ASM to 9.3

[ https://issues.apache.org/jira/browse/SPARK-39954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-39954. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37382

[jira] [Assigned] (SPARK-39954) Upgrade ASM to 9.3

[ https://issues.apache.org/jira/browse/SPARK-39954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-39954: --- Assignee: Dongjoon Hyun > Upgrade ASM to 9.3 > -- > > Key:

[jira] [Created] (SPARK-39958) Add warning log when unable to load custom metric object

L. C. Hsieh created SPARK-39958: --- Summary: Add warning log when unable to load custom metric object Key: SPARK-39958 URL: https://issues.apache.org/jira/browse/SPARK-39958 Project: Spark Issue

[jira] [Assigned] (SPARK-39785) Use setBufferedIo instead of withBufferedIo to cleanup log4j2 deprecated api usage

[ https://issues.apache.org/jira/browse/SPARK-39785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-39785: --- Assignee: Yang Jie > Use setBufferedIo instead of withBufferedIo to cleanup log4j2

[jira] [Resolved] (SPARK-39785) Use setBufferedIo instead of withBufferedIo to cleanup log4j2 deprecated api usage

[ https://issues.apache.org/jira/browse/SPARK-39785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-39785. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37193

[jira] [Resolved] (SPARK-39134) Add custom metric of skipped null values for stream join operator

[ https://issues.apache.org/jira/browse/SPARK-39134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-39134. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36491

[jira] [Assigned] (SPARK-39134) Add custom metric of skipped null values for stream join operator

[ https://issues.apache.org/jira/browse/SPARK-39134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-39134: --- Assignee: L. C. Hsieh > Add custom metric of skipped null values for stream join operator

[jira] [Created] (SPARK-39134) Add custom metric of skipped null values for stream join operator

L. C. Hsieh created SPARK-39134: --- Summary: Add custom metric of skipped null values for stream join operator Key: SPARK-39134 URL: https://issues.apache.org/jira/browse/SPARK-39134 Project: Spark

[jira] [Commented] (SPARK-39070) Pivoting a dataframe after two explodes raises an org.codehaus.commons.compiler.CompileException

[ https://issues.apache.org/jira/browse/SPARK-39070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17530850#comment-17530850 ] L. C. Hsieh commented on SPARK-39070: - SPARK-38285 is already backported to branch-3.2. It will be

[jira] [Commented] (SPARK-39070) Pivoting a dataframe after two explodes raises an org.codehaus.commons.compiler.CompileException

[ https://issues.apache.org/jira/browse/SPARK-39070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17530589#comment-17530589 ] L. C. Hsieh commented on SPARK-39070: - Hmm, SPARK-34638 is a feature requirement but this is a bug

[jira] [Updated] (SPARK-38918) Nested column pruning should filter out attributes that do not belong to the current relation

[ https://issues.apache.org/jira/browse/SPARK-38918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-38918: Fix Version/s: 3.1.3 3.0.4 > Nested column pruning should filter out

[jira] [Resolved] (SPARK-38918) Nested column pruning should filter out attributes that do not belong to the current relation

[ https://issues.apache.org/jira/browse/SPARK-38918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-38918. - Fix Version/s: 3.3.0 3.4.0 Assignee: Allison Wang Resolution:

[jira] [Resolved] (SPARK-39002) StringEndsWith/Contains support push down to Parquet so that we can leverage dictionary filter

[ https://issues.apache.org/jira/browse/SPARK-39002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-39002. - Fix Version/s: 3.4.0 Assignee: EdisonWang Resolution: Fixed >

[jira] [Resolved] (SPARK-38977) Fix schema pruning with correlated subqueries

[ https://issues.apache.org/jira/browse/SPARK-38977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-38977. - Fix Version/s: 3.3.0 3.2.2 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-38977) Fix schema pruning with correlated subqueries

[ https://issues.apache.org/jira/browse/SPARK-38977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-38977: --- Assignee: Anton Okolnychyi > Fix schema pruning with correlated subqueries >

[jira] [Commented] (SPARK-38824) Bug in async commit of Kafka offset in DirectKafkaInputDStream

[ https://issues.apache.org/jira/browse/SPARK-38824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17522996#comment-17522996 ] L. C. Hsieh commented on SPARK-38824: - Ideally, I think we might need to call commitAll in stop

[jira] [Commented] (SPARK-38824) Bug in async commit of Kafka offset in DirectKafkaInputDStream

[ https://issues.apache.org/jira/browse/SPARK-38824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17522995#comment-17522995 ] L. C. Hsieh commented on SPARK-38824: - Hmm, I think I can get the idea you want to point out. The

[jira] [Assigned] (SPARK-38840) Enable spark.sql.parquet.enableNestedColumnVectorizedReader on master branch by default

[ https://issues.apache.org/jira/browse/SPARK-38840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-38840: --- Assignee: Chao Sun > Enable spark.sql.parquet.enableNestedColumnVectorizedReader on master

[jira] [Resolved] (SPARK-38840) Enable spark.sql.parquet.enableNestedColumnVectorizedReader on master branch by default

[ https://issues.apache.org/jira/browse/SPARK-38840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-38840. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36124

[jira] [Resolved] (SPARK-34863) Support nested column in Spark Parquet vectorized readers

[ https://issues.apache.org/jira/browse/SPARK-34863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-34863. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34659

[jira] [Resolved] (SPARK-38605) Retrying on file manager operation in HDFSMetadataLog

[ https://issues.apache.org/jira/browse/SPARK-38605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-38605. - Resolution: Won't Fix > Retrying on file manager operation in HDFSMetadataLog >

[jira] [Commented] (SPARK-38605) Retrying on file manager operation in HDFSMetadataLog

[ https://issues.apache.org/jira/browse/SPARK-38605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509337#comment-17509337 ] L. C. Hsieh commented on SPARK-38605: - cc [~kabhwan] Please let me know if you have some concerns or

[jira] [Created] (SPARK-38605) Retrying on file manager operation in HDFSMetadataLog

L. C. Hsieh created SPARK-38605: --- Summary: Retrying on file manager operation in HDFSMetadataLog Key: SPARK-38605 URL: https://issues.apache.org/jira/browse/SPARK-38605 Project: Spark Issue

[jira] [Created] (SPARK-38593) Incorporate numRowsDroppedByWatermark metric from SessionWindowStateStoreRestoreExec into StateOperatorProgress

L. C. Hsieh created SPARK-38593: --- Summary: Incorporate numRowsDroppedByWatermark metric from SessionWindowStateStoreRestoreExec into StateOperatorProgress Key: SPARK-38593 URL:

[jira] [Updated] (SPARK-38529) Prevent GeneratorNestedColumnAliasing to be applied to non-Explode generators

[ https://issues.apache.org/jira/browse/SPARK-38529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-38529: Summary: Prevent GeneratorNestedColumnAliasing to be applied to non-Explode generators (was:

[jira] [Updated] (SPARK-38529) Prevent GeneratorNestedColumnAliasing to be applied to non-Explode generators

[ https://issues.apache.org/jira/browse/SPARK-38529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-38529: Issue Type: Improvement (was: Bug) > Prevent GeneratorNestedColumnAliasing to be applied to

[jira] [Resolved] (SPARK-38549) SessionWindowStateStoreRestoreExec should provide numRowsDroppedByWatermark metric

[ https://issues.apache.org/jira/browse/SPARK-38549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-38549. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35854

[jira] [Assigned] (SPARK-38549) SessionWindowStateStoreRestoreExec should provide numRowsDroppedByWatermark metric

[ https://issues.apache.org/jira/browse/SPARK-38549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-38549: --- Assignee: L. C. Hsieh > SessionWindowStateStoreRestoreExec should provide

[jira] [Created] (SPARK-38549) SessionWindowStateStoreRestoreExec should provide numRowsDroppedByWatermark metric

L. C. Hsieh created SPARK-38549: --- Summary: SessionWindowStateStoreRestoreExec should provide numRowsDroppedByWatermark metric Key: SPARK-38549 URL: https://issues.apache.org/jira/browse/SPARK-38549

[jira] [Commented] (SPARK-38531) "Prune unrequired child index" branch of ColumnPruning has wrong condition

[ https://issues.apache.org/jira/browse/SPARK-38531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505993#comment-17505993 ] L. C. Hsieh commented on SPARK-38531: - Oh, I think I get your point. The optimizer rule is to prune

[jira] [Resolved] (SPARK-36681) Fail to load Snappy codec

[ https://issues.apache.org/jira/browse/SPARK-36681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-36681. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35784

[jira] [Assigned] (SPARK-36681) Fail to load Snappy codec

[ https://issues.apache.org/jira/browse/SPARK-36681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-36681: --- Assignee: L. C. Hsieh > Fail to load Snappy codec > - > >

[jira] [Resolved] (SPARK-38458) Fix always false condition in LogDivertAppender#initLayout

[ https://issues.apache.org/jira/browse/SPARK-38458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-38458. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35780

[jira] [Assigned] (SPARK-38458) Fix always false condition in LogDivertAppender#initLayout

[ https://issues.apache.org/jira/browse/SPARK-38458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-38458: --- Assignee: Yang Jie > Fix always false condition in LogDivertAppender#initLayout >

[jira] [Resolved] (SPARK-38443) Document config STREAMING_SESSION_WINDOW_MERGE_SESSIONS_IN_LOCAL_PARTITION

[ https://issues.apache.org/jira/browse/SPARK-38443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-38443. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35762

[jira] [Assigned] (SPARK-38443) Document config STREAMING_SESSION_WINDOW_MERGE_SESSIONS_IN_LOCAL_PARTITION

[ https://issues.apache.org/jira/browse/SPARK-38443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-38443: --- Assignee: L. C. Hsieh > Document config

[jira] [Created] (SPARK-38443) Document config STREAMING_SESSION_WINDOW_MERGE_SESSIONS_IN_LOCAL_PARTITION

L. C. Hsieh created SPARK-38443: --- Summary: Document config STREAMING_SESSION_WINDOW_MERGE_SESSIONS_IN_LOCAL_PARTITION Key: SPARK-38443 URL: https://issues.apache.org/jira/browse/SPARK-38443 Project:

[jira] [Resolved] (SPARK-38285) ClassCastException: GenericArrayData cannot be cast to InternalRow

[ https://issues.apache.org/jira/browse/SPARK-38285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-38285. - Fix Version/s: 3.3.0 3.2.2 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-38285) ClassCastException: GenericArrayData cannot be cast to InternalRow

[ https://issues.apache.org/jira/browse/SPARK-38285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-38285: --- Assignee: L. C. Hsieh > ClassCastException: GenericArrayData cannot be cast to InternalRow

[jira] [Commented] (SPARK-38285) ClassCastException: GenericArrayData cannot be cast to InternalRow

[ https://issues.apache.org/jira/browse/SPARK-38285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17502123#comment-17502123 ] L. C. Hsieh commented on SPARK-38285: - No worries as bug fix is not blocked by code freeze. I've

[jira] [Commented] (SPARK-38285) ClassCastException: GenericArrayData cannot be cast to InternalRow

[ https://issues.apache.org/jira/browse/SPARK-38285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497826#comment-17497826 ] L. C. Hsieh commented on SPARK-38285: - Thanks for reporting this. I will take a look. >

[jira] [Resolved] (SPARK-38227) Apply strict nullability of nested column in time window / session window

[ https://issues.apache.org/jira/browse/SPARK-38227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-38227. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35543

[jira] [Assigned] (SPARK-38227) Apply strict nullability of nested column in time window / session window

[ https://issues.apache.org/jira/browse/SPARK-38227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-38227: --- Assignee: Jungtaek Lim > Apply strict nullability of nested column in time window /

[jira] [Commented] (SPARK-25271) Creating parquet table with all the column null throws exception

[ https://issues.apache.org/jira/browse/SPARK-25271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17494342#comment-17494342 ] L. C. Hsieh commented on SPARK-25271: - Based on this JIRA, we only have this fix since 2.4.8. I

  1   2   3   4   5   6   7   8   9   10   >