[jira] [Commented] (SPARK-37210) An error occurred while concurrently writing to different static partitions

[ https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17573218#comment-17573218 ] Apache Spark commented on SPARK-37210: -- User 'wForget' has created a pull request f

[jira] [Resolved] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao resolved SPARK-39909. Fix Version/s: 3.4.0 Resolution: Fixed > Organize the check of push down information for JD

[jira] [Assigned] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao reassigned SPARK-39909: -- Assignee: miracle > Organize the check of push down information for JDBCV2Suite > ---

[jira] [Commented] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17573197#comment-17573197 ] Mars commented on SPARK-39909: -- [~huaxingao] This is me :D [~miracle]  , my Jira id: miracl

[jira] [Commented] (SPARK-39908) Translate common used aggregate functions for pushdown (non ANSI)

[ https://issues.apache.org/jira/browse/SPARK-39908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17573194#comment-17573194 ] Sun BiaoBiao commented on SPARK-39908: -- all   aggregate functions doesn't follow AN

[jira] [Commented] (SPARK-38934) Provider TemporaryAWSCredentialsProvider has no credentials

[ https://issues.apache.org/jira/browse/SPARK-38934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17573193#comment-17573193 ] Lily Kim commented on SPARK-38934: -- Hey. It hasn't resolved yet. Why did you close it?

[jira] [Commented] (SPARK-39919) Hitting Interrupt when no cell is running kills pyspark jupyter kernel

[ https://issues.apache.org/jira/browse/SPARK-39919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17573175#comment-17573175 ] Dongjoon Hyun commented on SPARK-39919: --- Thank you for reporting, [~nsingl00] . I'

[jira] [Updated] (SPARK-39919) Hitting Interrupt when no cell is running kills pyspark jupyter kernel

[ https://issues.apache.org/jira/browse/SPARK-39919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39919: -- Target Version/s: (was: 3.2.2) > Hitting Interrupt when no cell is running kills pyspark jup

[jira] [Updated] (SPARK-39919) Hitting Interrupt when no cell is running kills pyspark jupyter kernel

[ https://issues.apache.org/jira/browse/SPARK-39919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Singla updated SPARK-39919: Description: In Pyspark 3.2 kernel with Jupyter notebook, When no cell is running and I hit inter

[jira] [Updated] (SPARK-39919) Hitting Interrupt when no cell is running kills pyspark jupyter kernel

[ https://issues.apache.org/jira/browse/SPARK-39919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Singla updated SPARK-39919: Description: In Pyspark 3.2 kernel with Jupyter notebook, When no cell is running and I hit interr

[jira] [Updated] (SPARK-39919) Hitting Interrupt when no cell is running kills pyspark jupyter kernel

[ https://issues.apache.org/jira/browse/SPARK-39919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Singla updated SPARK-39919: Summary: Hitting Interrupt when no cell is running kills pyspark jupyter kernel (was: Hitting Int

[jira] [Created] (SPARK-39926) Fix bug in existence DEFAULT value lookups for non-vectorized Parquet scans

Daniel created SPARK-39926: -- Summary: Fix bug in existence DEFAULT value lookups for non-vectorized Parquet scans Key: SPARK-39926 URL: https://issues.apache.org/jira/browse/SPARK-39926 Project: Spark

[jira] [Commented] (SPARK-38330) Certificate doesn't match any of the subject alternative names: [*.s3.amazonaws.com, s3.amazonaws.com]

[ https://issues.apache.org/jira/browse/SPARK-38330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17573141#comment-17573141 ] Steve Loughran commented on SPARK-38330: the hadoop 3.3.4 rC0 will fix this with

[jira] [Commented] (SPARK-38934) Provider TemporaryAWSCredentialsProvider has no credentials

[ https://issues.apache.org/jira/browse/SPARK-38934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17573138#comment-17573138 ] Steve Loughran commented on SPARK-38934: so that's a config problem? not a bug?

[jira] [Resolved] (SPARK-38934) Provider TemporaryAWSCredentialsProvider has no credentials

[ https://issues.apache.org/jira/browse/SPARK-38934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved SPARK-38934. Resolution: Invalid > Provider TemporaryAWSCredentialsProvider has no credentials > --

[jira] [Commented] (SPARK-38958) Override S3 Client in Spark Write/Read calls

[ https://issues.apache.org/jira/browse/SPARK-38958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17573137#comment-17573137 ] Steve Loughran commented on SPARK-38958: #. api is public, but we have changed t

[jira] [Commented] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17573091#comment-17573091 ] Huaxin Gao commented on SPARK-39909: Hi Chen Liang, do you have a jira id? > Organi

[jira] [Commented] (SPARK-39802) Support recursive references in Avro schemas in Spark

[ https://issues.apache.org/jira/browse/SPARK-39802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17573076#comment-17573076 ] Erik Krogen commented on SPARK-39802: - The Spark type system doesn't support recursi

[jira] [Updated] (SPARK-25718) Detect recursive reference in Avro schema and throw exception

[ https://issues.apache.org/jira/browse/SPARK-25718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated SPARK-25718: Description: Avro schema allows recursive reference, e.g. the example schema in https://avro.apac

[jira] [Updated] (SPARK-39893) Push limit 1 to the aggregate's child plan if grouping expressions and aggregate expressions are foldable

[ https://issues.apache.org/jira/browse/SPARK-39893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-39893: Description: If all group expressions are foldable, the result of this aggregate will always be OneRowRel

[jira] [Updated] (SPARK-39893) Push limit 1 to the aggregate's child plan if grouping expressions and aggregate expressions are foldable

[ https://issues.apache.org/jira/browse/SPARK-39893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-39893: Description: If all group expressions are foldable, the result of this aggregate will always be OneRowRel

[jira] [Updated] (SPARK-39893) Push limit 1 to the aggregate's child plan if grouping expressions and aggregate expressions are foldable

[ https://issues.apache.org/jira/browse/SPARK-39893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-39893: Description: If all group expressions are foldable, the result of this aggregate will always be OneRowRel

[jira] [Updated] (SPARK-39893) Push limit 1 to the aggregate's child plan if grouping expressions and aggregate expressions are foldable

[ https://issues.apache.org/jira/browse/SPARK-39893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-39893: Description: If all group expressions are foldable, the result of this aggregate will always be OneRowRel

[jira] [Updated] (SPARK-39893) Push limit 1 to the aggregate's child plan if grouping expressions and aggregate expressions are foldable

[ https://issues.apache.org/jira/browse/SPARK-39893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-39893: Summary: Push limit 1 to the aggregate's child plan if grouping expressions and aggregate expressions are

[jira] [Commented] (SPARK-39925) Add array_sort(column, comparator) overload to DataFrame operations

[ https://issues.apache.org/jira/browse/SPARK-39925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17573046#comment-17573046 ] Brandon Dahler commented on SPARK-39925: I'm working on a PR for this issue > A

[jira] [Created] (SPARK-39925) Add array_sort(column, comparator) overload to DataFrame operations

Brandon Dahler created SPARK-39925: -- Summary: Add array_sort(column, comparator) overload to DataFrame operations Key: SPARK-39925 URL: https://issues.apache.org/jira/browse/SPARK-39925 Project: Spar

[jira] [Assigned] (SPARK-39924) Extract `constructUnsupportedMergeException` method to deduplicate code in `AccumulatorV2`

[ https://issues.apache.org/jira/browse/SPARK-39924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39924: Assignee: (was: Apache Spark) > Extract `constructUnsupportedMergeException` method t

[jira] [Assigned] (SPARK-39924) Extract `constructUnsupportedMergeException` method to deduplicate code in `AccumulatorV2`

[ https://issues.apache.org/jira/browse/SPARK-39924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39924: Assignee: Apache Spark > Extract `constructUnsupportedMergeException` method to deduplic

[jira] [Commented] (SPARK-39924) Extract `constructUnsupportedMergeException` method to deduplicate code in `AccumulatorV2`

[ https://issues.apache.org/jira/browse/SPARK-39924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572964#comment-17572964 ] Apache Spark commented on SPARK-39924: -- User 'LuciferYang' has created a pull reque

[jira] [Created] (SPARK-39924) Extract `constructUnsupportedMergeException` method to deduplicate code in `AccumulatorV2`

Yang Jie created SPARK-39924: Summary: Extract `constructUnsupportedMergeException` method to deduplicate code in `AccumulatorV2` Key: SPARK-39924 URL: https://issues.apache.org/jira/browse/SPARK-39924 P

[jira] [Resolved] (SPARK-39849) Dataset.as(StructType) fills missing new columns with null value

[ https://issues.apache.org/jira/browse/SPARK-39849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39849. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37264 [https://gith

[jira] [Assigned] (SPARK-39849) Dataset.as(StructType) fills missing new columns with null value

[ https://issues.apache.org/jira/browse/SPARK-39849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39849: --- Assignee: Cheng Su > Dataset.as(StructType) fills missing new columns with null value > ---

[jira] [Assigned] (SPARK-37194) Avoid unnecessary sort in FileFormatWriter if it's not dynamic partition

[ https://issues.apache.org/jira/browse/SPARK-37194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37194: --- Assignee: XiDuo You > Avoid unnecessary sort in FileFormatWriter if it's not dynamic partit

[jira] [Resolved] (SPARK-37194) Avoid unnecessary sort in FileFormatWriter if it's not dynamic partition

[ https://issues.apache.org/jira/browse/SPARK-37194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37194. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37290 [https://gith

[jira] [Commented] (SPARK-39923) Put QueryContext to array instead of Option

[ https://issues.apache.org/jira/browse/SPARK-39923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572932#comment-17572932 ] Apache Spark commented on SPARK-39923: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-39923) Put QueryContext to array instead of Option

[ https://issues.apache.org/jira/browse/SPARK-39923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39923: Assignee: (was: Apache Spark) > Put QueryContext to array instead of Option > ---

[jira] [Assigned] (SPARK-39923) Put QueryContext to array instead of Option

[ https://issues.apache.org/jira/browse/SPARK-39923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39923: Assignee: Apache Spark > Put QueryContext to array instead of Option > --

[jira] [Created] (SPARK-39923) Put QueryContext to array instead of Option

Max Gekk created SPARK-39923: Summary: Put QueryContext to array instead of Option Key: SPARK-39923 URL: https://issues.apache.org/jira/browse/SPARK-39923 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-39918) Replace the wording "un-comparable" with "incomparable"

[ https://issues.apache.org/jira/browse/SPARK-39918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39918. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37338 [https://gi

[jira] [Commented] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572869#comment-17572869 ] Apache Spark commented on SPARK-39909: -- User 'yabola' has created a pull request fo

[jira] [Assigned] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39909: Assignee: Apache Spark > Organize the check of push down information for JDBCV2Suite > --

[jira] [Assigned] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39909: Assignee: (was: Apache Spark) > Organize the check of push down information for JDBCV

[jira] [Commented] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572867#comment-17572867 ] Apache Spark commented on SPARK-39909: -- User 'yabola' has created a pull request fo

[jira] [Commented] (SPARK-38639) Support ignoreCorruptRecord flag to ensure querying broken sequence file table smoothly

[ https://issues.apache.org/jira/browse/SPARK-38639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572858#comment-17572858 ] Apache Spark commented on SPARK-38639: -- User 'caican00' has created a pull request

[jira] [Commented] (SPARK-38639) Support ignoreCorruptRecord flag to ensure querying broken sequence file table smoothly

[ https://issues.apache.org/jira/browse/SPARK-38639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572857#comment-17572857 ] Apache Spark commented on SPARK-38639: -- User 'caican00' has created a pull request

[jira] [Commented] (SPARK-39922) Add a microbenchmark for `Utils#getIteratorSize`

[ https://issues.apache.org/jira/browse/SPARK-39922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572791#comment-17572791 ] Apache Spark commented on SPARK-39922: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-39922) Add a microbenchmark for `Utils#getIteratorSize`

[ https://issues.apache.org/jira/browse/SPARK-39922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39922: Assignee: (was: Apache Spark) > Add a microbenchmark for `Utils#getIteratorSize` > --

[jira] [Assigned] (SPARK-39922) Add a microbenchmark for `Utils#getIteratorSize`

[ https://issues.apache.org/jira/browse/SPARK-39922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39922: Assignee: Apache Spark > Add a microbenchmark for `Utils#getIteratorSize` > -

[jira] [Commented] (SPARK-39922) Add a microbenchmark for `Utils#getIteratorSize`

[ https://issues.apache.org/jira/browse/SPARK-39922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572790#comment-17572790 ] Apache Spark commented on SPARK-39922: -- User 'LuciferYang' has created a pull reque