[jira] [Resolved] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao resolved SPARK-39909. Fix Version/s: 3.4.0 Resolution: Fixed > Organize the check of push down information for

[jira] [Assigned] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao reassigned SPARK-39909: -- Assignee: miracle > Organize the check of push down information for JDBCV2Suite >

[jira] [Commented] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573197#comment-17573197 ] Mars commented on SPARK-39909: -- [~huaxingao] This is me :D [~miracle]  , my Jira id: miracle > Organize

[jira] [Commented] (SPARK-39908) Translate common used aggregate functions for pushdown (non ANSI)

[ https://issues.apache.org/jira/browse/SPARK-39908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573194#comment-17573194 ] Sun BiaoBiao commented on SPARK-39908: -- all   aggregate functions doesn't follow ANSI  list:    

[jira] [Commented] (SPARK-38934) Provider TemporaryAWSCredentialsProvider has no credentials

[ https://issues.apache.org/jira/browse/SPARK-38934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573193#comment-17573193 ] Lily Kim commented on SPARK-38934: -- Hey. It hasn't resolved yet. Why did you close it? > Provider

[jira] [Commented] (SPARK-39919) Hitting Interrupt when no cell is running kills pyspark jupyter kernel

[ https://issues.apache.org/jira/browse/SPARK-39919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573175#comment-17573175 ] Dongjoon Hyun commented on SPARK-39919: --- Thank you for reporting, [~nsingl00] . I'll take a look

[jira] [Updated] (SPARK-39919) Hitting Interrupt when no cell is running kills pyspark jupyter kernel

[ https://issues.apache.org/jira/browse/SPARK-39919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39919: -- Target Version/s: (was: 3.2.2) > Hitting Interrupt when no cell is running kills pyspark

[jira] [Updated] (SPARK-39919) Hitting Interrupt when no cell is running kills pyspark jupyter kernel

[ https://issues.apache.org/jira/browse/SPARK-39919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Singla updated SPARK-39919: Description: In Pyspark 3.2 kernel with Jupyter notebook, When no cell is running and I hit

[jira] [Updated] (SPARK-39919) Hitting Interrupt when no cell is running kills pyspark jupyter kernel

[ https://issues.apache.org/jira/browse/SPARK-39919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Singla updated SPARK-39919: Description: In Pyspark 3.2 kernel with Jupyter notebook, When no cell is running and I hit

[jira] [Updated] (SPARK-39919) Hitting Interrupt when no cell is running kills pyspark jupyter kernel

[ https://issues.apache.org/jira/browse/SPARK-39919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Singla updated SPARK-39919: Summary: Hitting Interrupt when no cell is running kills pyspark jupyter kernel (was: Hitting

[jira] [Created] (SPARK-39926) Fix bug in existence DEFAULT value lookups for non-vectorized Parquet scans

Daniel created SPARK-39926: -- Summary: Fix bug in existence DEFAULT value lookups for non-vectorized Parquet scans Key: SPARK-39926 URL: https://issues.apache.org/jira/browse/SPARK-39926 Project: Spark

[jira] [Commented] (SPARK-38330) Certificate doesn't match any of the subject alternative names: [*.s3.amazonaws.com, s3.amazonaws.com]

[ https://issues.apache.org/jira/browse/SPARK-38330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573141#comment-17573141 ] Steve Loughran commented on SPARK-38330: the hadoop 3.3.4 rC0 will fix this with that cut of the

[jira] [Commented] (SPARK-38934) Provider TemporaryAWSCredentialsProvider has no credentials

[ https://issues.apache.org/jira/browse/SPARK-38934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573138#comment-17573138 ] Steve Loughran commented on SPARK-38934: so that's a config problem? not a bug? closing >

[jira] [Resolved] (SPARK-38934) Provider TemporaryAWSCredentialsProvider has no credentials

[ https://issues.apache.org/jira/browse/SPARK-38934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved SPARK-38934. Resolution: Invalid > Provider TemporaryAWSCredentialsProvider has no credentials >

[jira] [Commented] (SPARK-38958) Override S3 Client in Spark Write/Read calls

[ https://issues.apache.org/jira/browse/SPARK-38958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573137#comment-17573137 ] Steve Loughran commented on SPARK-38958: #. api is public, but we have changed the api

[jira] [Commented] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573091#comment-17573091 ] Huaxin Gao commented on SPARK-39909: Hi Chen Liang, do you have a jira id? > Organize the check of

[jira] [Commented] (SPARK-39802) Support recursive references in Avro schemas in Spark

[ https://issues.apache.org/jira/browse/SPARK-39802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573076#comment-17573076 ] Erik Krogen commented on SPARK-39802: - The Spark type system doesn't support recursive types, so

[jira] [Updated] (SPARK-25718) Detect recursive reference in Avro schema and throw exception

[ https://issues.apache.org/jira/browse/SPARK-25718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated SPARK-25718: Description: Avro schema allows recursive reference, e.g. the example schema in

[jira] [Updated] (SPARK-39893) Push limit 1 to the aggregate's child plan if grouping expressions and aggregate expressions are foldable

[ https://issues.apache.org/jira/browse/SPARK-39893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-39893: Description: If all group expressions are foldable, the result of this aggregate will always be

[jira] [Updated] (SPARK-39893) Push limit 1 to the aggregate's child plan if grouping expressions and aggregate expressions are foldable

[ https://issues.apache.org/jira/browse/SPARK-39893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-39893: Description: If all group expressions are foldable, the result of this aggregate will always be

[jira] [Updated] (SPARK-39893) Push limit 1 to the aggregate's child plan if grouping expressions and aggregate expressions are foldable

[ https://issues.apache.org/jira/browse/SPARK-39893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-39893: Description: If all group expressions are foldable, the result of this aggregate will always be

[jira] [Updated] (SPARK-39893) Push limit 1 to the aggregate's child plan if grouping expressions and aggregate expressions are foldable

[ https://issues.apache.org/jira/browse/SPARK-39893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-39893: Description: If all group expressions are foldable, the result of this aggregate will always be

[jira] [Updated] (SPARK-39893) Push limit 1 to the aggregate's child plan if grouping expressions and aggregate expressions are foldable

[ https://issues.apache.org/jira/browse/SPARK-39893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-39893: Summary: Push limit 1 to the aggregate's child plan if grouping expressions and aggregate expressions are

[jira] [Commented] (SPARK-39925) Add array_sort(column, comparator) overload to DataFrame operations

[ https://issues.apache.org/jira/browse/SPARK-39925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573046#comment-17573046 ] Brandon Dahler commented on SPARK-39925: I'm working on a PR for this issue > Add

[jira] [Created] (SPARK-39925) Add array_sort(column, comparator) overload to DataFrame operations

Brandon Dahler created SPARK-39925: -- Summary: Add array_sort(column, comparator) overload to DataFrame operations Key: SPARK-39925 URL: https://issues.apache.org/jira/browse/SPARK-39925 Project:

[jira] [Assigned] (SPARK-39924) Extract `constructUnsupportedMergeException` method to deduplicate code in `AccumulatorV2`

[ https://issues.apache.org/jira/browse/SPARK-39924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39924: Assignee: (was: Apache Spark) > Extract `constructUnsupportedMergeException` method

[jira] [Assigned] (SPARK-39924) Extract `constructUnsupportedMergeException` method to deduplicate code in `AccumulatorV2`

[ https://issues.apache.org/jira/browse/SPARK-39924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39924: Assignee: Apache Spark > Extract `constructUnsupportedMergeException` method to

[jira] [Commented] (SPARK-39924) Extract `constructUnsupportedMergeException` method to deduplicate code in `AccumulatorV2`

[ https://issues.apache.org/jira/browse/SPARK-39924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572964#comment-17572964 ] Apache Spark commented on SPARK-39924: -- User 'LuciferYang' has created a pull request for this

[jira] [Created] (SPARK-39924) Extract `constructUnsupportedMergeException` method to deduplicate code in `AccumulatorV2`

Yang Jie created SPARK-39924: Summary: Extract `constructUnsupportedMergeException` method to deduplicate code in `AccumulatorV2` Key: SPARK-39924 URL: https://issues.apache.org/jira/browse/SPARK-39924

[jira] [Resolved] (SPARK-39849) Dataset.as(StructType) fills missing new columns with null value

[ https://issues.apache.org/jira/browse/SPARK-39849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39849. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37264

[jira] [Assigned] (SPARK-39849) Dataset.as(StructType) fills missing new columns with null value

[ https://issues.apache.org/jira/browse/SPARK-39849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39849: --- Assignee: Cheng Su > Dataset.as(StructType) fills missing new columns with null value >

[jira] [Assigned] (SPARK-37194) Avoid unnecessary sort in FileFormatWriter if it's not dynamic partition

[ https://issues.apache.org/jira/browse/SPARK-37194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37194: --- Assignee: XiDuo You > Avoid unnecessary sort in FileFormatWriter if it's not dynamic

[jira] [Resolved] (SPARK-37194) Avoid unnecessary sort in FileFormatWriter if it's not dynamic partition

[ https://issues.apache.org/jira/browse/SPARK-37194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37194. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37290

[jira] [Commented] (SPARK-39923) Put QueryContext to array instead of Option

[ https://issues.apache.org/jira/browse/SPARK-39923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572932#comment-17572932 ] Apache Spark commented on SPARK-39923: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39923) Put QueryContext to array instead of Option

[ https://issues.apache.org/jira/browse/SPARK-39923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39923: Assignee: (was: Apache Spark) > Put QueryContext to array instead of Option >

[jira] [Assigned] (SPARK-39923) Put QueryContext to array instead of Option

[ https://issues.apache.org/jira/browse/SPARK-39923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39923: Assignee: Apache Spark > Put QueryContext to array instead of Option >

[jira] [Created] (SPARK-39923) Put QueryContext to array instead of Option

Max Gekk created SPARK-39923: Summary: Put QueryContext to array instead of Option Key: SPARK-39923 URL: https://issues.apache.org/jira/browse/SPARK-39923 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-39918) Replace the wording "un-comparable" with "incomparable"

[ https://issues.apache.org/jira/browse/SPARK-39918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39918. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37338

[jira] [Commented] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572869#comment-17572869 ] Apache Spark commented on SPARK-39909: -- User 'yabola' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39909: Assignee: Apache Spark > Organize the check of push down information for JDBCV2Suite >

[jira] [Assigned] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39909: Assignee: (was: Apache Spark) > Organize the check of push down information for

[jira] [Commented] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572867#comment-17572867 ] Apache Spark commented on SPARK-39909: -- User 'yabola' has created a pull request for this issue:

[jira] [Commented] (SPARK-38639) Support ignoreCorruptRecord flag to ensure querying broken sequence file table smoothly

[ https://issues.apache.org/jira/browse/SPARK-38639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572858#comment-17572858 ] Apache Spark commented on SPARK-38639: -- User 'caican00' has created a pull request for this issue:

[jira] [Commented] (SPARK-38639) Support ignoreCorruptRecord flag to ensure querying broken sequence file table smoothly

[ https://issues.apache.org/jira/browse/SPARK-38639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572857#comment-17572857 ] Apache Spark commented on SPARK-38639: -- User 'caican00' has created a pull request for this issue:

[jira] [Commented] (SPARK-39922) Add a microbenchmark for `Utils#getIteratorSize`

[ https://issues.apache.org/jira/browse/SPARK-39922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572791#comment-17572791 ] Apache Spark commented on SPARK-39922: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-39922) Add a microbenchmark for `Utils#getIteratorSize`

[ https://issues.apache.org/jira/browse/SPARK-39922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39922: Assignee: (was: Apache Spark) > Add a microbenchmark for `Utils#getIteratorSize` >

[jira] [Assigned] (SPARK-39922) Add a microbenchmark for `Utils#getIteratorSize`

[ https://issues.apache.org/jira/browse/SPARK-39922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39922: Assignee: Apache Spark > Add a microbenchmark for `Utils#getIteratorSize` >

[jira] [Commented] (SPARK-39922) Add a microbenchmark for `Utils#getIteratorSize`

[ https://issues.apache.org/jira/browse/SPARK-39922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572790#comment-17572790 ] Apache Spark commented on SPARK-39922: -- User 'LuciferYang' has created a pull request for this

[jira] [Updated] (SPARK-39921) SkewJoin--Stream side skew in BroadcastJoin

[ https://issues.apache.org/jira/browse/SPARK-39921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wang-zhun updated SPARK-39921: -- Description: There is data skew in BroadcastJoin. Since LocalShuffle cannot cover all cases, we can

[jira] [Updated] (SPARK-39921) SkewJoin--Stream side skew in BroadcastJoin

[ https://issues.apache.org/jira/browse/SPARK-39921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wang-zhun updated SPARK-39921: -- Attachment: 1.png 2.png 3.png 4.png >

[jira] [Created] (SPARK-39922) Add a microbenchmark for `Utils#getIteratorSize`

Yang Jie created SPARK-39922: Summary: Add a microbenchmark for `Utils#getIteratorSize` Key: SPARK-39922 URL: https://issues.apache.org/jira/browse/SPARK-39922 Project: Spark Issue Type:

[jira] [Created] (SPARK-39921) SkewJoin--Stream side skew in BroadcastJoin

wang-zhun created SPARK-39921: - Summary: SkewJoin--Stream side skew in BroadcastJoin Key: SPARK-39921 URL: https://issues.apache.org/jira/browse/SPARK-39921 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-37210) An error occurred while concurrently writing to different static partitions

[ https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572754#comment-17572754 ] Zhen Wang edited comment on SPARK-37210 at 7/29/22 6:27 AM: {quote}I'm not