[jira] [Updated] (SPARK-39920) Interrupt on running spark job doesn't kill underneath spark job

2022-07-28 Thread Neha Singla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Singla updated SPARK-39920: Priority: Blocker (was: Major) > Interrupt on running spark job doesn't kill underneath spark

[jira] [Commented] (SPARK-37210) An error occurred while concurrently writing to different static partitions

2022-07-28 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572754#comment-17572754 ] Zhen Wang commented on SPARK-37210: --- > BTW, could you please reopen the PR freshly? I'll try to ping

[jira] [Updated] (SPARK-39920) Interrupt on running spark job doesn't kill underneath spark job

2022-07-28 Thread Neha Singla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Singla updated SPARK-39920: Attachment: interrupt_spark_job.ipynb > Interrupt on running spark job doesn't kill underneath

[jira] [Created] (SPARK-39920) Interrupt on running spark job doesn't kill underneath spark job

2022-07-28 Thread Neha Singla (Jira)
Neha Singla created SPARK-39920: --- Summary: Interrupt on running spark job doesn't kill underneath spark job Key: SPARK-39920 URL: https://issues.apache.org/jira/browse/SPARK-39920 Project: Spark

[jira] [Created] (SPARK-39919) Hitting Interrupt when no cell is running kills jupyter kernel

2022-07-28 Thread Neha Singla (Jira)
Neha Singla created SPARK-39919: --- Summary: Hitting Interrupt when no cell is running kills jupyter kernel Key: SPARK-39919 URL: https://issues.apache.org/jira/browse/SPARK-39919 Project: Spark

[jira] [Assigned] (SPARK-39918) Replace the wording "un-comparable" with "incomparable"

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39918: Assignee: Apache Spark (was: Gengliang Wang) > Replace the wording "un-comparable" with

[jira] [Commented] (SPARK-39918) Replace the wording "un-comparable" with "incomparable"

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572747#comment-17572747 ] Apache Spark commented on SPARK-39918: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-39918) Replace the wording "un-comparable" with "incomparable"

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39918: Assignee: Gengliang Wang (was: Apache Spark) > Replace the wording "un-comparable" with

[jira] [Created] (SPARK-39918) Replace the wording "un-comparable" with "incomparable"

2022-07-28 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-39918: -- Summary: Replace the wording "un-comparable" with "incomparable" Key: SPARK-39918 URL: https://issues.apache.org/jira/browse/SPARK-39918 Project: Spark

[jira] [Resolved] (SPARK-39905) Remove checkErrorClass()

2022-07-28 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-39905. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37322

[jira] [Assigned] (SPARK-39917) Use different error classes for numeric/interval arithmetic overflow

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39917: Assignee: Apache Spark (was: Gengliang Wang) > Use different error classes for

[jira] [Assigned] (SPARK-39917) Use different error classes for numeric/interval arithmetic overflow

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39917: Assignee: Gengliang Wang (was: Apache Spark) > Use different error classes for

[jira] [Commented] (SPARK-39917) Use different error classes for numeric/interval arithmetic overflow

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572743#comment-17572743 ] Apache Spark commented on SPARK-39917: -- User 'gengliangwang' has created a pull request for this

[jira] [Created] (SPARK-39917) Use different error classes for numeric/interval arithmetic overflow

2022-07-28 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-39917: -- Summary: Use different error classes for numeric/interval arithmetic overflow Key: SPARK-39917 URL: https://issues.apache.org/jira/browse/SPARK-39917 Project:

[jira] [Assigned] (SPARK-39906) Eliminate build warnings - 'sbt 0.13 shell syntax is deprecated; use slash syntax instead'

2022-07-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39906: Assignee: BingKun Pan > Eliminate build warnings - 'sbt 0.13 shell syntax is deprecated;

[jira] [Resolved] (SPARK-39906) Eliminate build warnings - 'sbt 0.13 shell syntax is deprecated; use slash syntax instead'

2022-07-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39906. -- Resolution: Fixed Issue resolved by pull request 37326

[jira] [Assigned] (SPARK-39916) Merge SchemaUtils from mlib to SQL

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39916: Assignee: (was: Apache Spark) > Merge SchemaUtils from mlib to SQL >

[jira] [Commented] (SPARK-39916) Merge SchemaUtils from mlib to SQL

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572726#comment-17572726 ] Apache Spark commented on SPARK-39916: -- User 'deshanxiao' has created a pull request for this

[jira] [Assigned] (SPARK-39916) Merge SchemaUtils from mlib to SQL

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39916: Assignee: Apache Spark > Merge SchemaUtils from mlib to SQL >

[jira] [Updated] (SPARK-39916) Merge SchemaUtils from mlib to SQL

2022-07-28 Thread deshanxiao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] deshanxiao updated SPARK-39916: --- Description: Today we have two SchemaUtils: SQL SchemaUtils and mllib SchemaUtils. the SchemaUtils

[jira] [Updated] (SPARK-39916) Merge SchemaUtils from mlib to SQL

2022-07-28 Thread deshanxiao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] deshanxiao updated SPARK-39916: --- Description: Today we have two SchemaUtils: SQL SchemaUtils and mllib SchemaUtils. the SchemaUtils

[jira] [Created] (SPARK-39916) Merge SchemaUtils from mlib to SQL

2022-07-28 Thread deshanxiao (Jira)
deshanxiao created SPARK-39916: -- Summary: Merge SchemaUtils from mlib to SQL Key: SPARK-39916 URL: https://issues.apache.org/jira/browse/SPARK-39916 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-39862) Fix bug in existence DEFAULT value lookups for V2 data sources

2022-07-28 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-39862: -- Assignee: Daniel > Fix bug in existence DEFAULT value lookups for V2 data sources >

[jira] [Resolved] (SPARK-39862) Fix bug in existence DEFAULT value lookups for V2 data sources

2022-07-28 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-39862. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37280

[jira] [Commented] (SPARK-39915) Dataset.repartition(N) may not create N partitions

2022-07-28 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572685#comment-17572685 ] Shixiong Zhu commented on SPARK-39915: -- cc [~cloud_fan]  > Dataset.repartition(N) may not create N

[jira] [Created] (SPARK-39915) Dataset.repartition(N) may not create N partitions

2022-07-28 Thread Shixiong Zhu (Jira)
Shixiong Zhu created SPARK-39915: Summary: Dataset.repartition(N) may not create N partitions Key: SPARK-39915 URL: https://issues.apache.org/jira/browse/SPARK-39915 Project: Spark Issue

[jira] [Commented] (SPARK-34788) Spark throws FileNotFoundException instead of IOException when disk is full

2022-07-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572634#comment-17572634 ] Dongjoon Hyun commented on SPARK-34788: --- No, what I meant is ensuring enough disk at your

[jira] [Commented] (SPARK-39895) pyspark drop doesn't accept *cols

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572623#comment-17572623 ] Apache Spark commented on SPARK-39895: -- User 'santosh-d3vpl3x' has created a pull request for this

[jira] [Commented] (SPARK-39887) Expression transform error

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572615#comment-17572615 ] Apache Spark commented on SPARK-39887: -- User 'peter-toth' has created a pull request for this

[jira] [Commented] (SPARK-39887) Expression transform error

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572613#comment-17572613 ] Apache Spark commented on SPARK-39887: -- User 'peter-toth' has created a pull request for this

[jira] [Commented] (SPARK-39895) pyspark drop doesn't accept *cols

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572601#comment-17572601 ] Apache Spark commented on SPARK-39895: -- User 'santosh-d3vpl3x' has created a pull request for this

[jira] [Assigned] (SPARK-39895) pyspark drop doesn't accept *cols

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39895: Assignee: Apache Spark > pyspark drop doesn't accept *cols >

[jira] [Assigned] (SPARK-39895) pyspark drop doesn't accept *cols

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39895: Assignee: (was: Apache Spark) > pyspark drop doesn't accept *cols >

[jira] [Commented] (SPARK-39895) pyspark drop doesn't accept *cols

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572600#comment-17572600 ] Apache Spark commented on SPARK-39895: -- User 'santosh-d3vpl3x' has created a pull request for this

[jira] [Resolved] (SPARK-39865) Show proper error messages on the overflow errors of table insert

2022-07-28 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-39865. Fix Version/s: 3.3.1 Resolution: Fixed Issue resolved by pull request 37311

[jira] [Assigned] (SPARK-39914) Add DS V2 Filter to V1 Filter conversion

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39914: Assignee: (was: Apache Spark) > Add DS V2 Filter to V1 Filter conversion >

[jira] [Assigned] (SPARK-39914) Add DS V2 Filter to V1 Filter conversion

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39914: Assignee: Apache Spark > Add DS V2 Filter to V1 Filter conversion >

[jira] [Commented] (SPARK-39914) Add DS V2 Filter to V1 Filter conversion

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572581#comment-17572581 ] Apache Spark commented on SPARK-39914: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Updated] (SPARK-39914) Add DS V2 Filter to V1 Filter conversion

2022-07-28 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-39914: --- Summary: Add DS V2 Filter to V1 Filter conversion (was: Add DS V2 Filter to V2 Filter conversion)

[jira] [Created] (SPARK-39914) Add DS V2 Filter to V2 Filter conversion

2022-07-28 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-39914: -- Summary: Add DS V2 Filter to V2 Filter conversion Key: SPARK-39914 URL: https://issues.apache.org/jira/browse/SPARK-39914 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-39743) Unable to set zstd compression level while writing parquet files

2022-07-28 Thread zhiming she (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572551#comment-17572551 ] zhiming she commented on SPARK-39743: - [~hyukjin.kwon]  As [~yeachan153]  said, is it possible to

[jira] [Commented] (SPARK-39743) Unable to set zstd compression level while writing parquet files

2022-07-28 Thread zhiming she (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572546#comment-17572546 ] zhiming she commented on SPARK-39743: - [~yeachan153]  Only some more important parameters are

[jira] [Commented] (SPARK-34788) Spark throws FileNotFoundException instead of IOException when disk is full

2022-07-28 Thread wenweijian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572536#comment-17572536 ] wenweijian commented on SPARK-34788: I can reproduce it.  If the performance impact of the proposal

[jira] [Assigned] (SPARK-39913) Upgrade Arrow to 9.0.0

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39913: Assignee: (was: Apache Spark) > Upgrade Arrow to 9.0.0 > -- > >

[jira] [Commented] (SPARK-39913) Upgrade Arrow to 9.0.0

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572520#comment-17572520 ] Apache Spark commented on SPARK-39913: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-39913) Upgrade Arrow to 9.0.0

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39913: Assignee: Apache Spark > Upgrade Arrow to 9.0.0 > -- > >

[jira] [Created] (SPARK-39913) Upgrade Arrow to 9.0.0

2022-07-28 Thread Yang Jie (Jira)
Yang Jie created SPARK-39913: Summary: Upgrade Arrow to 9.0.0 Key: SPARK-39913 URL: https://issues.apache.org/jira/browse/SPARK-39913 Project: Spark Issue Type: Improvement Components:

[jira] [Assigned] (SPARK-39912) Refine CatalogImpl

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39912: Assignee: Apache Spark > Refine CatalogImpl > -- > >

[jira] [Assigned] (SPARK-39912) Refine CatalogImpl

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39912: Assignee: (was: Apache Spark) > Refine CatalogImpl > -- > >

[jira] [Commented] (SPARK-39912) Refine CatalogImpl

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572508#comment-17572508 ] Apache Spark commented on SPARK-39912: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Created] (SPARK-39912) Refine CatalogImpl

2022-07-28 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-39912: --- Summary: Refine CatalogImpl Key: SPARK-39912 URL: https://issues.apache.org/jira/browse/SPARK-39912 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-34788) Spark throws FileNotFoundException instead of IOException when disk is full

2022-07-28 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572506#comment-17572506 ] wuyi commented on SPARK-34788: -- > Why don't you ensure enough disks from the beginning? On the write side,

[jira] [Commented] (SPARK-39901) Reconsider design of ignoreCorruptFiles feature

2022-07-28 Thread Wei Guo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572502#comment-17572502 ] Wei Guo commented on SPARK-39901: - The `ignoreCorruptFiles` features in

[jira] [Commented] (SPARK-37210) An error occurred while concurrently writing to different static partitions

2022-07-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572497#comment-17572497 ] Dongjoon Hyun commented on SPARK-37210: --- BTW, could you please reopen the PR freshly? I'll try to

[jira] [Commented] (SPARK-37210) An error occurred while concurrently writing to different static partitions

2022-07-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572495#comment-17572495 ] Dongjoon Hyun commented on SPARK-37210: --- I'm not sure if the concurrent static INSERT INTO

[jira] [Commented] (SPARK-34788) Spark throws FileNotFoundException instead of IOException when disk is full

2022-07-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572493#comment-17572493 ] Dongjoon Hyun commented on SPARK-34788: --- Why don't you ensure enough disks from the beginning?

[jira] [Commented] (SPARK-39911) Optimize global Sort to RepartitionByExpression

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572488#comment-17572488 ] Apache Spark commented on SPARK-39911: -- User 'ulysses-you' has created a pull request for this

[jira] [Commented] (SPARK-10000) Consolidate storage and execution memory management

2022-07-28 Thread sam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572486#comment-17572486 ] sam commented on SPARK-1: - Is there any way to disable Spark from evicting rdd partitions from memory

[jira] [Assigned] (SPARK-39911) Optimize global Sort to RepartitionByExpression

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39911: Assignee: (was: Apache Spark) > Optimize global Sort to RepartitionByExpression >

[jira] [Assigned] (SPARK-39911) Optimize global Sort to RepartitionByExpression

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39911: Assignee: Apache Spark > Optimize global Sort to RepartitionByExpression >

[jira] [Commented] (SPARK-39911) Optimize global Sort to RepartitionByExpression

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572487#comment-17572487 ] Apache Spark commented on SPARK-39911: -- User 'ulysses-you' has created a pull request for this

[jira] [Commented] (SPARK-14289) Support multiple eviction strategies for cached RDD partitions

2022-07-28 Thread sam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-14289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572485#comment-17572485 ] sam commented on SPARK-14289: - Is there any way to disable Spark from evicting rdd partitions from memory

[jira] [Commented] (SPARK-34788) Spark throws FileNotFoundException instead of IOException when disk is full

2022-07-28 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572478#comment-17572478 ] wuyi commented on SPARK-34788: -- [~leo wen] are you able to reproduce in your environment? If yes, we could

[jira] [Created] (SPARK-39911) Optimize global Sort to RepartitionByExpression

2022-07-28 Thread XiDuo You (Jira)
XiDuo You created SPARK-39911: - Summary: Optimize global Sort to RepartitionByExpression Key: SPARK-39911 URL: https://issues.apache.org/jira/browse/SPARK-39911 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-39528) Use V2 Filter in SupportsRuntimeFiltering

2022-07-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39528: --- Assignee: Huaxin Gao > Use V2 Filter in SupportsRuntimeFiltering >

[jira] [Resolved] (SPARK-39528) Use V2 Filter in SupportsRuntimeFiltering

2022-07-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39528. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36918

[jira] [Commented] (SPARK-39910) DataFrameReader API cannot read files from hadoop archives (.har)

2022-07-28 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-39910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572454#comment-17572454 ] Christophe Préaud commented on SPARK-39910: --- This is just a straightforward example to

[jira] [Updated] (SPARK-39910) DataFrameReader API cannot read files from hadoop archives (.har)

2022-07-28 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-39910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christophe Préaud updated SPARK-39910: -- Summary: DataFrameReader API cannot read files from hadoop archives (.har) (was:

[jira] [Updated] (SPARK-39910) DataFrameReader API cannot read files from hadoop archives(.har)

2022-07-28 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-39910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christophe Préaud updated SPARK-39910: -- Description: Reading a file from an hadoop archive using the DataFrameReader API

[jira] [Created] (SPARK-39910) DataFrameReader API cannot read files from hadoop archives(.har)

2022-07-28 Thread Jira
Christophe Préaud created SPARK-39910: - Summary: DataFrameReader API cannot read files from hadoop archives(.har) Key: SPARK-39910 URL: https://issues.apache.org/jira/browse/SPARK-39910 Project:

[jira] [Commented] (SPARK-39832) regexp_replace should support column arguments

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572433#comment-17572433 ] Apache Spark commented on SPARK-39832: -- User 'physinet' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39832) regexp_replace should support column arguments

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39832: Assignee: (was: Apache Spark) > regexp_replace should support column arguments >

[jira] [Commented] (SPARK-39832) regexp_replace should support column arguments

2022-07-28 Thread Brian Schaefer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572432#comment-17572432 ] Brian Schaefer commented on SPARK-39832: Yes, I've started a draft PR here:

[jira] [Assigned] (SPARK-39832) regexp_replace should support column arguments

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39832: Assignee: Apache Spark > regexp_replace should support column arguments >

[jira] [Updated] (SPARK-37210) An error occurred while concurrently writing to different static partitions

2022-07-28 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-37210: -- Description: An error occurred while concurrently writing to different static partitions. For

[jira] [Commented] (SPARK-37210) An error occurred while concurrently writing to different static partitions

2022-07-28 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572408#comment-17572408 ] Zhen Wang commented on SPARK-37210: --- The concurrent writes in INSERT INTO mode seems to have the same

[jira] [Commented] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

2022-07-28 Thread Mars (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572362#comment-17572362 ] Mars commented on SPARK-39909: -- assign [~miracle] > Organize the check of push down information for

[jira] [Updated] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

2022-07-28 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39909: --- Description: Currently, JDBCV2Suite have many test cases check the push-down information looks not

[jira] [Created] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

2022-07-28 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39909: -- Summary: Organize the check of push down information for JDBCV2Suite Key: SPARK-39909 URL: https://issues.apache.org/jira/browse/SPARK-39909 Project: Spark

[jira] [Created] (SPARK-39908) Translate common used aggregate functions for pushdown (non ANSI)

2022-07-28 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39908: -- Summary: Translate common used aggregate functions for pushdown (non ANSI) Key: SPARK-39908 URL: https://issues.apache.org/jira/browse/SPARK-39908 Project: Spark

[jira] [Comment Edited] (SPARK-39743) Unable to set zstd compression level while writing parquet files

2022-07-28 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572308#comment-17572308 ] Yeachan Park edited comment on SPARK-39743 at 7/28/22 8:29 AM: --- OK, thanks

[jira] [Comment Edited] (SPARK-39743) Unable to set zstd compression level while writing parquet files

2022-07-28 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572308#comment-17572308 ] Yeachan Park edited comment on SPARK-39743 at 7/28/22 8:28 AM: --- OK, thanks

[jira] [Commented] (SPARK-39743) Unable to set zstd compression level while writing parquet files

2022-07-28 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572308#comment-17572308 ] Yeachan Park commented on SPARK-39743: -- OK, thanks for the clarification. Should we look at

[jira] [Assigned] (SPARK-39907) Implement axis and skipna of Series.argmin

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39907: Assignee: (was: Apache Spark) > Implement axis and skipna of Series.argmin >

[jira] [Commented] (SPARK-39907) Implement axis and skipna of Series.argmin

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572293#comment-17572293 ] Apache Spark commented on SPARK-39907: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-39907) Implement axis and skipna of Series.argmin

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39907: Assignee: Apache Spark > Implement axis and skipna of Series.argmin >

[jira] [Created] (SPARK-39907) Implement axis and skipna of Series.argmin

2022-07-28 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-39907: - Summary: Implement axis and skipna of Series.argmin Key: SPARK-39907 URL: https://issues.apache.org/jira/browse/SPARK-39907 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-39906) Eliminate build warnings - 'sbt 0.13 shell syntax is deprecated; use slash syntax instead'

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39906: Assignee: Apache Spark > Eliminate build warnings - 'sbt 0.13 shell syntax is

[jira] [Assigned] (SPARK-39906) Eliminate build warnings - 'sbt 0.13 shell syntax is deprecated; use slash syntax instead'

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39906: Assignee: (was: Apache Spark) > Eliminate build warnings - 'sbt 0.13 shell syntax is

[jira] [Commented] (SPARK-39906) Eliminate build warnings - 'sbt 0.13 shell syntax is deprecated; use slash syntax instead'

2022-07-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572248#comment-17572248 ] Apache Spark commented on SPARK-39906: -- User 'panbingkun' has created a pull request for this

[jira] [Created] (SPARK-39906) Eliminate build warnings - 'sbt 0.13 shell syntax is deprecated; use slash syntax instead'

2022-07-28 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-39906: --- Summary: Eliminate build warnings - 'sbt 0.13 shell syntax is deprecated; use slash syntax instead' Key: SPARK-39906 URL: https://issues.apache.org/jira/browse/SPARK-39906