[jira] [Resolved] (SPARK-44920) Use await() instead of awaitUninterruptibly() in TransportClientFactory.createClient()

2023-08-22 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-44920. -- Fix Version/s: 3.3.4 3.5.0 4.0.0 3.4.2

[jira] [Assigned] (SPARK-44925) K8s default service token file should not be materialized into token

2023-08-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44925: - Assignee: Dongjoon Hyun > K8s default service token file should not be materialized

[jira] [Resolved] (SPARK-44925) K8s default service token file should not be materialized into token

2023-08-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44925. --- Fix Version/s: 3.3.4 3.5.0 4.0.0

[jira] [Resolved] (SPARK-44922) Disable o.a.p.h.InternalParquetRecordWriter logs for tests to reduce the log volume

2023-08-22 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-44922. -- Fix Version/s: 3.4.2 3.5.0 Assignee: Kent Yao Resolution: Fixed

[jira] [Resolved] (SPARK-44905) NullPointerException on stateful expression evaluation

2023-08-22 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-44905. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by

[jira] [Updated] (SPARK-44840) array_insert() give wrong results for ngative index

2023-08-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44840: -- Fix Version/s: 3.5.1 > array_insert() give wrong results for ngative index >

[jira] [Commented] (SPARK-44878) Address LRU cache insertion failure for RocksDB with strict limit

2023-08-22 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757801#comment-17757801 ] Snoot.io commented on SPARK-44878: -- User 'anishshri-db' has created a pull request for this issue:

[jira] [Commented] (SPARK-44878) Address LRU cache insertion failure for RocksDB with strict limit

2023-08-22 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757800#comment-17757800 ] Snoot.io commented on SPARK-44878: -- User 'anishshri-db' has created a pull request for this issue:

[jira] [Resolved] (SPARK-44878) Address LRU cache insertion failure for RocksDB with strict limit

2023-08-22 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-44878. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42567

[jira] [Assigned] (SPARK-44878) Address LRU cache insertion failure for RocksDB with strict limit

2023-08-22 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-44878: Assignee: Anish Shrigondekar > Address LRU cache insertion failure for RocksDB with

[jira] [Commented] (SPARK-44750) SparkSession.Builder should respect the options

2023-08-22 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757797#comment-17757797 ] Snoot.io commented on SPARK-44750: -- User 'michaelzhan-db' has created a pull request for this issue:

[jira] [Commented] (SPARK-42017) df["bad_key"] does not raise AnalysisException

2023-08-22 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757794#comment-17757794 ] Snoot.io commented on SPARK-42017: -- User 'zhengruifeng' has created a pull request for this issue:

[jira] [Commented] (SPARK-44903) Refine docstring of `approx_count_distinct`

2023-08-22 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757790#comment-17757790 ] Snoot.io commented on SPARK-44903: -- User 'LuciferYang' has created a pull request for this issue:

[jira] [Commented] (SPARK-44860) Implement SESSION_USER function

2023-08-22 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757788#comment-17757788 ] Snoot.io commented on SPARK-44860: -- User 'vitaliili-db' has created a pull request for this issue:

[jira] [Commented] (SPARK-44913) DS V2 supports push down V2 UDF that has magic method

2023-08-22 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757787#comment-17757787 ] Snoot.io commented on SPARK-44913: -- User 'ConeyLiu' has created a pull request for this issue:

[jira] [Commented] (SPARK-44925) K8s default service token file should not be materialized into token

2023-08-22 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757786#comment-17757786 ] Snoot.io commented on SPARK-44925: -- User 'dongjoon-hyun' has created a pull request for this issue:

[jira] [Commented] (SPARK-44925) K8s default service token file should not be materialized into token

2023-08-22 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757785#comment-17757785 ] Snoot.io commented on SPARK-44925: -- User 'dongjoon-hyun' has created a pull request for this issue:

[jira] [Updated] (SPARK-44923) Some directories should be cleared when regenerating files

2023-08-22 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-44923: Component/s: Build > Some directories should be cleared when regenerating files >

[jira] [Created] (SPARK-44925) K8s default service token file should not be materialized into token

2023-08-22 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-44925: - Summary: K8s default service token file should not be materialized into token Key: SPARK-44925 URL: https://issues.apache.org/jira/browse/SPARK-44925 Project:

[jira] [Resolved] (SPARK-44742) Add Spark version drop down to the PySpark doc site

2023-08-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44742. -- Fix Version/s: 4.0.0 3.5.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-44742) Add Spark version drop down to the PySpark doc site

2023-08-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44742: Assignee: BingKun Pan > Add Spark version drop down to the PySpark doc site >

[jira] [Created] (SPARK-44924) Add configurations for FileStreamSource cached files

2023-08-22 Thread kevin nacios (Jira)
kevin nacios created SPARK-44924: Summary: Add configurations for FileStreamSource cached files Key: SPARK-44924 URL: https://issues.apache.org/jira/browse/SPARK-44924 Project: Spark Issue

[jira] [Created] (SPARK-44923) Some directories should be cleared when regenerating files

2023-08-22 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-44923: --- Summary: Some directories should be cleared when regenerating files Key: SPARK-44923 URL: https://issues.apache.org/jira/browse/SPARK-44923 Project: Spark

[jira] [Created] (SPARK-44922) Disable o.a.p.h.InternalParquetRecordWriter logs for tests to reduce the log volume

2023-08-22 Thread Kent Yao (Jira)
Kent Yao created SPARK-44922: Summary: Disable o.a.p.h.InternalParquetRecordWriter logs for tests to reduce the log volume Key: SPARK-44922 URL: https://issues.apache.org/jira/browse/SPARK-44922 Project:

[jira] [Created] (SPARK-44921) Remove SqlBaseLexer.tokens from codebase

2023-08-22 Thread Rui Wang (Jira)
Rui Wang created SPARK-44921: Summary: Remove SqlBaseLexer.tokens from codebase Key: SPARK-44921 URL: https://issues.apache.org/jira/browse/SPARK-44921 Project: Spark Issue Type: Task

[jira] [Assigned] (SPARK-44920) Use await() instead of awaitUninterruptibly() in TransportClientFactory.createClient()

2023-08-22 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-44920: -- Assignee: Josh Rosen > Use await() instead of awaitUninterruptibly() in >

[jira] [Created] (SPARK-44920) Use await() instead of awaitUninterruptibly() in TransportClientFactory.createClient()

2023-08-22 Thread Josh Rosen (Jira)
Josh Rosen created SPARK-44920: -- Summary: Use await() instead of awaitUninterruptibly() in TransportClientFactory.createClient() Key: SPARK-44920 URL: https://issues.apache.org/jira/browse/SPARK-44920

[jira] [Assigned] (SPARK-44907) `DataFrame.join` should throw IllegalArgumentException for invalid join types

2023-08-22 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-44907: - Assignee: Ruifeng Zheng > `DataFrame.join` should throw IllegalArgumentException for

[jira] [Resolved] (SPARK-44907) `DataFrame.join` should throw IllegalArgumentException for invalid join types

2023-08-22 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-44907. --- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by

[jira] [Created] (SPARK-44919) Avro connector: convert a union of a single primitive type to a StructType

2023-08-22 Thread Tianhan Hu (Jira)
Tianhan Hu created SPARK-44919: -- Summary: Avro connector: convert a union of a single primitive type to a StructType Key: SPARK-44919 URL: https://issues.apache.org/jira/browse/SPARK-44919 Project:

[jira] [Commented] (SPARK-44857) Fix getBaseURI error in Spark Worker LogPage UI buttons

2023-08-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757691#comment-17757691 ] Dongjoon Hyun commented on SPARK-44857: --- Thank you for fixing the `Fix Version`, [~yumwang]. >

[jira] [Created] (SPARK-44918) Add named argument support for scalar Python/Pandas UDFs

2023-08-22 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-44918: - Summary: Add named argument support for scalar Python/Pandas UDFs Key: SPARK-44918 URL: https://issues.apache.org/jira/browse/SPARK-44918 Project: Spark

[jira] [Resolved] (SPARK-44916) Document Spark Driver Live Log UI

2023-08-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44916. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42615

[jira] [Assigned] (SPARK-44916) Document Spark Driver Live Log UI

2023-08-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44916: - Assignee: Dongjoon Hyun > Document Spark Driver Live Log UI >

[jira] [Resolved] (SPARK-44917) PySpark Streaming DataStreamWriter table API

2023-08-22 Thread Wei Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Liu resolved SPARK-44917. - Resolution: Not A Problem > PySpark Streaming DataStreamWriter table API >

[jira] [Updated] (SPARK-44914) Upgrade Apache ivy to 2.5.2

2023-08-22 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-44914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-44914: Affects Version/s: 3.5.0 > Upgrade Apache ivy to 2.5.2 > >

[jira] [Created] (SPARK-44917) PySpark Streaming DataStreamWriter table API

2023-08-22 Thread Wei Liu (Jira)
Wei Liu created SPARK-44917: --- Summary: PySpark Streaming DataStreamWriter table API Key: SPARK-44917 URL: https://issues.apache.org/jira/browse/SPARK-44917 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-44914) Upgrade Apache ivy to 2.5.2

2023-08-22 Thread Hudson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757647#comment-17757647 ] Hudson commented on SPARK-44914: User 'bjornjorgensen' has created a pull request for this issue:

[jira] [Resolved] (SPARK-44840) array_insert() give wrong results for ngative index

2023-08-22 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-44840. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42564

[jira] [Created] (SPARK-44916) Document Spark Driver Live Log UI

2023-08-22 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-44916: - Summary: Document Spark Driver Live Log UI Key: SPARK-44916 URL: https://issues.apache.org/jira/browse/SPARK-44916 Project: Spark Issue Type:

[jira] [Updated] (SPARK-44915) Validate checksum of remounted PVC's shuffle data before recovery

2023-08-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44915: -- Component/s: (was: Spark Core) > Validate checksum of remounted PVC's shuffle data before

[jira] [Created] (SPARK-44915) Validate checksum of remounted PVC's shuffle data before recovery

2023-08-22 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-44915: - Summary: Validate checksum of remounted PVC's shuffle data before recovery Key: SPARK-44915 URL: https://issues.apache.org/jira/browse/SPARK-44915 Project: Spark

[jira] [Assigned] (SPARK-44871) Fix PERCENTILE_DISC behaviour

2023-08-22 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-44871: Assignee: Peter Toth > Fix PERCENTILE_DISC behaviour > - > >

[jira] [Resolved] (SPARK-44871) Fix PERCENTILE_DISC behaviour

2023-08-22 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-44871. -- Fix Version/s: 3.4.2 (was: 3.5.0) (was: 4.0.0)

[jira] [Created] (SPARK-44914) Upgrade Apache ivy to 2.5.2

2023-08-22 Thread Jira
Bjørn Jørgensen created SPARK-44914: --- Summary: Upgrade Apache ivy to 2.5.2 Key: SPARK-44914 URL: https://issues.apache.org/jira/browse/SPARK-44914 Project: Spark Issue Type: Dependency

[jira] [Commented] (SPARK-44884) Spark doesn't create SUCCESS file when external path is passed

2023-08-22 Thread Dipayan Dev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757550#comment-17757550 ] Dipayan Dev commented on SPARK-44884: - [~ste...@apache.org] , I am running on DataProc but I am able

[jira] [Commented] (SPARK-44884) Spark doesn't create SUCCESS file when external path is passed

2023-08-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757547#comment-17757547 ] Steve Loughran commented on SPARK-44884: [~dipayandev] i don't think think anyone has disabled

[jira] [Commented] (SPARK-44817) Incremental Stats Collection

2023-08-22 Thread Rakesh Raushan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757539#comment-17757539 ] Rakesh Raushan commented on SPARK-44817: Sure. I would try to come up with a SPIP by this

[jira] [Updated] (SPARK-44871) Fix PERCENTILE_DISC behaviour

2023-08-22 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-44871: --- Affects Version/s: 3.4.0 3.3.2 3.3.1

[jira] [Updated] (SPARK-44871) Fix PERCENTILE_DISC behaviour

2023-08-22 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-44871: --- Affects Version/s: 3.4.1 3.3.3 (was: 3.3.0)

[jira] [Updated] (SPARK-44871) Fix PERCENTILE_DISC behaviour

2023-08-22 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-44871: --- Fix Version/s: 3.5.0 4.0.0 > Fix PERCENTILE_DISC behaviour >

[jira] [Created] (SPARK-44913) DS V2 supports push down V2 UDF that has magic method

2023-08-22 Thread Xianyang Liu (Jira)
Xianyang Liu created SPARK-44913: Summary: DS V2 supports push down V2 UDF that has magic method Key: SPARK-44913 URL: https://issues.apache.org/jira/browse/SPARK-44913 Project: Spark Issue

[jira] [Created] (SPARK-44912) Spark 3.4 multi-column sum slows with many columns

2023-08-22 Thread Brady Bickel (Jira)
Brady Bickel created SPARK-44912: Summary: Spark 3.4 multi-column sum slows with many columns Key: SPARK-44912 URL: https://issues.apache.org/jira/browse/SPARK-44912 Project: Spark Issue

[jira] [Created] (SPARK-44911) create hive table with invalid column should return error class

2023-08-22 Thread zzzzming95 (Jira)
ming95 created SPARK-44911: -- Summary: create hive table with invalid column should return error class Key: SPARK-44911 URL: https://issues.apache.org/jira/browse/SPARK-44911 Project: Spark

[jira] [Resolved] (SPARK-40156) url_decode() exposes a Java error

2023-08-22 Thread zzzzming95 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ming95 resolved SPARK-40156. Fix Version/s: 3.4.0 Resolution: Fixed > url_decode() exposes a Java error >

[jira] [Created] (SPARK-44910) Encoders.bean does not support superclasses with generic type arguments

2023-08-22 Thread Giambattista Bloisi (Jira)
Giambattista Bloisi created SPARK-44910: --- Summary: Encoders.bean does not support superclasses with generic type arguments Key: SPARK-44910 URL: https://issues.apache.org/jira/browse/SPARK-44910

[jira] [Assigned] (SPARK-44885) NullPointerException is thrown when column with ROWID type contains NULL values

2023-08-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44885: Assignee: Tim Nieradzik > NullPointerException is thrown when column with ROWID type

[jira] [Resolved] (SPARK-44885) NullPointerException is thrown when column with ROWID type contains NULL values

2023-08-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44885. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Updated] (SPARK-44786) Convert common Spark exceptions

2023-08-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-44786: - Fix Version/s: 3.5.0 > Convert common Spark exceptions > --- > >

[jira] [Resolved] (SPARK-44786) Convert common Spark exceptions

2023-08-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44786. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42472

[jira] [Assigned] (SPARK-44786) Convert common Spark exceptions

2023-08-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44786: Assignee: Yihong He > Convert common Spark exceptions > ---

[jira] [Commented] (SPARK-38958) Override S3 Client in Spark Write/Read calls

2023-08-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757364#comment-17757364 ] Steve Loughran commented on SPARK-38958: [~hershalb] we are about to merge the v2 sdk feature

[jira] [Created] (SPARK-44909) Skip starting torch distributor log streaming server when it is not available

2023-08-22 Thread Weichen Xu (Jira)
Weichen Xu created SPARK-44909: -- Summary: Skip starting torch distributor log streaming server when it is not available Key: SPARK-44909 URL: https://issues.apache.org/jira/browse/SPARK-44909 Project:

[jira] [Commented] (SPARK-44905) NullPointerException on stateful expression evaluation

2023-08-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757297#comment-17757297 ] ASF GitHub Bot commented on SPARK-44905: User 'yaooqinn' has created a pull request for this

[jira] [Commented] (SPARK-26398) Support building GPU docker images

2023-08-22 Thread comet (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757293#comment-17757293 ] comet commented on SPARK-26398: --- I see #23347 is closed without merge. Does that mean support for GPU is

[jira] [Commented] (SPARK-44908) Fix spark connect ML crossvalidator "foldCol" param

2023-08-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757292#comment-17757292 ] ASF GitHub Bot commented on SPARK-44908: User 'WeichenXu123' has created a pull request for this

[jira] [Created] (SPARK-44908) Fix spark connect ML crossvalidator "foldCol" param

2023-08-22 Thread Weichen Xu (Jira)
Weichen Xu created SPARK-44908: -- Summary: Fix spark connect ML crossvalidator "foldCol" param Key: SPARK-44908 URL: https://issues.apache.org/jira/browse/SPARK-44908 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-44908) Fix spark connect ML crossvalidator "foldCol" param

2023-08-22 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu reassigned SPARK-44908: -- Assignee: Weichen Xu > Fix spark connect ML crossvalidator "foldCol" param >

[jira] [Commented] (SPARK-44906) Move substituteAppNExecIds logic into kubernetesConf.annotations method

2023-08-22 Thread GridGain Integration (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757238#comment-17757238 ] GridGain Integration commented on SPARK-44906: -- User 'zwangsheng' has created a pull

[jira] [Resolved] (SPARK-42768) Enable cached plan apply AQE by default

2023-08-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-42768. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40390

[jira] [Assigned] (SPARK-42768) Enable cached plan apply AQE by default

2023-08-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-42768: --- Assignee: XiDuo You > Enable cached plan apply AQE by default >

[jira] [Assigned] (SPARK-44905) NullPointerException on stateful expression evaluation

2023-08-22 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-44905: Assignee: Kent Yao > NullPointerException on stateful expression evaluation >

[jira] [Updated] (SPARK-44907) `DataFrame.join` should throw IllegalArgumentException for invalid join types

2023-08-22 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-44907: -- Component/s: PySpark (was: Tests) > `DataFrame.join` should throw

[jira] [Created] (SPARK-44907) `DataFrame.join` should throw IllegalArgumentException for invalid join types

2023-08-22 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-44907: - Summary: `DataFrame.join` should throw IllegalArgumentException for invalid join types Key: SPARK-44907 URL: https://issues.apache.org/jira/browse/SPARK-44907

[jira] [Updated] (SPARK-44892) Add official image Dockerfile for Spark 3.3.3

2023-08-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44892: Fix Version/s: (was: 4.0.0) > Add official image Dockerfile for Spark 3.3.3 >

[jira] [Resolved] (SPARK-44892) Add official image Dockerfile for Spark 3.3.3

2023-08-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-44892. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 54

[jira] [Assigned] (SPARK-44892) Add official image Dockerfile for Spark 3.3.3

2023-08-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-44892: --- Assignee: Yuming Wang > Add official image Dockerfile for Spark 3.3.3 >

[jira] [Comment Edited] (SPARK-44890) Miswritten remarks in pom file

2023-08-22 Thread chenyu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756746#comment-17756746 ] chenyu edited comment on SPARK-44890 at 8/22/23 7:11 AM: - I had submit a patch 

[jira] [Comment Edited] (SPARK-44728) Improve PySpark documentations

2023-08-22 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757209#comment-17757209 ] Ruifeng Zheng edited comment on SPARK-44728 at 8/22/23 6:20 AM: A

[jira] [Comment Edited] (SPARK-44728) Improve PySpark documentations

2023-08-22 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757209#comment-17757209 ] Ruifeng Zheng edited comment on SPARK-44728 at 8/22/23 6:10 AM: A

[jira] [Assigned] (SPARK-44904) Correct the ‘versionadded’ of `sql.functions.approx_percentile` to 3.5.0.

2023-08-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44904: Assignee: Yang Jie > Correct the ‘versionadded’ of `sql.functions.approx_percentile` to

[jira] [Resolved] (SPARK-44904) Correct the ‘versionadded’ of `sql.functions.approx_percentile` to 3.5.0.

2023-08-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44904. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Comment Edited] (SPARK-44728) Improve PySpark documentations

2023-08-22 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757209#comment-17757209 ] Ruifeng Zheng edited comment on SPARK-44728 at 8/22/23 6:01 AM: A good

[jira] [Comment Edited] (SPARK-44728) Improve PySpark documentations

2023-08-22 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757209#comment-17757209 ] Ruifeng Zheng edited comment on SPARK-44728 at 8/22/23 5:59 AM: A good

[jira] [Comment Edited] (SPARK-44728) Improve PySpark documentations

2023-08-22 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757209#comment-17757209 ] Ruifeng Zheng edited comment on SPARK-44728 at 8/22/23 5:59 AM: A good