[jira] [Created] (SPARK-38800) Explicitly document the supported pandas version.

Haejoon Lee created SPARK-38800: --- Summary: Explicitly document the supported pandas version. Key: SPARK-38800 URL: https://issues.apache.org/jira/browse/SPARK-38800 Project: Spark Issue Type: T

[jira] [Commented] (SPARK-38800) Explicitly document the supported pandas version.

[ https://issues.apache.org/jira/browse/SPARK-38800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517830#comment-17517830 ] Haejoon Lee commented on SPARK-38800: - I'm working on this > Explicitly document th

[jira] [Commented] (SPARK-38581) List of supported pandas APIs for pandas API on Spark docs.

[ https://issues.apache.org/jira/browse/SPARK-38581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517829#comment-17517829 ] Apache Spark commented on SPARK-38581: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-38581) List of supported pandas APIs for pandas API on Spark docs.

[ https://issues.apache.org/jira/browse/SPARK-38581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38581: Assignee: Apache Spark > List of supported pandas APIs for pandas API on Spark docs. > --

[jira] [Commented] (SPARK-38581) List of supported pandas APIs for pandas API on Spark docs.

[ https://issues.apache.org/jira/browse/SPARK-38581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517827#comment-17517827 ] Apache Spark commented on SPARK-38581: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-38581) List of supported pandas APIs for pandas API on Spark docs.

[ https://issues.apache.org/jira/browse/SPARK-38581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38581: Assignee: (was: Apache Spark) > List of supported pandas APIs for pandas API on Spark

[jira] [Assigned] (SPARK-38760) Implement DataFrame.observe in PySpark

[ https://issues.apache.org/jira/browse/SPARK-38760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38760: Assignee: (was: Apache Spark) > Implement DataFrame.observe in PySpark >

[jira] [Commented] (SPARK-38760) Implement DataFrame.observe in PySpark

[ https://issues.apache.org/jira/browse/SPARK-38760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517826#comment-17517826 ] Apache Spark commented on SPARK-38760: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-38760) Implement DataFrame.observe in PySpark

[ https://issues.apache.org/jira/browse/SPARK-38760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38760: Assignee: Apache Spark > Implement DataFrame.observe in PySpark > ---

[jira] [Commented] (SPARK-38799) Fix scala license declaration

[ https://issues.apache.org/jira/browse/SPARK-38799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517814#comment-17517814 ] Apache Spark commented on SPARK-38799: -- User 'yaooqinn' has created a pull request

[jira] [Assigned] (SPARK-38799) Fix scala license declaration

[ https://issues.apache.org/jira/browse/SPARK-38799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38799: Assignee: (was: Apache Spark) > Fix scala license declaration > -

[jira] [Commented] (SPARK-38799) Fix scala license declaration

[ https://issues.apache.org/jira/browse/SPARK-38799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517812#comment-17517812 ] Apache Spark commented on SPARK-38799: -- User 'yaooqinn' has created a pull request

[jira] [Assigned] (SPARK-38799) Fix scala license declaration

[ https://issues.apache.org/jira/browse/SPARK-38799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38799: Assignee: Apache Spark > Fix scala license declaration > - >

[jira] [Created] (SPARK-38799) Fix scala license declaration

Kent Yao created SPARK-38799: Summary: Fix scala license declaration Key: SPARK-38799 URL: https://issues.apache.org/jira/browse/SPARK-38799 Project: Spark Issue Type: Bug Components: P

[jira] [Commented] (SPARK-38797) Runtime Filter support pushdown through window

[ https://issues.apache.org/jira/browse/SPARK-38797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517798#comment-17517798 ] Apache Spark commented on SPARK-38797: -- User 'wangyum' has created a pull request f

[jira] [Assigned] (SPARK-38797) Runtime Filter support pushdown through window

[ https://issues.apache.org/jira/browse/SPARK-38797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38797: Assignee: (was: Apache Spark) > Runtime Filter support pushdown through window >

[jira] [Assigned] (SPARK-38797) Runtime Filter support pushdown through window

[ https://issues.apache.org/jira/browse/SPARK-38797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38797: Assignee: Apache Spark > Runtime Filter support pushdown through window > ---

[jira] [Assigned] (SPARK-38798) Make `spark.file.transferTo` as an `ConfigEntry`

[ https://issues.apache.org/jira/browse/SPARK-38798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38798: Assignee: (was: Apache Spark) > Make `spark.file.transferTo` as an `ConfigEntry` > --

[jira] [Assigned] (SPARK-38798) Make `spark.file.transferTo` as an `ConfigEntry`

[ https://issues.apache.org/jira/browse/SPARK-38798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38798: Assignee: Apache Spark > Make `spark.file.transferTo` as an `ConfigEntry` > -

[jira] [Commented] (SPARK-38798) Make `spark.file.transferTo` as an `ConfigEntry`

[ https://issues.apache.org/jira/browse/SPARK-38798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517786#comment-17517786 ] Apache Spark commented on SPARK-38798: -- User 'LuciferYang' has created a pull reque

[jira] [Created] (SPARK-38798) Make `spark.file.transferTo` as an `ConfigEntry`

Yang Jie created SPARK-38798: Summary: Make `spark.file.transferTo` as an `ConfigEntry` Key: SPARK-38798 URL: https://issues.apache.org/jira/browse/SPARK-38798 Project: Spark Issue Type: Improvem

[jira] [Created] (SPARK-38797) Runtime Filter support pushdown through window

Yuming Wang created SPARK-38797: --- Summary: Runtime Filter support pushdown through window Key: SPARK-38797 URL: https://issues.apache.org/jira/browse/SPARK-38797 Project: Spark Issue Type: Impr

[jira] [Commented] (SPARK-38796) Implement the to_number and try_to_number SQL functions according to a new specification

[ https://issues.apache.org/jira/browse/SPARK-38796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517774#comment-17517774 ] Apache Spark commented on SPARK-38796: -- User 'dtenedor' has created a pull request

[jira] [Assigned] (SPARK-38796) Implement the to_number and try_to_number SQL functions according to a new specification

[ https://issues.apache.org/jira/browse/SPARK-38796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38796: Assignee: Apache Spark > Implement the to_number and try_to_number SQL functions accordin

[jira] [Assigned] (SPARK-38796) Implement the to_number and try_to_number SQL functions according to a new specification

[ https://issues.apache.org/jira/browse/SPARK-38796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38796: Assignee: (was: Apache Spark) > Implement the to_number and try_to_number SQL functio

[jira] [Updated] (SPARK-38796) Implement the to_number and try_to_number SQL functions according to a new specification

[ https://issues.apache.org/jira/browse/SPARK-38796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-38796: --- Description: This tracks implementing the 'to_number' and 'try_to_number' SQL function expressions accordin

[jira] [Updated] (SPARK-38796) Implement the to_number and try_to_number SQL functions according to a new specification

[ https://issues.apache.org/jira/browse/SPARK-38796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-38796: --- Description: This tracks implementing the 'to_number' and 'try_to_number' SQL function expressions accordin

[jira] [Updated] (SPARK-38796) Implement the to_number and try_to_number SQL functions according to a new specification

[ https://issues.apache.org/jira/browse/SPARK-38796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-38796: --- Description: This tracks implementing the 'to_number' and 'try_to_number' SQL function expressions accordin

[jira] [Updated] (SPARK-38796) Implement the to_number and try_to_number SQL functions according to a new specification

[ https://issues.apache.org/jira/browse/SPARK-38796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-38796: --- Description: This tracks implementing the 'to_number' and 'try_to_number' SQL function expressions accordin

[jira] [Updated] (SPARK-38796) Implement the to_number and try_to_number SQL functions according to a new specification

[ https://issues.apache.org/jira/browse/SPARK-38796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-38796: --- Description: This tracks implementing the `to_number` and `try_to_number` SQL function expressions accordin

[jira] [Updated] (SPARK-38796) Implement the to_number and try_to_number SQL functions according to a new specification

[ https://issues.apache.org/jira/browse/SPARK-38796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-38796: --- Description: This tracks implementing the `to_number` and `try_to_number` SQL function expressions accordin

[jira] [Updated] (SPARK-38796) Implement the to_number and try_to_number SQL functions according to a new specification

[ https://issues.apache.org/jira/browse/SPARK-38796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-38796: --- Description: This tracks implementing the `to_number` and `try_to_number` SQL function expressions accordin

[jira] [Created] (SPARK-38796) Implement the to_number and try_to_number SQL functions according to a new specification

Daniel created SPARK-38796: -- Summary: Implement the to_number and try_to_number SQL functions according to a new specification Key: SPARK-38796 URL: https://issues.apache.org/jira/browse/SPARK-38796 Project:

[jira] [Resolved] (SPARK-38786) Test Bug in StatisticsSuite "change stats after add/drop partition command"

[ https://issues.apache.org/jira/browse/SPARK-38786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-38786. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36075 [https://github.com

[jira] [Assigned] (SPARK-38786) Test Bug in StatisticsSuite "change stats after add/drop partition command"

[ https://issues.apache.org/jira/browse/SPARK-38786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-38786: Assignee: Kazuyuki Tanimura > Test Bug in StatisticsSuite "change stats after add/drop partition

[jira] [Commented] (SPARK-38795) Support INSERT INTO user specified column lists with DEFAULT values

[ https://issues.apache.org/jira/browse/SPARK-38795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517754#comment-17517754 ] Apache Spark commented on SPARK-38795: -- User 'dtenedor' has created a pull request

[jira] [Commented] (SPARK-38795) Support INSERT INTO user specified column lists with DEFAULT values

[ https://issues.apache.org/jira/browse/SPARK-38795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517755#comment-17517755 ] Apache Spark commented on SPARK-38795: -- User 'dtenedor' has created a pull request

[jira] [Assigned] (SPARK-38795) Support INSERT INTO user specified column lists with DEFAULT values

[ https://issues.apache.org/jira/browse/SPARK-38795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38795: Assignee: (was: Apache Spark) > Support INSERT INTO user specified column lists with

[jira] [Assigned] (SPARK-38795) Support INSERT INTO user specified column lists with DEFAULT values

[ https://issues.apache.org/jira/browse/SPARK-38795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38795: Assignee: Apache Spark > Support INSERT INTO user specified column lists with DEFAULT val

[jira] [Updated] (SPARK-38795) Support INSERT INTO user specified column lists with DEFAULT values

[ https://issues.apache.org/jira/browse/SPARK-38795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-38795: --- Shepherd: Gengliang Wang > Support INSERT INTO user specified column lists with DEFAULT values > ---

[jira] [Created] (SPARK-38795) Support INSERT INTO user specified column lists with DEFAULT values

Daniel created SPARK-38795: -- Summary: Support INSERT INTO user specified column lists with DEFAULT values Key: SPARK-38795 URL: https://issues.apache.org/jira/browse/SPARK-38795 Project: Spark Issu

[jira] [Commented] (SPARK-38637) pyspark.pandas.config.OptionError: "No such option: 'mode.chained_assignment'

[ https://issues.apache.org/jira/browse/SPARK-38637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517745#comment-17517745 ] Haejoon Lee commented on SPARK-38637: - [~pralabhkumar] Sure, pleas go ahead! > pysp

[jira] [Updated] (SPARK-38794) When ConfigMap creation fails, Spark driver starts but fails to start executors

[ https://issues.apache.org/jira/browse/SPARK-38794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Horrigan updated SPARK-38794: -- Description: When running Spark in Kubernetes client mode, all executors assume that a Config

[jira] [Created] (SPARK-38794) When ConfigMap creation fails, Spark driver starts but fails to start executors

Seth Horrigan created SPARK-38794: - Summary: When ConfigMap creation fails, Spark driver starts but fails to start executors Key: SPARK-38794 URL: https://issues.apache.org/jira/browse/SPARK-38794 Pro

[jira] [Commented] (SPARK-38793) Support `return_indexer` parameter of `Index/MultiIndex.sort_values`

[ https://issues.apache.org/jira/browse/SPARK-38793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517721#comment-17517721 ] Apache Spark commented on SPARK-38793: -- User 'xinrong-databricks' has created a pul

[jira] [Assigned] (SPARK-38793) Support `return_indexer` parameter of `Index/MultiIndex.sort_values`

[ https://issues.apache.org/jira/browse/SPARK-38793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38793: Assignee: Apache Spark > Support `return_indexer` parameter of `Index/MultiIndex.sort_val

[jira] [Assigned] (SPARK-38793) Support `return_indexer` parameter of `Index/MultiIndex.sort_values`

[ https://issues.apache.org/jira/browse/SPARK-38793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38793: Assignee: (was: Apache Spark) > Support `return_indexer` parameter of `Index/MultiInd

[jira] [Updated] (SPARK-38648) SPIP: Simplified API for DL Inferencing

[ https://issues.apache.org/jira/browse/SPARK-38648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lee Yang updated SPARK-38648: - Description: h1. Background and Motivation The deployment of deep learning (DL) models to Spark cluster

[jira] [Created] (SPARK-38793) Support `return_indexer` parameter of `Index/MultiIndex.sort_values`

Xinrong Meng created SPARK-38793: Summary: Support `return_indexer` parameter of `Index/MultiIndex.sort_values` Key: SPARK-38793 URL: https://issues.apache.org/jira/browse/SPARK-38793 Project: Spark

[jira] [Commented] (SPARK-38763) Pandas API on spark Can`t apply lamda to columns.

[ https://issues.apache.org/jira/browse/SPARK-38763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517719#comment-17517719 ] Xinrong Meng commented on SPARK-38763: -- [~bjornjorgensen] For sure :) The fix is in

[jira] [Updated] (SPARK-38648) SPIP: Simplified API for DL Inferencing

[ https://issues.apache.org/jira/browse/SPARK-38648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lee Yang updated SPARK-38648: - Description: h1. Background and Motivation The deployment of deep learning (DL) models to Spark cluster

[jira] [Assigned] (SPARK-38786) Test Bug in StatisticsSuite "change stats after add/drop partition command"

[ https://issues.apache.org/jira/browse/SPARK-38786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38786: Assignee: (was: Apache Spark) > Test Bug in StatisticsSuite "change stats after add/d

[jira] [Assigned] (SPARK-38786) Test Bug in StatisticsSuite "change stats after add/drop partition command"

[ https://issues.apache.org/jira/browse/SPARK-38786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38786: Assignee: Apache Spark > Test Bug in StatisticsSuite "change stats after add/drop partiti

[jira] [Commented] (SPARK-38786) Test Bug in StatisticsSuite "change stats after add/drop partition command"

[ https://issues.apache.org/jira/browse/SPARK-38786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517710#comment-17517710 ] Apache Spark commented on SPARK-38786: -- User 'kazuyukitanimura' has created a pull

[jira] [Updated] (SPARK-38792) Regression in time executor takes to do work since v3.0.1 ?

[ https://issues.apache.org/jira/browse/SPARK-38792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Guinther updated SPARK-38792: --- Description: Hello! I'm sorry to trouble you with this, but I'm seeing a noticeable regress

[jira] [Comment Edited] (SPARK-38792) Regression in time executor takes to do work since v3.0.1 ?

[ https://issues.apache.org/jira/browse/SPARK-38792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517704#comment-17517704 ] Danny Guinther edited comment on SPARK-38792 at 4/5/22 9:09 PM: --

[jira] [Commented] (SPARK-38792) Regression in time executor takes to do work since v3.0.1 ?

[ https://issues.apache.org/jira/browse/SPARK-38792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517704#comment-17517704 ] Danny Guinther commented on SPARK-38792: I don't know if it is helpful, but the

[jira] [Updated] (SPARK-38792) Regression in time executor takes to do work since v3.0.1 ?

[ https://issues.apache.org/jira/browse/SPARK-38792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Guinther updated SPARK-38792: --- Description: Hello! I'm sorry to trouble you with this, but I'm seeing a noticeable regress

[jira] [Updated] (SPARK-38792) Regression in time executor takes to do work since v3.0.1 ?

[ https://issues.apache.org/jira/browse/SPARK-38792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Guinther updated SPARK-38792: --- Attachment: what-s-up-with-exec-actions.jpg > Regression in time executor takes to do work s

[jira] [Created] (SPARK-38792) Regression in time executor takes to do work since v3.0.1 ?

Danny Guinther created SPARK-38792: -- Summary: Regression in time executor takes to do work since v3.0.1 ? Key: SPARK-38792 URL: https://issues.apache.org/jira/browse/SPARK-38792 Project: Spark

[jira] [Assigned] (SPARK-38791) Output parameter values of error classes in SQL style

[ https://issues.apache.org/jira/browse/SPARK-38791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38791: Assignee: Max Gekk (was: Apache Spark) > Output parameter values of error classes in SQL

[jira] [Commented] (SPARK-38791) Output parameter values of error classes in SQL style

[ https://issues.apache.org/jira/browse/SPARK-38791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517685#comment-17517685 ] Apache Spark commented on SPARK-38791: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-38791) Output parameter values of error classes in SQL style

[ https://issues.apache.org/jira/browse/SPARK-38791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38791: Assignee: Apache Spark (was: Max Gekk) > Output parameter values of error classes in SQL

[jira] [Created] (SPARK-38791) Output parameter values of error classes in SQL style

Max Gekk created SPARK-38791: Summary: Output parameter values of error classes in SQL style Key: SPARK-38791 URL: https://issues.apache.org/jira/browse/SPARK-38791 Project: Spark Issue Type: Sub

[jira] [Comment Edited] (SPARK-38790) Unexpected behaviour for "IN" operator in spark when null is involved in array

[ https://issues.apache.org/jira/browse/SPARK-38790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517674#comment-17517674 ] Bruce Robbins edited comment on SPARK-38790 at 4/5/22 7:50 PM: ---

[jira] [Commented] (SPARK-38790) Unexpected behaviour for "IN" operator in spark when null is involved in array

[ https://issues.apache.org/jira/browse/SPARK-38790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517674#comment-17517674 ] Bruce Robbins commented on SPARK-38790: --- I think that might be correct. At least,

[jira] [Assigned] (SPARK-38787) Possible correctness issue on stream-stream join when handling edge case

[ https://issues.apache.org/jira/browse/SPARK-38787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38787: Assignee: (was: Apache Spark) > Possible correctness issue on stream-stream join when

[jira] [Commented] (SPARK-38787) Possible correctness issue on stream-stream join when handling edge case

[ https://issues.apache.org/jira/browse/SPARK-38787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517659#comment-17517659 ] Apache Spark commented on SPARK-38787: -- User 'anishshri-db' has created a pull requ

[jira] [Assigned] (SPARK-38787) Possible correctness issue on stream-stream join when handling edge case

[ https://issues.apache.org/jira/browse/SPARK-38787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38787: Assignee: Apache Spark > Possible correctness issue on stream-stream join when handling e

[jira] [Comment Edited] (SPARK-38445) Are hadoop committers used in Structured Streaming?

[ https://issues.apache.org/jira/browse/SPARK-38445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517653#comment-17517653 ] Martin Andersson edited comment on SPARK-38445 at 4/5/22 7:04 PM:

[jira] [Commented] (SPARK-38445) Are hadoop committers used in Structured Streaming?

[ https://issues.apache.org/jira/browse/SPARK-38445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517653#comment-17517653 ] Martin Andersson commented on SPARK-38445: -- {quote}not suppoorted unless you pr

[jira] [Assigned] (SPARK-38666) Missing aggregate filter checks

[ https://issues.apache.org/jira/browse/SPARK-38666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38666: Assignee: (was: Apache Spark) > Missing aggregate filter checks > ---

[jira] [Assigned] (SPARK-38666) Missing aggregate filter checks

[ https://issues.apache.org/jira/browse/SPARK-38666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38666: Assignee: Apache Spark > Missing aggregate filter checks > --

[jira] [Commented] (SPARK-38666) Missing aggregate filter checks

[ https://issues.apache.org/jira/browse/SPARK-38666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517593#comment-17517593 ] Apache Spark commented on SPARK-38666: -- User 'bersprockets' has created a pull requ

[jira] [Updated] (SPARK-38790) Unexpected behaviour for "IN" operator in spark when null is involved in array

[ https://issues.apache.org/jira/browse/SPARK-38790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishnu K Suman updated SPARK-38790: --- Description:   `IN`operator in spark.sql is giving unexpected results 1 in (null, 1) => tr

[jira] [Updated] (SPARK-38790) Unexpected behaviour for "IN" operator in spark when null is involved in array

[ https://issues.apache.org/jira/browse/SPARK-38790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishnu K Suman updated SPARK-38790: --- Description:   `IN`operator in spark.sql is giving unexpected results 1 in (null, 1) => tr

[jira] [Updated] (SPARK-38790) Unexpected behaviour for "IN" operator in spark when null is involved in array

[ https://issues.apache.org/jira/browse/SPARK-38790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishnu K Suman updated SPARK-38790: --- Attachment: image-2022-04-05-22-21-27-695.png > Unexpected behaviour for "IN" operator in sp

[jira] [Updated] (SPARK-38790) Unexpected behaviour for in operator in spark when null is involved in array

[ https://issues.apache.org/jira/browse/SPARK-38790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishnu K Suman updated SPARK-38790: --- Summary: Unexpected behaviour for in operator in spark when null is involved in array (was:

[jira] [Updated] (SPARK-38790) Unexpected behaviour for "IN" operator in spark when null is involved in array

[ https://issues.apache.org/jira/browse/SPARK-38790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishnu K Suman updated SPARK-38790: --- Summary: Unexpected behaviour for "IN" operator in spark when null is involved in array (wa

[jira] [Created] (SPARK-38790) Unexpected behaviour for `in` operator in spark when null is involved in array

Vishnu K Suman created SPARK-38790: -- Summary: Unexpected behaviour for `in` operator in spark when null is involved in array Key: SPARK-38790 URL: https://issues.apache.org/jira/browse/SPARK-38790 Pr

[jira] [Commented] (SPARK-38445) Are hadoop committers used in Structured Streaming?

[ https://issues.apache.org/jira/browse/SPARK-38445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517556#comment-17517556 ] Steve Loughran commented on SPARK-38445: not suppoorted unless you provide the P

[jira] [Resolved] (SPARK-38384) Improve error messages of ParseException from ANTLR

[ https://issues.apache.org/jira/browse/SPARK-38384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-38384. - Fix Version/s: 3.3.0 Assignee: Xinyi Yu Resolution: Fixed > Improve error messages of Pa

[jira] [Assigned] (SPARK-37423) Inline type hints for python/pyspark/mllib/fpm.py

[ https://issues.apache.org/jira/browse/SPARK-37423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37423: -- Assignee: dch nguyen > Inline type hints for python/pyspark/mllib/fpm.py > --

[jira] [Resolved] (SPARK-37423) Inline type hints for python/pyspark/mllib/fpm.py

[ https://issues.apache.org/jira/browse/SPARK-37423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37423. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35067

[jira] [Updated] (SPARK-38789) Spark-Avro throws exception at to_avro after transformation although schema didn't change

[ https://issues.apache.org/jira/browse/SPARK-38789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Keppel updated SPARK-38789: -- Description: I use Structured Streaming to read Avro records from a Kafka topic A, do some tr

[jira] [Updated] (SPARK-38789) Spark-Avro throws exception at to_avro after transformation although schema didn't change

[ https://issues.apache.org/jira/browse/SPARK-38789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Keppel updated SPARK-38789: -- Description: I use Structured Streaming to read Avro records from a Kafka topic A, do some tr

[jira] [Created] (SPARK-38789) Spark-Avro throws exception at to_avro after transformation although schema didn't change

Julian Keppel created SPARK-38789: - Summary: Spark-Avro throws exception at to_avro after transformation although schema didn't change Key: SPARK-38789 URL: https://issues.apache.org/jira/browse/SPARK-38789

[jira] [Commented] (SPARK-37398) Inline type hints for python/pyspark/ml/classification.py

[ https://issues.apache.org/jira/browse/SPARK-37398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517460#comment-17517460 ] Apache Spark commented on SPARK-37398: -- User 'hi-zir' has created a pull request fo

[jira] [Commented] (SPARK-38637) pyspark.pandas.config.OptionError: "No such option: 'mode.chained_assignment'

[ https://issues.apache.org/jira/browse/SPARK-38637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517393#comment-17517393 ] pralabhkumar commented on SPARK-38637: -- [~itholic] can I work on this  > pyspark.p

[jira] [Commented] (SPARK-37997) Allow query parameters to be passed into spark.read

[ https://issues.apache.org/jira/browse/SPARK-37997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517333#comment-17517333 ] Plop commented on SPARK-37997: -- Is there a workaround for this? > Allow query parameters t

[jira] [Commented] (SPARK-38514) Download link on spark 3.2.1 for hadoop 3.2 is wrong

[ https://issues.apache.org/jira/browse/SPARK-38514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517307#comment-17517307 ] Bengt Sjögren commented on SPARK-38514: --- Okay cool! Thanks for informing me and fo

[jira] [Commented] (SPARK-38514) Download link on spark 3.2.1 for hadoop 3.2 is wrong

[ https://issues.apache.org/jira/browse/SPARK-38514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517304#comment-17517304 ] Hyukjin Kwon commented on SPARK-38514: -- This needed a big and invasive change so it

[jira] [Commented] (SPARK-38514) Download link on spark 3.2.1 for hadoop 3.2 is wrong

[ https://issues.apache.org/jira/browse/SPARK-38514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517296#comment-17517296 ] Bengt Sjögren commented on SPARK-38514: --- Hi, The tarball name reference still sta

[jira] [Updated] (SPARK-36555) DS V2 Filter support

[ https://issues.apache.org/jira/browse/SPARK-36555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-36555: - Epic Link: SPARK-38788 > DS V2 Filter support > > > Key: SPARK-3655

[jira] [Updated] (SPARK-36644) Push down boolean column filter

[ https://issues.apache.org/jira/browse/SPARK-36644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-36644: - Epic Link: SPARK-38788 > Push down boolean column filter > --- > >

[jira] [Updated] (SPARK-37483) Support push down top N to JDBC data source V2

[ https://issues.apache.org/jira/browse/SPARK-37483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-37483: - Epic Link: SPARK-38788 > Support push down top N to JDBC data source V2 > --

[jira] [Updated] (SPARK-37644) Support datasource v2 complete aggregate pushdown

[ https://issues.apache.org/jira/browse/SPARK-37644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-37644: - Epic Link: SPARK-38788 > Support datasource v2 complete aggregate pushdown > --

[jira] [Updated] (SPARK-37839) DS V2 supports partial aggregate push-down AVG

[ https://issues.apache.org/jira/browse/SPARK-37839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-37839: - Epic Link: SPARK-38788 > DS V2 supports partial aggregate push-down AVG > --

[jira] [Updated] (SPARK-36640) More operator pushdown API in Data Source V2

[ https://issues.apache.org/jira/browse/SPARK-36640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-36640: - Epic Link: SPARK-38788 > More operator pushdown API in Data Source V2 > ---

[jira] [Created] (SPARK-38788) More comprehensive DSV2 push down capabilities

Max Gekk created SPARK-38788: Summary: More comprehensive DSV2 push down capabilities Key: SPARK-38788 URL: https://issues.apache.org/jira/browse/SPARK-38788 Project: Spark Issue Type: Epic

  1   2   >