[jira] [Commented] (SPARK-39887) Expression transform error

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17578855#comment-17578855 ] Apache Spark commented on SPARK-39887: -- User 'peter-toth' has created a pull request for this

[jira] [Reopened] (SPARK-40013) DS V2 expressions should have the default implementation of toString

2022-08-12 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng reopened SPARK-40013: > DS V2 expressions should have the default implementation of toString >

[jira] [Created] (SPARK-40057) Cleanup "" in doctest

2022-08-12 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-40057: --- Summary: Cleanup "" in doctest Key: SPARK-40057 URL: https://issues.apache.org/jira/browse/SPARK-40057 Project: Spark Issue Type: Bug Components:

[jira] [Assigned] (SPARK-40013) DS V2 expressions should have the default toString

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40013: Assignee: Apache Spark > DS V2 expressions should have the default toString >

[jira] [Assigned] (SPARK-40013) DS V2 expressions should have the default toString

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40013: Assignee: Apache Spark > DS V2 expressions should have the default toString >

[jira] [Assigned] (SPARK-40013) DS V2 expressions should have the default toString

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40013: Assignee: (was: Apache Spark) > DS V2 expressions should have the default toString >

[jira] [Commented] (SPARK-40013) DS V2 expressions should have the default toString

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17578936#comment-17578936 ] Apache Spark commented on SPARK-40013: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40014) Support cast of decimals to ANSI intervals

2022-08-12 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-40014: Assignee: Max Gekk > Support cast of decimals to ANSI intervals >

[jira] [Resolved] (SPARK-40014) Support cast of decimals to ANSI intervals

2022-08-12 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-40014. -- Resolution: Fixed Issue resolved by pull request 37466 [https://github.com/apache/spark/pull/37466]

[jira] [Assigned] (SPARK-40057) Cleanup "" in doctest

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40057: Assignee: (was: Apache Spark) > Cleanup "" in doctest >

[jira] [Commented] (SPARK-40057) Cleanup "" in doctest

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17578886#comment-17578886 ] Apache Spark commented on SPARK-40057: -- User 'Yikun' has created a pull request for this issue:

[jira] [Commented] (SPARK-40057) Cleanup "" in doctest

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17578885#comment-17578885 ] Apache Spark commented on SPARK-40057: -- User 'Yikun' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40057) Cleanup "" in doctest

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40057: Assignee: Apache Spark > Cleanup "" in doctest > > >

[jira] [Created] (SPARK-40058) Avoid filter twice in HadoopFSUtils

2022-08-12 Thread ZiyueGuan (Jira)
ZiyueGuan created SPARK-40058: - Summary: Avoid filter twice in HadoopFSUtils Key: SPARK-40058 URL: https://issues.apache.org/jira/browse/SPARK-40058 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-40057) Cleanup "" in doctest

2022-08-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40057. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37492

[jira] [Commented] (SPARK-37690) Recursive view `df` detected (cycle: `df` -> `df`)

2022-08-12 Thread Daniel Darabos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17578901#comment-17578901 ] Daniel Darabos commented on SPARK-37690: It's fixed in Spark 3.3.0.

[jira] [Updated] (SPARK-40013) DS V2 expressions should have the default toString

2022-08-12 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-40013: --- Summary: DS V2 expressions should have the default toString (was: DS V2 expressions should have

[jira] [Updated] (SPARK-40058) Avoid filter twice in HadoopFSUtils

2022-08-12 Thread ZiyueGuan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZiyueGuan updated SPARK-40058: -- Component/s: Spark Core (was: SQL) > Avoid filter twice in HadoopFSUtils >

[jira] [Commented] (SPARK-39887) Expression transform error

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17578973#comment-17578973 ] Apache Spark commented on SPARK-39887: -- User 'peter-toth' has created a pull request for this

[jira] [Assigned] (SPARK-40057) Cleanup "" in doctest

2022-08-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40057: Assignee: Yikun Jiang > Cleanup "" in doctest > > >

[jira] [Resolved] (SPARK-40054) Restore the error handling syntax of try_cast()

2022-08-12 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-40054. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37486

[jira] [Created] (SPARK-40060) Add numberDecommissioningExecutors metric

2022-08-12 Thread Zhongwei Zhu (Jira)
Zhongwei Zhu created SPARK-40060: Summary: Add numberDecommissioningExecutors metric Key: SPARK-40060 URL: https://issues.apache.org/jira/browse/SPARK-40060 Project: Spark Issue Type:

[jira] [Updated] (SPARK-40058) Avoid filter twice in HadoopFSUtils

2022-08-12 Thread ZiyueGuan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZiyueGuan updated SPARK-40058: -- Affects Version/s: 3.4.0 (was: 3.2.2) > Avoid filter twice in

[jira] [Commented] (SPARK-40020) centralize the code of qualifying identifiers in SessionCatalog

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579014#comment-17579014 ] Apache Spark commented on SPARK-40020: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-40020) centralize the code of qualifying identifiers in SessionCatalog

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579013#comment-17579013 ] Apache Spark commented on SPARK-40020: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-40058) Avoid filter twice in HadoopFSUtils

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579048#comment-17579048 ] Apache Spark commented on SPARK-40058: -- User 'guanziyue' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40058) Avoid filter twice in HadoopFSUtils

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40058: Assignee: Apache Spark > Avoid filter twice in HadoopFSUtils >

[jira] [Assigned] (SPARK-40058) Avoid filter twice in HadoopFSUtils

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40058: Assignee: (was: Apache Spark) > Avoid filter twice in HadoopFSUtils >

[jira] [Commented] (SPARK-40058) Avoid filter twice in HadoopFSUtils

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579047#comment-17579047 ] Apache Spark commented on SPARK-40058: -- User 'guanziyue' has created a pull request for this issue:

[jira] [Commented] (SPARK-40060) Add numberDecommissioningExecutors metric

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579058#comment-17579058 ] Apache Spark commented on SPARK-40060: -- User 'warrenzhu25' has created a pull request for this

[jira] [Assigned] (SPARK-40060) Add numberDecommissioningExecutors metric

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40060: Assignee: (was: Apache Spark) > Add numberDecommissioningExecutors metric >

[jira] [Assigned] (SPARK-40060) Add numberDecommissioningExecutors metric

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40060: Assignee: Apache Spark > Add numberDecommissioningExecutors metric >

[jira] [Commented] (SPARK-40061) Document cast of ANSI intervals

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579096#comment-17579096 ] Apache Spark commented on SPARK-40061: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40061) Document cast of ANSI intervals

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40061: Assignee: Max Gekk (was: Apache Spark) > Document cast of ANSI intervals >

[jira] [Created] (SPARK-40059) Row indexes can overshadow user-created data

2022-08-12 Thread Ala Luszczak (Jira)
Ala Luszczak created SPARK-40059: Summary: Row indexes can overshadow user-created data Key: SPARK-40059 URL: https://issues.apache.org/jira/browse/SPARK-40059 Project: Spark Issue Type:

[jira] [Created] (SPARK-40061) Document cast of ANSI intervals

2022-08-12 Thread Max Gekk (Jira)
Max Gekk created SPARK-40061: Summary: Document cast of ANSI intervals Key: SPARK-40061 URL: https://issues.apache.org/jira/browse/SPARK-40061 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-40061) Document cast of ANSI intervals

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40061: Assignee: Apache Spark (was: Max Gekk) > Document cast of ANSI intervals >

[jira] [Updated] (SPARK-40056) Upgrade mvn-scalafmt from 1.0.4 to 1.1.1640084764.9f463a9

2022-08-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-40056: - Priority: Trivial (was: Minor) > Upgrade mvn-scalafmt from 1.0.4 to 1.1.1640084764.9f463a9 >

[jira] [Assigned] (SPARK-40056) Upgrade mvn-scalafmt from 1.0.4 to 1.1.1640084764.9f463a9

2022-08-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-40056: Assignee: BingKun Pan > Upgrade mvn-scalafmt from 1.0.4 to 1.1.1640084764.9f463a9 >

[jira] [Resolved] (SPARK-40056) Upgrade mvn-scalafmt from 1.0.4 to 1.1.1640084764.9f463a9

2022-08-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40056. -- Resolution: Fixed Issue resolved by pull request 37489

[jira] [Resolved] (SPARK-40061) Document cast of ANSI intervals

2022-08-12 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-40061. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37495

[jira] [Updated] (SPARK-40069) Extend the new heartbeat mechanism to Kubernetes

2022-08-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-40069: - Component/s: Kubernetes (was: Spark Core) > Extend the new heartbeat

[jira] [Commented] (SPARK-40063) pyspark.pandas .apply() changing rows ordering

2022-08-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579213#comment-17579213 ] Hyukjin Kwon commented on SPARK-40063: -- {quote} it ends up mixing the column's rows ordering.

[jira] [Commented] (SPARK-40048) Partitions are traversed multiple times invalidating Accumulator consistency

2022-08-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579214#comment-17579214 ] Hyukjin Kwon commented on SPARK-40048: -- Spark 2.4 is EOL. mind trying Spark 3.1+? > Partitions are

[jira] [Updated] (SPARK-40068) Extend new heartbeat mechanism to YARN

2022-08-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-40068: - Component/s: YARN (was: Spark Core) > Extend new heartbeat mechanism to

[jira] [Updated] (SPARK-40063) pyspark.pandas .apply() changing rows ordering

2022-08-12 Thread Marcelo Rossini Castro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Rossini Castro updated SPARK-40063: --- Summary: pyspark.pandas .apply() changing rows ordering (was:

[jira] [Updated] (SPARK-40063) pyspark.pandas .apply() chaging rows ordering

2022-08-12 Thread Marcelo Rossini Castro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Rossini Castro updated SPARK-40063: --- Description: When using the apply function to apply a function to a

[jira] [Updated] (SPARK-40063) pyspark.pandas .apply() chaging rows ordering

2022-08-12 Thread Marcelo Rossini Castro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Rossini Castro updated SPARK-40063: --- Description: When using the apply function to apply a function to a

[jira] [Updated] (SPARK-40063) pyspark.pandas .apply() chaging rows ordering

2022-08-12 Thread Marcelo Rossini Castro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Rossini Castro updated SPARK-40063: --- Description: When using the apply function to apply a function to a

[jira] [Updated] (SPARK-40063) pyspark.pandas .apply() changing rows ordering

2022-08-12 Thread Marcelo Rossini Castro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Rossini Castro updated SPARK-40063: --- Description: When using the apply function to apply a function to a

[jira] [Created] (SPARK-40065) Executor ConfigMap is not created if profile is not default

2022-08-12 Thread Nobuaki Sukegawa (Jira)
Nobuaki Sukegawa created SPARK-40065: Summary: Executor ConfigMap is not created if profile is not default Key: SPARK-40065 URL: https://issues.apache.org/jira/browse/SPARK-40065 Project: Spark

[jira] [Updated] (SPARK-40066) ANSI mode: always return null on invalid access to map column

2022-08-12 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-40066: --- Description: Since https://github.com/apache/spark/pull/30386, Spark always throws an error

[jira] [Commented] (SPARK-40049) Add adaptive plan case in ReplaceNullWithFalseInPredicateEndToEndSuite

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579138#comment-17579138 ] Apache Spark commented on SPARK-40049: -- User 'kazuyukitanimura' has created a pull request for this

[jira] [Assigned] (SPARK-40049) Add adaptive plan case in ReplaceNullWithFalseInPredicateEndToEndSuite

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40049: Assignee: (was: Apache Spark) > Add adaptive plan case in

[jira] [Assigned] (SPARK-40049) Add adaptive plan case in ReplaceNullWithFalseInPredicateEndToEndSuite

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40049: Assignee: Apache Spark > Add adaptive plan case in

[jira] [Updated] (SPARK-40063) pyspark.pandas .apply() changing rows ordering

2022-08-12 Thread Marcelo Rossini Castro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Rossini Castro updated SPARK-40063: --- Description: When using the apply function to apply a function to a

[jira] [Updated] (SPARK-39966) Use V2 Filter in SupportsDelete

2022-08-12 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-39966: --- Parent: SPARK-36555 Issue Type: Sub-task (was: Improvement) > Use V2 Filter in

[jira] [Updated] (SPARK-40065) Executor ConfigMap is not mounted if profile is not default

2022-08-12 Thread Nobuaki Sukegawa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nobuaki Sukegawa updated SPARK-40065: - Summary: Executor ConfigMap is not mounted if profile is not default (was: Executor

[jira] [Updated] (SPARK-40065) Executor ConfigMap is not mounted if profile is not default

2022-08-12 Thread Nobuaki Sukegawa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nobuaki Sukegawa updated SPARK-40065: - Description: When the resource profile is non-default, executor configmap is not

[jira] [Commented] (SPARK-40066) ANSI mode: always return null on invalid access to map column

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579200#comment-17579200 ] Apache Spark commented on SPARK-40066: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-40066) ANSI mode: always return null on invalid access to map column

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40066: Assignee: Apache Spark (was: Gengliang Wang) > ANSI mode: always return null on invalid

[jira] [Updated] (SPARK-40065) Executor ConfigMap is not mounted if profile is not default

2022-08-12 Thread Nobuaki Sukegawa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nobuaki Sukegawa updated SPARK-40065: - Description: When executor config map is made optional in SPARK-34316, mount volume is

[jira] [Updated] (SPARK-40065) Executor ConfigMap is not mounted if profile is not default

2022-08-12 Thread Nobuaki Sukegawa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nobuaki Sukegawa updated SPARK-40065: - Description: When executor config map is made optional in SPARK-34316, mount volume is

[jira] [Assigned] (SPARK-40066) ANSI mode: always return null on invalid access to map column

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40066: Assignee: Gengliang Wang (was: Apache Spark) > ANSI mode: always return null on invalid

[jira] [Commented] (SPARK-40065) Executor ConfigMap is not mounted if profile is not default

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579201#comment-17579201 ] Apache Spark commented on SPARK-40065: -- User 'nsuke' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40065) Executor ConfigMap is not mounted if profile is not default

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40065: Assignee: Apache Spark > Executor ConfigMap is not mounted if profile is not default >

[jira] [Assigned] (SPARK-40065) Executor ConfigMap is not mounted if profile is not default

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40065: Assignee: (was: Apache Spark) > Executor ConfigMap is not mounted if profile is not

[jira] [Created] (SPARK-40063) pyspark.pandas .apply() chaging rows order

2022-08-12 Thread Marcelo Rossini Castro (Jira)
Marcelo Rossini Castro created SPARK-40063: -- Summary: pyspark.pandas .apply() chaging rows order Key: SPARK-40063 URL: https://issues.apache.org/jira/browse/SPARK-40063 Project: Spark

[jira] [Updated] (SPARK-40063) pyspark.pandas .apply() chaging rows ordering

2022-08-12 Thread Marcelo Rossini Castro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Rossini Castro updated SPARK-40063: --- Summary: pyspark.pandas .apply() chaging rows ordering (was:

[jira] [Commented] (SPARK-40064) Use V2 Filter in SupportsOverwrite

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579149#comment-17579149 ] Apache Spark commented on SPARK-40064: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40064) Use V2 Filter in SupportsOverwrite

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40064: Assignee: Apache Spark > Use V2 Filter in SupportsOverwrite >

[jira] [Commented] (SPARK-40064) Use V2 Filter in SupportsOverwrite

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579150#comment-17579150 ] Apache Spark commented on SPARK-40064: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40064) Use V2 Filter in SupportsOverwrite

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40064: Assignee: (was: Apache Spark) > Use V2 Filter in SupportsOverwrite >

[jira] [Resolved] (SPARK-38969) Graceful decomissionning on Kubernetes fails / decom script error

2022-08-12 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-38969. -- Fix Version/s: 3.4.0 Assignee: Holden Karau Resolution: Fixed Updated

[jira] [Created] (SPARK-40066) ANSI mode: always return null on invalid access to map column

2022-08-12 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-40066: -- Summary: ANSI mode: always return null on invalid access to map column Key: SPARK-40066 URL: https://issues.apache.org/jira/browse/SPARK-40066 Project: Spark

[jira] [Updated] (SPARK-40067) Add table name to Spark plan node in SparkUI

2022-08-12 Thread Sumeet (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumeet updated SPARK-40067: --- Description: [SPARK-39902|https://issues.apache.org/jira/browse/SPARK-39902] introduced `Scan#name()` API

[jira] [Created] (SPARK-40068) Extend new heartbeat mechanism to YARN

2022-08-12 Thread Kai-Hsun Chen (Jira)
Kai-Hsun Chen created SPARK-40068: - Summary: Extend new heartbeat mechanism to YARN Key: SPARK-40068 URL: https://issues.apache.org/jira/browse/SPARK-40068 Project: Spark Issue Type:

[jira] [Created] (SPARK-40062) Spark - Creating Sub Folder while writing to Partitioned Hive Table

2022-08-12 Thread dinesh (Jira)
string) Partitioned by (process_date string) stored as parquet location '/mytable/a/b/c/org=employee';_   While writing to table HDFS path being written looks something like this -  {_}/mytable/a/b/c/org=employee/{_}{_}process_date=20220812/{_}{color:#de350b}_org=employee_{color

[jira] [Commented] (SPARK-39926) Fix bug in existence DEFAULT value lookups for non-vectorized Parquet scans

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579142#comment-17579142 ] Apache Spark commented on SPARK-39926: -- User 'dtenedor' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39926) Fix bug in existence DEFAULT value lookups for non-vectorized Parquet scans

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39926: Assignee: Apache Spark > Fix bug in existence DEFAULT value lookups for non-vectorized

[jira] [Assigned] (SPARK-39926) Fix bug in existence DEFAULT value lookups for non-vectorized Parquet scans

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39926: Assignee: (was: Apache Spark) > Fix bug in existence DEFAULT value lookups for

[jira] [Updated] (SPARK-40063) pyspark.pandas .apply() changing rows ordering

2022-08-12 Thread Marcelo Rossini Castro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Rossini Castro updated SPARK-40063: --- Description: When using the apply function to apply a function to a

[jira] [Commented] (SPARK-40065) Executor ConfigMap is not mounted if profile is not default

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579202#comment-17579202 ] Apache Spark commented on SPARK-40065: -- User 'nsuke' has created a pull request for this issue:

[jira] [Updated] (SPARK-40063) pyspark.pandas .apply() changing rows ordering

2022-08-12 Thread Marcelo Rossini Castro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Rossini Castro updated SPARK-40063: --- Language: Python Environment: Databricks Runtime 11.1

[jira] [Created] (SPARK-40064) Use V2 Filter in SupportsOverwrite

2022-08-12 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-40064: -- Summary: Use V2 Filter in SupportsOverwrite Key: SPARK-40064 URL: https://issues.apache.org/jira/browse/SPARK-40064 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-39528) Use V2 Filter in SupportsRuntimeFiltering

2022-08-12 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-39528: --- Parent: SPARK-36555 Issue Type: Sub-task (was: Improvement) > Use V2 Filter in

[jira] [Assigned] (SPARK-40052) Handle direct byte buffers in VectorizedDeltaBinaryPackedReader

2022-08-12 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-40052: Assignee: Ivan Sadikov > Handle direct byte buffers in VectorizedDeltaBinaryPackedReader >

[jira] [Resolved] (SPARK-40052) Handle direct byte buffers in VectorizedDeltaBinaryPackedReader

2022-08-12 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-40052. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37485

[jira] [Updated] (SPARK-40037) Upgrade com.google.crypto.tink:tink from 1.6.1 to 1.7.0

2022-08-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-40037: - Priority: Minor (was: Major) > Upgrade com.google.crypto.tink:tink from 1.6.1 to 1.7.0 >

[jira] [Assigned] (SPARK-40037) Upgrade com.google.crypto.tink:tink from 1.6.1 to 1.7.0

2022-08-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-40037: Assignee: Bjørn Jørgensen > Upgrade com.google.crypto.tink:tink from 1.6.1 to 1.7.0 >

[jira] [Resolved] (SPARK-40037) Upgrade com.google.crypto.tink:tink from 1.6.1 to 1.7.0

2022-08-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40037. -- Fix Version/s: 3.4.0 Resolution: Fixed Resolved by

[jira] [Assigned] (SPARK-40049) Add adaptive plan case in ReplaceNullWithFalseInPredicateEndToEndSuite

2022-08-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-40049: - Assignee: Kazuyuki Tanimura > Add adaptive plan case in

[jira] [Resolved] (SPARK-40049) Add adaptive plan case in ReplaceNullWithFalseInPredicateEndToEndSuite

2022-08-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40049. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37500

[jira] [Updated] (SPARK-40065) Executor ConfigMap is not created if profile is not default

2022-08-12 Thread Nobuaki Sukegawa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nobuaki Sukegawa updated SPARK-40065: - Affects Version/s: 3.2.2 > Executor ConfigMap is not created if profile is not default

[jira] [Updated] (SPARK-40065) Executor ConfigMap is not created if profile is not default

2022-08-12 Thread Nobuaki Sukegawa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nobuaki Sukegawa updated SPARK-40065: - Affects Version/s: 3.2.1 3.2.0 > Executor ConfigMap is not

[jira] [Created] (SPARK-40067) Add table name to Spark plan node in SparkUI

2022-08-12 Thread Sumeet (Jira)
Sumeet created SPARK-40067: -- Summary: Add table name to Spark plan node in SparkUI Key: SPARK-40067 URL: https://issues.apache.org/jira/browse/SPARK-40067 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-40067) Add table name to Spark plan node in SparkUI

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40067: Assignee: (was: Apache Spark) > Add table name to Spark plan node in SparkUI >

[jira] [Assigned] (SPARK-40067) Add table name to Spark plan node in SparkUI

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40067: Assignee: Apache Spark > Add table name to Spark plan node in SparkUI >

[jira] [Commented] (SPARK-40067) Add table name to Spark plan node in SparkUI

2022-08-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579209#comment-17579209 ] Apache Spark commented on SPARK-40067: -- User 'sumeetgajjar' has created a pull request for this

[jira] [Created] (SPARK-40069) Extend the new heartbeat mechanism to Kubernetes

2022-08-12 Thread Kai-Hsun Chen (Jira)
Kai-Hsun Chen created SPARK-40069: - Summary: Extend the new heartbeat mechanism to Kubernetes Key: SPARK-40069 URL: https://issues.apache.org/jira/browse/SPARK-40069 Project: Spark Issue

  1   2   >