[jira] [Created] (SPARK-38300) Refactor `fileToString` and `resourceToBytes` in catalyst.util to clean up duplicate codes

2022-02-22 Thread Yang Jie (Jira)
Yang Jie created SPARK-38300: Summary: Refactor `fileToString` and `resourceToBytes` in catalyst.util to clean up duplicate codes Key: SPARK-38300 URL: https://issues.apache.org/jira/browse/SPARK-38300

[jira] [Assigned] (SPARK-38299) Clean up deprecated usage of `StringBuilder.newBuilder`

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38299: Assignee: Apache Spark > Clean up deprecated usage of `StringBuilder.newBuilder` >

[jira] [Assigned] (SPARK-38299) Clean up deprecated usage of `StringBuilder.newBuilder`

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38299: Assignee: (was: Apache Spark) > Clean up deprecated usage of

[jira] [Commented] (SPARK-38299) Clean up deprecated usage of `StringBuilder.newBuilder`

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496544#comment-17496544 ] Apache Spark commented on SPARK-38299: -- User 'LuciferYang' has created a pull request for this

[jira] [Created] (SPARK-38299) Clean up deprecated usage of `StringBuilder.newBuilder`

2022-02-22 Thread Yang Jie (Jira)
Yang Jie created SPARK-38299: Summary: Clean up deprecated usage of `StringBuilder.newBuilder` Key: SPARK-38299 URL: https://issues.apache.org/jira/browse/SPARK-38299 Project: Spark Issue Type:

[jira] [Commented] (SPARK-38294) DDLUtils.verifyNotReadPath should check target is subDir

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496538#comment-17496538 ] Apache Spark commented on SPARK-38294: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-38294) DDLUtils.verifyNotReadPath should check target is subDir

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38294: Assignee: Apache Spark > DDLUtils.verifyNotReadPath should check target is subDir >

[jira] [Commented] (SPARK-38294) DDLUtils.verifyNotReadPath should check target is subDir

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496537#comment-17496537 ] Apache Spark commented on SPARK-38294: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-38294) DDLUtils.verifyNotReadPath should check target is subDir

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38294: Assignee: (was: Apache Spark) > DDLUtils.verifyNotReadPath should check target is

[jira] [Resolved] (SPARK-38162) Optimize one row plan in normal and AQE Optimizer

2022-02-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38162. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35473

[jira] [Assigned] (SPARK-38240) Improve RuntimeReplaceable and add a guideline for adding new functions

2022-02-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-38240: --- Assignee: Wenchen Fan > Improve RuntimeReplaceable and add a guideline for adding new

[jira] [Resolved] (SPARK-38240) Improve RuntimeReplaceable and add a guideline for adding new functions

2022-02-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38240. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35534

[jira] [Commented] (SPARK-36967) Report accurate shuffle block size if its skewed

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496509#comment-17496509 ] Apache Spark commented on SPARK-36967: -- User 'wankunde' has created a pull request for this issue:

[jira] [Commented] (SPARK-36967) Report accurate shuffle block size if its skewed

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496508#comment-17496508 ] Apache Spark commented on SPARK-36967: -- User 'wankunde' has created a pull request for this issue:

[jira] [Commented] (SPARK-38298) Fix DataExpressionSuite, NullExpressionsSuite, StringExpressionsSuite, complexTypesSuite, CastSuite under ANSI mode

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496491#comment-17496491 ] Apache Spark commented on SPARK-38298: -- User 'anchovYu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38298) Fix DataExpressionSuite, NullExpressionsSuite, StringExpressionsSuite, complexTypesSuite, CastSuite under ANSI mode

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38298: Assignee: Apache Spark > Fix DataExpressionSuite, NullExpressionsSuite,

[jira] [Assigned] (SPARK-38298) Fix DataExpressionSuite, NullExpressionsSuite, StringExpressionsSuite, complexTypesSuite, CastSuite under ANSI mode

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38298: Assignee: (was: Apache Spark) > Fix DataExpressionSuite, NullExpressionsSuite,

[jira] [Commented] (SPARK-38298) Fix DataExpressionSuite, NullExpressionsSuite, StringExpressionsSuite, complexTypesSuite, CastSuite under ANSI mode

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496490#comment-17496490 ] Apache Spark commented on SPARK-38298: -- User 'anchovYu' has created a pull request for this issue:

[jira] [Created] (SPARK-38298) Fix DataExpressionSuite, NullExpressionsSuite, StringExpressionsSuite, complexTypesSuite, CastSuite

2022-02-22 Thread Xinyi Yu (Jira)
Xinyi Yu created SPARK-38298: Summary: Fix DataExpressionSuite, NullExpressionsSuite, StringExpressionsSuite, complexTypesSuite, CastSuite Key: SPARK-38298 URL: https://issues.apache.org/jira/browse/SPARK-38298

[jira] [Updated] (SPARK-38298) Fix DataExpressionSuite, NullExpressionsSuite, StringExpressionsSuite, complexTypesSuite, CastSuite under ANSI mode

2022-02-22 Thread Xinyi Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinyi Yu updated SPARK-38298: - Summary: Fix DataExpressionSuite, NullExpressionsSuite, StringExpressionsSuite, complexTypesSuite,

[jira] [Updated] (SPARK-38298) Fix DataExpressionSuite, NullExpressionsSuite, StringExpressionsSuite, complexTypesSuite, CastSuite Under ANSI mode

2022-02-22 Thread Xinyi Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinyi Yu updated SPARK-38298: - Summary: Fix DataExpressionSuite, NullExpressionsSuite, StringExpressionsSuite, complexTypesSuite,

[jira] [Resolved] (SPARK-38297) Fix mypy failure on DataFrame.to_numpy in pandas API on Spark

2022-02-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38297. -- Fix Version/s: 3.3.0 3.2.2 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-38297) Fix mypy failure on DataFrame.to_numpy in pandas API on Spark

2022-02-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38297: Assignee: Hyukjin Kwon > Fix mypy failure on DataFrame.to_numpy in pandas API on Spark >

[jira] [Assigned] (SPARK-38297) Fix mypy failure on DataFrame.to_numpy in pandas API on Spark

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38297: Assignee: (was: Apache Spark) > Fix mypy failure on DataFrame.to_numpy in pandas API

[jira] [Commented] (SPARK-38297) Fix mypy failure on DataFrame.to_numpy in pandas API on Spark

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496470#comment-17496470 ] Apache Spark commented on SPARK-38297: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-38297) Fix mypy failure on DataFrame.to_numpy in pandas API on Spark

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38297: Assignee: Apache Spark > Fix mypy failure on DataFrame.to_numpy in pandas API on Spark >

[jira] [Created] (SPARK-38297) Fix mypy failure on DataFrame.to_numpy in pandas API on Spark

2022-02-22 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-38297: Summary: Fix mypy failure on DataFrame.to_numpy in pandas API on Spark Key: SPARK-38297 URL: https://issues.apache.org/jira/browse/SPARK-38297 Project: Spark

[jira] [Updated] (SPARK-38296) Error Classes AnalysisException is not propagated in FunctionRegistry

2022-02-22 Thread Andrew Murphy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Murphy updated SPARK-38296: -- Description: While trying to migrate invalidLiteralForWindowDurationError to use Error

[jira] [Updated] (SPARK-38296) Error Classes AnalysisException is not propagated in FunctionRegistry

2022-02-22 Thread Andrew Murphy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Murphy updated SPARK-38296: -- External issue ID: (was: SPARK-38110) External issue URL: (was:

[jira] [Created] (SPARK-38296) Error Classes AnalysisException is not propagated in FunctionRegistry

2022-02-22 Thread Andrew Murphy (Jira)
Andrew Murphy created SPARK-38296: - Summary: Error Classes AnalysisException is not propagated in FunctionRegistry Key: SPARK-38296 URL: https://issues.apache.org/jira/browse/SPARK-38296 Project:

[jira] [Resolved] (SPARK-38241) Close KubernetesClient in K8S integrations tests

2022-02-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38241. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 3

[jira] [Assigned] (SPARK-38241) Close KubernetesClient in K8S integrations tests

2022-02-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38241: - Assignee: Martin Tzvetanov Grigorov > Close KubernetesClient in K8S integrations tests

[jira] [Assigned] (SPARK-38295) Fix ArithmeticExpressionSuite under ANSI mode

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38295: Assignee: (was: Apache Spark) > Fix ArithmeticExpressionSuite under ANSI mode >

[jira] [Assigned] (SPARK-38295) Fix ArithmeticExpressionSuite under ANSI mode

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38295: Assignee: Apache Spark > Fix ArithmeticExpressionSuite under ANSI mode >

[jira] [Commented] (SPARK-38295) Fix ArithmeticExpressionSuite under ANSI mode

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496447#comment-17496447 ] Apache Spark commented on SPARK-38295: -- User 'anchovYu' has created a pull request for this issue:

[jira] [Updated] (SPARK-38295) Fix ArithmeticExpressionSuite under ANSI mode

2022-02-22 Thread Xinyi Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinyi Yu updated SPARK-38295: - Description: There are about 8 failure test cases in this suite. (was: There are about 20 failure test

[jira] [Created] (SPARK-38295) Fix ArithmeticExpressionSuite under ANSI mode

2022-02-22 Thread Xinyi Yu (Jira)
Xinyi Yu created SPARK-38295: Summary: Fix ArithmeticExpressionSuite under ANSI mode Key: SPARK-38295 URL: https://issues.apache.org/jira/browse/SPARK-38295 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-38260) Remove dependence on commons-net

2022-02-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38260: - Assignee: Yang Jie > Remove dependence on commons-net >

[jira] [Resolved] (SPARK-38260) Remove dependence on commons-net

2022-02-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38260. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35582

[jira] [Commented] (SPARK-38287) Upgrade h2 from 2.0.204 to 2.1.210 in /sql/core

2022-02-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496439#comment-17496439 ] Hyukjin Kwon commented on SPARK-38287: -- [~bjornjorgensen] please go ahead for a PR! > Upgrade h2

[jira] [Updated] (SPARK-38272) Fix wrong context name in spark k8s integration test on docker-desktop

2022-02-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38272: -- Component/s: Tests > Fix wrong context name in spark k8s integration test on docker-desktop >

[jira] [Updated] (SPARK-38272) Fix wrong context name in spark k8s integration test on docker-desktop

2022-02-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38272: -- Issue Type: Improvement (was: Bug) > Fix wrong context name in spark k8s integration test on

[jira] [Updated] (SPARK-38272) Use docker-desktop instead of docker-for-desktop for Docker K8S IT deployMode and context name

2022-02-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38272: -- Summary: Use docker-desktop instead of docker-for-desktop for Docker K8S IT deployMode and

[jira] [Resolved] (SPARK-38272) Fix wrong context name in spark k8s integration test on docker-desktop

2022-02-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38272. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35595

[jira] [Commented] (SPARK-38288) Aggregate push down doesnt work using Spark SQL jdbc datasource with postgresql

2022-02-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496437#comment-17496437 ] Hyukjin Kwon commented on SPARK-38288: -- cc [~beliefer] FYI > Aggregate push down doesnt work

[jira] [Assigned] (SPARK-38272) Fix wrong context name in spark k8s integration test on docker-desktop

2022-02-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38272: - Assignee: Yikun Jiang > Fix wrong context name in spark k8s integration test on

[jira] [Created] (SPARK-38294) DDLUtils.verifyNotReadPath should check target is subDir

2022-02-22 Thread angerszhu (Jira)
angerszhu created SPARK-38294: - Summary: DDLUtils.verifyNotReadPath should check target is subDir Key: SPARK-38294 URL: https://issues.apache.org/jira/browse/SPARK-38294 Project: Spark Issue

[jira] [Resolved] (SPARK-38062) FallbackStorage shouldn't attempt to resolve arbitrary "remote" hostname

2022-02-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38062. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35358

[jira] [Assigned] (SPARK-38062) FallbackStorage shouldn't attempt to resolve arbitrary "remote" hostname

2022-02-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-38062: --- Assignee: Erik Krogen > FallbackStorage shouldn't attempt to resolve arbitrary "remote"

[jira] [Updated] (SPARK-38270) SQL CLI AM should keep same exitcode with client

2022-02-22 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-38270: -- Parent: SPARK-36623 Issue Type: Sub-task (was: Task) > SQL CLI AM should keep same exitcode

[jira] [Updated] (SPARK-38286) Union's maxRows and maxRowsPerPartition may overflow

2022-02-22 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-38286: - Summary: Union's maxRows and maxRowsPerPartition may overflow (was: check Union's maxRows and

[jira] [Commented] (SPARK-38293) Fix flaky text of HealthTrackerIntegrationSuite

2022-02-22 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496422#comment-17496422 ] angerszhu commented on SPARK-38293: --- cc [~dongjoon] [~hyukjin.kwon] Have meet this for some times. >

[jira] [Created] (SPARK-38293) Fix flaky text of HealthTrackerIntegrationSuite

2022-02-22 Thread angerszhu (Jira)
angerszhu created SPARK-38293: - Summary: Fix flaky text of HealthTrackerIntegrationSuite Key: SPARK-38293 URL: https://issues.apache.org/jira/browse/SPARK-38293 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-38292) Support `na_filter` for pyspark.pandas.read_csv

2022-02-22 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-38292: --- Summary: Support `na_filter` for pyspark.pandas.read_csv Key: SPARK-38292 URL: https://issues.apache.org/jira/browse/SPARK-38292 Project: Spark Issue Type:

[jira] [Updated] (SPARK-38245) Avro Complex Union Type return `member$I`

2022-02-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38245: - Labels: avro (was: avro newbie) > Avro Complex Union Type return `member$I` >

[jira] [Assigned] (SPARK-38235) Add test util for testing grouped aggregate pandas UDF.

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38235: Assignee: (was: Apache Spark) > Add test util for testing grouped aggregate pandas

[jira] [Assigned] (SPARK-38235) Add test util for testing grouped aggregate pandas UDF.

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38235: Assignee: Apache Spark > Add test util for testing grouped aggregate pandas UDF. >

[jira] [Commented] (SPARK-38235) Add test util for testing grouped aggregate pandas UDF.

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496397#comment-17496397 ] Apache Spark commented on SPARK-38235: -- User 'itholic' has created a pull request for this issue:

[jira] [Resolved] (SPARK-22947) SPIP: as-of join in Spark SQL

2022-02-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22947. -- Resolution: Duplicate I completely forgot that we have a JIRA here already :-). > SPIP:

[jira] [Commented] (SPARK-22947) SPIP: as-of join in Spark SQL

2022-02-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496394#comment-17496394 ] Hyukjin Kwon commented on SPARK-22947: -- Oh actually [~icexelloss] I believe this is fixed in

[jira] [Commented] (SPARK-38245) Avro Complex Union Type return `member$I`

2022-02-22 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496337#comment-17496337 ] Erik Krogen commented on SPARK-38245: - FWIW, though I don't have context for when this logic was

[jira] [Resolved] (SPARK-38291) Upgrade postgresql from 42.3.0 to 42.3.3

2022-02-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38291. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35614

[jira] [Assigned] (SPARK-38291) Upgrade postgresql from 42.3.0 to 42.3.3

2022-02-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38291: - Assignee: Bjørn Jørgensen > Upgrade postgresql from 42.3.0 to 42.3.3 >

[jira] [Updated] (SPARK-38291) Upgrade postgresql from 42.3.0 to 42.3.3

2022-02-22 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-38291: Summary: Upgrade postgresql from 42.3.0 to 42.3.3 (was: CVE-2022-21724 and

[jira] [Updated] (SPARK-38291) CVE-2022-21724 and GHSA-673j-qm5f-xpv8

2022-02-22 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-38291: Summary: CVE-2022-21724 and GHSA-673j-qm5f-xpv8 (was: Upgrade postgresql from 42.3.0 to

[jira] [Updated] (SPARK-38291) Upgrade postgresql from 42.3.0 to 42.3.3

2022-02-22 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-38291: Description: postgresql 42.3.0 has two security issues.

[jira] [Updated] (SPARK-38291) Upgrade postgresql from 42.3.0 to 42.3.3

2022-02-22 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-38291: Description: postgresql 42.3.0 have two security issues.

[jira] [Commented] (SPARK-38291) Upgrade postgresql from 42.3.0 to 42.3.3

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496222#comment-17496222 ] Apache Spark commented on SPARK-38291: -- User 'bjornjorgensen' has created a pull request for this

[jira] [Assigned] (SPARK-38291) Upgrade postgresql from 42.3.0 to 42.3.3

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38291: Assignee: (was: Apache Spark) > Upgrade postgresql from 42.3.0 to 42.3.3 >

[jira] [Commented] (SPARK-38291) Upgrade postgresql from 42.3.0 to 42.3.3

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496219#comment-17496219 ] Apache Spark commented on SPARK-38291: -- User 'bjornjorgensen' has created a pull request for this

[jira] [Assigned] (SPARK-38291) Upgrade postgresql from 42.3.0 to 42.3.3

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38291: Assignee: Apache Spark > Upgrade postgresql from 42.3.0 to 42.3.3 >

[jira] [Resolved] (SPARK-38290) Fix JsonSuite and ParquetIOSuite under ANSI mode

2022-02-22 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-38290. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35611

[jira] [Created] (SPARK-38291) Upgrade postgresql from 42.3.0 to 42.3.3

2022-02-22 Thread Jira
Bjørn Jørgensen created SPARK-38291: --- Summary: Upgrade postgresql from 42.3.0 to 42.3.3 Key: SPARK-38291 URL: https://issues.apache.org/jira/browse/SPARK-38291 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-38273) Native memory leak in SparkPlan

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38273: Assignee: (was: Apache Spark) > Native memory leak in SparkPlan >

[jira] [Commented] (SPARK-38273) Native memory leak in SparkPlan

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496183#comment-17496183 ] Apache Spark commented on SPARK-38273: -- User 'kevins-29' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38273) Native memory leak in SparkPlan

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38273: Assignee: Apache Spark > Native memory leak in SparkPlan >

[jira] [Commented] (SPARK-38273) Native memory leak in SparkPlan

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496184#comment-17496184 ] Apache Spark commented on SPARK-38273: -- User 'kevins-29' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38060) Inconsistent behavior from JSON option allowNonNumericNumbers

2022-02-22 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-38060: Assignee: Andy Grove > Inconsistent behavior from JSON option allowNonNumericNumbers >

[jira] [Commented] (SPARK-22947) SPIP: as-of join in Spark SQL

2022-02-22 Thread Li Jin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496142#comment-17496142 ] Li Jin commented on SPARK-22947: For those who are interested, I will no longer on work on this SPIP.

[jira] [Resolved] (SPARK-38060) Inconsistent behavior from JSON option allowNonNumericNumbers

2022-02-22 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38060. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35573

[jira] [Assigned] (SPARK-38289) Refactor SQL CLI exit code related code

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38289: Assignee: (was: Apache Spark) > Refactor SQL CLI exit code related code >

[jira] [Assigned] (SPARK-38289) Refactor SQL CLI exit code related code

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38289: Assignee: Apache Spark > Refactor SQL CLI exit code related code >

[jira] [Commented] (SPARK-38289) Refactor SQL CLI exit code related code

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496140#comment-17496140 ] Apache Spark commented on SPARK-38289: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-38290) Fix JsonSuite and ParquetIOSuite under ANSI mode

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496119#comment-17496119 ] Apache Spark commented on SPARK-38290: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-38290) Fix JsonSuite and ParquetIOSuite under ANSI mode

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38290: Assignee: Apache Spark (was: Gengliang Wang) > Fix JsonSuite and ParquetIOSuite under

[jira] [Commented] (SPARK-38290) Fix JsonSuite and ParquetIOSuite under ANSI mode

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496118#comment-17496118 ] Apache Spark commented on SPARK-38290: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-38290) Fix JsonSuite and ParquetIOSuite under ANSI mode

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38290: Assignee: Gengliang Wang (was: Apache Spark) > Fix JsonSuite and ParquetIOSuite under

[jira] [Created] (SPARK-38290) Fix JsonSuite and ParquetIOSuite under ANSI mode

2022-02-22 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-38290: -- Summary: Fix JsonSuite and ParquetIOSuite under ANSI mode Key: SPARK-38290 URL: https://issues.apache.org/jira/browse/SPARK-38290 Project: Spark Issue

[jira] [Created] (SPARK-38289) Refactor SQL CLI exit code related code

2022-02-22 Thread angerszhu (Jira)
angerszhu created SPARK-38289: - Summary: Refactor SQL CLI exit code related code Key: SPARK-38289 URL: https://issues.apache.org/jira/browse/SPARK-38289 Project: Spark Issue Type: Task

[jira] [Resolved] (SPARK-38271) PoissonSampler may output more rows than MaxRows

2022-02-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38271. - Fix Version/s: 3.3.0 3.2.2 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-38271) PoissonSampler may output more rows than MaxRows

2022-02-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-38271: --- Assignee: zhengruifeng > PoissonSampler may output more rows than MaxRows >

[jira] [Updated] (SPARK-38288) Aggregate push down doesnt work using Spark SQL jdbc datasource with postgresql

2022-02-22 Thread Luis Lozano Coira (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luis Lozano Coira updated SPARK-38288: -- Description: I am establishing a connection with postgresql using the Spark SQL jdbc

[jira] [Created] (SPARK-38288) Aggregate push down doesnt work using Spark SQL jdbc datasource with postgresql

2022-02-22 Thread Luis Lozano Coira (Jira)
Luis Lozano Coira created SPARK-38288: - Summary: Aggregate push down doesnt work using Spark SQL jdbc datasource with postgresql Key: SPARK-38288 URL: https://issues.apache.org/jira/browse/SPARK-38288

[jira] [Created] (SPARK-38287) Upgrade h2 from 2.0.204 to 2.1.210 in /sql/core

2022-02-22 Thread Jira
Bjørn Jørgensen created SPARK-38287: --- Summary: Upgrade h2 from 2.0.204 to 2.1.210 in /sql/core Key: SPARK-38287 URL: https://issues.apache.org/jira/browse/SPARK-38287 Project: Spark Issue

[jira] [Assigned] (SPARK-38286) check Union's maxRows and maxRowsPerPartition

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38286: Assignee: (was: Apache Spark) > check Union's maxRows and maxRowsPerPartition >

[jira] [Commented] (SPARK-38286) check Union's maxRows and maxRowsPerPartition

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496041#comment-17496041 ] Apache Spark commented on SPARK-38286: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-38286) check Union's maxRows and maxRowsPerPartition

2022-02-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38286: Assignee: Apache Spark > check Union's maxRows and maxRowsPerPartition >

[jira] [Updated] (SPARK-38286) check Union's maxRows and maxRowsPerPartition

2022-02-22 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-38286: - Description: {code:java} scala> val df1 = spark.range(0, Long.MaxValue, 1, 1) df1:

[jira] [Created] (SPARK-38286) check Union's maxRows and maxRowsPerPartition

2022-02-22 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-38286: Summary: check Union's maxRows and maxRowsPerPartition Key: SPARK-38286 URL: https://issues.apache.org/jira/browse/SPARK-38286 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-37422) Inline type hints for python/pyspark/mllib/feature.py

2022-02-22 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37422. Fix Version/s: 3.3.0 Assignee: Maciej Szymkiewicz Resolution:

  1   2   >