[jira] [Assigned] (SPARK-40369) Migrate the type check failures of calls via reflection onto error classes

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40369: Assignee: (was: Apache Spark) > Migrate the type check failures of calls via

[jira] [Assigned] (SPARK-40369) Migrate the type check failures of calls via reflection onto error classes

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40369: Assignee: Apache Spark > Migrate the type check failures of calls via reflection onto

[jira] [Commented] (SPARK-40369) Migrate the type check failures of calls via reflection onto error classes

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619239#comment-17619239 ] Apache Spark commented on SPARK-40369: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-40828) Drop Python test tables before and after unit tests

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40828: Assignee: Apache Spark > Drop Python test tables before and after unit tests >

[jira] [Commented] (SPARK-40828) Drop Python test tables before and after unit tests

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619226#comment-17619226 ] Apache Spark commented on SPARK-40828: -- User 'amaliujia' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40828) Drop Python test tables before and after unit tests

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40828: Assignee: (was: Apache Spark) > Drop Python test tables before and after unit tests

[jira] [Created] (SPARK-40828) Drop Python test tables before and after unit tests

2022-10-17 Thread Rui Wang (Jira)
Rui Wang created SPARK-40828: Summary: Drop Python test tables before and after unit tests Key: SPARK-40828 URL: https://issues.apache.org/jira/browse/SPARK-40828 Project: Spark Issue Type:

[jira] [Created] (SPARK-40827) Re-enable the DataFrame.corrwith test after fixing in future pandas.

2022-10-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-40827: --- Summary: Re-enable the DataFrame.corrwith test after fixing in future pandas. Key: SPARK-40827 URL: https://issues.apache.org/jira/browse/SPARK-40827 Project: Spark

[jira] [Commented] (SPARK-40779) Fix `corrwith` to work properly with different anchor.

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619224#comment-17619224 ] Apache Spark commented on SPARK-40779: -- User 'itholic' has created a pull request for this issue:

[jira] [Resolved] (SPARK-40774) Add Sample to proto and DSL

2022-10-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-40774. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38227

[jira] [Assigned] (SPARK-40774) Add Sample to proto and DSL

2022-10-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-40774: --- Assignee: Rui Wang > Add Sample to proto and DSL > --- > >

[jira] [Assigned] (SPARK-40779) Fix `corrwith` to work properly with different anchor.

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40779: Assignee: (was: Apache Spark) > Fix `corrwith` to work properly with different

[jira] [Assigned] (SPARK-40779) Fix `corrwith` to work properly with different anchor.

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40779: Assignee: Apache Spark > Fix `corrwith` to work properly with different anchor. >

[jira] [Commented] (SPARK-40793) Row-level Runtime Filtering cannot be enabled when externalTable has no stats

2022-10-17 Thread chenminghua (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619211#comment-17619211 ] chenminghua commented on SPARK-40793: - I think a better way to improve this problem: when the

[jira] [Assigned] (SPARK-40826) Add additional checkpoint rename file check

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40826: Assignee: Apache Spark > Add additional checkpoint rename file check >

[jira] [Commented] (SPARK-40826) Add additional checkpoint rename file check

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619208#comment-17619208 ] Apache Spark commented on SPARK-40826: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-40826) Add additional checkpoint rename file check

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619207#comment-17619207 ] Apache Spark commented on SPARK-40826: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40826) Add additional checkpoint rename file check

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40826: Assignee: (was: Apache Spark) > Add additional checkpoint rename file check >

[jira] [Created] (SPARK-40826) Add additional checkpoint rename file check

2022-10-17 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-40826: --- Summary: Add additional checkpoint rename file check Key: SPARK-40826 URL: https://issues.apache.org/jira/browse/SPARK-40826 Project: Spark Issue Type:

[jira] [Commented] (SPARK-40799) Enforce Scalafmt for Spark Connect Module

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619202#comment-17619202 ] Apache Spark commented on SPARK-40799: -- User 'amaliujia' has created a pull request for this issue:

[jira] [Commented] (SPARK-40799) Enforce Scalafmt for Spark Connect Module

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619200#comment-17619200 ] Apache Spark commented on SPARK-40799: -- User 'amaliujia' has created a pull request for this issue:

[jira] [Commented] (SPARK-40756) Migrate type check failures of string expressions onto error classes

2022-10-17 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619198#comment-17619198 ] BingKun Pan commented on SPARK-40756: - I will work on it. > Migrate type check failures of string

[jira] [Resolved] (SPARK-40743) StructType should contain a list of StructField and each field should have a name

2022-10-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-40743. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38200

[jira] [Assigned] (SPARK-40743) StructType should contain a list of StructField and each field should have a name

2022-10-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-40743: --- Assignee: Rui Wang > StructType should contain a list of StructField and each field should

[jira] [Commented] (SPARK-40563) Error at where clause, when sql case executes by else branch

2022-10-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619189#comment-17619189 ] Yuming Wang commented on SPARK-40563: - Thank you [~Zing] > Error at where clause, when sql case

[jira] [Comment Edited] (SPARK-40802) Enhance JDBC Connector to use PreparedStatement.getMetaData() to resolve schema instead of PreparedStatement.executeQuery()

2022-10-17 Thread Mingli Rui (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619124#comment-17619124 ] Mingli Rui edited comment on SPARK-40802 at 10/18/22 1:38 AM: -- Hi,

[jira] [Resolved] (SPARK-40818) Add Intersect to Connect proto and DSL

2022-10-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40818. -- Resolution: Duplicate > Add Intersect to Connect proto and DSL >

[jira] [Created] (SPARK-40825) Topic-partition in offset deleted before commiting

2022-10-17 Thread Nirakar (Jira)
Nirakar created SPARK-40825: --- Summary: Topic-partition in offset deleted before commiting Key: SPARK-40825 URL: https://issues.apache.org/jira/browse/SPARK-40825 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-40824) Certain aggregations cause extra exchange steps on unioned and bucketed tables

2022-10-17 Thread Michael Tong (Jira)
Michael Tong created SPARK-40824: Summary: Certain aggregations cause extra exchange steps on unioned and bucketed tables Key: SPARK-40824 URL: https://issues.apache.org/jira/browse/SPARK-40824

[jira] [Commented] (SPARK-40802) Enhance JDBC Connector to use PreparedStatement.getMetaData() to resolve schema instead of PreparedStatement.executeQuery()

2022-10-17 Thread Mingli Rui (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619124#comment-17619124 ] Mingli Rui commented on SPARK-40802: Hi, [~hyukjin.kwon], Could you please explain more about +We

[jira] [Assigned] (SPARK-40823) Connect Proto should carry unparsed identifiers

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40823: Assignee: Apache Spark > Connect Proto should carry unparsed identifiers >

[jira] [Commented] (SPARK-40823) Connect Proto should carry unparsed identifiers

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619117#comment-17619117 ] Apache Spark commented on SPARK-40823: -- User 'amaliujia' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40823) Connect Proto should carry unparsed identifiers

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40823: Assignee: (was: Apache Spark) > Connect Proto should carry unparsed identifiers >

[jira] [Created] (SPARK-40823) Connect Proto should carry unparsed identifiers

2022-10-17 Thread Rui Wang (Jira)
Rui Wang created SPARK-40823: Summary: Connect Proto should carry unparsed identifiers Key: SPARK-40823 URL: https://issues.apache.org/jira/browse/SPARK-40823 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-40821) Fix late record filtering to support chaining of steteful operators

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619114#comment-17619114 ] Apache Spark commented on SPARK-40821: -- User 'alex-balikov' has created a pull request for this

[jira] [Assigned] (SPARK-40821) Fix late record filtering to support chaining of steteful operators

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40821: Assignee: (was: Apache Spark) > Fix late record filtering to support chaining of

[jira] [Assigned] (SPARK-40821) Fix late record filtering to support chaining of steteful operators

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40821: Assignee: Apache Spark > Fix late record filtering to support chaining of steteful

[jira] [Commented] (SPARK-40821) Fix late record filtering to support chaining of steteful operators

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619113#comment-17619113 ] Apache Spark commented on SPARK-40821: -- User 'alex-balikov' has created a pull request for this

[jira] [Commented] (SPARK-40796) Check the generated python protos in GitHub Actions

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619083#comment-17619083 ] Apache Spark commented on SPARK-40796: -- User 'amaliujia' has created a pull request for this issue:

[jira] [Commented] (SPARK-40796) Check the generated python protos in GitHub Actions

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619082#comment-17619082 ] Apache Spark commented on SPARK-40796: -- User 'amaliujia' has created a pull request for this issue:

[jira] [Commented] (SPARK-40802) Enhance JDBC Connector to use PreparedStatement.getMetaData() to resolve schema instead of PreparedStatement.executeQuery()

2022-10-17 Thread Mingli Rui (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619043#comment-17619043 ] Mingli Rui commented on SPARK-40802: [~hyukjin.kwon] Thanks for the comment! Do we know any JDBC

[jira] [Created] (SPARK-40822) Use stable derived-column-alias algorithm, suitable for CREATE VIEW

2022-10-17 Thread Serge Rielau (Jira)
Serge Rielau created SPARK-40822: Summary: Use stable derived-column-alias algorithm, suitable for CREATE VIEW Key: SPARK-40822 URL: https://issues.apache.org/jira/browse/SPARK-40822 Project: Spark

[jira] [Created] (SPARK-40821) Fix late record filtering to support chaining of steteful operators

2022-10-17 Thread Alex Balikov (Jira)
Alex Balikov created SPARK-40821: Summary: Fix late record filtering to support chaining of steteful operators Key: SPARK-40821 URL: https://issues.apache.org/jira/browse/SPARK-40821 Project: Spark

[jira] [Assigned] (SPARK-40657) Add support for compiled classes (Java classes)

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40657: Assignee: Apache Spark > Add support for compiled classes (Java classes) >

[jira] [Commented] (SPARK-40657) Add support for compiled classes (Java classes)

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619012#comment-17619012 ] Apache Spark commented on SPARK-40657: -- User 'rangadi' has created a pull request for this issue:

[jira] [Commented] (SPARK-40657) Add support for compiled classes (Java classes)

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17619013#comment-17619013 ] Apache Spark commented on SPARK-40657: -- User 'rangadi' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40657) Add support for compiled classes (Java classes)

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40657: Assignee: (was: Apache Spark) > Add support for compiled classes (Java classes) >

[jira] [Updated] (SPARK-29729) Upgrade ASM to 7.2

2022-10-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-29729: --- Description: this patch is required for spark to build with any version of bouncy castle

[jira] [Updated] (SPARK-40563) Error at where clause, when sql case executes by else branch

2022-10-17 Thread zzzzming95 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ming95 updated SPARK-40563: --- Fix Version/s: 3.4.0 (was: 3.3.1) > Error at where clause, when sql case

[jira] [Commented] (SPARK-40820) Creating StructType from Json

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17618976#comment-17618976 ] Apache Spark commented on SPARK-40820: -- User 'anthonywainer' has created a pull request for this

[jira] [Assigned] (SPARK-40820) Creating StructType from Json

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40820: Assignee: Apache Spark > Creating StructType from Json > - >

[jira] [Assigned] (SPARK-40820) Creating StructType from Json

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40820: Assignee: (was: Apache Spark) > Creating StructType from Json >

[jira] [Commented] (SPARK-40820) Creating StructType from Json

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17618974#comment-17618974 ] Apache Spark commented on SPARK-40820: -- User 'anthonywainer' has created a pull request for this

[jira] [Commented] (SPARK-40563) Error at where clause, when sql case executes by else branch

2022-10-17 Thread zzzzming95 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17618973#comment-17618973 ] ming95 commented on SPARK-40563: [~yumwang]  The problem was not fixed in branch-3.3. > Error

[jira] [Updated] (SPARK-40820) Creating StructType from Json

2022-10-17 Thread Anthony Wainer Cachay Guivin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anthony Wainer Cachay Guivin updated SPARK-40820: - Summary: Creating StructType from Json (was: creating

[jira] [Updated] (SPARK-40820) creating StructType from Json

2022-10-17 Thread Anthony Wainer Cachay Guivin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anthony Wainer Cachay Guivin updated SPARK-40820: - Language: Python > creating StructType from Json >

[jira] [Updated] (SPARK-38591) Add flatMapSortedGroups and cogroupSorted to KeyValueGroupedDataset

2022-10-17 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-38591: -- Description: The existing methods {{KeyValueGroupedDataset.flatMapGroups}} and

[jira] [Updated] (SPARK-40820) creating StructType from Json

2022-10-17 Thread Anthony Wainer Cachay Guivin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anthony Wainer Cachay Guivin updated SPARK-40820: - Attachment: (was: bug .png) > creating StructType from Json

[jira] [Updated] (SPARK-40820) creating StructType from Json

2022-10-17 Thread Anthony Wainer Cachay Guivin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anthony Wainer Cachay Guivin updated SPARK-40820: - Description: When create a StructType from a Python dictionary

[jira] [Updated] (SPARK-40820) creating StructType from Json

2022-10-17 Thread Anthony Wainer Cachay Guivin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anthony Wainer Cachay Guivin updated SPARK-40820: - Description: When create a StructType from a Python dictionary

[jira] [Updated] (SPARK-40820) creating StructType from Json

2022-10-17 Thread Anthony Wainer Cachay Guivin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anthony Wainer Cachay Guivin updated SPARK-40820: - Description: When create a StructType from a Python dictionary

[jira] [Updated] (SPARK-40820) creating StructType from Json

2022-10-17 Thread Anthony Wainer Cachay Guivin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anthony Wainer Cachay Guivin updated SPARK-40820: - Attachment: bug .png > creating StructType from Json >

[jira] [Created] (SPARK-40820) creating StructType from Json

2022-10-17 Thread Anthony Wainer Cachay Guivin (Jira)
Anthony Wainer Cachay Guivin created SPARK-40820: Summary: creating StructType from Json Key: SPARK-40820 URL: https://issues.apache.org/jira/browse/SPARK-40820 Project: Spark

[jira] [Assigned] (SPARK-40799) Enforce Scalafmt for Spark Connect Module

2022-10-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40799: Assignee: Martin Grund > Enforce Scalafmt for Spark Connect Module >

[jira] [Resolved] (SPARK-40799) Enforce Scalafmt for Spark Connect Module

2022-10-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40799. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38258

[jira] [Commented] (SPARK-40808) Infer schema for CSV files - wrong behavior using header + merge schema

2022-10-17 Thread ohad (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17618839#comment-17618839 ] ohad commented on SPARK-40808: -- [~Zing] [~hyukjin.kwon]  File test_csv.py attached As you can see the

[jira] [Updated] (SPARK-40808) Infer schema for CSV files - wrong behavior using header + merge schema

2022-10-17 Thread ohad (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ohad updated SPARK-40808: - Attachment: test_csv.py > Infer schema for CSV files - wrong behavior using header + merge schema >

[jira] [Resolved] (SPARK-40556) Unpersist the intermediate datasets cached in AttachDistributedSequenceExec

2022-10-17 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-40556. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38130

[jira] [Assigned] (SPARK-40556) Unpersist the intermediate datasets cached in AttachDistributedSequenceExec

2022-10-17 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-40556: - Assignee: Ruifeng Zheng > Unpersist the intermediate datasets cached in

[jira] [Created] (SPARK-40819) Parquet INT64 (TIMESTAMP(NANOS,true)) now throwing Illegal Parquet type instead of automatically converting to LongType

2022-10-17 Thread Alfred Davidson (Jira)
Alfred Davidson created SPARK-40819: --- Summary: Parquet INT64 (TIMESTAMP(NANOS,true)) now throwing Illegal Parquet type instead of automatically converting to LongType Key: SPARK-40819 URL:

[jira] [Commented] (SPARK-40796) Check the generated python protos in GitHub Actions

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17618824#comment-17618824 ] Apache Spark commented on SPARK-40796: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-40796) Check the generated python protos in GitHub Actions

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17618825#comment-17618825 ] Apache Spark commented on SPARK-40796: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-40818) Add Intersect to Connect proto and DSL

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40818: Assignee: (was: Apache Spark) > Add Intersect to Connect proto and DSL >

[jira] [Assigned] (SPARK-40818) Add Intersect to Connect proto and DSL

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40818: Assignee: Apache Spark > Add Intersect to Connect proto and DSL >

[jira] [Commented] (SPARK-40818) Add Intersect to Connect proto and DSL

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17618808#comment-17618808 ] Apache Spark commented on SPARK-40818: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-40803) LZ4CompressionCodec looks up configuration on each stream creation

2022-10-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-40803: - Assignee: Emil Ejbyfeldt > LZ4CompressionCodec looks up configuration on each stream

[jira] [Resolved] (SPARK-40803) LZ4CompressionCodec looks up configuration on each stream creation

2022-10-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40803. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38282

[jira] [Created] (SPARK-40818) Add Intersect to Connect proto and DSL

2022-10-17 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-40818: Summary: Add Intersect to Connect proto and DSL Key: SPARK-40818 URL: https://issues.apache.org/jira/browse/SPARK-40818 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-40646) Fix returning partial results in JSON data source and JSON functions

2022-10-17 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-40646. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38090

[jira] [Assigned] (SPARK-40646) Fix returning partial results in JSON data source and JSON functions

2022-10-17 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-40646: Assignee: Ivan Sadikov > Fix returning partial results in JSON data source and JSON functions >

[jira] [Created] (SPARK-40817) Remote spark.jars URIs ignored for Spark on Kubernetes in cluster mode

2022-10-17 Thread Anton Ippolitov (Jira)
Anton Ippolitov created SPARK-40817: --- Summary: Remote spark.jars URIs ignored for Spark on Kubernetes in cluster mode Key: SPARK-40817 URL: https://issues.apache.org/jira/browse/SPARK-40817

[jira] [Updated] (SPARK-40817) Remote spark.jars URIs ignored for Spark on Kubernetes in cluster mode

2022-10-17 Thread Anton Ippolitov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Ippolitov updated SPARK-40817: Description: I discovered that remote URIs in {{spark.jars}} get discarded when launching

[jira] [Updated] (SPARK-40817) Remote spark.jars URIs ignored for Spark on Kubernetes in cluster mode

2022-10-17 Thread Anton Ippolitov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Ippolitov updated SPARK-40817: Attachment: image-2022-10-17-10-44-46-862.png > Remote spark.jars URIs ignored for Spark

[jira] [Resolved] (SPARK-40790) Check error classes in DDL parsing tests

2022-10-17 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-40790. -- Resolution: Fixed Issue resolved by pull request 38280 [https://github.com/apache/spark/pull/38280]

[jira] [Assigned] (SPARK-40790) Check error classes in DDL parsing tests

2022-10-17 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-40790: Assignee: BingKun Pan > Check error classes in DDL parsing tests >

[jira] [Resolved] (SPARK-40528) Add dockerfile template

2022-10-17 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40528. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 12

[jira] [Resolved] (SPARK-40805) Use `spark` username in official image

2022-10-17 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40805. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 11

[jira] [Updated] (SPARK-40793) Row-level Runtime Filtering cannot be enabled when externalTable has no stats

2022-10-17 Thread chenminghua (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenminghua updated SPARK-40793: Description: When using external tables, Row-level Runtime Filtering cannot be enabled anyway

[jira] [Commented] (SPARK-40803) LZ4CompressionCodec looks up configuration on each stream creation

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17618682#comment-17618682 ] Apache Spark commented on SPARK-40803: -- User 'eejbyfeldt' has created a pull request for this

[jira] [Assigned] (SPARK-40803) LZ4CompressionCodec looks up configuration on each stream creation

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40803: Assignee: Apache Spark > LZ4CompressionCodec looks up configuration on each stream

[jira] [Commented] (SPARK-40803) LZ4CompressionCodec looks up configuration on each stream creation

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17618681#comment-17618681 ] Apache Spark commented on SPARK-40803: -- User 'eejbyfeldt' has created a pull request for this

[jira] [Assigned] (SPARK-40803) LZ4CompressionCodec looks up configuration on each stream creation

2022-10-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40803: Assignee: (was: Apache Spark) > LZ4CompressionCodec looks up configuration on each

[jira] [Assigned] (SPARK-40796) Check the generated python protos in GitHub Actions

2022-10-17 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-40796: - Assignee: Ruifeng Zheng > Check the generated python protos in GitHub Actions >

[jira] [Resolved] (SPARK-40796) Check the generated python protos in GitHub Actions

2022-10-17 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-40796. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38253