[jira] [Created] (SPARK-32518) CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources

wuyi created SPARK-32518: Summary: CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources Key: SPARK-32518 URL: https://issues.apache.org/jira/browse/SPARK-32518 Project:

[jira] [Updated] (SPARK-32495) Update jackson-databind versions to fix various vulnerabilities.

[ https://issues.apache.org/jira/browse/SPARK-32495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-32495: Summary: Update jackson-databind versions to fix various vulnerabilities. (was: Update

[jira] [Commented] (SPARK-32517) Add StorageLevel.DISK_ONLY_3

[ https://issues.apache.org/jira/browse/SPARK-32517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169739#comment-17169739 ] Apache Spark commented on SPARK-32517: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-32517) Add StorageLevel.DISK_ONLY_3

[ https://issues.apache.org/jira/browse/SPARK-32517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32517: Assignee: (was: Apache Spark) > Add StorageLevel.DISK_ONLY_3 >

[jira] [Assigned] (SPARK-32517) Add StorageLevel.DISK_ONLY_3

[ https://issues.apache.org/jira/browse/SPARK-32517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32517: Assignee: Apache Spark > Add StorageLevel.DISK_ONLY_3 > > >

[jira] [Commented] (SPARK-31754) Spark Structured Streaming: NullPointerException in Stream Stream join

[ https://issues.apache.org/jira/browse/SPARK-31754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169736#comment-17169736 ] fritz commented on SPARK-31754: --- Thanks for your response [~puviarasu] Yes. Batch processing is not have

[jira] [Commented] (SPARK-32432) Add support for reading ORC/Parquet files with SymlinkTextInputFormat

[ https://issues.apache.org/jira/browse/SPARK-32432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169734#comment-17169734 ] Apache Spark commented on SPARK-32432: -- User 'moomindani' has created a pull request for this

[jira] [Created] (SPARK-32517) Add StorageLevel.DISK_ONLY_3

Dongjoon Hyun created SPARK-32517: - Summary: Add StorageLevel.DISK_ONLY_3 Key: SPARK-32517 URL: https://issues.apache.org/jira/browse/SPARK-32517 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-32432) Add support for reading ORC/Parquet files with SymlinkTextInputFormat

[ https://issues.apache.org/jira/browse/SPARK-32432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169733#comment-17169733 ] Apache Spark commented on SPARK-32432: -- User 'moomindani' has created a pull request for this

[jira] [Assigned] (SPARK-32432) Add support for reading ORC/Parquet files with SymlinkTextInputFormat

[ https://issues.apache.org/jira/browse/SPARK-32432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32432: Assignee: Apache Spark > Add support for reading ORC/Parquet files with

[jira] [Assigned] (SPARK-32432) Add support for reading ORC/Parquet files with SymlinkTextInputFormat

[ https://issues.apache.org/jira/browse/SPARK-32432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32432: Assignee: (was: Apache Spark) > Add support for reading ORC/Parquet files with

[jira] [Commented] (SPARK-31754) Spark Structured Streaming: NullPointerException in Stream Stream join

[ https://issues.apache.org/jira/browse/SPARK-31754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169727#comment-17169727 ] Puviarasu commented on SPARK-31754: --- Hello [~fritzwijaya],  We still have the issue as of

[jira] [Resolved] (SPARK-32509) Unused DPP Filter causes issue in canonicalization and prevents reuse exchange

[ https://issues.apache.org/jira/browse/SPARK-32509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32509. - Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-32509) Unused DPP Filter causes issue in canonicalization and prevents reuse exchange

[ https://issues.apache.org/jira/browse/SPARK-32509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32509: --- Assignee: Prakhar Jain > Unused DPP Filter causes issue in canonicalization and prevents

[jira] [Resolved] (SPARK-32510) JDBC doesn't check duplicate column names in nested structures

[ https://issues.apache.org/jira/browse/SPARK-32510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32510. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29317

[jira] [Assigned] (SPARK-32510) JDBC doesn't check duplicate column names in nested structures

[ https://issues.apache.org/jira/browse/SPARK-32510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32510: --- Assignee: Maxim Gekk > JDBC doesn't check duplicate column names in nested structures >

[jira] [Resolved] (SPARK-32274) Add in the ability for a user to replace the serialization format of the cache

[ https://issues.apache.org/jira/browse/SPARK-32274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32274. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29067

[jira] [Assigned] (SPARK-32274) Add in the ability for a user to replace the serialization format of the cache

[ https://issues.apache.org/jira/browse/SPARK-32274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32274: --- Assignee: Robert Joseph Evans > Add in the ability for a user to replace the serialization

[jira] [Commented] (SPARK-31754) Spark Structured Streaming: NullPointerException in Stream Stream join

[ https://issues.apache.org/jira/browse/SPARK-31754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169677#comment-17169677 ] fritz commented on SPARK-31754: --- Hi [~puviarasu] [~kabhwan], recently facing similar issue with the NPE

[jira] [Commented] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169673#comment-17169673 ] Takeshi Yamamuro commented on SPARK-32515: -- Thanks for the report. I updated some fields (e.g.,

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-32515: - Component/s: (was: PySpark) SQL > Distinct Function Weird Bug >

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-32515: - Target Version/s: (was: 2.4.6) > Distinct Function Weird Bug >

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-32515: - Labels: (was: distinct groupby load read) > Distinct Function Weird Bug >

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-32515: - Priority: Major (was: Blocker) > Distinct Function Weird Bug >

[jira] [Updated] (SPARK-28818) FrequentItems applies an incorrect schema to the resulting dataframe when nulls are present

[ https://issues.apache.org/jira/browse/SPARK-28818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-28818: - Fix Version/s: 2.4.7 > FrequentItems applies an incorrect schema to the resulting

[jira] [Resolved] (SPARK-32490) Upgrade netty-all to 4.1.51.Final

[ https://issues.apache.org/jira/browse/SPARK-32490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32490. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29299

[jira] [Assigned] (SPARK-32490) Upgrade netty-all to 4.1.51.Final

[ https://issues.apache.org/jira/browse/SPARK-32490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32490: - Assignee: Yang Jie > Upgrade netty-all to 4.1.51.Final >

[jira] [Commented] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169658#comment-17169658 ] Jayce Jiang commented on SPARK-32515: -

[jira] [Commented] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169657#comment-17169657 ] Jayce Jiang commented on SPARK-32515: - Hey, here is a sample data. Please have a try. > Distinct

[jira] [Comment Edited] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169653#comment-17169653 ] JinxinTang edited comment on SPARK-32515 at 8/2/20, 11:09 PM: -- Hi

[jira] [Comment Edited] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169653#comment-17169653 ] JinxinTang edited comment on SPARK-32515 at 8/2/20, 11:08 PM: -- Hi

[jira] [Comment Edited] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169653#comment-17169653 ] JinxinTang edited comment on SPARK-32515 at 8/2/20, 11:08 PM: -- Hi

[jira] [Commented] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169653#comment-17169653 ] JinxinTang commented on SPARK-32515: Hi [~tigaiii123],  Could you please try the data or could you

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-32515: --- Attachment: image-2020-08-03-07-03-55-716.png > Distinct Function Weird Bug >

[jira] [Commented] (SPARK-32516) path option is treated differently for 'format("parquet").load(path)' vs. 'parquet(path)'

[ https://issues.apache.org/jira/browse/SPARK-32516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169639#comment-17169639 ] Apache Spark commented on SPARK-32516: -- User 'imback82' has created a pull request for this issue:

[jira] [Commented] (SPARK-32516) path option is treated differently for 'format("parquet").load(path)' vs. 'parquet(path)'

[ https://issues.apache.org/jira/browse/SPARK-32516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169638#comment-17169638 ] Apache Spark commented on SPARK-32516: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32516) path option is treated differently for 'format("parquet").load(path)' vs. 'parquet(path)'

[ https://issues.apache.org/jira/browse/SPARK-32516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32516: Assignee: Apache Spark > path option is treated differently for

[jira] [Assigned] (SPARK-32516) path option is treated differently for 'format("parquet").load(path)' vs. 'parquet(path)'

[ https://issues.apache.org/jira/browse/SPARK-32516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32516: Assignee: (was: Apache Spark) > path option is treated differently for

[jira] [Created] (SPARK-32516) path option is treated differently for 'format("parquet").load(path)' vs. 'parquet(path)'

Terry Kim created SPARK-32516: - Summary: path option is treated differently for 'format("parquet").load(path)' vs. 'parquet(path)' Key: SPARK-32516 URL: https://issues.apache.org/jira/browse/SPARK-32516

[jira] [Commented] (SPARK-32427) Omit USING in CREATE TABLE via JDBC Table Catalog

[ https://issues.apache.org/jira/browse/SPARK-32427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169610#comment-17169610 ] L. C. Hsieh commented on SPARK-32427: - Do you mean "CREATE TABLE .." without USING indicates using

[jira] [Updated] (SPARK-22231) Support of map, filter, withField, dropFields in nested list of structures

[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-22231: Summary: Support of map, filter, withField, dropFields in nested list of structures (was: Support of

[jira] [Commented] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169574#comment-17169574 ] Jayce Jiang commented on SPARK-32515: - I re-uploaded, as you can see, the CSV file gets loaded into

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayce Jiang updated SPARK-32515: Attachment: Capture1.png > Distinct Function Weird Bug > --- > >

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayce Jiang updated SPARK-32515: Attachment: Capture2.PNG > Distinct Function Weird Bug > --- > >

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayce Jiang updated SPARK-32515: Attachment: Capture.PNG > Distinct Function Weird Bug > --- > >

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayce Jiang updated SPARK-32515: Attachment: unknown2.png unknown1.png unknown.png > Distinct

[jira] [Commented] (SPARK-28818) FrequentItems applies an incorrect schema to the resulting dataframe when nulls are present

[ https://issues.apache.org/jira/browse/SPARK-28818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169501#comment-17169501 ] Apache Spark commented on SPARK-28818: -- User 'maropu' has created a pull request for this issue:

[jira] [Commented] (SPARK-28818) FrequentItems applies an incorrect schema to the resulting dataframe when nulls are present

[ https://issues.apache.org/jira/browse/SPARK-28818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169500#comment-17169500 ] Apache Spark commented on SPARK-28818: -- User 'maropu' has created a pull request for this issue:

[jira] [Commented] (SPARK-32502) Please fix CVE related to Guava 14.0.1

[ https://issues.apache.org/jira/browse/SPARK-32502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169466#comment-17169466 ] Apache Spark commented on SPARK-32502: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-32502) Please fix CVE related to Guava 14.0.1

[ https://issues.apache.org/jira/browse/SPARK-32502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169464#comment-17169464 ] Apache Spark commented on SPARK-32502: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32502) Please fix CVE related to Guava 14.0.1

[ https://issues.apache.org/jira/browse/SPARK-32502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32502: Assignee: (was: Apache Spark) > Please fix CVE related to Guava 14.0.1 >

[jira] [Assigned] (SPARK-32502) Please fix CVE related to Guava 14.0.1

[ https://issues.apache.org/jira/browse/SPARK-32502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32502: Assignee: Apache Spark > Please fix CVE related to Guava 14.0.1 >

[jira] [Commented] (SPARK-32502) Please fix CVE related to Guava 14.0.1

[ https://issues.apache.org/jira/browse/SPARK-32502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169465#comment-17169465 ] Apache Spark commented on SPARK-32502: -- User 'viirya' has created a pull request for this issue: