[jira] [Commented] (SPARK-32495) Update jackson-databind versions to fix various vulnerabilities.

[ https://issues.apache.org/jira/browse/SPARK-32495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169756#comment-17169756 ] Prashant Sharma commented on SPARK-32495: - As per the issues and commit: [https

[jira] [Commented] (SPARK-32518) CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources

[ https://issues.apache.org/jira/browse/SPARK-32518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169753#comment-17169753 ] Apache Spark commented on SPARK-32518: -- User 'Ngone51' has created a pull request f

[jira] [Assigned] (SPARK-32518) CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources

[ https://issues.apache.org/jira/browse/SPARK-32518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32518: Assignee: Apache Spark > CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consi

[jira] [Commented] (SPARK-32518) CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources

[ https://issues.apache.org/jira/browse/SPARK-32518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169752#comment-17169752 ] Apache Spark commented on SPARK-32518: -- User 'Ngone51' has created a pull request f

[jira] [Assigned] (SPARK-32518) CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources

[ https://issues.apache.org/jira/browse/SPARK-32518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32518: Assignee: (was: Apache Spark) > CoarseGrainedSchedulerBackend.maxNumConcurrentTasks s

[jira] [Commented] (SPARK-31754) Spark Structured Streaming: NullPointerException in Stream Stream join

[ https://issues.apache.org/jira/browse/SPARK-31754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169746#comment-17169746 ] Puviarasu commented on SPARK-31754: --- Hello [~fritzwijaya],  Restarting[after clearing

[jira] [Resolved] (SPARK-24884) Implement regexp_extract_all

[ https://issues.apache.org/jira/browse/SPARK-24884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24884. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27507 [https://gith

[jira] [Assigned] (SPARK-24884) Implement regexp_extract_all

[ https://issues.apache.org/jira/browse/SPARK-24884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-24884: --- Assignee: jiaan.geng > Implement regexp_extract_all > > >

[jira] [Created] (SPARK-32518) CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources

wuyi created SPARK-32518: Summary: CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources Key: SPARK-32518 URL: https://issues.apache.org/jira/browse/SPARK-32518 Project: S

[jira] [Updated] (SPARK-32495) Update jackson-databind versions to fix various vulnerabilities.

[ https://issues.apache.org/jira/browse/SPARK-32495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-32495: Summary: Update jackson-databind versions to fix various vulnerabilities. (was: Update ja

[jira] [Commented] (SPARK-32517) Add StorageLevel.DISK_ONLY_3

[ https://issues.apache.org/jira/browse/SPARK-32517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169739#comment-17169739 ] Apache Spark commented on SPARK-32517: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-32517) Add StorageLevel.DISK_ONLY_3

[ https://issues.apache.org/jira/browse/SPARK-32517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32517: Assignee: (was: Apache Spark) > Add StorageLevel.DISK_ONLY_3 > --

[jira] [Assigned] (SPARK-32517) Add StorageLevel.DISK_ONLY_3

[ https://issues.apache.org/jira/browse/SPARK-32517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32517: Assignee: Apache Spark > Add StorageLevel.DISK_ONLY_3 > > >

[jira] [Commented] (SPARK-31754) Spark Structured Streaming: NullPointerException in Stream Stream join

[ https://issues.apache.org/jira/browse/SPARK-31754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169736#comment-17169736 ] fritz commented on SPARK-31754: --- Thanks for your response [~puviarasu] Yes. Batch process

[jira] [Commented] (SPARK-32432) Add support for reading ORC/Parquet files with SymlinkTextInputFormat

[ https://issues.apache.org/jira/browse/SPARK-32432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169734#comment-17169734 ] Apache Spark commented on SPARK-32432: -- User 'moomindani' has created a pull reques

[jira] [Created] (SPARK-32517) Add StorageLevel.DISK_ONLY_3

Dongjoon Hyun created SPARK-32517: - Summary: Add StorageLevel.DISK_ONLY_3 Key: SPARK-32517 URL: https://issues.apache.org/jira/browse/SPARK-32517 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-32432) Add support for reading ORC/Parquet files with SymlinkTextInputFormat

[ https://issues.apache.org/jira/browse/SPARK-32432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32432: Assignee: Apache Spark > Add support for reading ORC/Parquet files with SymlinkTextInputF

[jira] [Assigned] (SPARK-32432) Add support for reading ORC/Parquet files with SymlinkTextInputFormat

[ https://issues.apache.org/jira/browse/SPARK-32432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32432: Assignee: (was: Apache Spark) > Add support for reading ORC/Parquet files with Symlin

[jira] [Commented] (SPARK-32432) Add support for reading ORC/Parquet files with SymlinkTextInputFormat

[ https://issues.apache.org/jira/browse/SPARK-32432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169733#comment-17169733 ] Apache Spark commented on SPARK-32432: -- User 'moomindani' has created a pull reques

[jira] [Commented] (SPARK-31754) Spark Structured Streaming: NullPointerException in Stream Stream join

[ https://issues.apache.org/jira/browse/SPARK-31754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169727#comment-17169727 ] Puviarasu commented on SPARK-31754: --- Hello [~fritzwijaya],  We still have the issue a

[jira] [Resolved] (SPARK-32509) Unused DPP Filter causes issue in canonicalization and prevents reuse exchange

[ https://issues.apache.org/jira/browse/SPARK-32509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32509. - Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue resolved by pull re

[jira] [Assigned] (SPARK-32509) Unused DPP Filter causes issue in canonicalization and prevents reuse exchange

[ https://issues.apache.org/jira/browse/SPARK-32509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32509: --- Assignee: Prakhar Jain > Unused DPP Filter causes issue in canonicalization and prevents re

[jira] [Resolved] (SPARK-32510) JDBC doesn't check duplicate column names in nested structures

[ https://issues.apache.org/jira/browse/SPARK-32510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32510. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29317 [https://gith

[jira] [Assigned] (SPARK-32510) JDBC doesn't check duplicate column names in nested structures

[ https://issues.apache.org/jira/browse/SPARK-32510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32510: --- Assignee: Maxim Gekk > JDBC doesn't check duplicate column names in nested structures > --

[jira] [Resolved] (SPARK-32274) Add in the ability for a user to replace the serialization format of the cache

[ https://issues.apache.org/jira/browse/SPARK-32274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32274. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29067 [https://gith

[jira] [Assigned] (SPARK-32274) Add in the ability for a user to replace the serialization format of the cache

[ https://issues.apache.org/jira/browse/SPARK-32274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32274: --- Assignee: Robert Joseph Evans > Add in the ability for a user to replace the serialization

[jira] [Commented] (SPARK-31754) Spark Structured Streaming: NullPointerException in Stream Stream join

[ https://issues.apache.org/jira/browse/SPARK-31754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169677#comment-17169677 ] fritz commented on SPARK-31754: --- Hi [~puviarasu] [~kabhwan], recently facing similar issue

[jira] [Commented] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169673#comment-17169673 ] Takeshi Yamamuro commented on SPARK-32515: -- Thanks for the report. I updated so

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-32515: - Component/s: (was: PySpark) SQL > Distinct Function Weird Bug > ---

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-32515: - Target Version/s: (was: 2.4.6) > Distinct Function Weird Bug > ---

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-32515: - Labels: (was: distinct groupby load read) > Distinct Function Weird Bug >

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-32515: - Fix Version/s: (was: 2.4.6) > Distinct Function Weird Bug >

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-32515: - Priority: Major (was: Blocker) > Distinct Function Weird Bug >

[jira] [Updated] (SPARK-28818) FrequentItems applies an incorrect schema to the resulting dataframe when nulls are present

[ https://issues.apache.org/jira/browse/SPARK-28818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-28818: - Fix Version/s: 2.4.7 > FrequentItems applies an incorrect schema to the resulting datafr

[jira] [Resolved] (SPARK-32490) Upgrade netty-all to 4.1.51.Final

[ https://issues.apache.org/jira/browse/SPARK-32490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32490. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29299 [https://

[jira] [Assigned] (SPARK-32490) Upgrade netty-all to 4.1.51.Final

[ https://issues.apache.org/jira/browse/SPARK-32490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32490: - Assignee: Yang Jie > Upgrade netty-all to 4.1.51.Final > --

[jira] [Commented] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169658#comment-17169658 ] Jayce Jiang commented on SPARK-32515: - https://drive.google.com/file/d/1542jrV1-qXiY

[jira] [Commented] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169657#comment-17169657 ] Jayce Jiang commented on SPARK-32515: - Hey, here is a sample data. Please have a try

[jira] [Comment Edited] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169653#comment-17169653 ] JinxinTang edited comment on SPARK-32515 at 8/2/20, 11:09 PM:

[jira] [Comment Edited] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169653#comment-17169653 ] JinxinTang edited comment on SPARK-32515 at 8/2/20, 11:08 PM:

[jira] [Comment Edited] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169653#comment-17169653 ] JinxinTang edited comment on SPARK-32515 at 8/2/20, 11:08 PM:

[jira] [Commented] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169653#comment-17169653 ] JinxinTang commented on SPARK-32515: Hi [~tigaiii123],  Could you please try the da

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-32515: --- Attachment: image-2020-08-03-07-03-55-716.png > Distinct Function Weird Bug > --

[jira] [Commented] (SPARK-32516) path option is treated differently for 'format("parquet").load(path)' vs. 'parquet(path)'

[ https://issues.apache.org/jira/browse/SPARK-32516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169639#comment-17169639 ] Apache Spark commented on SPARK-32516: -- User 'imback82' has created a pull request

[jira] [Assigned] (SPARK-32516) path option is treated differently for 'format("parquet").load(path)' vs. 'parquet(path)'

[ https://issues.apache.org/jira/browse/SPARK-32516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32516: Assignee: Apache Spark > path option is treated differently for 'format("parquet").load(p

[jira] [Assigned] (SPARK-32516) path option is treated differently for 'format("parquet").load(path)' vs. 'parquet(path)'

[ https://issues.apache.org/jira/browse/SPARK-32516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32516: Assignee: (was: Apache Spark) > path option is treated differently for 'format("parqu

[jira] [Commented] (SPARK-32516) path option is treated differently for 'format("parquet").load(path)' vs. 'parquet(path)'

[ https://issues.apache.org/jira/browse/SPARK-32516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169638#comment-17169638 ] Apache Spark commented on SPARK-32516: -- User 'imback82' has created a pull request

[jira] [Created] (SPARK-32516) path option is treated differently for 'format("parquet").load(path)' vs. 'parquet(path)'

Terry Kim created SPARK-32516: - Summary: path option is treated differently for 'format("parquet").load(path)' vs. 'parquet(path)' Key: SPARK-32516 URL: https://issues.apache.org/jira/browse/SPARK-32516 P

[jira] [Commented] (SPARK-32427) Omit USING in CREATE TABLE via JDBC Table Catalog

[ https://issues.apache.org/jira/browse/SPARK-32427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169610#comment-17169610 ] L. C. Hsieh commented on SPARK-32427: - Do you mean "CREATE TABLE .." without USING i

[jira] [Updated] (SPARK-22231) Support of map, filter, withField, dropFields in nested list of structures

[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-22231: Summary: Support of map, filter, withField, dropFields in nested list of structures (was: Support of map,

[jira] [Commented] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169574#comment-17169574 ] Jayce Jiang commented on SPARK-32515: - I re-uploaded, as you can see, the CSV file g

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayce Jiang updated SPARK-32515: Attachment: Capture1.png > Distinct Function Weird Bug > --- > >

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayce Jiang updated SPARK-32515: Attachment: Capture2.PNG > Distinct Function Weird Bug > --- > >

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayce Jiang updated SPARK-32515: Attachment: Capture.PNG > Distinct Function Weird Bug > --- > >

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayce Jiang updated SPARK-32515: Attachment: unknown2.png unknown1.png unknown.png > Distinct Funct

[jira] [Commented] (SPARK-28818) FrequentItems applies an incorrect schema to the resulting dataframe when nulls are present

[ https://issues.apache.org/jira/browse/SPARK-28818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169501#comment-17169501 ] Apache Spark commented on SPARK-28818: -- User 'maropu' has created a pull request fo

[jira] [Commented] (SPARK-28818) FrequentItems applies an incorrect schema to the resulting dataframe when nulls are present

[ https://issues.apache.org/jira/browse/SPARK-28818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169500#comment-17169500 ] Apache Spark commented on SPARK-28818: -- User 'maropu' has created a pull request fo