[jira] [Resolved] (SPARK-41470) SPJ: Spark shouldn't assume InternalRow implements equals and hashCode

[ https://issues.apache.org/jira/browse/SPARK-41470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-41470. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 39687

[jira] [Assigned] (SPARK-42353) Cleanup orphan sst and log files in RocksDB checkpoint directory

[ https://issues.apache.org/jira/browse/SPARK-42353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42353: Assignee: (was: Apache Spark) > Cleanup orphan sst and log files in RocksDB

[jira] [Assigned] (SPARK-42353) Cleanup orphan sst and log files in RocksDB checkpoint directory

[ https://issues.apache.org/jira/browse/SPARK-42353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42353: Assignee: Apache Spark > Cleanup orphan sst and log files in RocksDB checkpoint

[jira] [Assigned] (SPARK-42255) Assign name to _LEGACY_ERROR_TEMP_2430

[ https://issues.apache.org/jira/browse/SPARK-42255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-42255: Assignee: Haejoon Lee > Assign name to _LEGACY_ERROR_TEMP_2430 >

[jira] [Resolved] (SPARK-42255) Assign name to _LEGACY_ERROR_TEMP_2430

[ https://issues.apache.org/jira/browse/SPARK-42255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-42255. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39839

[jira] [Assigned] (SPARK-42360) Transform LeftOuter join with IsNull filter on right side to Anti join

[ https://issues.apache.org/jira/browse/SPARK-42360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42360: Assignee: (was: Apache Spark) > Transform LeftOuter join with IsNull filter on right

[jira] [Assigned] (SPARK-42360) Transform LeftOuter join with IsNull filter on right side to Anti join

[ https://issues.apache.org/jira/browse/SPARK-42360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42360: Assignee: Apache Spark > Transform LeftOuter join with IsNull filter on right side to

[jira] [Commented] (SPARK-42360) Transform LeftOuter join with IsNull filter on right side to Anti join

[ https://issues.apache.org/jira/browse/SPARK-42360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684745#comment-17684745 ] Apache Spark commented on SPARK-42360: -- User 'wankunde' has created a pull request for this issue:

[jira] [Created] (SPARK-42360) Transform LeftOuter join with IsNull filter on right side to Anti join

Wan Kun created SPARK-42360: --- Summary: Transform LeftOuter join with IsNull filter on right side to Anti join Key: SPARK-42360 URL: https://issues.apache.org/jira/browse/SPARK-42360 Project: Spark

[jira] [Commented] (SPARK-42359) Support row skipping when reading CSV files

[ https://issues.apache.org/jira/browse/SPARK-42359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684705#comment-17684705 ] Apache Spark commented on SPARK-42359: -- User 'ted-jenks' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42359) Support row skipping when reading CSV files

[ https://issues.apache.org/jira/browse/SPARK-42359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42359: Assignee: Apache Spark > Support row skipping when reading CSV files >

[jira] [Assigned] (SPARK-42359) Support row skipping when reading CSV files

[ https://issues.apache.org/jira/browse/SPARK-42359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42359: Assignee: (was: Apache Spark) > Support row skipping when reading CSV files >

[jira] [Commented] (SPARK-42359) Support row skipping when reading CSV files

[ https://issues.apache.org/jira/browse/SPARK-42359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684689#comment-17684689 ] Willi Raschkowski commented on SPARK-42359: --- This repeats SPARK-26406 but it's worth

[jira] [Commented] (SPARK-42359) Support row skipping when reading CSV files

[ https://issues.apache.org/jira/browse/SPARK-42359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684683#comment-17684683 ] Willi Raschkowski commented on SPARK-42359: --- In our experience such CSV files tend to be Excel

[jira] [Updated] (SPARK-42359) Support row skipping when reading CSV files

[ https://issues.apache.org/jira/browse/SPARK-42359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Willi Raschkowski updated SPARK-42359: -- Attachment: Screenshot 2023-02-06 at 13.23.34.png > Support row skipping when reading

[jira] [Created] (SPARK-42359) Support row skipping when reading CSV files

Willi Raschkowski created SPARK-42359: - Summary: Support row skipping when reading CSV files Key: SPARK-42359 URL: https://issues.apache.org/jira/browse/SPARK-42359 Project: Spark Issue

[jira] [Resolved] (SPARK-42346) distinct(count colname) with UNION ALL causes query analyzer bug

[ https://issues.apache.org/jira/browse/SPARK-42346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42346. - Fix Version/s: 3.3.2 3.4.0 3.5.0 Assignee: Peter

[jira] [Commented] (SPARK-41962) Update the import order of scala package in class SpecificParquetRecordReaderBase

[ https://issues.apache.org/jira/browse/SPARK-41962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684675#comment-17684675 ] Apache Spark commented on SPARK-41962: -- User 'wayneguow' has created a pull request for this issue:

[jira] [Commented] (SPARK-41962) Update the import order of scala package in class SpecificParquetRecordReaderBase

[ https://issues.apache.org/jira/browse/SPARK-41962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684674#comment-17684674 ] Apache Spark commented on SPARK-41962: -- User 'wayneguow' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41962) Update the import order of scala package in class SpecificParquetRecordReaderBase

[ https://issues.apache.org/jira/browse/SPARK-41962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41962: Assignee: Apache Spark > Update the import order of scala package in class >

[jira] [Assigned] (SPARK-41962) Update the import order of scala package in class SpecificParquetRecordReaderBase

[ https://issues.apache.org/jira/browse/SPARK-41962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41962: Assignee: (was: Apache Spark) > Update the import order of scala package in class >

[jira] [Commented] (SPARK-40819) Parquet INT64 (TIMESTAMP(NANOS,true)) now throwing Illegal Parquet type instead of automatically converting to LongType

[ https://issues.apache.org/jira/browse/SPARK-40819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684673#comment-17684673 ] Apache Spark commented on SPARK-40819: -- User 'awdavidson' has created a pull request for this

[jira] [Commented] (SPARK-40819) Parquet INT64 (TIMESTAMP(NANOS,true)) now throwing Illegal Parquet type instead of automatically converting to LongType

[ https://issues.apache.org/jira/browse/SPARK-40819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684623#comment-17684623 ] Apache Spark commented on SPARK-40819: -- User 'awdavidson' has created a pull request for this

[jira] [Assigned] (SPARK-42358) Provide more details in ExecutorUpdated sent in Master.removeWorker

[ https://issues.apache.org/jira/browse/SPARK-42358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42358: Assignee: (was: Apache Spark) > Provide more details in ExecutorUpdated sent in

[jira] [Commented] (SPARK-42358) Provide more details in ExecutorUpdated sent in Master.removeWorker

[ https://issues.apache.org/jira/browse/SPARK-42358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684604#comment-17684604 ] Apache Spark commented on SPARK-42358: -- User 'bozhang2820' has created a pull request for this

[jira] [Assigned] (SPARK-42358) Provide more details in ExecutorUpdated sent in Master.removeWorker

[ https://issues.apache.org/jira/browse/SPARK-42358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42358: Assignee: Apache Spark > Provide more details in ExecutorUpdated sent in

[jira] [Updated] (SPARK-34661) Replaces `OriginalType` with `LogicalTypeAnnotation` in VectorizedColumnReader

[ https://issues.apache.org/jira/browse/SPARK-34661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-34661: -- Affects Version/s: 3.2.3 3.3.1 3.2.2

[jira] [Updated] (SPARK-34661) Replaces `OriginalType` with `LogicalTypeAnnotation` in VectorizedColumnReader

[ https://issues.apache.org/jira/browse/SPARK-34661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-34661: -- Affects Version/s: (was: 3.3.0) (was: 3.2.1)

[jira] [Resolved] (SPARK-42302) Assign name to _LEGACY_ERROR_TEMP_2135

[ https://issues.apache.org/jira/browse/SPARK-42302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-42302. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39872

[jira] [Assigned] (SPARK-42302) Assign name to _LEGACY_ERROR_TEMP_2135

[ https://issues.apache.org/jira/browse/SPARK-42302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-42302: Assignee: Haejoon Lee > Assign name to _LEGACY_ERROR_TEMP_2135 >

[jira] [Assigned] (SPARK-42320) Assign name to _LEGACY_ERROR_TEMP_2188

[ https://issues.apache.org/jira/browse/SPARK-42320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-42320: Assignee: Haejoon Lee > Assign name to _LEGACY_ERROR_TEMP_2188 >

[jira] [Resolved] (SPARK-42320) Assign name to _LEGACY_ERROR_TEMP_2188

[ https://issues.apache.org/jira/browse/SPARK-42320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-42320. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39888

[jira] [Created] (SPARK-42358) Provide more details in ExecutorUpdated sent in Master.removeWorker

Bo Zhang created SPARK-42358: Summary: Provide more details in ExecutorUpdated sent in Master.removeWorker Key: SPARK-42358 URL: https://issues.apache.org/jira/browse/SPARK-42358 Project: Spark

[jira] [Assigned] (SPARK-42349) Support pandas cogroup with multiple df

[ https://issues.apache.org/jira/browse/SPARK-42349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42349: Assignee: Apache Spark > Support pandas cogroup with multiple df >

[jira] [Commented] (SPARK-42349) Support pandas cogroup with multiple df

[ https://issues.apache.org/jira/browse/SPARK-42349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684568#comment-17684568 ] Apache Spark commented on SPARK-42349: -- User 'santosh-d3vpl3x' has created a pull request for this

[jira] [Assigned] (SPARK-42349) Support pandas cogroup with multiple df

[ https://issues.apache.org/jira/browse/SPARK-42349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42349: Assignee: (was: Apache Spark) > Support pandas cogroup with multiple df >

[jira] [Commented] (SPARK-40819) Parquet INT64 (TIMESTAMP(NANOS,true)) now throwing Illegal Parquet type instead of automatically converting to LongType

[ https://issues.apache.org/jira/browse/SPARK-40819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684565#comment-17684565 ] Apache Spark commented on SPARK-40819: -- User 'awdavidson' has created a pull request for this

[jira] [Commented] (SPARK-42357) Log `exitCode` when `SparkContext.stop` starts

[ https://issues.apache.org/jira/browse/SPARK-42357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684564#comment-17684564 ] Apache Spark commented on SPARK-42357: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-42357) Log `exitCode` when `SparkContext.stop` starts

[ https://issues.apache.org/jira/browse/SPARK-42357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42357: Assignee: Apache Spark > Log `exitCode` when `SparkContext.stop` starts >

[jira] [Assigned] (SPARK-42357) Log `exitCode` when `SparkContext.stop` starts

[ https://issues.apache.org/jira/browse/SPARK-42357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42357: Assignee: (was: Apache Spark) > Log `exitCode` when `SparkContext.stop` starts >

[jira] [Created] (SPARK-42357) Log `exitCode` when `SparkContext.stop` starts

Dongjoon Hyun created SPARK-42357: - Summary: Log `exitCode` when `SparkContext.stop` starts Key: SPARK-42357 URL: https://issues.apache.org/jira/browse/SPARK-42357 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-42002) Implement DataFrameWriterV2 (ReadwriterV2Tests)

[ https://issues.apache.org/jira/browse/SPARK-42002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42002. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39614

[jira] [Assigned] (SPARK-42002) Implement DataFrameWriterV2 (ReadwriterV2Tests)

[ https://issues.apache.org/jira/browse/SPARK-42002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42002: Assignee: Sandeep Singh > Implement DataFrameWriterV2 (ReadwriterV2Tests) >

[jira] [Assigned] (SPARK-40819) Parquet INT64 (TIMESTAMP(NANOS,true)) now throwing Illegal Parquet type instead of automatically converting to LongType

[ https://issues.apache.org/jira/browse/SPARK-40819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40819: Assignee: Alfred Davidson > Parquet INT64 (TIMESTAMP(NANOS,true)) now throwing Illegal

[jira] [Resolved] (SPARK-40819) Parquet INT64 (TIMESTAMP(NANOS,true)) now throwing Illegal Parquet type instead of automatically converting to LongType

[ https://issues.apache.org/jira/browse/SPARK-40819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40819. -- Resolution: Fixed Fixed in https://github.com/apache/spark/pull/38312 > Parquet INT64

[jira] [Updated] (SPARK-40819) Parquet INT64 (TIMESTAMP(NANOS,true)) now throwing Illegal Parquet type instead of automatically converting to LongType

[ https://issues.apache.org/jira/browse/SPARK-40819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-40819: - Fix Version/s: 3.4.0 > Parquet INT64 (TIMESTAMP(NANOS,true)) now throwing Illegal Parquet type

[jira] [Resolved] (SPARK-42356) Cannot resolve orderby attributes in DISTINCT

[ https://issues.apache.org/jira/browse/SPARK-42356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tongwei resolved SPARK-42356. - Resolution: Won't Fix > Cannot resolve orderby attributes in DISTINCT >

[jira] [Updated] (SPARK-42356) Cannot resolve orderby attributes in DISTINCT

[ https://issues.apache.org/jira/browse/SPARK-42356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tongwei updated SPARK-42356: Description: query: {code:java} CREATE TABLE students (name VARCHAR(64), address VARCHAR(64)) USING

[jira] [Updated] (SPARK-42356) Cannot resolve orderby attributes in DISTINCT

[ https://issues.apache.org/jira/browse/SPARK-42356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tongwei updated SPARK-42356: Description: query:   {code:java} CREATE TABLE students (name VARCHAR(64), address VARCHAR(64)) USING

[jira] [Created] (SPARK-42356) Cannot resolve orderby attributes in DISTINCT

Tongwei created SPARK-42356: --- Summary: Cannot resolve orderby attributes in DISTINCT Key: SPARK-42356 URL: https://issues.apache.org/jira/browse/SPARK-42356 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-42355) Upgrade some maven-plugins

[ https://issues.apache.org/jira/browse/SPARK-42355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42355: Assignee: Apache Spark > Upgrade some maven-plugins > -- > >

[jira] [Assigned] (SPARK-42355) Upgrade some maven-plugins

[ https://issues.apache.org/jira/browse/SPARK-42355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42355: Assignee: (was: Apache Spark) > Upgrade some maven-plugins >

[jira] [Commented] (SPARK-42355) Upgrade some maven-plugins

[ https://issues.apache.org/jira/browse/SPARK-42355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684505#comment-17684505 ] Apache Spark commented on SPARK-42355: -- User 'LuciferYang' has created a pull request for this

[jira] [Created] (SPARK-42355) Upgrade maven-plugins

Yang Jie created SPARK-42355: Summary: Upgrade maven-plugins Key: SPARK-42355 URL: https://issues.apache.org/jira/browse/SPARK-42355 Project: Spark Issue Type: Improvement Components:

[jira] [Updated] (SPARK-42355) Upgrade some maven-plugins

[ https://issues.apache.org/jira/browse/SPARK-42355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42355: - Summary: Upgrade some maven-plugins (was: Upgrade maven-plugins) > Upgrade some maven-plugins >

[jira] [Assigned] (SPARK-42354) Upgrade Jackson to 2.14.2

[ https://issues.apache.org/jira/browse/SPARK-42354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42354: Assignee: (was: Apache Spark) > Upgrade Jackson to 2.14.2 >

[jira] [Commented] (SPARK-42353) Cleanup orphan sst and log files in RocksDB checkpoint directory

[ https://issues.apache.org/jira/browse/SPARK-42353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684501#comment-17684501 ] Apache Spark commented on SPARK-42353: -- User 'chaoqin-li1123' has created a pull request for this

[jira] [Commented] (SPARK-42354) Upgrade Jackson to 2.14.2

[ https://issues.apache.org/jira/browse/SPARK-42354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684502#comment-17684502 ] Apache Spark commented on SPARK-42354: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-42354) Upgrade Jackson to 2.14.2

[ https://issues.apache.org/jira/browse/SPARK-42354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42354: Assignee: Apache Spark > Upgrade Jackson to 2.14.2 > - > >

[jira] [Created] (SPARK-42354) Upgrade Jackson to 2.14.2

Yang Jie created SPARK-42354: Summary: Upgrade Jackson to 2.14.2 Key: SPARK-42354 URL: https://issues.apache.org/jira/browse/SPARK-42354 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-41234) High-order function: array_insert

[ https://issues.apache.org/jira/browse/SPARK-41234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41234. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38867

[jira] [Assigned] (SPARK-41234) High-order function: array_insert

[ https://issues.apache.org/jira/browse/SPARK-41234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41234: - Assignee: Daniel Davies > High-order function: array_insert >

<    1   2