[jira] [Commented] (SPARK-36761) spark-examples_2.12-3.0.2.jar DFSReadWriteTest S3A Implementation

[ https://issues.apache.org/jira/browse/SPARK-36761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416463#comment-17416463 ] Hyukjin Kwon commented on SPARK-36761: -- does the issue happen with spark submit whether it's the

[jira] [Commented] (SPARK-36761) spark-examples_2.12-3.0.2.jar DFSReadWriteTest S3A Implementation

[ https://issues.apache.org/jira/browse/SPARK-36761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416462#comment-17416462 ] Raj commented on SPARK-36761: - Hey Hyukjin Kwon I am able to read and write to the s3a bucket(Inside Ceph

[jira] [Updated] (SPARK-36781) The log could not get the correct line number

[ https://issues.apache.org/jira/browse/SPARK-36781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36781: - Affects Version/s: 3.0.3 > The log could not get the correct line number >

[jira] [Updated] (SPARK-36778) Support ILIKE API on Scala(dataframe)

[ https://issues.apache.org/jira/browse/SPARK-36778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36778: - Parent: SPARK-36752 Issue Type: Sub-task (was: New Feature) > Support ILIKE API on

[jira] [Updated] (SPARK-36778) Support ILIKE API on Scala(dataframe)

[ https://issues.apache.org/jira/browse/SPARK-36778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36778: - External issue ID: (was: SPARK-36752) > Support ILIKE API on Scala(dataframe) >

[jira] [Updated] (SPARK-36752) Support ILIKE by Scala/Java, PySpark and R APIs

[ https://issues.apache.org/jira/browse/SPARK-36752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36752: - Issue Type: Umbrella (was: New Feature) > Support ILIKE by Scala/Java, PySpark and R APIs >

[jira] [Commented] (SPARK-36781) The log could not get the correct line number

[ https://issues.apache.org/jira/browse/SPARK-36781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416455#comment-17416455 ] chenxusheng commented on SPARK-36781: - [~hyukjin.kwon]  Yes, when I use Spark3.0, the problem still

[jira] [Commented] (SPARK-36761) spark-examples_2.12-3.0.2.jar DFSReadWriteTest S3A Implementation

[ https://issues.apache.org/jira/browse/SPARK-36761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416453#comment-17416453 ] Hyukjin Kwon commented on SPARK-36761: -- does other spark application work? seems like it's an issue

[jira] [Resolved] (SPARK-36766) Spark SQL DDL does not recognize fs.s3.impl implied filesystem in LOCATION tag

[ https://issues.apache.org/jira/browse/SPARK-36766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36766. -- Resolution: Invalid > Spark SQL DDL does not recognize fs.s3.impl implied filesystem in

[jira] [Commented] (SPARK-36766) Spark SQL DDL does not recognize fs.s3.impl implied filesystem in LOCATION tag

[ https://issues.apache.org/jira/browse/SPARK-36766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416452#comment-17416452 ] Hyukjin Kwon commented on SPARK-36766: -- The error message is from iceberg. Seems like it's iceberg

[jira] [Commented] (SPARK-36776) Partition filter of DataSourceV2ScanRelation can not push down when select none dataSchema from FileScan

[ https://issues.apache.org/jira/browse/SPARK-36776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416451#comment-17416451 ] Hyukjin Kwon commented on SPARK-36776: -- I think this is fixed at SPARK-36351. cc [~huaxingao] FYI

[jira] [Commented] (SPARK-36765) Spark Support for MS Sql JDBC connector with Kerberos/Keytab

[ https://issues.apache.org/jira/browse/SPARK-36765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416441#comment-17416441 ] Hyukjin Kwon commented on SPARK-36765: -- cc [~gaborgsomogyi] FYI > Spark Support for MS Sql JDBC

[jira] [Updated] (SPARK-36765) Spark Support for MS Sql JDBC connector with Kerberos/Keytab

[ https://issues.apache.org/jira/browse/SPARK-36765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36765: - Description: Hi Team,   We are using the Spark-3.0.2 to connect to MS SqlServer with the

[jira] [Commented] (SPARK-36781) The log could not get the correct line number

[ https://issues.apache.org/jira/browse/SPARK-36781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416440#comment-17416440 ] Hyukjin Kwon commented on SPARK-36781: -- Spark 2.x is EOL. Can you check if it works w/ Spark 3.x?

[jira] [Updated] (SPARK-36787) Kryo Buffer underflow error in Spark 3.0

[ https://issues.apache.org/jira/browse/SPARK-36787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36787: - Priority: Major (was: Blocker) > Kryo Buffer underflow error in Spark 3.0 >

[jira] [Updated] (SPARK-27994) Spark Avro Failed to read logical type decimal backed by bytes

[ https://issues.apache.org/jira/browse/SPARK-27994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Pascal updated SPARK-27994: --- Description: Fields with this following schema provokes Spark to fail reading the Avro

[jira] [Commented] (SPARK-36708) Support numpy.typing for annotating ArrayType

[ https://issues.apache.org/jira/browse/SPARK-36708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416432#comment-17416432 ] Apache Spark commented on SPARK-36708: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-36708) Support numpy.typing for annotating ArrayType

[ https://issues.apache.org/jira/browse/SPARK-36708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36708: Assignee: Apache Spark > Support numpy.typing for annotating ArrayType >

[jira] [Assigned] (SPARK-36708) Support numpy.typing for annotating ArrayType

[ https://issues.apache.org/jira/browse/SPARK-36708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36708: Assignee: (was: Apache Spark) > Support numpy.typing for annotating ArrayType >

[jira] [Commented] (SPARK-36708) Support numpy.typing for annotating ArrayType

[ https://issues.apache.org/jira/browse/SPARK-36708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416431#comment-17416431 ] Apache Spark commented on SPARK-36708: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-36788) Change log level of AQE for non-supported plans from warning to debug

[ https://issues.apache.org/jira/browse/SPARK-36788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36788: Assignee: Hyukjin Kwon > Change log level of AQE for non-supported plans from warning to

[jira] [Resolved] (SPARK-36788) Change log level of AQE for non-supported plans from warning to debug

[ https://issues.apache.org/jira/browse/SPARK-36788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36788. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 34026

[jira] [Resolved] (SPARK-36783) ScanOperation should not push Filter through nondeterministic Project

[ https://issues.apache.org/jira/browse/SPARK-36783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36783. - Fix Version/s: 3.1.3 3.2.0 3.0.4 Resolution: Fixed

[jira] [Assigned] (SPARK-36783) ScanOperation should not push Filter through nondeterministic Project

[ https://issues.apache.org/jira/browse/SPARK-36783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36783: --- Assignee: Wenchen Fan > ScanOperation should not push Filter through nondeterministic

[jira] [Commented] (SPARK-36773) Fix the uts to check the parquet compression

[ https://issues.apache.org/jira/browse/SPARK-36773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416405#comment-17416405 ] Hyukjin Kwon commented on SPARK-36773: -- Fixed in https://github.com/apache/spark/pull/34012 > Fix

[jira] [Updated] (SPARK-36773) Fix the uts to check the parquet compression

[ https://issues.apache.org/jira/browse/SPARK-36773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36773: - Fix Version/s: 3.3.0 > Fix the uts to check the parquet compression >

[jira] [Assigned] (SPARK-36773) Fix the uts to check the parquet compression

[ https://issues.apache.org/jira/browse/SPARK-36773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36773: Assignee: jingpan xiong > Fix the uts to check the parquet compression >

[jira] [Resolved] (SPARK-36773) Fix the uts to check the parquet compression

[ https://issues.apache.org/jira/browse/SPARK-36773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jingpan xiong resolved SPARK-36773. --- Resolution: Fixed > Fix the uts to check the parquet compression >

[jira] [Assigned] (SPARK-36778) Support ILIKE API on Scala(dataframe)

[ https://issues.apache.org/jira/browse/SPARK-36778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36778: Assignee: (was: Apache Spark) > Support ILIKE API on Scala(dataframe) >

[jira] [Updated] (SPARK-36773) Fix the uts to check the parquet compression

[ https://issues.apache.org/jira/browse/SPARK-36773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36773: - Affects Version/s: (was: 3.3) 3.3.0 > Fix the uts to check the

[jira] [Updated] (SPARK-36773) Fix the uts to check the parquet compression

[ https://issues.apache.org/jira/browse/SPARK-36773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36773: - Fix Version/s: (was: 3.3) > Fix the uts to check the parquet compression >

[jira] [Commented] (SPARK-36778) Support ILIKE API on Scala(dataframe)

[ https://issues.apache.org/jira/browse/SPARK-36778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416401#comment-17416401 ] Apache Spark commented on SPARK-36778: -- User 'yoda-mon' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36778) Support ILIKE API on Scala(dataframe)

[ https://issues.apache.org/jira/browse/SPARK-36778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36778: Assignee: Apache Spark > Support ILIKE API on Scala(dataframe) >

[jira] [Updated] (SPARK-36778) Support ILIKE Scala(dataframe) API

[ https://issues.apache.org/jira/browse/SPARK-36778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leona Yoda updated SPARK-36778: --- Summary: Support ILIKE Scala(dataframe) API (was: Support Scala(dataframe) API on ILIKE) >

[jira] [Updated] (SPARK-36778) Support ILIKE API on Scala(dataframe)

[ https://issues.apache.org/jira/browse/SPARK-36778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leona Yoda updated SPARK-36778: --- Summary: Support ILIKE API on Scala(dataframe) (was: Support ILIKE Scala(dataframe) API) >

[jira] [Updated] (SPARK-36787) Kryo Buffer underflow error in Spark 3.0

[ https://issues.apache.org/jira/browse/SPARK-36787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu updated SPARK-36787: -- Description: Spark JavaPairRDD processing fails with error {code:java}

[jira] [Commented] (SPARK-36788) Change log level of AQE for non-supported plans from warning to debug

[ https://issues.apache.org/jira/browse/SPARK-36788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416386#comment-17416386 ] Apache Spark commented on SPARK-36788: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-36788) Change log level of AQE for non-supported plans from warning to debug

[ https://issues.apache.org/jira/browse/SPARK-36788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36788: Assignee: Apache Spark > Change log level of AQE for non-supported plans from warning to

[jira] [Assigned] (SPARK-36788) Change log level of AQE for non-supported plans from warning to debug

[ https://issues.apache.org/jira/browse/SPARK-36788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36788: Assignee: (was: Apache Spark) > Change log level of AQE for non-supported plans from

[jira] [Created] (SPARK-36788) Change log level of AQE for non-supported plans from warning to debug

Hyukjin Kwon created SPARK-36788: Summary: Change log level of AQE for non-supported plans from warning to debug Key: SPARK-36788 URL: https://issues.apache.org/jira/browse/SPARK-36788 Project: Spark

[jira] [Created] (SPARK-36787) Kryo Buffer underflow error in Spark 3.0

Raghu created SPARK-36787: - Summary: Kryo Buffer underflow error in Spark 3.0 Key: SPARK-36787 URL: https://issues.apache.org/jira/browse/SPARK-36787 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-36786) SPIP: Improving the compile time performance, by improving a couple of rules, from 24 hrs to under 8 minutes

[ https://issues.apache.org/jira/browse/SPARK-36786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-36786: - Labels: SPIP (was: ) Summary: SPIP: Improving the compile time performance, by improving a couple of

[jira] [Created] (SPARK-36786) Improving the compile time performance, by improving a couple of rules, from 24 hrs to under 8 minutes

Asif created SPARK-36786: Summary: Improving the compile time performance, by improving a couple of rules, from 24 hrs to under 8 minutes Key: SPARK-36786 URL: https://issues.apache.org/jira/browse/SPARK-36786

[jira] [Assigned] (SPARK-36779) Error when list of data type tuples has len = 1

[ https://issues.apache.org/jira/browse/SPARK-36779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36779: Assignee: dgd_contributor > Error when list of data type tuples has len = 1 >

[jira] [Resolved] (SPARK-36779) Error when list of data type tuples has len = 1

[ https://issues.apache.org/jira/browse/SPARK-36779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36779. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34019

[jira] [Resolved] (SPARK-36774) Use SparkSubmitTestUtils to core and use it in SparkSubmitSuite

[ https://issues.apache.org/jira/browse/SPARK-36774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-36774. Fix Version/s: 3.2.0 Resolution: Fixed > Use SparkSubmitTestUtils to core and use it in

[jira] [Updated] (SPARK-33152) SPIP: Constraint Propagation code causes OOM issues or increasing compilation time to hours

[ https://issues.apache.org/jira/browse/SPARK-33152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-33152: - Description: h2. Q1. What are you trying to do? Articulate your objectives using absolutely no jargon.

[jira] [Updated] (SPARK-33152) SPIP: Constraint Propagation code causes OOM issues or increasing compilation time to hours

[ https://issues.apache.org/jira/browse/SPARK-33152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-33152: - Labels: SPIP (was: ) > SPIP: Constraint Propagation code causes OOM issues or increasing compilation > time

[jira] [Updated] (SPARK-33152) SPIP: Constraint Propagation code causes OOM issues or increasing compilation time to hours

[ https://issues.apache.org/jira/browse/SPARK-33152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-33152: - Summary: SPIP: Constraint Propagation code causes OOM issues or increasing compilation time to hours (was:

[jira] [Assigned] (SPARK-36673) Incorrect Unions of struct with mismatched field name case

[ https://issues.apache.org/jira/browse/SPARK-36673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36673: Assignee: (was: Apache Spark) > Incorrect Unions of struct with mismatched field

[jira] [Assigned] (SPARK-36673) Incorrect Unions of struct with mismatched field name case

[ https://issues.apache.org/jira/browse/SPARK-36673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36673: Assignee: Apache Spark > Incorrect Unions of struct with mismatched field name case >

[jira] [Commented] (SPARK-36673) Incorrect Unions of struct with mismatched field name case

[ https://issues.apache.org/jira/browse/SPARK-36673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416311#comment-17416311 ] Apache Spark commented on SPARK-36673: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-36784) Handle DNS failures in Executor without causing FetchFailed

[ https://issues.apache.org/jira/browse/SPARK-36784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416299#comment-17416299 ] Apache Spark commented on SPARK-36784: -- User 'thejdeep' has created a pull request for this issue:

[jira] [Commented] (SPARK-36784) Handle DNS failures in Executor without causing FetchFailed

[ https://issues.apache.org/jira/browse/SPARK-36784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416300#comment-17416300 ] Apache Spark commented on SPARK-36784: -- User 'thejdeep' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36784) Handle DNS failures in Executor without causing FetchFailed

[ https://issues.apache.org/jira/browse/SPARK-36784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36784: Assignee: Apache Spark > Handle DNS failures in Executor without causing FetchFailed >

[jira] [Assigned] (SPARK-36784) Handle DNS failures in Executor without causing FetchFailed

[ https://issues.apache.org/jira/browse/SPARK-36784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36784: Assignee: (was: Apache Spark) > Handle DNS failures in Executor without causing

[jira] [Updated] (SPARK-36784) Handle DNS failures in Executor without causing FetchFailed

[ https://issues.apache.org/jira/browse/SPARK-36784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejdeep Gudivada updated SPARK-36784: -- Fix Version/s: 3.3.0 > Handle DNS failures in Executor without causing FetchFailed >

[jira] [Commented] (SPARK-36785) Fix ps.DataFrame.isin

[ https://issues.apache.org/jira/browse/SPARK-36785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416283#comment-17416283 ] dgd_contributor commented on SPARK-36785: - I am working on this. > Fix ps.DataFrame.isin >

[jira] [Created] (SPARK-36785) Fix ps.DataFrame.isin

dgd_contributor created SPARK-36785: --- Summary: Fix ps.DataFrame.isin Key: SPARK-36785 URL: https://issues.apache.org/jira/browse/SPARK-36785 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36784) Handle DNS failures in Executor without causing FetchFailed

Thejdeep Gudivada created SPARK-36784: - Summary: Handle DNS failures in Executor without causing FetchFailed Key: SPARK-36784 URL: https://issues.apache.org/jira/browse/SPARK-36784 Project: Spark

[jira] [Updated] (SPARK-36770) Improve run-time performance for window function first and last against UnboundedFollowing window ROWS frame

[ https://issues.apache.org/jira/browse/SPARK-36770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guibin Zhang updated SPARK-36770: - Description: *Context* The *UnboundedFollowingWindowFunctionFrame* has the time complexity of

[jira] [Updated] (SPARK-33152) Constraint Propagation code causes OOM issues or increasing compilation time to hours

[ https://issues.apache.org/jira/browse/SPARK-33152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-33152: - Affects Version/s: 3.1.2 Issue Type: Improvement (was: Bug) > Constraint Propagation code causes

[jira] [Assigned] (SPARK-36783) ScanOperation should not push Filter through nondeterministic Project

[ https://issues.apache.org/jira/browse/SPARK-36783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36783: Assignee: (was: Apache Spark) > ScanOperation should not push Filter through

[jira] [Assigned] (SPARK-36783) ScanOperation should not push Filter through nondeterministic Project

[ https://issues.apache.org/jira/browse/SPARK-36783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36783: Assignee: Apache Spark > ScanOperation should not push Filter through nondeterministic

[jira] [Commented] (SPARK-36783) ScanOperation should not push Filter through nondeterministic Project

[ https://issues.apache.org/jira/browse/SPARK-36783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416239#comment-17416239 ] Apache Spark commented on SPARK-36783: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Created] (SPARK-36783) ScanOperation should not push Filter through nondeterministic Project

Wenchen Fan created SPARK-36783: --- Summary: ScanOperation should not push Filter through nondeterministic Project Key: SPARK-36783 URL: https://issues.apache.org/jira/browse/SPARK-36783 Project: Spark

[jira] [Comment Edited] (SPARK-31646) Remove unused registeredConnections counter from ShuffleMetrics

[ https://issues.apache.org/jira/browse/SPARK-31646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416225#comment-17416225 ] Yongjun Zhang edited comment on SPARK-31646 at 9/16/21, 4:36 PM: - Thanks

[jira] [Comment Edited] (SPARK-31646) Remove unused registeredConnections counter from ShuffleMetrics

[ https://issues.apache.org/jira/browse/SPARK-31646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416225#comment-17416225 ] Yongjun Zhang edited comment on SPARK-31646 at 9/16/21, 4:35 PM: - Thanks

[jira] [Commented] (SPARK-31646) Remove unused registeredConnections counter from ShuffleMetrics

[ https://issues.apache.org/jira/browse/SPARK-31646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416225#comment-17416225 ] Yongjun Zhang commented on SPARK-31646: --- Thanks a lot for your quick response [~mauzhang]. Sorry

[jira] [Comment Edited] (SPARK-33772) Build and Run Spark on Java 17

[ https://issues.apache.org/jira/browse/SPARK-33772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416218#comment-17416218 ] Dongjoon Hyun edited comment on SPARK-33772 at 9/16/21, 4:28 PM: - I used

[jira] [Commented] (SPARK-33772) Build and Run Spark on Java 17

[ https://issues.apache.org/jira/browse/SPARK-33772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416221#comment-17416221 ] Dongjoon Hyun commented on SPARK-33772: --- BTW, [~lrytz]. As you see, this issue is growing by

[jira] [Commented] (SPARK-33772) Build and Run Spark on Java 17

[ https://issues.apache.org/jira/browse/SPARK-33772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416218#comment-17416218 ] Dongjoon Hyun commented on SPARK-33772: --- I used your example. It seems that Spark can use that

[jira] [Commented] (SPARK-33772) Build and Run Spark on Java 17

[ https://issues.apache.org/jira/browse/SPARK-33772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416217#comment-17416217 ] Dongjoon Hyun commented on SPARK-33772: --- Yep, that's the one I asked, [~lrytz]. Thanks. :) >

[jira] [Updated] (SPARK-35924) Add Java 17 ea build test to GitHub action

[ https://issues.apache.org/jira/browse/SPARK-35924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35924: -- Parent: SPARK-33772 Issue Type: Sub-task (was: Improvement) > Add Java 17 ea build

[jira] [Updated] (SPARK-36780) Make dev/mima runs on Java 17

[ https://issues.apache.org/jira/browse/SPARK-36780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-36780: Summary: Make dev/mima runs on Java 17 (was: Make dev/mima runs on Java 17+) > Make dev/mima

[jira] [Commented] (SPARK-36780) Make dev/mima runs on Java 17+

[ https://issues.apache.org/jira/browse/SPARK-36780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416181#comment-17416181 ] Apache Spark commented on SPARK-36780: -- User 'RabbidHY' has created a pull request for this issue:

[jira] [Commented] (SPARK-36433) Logs should show correct URL of where HistoryServer is started

[ https://issues.apache.org/jira/browse/SPARK-36433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416150#comment-17416150 ] Thejdeep Gudivada commented on SPARK-36433: --- Thanks [~Gengliang.Wang] > Logs should show

[jira] [Commented] (SPARK-31646) Remove unused registeredConnections counter from ShuffleMetrics

[ https://issues.apache.org/jira/browse/SPARK-31646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416147#comment-17416147 ] Manu Zhang commented on SPARK-31646: [~yzhangal], please check

[jira] [Updated] (SPARK-36782) Deadlock between map-output-dispatcher and dispatcher-BlockManagerMaster upon migrating shuffle blocks

[ https://issues.apache.org/jira/browse/SPARK-36782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabian Thiele updated SPARK-36782: -- Description: I can observe a deadlock on the driver that can be triggered rather reliably in

[jira] [Updated] (SPARK-36782) Deadlock between map-output-dispatcher and dispatcher-BlockManagerMaster upon migrating shuffle blocks

[ https://issues.apache.org/jira/browse/SPARK-36782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabian Thiele updated SPARK-36782: -- Attachment: 0001-Add-test-showing-that-decommission-might-deadlock.patch > Deadlock between

[jira] [Updated] (SPARK-36782) Deadlock between map-output-dispatcher and dispatcher-BlockManagerMaster upon migrating shuffle blocks

[ https://issues.apache.org/jira/browse/SPARK-36782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabian Thiele updated SPARK-36782: -- Attachment: spark_stacktrace_deadlock.txt > Deadlock between map-output-dispatcher and

[jira] [Created] (SPARK-36782) Deadlock between map-output-dispatcher and dispatcher-BlockManagerMaster upon migrating shuffle blocks

Fabian Thiele created SPARK-36782: - Summary: Deadlock between map-output-dispatcher and dispatcher-BlockManagerMaster upon migrating shuffle blocks Key: SPARK-36782 URL:

[jira] [Commented] (SPARK-36642) Add df.withMetadata: a syntax suger to update the metadata of a dataframe

[ https://issues.apache.org/jira/browse/SPARK-36642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416120#comment-17416120 ] Apache Spark commented on SPARK-36642: -- User 'liangz1' has created a pull request for this issue:

[jira] [Commented] (SPARK-34183) DataSource V2: Support required distribution and ordering in SS

[ https://issues.apache.org/jira/browse/SPARK-34183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416116#comment-17416116 ] Jungtaek Lim commented on SPARK-34183: -- I just changed the affected version as it doesn't seem to

[jira] [Updated] (SPARK-34183) DataSource V2: Support required distribution and ordering in SS

[ https://issues.apache.org/jira/browse/SPARK-34183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-34183: - Affects Version/s: (was: 3.2.0) 3.3.0 > DataSource V2: Support

[jira] [Updated] (SPARK-36762) Fix Series.isin when Series has NaN values

[ https://issues.apache.org/jira/browse/SPARK-36762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dgd_contributor updated SPARK-36762: Affects Version/s: (was: 3.3) 3.3.0 > Fix Series.isin when

[jira] [Updated] (SPARK-36762) Fix Series.isin when Series has NaN values

[ https://issues.apache.org/jira/browse/SPARK-36762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dgd_contributor updated SPARK-36762: Affects Version/s: 3.2.0 > Fix Series.isin when Series has NaN values >

[jira] [Commented] (SPARK-33772) Build and Run Spark on Java 17

[ https://issues.apache.org/jira/browse/SPARK-33772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416071#comment-17416071 ] Lukas Rytz commented on SPARK-33772: [~dongjoon] do you mean an example like this? {code:java} $>

[jira] [Updated] (SPARK-36779) Error when list of data type tuples has len = 1

[ https://issues.apache.org/jira/browse/SPARK-36779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dgd_contributor updated SPARK-36779: Summary: Error when list of data type tuples has len = 1 (was: Fix when list of data

[jira] [Commented] (SPARK-36759) Upgrade Scala to 2.12.15

[ https://issues.apache.org/jira/browse/SPARK-36759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416060#comment-17416060 ] Apache Spark commented on SPARK-36759: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-36777) Move Java 17 on GitHub Actions from EA to LTS release

[ https://issues.apache.org/jira/browse/SPARK-36777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-36777: Affects Version/s: (was: 3.3.0) 3.2.0 > Move Java 17 on GitHub Actions

[jira] [Updated] (SPARK-36781) The log could not get the correct line number

[ https://issues.apache.org/jira/browse/SPARK-36781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenxusheng updated SPARK-36781: Description: INFO 18:16:46 [Thread-1]

[jira] [Created] (SPARK-36781) The log could not get the correct line number

chenxusheng created SPARK-36781: --- Summary: The log could not get the correct line number Key: SPARK-36781 URL: https://issues.apache.org/jira/browse/SPARK-36781 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-36780) Make dev/mima runs on Java 17+

Yuming Wang created SPARK-36780: --- Summary: Make dev/mima runs on Java 17+ Key: SPARK-36780 URL: https://issues.apache.org/jira/browse/SPARK-36780 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-36777) Move Java 17 on GitHub Actions from EA to LTS release

[ https://issues.apache.org/jira/browse/SPARK-36777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-36777. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 34017

[jira] [Commented] (SPARK-23814) Couldn't read file with colon in name and new line character in one of the field.

[ https://issues.apache.org/jira/browse/SPARK-23814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416025#comment-17416025 ] Khaled Radmal commented on SPARK-23814: --- version 3.1.1 also returns the same error, however

[jira] [Commented] (SPARK-36779) Fix when list of data type tuples has len = 1

[ https://issues.apache.org/jira/browse/SPARK-36779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416015#comment-17416015 ] Apache Spark commented on SPARK-36779: -- User 'dgd-contributor' has created a pull request for this

[jira] [Assigned] (SPARK-36779) Fix when list of data type tuples has len = 1

[ https://issues.apache.org/jira/browse/SPARK-36779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36779: Assignee: (was: Apache Spark) > Fix when list of data type tuples has len = 1 >

[jira] [Assigned] (SPARK-36779) Fix when list of data type tuples has len = 1

[ https://issues.apache.org/jira/browse/SPARK-36779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36779: Assignee: Apache Spark > Fix when list of data type tuples has len = 1 >

[jira] [Commented] (SPARK-36779) Fix when list of data type tuples has len = 1

[ https://issues.apache.org/jira/browse/SPARK-36779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416014#comment-17416014 ] Apache Spark commented on SPARK-36779: -- User 'dgd-contributor' has created a pull request for this

  1   2   >