[jira] [Comment Edited] (SPARK-36229) conv() inconsistently handles invalid strings with > 64 invalid characters

[ https://issues.apache.org/jira/browse/SPARK-36229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384666#comment-17384666 ] dgd_contributor edited comment on SPARK-36229 at 7/21/21, 5:44 AM: ---

[jira] [Commented] (SPARK-36229) conv() inconsistently handles invalid strings with > 64 invalid characters

[ https://issues.apache.org/jira/browse/SPARK-36229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384666#comment-17384666 ] dgd_contributor commented on SPARK-36229: - After look closely, I found out that the overflow

[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384660#comment-17384660 ] Gengliang Wang commented on SPARK-25075: [~dongjoon]Thanks for the ping. Yes, we should have

[jira] [Commented] (SPARK-36030) Support DS v2 metrics at writing path

[ https://issues.apache.org/jira/browse/SPARK-36030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384659#comment-17384659 ] Apache Spark commented on SPARK-36030: -- User 'viirya' has created a pull request for this issue:

[jira] [Created] (SPARK-36236) RocksDB state store: Add additional metrics for better observability into state store operations

Venki Korukanti created SPARK-36236: --- Summary: RocksDB state store: Add additional metrics for better observability into state store operations Key: SPARK-36236 URL:

[jira] [Commented] (SPARK-36030) Support DS v2 metrics at writing path

[ https://issues.apache.org/jira/browse/SPARK-36030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384658#comment-17384658 ] Apache Spark commented on SPARK-36030: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-36030) Support DS v2 metrics at writing path

[ https://issues.apache.org/jira/browse/SPARK-36030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384657#comment-17384657 ] Apache Spark commented on SPARK-36030: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-36030) Support DS v2 metrics at writing path

[ https://issues.apache.org/jira/browse/SPARK-36030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384654#comment-17384654 ] Apache Spark commented on SPARK-36030: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-36030) Support DS v2 metrics at writing path

[ https://issues.apache.org/jira/browse/SPARK-36030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384653#comment-17384653 ] Apache Spark commented on SPARK-36030: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-36206) Diagnose shuffle data corruption by checksum

[ https://issues.apache.org/jira/browse/SPARK-36206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384643#comment-17384643 ] Apache Spark commented on SPARK-36206: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36206) Diagnose shuffle data corruption by checksum

[ https://issues.apache.org/jira/browse/SPARK-36206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36206: Assignee: Apache Spark > Diagnose shuffle data corruption by checksum >

[jira] [Commented] (SPARK-36206) Diagnose shuffle data corruption by checksum

[ https://issues.apache.org/jira/browse/SPARK-36206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384642#comment-17384642 ] Apache Spark commented on SPARK-36206: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36206) Diagnose shuffle data corruption by checksum

[ https://issues.apache.org/jira/browse/SPARK-36206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36206: Assignee: (was: Apache Spark) > Diagnose shuffle data corruption by checksum >

[jira] [Commented] (SPARK-35809) Add `index_col` argument for ps.sql.

[ https://issues.apache.org/jira/browse/SPARK-35809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384639#comment-17384639 ] Apache Spark commented on SPARK-35809: -- User 'itholic' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35809) Add `index_col` argument for ps.sql.

[ https://issues.apache.org/jira/browse/SPARK-35809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35809: Assignee: Apache Spark > Add `index_col` argument for ps.sql. >

[jira] [Assigned] (SPARK-35809) Add `index_col` argument for ps.sql.

[ https://issues.apache.org/jira/browse/SPARK-35809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35809: Assignee: (was: Apache Spark) > Add `index_col` argument for ps.sql. >

[jira] [Commented] (SPARK-35809) Add `index_col` argument for ps.sql.

[ https://issues.apache.org/jira/browse/SPARK-35809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384636#comment-17384636 ] Apache Spark commented on SPARK-35809: -- User 'itholic' has created a pull request for this issue:

[jira] [Resolved] (SPARK-36235) Supported datatype check should check inner field

[ https://issues.apache.org/jira/browse/SPARK-36235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-36235. --- Resolution: Not A Bug > Supported datatype check should check inner field >

[jira] [Commented] (SPARK-10816) EventTime based sessionization (session window)

[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384626#comment-17384626 ] Jungtaek Lim commented on SPARK-10816: -- I don't think I could finalize this one without your

[jira] [Resolved] (SPARK-36030) Support DS v2 metrics at writing path

[ https://issues.apache.org/jira/browse/SPARK-36030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-36030. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33239

[jira] [Created] (SPARK-36235) Supported datatype check should check inner field

angerszhu created SPARK-36235: - Summary: Supported datatype check should check inner field Key: SPARK-36235 URL: https://issues.apache.org/jira/browse/SPARK-36235 Project: Spark Issue Type:

[jira] [Commented] (SPARK-36229) conv() inconsistently handles invalid strings with > 64 invalid characters

[ https://issues.apache.org/jira/browse/SPARK-36229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384610#comment-17384610 ] dgd_contributor commented on SPARK-36229: - thanks, I will look into this   > conv()

[jira] [Updated] (SPARK-36153) Add SQL doc about transform for current behavior

[ https://issues.apache.org/jira/browse/SPARK-36153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-36153: - Summary: Add SQL doc about transform for current behavior (was: Add SLQ doc about transform

[jira] [Resolved] (SPARK-36153) Add SLQ doc about transform for current behavior

[ https://issues.apache.org/jira/browse/SPARK-36153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-36153. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33362

[jira] [Assigned] (SPARK-36153) Add SLQ doc about transform for current behavior

[ https://issues.apache.org/jira/browse/SPARK-36153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-36153: Assignee: angerszhu > Add SLQ doc about transform for current behavior >

[jira] [Updated] (SPARK-36153) Add SLQ doc about transform for current behavior

[ https://issues.apache.org/jira/browse/SPARK-36153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-36153: - Priority: Minor (was: Major) > Add SLQ doc about transform for current behavior >

[jira] [Assigned] (SPARK-35658) Document Parquet encryption feature in Spark

[ https://issues.apache.org/jira/browse/SPARK-35658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-35658: Assignee: Gidon Gershinsky > Document Parquet encryption feature in Spark >

[jira] [Updated] (SPARK-35658) Document Parquet encryption feature in Spark

[ https://issues.apache.org/jira/browse/SPARK-35658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-35658: - Priority: Minor (was: Major) > Document Parquet encryption feature in Spark >

[jira] [Resolved] (SPARK-35658) Document Parquet encryption feature in Spark

[ https://issues.apache.org/jira/browse/SPARK-35658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-35658. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 32895

[jira] [Commented] (SPARK-10816) EventTime based sessionization (session window)

[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384592#comment-17384592 ] L. C. Hsieh commented on SPARK-10816: - Yea, excited to have the great work in the next 3.2 release!

[jira] [Updated] (SPARK-35027) Close the inputStream in FileAppender when writing the logs failure

[ https://issues.apache.org/jira/browse/SPARK-35027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-35027: - Priority: Minor (was: Major) Resolved by https://github.com/apache/spark/pull/33263 > Close

[jira] [Resolved] (SPARK-35027) Close the inputStream in FileAppender when writing the logs failure

[ https://issues.apache.org/jira/browse/SPARK-35027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-35027. -- Fix Version/s: 3.0.4 3.1.3 3.2.0 Resolution:

[jira] [Assigned] (SPARK-35027) Close the inputStream in FileAppender when writing the logs failure

[ https://issues.apache.org/jira/browse/SPARK-35027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-35027: Assignee: Jack Hu > Close the inputStream in FileAppender when writing the logs failure

[jira] [Reopened] (SPARK-35027) Close the inputStream in FileAppender when writing the logs failure

[ https://issues.apache.org/jira/browse/SPARK-35027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reopened SPARK-35027: -- > Close the inputStream in FileAppender when writing the logs failure >

[jira] [Commented] (SPARK-10816) EventTime based sessionization (session window)

[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384586#comment-17384586 ] Yuanjian Li commented on SPARK-10816: - Thrilled to see this issue got resolved finally! Thank you

[jira] [Commented] (SPARK-36088) 'spark.archives' does not extract the archive file into the driver under client mode

[ https://issues.apache.org/jira/browse/SPARK-36088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384576#comment-17384576 ] rickcheng commented on SPARK-36088: --- Hi, [~srowen] Thanks for the comment. I agree that in client

[jira] [Assigned] (SPARK-35310) Bump Breeze from 1.0 to 1.2

[ https://issues.apache.org/jira/browse/SPARK-35310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35310: Assignee: Apache Spark > Bump Breeze from 1.0 to 1.2 > --- > >

[jira] [Assigned] (SPARK-35310) Bump Breeze from 1.0 to 1.2

[ https://issues.apache.org/jira/browse/SPARK-35310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35310: Assignee: (was: Apache Spark) > Bump Breeze from 1.0 to 1.2 >

[jira] [Commented] (SPARK-35310) Bump Breeze from 1.0 to 1.2

[ https://issues.apache.org/jira/browse/SPARK-35310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384570#comment-17384570 ] Apache Spark commented on SPARK-35310: -- User 'srowen' has created a pull request for this issue:

[jira] [Commented] (SPARK-35310) Bump Breeze from 1.0 to 1.2

[ https://issues.apache.org/jira/browse/SPARK-35310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384571#comment-17384571 ] Apache Spark commented on SPARK-35310: -- User 'srowen' has created a pull request for this issue:

[jira] [Commented] (SPARK-36088) 'spark.archives' does not extract the archive file into the driver under client mode

[ https://issues.apache.org/jira/browse/SPARK-36088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384569#comment-17384569 ] rickcheng commented on SPARK-36088: --- Hi, [~hyukjin.kwon] Thanks for the comment. After my test, under

[jira] [Resolved] (SPARK-10816) EventTime based sessionization (session window)

[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-10816. -- Fix Version/s: 3.2.0 Assignee: Jungtaek Lim Resolution: Fixed I'm resolving

[jira] [Assigned] (SPARK-36188) Add categories setter to CategoricalAccessor and CategoricalIndex.

[ https://issues.apache.org/jira/browse/SPARK-36188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36188: Assignee: Apache Spark > Add categories setter to CategoricalAccessor and

[jira] [Commented] (SPARK-36188) Add categories setter to CategoricalAccessor and CategoricalIndex.

[ https://issues.apache.org/jira/browse/SPARK-36188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384567#comment-17384567 ] Apache Spark commented on SPARK-36188: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36188) Add categories setter to CategoricalAccessor and CategoricalIndex.

[ https://issues.apache.org/jira/browse/SPARK-36188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36188: Assignee: (was: Apache Spark) > Add categories setter to CategoricalAccessor and

[jira] [Resolved] (SPARK-36172) Document session window in Structured Streaming guide doc

[ https://issues.apache.org/jira/browse/SPARK-36172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-36172. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33433

[jira] [Assigned] (SPARK-36172) Document session window in Structured Streaming guide doc

[ https://issues.apache.org/jira/browse/SPARK-36172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-36172: Assignee: Jungtaek Lim > Document session window in Structured Streaming guide doc >

[jira] [Resolved] (SPARK-36186) Add as_ordered/as_unordered to CategoricalAccessor and CategoricalIndex.

[ https://issues.apache.org/jira/browse/SPARK-36186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36186. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Issue

[jira] [Commented] (SPARK-36145) Remove Python 3.6 support in codebase and CI

[ https://issues.apache.org/jira/browse/SPARK-36145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384544#comment-17384544 ] Hyukjin Kwon commented on SPARK-36145: -- Oh, this JIRA actually targets to remove Python 3.9 out

[jira] [Commented] (SPARK-35848) Spark Bloom Filter, others using treeAggregate can throw OutOfMemoryError

[ https://issues.apache.org/jira/browse/SPARK-35848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384540#comment-17384540 ] Sean R. Owen commented on SPARK-35848: -- I think you'd get "Requested array size exceeds VM limit"

[jira] [Commented] (SPARK-35848) Spark Bloom Filter, others using treeAggregate can throw OutOfMemoryError

[ https://issues.apache.org/jira/browse/SPARK-35848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384527#comment-17384527 ] Sai Polisetty commented on SPARK-35848: --- Thanks for taking a look at it, Sean. I am using [Azure

[jira] [Created] (SPARK-36234) Consider mapper location and shuffle block size in OptimizeLocalShuffleReader

Michael Zhang created SPARK-36234: - Summary: Consider mapper location and shuffle block size in OptimizeLocalShuffleReader Key: SPARK-36234 URL: https://issues.apache.org/jira/browse/SPARK-36234

[jira] [Commented] (SPARK-36218) Flaky Test: TPC-DS in PR builder

[ https://issues.apache.org/jira/browse/SPARK-36218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384495#comment-17384495 ] Dongjoon Hyun commented on SPARK-36218: --- Oh, thank you for sharing, [~hyukjin.kwon]. > Flaky

[jira] [Commented] (SPARK-35310) Bump Breeze from 1.0 to 1.2

[ https://issues.apache.org/jira/browse/SPARK-35310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384491#comment-17384491 ] Sean R. Owen commented on SPARK-35310: -- FWIW it's easy to work around the compile error, I did

[jira] [Updated] (SPARK-36000) Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled

[ https://issues.apache.org/jira/browse/SPARK-36000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36000: - Description: The creation and operations of ps.Series/Index with Decimal('NaN') doesn't work

[jira] [Updated] (SPARK-36000) Support creation and operations of ps.Series/Index with Decimal('NaN')

[ https://issues.apache.org/jira/browse/SPARK-36000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36000: - Summary: Support creation and operations of ps.Series/Index with Decimal('NaN') (was: Support

[jira] [Updated] (SPARK-36000) Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled

[ https://issues.apache.org/jira/browse/SPARK-36000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36000: - Description: The creation and operations of ps.Series/Index have bugs. Please refer to

[jira] [Updated] (SPARK-36232) Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled

[ https://issues.apache.org/jira/browse/SPARK-36232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36232: - Description:   {code:java} >>> import decimal as d >>> import pyspark.pandas as ps >>> import

[jira] [Created] (SPARK-36233) Spark Kube Integration tests must be run from project root

Holden Karau created SPARK-36233: Summary: Spark Kube Integration tests must be run from project root Key: SPARK-36233 URL: https://issues.apache.org/jira/browse/SPARK-36233 Project: Spark

[jira] [Commented] (SPARK-36141) Support arithmetic operations of Series containing Decimal(np.nan)

[ https://issues.apache.org/jira/browse/SPARK-36141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384485#comment-17384485 ] Xinrong Meng commented on SPARK-36141: -- It is closed because it is duplicated from 

[jira] [Created] (SPARK-36232) Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled

Xinrong Meng created SPARK-36232: Summary: Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled Key: SPARK-36232 URL: https://issues.apache.org/jira/browse/SPARK-36232

[jira] [Created] (SPARK-36231) Support arithmetic operations of Series containing Decimal(np.nan)

Xinrong Meng created SPARK-36231: Summary: Support arithmetic operations of Series containing Decimal(np.nan) Key: SPARK-36231 URL: https://issues.apache.org/jira/browse/SPARK-36231 Project: Spark

[jira] [Resolved] (SPARK-36141) Support arithmetic operations of Series containing Decimal(np.nan)

[ https://issues.apache.org/jira/browse/SPARK-36141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-36141. -- Resolution: Duplicate > Support arithmetic operations of Series containing Decimal(np.nan) >

[jira] [Created] (SPARK-36230) hasnans for Series of Decimal(`NaN`)

Xinrong Meng created SPARK-36230: Summary: hasnans for Series of Decimal(`NaN`) Key: SPARK-36230 URL: https://issues.apache.org/jira/browse/SPARK-36230 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384469#comment-17384469 ] Dongjoon Hyun commented on SPARK-25075: --- > It looks like scala 2.12 will still be the default,

[jira] [Updated] (SPARK-36210) Preserve column insertion order in Dataset.withColumns

[ https://issues.apache.org/jira/browse/SPARK-36210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-36210: Affects Version/s: 3.2.0 3.0.3 > Preserve column insertion order in

[jira] [Updated] (SPARK-36229) conv() inconsistently handles invalid strings with > 64 invalid characters

[ https://issues.apache.org/jira/browse/SPARK-36229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong updated SPARK-36229: -- Environment: (was: SPARK-33428 fixed ArrayIndexOutofBoundsException but introduced a new

[jira] [Commented] (SPARK-36229) conv() inconsistently handles invalid strings with > 64 invalid characters

[ https://issues.apache.org/jira/browse/SPARK-36229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384441#comment-17384441 ] Tim Armstrong commented on SPARK-36229: --- [~dgd_contributor] [~wenchen] > conv() inconsistently

[jira] [Updated] (SPARK-36229) conv() inconsistently handles invalid strings with > 64 invalid characters

[ https://issues.apache.org/jira/browse/SPARK-36229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong updated SPARK-36229: -- Description: SPARK-33428 fixed ArrayIndexOutofBoundsException but introduced a new

[jira] [Created] (SPARK-36229) conv() inconsistently handles invalid strings with > 64 invalid characters

Tim Armstrong created SPARK-36229: - Summary: conv() inconsistently handles invalid strings with > 64 invalid characters Key: SPARK-36229 URL: https://issues.apache.org/jira/browse/SPARK-36229

[jira] [Updated] (SPARK-36229) conv() inconsistently handles invalid strings with > 64 invalid characters

[ https://issues.apache.org/jira/browse/SPARK-36229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong updated SPARK-36229: -- Affects Version/s: (was: 3.1.2) 3.2.0 > conv() inconsistently

[jira] [Resolved] (SPARK-35546) Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the state in a better way

[ https://issues.apache.org/jira/browse/SPARK-35546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou resolved SPARK-35546. - Fix Version/s: 3.2.0 Target Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull

[jira] [Updated] (SPARK-36143) Adjust astype of Series with missing values to follow pandas

[ https://issues.apache.org/jira/browse/SPARK-36143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36143: - Summary: Adjust astype of Series with missing values to follow pandas (was: Adjust astype of

[jira] [Assigned] (SPARK-36215) Add logging for slow fetches to diagnose external shuffle service issues

[ https://issues.apache.org/jira/browse/SPARK-36215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36215: Assignee: (was: Apache Spark) > Add logging for slow fetches to diagnose external

[jira] [Assigned] (SPARK-36215) Add logging for slow fetches to diagnose external shuffle service issues

[ https://issues.apache.org/jira/browse/SPARK-36215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36215: Assignee: Apache Spark > Add logging for slow fetches to diagnose external shuffle

[jira] [Commented] (SPARK-36215) Add logging for slow fetches to diagnose external shuffle service issues

[ https://issues.apache.org/jira/browse/SPARK-36215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384428#comment-17384428 ] Apache Spark commented on SPARK-36215: -- User 'shardulm94' has created a pull request for this

[jira] [Assigned] (SPARK-36227) Remove TimestampNTZ type support in Spark 3.2

[ https://issues.apache.org/jira/browse/SPARK-36227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36227: Assignee: Gengliang Wang (was: Apache Spark) > Remove TimestampNTZ type support in

[jira] [Assigned] (SPARK-35848) Spark Bloom Filter, others using treeAggregate can throw OutOfMemoryError

[ https://issues.apache.org/jira/browse/SPARK-35848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35848: Assignee: Sean R. Owen (was: Apache Spark) > Spark Bloom Filter, others using

[jira] [Assigned] (SPARK-36227) Remove TimestampNTZ type support in Spark 3.2

[ https://issues.apache.org/jira/browse/SPARK-36227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36227: Assignee: Apache Spark (was: Gengliang Wang) > Remove TimestampNTZ type support in

[jira] [Commented] (SPARK-36227) Remove TimestampNTZ type support in Spark 3.2

[ https://issues.apache.org/jira/browse/SPARK-36227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384403#comment-17384403 ] Apache Spark commented on SPARK-36227: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-35848) Spark Bloom Filter, others using treeAggregate can throw OutOfMemoryError

[ https://issues.apache.org/jira/browse/SPARK-35848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35848: Assignee: Apache Spark (was: Sean R. Owen) > Spark Bloom Filter, others using

[jira] [Commented] (SPARK-35848) Spark Bloom Filter, others using treeAggregate can throw OutOfMemoryError

[ https://issues.apache.org/jira/browse/SPARK-35848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384402#comment-17384402 ] Apache Spark commented on SPARK-35848: -- User 'srowen' has created a pull request for this issue:

[jira] [Commented] (SPARK-36228) Skip splitting a reducer partition when some mapStatus is null

[ https://issues.apache.org/jira/browse/SPARK-36228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384400#comment-17384400 ] Apache Spark commented on SPARK-36228: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36228) Skip splitting a reducer partition when some mapStatus is null

[ https://issues.apache.org/jira/browse/SPARK-36228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36228: Assignee: Apache Spark > Skip splitting a reducer partition when some mapStatus is null

[jira] [Assigned] (SPARK-36228) Skip splitting a reducer partition when some mapStatus is null

[ https://issues.apache.org/jira/browse/SPARK-36228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36228: Assignee: (was: Apache Spark) > Skip splitting a reducer partition when some

[jira] [Resolved] (SPARK-35353) Cross-building docker images to ARM64 is failing (with Ubuntu host)

[ https://issues.apache.org/jira/browse/SPARK-35353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-35353. -- Resolution: Not A Problem OK, that doesn't seem directly related to Spark as distributed by

[jira] [Resolved] (SPARK-35365) spark3.1.1 use too long time to analyze table fields

[ https://issues.apache.org/jira/browse/SPARK-35365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-35365. -- Resolution: Invalid This is more a question for the user@ list than a specific issue report

[jira] [Resolved] (SPARK-35517) Critical Vulnerabilities: jackson-databind 2.4.0 shipped with htrace-core4-4.1.0-incubating.jar

[ https://issues.apache.org/jira/browse/SPARK-35517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-35517. -- Resolution: Not A Problem > Critical Vulnerabilities: jackson-databind 2.4.0 shipped with >

[jira] [Commented] (SPARK-35517) Critical Vulnerabilities: jackson-databind 2.4.0 shipped with htrace-core4-4.1.0-incubating.jar

[ https://issues.apache.org/jira/browse/SPARK-35517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384395#comment-17384395 ] Sean R. Owen commented on SPARK-35517: -- Spark 3.2 is coming in a month or so. Spark doesn't use it

[jira] [Commented] (SPARK-35519) Critical Vulnerabilities: nimbusds_nimbus-jose-jwt 4.41.1 shipped

[ https://issues.apache.org/jira/browse/SPARK-35519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384394#comment-17384394 ] Sean R. Owen commented on SPARK-35519: -- We generally do not accept reports like "my static analyzer

[jira] [Commented] (SPARK-35518) Critical Vulnerabilities: log4j_log4j 1.2.17 shipped

[ https://issues.apache.org/jira/browse/SPARK-35518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384387#comment-17384387 ] Sean R. Owen commented on SPARK-35518: -- I tried this a looong time ago and it was very hard. Spark

[jira] [Created] (SPARK-36228) Skip splitting a reducer partition when some mapStatus is null

Wenchen Fan created SPARK-36228: --- Summary: Skip splitting a reducer partition when some mapStatus is null Key: SPARK-36228 URL: https://issues.apache.org/jira/browse/SPARK-36228 Project: Spark

[jira] [Commented] (SPARK-35837) Recommendations for Common Query Problems

[ https://issues.apache.org/jira/browse/SPARK-35837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384375#comment-17384375 ] Sean R. Owen commented on SPARK-35837: -- What would this look like in Spark though? how do you

[jira] [Assigned] (SPARK-35848) Spark Bloom Filter, others using treeAggregate can throw OutOfMemoryError

[ https://issues.apache.org/jira/browse/SPARK-35848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-35848: Component/s: ML Target Version/s: 3.3.0 Affects Version/s: (was:

[jira] [Created] (SPARK-36227) Remove TimestampNTZ type support in Spark 3.2

Gengliang Wang created SPARK-36227: -- Summary: Remove TimestampNTZ type support in Spark 3.2 Key: SPARK-36227 URL: https://issues.apache.org/jira/browse/SPARK-36227 Project: Spark Issue

[jira] [Resolved] (SPARK-36222) Step by days in the Sequence expression for dates

[ https://issues.apache.org/jira/browse/SPARK-36222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-36222. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33439

[jira] [Assigned] (SPARK-36222) Step by days in the Sequence expression for dates

[ https://issues.apache.org/jira/browse/SPARK-36222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-36222: Assignee: jiaan.geng > Step by days in the Sequence expression for dates >

[jira] [Resolved] (SPARK-36210) Preserve column insertion order in Dataset.withColumns

[ https://issues.apache.org/jira/browse/SPARK-36210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-36210. - Fix Version/s: 3.1.3 3.2.0 3.0.4 Resolution: Fixed

[jira] [Assigned] (SPARK-36210) Preserve column insertion order in Dataset.withColumns

[ https://issues.apache.org/jira/browse/SPARK-36210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-36210: --- Assignee: koert kuipers > Preserve column insertion order in Dataset.withColumns >

[jira] [Commented] (SPARK-36020) Check logical link in remove redundant projects

[ https://issues.apache.org/jira/browse/SPARK-36020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384360#comment-17384360 ] Apache Spark commented on SPARK-36020: -- User 'cloud-fan' has created a pull request for this issue:

  1   2   >