[jira] [Updated] (SPARK-48428) IllegalStateException due to nested column aliasing

2024-05-27 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ejbyfeldt updated SPARK-48428: --- Description: {code:java} val f = udf[((Int, Int), Int), ((Int, Int), Int)](identity) val

[jira] [Created] (SPARK-48428) IllegalStateException due to nested column aliasing

2024-05-27 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-48428: -- Summary: IllegalStateException due to nested column aliasing Key: SPARK-48428 URL: https://issues.apache.org/jira/browse/SPARK-48428 Project: Spark

[jira] [Updated] (SPARK-47927) Nullability after join not respected in UDF

2024-04-24 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ejbyfeldt updated SPARK-47927: --- Labels: correctness pull-request-available (was: pull-request-available) > Nullability

[jira] [Created] (SPARK-47927) Nullability after join not respected in UDF

2024-04-21 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-47927: -- Summary: Nullability after join not respected in UDF Key: SPARK-47927 URL: https://issues.apache.org/jira/browse/SPARK-47927 Project: Spark Issue Type:

[jira] [Updated] (SPARK-46421) Broken support for explode on a Map in typed API

2023-12-15 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ejbyfeldt updated SPARK-46421: --- Description:   {code:java} scala> spark.createDataset(Seq(Tuple1(Map(1 ->

[jira] [Created] (SPARK-46421) Broken support for explode on a Map in typed API

2023-12-15 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-46421: -- Summary: Broken support for explode on a Map in typed API Key: SPARK-46421 URL: https://issues.apache.org/jira/browse/SPARK-46421 Project: Spark Issue

[jira] [Updated] (SPARK-45592) AQE and InMemoryTableScanExec correctness bug

2023-11-09 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ejbyfeldt updated SPARK-45592: --- Description: The following query should return 100 {code:java} import

[jira] [Commented] (SPARK-45282) Join loses records for cached datasets

2023-11-09 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784320#comment-17784320 ] Emil Ejbyfeldt commented on SPARK-45282: Created this

[jira] [Created] (SPARK-45849) Remove uneccessary toSeq when encoding Set to catalyst

2023-11-09 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-45849: -- Summary: Remove uneccessary toSeq when encoding Set to catalyst Key: SPARK-45849 URL: https://issues.apache.org/jira/browse/SPARK-45849 Project: Spark

[jira] [Commented] (SPARK-45282) Join loses records for cached datasets

2023-11-08 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784109#comment-17784109 ] Emil Ejbyfeldt commented on SPARK-45282: The code reproducing the bug looks quite similar to

[jira] [Created] (SPARK-45820) Support encoding of scala.collection.immutable.ArraySeq

2023-11-07 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-45820: -- Summary: Support encoding of scala.collection.immutable.ArraySeq Key: SPARK-45820 URL: https://issues.apache.org/jira/browse/SPARK-45820 Project: Spark

[jira] [Created] (SPARK-45592) AQE and InMemoryTableScanExec correctness bug

2023-10-18 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-45592: -- Summary: AQE and InMemoryTableScanExec correctness bug Key: SPARK-45592 URL: https://issues.apache.org/jira/browse/SPARK-45592 Project: Spark Issue

[jira] [Created] (SPARK-45386) Correctness issue when persisting using StorageLevel.NONE

2023-09-30 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-45386: -- Summary: Correctness issue when persisting using StorageLevel.NONE Key: SPARK-45386 URL: https://issues.apache.org/jira/browse/SPARK-45386 Project: Spark

[jira] [Updated] (SPARK-38101) MetadataFetchFailedException due to decommission block migrations

2023-09-04 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ejbyfeldt updated SPARK-38101: --- Description: As noted in SPARK-34939 there is race when using broadcast for map output

[jira] [Created] (SPARK-44777) Allow to specify eagerness to RDD.checkpoint

2023-08-11 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-44777: -- Summary: Allow to specify eagerness to RDD.checkpoint Key: SPARK-44777 URL: https://issues.apache.org/jira/browse/SPARK-44777 Project: Spark Issue Type:

[jira] [Created] (SPARK-44376) Build using maven is broken using 2.13 and Java 11 and Java 17

2023-07-11 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-44376: -- Summary: Build using maven is broken using 2.13 and Java 11 and Java 17 Key: SPARK-44376 URL: https://issues.apache.org/jira/browse/SPARK-44376 Project: Spark

[jira] [Created] (SPARK-44311) UDF should support function taking value classes

2023-07-05 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-44311: -- Summary: UDF should support function taking value classes Key: SPARK-44311 URL: https://issues.apache.org/jira/browse/SPARK-44311 Project: Spark Issue

[jira] [Updated] (SPARK-43378) SerializerHelper.deserializeFromChunkedBuffer leaks deserialization streams

2023-06-06 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ejbyfeldt updated SPARK-43378: --- Summary: SerializerHelper.deserializeFromChunkedBuffer leaks deserialization streams (was:

[jira] [Created] (SPARK-43378) SerializerHelper.deserializeFromChunkedBuffer

2023-05-04 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-43378: -- Summary: SerializerHelper.deserializeFromChunkedBuffer Key: SPARK-43378 URL: https://issues.apache.org/jira/browse/SPARK-43378 Project: Spark Issue

[jira] [Commented] (SPARK-43138) ClassNotFoundException during RDD block replication/migration

2023-04-16 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17712781#comment-17712781 ] Emil Ejbyfeldt commented on SPARK-43138: No. The class `com.class.from.user.jar.ClassName` is

[jira] [Updated] (SPARK-43138) ClassNotFoundException during RDD block replication/migration

2023-04-14 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ejbyfeldt updated SPARK-43138: --- Summary: ClassNotFoundException during RDD block replication/migration (was: ClassNotFound

[jira] [Created] (SPARK-43138) ClassNotFound during RDD block replication/migration

2023-04-14 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-43138: -- Summary: ClassNotFound during RDD block replication/migration Key: SPARK-43138 URL: https://issues.apache.org/jira/browse/SPARK-43138 Project: Spark

[jira] [Updated] (SPARK-39696) Uncaught exception in thread executor-heartbeater java.util.ConcurrentModificationException: mutation occurred during iteration

2023-04-05 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ejbyfeldt updated SPARK-39696: --- Attachment: (was: 0001-Spec-that-produces-race-condition-with-hearbeater-th.patch) >

[jira] [Updated] (SPARK-39696) Uncaught exception in thread executor-heartbeater java.util.ConcurrentModificationException: mutation occurred during iteration

2023-04-05 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ejbyfeldt updated SPARK-39696: --- Fix Version/s: 3.4.0 > Uncaught exception in thread executor-heartbeater >

[jira] [Updated] (SPARK-39696) Uncaught exception in thread executor-heartbeater java.util.ConcurrentModificationException: mutation occurred during iteration

2023-04-05 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ejbyfeldt updated SPARK-39696: --- Affects Version/s: 3.4.0 > Uncaught exception in thread executor-heartbeater >

[jira] [Commented] (SPARK-39696) Uncaught exception in thread executor-heartbeater java.util.ConcurrentModificationException: mutation occurred during iteration

2023-04-04 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17708453#comment-17708453 ] Emil Ejbyfeldt commented on SPARK-39696: Created a test case that consistently reproduces the

[jira] [Updated] (SPARK-39696) Uncaught exception in thread executor-heartbeater java.util.ConcurrentModificationException: mutation occurred during iteration

2023-04-04 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ejbyfeldt updated SPARK-39696: --- Attachment: 0001-Spec-that-produces-race-condition-with-hearbeater-th.patch > Uncaught

[jira] [Updated] (SPARK-40950) isRemoteAddressMaxedOut performance overhead on scala 2.13

2022-10-28 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ejbyfeldt updated SPARK-40950: --- Summary: isRemoteAddressMaxedOut performance overhead on scala 2.13 (was: On scala 2.13

[jira] [Created] (SPARK-40950) On scala 2.13 isRemoteAddressMaxedOut performance overhead

2022-10-28 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-40950: -- Summary: On scala 2.13 isRemoteAddressMaxedOut performance overhead Key: SPARK-40950 URL: https://issues.apache.org/jira/browse/SPARK-40950 Project: Spark

[jira] [Updated] (SPARK-40912) Overhead of Exceptions in DeserializationStream

2022-10-25 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ejbyfeldt updated SPARK-40912: --- Description: The interface of DeserializationStream forces implementation to raise

[jira] [Created] (SPARK-40912) Large overhead of Exceptions in DeserializationStream

2022-10-25 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-40912: -- Summary: Large overhead of Exceptions in DeserializationStream Key: SPARK-40912 URL: https://issues.apache.org/jira/browse/SPARK-40912 Project: Spark

[jira] [Updated] (SPARK-40912) Overhead of Exceptions in DeserializationStream

2022-10-25 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ejbyfeldt updated SPARK-40912: --- Summary: Overhead of Exceptions in DeserializationStream (was: Large overhead of

[jira] [Created] (SPARK-40803) LZ4CompressionCodec looks up configuration on each stream creation

2022-10-15 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-40803: -- Summary: LZ4CompressionCodec looks up configuration on each stream creation Key: SPARK-40803 URL: https://issues.apache.org/jira/browse/SPARK-40803 Project:

[jira] [Created] (SPARK-40771) Estimated size in log message can overflow Int

2022-10-12 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-40771: -- Summary: Estimated size in log message can overflow Int Key: SPARK-40771 URL: https://issues.apache.org/jira/browse/SPARK-40771 Project: Spark Issue

[jira] [Resolved] (SPARK-40662) Serialization of MapStatuses is somtimes much larger on scala 2.13

2022-10-07 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ejbyfeldt resolved SPARK-40662. Resolution: Invalid The increase was caused by change in hashCode between 2.12 and 2.13

[jira] [Created] (SPARK-40662) Serialization of MapStatuses is somtimes much larger on scala 2.13

2022-10-05 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-40662: -- Summary: Serialization of MapStatuses is somtimes much larger on scala 2.13 Key: SPARK-40662 URL: https://issues.apache.org/jira/browse/SPARK-40662 Project:

[jira] [Created] (SPARK-40385) Classes with companion object constructor fails interpreted path

2022-09-07 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-40385: -- Summary: Classes with companion object constructor fails interpreted path Key: SPARK-40385 URL: https://issues.apache.org/jira/browse/SPARK-40385 Project: Spark

[jira] [Commented] (SPARK-38681) Support nested generic case classes

2022-05-19 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539441#comment-17539441 ] Emil Ejbyfeldt commented on SPARK-38681: While testing the spark 3.3.0 release candidate I

[jira] [Created] (SPARK-38681) Support nested generic case classes

2022-03-29 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-38681: -- Summary: Support nested generic case classes Key: SPARK-38681 URL: https://issues.apache.org/jira/browse/SPARK-38681 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-38502) Distribution with hadoop-provided is missing log4j2

2022-03-10 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ejbyfeldt resolved SPARK-38502. Resolution: Duplicate > Distribution with hadoop-provided is missing log4j2 >

[jira] [Commented] (SPARK-38502) Distribution with hadoop-provided is missing log4j2

2022-03-10 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17504749#comment-17504749 ] Emil Ejbyfeldt commented on SPARK-38502: Duplicate of SPARK-38516 > Distribution with

[jira] [Updated] (SPARK-38502) Distribution with hadoop-provided is missing log4j2

2022-03-10 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ejbyfeldt updated SPARK-38502: --- Summary: Distribution with hadoop-provided is missing log4j2 (was: Distribution with

[jira] [Created] (SPARK-38502) Distribution with hadoop-provided and log4j2

2022-03-10 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-38502: -- Summary: Distribution with hadoop-provided and log4j2 Key: SPARK-38502 URL: https://issues.apache.org/jira/browse/SPARK-38502 Project: Spark Issue Type:

[jira] [Updated] (SPARK-38502) Distribution with hadoop-provided and log4j2

2022-03-10 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emil Ejbyfeldt updated SPARK-38502: --- Description: Currently building spark 3.3.0-SNAPSHOT using `./dev/make-distribution.sh

[jira] [Commented] (SPARK-38101) MetadataFetchFailedException due to decommission block migrations

2022-03-09 Thread Emil Ejbyfeldt (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17504028#comment-17504028 ] Emil Ejbyfeldt commented on SPARK-38101: The race condition only exists when broadcast is used

[jira] [Created] (SPARK-38101) MetadataFetchFailedException due to decommission block migrations

2022-02-03 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-38101: -- Summary: MetadataFetchFailedException due to decommission block migrations Key: SPARK-38101 URL: https://issues.apache.org/jira/browse/SPARK-38101 Project: Spark

[jira] [Created] (SPARK-37071) OpenHashMap should be serializable without reference tracking

2021-10-20 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-37071: -- Summary: OpenHashMap should be serializable without reference tracking Key: SPARK-37071 URL: https://issues.apache.org/jira/browse/SPARK-37071 Project: Spark

[jira] [Created] (SPARK-35653) [SQL] CatalystToExternalMap interpreted path fails for Map with case classes as keys or values

2021-06-04 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-35653: -- Summary: [SQL] CatalystToExternalMap interpreted path fails for Map with case classes as keys or values Key: SPARK-35653 URL: