unsubscribe

unsubscribe

Pass xmx values to SparkLauncher launched Java process

Hi, We have a usecase where we are submitting multiple spark jobs using SparkLauncher from a Java class. We are currently in a memory crunch situation on our edge node where we see that the Java processes spawned by the launcher is taking around 1 GB. Is there a way to pass JMX parameters to this

How grouping rows without shuffle

Hi all, I'm trying to group X rows in a single one without shuffling the date. I was thinking doing something like that : val myDF = Seq(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11).toDF("myColumn") myDF.withColumn("myColumn", expr("sliding(myColumn, 3)")) expected result: myColumn [1,2,3] [4,5,6]

help needed with SPARK-45598 and SPARK-45769

Greetings, tl;dr there must have been a regression in spark *connect*'s ability to retrieve data, more details in linked issues https://issues.apache.org/jira/browse/SPARK-45598 https://issues.apache.org/jira/browse/SPARK-45769 we have projects that depend on spark connect 3.5 and we'd