Join push down in DSv2

2023-11-27 Thread Stefan Hagedorn
Hi, At the Spark Summit 2017 Ioana Delaney presented an approach for join pushdown in Apache Spark [1]. Is there any intent to actually bring this into Spark, especially in the DSv2 interface? Does anyone know if there's ongoing work or a document about this? [1]

unsubscribe

2023-11-03 Thread Stefan Hagedorn

Re: Spark Connect: API mismatch in SparkSesession#execute

2023-08-29 Thread Stefan Hagedorn
Thank you, Martin! I got it working now using the same shading rules in my project as in Spark. From: Martin Grund Date: Monday, 28. August 2023 at 17:58 To: Stefan Hagedorn Cc: dev@spark.apache.org Subject: Re: Spark Connect: API mismatch in SparkSesession#execute Hi Stefan, There are some

Spark Connect: API mismatch in SparkSesession#execute

2023-08-28 Thread Stefan Hagedorn
Hi everyone, Trying my luck here, after no success in the user mailing list :) I’m trying to use the "extension" feature of the Spark Connect CommandPlugin (Spark 3.4.1) [1]. I created a simple protobuf message `MyMessage` that I want to send from the connect client-side to the connect server