No, programs can't read information out of other processes's memory - this
is true of all software. A cached DataFrame is tied to a Spark
application, but many things could be running inside that app. You may be
thinking of something like, exposing some kind of service on the driver
that responds
Thanks for response. I want to figure out is there a way to share data without
writing it to disk because of performance issues.
Chenyang Zhang
Software Engineering Intern, Platform
Redwood City, California
Spark is decoupled from storage. You can write data to any storage you
like. Anything that can read that data, can read that data - Spark or not,
different session or not. Temp views are specific to a session and do not
store data. I think this is trivial and no problem at all, or else I'm not
Hi,
Thanks so much for the response from
https://issues.apache.org/jira/browse/SPARK-39549. I am curious what do you
mean by write down to a table and read it from a different Spark Application.
Do you mean a table in a database or the Spark to_table() api? Could I read the
table created in
