Umesh,
I found the following write-up dealing with architecture and memory
considerations elaborately. There are updates on memory, but it would
be a good start for you:
https://0x0fff.com/spark-architecture/
Any additional source(s) of info. are welcome from others too.
- Venkat.
On Sun, Sep
Hi I have application which servers as ETL job and I have hundreds of such
ETL jobs which runs daily now as of now I have just one spark session which
is shared by all these jobs and sometimes all of these jobs run at the same
time causing spark session to die due memory issues mostly. Is this a
