Re: [External]Re: spark 2.x design docs

2019-09-19 Thread yeikel valdes
-core sources then building and running jobs on it. Form printed logs I try to understand execution flows.   From: Vipul Rajan Sent: Thursday, September 19, 2019 12:23 PM To: Kamal7 Kumar Cc: spark-user Subject: [External]Re: spark 2.x design docs   The e-mail below is from an external

Re: [External]Re: spark 2.x design docs

2019-09-19 Thread Vipul Rajan
hursday, September 19, 2019 12:23 PM > *To:* Kamal7 Kumar > *Cc:* spark-user > *Subject:* [External]Re: spark 2.x design docs > > > > The e-mail below is from an external source. Please do not open > attachments or click links from an unknown or suspicious origin. >

RE: [External]Re: spark 2.x design docs

2019-09-19 Thread Kamal7.Kumar
:23 PM To: Kamal7 Kumar Cc: spark-user Subject: [External]Re: spark 2.x design docs The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin. https://github.com/JerryLead/SparkInternals/blob/master/EnglishVersion/2

Re: spark 2.x design docs

2019-09-19 Thread Vipul Rajan
https://github.com/JerryLead/SparkInternals/blob/master/EnglishVersion/2-JobLogicalPlan.md This is pretty old. but it might help a little bit. I myself am going through the source code and trying to reverse engineer stuff. Let me know if you'd like to pool resources sometime. Regards On Thu, Sep

spark 2.x design docs

2019-09-19 Thread Kamal7.Kumar
Hi , Can someone provide documents/links (apart from official documentation) for understanding internal workings of spark-core, Document containing components pseudo codes, class diagrams, execution flows , etc. Thanks, Kamal "Confidentiality Warning: This message and any attachments are