Re: Call Site - Spark Context

[Sandip Mehta]

Thanks Robin. 

Regards
SM
> On 01-Oct-2015, at 3:15 pm, Robin East  wrote:
> 
> From the comments in the code:
> 
> When called inside a class in the spark package, returns the name of the user 
> code class (outside the spark package) that called into Spark, as well as 
> which Spark method they called. This is used, for example, to tell users 
> where in their code each RDD got created.
> Keep crawling up the stack trace until we find the first function not inside 
> of the spark package. We track the last (shallowest) contiguous Spark method. 
> This might be an RDD transformation, a SparkContext function (such as 
> parallelize), or anything else that leads to instantiation of an RDD. We also 
> track the first (deepest) user method, file, and line.
> So basically it’s a mechanism to report where in the user’s code an RDD is 
> created.
> ---
> Robin East
> Spark GraphX in Action Michael Malak and Robin East
> Manning Publications Co.
> http://www.manning.com/books/spark-graphx-in-action 
> 
> 
> 
> 
> 
> 
>> On 1 Oct 2015, at 23:06, Sandip Mehta > > wrote:
>> 
>> Hi,
>> 
>> I wanted to understand what is the purpose of Call Site in Spark Context?
>> 
>> Regards
>> SM
>> 
>> -
>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org 
>> 
>> For additional commands, e-mail: user-h...@spark.apache.org 
>> 
>> 
> 

Re: Call Site - Spark Context

[Sandip Mehta]

Thanks Robin. 

Regards
SM
> On 01-Oct-2015, at 3:15 pm, Robin East  wrote:
> 
> From the comments in the code:
> 
> When called inside a class in the spark package, returns the name of the user 
> code class (outside the spark package) that called into Spark, as well as 
> which Spark method they called. This is used, for example, to tell users 
> where in their code each RDD got created.
> Keep crawling up the stack trace until we find the first function not inside 
> of the spark package. We track the last (shallowest) contiguous Spark method. 
> This might be an RDD transformation, a SparkContext function (such as 
> parallelize), or anything else that leads to instantiation of an RDD. We also 
> track the first (deepest) user method, file, and line.
> So basically it’s a mechanism to report where in the user’s code an RDD is 
> created.
> ---
> Robin East
> Spark GraphX in Action Michael Malak and Robin East
> Manning Publications Co.
> http://www.manning.com/books/spark-graphx-in-action 
> 
> 
> 
> 
> 
> 
>> On 1 Oct 2015, at 23:06, Sandip Mehta > > wrote:
>> 
>> Hi,
>> 
>> I wanted to understand what is the purpose of Call Site in Spark Context?
>> 
>> Regards
>> SM
>> 
>> -
>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org 
>> 
>> For additional commands, e-mail: user-h...@spark.apache.org 
>> 
>> 
> 

Call Site - Spark Context

[Sandip Mehta]

Hi,

I wanted to understand what is the purpose of Call Site in Spark Context?

Regards
SM

-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org