Re: Spark dev-setup

2016-08-25 Thread Nishadi Kirielle
Thanks a lot for the guidelines. I could successfully configure and debug On Wed, Aug 24, 2016 at 7:05 PM, Jacek Laskowski wrote: > On Wed, Aug 24, 2016 at 2:32 PM, Steve Loughran > wrote: > > > no reason; the key thing is : not in cluster mode, as

Re: Spark dev-setup

2016-08-24 Thread Jacek Laskowski
On Wed, Aug 24, 2016 at 2:32 PM, Steve Loughran wrote: > no reason; the key thing is : not in cluster mode, as there your work happens > elsewhere Right! Anything but cluster mode should make it easy (that leaves us with local). Jacek

Re: Spark dev-setup

2016-08-24 Thread Steve Loughran
> On 24 Aug 2016, at 11:38, Jacek Laskowski wrote: > > On Wed, Aug 24, 2016 at 11:13 AM, Steve Loughran > wrote: > >> I'd recommend > > ...which I mostly agree to with some exceptions :) > >> -stark spark standalone from there > > Why spark

Re: Spark dev-setup

2016-08-24 Thread Jacek Laskowski
On Wed, Aug 24, 2016 at 11:13 AM, Steve Loughran wrote: > I'd recommend ...which I mostly agree to with some exceptions :) > -stark spark standalone from there Why spark standalone since the OP asked about "learning how query execution flow occurs in Spark SQL"? How

Re: Spark dev-setup

2016-08-24 Thread Steve Loughran
On 24 Aug 2016, at 07:10, Nishadi Kirielle > wrote: Hi, I'm engaged in learning how query execution flow occurs in Spark SQL. In order to understand the query execution flow, I'm attempting to run an example in debug mode with intellij IDEA. It

Re: Spark dev-setup

2016-08-24 Thread Nishadi Kirielle
Hi, I'm engaged in learning how query execution flow occurs in Spark SQL. In order to understand the query execution flow, I'm attempting to run an example in debug mode with intellij IDEA. It would be great if anyone can help me with debug configurations. Thanks & Regards Nishadi On Tue, Jun