Re: Comprehensive Port Configuration reference?

2014-05-11 Thread Mark Baker
On Tue, May 6, 2014 at 9:09 AM, Jacob Eisinger jeis...@us.ibm.com wrote: In a nut shell, Spark opens up a couple of well known ports. And,then the workers and the shell open up dynamic ports for each job. These dynamic ports make securing the Spark network difficult. Indeed. Judging by

Re: Pig on Spark

2014-04-25 Thread Mark Baker
I've only had a quick look at Pig, but it seems that a declarative layer on top of Spark couldn't be anything other than a big win, as it allows developers to declare *what* they want, permitting the compiler to determine how best poke at the RDD API to implement it. In my brief time with Spark,

Re: Master registers itself at startup?

2014-04-12 Thread Mark Baker
On Sat, Apr 12, 2014 at 9:19 AM, ge ko koenig@gmail.com wrote: Hi, I'm wondering why the master is registering itself at startup, exactly 3 times (same number as the number of workers). Log excerpt: 2014-04-11 21:08:15,363 INFO akka.event.slf4j.Slf4jLogger: Slf4jLogger started