Re: SQLContext within foreachRDD

2015-10-12 Thread Daniel Haviv
v > Date: Monday, October 12, 2015 at 12:52 PM > To: user > Subject: SQLContext within foreachRDD > > Hi, > As things that run inside foreachRDD run at the driver, does that mean > that if we use SQLContext inside foreachRDD the data is sent back to the > driver and only

Re: SQLContext within foreachRDD

2015-10-12 Thread Adrian Tanase
/scala/com/databricks/apps/logs/chapter1/LogAnalyzerStreamingSQL.scala -adrian From: Daniel Haviv Date: Monday, October 12, 2015 at 12:52 PM To: user Subject: SQLContext within foreachRDD Hi, As things that run inside foreachRDD run at the driver, does that mean that if we use SQLContext inside

SQLContext within foreachRDD

2015-10-12 Thread Daniel Haviv
Hi, As things that run inside foreachRDD run at the driver, does that mean that if we use SQLContext inside foreachRDD the data is sent back to the driver and only then the query is executed or is it executed at the executors? Thank you. Daniel