Try setting spark.driver.host to the actual ip or hostname of the box
submitting the work. More info the networking section in this link:
http://spark.apache.org/docs/latest/configuration.html
Also check the spark config for your application for these driver settings in
the application web UI
I've been querying Zookeeper directly via the Zookeeper client tools, it has
the ip of the current master leader in the master_status data. We are also
running Exhibitor for zookeeper which has a nice UI for exploring if you want
to look up manually
Thanks,
Michal
On May 12, 2015, at 1:28
According to the docs it should go like this:
spark://host1:port1,host2:port2
https://spark.apache.org/docs/latest/spark-standalone.html#standby-masters-with-zookeeper
Thanks
M
On Apr 28, 2015, at 8:13 AM, James King jakwebin...@gmail.com wrote:
I have multiple masters running and I'm
Not sure if there's a spark native way but we've been using consul for this.
M
On Apr 26, 2015, at 5:17 AM, James King jakwebin...@gmail.com wrote:
Thanks for the response.
But no this does not answer the question.
The question was: Is there a way (via some API call) to query the
A SparkContext can submit jobs remotely.
The spark-submit options in general can be populated into a SparkConf and
passed in when you create a SparkContext.
We personally have not had too much success with yarn-client remote submission,
but standalone cluster mode was easy to get going.
M
and would like to just divy out the result set.
M
On Mar 1, 2015, at 5:18 AM, Jörn Franke jornfra...@gmail.com wrote:
What database are you using?
Le 28 févr. 2015 18:15, Michal Klos michal.klo...@gmail.com a écrit :
Hi Spark community,
We have a use case where we need to pull huge amounts
the query results into another table in
your database and then query that using the normal approach?
--eric
On 3/1/15 4:28 AM, michal.klo...@gmail.com wrote:
Jorn: Vertica
Cody: I posited the limit just as an example of how jdbcrdd could be used
least invasively. Let's say we used a partition
