Hi Chetan, You can pretty much use any client to do this. When I was using Spark at a previous job, we used OkHttp, but I'm sure there are plenty of others. In our case, we had a startup phase in which we gathered metadata via a REST API and then broadcast it to the workers. I think if you need all the workers to have access to whatever you're getting from the API, that's the way to do it.
Jerry On Thu, May 14, 2020 at 5:03 PM Chetan Khatri <chetan.opensou...@gmail.com> wrote: > Hi Spark Users, > > How can I invoke the Rest API call from Spark Code which is not only > running on Spark Driver but distributed / parallel? > > Spark with Scala is my tech stack. > > Thanks > > > -- http://www.google.com/profiles/grapesmoker
