I'm looking at Distributed Shell as an example for writing a YARN
application.
My question is why are the script path and associated metadata saved as
environment variables? Are there any other ways besides environment
variables or command line arguments for passing data from the Client to
t
Command line arguments & env variables are the most direct options.
A more onerous option is to write some data to a file in HDFS, use
LocalResource to ship it to the container on each node and get application code
to read that file locally. (In MRv1 parlance that is "Distributed Cache").
hth,
while your at it, what about adding values to the Configuration() object,
does that still work as a hack for information passing?
On Wed, Jan 29, 2014 at 5:25 PM, Arun C Murthy wrote:
> Command line arguments & env variables are the most direct options.
>
> A more onerous option is to write som
Adding values to a Configuration object does not really work unless you
serialize the config into a file and send it over to the AM and containers as a
local resource. The application code would then need to load in this file using
Configuration::addResource(). MapReduce does this by taking in a
