Steve,
On May 15, 2008, at 5:05 AM, Steve Loughran wrote:
I have a question for users: how do they ensure their client apps
have configuration XML file that are kept up to date?
I know how I do it to date (get the site config off the site team,
have my private copy in SVN), but that is too brittle, and
diagnosing failures is pretty tricky. All you get is "Failed to
Submit Job!" exceptions and local stack traces, from which you have
to work backwards to the work.
I'm thinking of looking at what it would take for a job submitter
to ask the tracker for its config data, to get things like the
various directory bases from the cluster, instead of being compiled
into the client. Then the management problem becomes one of keeping
the cluster configuration under control, which is a much easier
proposition.
Fair analysis. http://issues.apache.org/jira/browse/HADOOP-3135 is
quite close, should get in hadoop-0.18 and will help a great deal. It
does precisely what you proposed: fix the job-submitter to query the
JT to get the right configs.
Arun
