I think it would be better to have the client retrieve the default configuration. Not all configuration settings are simple overrides. Some are read-modify-write operations.
This also fits the current code better. On 5/19/08 6:38 AM, "Steve Loughran" <[EMAIL PROTECTED]> wrote: > Alejandro Abdelnur wrote: >> A while ago I've opened an issue related to this topic >> >> https://issues.apache.org/jira/browse/HADOOP-3287 >> >> My take is a little different, when submitting a job, the clients >> should only send to the jobtracker the configuration they explicitly >> set, then the job tracker would apply the defaults for all the other >> configuration. >> >> By doing this the cluster admin can modify things at any time and >> changes on default values take effect for all clients without having >> to distribute a new configuration to all clients. >> >> IMO, this approach was the intended behavior at some point, according >> to the Configuration.write(OutputStream) javadocs ' Writes non-default >> properties in this configuration.'. But as the write method is writing >> default properties this is not happening. > > I'll keep an eye on that issue. I think a key problem right now is that > clients take their config from the configuration file in the core jar, > and from their own settings, You need to keep the settings in sync > somehow, and have to take what the core jar provides. > > >> This approach would also get rid of the separate mechanism (zookeeper, >> svn, etc) to keep clients synchronized as there would be no need to do >> so. > > zookeeper and similar are to keep the cluster alive; they shouldnt be > needed for clients, which should only need some URL of a job tracker to > talk to.
