Hi Edward, Thanks for committing it.
In the future, it would be nicer to commit only the changes related to the respective ticket/patch, to make it easier for others to follow/review the development. In this particular case, you committed as well changes relative to the job paths. It would be easier for others if those changes came in a separate commit. cheers On Mon, Sep 27, 2010 at 5:28 AM, <[email protected]> wrote: > Author: edwardyoon > Date: Mon Sep 27 04:28:32 2010 > New Revision: 1001586 > > URL: http://svn.apache.org/viewvc?rev=1001586&view=rev > Log: > BSPMaster should use the bsp.master.port config property when creating its > InetSocketAddr instance. > > Modified: > incubator/hama/trunk/CHANGES.txt > incubator/hama/trunk/conf/hama-default.xml > incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPJobClient.java > incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPMaster.java > incubator/hama/trunk/src/java/org/apache/hama/bsp/GroomServer.java > incubator/hama/trunk/src/java/org/apache/hama/bsp/JobInProgress.java > > Modified: incubator/hama/trunk/CHANGES.txt > URL: > http://svn.apache.org/viewvc/incubator/hama/trunk/CHANGES.txt?rev=1001586&r1=1001585&r2=1001586&view=diff > ============================================================================== > --- incubator/hama/trunk/CHANGES.txt (original) > +++ incubator/hama/trunk/CHANGES.txt Mon Sep 27 04:28:32 2010 > @@ -46,6 +46,8 @@ Trunk (unreleased changes) > > IMPROVEMENTS > > + HAMA-287: BSPMaster should use the bsp.master.port config property > + when creating its InetSocketAddr instance (Filipe > Manana via edwardyoon) > HAMA-283: Removing duplicate code (Filipe Manana via edwardyoon) > HAMA-277: Add default number of bsp task (edwardyoon) > HAMA-273: Implement killJob() method for local job (edwardyoon) > > Modified: incubator/hama/trunk/conf/hama-default.xml > URL: > http://svn.apache.org/viewvc/incubator/hama/trunk/conf/hama-default.xml?rev=1001586&r1=1001585&r2=1001586&view=diff > ============================================================================== > --- incubator/hama/trunk/conf/hama-default.xml (original) > +++ incubator/hama/trunk/conf/hama-default.xml Mon Sep 27 04:28:32 2010 > @@ -26,7 +26,8 @@ > <name>bsp.master.address</name> > <value>local</value> > <description>The address of the bsp master server. Either the > - literal string "local" or a host:port for distributed mode > + literal string "local" or a host[:port] (where host is a name or > + IP address) for distributed mode. > </description> > </property> > <property> > > Modified: incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPJobClient.java > URL: > http://svn.apache.org/viewvc/incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPJobClient.java?rev=1001586&r1=1001585&r2=1001586&view=diff > ============================================================================== > --- incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPJobClient.java > (original) > +++ incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPJobClient.java Mon > Sep 27 04:28:32 2010 > @@ -237,6 +237,8 @@ public class BSPJobClient extends Config > Path submitJarFile = new Path(submitJobDir, "job.jar"); > Path submitJobFile = new Path(submitJobDir, "job.xml"); > > + LOG.debug("BSPJobClient.submitJobDir: " + submitJobDir); > + > /* > * set this user's id in job configuration, so later job files can be > * accessed using this user's id > > Modified: incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPMaster.java > URL: > http://svn.apache.org/viewvc/incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPMaster.java?rev=1001586&r1=1001585&r2=1001586&view=diff > ============================================================================== > --- incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPMaster.java > (original) > +++ incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPMaster.java Mon Sep > 27 04:28:32 2010 > @@ -237,16 +237,10 @@ public class BSPMaster implements JobSub > } > > public static InetSocketAddress getAddress(Configuration conf) { > - String hamaMasterStr = conf.get("bsp.master.address", "localhost:40000"); > - return NetUtils.createSocketAddr(hamaMasterStr); > - } > - > - public int getPort() { > - return this.conf.getInt("bsp.master.port", 0); > - } > + String hamaMasterStr = conf.get("bsp.master.address", "localhost"); > + int defaultPort = conf.getInt("bsp.master.port", 40000); > > - public Configuration getConf() { > - return this.conf; > + return NetUtils.createSocketAddr(hamaMasterStr, defaultPort); > } > > private static SimpleDateFormat getDateFormat() { > > Modified: incubator/hama/trunk/src/java/org/apache/hama/bsp/GroomServer.java > URL: > http://svn.apache.org/viewvc/incubator/hama/trunk/src/java/org/apache/hama/bsp/GroomServer.java?rev=1001586&r1=1001585&r2=1001586&view=diff > ============================================================================== > --- incubator/hama/trunk/src/java/org/apache/hama/bsp/GroomServer.java > (original) > +++ incubator/hama/trunk/src/java/org/apache/hama/bsp/GroomServer.java Mon > Sep 27 04:28:32 2010 > @@ -391,23 +391,32 @@ public class GroomServer implements Runn > this.task = task; > } > > + static final String SUBDIR = "groomServer"; > + > public void launchTask() { > // until job is completed, don't accept new task > acceptNewTasks = false; > > try { > // TODO: need to move this code to TaskRunner > - Path localJobFile = new > Path(task.getJobFile().replace(systemFS.getUri().toString(), "")); > - String localJarFile = localJobFile.toString().replace(".xml", > ".jar"); > + task.getJobFile(); > + conf.addResource(task.getJobFile()); > + BSPJob defaultJobConf = new BSPJob((HamaConfiguration) conf); > + > + Path localJobFile = > + > defaultJobConf.getLocalPath(SUBDIR+"/"+task.getTaskID()+"/"+"job.xml"); > + Path localJarFile = > + > defaultJobConf.getLocalPath(SUBDIR+"/"+task.getTaskID()+"/"+"job.jar"); > + > LOG.debug("localJobFile: "+ localJobFile); > > systemFS.copyToLocalFile(new Path(task.getJobFile()), localJobFile); > - systemFS.copyToLocalFile(new Path(task.getJobFile().replace(".xml", > ".jar")), new Path(localJarFile)); > + systemFS.copyToLocalFile(new Path(task.getJobFile().replace(".xml", > ".jar")), localJarFile); > > HamaConfiguration conf = new HamaConfiguration(); > conf.addResource(localJobFile); > BSPJob jobConf = new BSPJob(conf, task.getJobID()); > - jobConf.setJar(localJarFile); > + jobConf.setJar(localJarFile.toString()); > > BSP bsp = (BSP) ReflectionUtils.newInstance(jobConf.getBspClass(), > conf); > bsp.setPeer(bspPeer); > > Modified: incubator/hama/trunk/src/java/org/apache/hama/bsp/JobInProgress.java > URL: > http://svn.apache.org/viewvc/incubator/hama/trunk/src/java/org/apache/hama/bsp/JobInProgress.java?rev=1001586&r1=1001585&r2=1001586&view=diff > ============================================================================== > --- incubator/hama/trunk/src/java/org/apache/hama/bsp/JobInProgress.java > (original) > +++ incubator/hama/trunk/src/java/org/apache/hama/bsp/JobInProgress.java Mon > Sep 27 04:28:32 2010 > @@ -76,6 +76,10 @@ class JobInProgress { > + ".xml"); > this.localJarFile = master.getLocalPath(BSPMaster.SUBDIR + "/" + jobId > + ".jar"); > + > + LOG.debug("JobInProgress.localJobFile: " + this.localJobFile); > + LOG.debug("JobInProgress.localJarFile: " + this.localJarFile); > + > Path jobDir = master.getSystemDirectoryForJob(jobId); > FileSystem fs = jobDir.getFileSystem(conf); > jobFile = new Path(jobDir, "job.xml"); > > > -- Filipe David Manana, [email protected], [email protected] "Reasonable men adapt themselves to the world. Unreasonable men adapt the world to themselves. That's why all progress depends on unreasonable men."
