Hello Luca Favatella, and friends,
Octave can be used in ways Matlab cannot be, for example
in problems which have data-parallelism ( same problem solved
for different initial conditions/ data ) Octave gives a large speedup
factors.
A master Octave can spawn several Octave processess whose parameters
are written in a disk and then wait for all of them to end, and pickup
the resulting data
from known saved files. So this would easily allow throwing
multicore resources at a problem which allows data parallelism, which otherwise
would run as just a single Octave serial process.
In a GA toolbox we evaluate same function/program for different
data and this can make use of the data parallelism.
Under the assumptions of having a Fitness functions (f(x)) to be
specified as a string, and
[ncores] has to be the number of cores to be used in the GA fitness evaluation,
specified as an attribute to GA problem structure, we can do this by creating
replacing __ga_scores__.m script with the following pair of scripts,
__ga_multicore_scores__.m
__ga_multicore_helper__.m
You can test these scripts by, running test_multicore.m .
(please see attachments. )
We need fitness functions as a string, since child processess donot share
memory with the parent/master process.
Ofcourse the benefit of a multicore fitness function evaluator doesnt
show up until
out f(x) is very time-consuming. This is a neat problem with explicit
parallelism.
I would like to help integrate these scripts with the current design
of the GA toolbox
by adding the attributes to the GA.m, so I am stopping from directly
editing the rest
of the toolbox.
Best,
-Muthu
## Copyright (C) 2010 Muthiah Annamalai <[email protected]>
##
## This program is free software; you can redistribute it and/or modify
## it under the terms of the GNU General Public License as published by
## the Free Software Foundation; either version 2 of the License, or
## (at your option) any later version.
##
## This program is distributed in the hope that it will be useful,
## but WITHOUT ANY WARRANTY; without even the implied warranty of
## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
## GNU General Public License for more details.
##
## You should have received a copy of the GNU General Public License
## along with this program; If not, see <http://www.gnu.org/licenses/>.
##
## usage: octave -q __ga_multicore_helper__.m fitnessfcn matfile
## and save the result in proc-$matfile in the variable fitnessval.
## load(procfile); ## has variable fitnessval.
##
more off
parameters = argv();
fprintf('Child process %d: command line received :\n',getpid());
parameters
fitnessfcn = parameters{1}; matfile = parameters{2};
load(matfile);
fitnessval = feval( fitnessfcn, data(:) );
## fitnessval
save('-mat',sprintf('proc-%s',matfile),'fitnessval');
## Copyright (C) 2010 Muthiah Annamalai <[email protected]>
## Copyright (C) 2008 Luca Favatella <[email protected]>
##
## This program is free software; you can redistribute it and/or modify
## it under the terms of the GNU General Public License as published by
## the Free Software Foundation; either version 2 of the License, or
## (at your option) any later version.
##
## This program is distributed in the hope that it will be useful,
## but WITHOUT ANY WARRANTY; without even the implied warranty of
## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
## GNU General Public License for more details.
##
## You should have received a copy of the GNU General Public License
## along with this program; If not, see <http://www.gnu.org/licenses/>.
## Author: Luca Favatella <[email protected]>
## Version: 5.3.1
##
## Modified for use with Multicore CPU to utilize data-parallel
## nature of GA fitness functions, which we cannot do with limited
## License software. Go Octave!
##
## Assumptions, Fitness functions have to be a m-file, since we
## donot share memory with the child process.
## [ncores] has to be the number of cores to be used in the GA
## fitness evaluation, obtained as struct element of GA problem,
## and passed down to us.
##
function Scores = __ga_multicore_scores__ (fitnessfcn, Population, ncores, InitialScores = [])
[nrPopulation ncPopulation] = size (Population);
[nrInitialScores ncInitialScores] = size (InitialScores);
#assert ((ncInitialScores == 0) || (ncInitialScores == 1)); ## DEBUG
#assert (nrInitialScores <= nrPopulation); ## DEBUG
if (! isempty (InitialScores))
Scores(1:nrInitialScores, 1) = InitialScores;
endif
probstr = sprintf('GA-%d-Pop-%dx%d-',round(rand()*1e6),nrPopulation,ncPopulation);
datestr = strrep(ctime(time),':','-');
matfileprefix = sprintf('%s-%s',probstr,datestr);
matfileprefix = strrep(matfileprefix,' ','-');
index = (nrInitialScores + 1);
while ( index <= nrPopulation )
matfilename = {};
## save ncore data elements to file
idx = index;
while ( idx <= nrPopulation && idx < index + ncores )
matfile = sprintf('%s-%d.mat',matfileprefix(1:end-1),idx);
matfilename{idx-index + 1} = matfile;
data = Population(index, 1:ncPopulation);
save('-mat',matfile,'data');
idx = idx + 1;
end
nprocs = idx-index; ## number of processess to be started.
## start the processess with the marshaller function
idx = index; pid_child = zeros( nprocs, 1 );
while ( idx <= nrPopulation && idx < index + ncores )
matfile = matfilename{idx-index + 1};
pid_child( idx - index + 1 ) = system(sprintf('octave -q __ga_multicore_helper__.m "%s" "%s" ',fitnessfcn, matfile),'','async');
idx = idx + 1;
end
## wait for the processess to finish
for idx = 1:nprocs
waitpid( pid_child(idx), 0 );
fprintf('Terminated Child %d\n',pid_child(idx));
end
## matfilename; nprocs
## load results from the processed cousins
for idx = 1:nprocs
matfile = matfilename{idx}
procfile = sprintf('proc-%s',matfile);
load(procfile); ## has variable fitnessval.
Scores(index+idx-1,1) = fitnessval;
unlink(matfile);
unlink(procfile);
end
index = index + ncores;
end
## Scores; nrPopulation; size( Scores );
Scores(1:nrPopulation, 1) = Scores;
endfunction
pop = [[1:10]' rand(10,1)];
__ga_multicore_scores__( "length", pop, 3 )
------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com
_______________________________________________
Octave-dev mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/octave-dev
