More or less as you described. I exploit jMetal to build the problem and the algorithm is it really possible to integrate Flink in jMetal?
Best, Andrea > Il giorno 13/nov/2014, alle ore 21:44, <[email protected]> > <[email protected]> ha scritto: > > Hi Andrea, > > Please correct me if I got something wrong. > You have a population of several classifiers that you want to evolve and > improve. > In each iteration, you test each classifier with all your test data and > evaluate the fitness of each classifier. Then, you create a new population of > classifiers by removing bad classifiers and mutating (and crossing) the > better ones. > > I think you can do that with Flink as follows: > you use a Map over the test data and a broadcast set as the classifiers to > check the classification of a single attribute. With a following reduce, you > aggregate the fitness of each classifier and use a reduce all to build a new > population of classifiers. > In the next iteration, the new population is broadcasted to the Map over the > test data. > > This is quite similar to what our KMeans example does. You should have a look > at it. > > Best, Fabian > > From: Kostas Tzoumas <mailto:[email protected]> > Sent: Thursday, 13. November, 2014 17:11 > To: [email protected] <mailto:[email protected]> > Cc: Andrea Ferranti <mailto:[email protected]> > > I am forwarding this here in case someone with better knowledge of genetic > algorithms picks it up. > > Kostas > > ---------- Forwarded message ---------- > From: Andrea Ferranti <[email protected] > <mailto:[email protected]>> > Date: Thu, Nov 13, 2014 at 4:46 PM > Subject: Re: Information on Flink > To: Kostas Tzoumas <[email protected] <mailto:[email protected]>> > > > Thanks very much for your reply. > > First, can I forward this to the Flink user mailing list? Perhaps someone > over there has a better answer. > > Yes, of course. > > Can you describe very briefly how fitness evaluation is computed in your > algorithm? > > My fitness evaluation is basically an evaluation of accuracy in a > classification problem, so i must read every line of file(in which is present > some attribute and a class) and verify if my classification work well. > So in each iteration of a genetic algorithm i change some chromosome and than > evaluate the solution. > > At the moment the entire program is written in C++ but I would take it in > java using jMetal > > Best regards, > Andrea > > Il giorno 13/nov/2014, alle ore 16:25, Kostas Tzoumas <[email protected] > <mailto:[email protected]>> ha scritto: > > Hey, > > First, can I forward this to the Flink user mailing list? Perhaps someone > over there has a better answer. > > Can you describe very briefly how fitness evaluation is computed in your > algorithm? > > Kostas > > On Thu, Nov 13, 2014 at 4:08 PM, Andrea Ferranti <[email protected] > <mailto:[email protected]>> wrote: > Dear Kostas Tzoumas, > I'm Andrea Ferranti, a student of Computer Engineering at the University of > Pisa. > In my thesis I would like to exploit Flink to parallelize a Evolutionary > algorithm, in particular the fitness evaluation. My problem and algorithm are > written in Java (jMetal). > > > Do you think that flink can be a good tool for the parallelization of > fitness? In my problem the fitness is evaluate on very big datasets. > > Best regards, > Andrea
