Hi, Yes, you are right, although I guess you can just pass a null if you do not need resources.
Is the multi-threading working for you? R On Mon, Nov 24, 2014 at 6:26 PM, nikhil jain <nikhil_jain1...@yahoo.com> wrote: > Hi Rodrigo, > > I was trying to call train method without resource but I was getting some > errors. I did not find any train method without resources. > > I found these train methods in class NameFinderME: > > 1. train(String languageCode, String type, ObjectStream<NameSample> > samples,TrainingParameters trainParams, byte[] featureGeneratorBytes, > Map<String,Object> resources) > 2. train(String languageCode, String type, ObjectStream<NameSample> > samples,TrainingParameters trainParams, AdaptiveFeatureGenerator > generator,Map<String,Object> resources) > 3. train(String languageCode, String type, ObjectStream<NameSample> > samples,Map<String,Object> resources) > 4. train(String languageCode, String type, ObjectStream<NameSample> > samples,AdaptiveFeatureGenerator generator, Map<String,Object> resources, > int iterations, int cutoff) > > Am I missing something, Could you please tell me how can I do so? > > Thanks > Nikhil > > ________________________________ > From: Rodrigo Agerri <rage...@apache.org> > To: nikhil jain <nikhil_jain1...@yahoo.com> > Sent: Friday, November 21, 2014 12:12 AM > > Subject: Re: Need to speed up the model creation process of OpenNLP > > Hi Nikhil, > > It looks good, but you do not seem to need the resources, though, you > why do not use the train method without the resources? > > Also, do you have 50 threads? > > Rodrigo > > > > On Thu, Nov 20, 2014 at 5:57 PM, nikhil jain <nikhil_jain1...@yahoo.com> > wrote: >> Thanks for the feedback Rodrigo. >> Yes I am trying to create a model based on maximum entropy. As I am using >> API's for building the model, so I tried adding thread param in the >> Training >> parameters object but I am not sure whether I am adding the param >> correctly >> or not. I haven't find any clue in documentation as well. >> >> Here is my code developed with the help of openNLP documentation. Is it >> the >> correct way of creating a maxent model using multi threads? >> >> TrainingParameters tp = new TrainingParameters(); >> tp.put(TrainingParameters.ALGORITHM_PARAM, "MAXENT"); >> tp.put(TrainingParameters.ITERATIONS_PARAM, Integer.toString(100)); >> tp.put(TrainingParameters.CUTOFF_PARAM, Integer.toString(5)); >> tp.put("Threads", "50"); >> >> Map<String, Object> resources = new HashMap<String, Object>(); >> model = NameFinderME.train( "en", "sample", sampleStream, tp, generator, >> resources); >> Thanks >> Nikhil >> >> >> ________________________________ >> From: Rodrigo Agerri <rage...@apache.org> >> To: nikhil jain <nikhil_jain1...@yahoo.com> >> Sent: Thursday, November 20, 2014 11:35 AM >> >> Subject: Re: Need to speed up the model creation process of OpenNLP >> >> Hi Nikhil >> The maxent trainer already allows multi thread training. If you are using >> the cli specify the Threads in your Trainparams file. Check the paramaters >> file sample distributed with opennlp. >> If using it via API perhaps the easiest is to create a TrainingParameters >> object with the threads param specified. >> HTH >> R >> >> >> On 19 Nov 2014 21:19, "nikhil jain" <nikhil_jain1...@yahoo.com> wrote: >> >> Hi Rodrigo, >> >> No, I am not using multi-threading, it's a simple Java program, took help >> from openNLP documentation but it is worth mentioning over here is that as >> the corpus is containing 4 million records so my Java program running in >> eclipse was frequently giving me java heap space issue (out of memory >> issue) >> so I investigate a bit and found that process was taking around 10GB >> memory >> for building the model so i increased the memory to 10 GB using -Xmx >> parameter. so it worked properly but took 3 hours. >> >> Thanks >> -NIkhil >> >> ________________________________ >> From: Rodrigo Agerri <rage...@apache.org> >> To: "dev@opennlp.apache.org" <dev@opennlp.apache.org>; nikhil jain >> <nikhil_jain1...@yahoo.com> >> Cc: "us...@opennlp.apache.org" <us...@opennlp.apache.org> >> Sent: Wednesday, November 19, 2014 2:17 AM >> Subject: Re: Need to speed up the model creation process of OpenNLP >> >> Hi, >> >> Are you using multithreading, lots of threads, RAM memory? >> >> R >> >> >> >> >> On Tue, Nov 18, 2014 at 5:46 PM, nikhil jain >> <nikhil_jain1...@yahoo.com.invalid> wrote: >>> Hi, >>> I asked below question yesterday, did anyone get a chance to look at >>> this. >>> I am new in OpenNLP and really need some help. Please provide some clue >>> or >>> link or example. >>> ThanksNIkhil >>> From: nikhil jain <nikhil_jain1...@yahoo.com.INVALID> >>> To: "us...@opennlp.apache.org" <us...@opennlp.apache.org>; Dev at >>> Opennlp >>> Apache <dev@opennlp.apache.org> >>> Sent: Tuesday, November 18, 2014 12:02 AM >>> Subject: Need to speed up the model creation process of OpenNLP >>> >>> Hi, >>> I am using OpenNLP Token Name Finder for parsing the unstructured data. I >>> have created a corpus of about 4 million records. When I am creating a >>> model >>> out of the training set using openNLP API's in Eclipse using default >>> setting >>> (cut-off 5 and iterations 100), process is taking a good amount of time, >>> around 2-3 hours. >>> Can someone suggest me how can I reduce the time as I want to experiment >>> with different iterations but as the model creation process is taking so >>> much time, I am not able to experiment with it. This is really a time >>> consuming process. >>> Please provide some feedback. >>> Thanks in advance.Nikhil Jain >>> >>> >> >> >> >> > >