If you're running low-res searches (i.e. fragment_bin_size = 1.0005) then you can probably afford to set this value even higher. Just try starting a search and keeping an eye on memory use. As soon as each batch of spectra is loaded and being searched against the sequence database, that should be approximately the maximum amount of memory that Comet will be using. So you want to set the batch size as big as possible without running out of memory, say shoot for 80% to 90% memory use. The larger the batch size, the more efficient the search is because it reduces some redundant calculations. At some point, the performance gains do level off.
Figure 3 of the manuscript "A deeper look into Comet--implementation and features." shows the trade off between batch size, memory use and run times: http://link.springer.com/article/10.1007%2Fs13361-015-1179-x On Wed, Aug 10, 2016 at 1:02 PM, Eslam Nouri <eslamno...@gmail.com> wrote: > Hi Jimmy, > > Thanks a lot for the recommendation. I changed the spectra batch size from > 1000 to 5000 and could analyze the same DDA sample with 50% improvement in > the analysis time. I will surely check the 10000 spectra batch size to see > if I can obtain shorter analysis time. > > Thanks, Eslam > > On Wed, Aug 10, 2016 at 7:54 AM, Jimmy Eng <jke...@gmail.com> wrote: > >> How much memory/RAM do you have on your computer? It won't make a huge >> difference but larger spectrum batch sizes are more efficient. So see if >> you have enough free memory to run with that parameter set at say 5000. >> With our 16GB linux computers performing high-res ms/ms searches, I use >> batch sizes of 10000. >> >> On Wed, Aug 10, 2016 at 12:27 AM, Eslam Nouri <eslamno...@gmail.com> >> wrote: >> >>> Thanks a lot Jimmy. It is a relief to know that I am doing the best that >>> I can do with my current system. >>> >>> I have already removed all the dynamic modifications that I had (no >>> methionine oxidation or phosphorylations were defined). I have also >>> confined the analysis to 1000 spectra batch size. Database search takes a >>> long time, but it identifies a large number of microbiome proteins with >>> high confidence. >>> >>> Eslam >>> >>> On Tue, Aug 9, 2016 at 11:39 PM, Jimmy Eng <jke...@gmail.com> wrote: >>> >>>> Eslam, >>>> >>>> Setting "num_threads = 0" is the best you can do. There's no free >>>> lunch ... a database that's 100 times larger than the human UniProt fasta >>>> will take approximately 100 times longer to run, e.g. a 10 minute human >>>> search would translate to a 17 hour search if the database were 100 times >>>> larger. You can give things up (no variable modifications, no missed >>>> cleavages, narrow precursor mass tolerance, etc.) to hopefully speed things >>>> up a little bit. Otherwise you just have to throw more horsepower at it. >>>> You can get Broadwell-class Xeon processors with up to 24 cores in them. >>>> >>>> On Tue, Aug 9, 2016 at 11:30 PM, Eslam Nouri <eslamno...@gmail.com> >>>> wrote: >>>> >>>>> Hi Eric, >>>>> >>>>> I am using comet. In my comet file the number of threads is equal to >>>>> zero "num_threads = 0". I have attached my system information. Is >>>>> there any way to increase the data analysis time by defining a thread >>>>> number? >>>>> >>>>> Thanks, Eslam >>>>> >>>>> On Tue, Aug 9, 2016 at 11:18 PM, Eric Deutsch < >>>>> edeut...@systemsbiology.org> wrote: >>>>> >>>>>> Which search engine are you using? Comet? Tandem? >>>>>> >>>>>> >>>>>> >>>>>> Comet has num_threads: >>>>>> >>>>>> http://comet-ms.sourceforge.net/parameters/parameters_201601 >>>>>> /num_threads.php >>>>>> >>>>>> >>>>>> >>>>>> Tandem has threads: >>>>>> >>>>>> http://www.thegpm.org/TANDEM/api/st.html >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> *From:* spctools-discuss@googlegroups.com [mailto: >>>>>> spctools-discuss@googlegroups.com] *On Behalf Of *Eslam Nouri >>>>>> *Sent:* Tuesday, August 9, 2016 11:05 PM >>>>>> *To:* spctools-discuss@googlegroups.com >>>>>> *Subject:* [spctools-discuss] Multi core analysis >>>>>> >>>>>> >>>>>> >>>>>> Hello_ >>>>>> >>>>>> >>>>>> >>>>>> I am using TPP for the database search of microbiome proteins. Since >>>>>> the fasta file that I have is more than 100 times larger than human >>>>>> uniprot >>>>>> fasta file, the database search for a single DDA run takes more than 12 >>>>>> hours. Is there any way to have faster analysis using multi-cores? >>>>>> >>>>>> >>>>>> >>>>>> Thanks, Eslam >>>>>> >>>>>> Proteomics core facility >>>>>> >>>>>> University of southern California >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "spctools-discuss" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to spctools-discuss+unsubscr...@googlegroups.com. >>>>>> To post to this group, send email to spctools-discuss@googlegroups. >>>>>> com. >>>>>> Visit this group at https://groups.google.com/group/spctools-discuss. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "spctools-discuss" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to spctools-discuss+unsubscr...@googlegroups.com. >>>>>> To post to this group, send email to spctools-discuss@googlegroups. >>>>>> com. >>>>>> Visit this group at https://groups.google.com/group/spctools-discuss. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "spctools-discuss" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to spctools-discuss+unsubscr...@googlegroups.com. >>>>> To post to this group, send email to spctools-discuss@googlegroups.com >>>>> . >>>>> Visit this group at https://groups.google.com/group/spctools-discuss. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "spctools-discuss" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to spctools-discuss+unsubscr...@googlegroups.com. >>>> To post to this group, send email to spctools-discuss@googlegroups.com. >>>> Visit this group at https://groups.google.com/group/spctools-discuss. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "spctools-discuss" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to spctools-discuss+unsubscr...@googlegroups.com. >>> To post to this group, send email to spctools-discuss@googlegroups.com. >>> Visit this group at https://groups.google.com/group/spctools-discuss. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "spctools-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to spctools-discuss+unsubscr...@googlegroups.com. >> To post to this group, send email to spctools-discuss@googlegroups.com. >> Visit this group at https://groups.google.com/group/spctools-discuss. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "spctools-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to spctools-discuss+unsubscr...@googlegroups.com. > To post to this group, send email to spctools-discuss@googlegroups.com. > Visit this group at https://groups.google.com/group/spctools-discuss. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "spctools-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to spctools-discuss+unsubscr...@googlegroups.com. To post to this group, send email to spctools-discuss@googlegroups.com. Visit this group at https://groups.google.com/group/spctools-discuss. For more options, visit https://groups.google.com/d/optout.
