On Sat, Jan 10, 2009 at 9:47 PM, Matthew Toseland <toad at amphibian.dyndns.org> wrote: > On Saturday 10 January 2009 13:26, Daniel Cheng wrote: >> On Sat, Jan 10, 2009 at 8:19 PM, Matthew Toseland >> <toad at amphibian.dyndns.org> wrote: >> > On Saturday 10 January 2009 11:44, Daniel Cheng wrote: >> >> On Sat, Jan 10, 2009 at 9:17 AM, Matthew Toseland >> >> <toad at amphibian.dyndns.org> wrote: >> >> > On Monday 22 December 2008 16:04, j16sdiz at freenetproject.org wrote: >> >> >> Author: j16sdiz >> >> >> Date: 2008-12-22 16:04:07 +0000 (Mon, 22 Dec 2008) >> >> >> New Revision: 24747 >> >> >> >> >> >> Modified: >> >> >> trunk/plugins/XMLSpider/XMLSpider.java >> >> >> Log: >> >> >> no retry, fail faster >> >> > >> >> > Why? BTW retries = 1 still means 1 retry i.e. 2 attempts iirc. It must > be >> > no >> >> > more than 2 to avoid the cooldown queue. >> >> >> >> higher turnover rate. >> > >> > I thought you were having performance issues with XMLSpider core, in which >> > case it makes more sense to try more? >> >> Performance on this is not a issue after migrating to perst. >> >> There is two remaining bottleneck - >> (1) The content filter is slow. I have no idea why, but checking >> for valid URI is really slow > > Do you think this contributes to fproxy lack of performance? >
no idea. xmlspider always run in lower priority thread.. maybe just a priority issue. >> (2) ClientGetter.start() try to check the data store before return. >> We are in a low priority thread, it have problem getting the >> share of synchronization locks. > > Doesn't happen on the db4o branch - datastore checking is serialised on a > separate thead. >> >> >> >> Modified: trunk/plugins/XMLSpider/XMLSpider.java >> >> >> =================================================================== >> >> >> --- trunk/plugins/XMLSpider/XMLSpider.java 2008-12-22 16:03:41 UTC >> > (rev >> >> > 24746) >> >> >> +++ trunk/plugins/XMLSpider/XMLSpider.java 2008-12-22 16:04:07 UTC >> > (rev >> >> > 24747) >> >> >> @@ -526,8 +526,8 @@ >> >> >> >> >> >> /* Initialize Fetch Context */ >> >> >> this.ctx = pr.getHLSimpleClient().getFetchContext(); >> >> >> - ctx.maxSplitfileBlockRetries = 2; >> >> >> - ctx.maxNonSplitfileRetries = 2; >> >> >> + ctx.maxSplitfileBlockRetries = 1; >> >> >> + ctx.maxNonSplitfileRetries = 1; >> >> >> ctx.maxTempLength = 2 * 1024 * 1024; >> >> >> ctx.maxOutputLength = 2 * 1024 * 1024; >> >> >> allowedMIMETypes = new HashSet<String>(); >> >> >> >> >> >> _______________________________________________ >> >> >> cvs mailing list >> >> >> cvs at freenetproject.org >> >> >> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs >> >> >> >> >> >> >> >> > >> >> > _______________________________________________ >> >> > Devl mailing list >> >> > Devl at freenetproject.org >> >> > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl >> >> > >> >> _______________________________________________ >> >> Devl mailing list >> >> Devl at freenetproject.org >> >> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl >> >> >> >> >> > >> > _______________________________________________ >> > Devl mailing list >> > Devl at freenetproject.org >> > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl >> > >> _______________________________________________ >> Devl mailing list >> Devl at freenetproject.org >> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl >> >> > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl >
