Hey ,Nicolas, I am not sure whether that was really caused by network. I was running this stand-alone app in both high performance network(school) and my home(crazy slow..), but they got almost the same speed rate.
Or would it possible that BOINC itself restrict the performance ? (by set upperbound on network bandwidth for app regardless of network status ?) --- On Fri, 10/23/09, Nicolás Alvarez <[email protected]> wrote: > From: Nicolás Alvarez <[email protected]> > Subject: Re: [boinc_dev] I/O affects the performance ? > To: [email protected] > Cc: "Kunsheng Chen" <[email protected]> > Date: Friday, October 23, 2009, 11:42 PM > El Vie 23 Oct 2009 20:16:34 Kunsheng > Chen escribió: > > I am using Libcurl to fetch http code for a list of > URLs in crawling > > application. > > > > When I tested that in a stand-alone machine mode, the > speed just not > > satisfying (and I think the same when it came to BOINC > infrastructure) > > > > I am wondering if I/O in BOINC affects the speed ? > > > > Currently the algorithms are as below: > > > > For each link in the buffer > > { > > (1) fetch http code for link > using Libcurl > > (2) print out into 'output' > immediately (fprintf function) > > } > > > > The time for each loop above is around 2 seconds, and > since there are > > usually more than 20 links inside the buffer. It takes > around one minute to > > finish 20 links. > > > > Does anyone using Libcurl in BOINC know something > about that ? > > > > I just want to know whether this is reasonable or > whether another way such > > as: > > > > -------------------------- > > > > For each link in the buffer > > { > > fetch http code for link and store in > memory > > } > > > > Print all links in the memory into 'output' at one > time(no loop) > > > > > > ----------------------- > > > > might be better ? But this way takes more memory in > client side and I > > really don't want to do that unless the speed has a > huge change. > > Your bottleneck is clearly the network. I really don't > think the three bytes > you're writing to the disk are causing any slowdown... > > -- > Nicolas > _______________________________________________ boinc_dev mailing list [email protected] http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address.
