In the first 10 minutes of being back up and running, I would say most projects could respond to every attached client, so long as the protocol and scoring system are sufficiantly simple.
10 minutes = 600 seconds, say 10 interactions per second? 6000 hosts. Which for many tiny budget projects is all they will have. For a larger project with 60,000 hosts, it is still reasonable to expect you would only have hits from 10% of them in the first 10 minutes. So, you've still kept up with the workload. ...and if not, that's why the client does retries. Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running rose...@home just might! http://boinc.bakerlab.org/rosetta/ --- On Sat, 7/11/09, Lynn W. Taylor <[email protected]> wrote: > From: Lynn W. Taylor <[email protected]> > Subject: Re: [boinc_dev] Optimizing uploads..... > To: [email protected] > Cc: "BOINC dev" <[email protected]> > Date: Saturday, July 11, 2009, 5:33 PM > Mark, > > What if the server is so busy you can't connect to it to > get that message back? > > We have three cases: > > 1) Everything is happy. > > 2) Things are really busy. > > 3) A million pounds of (fill in whatever you'd like) in a > five pound bag. > > Things can go from #1 to #3 quickly if the project is big, > and funding is low (which is the goal, to bring distributed > computing to projects with tiny funding). > > So, while it would be ideal for the servers to be able to > say "no, and please slow way, way down" I think we have to > assume that all of the project's servers are unreachable > under an extreme load. > > That is when the need is most extreme, and the hardest to > deal with, so that's the only one I'm really thinking > about. > > -- Lynn > > Mark Pottorff wrote: > > It would seem that having the server be able to accept > a connection, and then direct the client on any preferred > backoff would be ideal. The server is in the best position > to know how long it was down, how long it will take to > recover to normal bw levels, how many active connections it > already has, whether deadlines have been extended etc. etc. > > > > The client is in the best position to know how > frequently it has an internet connection available, what the > available bw to the server tends to be, how much data it has > to send, etc. > > > > If the upload server could respond to requests with a > message, to the effect of "I'm trying to focus only on > uploads that are within 12 hours of deadline" and negotiate > with the client as to whether the uploads are within that > objective, whether it expects to have an internet connection > available later, number of files to upload etc. then it > should be possible to arrive at a single backoff to a point > in time that the server will be available to handle the > request at full speed. "OK, we'll upload one file now and > hit you in 3hrs 12 minutes with the other 4." > > > > As the server gets caught up, and bw utilization > begins to drop, or the server sees holes in the commitments > it has made, the window extends on the uploads out to 24hrs > from deadline etc. > > > > When an upload completes, a confirmation is made that > it arrived with all of the data. It would seem an ideal time > for an overloaded server to convey that it would prefer to > delay any further uploads. Or establish a simple 1-5 system > expressing how busy it is. The client then weighs time to > deadline against likelihood of future internet connection to > assess whether it has an upload of sufficient priority or > not. And if not, it delays starting next upload. > > > > Really the objective is that all result files across > the user base get uploaded in priority order. Where priority > takes in to account bandwidth user and project have > available, deadline, file size, etc. > > > > Same is true for downloads. If server is drowning in > requests, you want to be servicing the starving clients > first, not those with full-time networks and fat work > caches. And only to the extent that they aren't starving > anymore, until the contention is resolved. > > Having a machine sitting idle waiting for downloads > can occur, and the order of downloads isn't optimized > either. One more file and I'll be able to start processing a > task, but that's not necessarily the file being downloaded > right now. Just depends on the backoffs it was assigned etc. > as retries occurred and connections were satisfied. > > > > > > Running Microsoft's "System Idle Process" will never > help cure cancer, > > AIDS nor Alzheimer's. But running rose...@home > just might! > > http://boinc.bakerlab.org/rosetta/ > > > > > > > _______________________________________________ > > boinc_dev mailing list > > [email protected] > > http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev > > To unsubscribe, visit the above URL and > > (near bottom of page) enter your email address. > > > _______________________________________________ boinc_dev mailing list [email protected] http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address.
