Mark, maybe I missed this from earlier on, but do you have any plans to document this or open-source any related code? Sounds pretty interesting!
-- Joshua Gross Christian / SpanDeX.io / BA Candidate of Computer Science, UW-Madison 2013 414-377-1041 / http://www.joshisgross.com On Oct 6, 2012, at 2:35 PM, Mark Hahn <m...@hahnca.com> wrote: > > So what server inserts the task into the database? > > The tasks come from user actions. Each user is only connected to one host > server. So only one server creates a task.. When a task is finished its > results are shared by all hosts. There is no concept of two tasks being > equal and no possible race during creation. > > If a task server claims a task and then crashes, the others notice a time-out > condition (actually a lack of heartbeat) and the task is up for grabs again. > If two servers finish the same task before the collision is noticed, then > their results go into the same doc so eventual consistency wins again. It > has taken me a long time to make this fool-proof but it works now. > > > On Fri, Oct 5, 2012 at 10:07 PM, Tom <tommed...@gmail.com> wrote: > I have multiple mongodb servers running in a cluster, so they are not always > consistent. > > @Mark, I see a problem with your suggestion: "processes get the tasks from a > db record". So what server inserts the task into the database? If all servers > try to insert the task then multiple tasks will be created. If only one > server creates tasks then you will have a problem when this server goes > offline. If each server first checks if a task was not already created, I see > two problems: 1) how do you identify two tasks as being equal? 2) what if > there is a race condition in inserting the task, so server B cannot read the > task but server A is already in the process of writing it. > > @Ryan, since the aim is to coordinate a sole task among multiple servers, > simply reducing the cluster to one server does not really achieve my > objective. I think that it is possible to coordinate such task. Maybe with a > distributed event system, allowing servers to coordinate who is going to > perform a task? > > Tom > > Op zaterdag 6 oktober 2012 09:27:55 UTC+7 schreef ryandesign het volgende: > On Oct 5, 2012, at 10:07, Tom <tomm...@gmail.com> wrote: > > > Unfortunately I'm afraid that I don't see how a scheduler can avoid the > > concurrency problems. Note that the advantages (e.g. in availability) of > > having a cluster should be maintained here, and so you cannot run a > > scheduler in a separate process on a single server. > > I would say if you want a task run *once* every five minutes, then you should > run it on one server. > > If high availability is essential, perhaps you designate one primary server > to run the task, and a backup server that monitors the primary server and > takes over if the primary goes offline. > > > > If every server would be running the scheduler, the same concurrency > > problems would arise. > > I agree they would. > > What is this task you want to run every five minutes? Is there a way you > could partition the work so that each server runs the script every five > minutes and each server only does a specific fraction of the work? That would > help distribute the workload as it increases, but wouldn't help a server goes > offline. > > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to nodejs@googlegroups.com > To unsubscribe from this group, send email to > nodejs+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to nodejs@googlegroups.com > To unsubscribe from this group, send email to > nodejs+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to nodejs@googlegroups.com To unsubscribe from this group, send email to nodejs+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
