Adam R. B. Jack wrote:
[snip]
Server component to look at the projects and work out which ones Gump
needs to try to build next. To start off, the algorithm stays as it is
now. In future we can look to detect if there have been CVS commits
since last build and so on. Server component puts "n" work pieces into
a pool.
[snip]
I'll respond as no one else has yet. I am very new to Python - I can play about with simple things like checking of executables but pretty much anything else is beyond me, so I am unlikely to be able to help with coding for this. That said...Service/Client component takes one project out of the pool, performs cvs
update and attempts to build (again using the current algorithm). Puts
the results into a second pool.
I've been thinking about this, but for threads. I recently split Gump into pieces with some for of listener & event/request pattern, so we could dispatch to unnamed parties. I reworked a 'runner' so we could have various types, my goal is one that take 'next available module or project' from a list, so we can have multiple concurrent threads working it.
View component takes the results from the second pool and produces the
web site. Alternatively, the second pool could be replaced with a
filestore and the view generates the pages on the fly from the
filestore. The view component is forrest with a little glue logic.
I agree, something (lower priority?) could be building a WWW site at the same time that real biulds are going on.
I keep thinking we could serialize the context (the build information) to a file, but I like having lots of information & haven't found time to serialzie it all.
The nice thing about this is that it opens up many more possibilities
for the future. For instance, it would enable distributed gump at the
client component.
Yup, very nice.
My personal goals are for cascading gumps, then multi-threaded (to make use of the two CPUs on Brutus), and one day, distributed.
Thanks for the thoughts. Now, how do we make the incremental progress to get
there?
You said that you have recently split Gump up into pieces using an event/listener pattern. I infer that the listener works across the equivalent of my proposed server/client interface but you don't have an equivalent between client and view. The steps would seem to be:
1a. Find a way of describing a piece of work to the client (a 'type' as you put it - a project or module). For the moment, I think we can assume that server and client have a consistent view of the metadata, so just the module or project name should suffice for the moment.
1b. Rework the runner to work over some form of sockets. The client will request a piece of work, the server will describe the piece of work as per [1a] above and mark it as checked out to the client (simply remove the work piece from its list in the initial implementation). When the server runs out of work pieces, it starts again from the beginning.
2a. Find a way of describing the results of a build. We need to decide whether only the log files are to be supplied; log files and location of the build on the client (assuming it is public access); or log files plus the actual build.
2b. The client will contact the view and supply the results of the build as per [2a] above and the view will acknowledge receipt.
2c. Rework the view to understand how to process the results from the client.
The actual sockets layer could be implemented using ftp, http, svn or our own protocol.
Later we can add a feedback loop from view to server to let the server know that the client has supplied the results and make the mechanism more bulletproof by timing out a client if it doesn't supply the results in a timely manner (so the work piece can be supplied to a different client). We could also send the work out to multiple clients to ensure they agree upon the results.
-- Michael
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
