narayan, paul, brandon
i am puzzled by this discussion. the task narayan brings up,
that of staging a new service, seems completely independent of
the issues around version control. and conflating the two seems
confusing.
i think what narayan wants, and the task requires, is a way of
referring to various cluster properties, namely
a) there exists a functioning new ntp server
b) all nodes use the new ntp server
c) the node with the old ntp server has that old server deleted
then what narayan is saying is we implement a) by moving
to version 301. when that is stable, we move to 302 and wait until
all nodes use the new server. then we can move to 303.
all true and all, but hopeless because the thing you want is a),
b) and c), and not 301-3. the steps are then
A) pick a node, and give it the new ntp server; verify property a) is
true.
this might be rev 301.
B) point all nodes to teh new server; verify property b) is true. this
might be rev 304.
C) delete the old ntp server; verify property c). this might be rev 312.
now, everything is clearer. other things can happen in parallel and no
one cares
because we make the changes when it is safe to do so-- when the
previous step has
succeeded. now even brandon can make this automatic. (i realise that
part of brandon's
comment also refers to the fact that human approval steps often serve
to act as a check
that things external to the systems' state haven't arisen, but i regard
this as a purely process
thing; if people want to make it automatic, i want to support that.)
i know i harp on this insessantly, but node/group properties are the
ONLY things
that matter in the long run. config changes, or revisions, are just
means to an end.
so issuing the change to point nodes to teh new server is merely
interesting unless
you combine it with a check that they are doing so.
as for paul's comments, i have already spoken to most of them; you
apply changes
when you are ready to deploy them. for the person designing the
changes, it means that you
don't simply edit the configs, you write programs to make the
(hopefully) simple changes.
if this is impractical or too hard, then make them in real time. to do
otherwise is to impose
the complexity of your change management flow onto the version control
system,
which can barely cope with it (to say nothing of users).
athough he did not say so explicitly, i took paul's comments to also
cover the
issue of feature interaction, which is much harder. an example might be
that
an urgent fix might require that the new ntp server has to be another
node
and that this occurs during step B). the best thing would likely be to
point everyone
at the old server, implement the new urgent thing, then start step B)
over again.
there might be some transient issues, but that's teh price you pay for
urgent things.
do i have this right? or is there some other issue underneath that
justifies
enmeshing version control with staging and verifying actions?
andrew
On Nov 2, 2006, at 11:04 AM, Narayan Desai wrote:
Here is how we implemented this. First, you commit three revisions to
the svn repository. In the first, you enable the new ntp server. Say
this gets repo revision 301. Then you commit a change that points
clients at the new server; this gets revision 302. Then you commit a
change that disables the old server; this gets revision 303.
At this point, the server is still running off of revision 300. You
set things into motion by updating it to revision 301. We have a
script that checks if clients have the right versions of particular
configuration entries and groupings. In this case, once the new ntp
server has the ntp server properly configured at r301, we can move to
r302. Once r302 is deployed, clients begin to change over to the new
ntp server. Once all clients are showing correct ntp @ r301, we can
move to r302 and decommission the old server.
----
Andrew Hume (best -> Telework) +1 732-886-1886
[EMAIL PROTECTED] (Work) +1 973-360-8651
AT&T Labs - Research; member of USENIX and LOPSA
_______________________________________________
lssconf-discuss mailing list
lssconf-discuss@inf.ed.ac.uk
http://lists.inf.ed.ac.uk/mailman/listinfo/lssconf-discuss
