Re: XO in-field upgrades
On Sun, 2007-06-24 at 13:24 -0400, C. Scott Ananian wrote: On 6/24/07, Ivan Krstić [EMAIL PROTECTED] wrote: I should have a concrete spec ready for discussion later today. I will wait with bated breath. =) Some concrete concerns -- I've got some answers to these, but I'll try to just present the questions at this point: a) Robustness -- what can break our upgrade mechanism? Do we have a fallback? What dependencies does the upgrade mechanism have? b) Current vserver code requires restarting the containers when files inside the container are modified by the root context. There is also a relinking process necessary. Have we thought through these interactions? I haven't really seen a viable way to use vserver for updating the root filesystem. Is there a proposal for how this would work? I had a different idea to do a very robust root filesystem update. Given the layout and format of jffs2 (the filesystem we're using) it is very easy to add filessystem transaction support. That is, we'd have a way to trigger a transaction at the start of an update, then we just do all the changes we want (including just updating the kernel image file). Then when the update is done, we commit the transaction. If at any time we lose power or anything like that, none of the changes after the transaction start will be visible on the next boot. And also, if we have any problems during update (e.g. out of flash) we an rollback and abort. ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: XO in-field upgrades
On 6/25/07, Christopher Blizzard [EMAIL PROTECTED] wrote: Most of the stuff that Alex has done is (carefully) independent of any vserver or container discussion. Specifically, the update system in question could be applied inside of a container as easy as it would be outside of the container. I not sure I agree that the filesystem portion of the implementation ought to be completely independent of the network portion. I think that networking desiderata are going to impact our choices among the various upgrade schemes, and I'd prefer to have a high level design for the whole thing before we get too attached to particular bits of code. As one example, since network reliability goes down quickly as message size increases, it seems (to me, at least) that our upgrade data messages should be made as small as possible. It's not clear to me that our current design minimizes upgrade transfer size. Re: broadcast, that's basically the same as any laptop exposing presence information. For _transmission_ of an update, it's an interesting question as to whether or not to use a multicast update kind of thing. Do laptops usually wake each other up to process presence information? [Hopefully not.] Should they do so for urgent security upgrades? [Hopefully.] Here's a draft proposal: We listen to multicast address foo:bar:xxx:xxx:n + 1 if we currently have version # n installed on the machine. Then we won't be woken up by our friends announcing that they have version n like we do, but we will be woken up as soon as someone gets version n+1. On 6/25 Alex L wrote: mDNS *is* multicast. But the blobs won't be exposed over mDNS, that is far to much data for a protocol like that. Really? Do we know that? What's a typical 0-day patch look like? Have we tried to see how few bits it could be squashed into? Binary diffs seem much less useful. They enforce a specific base version that you have to have around, and they enforce the direction of upgrade. This is *exactly* why we need to have the big picture view. My understanding is that we *are* expecting all laptops to have identical bases, that the upgrade propagation rate and upgrade (in)frequency will be sufficient that all laptops will be running the same version of the software (save for a few stragglers who go on a long trip for a few weeks), and that the vserver copy-on-write mechanism will be used to perform rollback (so that we only need to worry about the forward direction). I'm not saying that my assumptions are correct. But I feel that deciding file formats before we've come to a big picture consensus may be premature. If you can cheaply generate at runtime (on the client) a minimal diff between any two versions you can avoid storing unnecessary information, Not sure exactly what you're getting at here: that we transfer complete blobs over the network but store them on the XO as binary diffs? My working assumption is that network and storage costs on the XO should be minimized as much as possible. Transferring complete blobs fails on these grounds. --scott -- ( http://cscott.net/ ) ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: XO in-field upgrades
On Mon, 2007-06-25 at 12:25 -0400, C. Scott Ananian wrote: mDNS *is* multicast. But the blobs won't be exposed over mDNS, that is far to much data for a protocol like that. Really? Do we know that? What's a typical 0-day patch look like? Have we tried to see how few bits it could be squashed into? The broadcast just contains a product/version ID - doesn't have to include the entire update. No more expensive than the presence stuff we have today. Binary diffs seem much less useful. They enforce a specific base version that you have to have around, and they enforce the direction of upgrade. This is *exactly* why we need to have the big picture view. My understanding is that we *are* expecting all laptops to have identical bases, that the upgrade propagation rate and upgrade (in)frequency will be sufficient that all laptops will be running the same version of the software (save for a few stragglers who go on a long trip for a few weeks), and that the vserver copy-on-write mechanism will be used to perform rollback (so that we only need to worry about the forward direction). I think that what Alex here can do it either way. You could use the vserver copy on write stuff if you want to use the hammer of a filesystem or you could use the image code he has to move it back whether or not the copy on write stuff is there. Or even after you have committed via vserver. I strongly suggest that we make sure that we keep the ability to go both directions. It gives us a lot more flexibility down the road, and for the countries and our users as well. I'm not saying that my assumptions are correct. But I feel that deciding file formats before we've come to a big picture consensus may be premature. If you can cheaply generate at runtime (on the client) a minimal diff between any two versions you can avoid storing unnecessary information, Not sure exactly what you're getting at here: that we transfer complete blobs over the network but store them on the XO as binary diffs? My working assumption is that network and storage costs on the XO should be minimized as much as possible. Transferring complete blobs fails on these grounds. When you hear complete blobs can you describe what you mean? I suspect that you're thinking of something different than what Alex has actually implemented. --Chris ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
[PATCH] xf86-amd-devel: fix lx Xv downscaling
Try #2.
diff --git a/src/amd_lx_video.c b/src/amd_lx_video.c
index e1e51aa..a4cd91b 100644
--- a/src/amd_lx_video.c
+++ b/src/amd_lx_video.c
@@ -365,6 +365,7 @@ LXDisplayVideo(ScrnInfoPtr pScrni, int id, short width,
short height,
unsigned long yExtra, uvExtra = 0;
DF_VIDEO_POSITION vidPos;
DF_VIDEO_SOURCE_PARAMS vSrcParams;
+ int err;
memset(vSrcParams, 0, sizeof(vSrcParams));
@@ -401,16 +402,14 @@ LXDisplayVideo(ScrnInfoPtr pScrni, int id, short width,
short height,
/* Set up scaling */
df_set_video_filter_coefficients(NULL, 1);
-
- if ((drawW = srcW) (drawH = srcH))
-df_set_video_scale(width, height, drawW, drawH,
- DF_SCALEFLAG_CHANGEX | DF_SCALEFLAG_CHANGEY);
- else if (drawW srcW)
-df_set_video_scale(drawW, height, drawW, drawH,
- DF_SCALEFLAG_CHANGEX | DF_SCALEFLAG_CHANGEY);
- else if (drawH srcH)
-df_set_video_scale(width, drawH, drawW, drawH,
- DF_SCALEFLAG_CHANGEX | DF_SCALEFLAG_CHANGEY);
+
+ err = df_set_video_scale(width, height, drawW, drawH,
+DF_SCALEFLAG_CHANGEX | DF_SCALEFLAG_CHANGEY);
+ if (err != CIM_STATUS_OK) {
+/* Note the problem, but do nothing for now. */
+ErrorF(Video scale factor too large: %dx%d - %dx%d\n,
+ width, height, drawW, drawH);
+ }
/* Figure out clipping */
___
Devel mailing list
Devel@lists.laptop.org
http://lists.laptop.org/listinfo/devel
Re: XO in-field upgrades
On 6/25/07, Christopher Blizzard [EMAIL PROTECTED] wrote: The broadcast just contains a product/version ID - doesn't have to include the entire update. No more expensive than the presence stuff we have today. You're misunderstanding me. My concern is with waking machines up by broadcasting this information. We don't wake up on presence, but we do want to wake up on (some, urgent) upgrades. I think that what Alex here can do it either way. You could use the vserver copy on write stuff if you want to use the hammer of a Again, you're misunderstanding me slightly. Vserver has *very odd* semantics for its copy-on-write and for writing inside containers. We need to sync up on that come up with a plan, or we run the risk of creating a useless tool. Binary diffs are also bidirectional, FWIW. When you hear complete blobs can you describe what you mean? I suspect that you're thinking of something different than what Alex has actually implemented. Quoting from https://hosted.fedoraproject.org/projects/updatinator/ : --- Each computer running Updatinator tracks a specific image id, and runs a specific version of that image. Whenever it finds a manifest for a later version of that image id (and that manifest is signed by the right gpg key) that manifest and the required blobs for updating to it is automatically downloaded. When the manifest and all required blobs are downloaded Updatinator sends a dbus signal so that the system may let the user apply the update (e.g. automatically, or by showing some ui asking the user if he wants to update). --- My next post will give some concrete #s justifing the use of binary diffs. --scott -- ( http://cscott.net/ ) ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: XO in-field upgrades
[ Fixing Alex's address. ] On Mon, 2007-06-25 at 14:58 -0400, C. Scott Ananian wrote: On 6/25/07, Christopher Blizzard [EMAIL PROTECTED] wrote: The broadcast just contains a product/version ID - doesn't have to include the entire update. No more expensive than the presence stuff we have today. You're misunderstanding me. My concern is with waking machines up by broadcasting this information. We don't wake up on presence, but we do want to wake up on (some, urgent) upgrades. That's going to be interesting, yeah. You would need to teach the wireless firmware about it? How about just checking on wakeup? Some kind of wake-on-lan signal? I think that what Alex here can do it either way. You could use the vserver copy on write stuff if you want to use the hammer of a Again, you're misunderstanding me slightly. Vserver has *very odd* semantics for its copy-on-write and for writing inside containers. We need to sync up on that come up with a plan, or we run the risk of creating a useless tool. Can you explain how they are odd? It sure would help everyone. Binary diffs are also bidirectional, FWIW. Yeah, but you always need both sets of information to be able to generate them. So you have to host full file + diff data if you want to host an update. The nice thing about Alex's system is that you only have to host the file data that you're using on your system instead of file + diff data. You end up using less space that way. If you want to downgrade, you have to get the files or use the vserver versions (maybe you could just use the old files handled by the CoW stuff to drive the update system - that might work pretty well!) When you hear complete blobs can you describe what you mean? I suspect that you're thinking of something different than what Alex has actually implemented. Quoting from https://hosted.fedoraproject.org/projects/updatinator/ : --- Each computer running Updatinator tracks a specific image id, and runs a specific version of that image. Whenever it finds a manifest for a later version of that image id (and that manifest is signed by the right gpg key) that manifest and the required blobs for updating to it is automatically downloaded. When the manifest and all required blobs are downloaded Updatinator sends a dbus signal so that the system may let the user apply the update (e.g. automatically, or by showing some ui asking the user if he wants to update). --- My next post will give some concrete #s justifing the use of binary diffs. --scott Keep in mind that those blobs he's talking about are just files. The only place where we would add binary diffs would be for individual files, not entire trees. So what we're downloading today is only the changed files, largely for the sake of expediency and what I describe above for the space savings. Speaking for myself I've never been opposed to use binary diffs. To be sure, all of my original ideas included them. But if I have to choose between having something that works today with full files and saves some space and adding the complexity of binary diffs later, I will use the former. I would love to hear what you have to say about numbers for binary diffs. (I believe that in earlier discussions with people who tried these systems before that any diff system that didn't understand elf was doomed to failure, but I am ready to be shown the money.) I think that both Alex and I are happy to listen to ideas here (esp about the vserver stuff that you mention!) but it's June 25th and we have to be pretty practical about what we can do between now and when we have to ship. The update system needs to be very simple, easy to test and easy to validate + understand. The key to that is using a simple design and simple implementation. Added complexity has a high bar for inclusion here. --Chris ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Upgrades and image manifests
On Mon, 2007-06-25 at 15:45 +0100, David Woodhouse wrote: On Tue, 2007-06-19 at 14:39 +0200, Alexander Larsson wrote: Does OLPC use selinux or xattrs? Because if so we have to extend the manifest format. Not yet, but it's likely to in the near future when we ditch the short-term hacks and manage to implement the proper long-term security plan. So it's worth designing for it from the start. Yeah, I'm pretty sure that want to have support for at least describing xattrs (especially if we want to use this for regular filesystems as well - lots of people are using xattrs for all kinds of things.) --Chris ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Fwd: XO in-field upgrades
[grumble, sent this from the wrong email address.] -- Forwarded message -- Here are some quick numbers justifying the use of binary diffs as an upgrade delivery format. I just grabbed the latest debian security advisory for Etch, which happened to be: http://www.us.debian.org/security/2007/dsa-1320 on clamav. This upgrade appeared not atypical, but feel free to point out if it is widely uncharacteristic for a lightweight package (ie, not mozilla). I downloaded and unpacked the packages corresponding to the 'before' and 'after' versions of all the packages affected. For reference, those versions are 0.90.1-3etch2 and 0.90.1-3etch3. I then deleted the files in /usr/share/doc and /usr/share/man (neither of which 'should' have changed, but apparently they did) and /usr/lib/debug. I wrote a script to generate binary diffs using both xdiff and bsdiff for the files which changed. (I also munged slashes to dashes.) These files were: usr-bin-clamconf usr-bin-freshclamusr-lib-libclamav.so.2.0.1 usr-bin-clamdscan usr-bin-sigtool usr-sbin-clamav-milter usr-bin-clamscan usr-lib-libclamav.a usr-sbin-clamd For all files other than libclamav.a and libclamav.so.2.0.1, the binary diff was less than 164/225 bytes (xdiff/bsdiff). libclamav.a had a 3529/6571 byte diff, and libclamav.so.2.0.1 had a 13398/74461 byte diff. (Note larger diff because all the relocation table entries moved. Maybe a smarter diff is possible.) In comparison, the full libclamav.a blob is 567,140 bytes and libclamav.so.2.0.1 is 432,668 bytes. Details: blob: 17168 usr/bin/clamconf 33120 usr/bin/clamdscan 49600 usr/bin/clamscan 65584 usr/bin/freshclam 77868 usr/bin/sigtool 567140 usr/lib/libclamav.a 432668 usr/lib/libclamav.so.2.0.1 76260 usr/sbin/clamav-milter 49948 usr/sbin/clamd bdiff: 158 usr-bin-clamconf 158 usr-bin-clamdscan 161 usr-bin-clamscan 159 usr-bin-freshclam 162 usr-bin-sigtool 3529 usr-lib-libclamav.a 13398 usr-lib-libclamav.so.2.0.1 164 usr-sbin-clamav-milter 162 usr-sbin-clamd Total bsdiff size: 18,051 Total raw size: 1,369,356 bytes. (75x larger) Further, although my goal of single packet upgrades seems overoptimistic, all but two of the file diffs can fit in a single packet, helping assure atomicity. The total upgrade information fits in 13 packets. To my mind, this makes it clear that the on-wire format should be a binary diff. We could uncompress these diffs on receipt and maintain a blob store as in the current proposal, but it seems much more reasonable to sign, authenticate, and store the *diffs* rather than the *blobs*, so that we don't need to recompute diffs when we want to pass on our upgrade to a neighboring XO. --scott -- ( http://cscott.net/ ) -- ( http://cscott.net/ ) ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: XO in-field upgrades
C. Scott Ananian wrote:
On 6/25/07, Christopher Blizzard [EMAIL PROTECTED] wrote:
That's going to be interesting, yeah. You would need to teach the
wireless firmware about it? How about just checking on wakeup? Some
kind of wake-on-lan signal?
Binding upgrade notifications to a multicast address as I previously
proposed fixes this problem without any kind of firmware hacking.
Can you explain how they are odd? It sure would help everyone.
Caveat: I'm not an expert here. I haven't read the code, just the
documentation. So we can all follow along, start here:
http://linux-vserver.org/Paper#Unification
http://linux-vserver.org/Frequently_Asked_Questions#What_is_vhashify.3F
Basically, copy-on-write works by running a tool ('vhashify') which
looks for identical files in the different containers and hard links
them together, then marks them immutable. The copy-on-write mechanism
works by intercepting writes to immutable files and cloning the file
before making it writable by the container.
It is worth noting we are not using vhashify or any of the other util
scripts. The rainbow daemon sets up the chroot for each activity itself.
We are a bit non-standard in that we are doing process-level
containerization, instead of a more guest-OS system like many vserver
users (most?).
--Noah
signature.asc
Description: OpenPGP digital signature
___
Devel mailing list
Devel@lists.laptop.org
http://lists.laptop.org/listinfo/devel
Re: XO in-field upgrades
On Mon, 2007-06-25 at 15:35 -0400, C. Scott Ananian wrote:
On 6/25/07, Christopher Blizzard [EMAIL PROTECTED] wrote:
That's going to be interesting, yeah. You would need to teach the
wireless firmware about it? How about just checking on wakeup? Some
kind of wake-on-lan signal?
Binding upgrade notifications to a multicast address as I previously
proposed fixes this problem without any kind of firmware hacking.
Ahh, sorry, I thought you meant _really_ asleep - like not waking up on
network events. Although does our independent firmware know enough to
wake us up on multicast traffic? I thought that it only worked on the
lower level protocols and that a packet had to be specifically destined
for our MAC address to get a wake up event. I'll show my ignorance of
multicast here: does it include specific MAC addresses or is it a wide
broadcast at that layer? I always assumed that it was the latter.
Can you explain how they are odd? It sure would help everyone.
Caveat: I'm not an expert here. I haven't read the code, just the
documentation. So we can all follow along, start here:
http://linux-vserver.org/Paper#Unification
http://linux-vserver.org/Frequently_Asked_Questions#What_is_vhashify.3F
Basically, copy-on-write works by running a tool ('vhashify') which
looks for identical files in the different containers and hard links
them together, then marks them immutable. The copy-on-write mechanism
works by intercepting writes to immutable files and cloning the file
before making it writable by the container.
Quoting from their FAQ:
(when running vhashify:) The guest needs to be running because
vhashify tries to figure out what files not to hashify by calling the
package manager of the guest via vserver enter.
In order for the OS cache to benefit from the hardlinking, you'll have
to restart the vservers.
Holy crap, this sounds like a steaming _pile_ of complexity. Are we
seriously going to try to deploy on this?
Since vserver is doing file hashification anyway, it seems like it
would be a much better idea to use this rather than reinvent the
wheel. Some other issues:
a) we may need to be aware of which files are hardlinked where in
order to do proper updates.
b) not clear how/if we can make updates to an entire tree atomically
c) how/can we restart the vservers? how important is this?
I think we need to bring in a vserver expert (ie, not me) to get these
details right at the start.
Am happy to get more advice on this, for sure. I suspect that all of
the vserver people we can call on are on this list.
Our current thinking basically is that we can do an update as part of an
update/shutdown procedure. So you can apply the updates on the way
down, get a new env on restart. That would handle the vserver restarts
and also how to get a new kernel issue that no one else has mentioned.
I'm not sure if hashing for updates and hashing for vserver are the
kinds of things we want to share or not. I would love to hear more
about how vserver does its hashing and see if we can share. I still
feel that keeping the update system as simple and uncomplex as possible
is a very good way to go - it lets us advance in parallel and come up
with something that works well.
It also sounds like it's pretty easy to do something like:
o Start root container
o Start guest container
o Apply update
o Start activities in that guest container
And the hardlink/CoW stuff will then give us an updated container.
Still doesn't help with updates on the base system nor kernel bits, but
it's a start.
Yeah, but you always need both sets of information to be able to
generate them. So you have to host full file + diff data if you want to
host an update.
My proposal would be that XOs pass around binary diffs *only* among
themselves, and that if someone needs an older version or to leapfrog
versions, they ask the school server. This allows XOs to cheaply host
updates in the common case.
You could do that with Alex's system as well. But in Alex's case the XO
doesn't have to carry both the system it's using + diff. Because the
system you're using is the update.
The nice thing about Alex's system is that you only
have to host the file data that you're using on your system instead of
file + diff data. You end up using less space that way.
If you look at the numbers I just posted, file+diff is 1.3% larger
than just files.
If you want to
downgrade, you have to get the files or use the vserver versions (maybe
you could just use the old files handled by the CoW stuff to drive the
update system - that might work pretty well!)
Now we're talking! ;-)
Keep in mind that those blobs he's talking about are just files. The
only place where we would add binary diffs would be for individual
files, not entire trees. So what we're downloading today is only the
changed files, largely for the sake of expediency and what I describe
Re: XO in-field upgrades
You're misunderstanding me. My concern is with waking machines up by broadcasting this information. We don't wake up on presence, but we do want to wake up on (some, urgent) upgrades. That's going to be interesting, yeah. You would need to teach the wireless firmware about it? How about just checking on wakeup? Some kind of wake-on-lan signal? In general, we won't be waking up the host on broadcast/multicast frames. We can make an exception for anycast addresses (that the firmware listens for), however I can't see a scenario where we wake up on broadcast. M. ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: XO in-field upgrades
On Mon, 2007-06-25 at 15:56 -0400, Noah Kantrowitz wrote: Design docs? We're still at the proof-of-concept phase really ;-). But yes, each chroot needs to be generated on the fly when a new activity starts (unless we do some funky magic with unionfs, which is probably not a great idea). The load of a few directory hardlinks should be minimal. Are we expecting to do online updating or will it be more of a windows-style shut it all down then patch? I think that for phase one we can do it during a shutdown/restart cycle. --Chris ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Fwd: XO in-field upgrades
On 6/25/07, Christopher Blizzard [EMAIL PROTECTED] wrote: On Mon, 2007-06-25 at 15:38 -0400, C. Scott Ananian wrote: To my mind, this makes it clear that the on-wire format should be a binary diff. We could uncompress these diffs on receipt and maintain a blob store as in the current proposal, but it seems much more reasonable to sign, authenticate, and store the *diffs* rather than the *blobs*, so that we don't need to recompute diffs when we want to pass on our upgrade to a neighboring XO. It's not clear what that data means, exactly. Do those files like this: 158 usr-bin-clamconf 158 usr-bin-clamdscan 161 usr-bin-clamscan 159 usr-bin-freshclam 162 usr-bin-sigtool Actually include binary files changes or are they just permissions changes? If so you're 75x larger charge seems a little over the top since they wouldn't be transferred at all in Alex's case outside of the manifest. :) No, they are actually 4-byte changes to each binary, buried in the linker-generated section, presumably caused by an offset change in the library. I stared at this and the patches for a while. You're correct that most of the size of this diff is spent coding the filename, but there is a real diff there. There's a 60x change if you just look at the libraries. So this savings is definitely real. That being said, this is pretty compelling stuff. I would love to see this done against various versions of libgklayout.so, which will probably be our largest offender. If I were bored, I might indulge you. ;-) It also might be interesting to layer a diff system on top of what Alex has proposed so you could do it either way - diffs if ya got 'em, raw if ya don't. That sounds suspiciously like the complexity you're afraid of. Let's just do diffs and call it done. --scott -- ( http://cscott.net/ ) ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [PATCH] xf86-amd-devel: fix lx Xv downscaling
On Mon, 2007-06-25 at 17:21 -0400, Bernardo Innocenti wrote: Dan Williams wrote: Try #2. Does ti fix http://dev.laptop.org/ticket/1601 perhaps? _maybe_, but this was tested on Xorg 1.1.99, so there may be more stuff with 1.3 that's broken. Dan ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
A different proposal for XO upgrade.
As food for discussion, here's a counter proposal for an XO upgrade mechanism, focusing on the network side. - Design goals: - minimize round-trips necessary for successful upgrade - minimize size of upgrades Software versions are assigned sequential integers. Version 1, 2, etc. We assume that every machine should discover an available upgrade within a day, on average. The constants below can be tweaked to adjust this interval. Upgrades consist of a number of messages U_0 through U_N with a maximum size Umax. The maximum size is chosen either so that a) each U_i fits in a single network packet, or b) each U_i can fits into a reserved storage area on the XO (to allow it to be shared with neighbors). Upgrade messages are independent. It may be worthwhile for upgrade messages to be applied in any order, but for simplicity we'll constrain the order of application to be sequential. U_i is authenticated and self-labeling: it contains a signature as well as a field specifying the version # of the upgrade as well as i, the sequence number of the upgrade message. Part 1: discovering an upgrade. Every hour, the XO looks to see how many mesh neighbors it has, counting itself. Call this number N. N is always at least 1. It then generates a random number in [0, N*24). If the number is 0, does a DNS lookup on update.laptop.org to check for a new version. (Note that the DNS lookup may be cached by the mesh portal or school server.) XOs which have version V of the base system also listen to the multicast address ::::V+1 Part 2: announcing the upgrade. If you are the person to discover a new version V', you are the Upgrade Leader. While the version on your laptop (V) is less than V' do the following: a) Set i=0. Stop listening to the multicast address. b) download U_i for (V+1) via http from update.laptop.org (again, these queries are often cached by a school server). Apply it to our local copy (using COW techniques to prevent the application from being globally visible until the upgrade is complete) c) Broadcast U_i to the multicast address ::::V+1. d) If U_i is not the last upgrade message, increment i, and repeat from step b) e) Otherwise, atomically make our updates visible to the system, increment V, and restart part 2 if necessary. f) Start listening to the multicast address ::::V'+1 Part 3: casual bystanders. If you hear a piece of the upgrade U_i on the multicast address ::::V+1: a) If there is a piece U_j between U_0 and U_i which you have not already heard, ignore this packet. b) Otherwise, apply U_i and (re)set the Upgrade Timer to expire some random time in the future (some 10s of minutes). c) If U_i was the last upgrade message, atomically make the updates visible, delete the Upgrade timer, increment our local version, and rebind our multicast listening address to ::::V+1 for our new version V. d) If the Upgrade Timer expires, become a new Upgrade Leader: set i = the smallest i such that we don't have U_i, and jump to Part 2 step b. -- Structure of upgrade messages: Version # from Version # to Sequence # final message flag? msg length 1 or more of: filename to patch pre-patch hash post-patch hash bsdiff format diff signature We might keep a git-style manifest for system, patching this like any other file. This can be used like fsck to sanity-check/verify the end-result. Further, although the 'from' version and 'to' version are separated by exactly 1 in typical usage, a sequence of upgrade messages can be used to perform any desired amount of up/down grade. I would propose keeping 'up-by-one' and 'down-by-one' upgrade message sequences on update.laptop.org, so that the user can downgrade to any chosen revision. We might either store 'skip-by-N' sequences on the server (for N=1,2,4,8,16,32,...) or generate upgrade messages on the fly if we want faster/further upgrades/downgrades. - Notes: a) the school server can trigger updates on all machines by becoming an Upgrade Leader and broadcasting bits. b) Upgrades are distributed very efficiently over the air if mesh broadcast is working (ie, not too many nodes drop packets). It could be even more efficient if we were allowed to cache entire upgrade message sequences on the XO and/or apply upgrade messages out-of-order. c) The Upgrade Timer timeout should probably depend on the number of messages in the upgrade and the # of the message you lost. d) Although I've described the process above as an automatic upgrade, once the sequence of upgrades has been applied, the actual atomic swap of current-tree to upgraded-tree could be deferred until some point in the future. e) Limiting the size of the upgrade messages U_i is
Ottawa Linux Symposium
Hello, tomorrow I'll be heading to Ottawa for attending the OLS. If anyone from Boston wants to join the trip, we can share fuel. According to Google, it will be approximately 7hrs (more if you consider a few breaks). I've not yet decided when to return. I'd like to do a little stop for visiting Montreal on the way back. I won't mind skipping it if I'm traveling with people who need to be back earlier. -- // Bernardo Innocenti \X/ http://www.codewiz.org/ ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Network Traffic and Multicast
Hi. After EduKT and AmiGO, we are working on ITv which has been develop for any OS except XO because the usage of 2 very dynamic libraries: tubes and gst (for regular linux/windows we are using vlc/twisted) the next week we will launch a video demo of ITv (Interactive Television), which allows 1 video channel and 3 real time data channel however i have to know about the performance of the network, bandwidth usage for multicast, dead times over a stream, coverage (ideal range) of the mesh... the goal is optimize the traffic consume and low latency issues. Thanks, -- Alfonso de la Guarda ICTEC SAC www.cosperu.com www.delaguarda.info Telef. 97550914 4726906 ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
