Re: [Monotone-devel] mtnplain (aka dumb) revision order, epoch
Hi, Nathaniel Smith <[EMAIL PROTECTED]> wrote: > On Fri, Jun 23, 2006 at 09:40:31AM +0200, Zbigniew Zagórski wrote: > > ... > > Question: How to obtain parent REV(s) from rdata packet without using 'mtn > read' > > ? > > I thought that someone else already fixed this in the current head of > the .dumb branch -- the better solution is to make sure that when we > write things out to the DATA file, we write them in the correct order, > so that later on they can just be read in directly like they are now. > > It's possible that this is in fact fixed, but you're trying to pull > from a "dumb" repository that was created before this was fixed, so > the packets are still in the wrong order. > > I suggest doing a push into a fresh dumb repository, and then pulling > from that, and seeing if that fixes the problem. Unfortunately, ... you're right. I followed probably outdated Riccardo's e-mail in which he written reordering is not ready :/. It works. Now I see that my effort was so stupid :/. Ach ... And yes I must have got some old 'dumb' repository which i overlooked. Sorry for bugging with this. > > Second problem. > > > What needs to be done to support them in mtn-dumb is: > 1) create a new packet type for epochs > 2) for each branch in the db, add this packet to the merkle trie > being synced. Make sure to put these packets as early as > possible; just like revisions should come before certs, epochs > should come before revisions, and even before files. > > ... > "mtn read" for locks on the database.) (This is exactly how an > in-monotone implementation of epoch packets would do, except with > fewer caveats.) > Now after reading a little i think that 2) is perfect solution. I'll do it so. When it'll be finished, then 'mtnplain' will be quite complete solution. > > PS. I think that 'plain' is better name for 'dumb'. And my proposition is > > to name the tool is 'mtnplain'. > > Hmm, I'm not sure. There are two problems: > 1) "dumb" is sort of a stupid name, but it has a lot of history > behind it :-). The standard word for using HTTP/FTP/etc. as a > VCS transport has been "dumb" for many years now, and it's been > used by a lot of different people (starting with the Arch > people). > 2) "plain" is very ambiguous. For instance, surely "plain" monotone > is what you get when you go to > http://venge.net/monotone/downloads/ > right? Hmmm, for now lets leave it to others. In fact i don't care about name. > ... > Want to send me a key, so you can use the venge.net server to > distribute your work? You've already added it some time ago, but from then i went sleep (in a mean of development) and didn't pushed anything. I'll push these changes ASAISFTFOS (as soon as.i find some free time for open source). -- Best regards, Zbyszek ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] mtnplain (aka dumb) revision order, epoch
On Fri, Jun 23, 2006 at 09:40:31AM +0200, Zbigniew Zagórski wrote: > Hi, > > Recently I recognized that plain (aka dumb) that is rather far from working > with > mtn 0.26. > On 0.27 it's better because it works on windows (automate stdio newlines > fixed). > There is a problem with revision reordering which I almost fixed in my > sandbox. > > This fix is to recognize all the certs received by Feeder and > queue them accordingly to the type: > - fdata - write them immediately > - fdelta - write them immediately > - pubkey - queue > - rdata - queue > - rcert - queue > Then when everything is fed to the Feeder: > - write all 'pubkey' - this probably can go unordered > - write all 'rdata' (*) > - write all 'rcert' > > (*) The hack is with writing rdata in correct order. Currently after writing > each rdata packet mtn 'stderr' is checked if contains: >missing prerequisite revision 'REVID' > and if text is found then this revision is queued as a child of REVID. Then > when > REVID is written then all its children are queued again. > > This is almost correct but certainly that 'stderr' matching is awfully evil. > Yes > i know. > Question: How to obtain parent REV(s) from rdata packet without using 'mtn > read' > ? I thought that someone else already fixed this in the current head of the .dumb branch -- the better solution is to make sure that when we write things out to the DATA file, we write them in the correct order, so that later on they can just be read in directly like they are now. It's possible that this is in fact fixed, but you're trying to pull from a "dumb" repository that was created before this was fixed, so the packets are still in the wrong order. I suggest doing a push into a fresh dumb repository, and then pulling from that, and seeing if that fixes the problem. > Second problem. > After finishing packet reordering I found that 'mtnplain' 'pull' to a clean > database fails. > This error was after following steps: > 1. 'mtnplain' -d DB.mtn push SOMEPLACE > 2. mtn -d DB2.mtn db init > 3. 'mtnplain' -d DB2.mtn pull SOMEPLACE > 4. mtn -D DB2.mtn pull file:DB.mtn > 4th step was to check if all data were transfered by mtnplain if yes - netsync > should transfer nothing; but apparently mtn failed with following error > > |06:07:16#23|[EMAIL PROTECTED]|..umb/plain-zbigg|$ mtn -d DB2.mtn pull > file:DB.mtn > 'net.*' > mtn: connecting to file:DB.mtn > mtn: finding items to synchronize: > mtn: certificates | keys | revisions > mtn: 162 |3 |54 > mtn: warning: error: Mismatched epoch on branch net.venge.monotone.dumb. > Server > has > 'e748431d4147df17a39df80f0c2d72c9d2fa9fb2', client has > ''. > mtn: peer file:DB.mtn disconnected after we informed them of error > mtn: warning: protocol error while processing peer stdio: 'received network > error: Mismatched epoch on branch net.venge. > monotone.dumb. Server has 'e748431d4147df17a39df80f0c2d72c9d2fa9fb2', client > has > ''.' > mtn: bytes in | bytes out | certs in | revs in > mtn: 129 | 235 |0 | 0 > > > Hmm, what is epoch and what should I do while exporting/importing to preserve > epoch of a branch? Ah, good question! The documentation on epochs is here: http://venge.net/monotone/docs/Rebuilding-ancestry.html (It could probably be more clear.) What needs to be done to support them in mtn-dumb is: 1) create a new packet type for epochs 2) for each branch in the db, add this packet to the merkle trie being synced. Make sure to put these packets as early as possible; just like revisions should come before certs, epochs should come before revisions, and even before files. (1) is best done in monotone itself. I'll put it on my list to do, but I already have a bunch of "do this immediately!" things on my list, so if you want to do it, or anyone else out there does, that will likely be quicker :-). You could also, for now, implement (1) yourself -- basically make up a special format that mtn-dumb will recognize. Then generate packets by isng "mtn ls epochs", and when reading in packets, check for the special format, and if you see it then first use "mtn ls epochs" to make sure that the local db has no epoch for the branch in question, if it does have an epoch then error out, and if it doesn't then call "mtn db set_epoch" of passing it to "mtn read". (You might need to queue the set_epoch calls up until the end, so as not to to fight with "mtn read" for locks on the database.) (This is exactly how an in-monotone implementation of epoch packets would do, except with fewer caveats.) > PS. I think that 'plain' is better name for 'dumb'. And my proposition is > to name the tool is 'mtnplain'. Hmm, I'm not sure. There are two problems: 1) "dumb" is sort of a stupid name, but it has a lot of history behind it :-). The standard word for using HTTP/FTP
[Monotone-devel] mtnplain (aka dumb) revision order, epoch
Hi, Recently I recognized that plain (aka dumb) that is rather far from working with mtn 0.26. On 0.27 it's better because it works on windows (automate stdio newlines fixed). There is a problem with revision reordering which I almost fixed in my sandbox. This fix is to recognize all the certs received by Feeder and queue them accordingly to the type: - fdata - write them immediately - fdelta - write them immediately - pubkey - queue - rdata - queue - rcert - queue Then when everything is fed to the Feeder: - write all 'pubkey' - this probably can go unordered - write all 'rdata' (*) - write all 'rcert' (*) The hack is with writing rdata in correct order. Currently after writing each rdata packet mtn 'stderr' is checked if contains: missing prerequisite revision 'REVID' and if text is found then this revision is queued as a child of REVID. Then when REVID is written then all its children are queued again. This is almost correct but certainly that 'stderr' matching is awfully evil. Yes i know. Question: How to obtain parent REV(s) from rdata packet without using 'mtn read' ? Second problem. After finishing packet reordering I found that 'mtnplain' 'pull' to a clean database fails. This error was after following steps: 1. 'mtnplain' -d DB.mtn push SOMEPLACE 2. mtn -d DB2.mtn db init 3. 'mtnplain' -d DB2.mtn pull SOMEPLACE 4. mtn -D DB2.mtn pull file:DB.mtn 4th step was to check if all data were transfered by mtnplain if yes - netsync should transfer nothing; but apparently mtn failed with following error |06:07:16#23|[EMAIL PROTECTED]|..umb/plain-zbigg|$ mtn -d DB2.mtn pull file:DB.mtn 'net.*' mtn: connecting to file:DB.mtn mtn: finding items to synchronize: mtn: certificates | keys | revisions mtn: 162 |3 |54 mtn: warning: error: Mismatched epoch on branch net.venge.monotone.dumb. Server has 'e748431d4147df17a39df80f0c2d72c9d2fa9fb2', client has ''. mtn: peer file:DB.mtn disconnected after we informed them of error mtn: warning: protocol error while processing peer stdio: 'received network error: Mismatched epoch on branch net.venge. monotone.dumb. Server has 'e748431d4147df17a39df80f0c2d72c9d2fa9fb2', client has ''.' mtn: bytes in | bytes out | certs in | revs in mtn: 129 | 235 |0 | 0 Hmm, what is epoch and what should I do while exporting/importing to preserve epoch of a branch? PS. I think that 'plain' is better name for 'dumb'. And my proposition is to name the tool is 'mtnplain'. PS2. Here is latest version of 'mtnplain' http://zzagorski.strony.wi.ps.pl/mtnplain/mtnplain-0.0.3.zip -- Best regards, Zbyszek ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
