Re: [Monotone-devel] date certs on net.venge.monotone
Hi, Nathaniel Smith wrote: > Make warns about clock skew (between your local host and its NFS > server) because such clock skew breaks Make. Agreed. > Right now, clock skew (between all the hosts used by everyone in a > distributed development project) does not break monotone; This depends on whether you count invalid date certs as breakage or not. You obviously don't, I do. > it continues > to provide its basic functionality of managing revisions just fine. > Your proposal would compromise this core functionality (either by > causing revisions to disappear, or just printing annoying warning > messages), so as to better provide the functionality of letting people > know that someone else's clock that they can't fix is broken. The main motivation is making dates cert information more reliable. But it looks like that's not considered "core functionality" of monotone. > I understand the impulse, Phew, good to hear. > but I don't see how this is the right place > for it -- if I really want to know about clock skew, there are better > ways to find out, and if don't have access to those better ways, I can > just run that script you wrote over my monotone history anyway. My question is: how do I avoid silly situations like this one here: # mtn -d $BOTAN_DB log \ > --brief --last=3 \ > --from 1cc7ac30ff0c2871e24e3a76d147cab907b5429e o 1cc7ac30ff0c2871e24e3a76d147cab907b5429e [EMAIL PROTECTED] | 2006-09-26T19:33:01 net.randombit.botan o fb8114127ac9e23c9870bba631a79c41ece29355 [EMAIL PROTECTED] | 2006-09-27T03:44:57 net.randombit.botan o 3c74c0b06690f73b9d34435e77432e6776c1f7e3 [EMAIL PROTECTED] 2006-09-27T01:29:54 net.randombit.botan Why does monotone show a date at all, if it's so unreliable, unchecked and not considered "core functionality" by monotone? Anyway, I think there's enough resistance to drop the idea for now. Regards Markus Wanner ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] date certs on net.venge.monotone
On Thu, Oct 23, 2008 at 5:12 AM, Markus Wanner <[EMAIL PROTECTED]> wrote: > Thomas Moschny wrote: >> So, I don't think we should warn there. > > Even make hints about clock skews. Why is that so unwanted for monotone? Make warns about clock skew (between your local host and its NFS server) because such clock skew breaks Make. Right now, clock skew (between all the hosts used by everyone in a distributed development project) does not break monotone; it continues to provide its basic functionality of managing revisions just fine. Your proposal would compromise this core functionality (either by causing revisions to disappear, or just printing annoying warning messages), so as to better provide the functionality of letting people know that someone else's clock that they can't fix is broken. I understand the impulse, but I don't see how this is the right place for it -- if I really want to know about clock skew, there are better ways to find out, and if don't have access to those better ways, I can just run that script you wrote over my monotone history anyway. -- Nathaniel ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] date certs on net.venge.monotone
Hi, Thomas Moschny wrote: > Monotone stores timestamps in the date certs in UTC, of course. Nevertheless, > clocks for each developer might differ a bit such that commits appear to be > made in non-chronological order. That's theoretically possible, though pretty rare in practice. > So, I don't think we should warn there. Even make hints about clock skews. Why is that so unwanted for monotone? Regards Markus Wanner ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] date certs on net.venge.monotone
Hi, Stephen Leake wrote: > If there is no global shared clock, then developers working in > different time zones can easily commit revisions to a shared server > that have timestamps using local times that appear to be "out of > order" with respect to each other. Any sane OS today knows the difference between local time and UTC. Measurement shows that this is not an issue (except maybe for Xaraya, which needs further investigation) > On the other hand, there are such things as global clocks; we use them > all the time at NASA. If each developer on a team commits to > configuring their computer such that the functions "local_to_UTC" and > "UTC_to_local" work properly, My assumption is that most clocks are close enough to "the" global clock to be useful for mtns purpose. Looking at the dates in nvm, this assumption holds true for the vast majority of committed revisions. > monotone could support an option to use > UTC times for timestamps. Monotone currently uses UTC dates for its certs, which might not be overly user friendly. However, we do not take leap seconds into account within our own date calculation routines. > TAI might be a better global time choice, > since it doesn't do leap seconds, and hence is monotonic. Conversion between UTC and local time is pretty simple on most systems, where as UTC to TAI and back is not as common. IMO the user shouldn't need to care about anything except local time. Regards Markus Wanner ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] date certs on net.venge.monotone
Stephen Leake wrote: > Markus Wanner <[EMAIL PROTECTED]> writes: > > I'm just proposing to check the date and warn the user if it obviously > > doesn't match. It would help making that meta-information more reliable, > > nothing more, nothing less. > > If there is no global shared clock, then developers working in > different time zones can easily commit revisions to a shared server > that have timestamps using local times that appear to be "out of > order" with respect to each other. Monotone stores timestamps in the date certs in UTC, of course. Nevertheless, clocks for each developer might differ a bit such that commits appear to be made in non-chronological order. > Why would you want to warn about that? So, I don't think we should warn there. - Thomas -- Thomas Moschny <[EMAIL PROTECTED]> signature.asc Description: This is a digitally signed message part. ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] date certs on net.venge.monotone
Markus Wanner <[EMAIL PROTECTED]> writes: > I'm just proposing to check the date and warn the user if it obviously > doesn't match. It would help making that meta-information more reliable, > nothing more, nothing less. If there is no global shared clock, then developers working in different time zones can easily commit revisions to a shared server that have timestamps using local times that appear to be "out of order" with respect to each other. Why would you want to warn about that? On the other hand, there are such things as global clocks; we use them all the time at NASA. If each developer on a team commits to configuring their computer such that the functions "local_to_UTC" and "UTC_to_local" work properly, monotone could support an option to use UTC times for timestamps. TAI might be a better global time choice, since it doesn't do leap seconds, and hence is monotonic. -- -- Stephe ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] date certs on net.venge.monotone
Hi, Ethan Blanton wrote: > That is correct. It actually caused problems for us at some point, > when something related to monotone started rejecting those fractional > second values. The oldest revisions in that Pidgin data started out > as CVS, were migrated to SVN, and then subsequently migrated to > Monotone. There's plenty of weird and wonderful-ness in there. ;-) Aha, thanks for confirmation. > Which brings me to a (yet another) reason that I don't think this > mucking about with date certs is a good idea -- there are almost > certainly *correct* date certs in the Pidgin repository which are out > of order. I'm not sure what you mean be *correct* dates certs being out of order. I've counted exactly 36 discrepancies in the pidgin repository, see my mail starting this thread: > pidgin: 36 of 28349 (0.13%) > During our migration, there were a number of revisions > which, for one reason or another, had problems which made them > difficult to migrate (svn is capable of creating revisions under > normal circumstances which it cannot subsequently handle gracefully). Yeah, svn is capable of doing lots of nasty things... > There are sequences of such revisions which were collapsed to one > "patch-up" revision in the migration; the date stamp on the patch-up > revision is the date of its creation, not the date of the original > changes. This causes one date in the middle of a sequence of valid > revisions to be months or even years newer than its ancestors or > descendants. Agreed. > Generally speaking, date information in monotone is very much > meta-information. Introducing the idea of global clocks in this > manner takes it from being meta-information to being revision > information. I'm just proposing to check the date and warn the user if it obviously doesn't match. It would help making that meta-information more reliable, nothing more, nothing less. > I think this is a bad idea. If I've had a patch sitting > on my disk for 2 months, and I just now get around to committing it, > it may be a reasonable and correct thing for me to want to do to use a > --date argument to the commit of the creation of the patch, not the > commit of the patch. In that case you should certainly also commit against the parent revision you've created the patch against. So that would not result in timestamp inconsistencies. > I guess, basically, I'm not a fan of validating anything for > consistency which does not have a clear definition of consistency. In > the absence of global clocks, dates do not have consistency. The fact > that monotone currently recognizes and ignores this information is a > feature, not a bug. Monotone does not ignore the information, but create date certs from it. Ignoring it would mean dropping date certs entirely. Please keep in mind that for us human beings there *is* a global clock and we like to express ourselves in terms of date and time. Of course, monotone cannot rely on the system clock being in sync with the global clock. But as pointed out, the assumption is good enough to provide that convenience feature called "date cert". To help the user making that convenience feature more reliable, it would help if monotone checked date certs, IMO, without hurting anything else. Regards Markus Wanner ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] date certs on net.venge.monotone
Bruce Stephens spake unto us the following wisdom: > Markus Wanner <[EMAIL PROTECTED]> writes: > > What surprised me about the pidgin repository is, that there are some > > date certs with six digits (!) for milliseconds in the date, i.e.: > > "2006-11-01T21:39:36.807940" > > IIRC Tailor did that at one point, so likely that's a result of > converting from whatever they used before (subversion?). That is correct. It actually caused problems for us at some point, when something related to monotone started rejecting those fractional second values. The oldest revisions in that Pidgin data started out as CVS, were migrated to SVN, and then subsequently migrated to Monotone. There's plenty of weird and wonderful-ness in there. ;-) Which brings me to a (yet another) reason that I don't think this mucking about with date certs is a good idea -- there are almost certainly *correct* date certs in the Pidgin repository which are out of order. During our migration, there were a number of revisions which, for one reason or another, had problems which made them difficult to migrate (svn is capable of creating revisions under normal circumstances which it cannot subsequently handle gracefully). There are sequences of such revisions which were collapsed to one "patch-up" revision in the migration; the date stamp on the patch-up revision is the date of its creation, not the date of the original changes. This causes one date in the middle of a sequence of valid revisions to be months or even years newer than its ancestors or descendants. Generally speaking, date information in monotone is very much meta-information. Introducing the idea of global clocks in this manner takes it from being meta-information to being revision information. I think this is a bad idea. If I've had a patch sitting on my disk for 2 months, and I just now get around to committing it, it may be a reasonable and correct thing for me to want to do to use a --date argument to the commit of the creation of the patch, not the commit of the patch. I guess, basically, I'm not a fan of validating anything for consistency which does not have a clear definition of consistency. In the absence of global clocks, dates do not have consistency. The fact that monotone currently recognizes and ignores this information is a feature, not a bug. Ethan -- The laws that forbid the carrying of arms are laws [that have no remedy for evils]. They disarm only those who are neither inclined nor determined to commit crimes. -- Cesare Beccaria, "On Crimes and Punishments", 1764 signature.asc Description: Digital signature ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] date certs on net.venge.monotone
Hi, Daniel Carosone wrote: > If we're only storing up to seconds, it could happen readily for a > multi-way merge that created several intermediate merge nodes in the > one command. Sure. See the long list of revisions I've posted, there's one such case within "net.venge.monotone*": ee293e066aa9e8222b7ce3b854affe49ddf9317c 4dd538bd7bb906041b817b8bc6a93e87fb3759d5 Both of these revisions have the timestamp "2008-04-30T15:32:06". But AFAICT this is the only occurrence within the monotone repository so far. And the Oscar for the quickest merge goes to: Author: [EMAIL PROTECTED] :-) Regards Markus Wanner > - > Revision: 4dd538bd7bb906041b817b8bc6a93e87fb3759d5 > Ancestor: ee293e066aa9e8222b7ce3b854affe49ddf9317c > Ancestor: ee5efcb81167e04e6c915ce21bb32b832b7ad964 > Author: [EMAIL PROTECTED] > Date: 2008-04-30T15:32:06 > > - > Revision: ee293e066aa9e8222b7ce3b854affe49ddf9317c > Ancestor: 301352168b24a1633d45b92051592ed44142471b > Ancestor: e2ed2c1ee19f1f3ad1616838e6cbd1be66fc020a > Author: [EMAIL PROTECTED] > Date: 2008-04-30T15:32:06 ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] date certs on net.venge.monotone
On Mon, Oct 20, 2008 at 10:49:12AM +0200, Markus Wanner wrote: > (Yes, this currently includes counting > dependent revisions with equal timestamps as invalid, which is certainly > bogus, just highly unlikely). If we're only storing up to seconds, it could happen readily for a multi-way merge that created several intermediate merge nodes in the one command. -- Dan. pgpnZItPOrMKh.pgp Description: PGP signature ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] date certs on net.venge.monotone
Markus Wanner <[EMAIL PROTECTED]> writes: [...] > What surprised me about the pidgin repository is, that there are some > date certs with six digits (!) for milliseconds in the date, i.e.: > "2006-11-01T21:39:36.807940" IIRC Tailor did that at one point, so likely that's a result of converting from whatever they used before (subversion?). [...] ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
[Monotone-devel] date certs on net.venge.monotone
Hi, as mentioned in other threads already, I'm currently analyzing date certs and comparing them against the revision ancestry. The obvious first test database includes all of "net.venge.monotone*" (and nothing else). At the time of testing, it had 14156 revisions with 17737 relations between them stored in the table revision_ancestry. 32 of the ancestry relations are initial revisions with the null_id as parent. All revisions had at least one date cert. If there are multiple date certs for the same revision, I've decided to counting the earliest date for the revision. Checking these dates against the revision ancestry reveals, that only 58 (0.33%) of all of these dependencies turn out to have inversed date information (see [1] for a full list). For all of the remaining 17647 (99.67%) ancestory relations, the date information is plausible (i.e. the timestamp of the date cert on the ancestor revision is lower than that of the descendant revision) (Yes, this currently includes counting dependent revisions with equal timestamps as invalid, which is certainly bogus, just highly unlikely). For Botan and Pidgin repositories it's pretty similar: Botan: 3 of 1786 (0.17%) Pidgin: 36 of 28349 (0.13%) What surprised me about the pidgin repository is, that there are some date certs with six digits (!) for milliseconds in the date, i.e.: "2006-11-01T21:39:36.807940" Thus, I've added support for that format (without parsing the milliseconds part, though) in nvm.dates. OpenEmbedded (latest mtn snapshot) is slightly higher: 501 of 32686 (1.53%) But what currently puzzles me is the Xaraya repository: Xaraya: 6037 or 23413 (25%) I didn't investigate further, but the time gaps are pretty large... however, maybe choosing the earliest date just isn't clever enough. Regards Markus Wanner [1]: full list of date mismatches: > mtn: checking timestamps of ancestor revisions > mtn:rev 006972c572eb094a3c03d3ea6105e7f31f5bfa05 -> rev > c6bd5efd89266879cf9cbe7006b0fe5ca5d5b4dc > mtn:but date 2003-10-20T22:15:28 ! -> 2003-10-20T21:38:09 > mtn:(difference: 2239 seconds) > mtn:rev 02ef37090f20abd12a6aba5cd0244ca78a05ab11 -> rev > a43b9f2bec32d4a63f073e414f067862d676f04c > mtn:but date 2004-05-03T00:48:36 ! -> 2004-05-03T00:05:41 > mtn:(difference: 2575 seconds) > mtn:rev 05a0f9ce71013e8bc5c04c825a53d17a5e3007a9 -> rev > 3011908f7f234e48e5d90160cb406f02f09c19d2 > mtn:but date 2007-09-02T05:29:33 ! -> 2007-09-02T03:35:35 > mtn:(difference: 6838 seconds) > mtn:rev 05ee0ead1ba7806116bd8f887b8206790a2a7eec -> rev > 692d74c8ef9a112ba23ece0018daf82ad4382942 > mtn:but date 2005-06-25T01:00:01 ! -> 2005-06-25T00:06:00 > mtn:(difference: 3241 seconds) > mtn:rev 080864450f24829523bed008fbad268a6c7eb05c -> rev > b664839edf950807096ce987038b1d8ea0ebd0e9 > mtn:but date 2004-08-09T17:21:34 ! -> 2004-08-05T23:09:44 > mtn:(difference: 324710 seconds) > mtn:rev 0826abb7a5e5aacbcda1d3634178b718aada8378 -> rev > 00900acfa10d85988692ee7571028e7364afef79 > mtn:but date 2005-07-24T02:09:56 ! -> 2005-07-23T23:30:22 > mtn:(difference: 9574 seconds) > mtn:rev 087994568abcbf9aee0347356bad0e4af09152d2 -> rev > a9cd9f0a9d3d93dcd451ac49254d92603ec1272f > mtn:but date 2005-05-03T11:00:00 ! -> 2005-05-03T10:55:12 > mtn:(difference: 288 seconds) > mtn:rev 13df407d1c24805c9a350dc49257c48a9ff3f2d4 -> rev > 91891a99890d2a9e497f3055f1f787d62d408d67 > mtn:but date 2005-10-19T07:52:39 ! -> 2005-10-19T06:23:00 > mtn:(difference: 5379 seconds) > mtn:rev 14312f90136babc24be6a746397f99b121e10353 -> rev > 650d4a285dbd92d434399202ec7fd3d412ba444d > mtn:but date 2005-11-21T01:18:57 ! -> 2005-11-21T00:53:23 > mtn:(difference: 1534 seconds) > mtn:rev 14e42d73aa188b25687c93aecf799fc4b7b9f1e6 -> rev > f128c0fb1cd5bab53172c3fb443d3a7207182844 > mtn:but date 2006-03-13T06:28:22 ! -> 2006-03-13T00:31:35 > mtn:(difference: 21407 seconds) > mtn:rev 1ca6b8755c5fed18ca636aee06149963c0b2eef4 -> rev > 3e8a78b0f8151f58578134cad523b7c664229760 > mtn:but date 2005-05-03T07:55:58 ! -> 2005-05-03T07:46:23 > mtn:(difference: 575 seconds) > mtn:rev 1d11d03b9643ccf9305e332b598e59622231f998 -> rev > fa4cedc2a54a54225587fd0f788f63fa887cc6e9 > mtn:but date 2006-08-06T09:04:46 ! -> 2006-08-06T08:57:07 > mtn:(difference: 459 seconds) > mtn:rev 25609512782fe474133707a82d73349aa56f1f5c -> rev > 40bbc03627a68c9e4afd5fddd9fc9401e6e26b2f > mtn:but date 2007-12-16T17:10:30 ! -> 2007-12-16T17:08:30 > mtn:(difference: 120 seconds) > mtn:rev 296d8fb7e07fbead3128638ca638e525e1ab38fc -> rev > 5c9e19109f4627ad83b51037afec5157e4be06ca > mtn:but date 2005-07-27T11:41:38 ! -> 2005-07-27T11:19:22 > mtn:(difference: 1336 seconds) > mtn:rev 2b1e8f23addc3eb4aaf326f5632ce1e6fef5dad1 -> rev > 059e2e79d9de7a2a30a9a67947308df4cc970c62 > mtn:but date 2005-07-14T00:44:41 ! -> 2005-07-13T23:58:09 > mtn:(difference: 2792 seconds)
