Re: [Monotone-devel] ..more notes about .nuskool
Hi, Zbigniew Zagórski wrote: Just for curiosity how big databases you've managed to sync ? Is it useful for syncing n.v.monotone* branch set ? Uh.. nuskool isn't quite there, yet, sorry. With the oldskool netsync, I'm regularly synching a database over 500mb. Maybe a little not related opinion. I've been thinking a lot about all of this while hacking on n.v.m.dumb. I know that current dumb tool can be much more faster if it would be possible to retrieve complete merkle_dir (or anything representing state of the database/project) in some fast way. The HTTP time wasn't too important, the most time consuming was retrieving/toposorting all revisions (not so bad) and all the certs for all revisions (ages). I would like to bring together nuskool and nvm.dumb. Maybe even integrate compatibility to download from dumb servers into monotone. However, the primary point of nuskool is not requiring to build merkle tries anymore. Instead, it uses a DAG based refinement [1], as do most other similar VCSes. So nvm.dumb will probably have to adapt as well. It's slow also for current netsync protocol so I think there must be some idea shift about synchronization. How to synchronize databases without retrieving WHOLE of tree and ALL certs. Well, without a merkle trie, certs will have to be treated differently. Graydon proposed a DAG based synchronization for the certs as well. This would involve creating revisions in "cert branches" - somewhat similar to policy branches. I still need to get my mind around that, but it sounds very interesting to me. See the IRC log here [2]. Regards Markus [1]: Monotone wiki about DAG based refinement: http://www.venge.net/mtn-wiki/DagBasedRefinement [2]: Monotone IRC log: http://colabti.de/irclogger/irclogger_log/monotone?date=2008-01-31,Thu#l51 ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] ..more notes about .nuskool
2008/2/4, Markus Schiltknecht <[EMAIL PROTECTED]>: > However, ATM nvm.nuskool is pretty simplistic and works over HTTP > exclusively, ignoring any branch patterns and synchronizing all > revisions in the database. Just for curiosity how big databases you've managed to sync ? Is it useful for syncing n.v.monotone* branch set ? > I'd like to go for something more RESTful, as far as the HTTP interface > is concerned. This should make better use of the HTTP caching > infrastructure and would probably even allow something like the "mtn > dumb" server interface. > > If we still need olskool netsync (do we?), we should encapsulate the > communication channel stuff from the nuskool new DAG based refinement > thingie, very much like the current refiner is separate. >... > > 1.) sync from mtn client to dumb http server with static files > 2.) sync from mtn client to clever http server via scgi to mtn server > 3.) sync from mtn client to clever mtn server via netsync > > While the first two require a state-less approach, fitting HTTP, the > third option doesn't. I'm not sure if it's worth exploiting that. > Probably not. And the first option is quite different in that we cannot > put any logic into the server, for obvious reasons. Maybe a little not related opinion. I've been thinking a lot about all of this while hacking on n.v.m.dumb. I know that current dumb tool can be much more faster if it would be possible to retrieve complete merkle_dir (or anything representing state of the database/project) in some fast way. The HTTP time wasn't too important, the most time consuming was retrieving/toposorting all revisions (not so bad) and all the certs for all revisions (ages). It's slow also for current netsync protocol so I think there must be some idea shift about synchronization. How to synchronize databases without retrieving WHOLE of tree and ALL certs. -- Zbigniew -zbigg- Zagórski / software developer / geek / happy daddy / ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] ..more notes about .nuskool
Hello Zach, Zack Weinberg wrote: The present netsync protocol uses a cryptographic primitive (RSA decryption) that isn't supported by ssh-agent.¹ This is why you get prompted for your passphrase on 'mtn sync' even if your key is already loaded into the agent. In implementing the new network protocol, please consider making the crypto handshake require *only* RSA/DSA signatures, as that is the only primitive that seems to be universally supported by ssh-agent. I'm not sure how to do that, but evidently ssh itself does, so it must be possible. Thank you for this note. I'm not exactly a crypto expert, so that's very helpful. However, ATM nvm.nuskool is pretty simplistic and works over HTTP exclusively, ignoring any branch patterns and synchronizing all revisions in the database. I'd like to go for something more RESTful, as far as the HTTP interface is concerned. This should make better use of the HTTP caching infrastructure and would probably even allow something like the "mtn dumb" server interface. If we still need olskool netsync (do we?), we should encapsulate the communication channel stuff from the nuskool new DAG based refinement thingie, very much like the current refiner is separate. What I'm pretty dubious about HTTP/JSON with regard to binary data. On [1] they say, that JSON strings allow all Unicode characters, with some exceptions, which can be hex encoded. But writing out valid Unicode from binary data seems troublesome to me. And for the format to be really useful from browsers and such, we'd certainly want to make the format standard compliant. So via HTTP, we are probably going to send around hex or base64 encoded data. Covering all use cases I can currently think of would lead to three, slightly different refiner procedures: 1.) sync from mtn client to dumb http server with static files 2.) sync from mtn client to clever http server via scgi to mtn server 3.) sync from mtn client to clever mtn server via netsync While the first two require a state-less approach, fitting HTTP, the third option doesn't. I'm not sure if it's worth exploiting that. Probably not. And the first option is quite different in that we cannot put any logic into the server, for obvious reasons. Still quite a long way to go... Regards Markus [1]: http://www.json.org/ ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
