[Monotone-devel] cvssync, ssh, serve stdio need a missing common infrastructure
Hi Nathaniel, Hi Jon (Bright), while dusting off the ssh (and serve stdio) branch I realized that all these works need a common infrastructure: The ability to (cross-platform!) deal with pipes. Since serve stdio seemed the least intrusive I looked into separating it from the rest ... but ... serve stdio needs a Netxx Stream which uses one fd for reading (0) and one fd for writing (1). While I know enough of unix architecture to design such a beast (it is part of the ssh branch) I don't know deeply enough how Windows handles such things. I need one Netxx::Stream which handles stdi/o (two file descriptors accessed via read/select/write on unix) and one Stream which handles a bidirectional pipe (the same on unix!). Are these tasks on Windows! similar enough to get covered by one class? I have a working example of a pipe invocation on Windows (2*CreatePipe + CreateProcess) but the HANDLE s are accessed via Read/WriteFile and not via read/write (are write and HANDLEs compatible?). On the other hand my Pipe class is derived from Netxx::Stream but IIRC netxx is designed to handle sockets which are totally different on Windows (I see that get_last_error always uses WSAGetLastError which is nonsense if you access files). So if my worst fears come true netxx::Stream is uncapable on handling even the writing/reading half of neither pipes nor stdio. looking into the source the (undocumented :-( ) Netxx::Socket uses recv() and only handles files on unix because of unix' philosophy. :-( So it looks like I have to write a Socket replacement, too. So my questions: Do you see any alternative to this (e.g. using c++ streambufs in netsync (which is a common interface/superset of netxx::Sockets and files))? IIRC likely to fail because streambufs have no similar concept to select()/Netxx::Probe(). Should I hack netsync (I need a pairint,int instead of int for sockets) in a way which disables these features on Windows. (serve stdio is useless on Windows unless combined with ssh:/file:) Jon: Do you think that stdio and 2*pipe are similar enough on Windows to get covered by one single class? Would you suggest to use read() or ReadFile()? Christof PS: This nicely explains why windows compatibility indead is the greatest hurdle to merging any of these branches into mainline. signature.asc Description: OpenPGP digital signature ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] monotone automate stdio (update)
Timothy Brownawell wrote: Input for monotone automate stdio now has the format 'l'length':'arg[length':'arg...]'e' , as there were problems mentioned with the previous format. Characters between the 'e' for one command and the 'l' for the next are ignored (except for 'l', of course). Output now has the format cmdnum':'err':'last?':'size':'output , where cmdnum is incremented by 1 for each command (statring at 0), err is 0 for success, 1 for a syntax error, or 2 for other errors, last? is 'l' if this is the last piece of output for this command or 'm' if there is more, and size is the number of bytes in this piece of output. Output is broken into pieces not more than 1024 bytes (arbitrarily chosen size) each. As mentioned this isn't a network protocol, so I didn't go with the 3-digit code for each line. Also, this format seems somewhat simpler while still providing all needed information. It also works better with binary output, which was mentioned as a possibility. I've written a client which seems to work (see Automation at the bottom); http://grahame.angrygoats.net/viewmtn/fileinbranch.psp?branch=net.angrygoats.viewmtnpath=monotone.py Just for reference of others trying to use it :-) Cheers Grahame ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] cvssync branch feedback
Matt == Matt Lavin [EMAIL PROTECTED] writes: Matt If you are targeting monotone as the new SCM then cvssync is Matt heads above tailor.py in terms of speed and correct file Matt generation. No doubt on former, given the different overall goals of the two, but I'd like to better understand the latter. Matt I got exactly what I wanted from the cvssync branch in Matt almost the first try as compared to working the tailor.py Matt over the course of a week to finally just give up. I'm sad to hear that, sorry for the waste of time. Maybe I could have been of some help, knowing that. Matt Specifically, cvssync seemed to handle the use of modules Matt and deleted files/directories much better than tailor.py. Well, here I'd gladly accept some more context, and possibly some more details on the errors you found, expecially if they are related to how tailor failed to build up reasonable changesets from upstream CVS. If on the contrary the problem is related to monotone support, I'm sure somebody could be interested as well. Thank you, ciao, lele. -- nickname: Lele Gaifax | Quando vivr di quello che ho pensato ieri real: Emanuele Gaifas | comincer ad aver paura di chi mi copia. email: [EMAIL PROTECTED]| -- Fortunato Depero, 1929. ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] cvssync branch feedback
I hope I didn't appear to be slamming tailor.py, I was trying mostly to compliment cvssync. I'll try to clear up some of my statements. 1. correct file generation should have been correct directory contents. I have a rather large project that I was trying to convert ( ~3500 patches) and at the end of the convert I did a simple diff between a fresh CVS checkout and a darcs (for tailor.py) or monotone (for cvssync) checkout of the converted project. For the cvssync checkout the directory contents matched exactly (ignoring CVS, .cvsignore, and MT directories). For the tailor.py build project I had a couple of extra empty directories and 1 missing file. One thing I noticed about the empty directories is that in the final CVS checkout from tailor.py they all had completely blank CVS/Entries files. My 30 second guess for the problem was that cvs update would not delete empty directories when the last file is deleted. About the missing file, I have no idea why it would happen. 2. The wasted week. Most of the time was spent waiting to see the results of the convert. I was originally converting over pserver to a remote location ( before I was banned by my local admin =). Interestingly, it was not significantly faster when I convinced him to provide CVSROOT tarballs and I tried build from a local directory. During the week, I did find a slow leak of file handles in tailor.py which I submitted a patch for, and is now included, so it was hardly a waste of time. Another time sink was trying to figure out the exact time stamp to use to get the entire history of the project. Maybe I'm doing something wrong, but it was quite hard for me to pick just the right time so that the whole history would be picked up. (I finally used cscvs to read the repo, and used the datetime of the initial commit it found). 3. Lack of CVS module support. When I tried to use tailor.py on a CVS project which used modules it complained: Traceback (most recent call last): File /home/matt/projects/cvsync/tailor.py, line 36, in ? main() File /home/matt/projects/cvsync/vcpx/tailor.py, line 615, in main options.subdir) File /home/matt/projects/cvsync/vcpx/tailor.py, line 325, in bootstrap logger=self.logger) File /home/matt/projects/cvsync/vcpx/dualwd.py, line 74, in checkoutUpstreamRevision revision, **kwargs) File /home/matt/projects/cvsync/vcpx/source.py, line 179, in checkoutUpstreamRevision module, revision, File /home/matt/projects/cvsync/vcpx/cvsps.py, line 343, in _checkoutUpstreamRevision csets = self.getUpstreamChangesets(wdir, repository, module) File /home/matt/projects/cvsync/vcpx/cvs.py, line 318, in getUpstreamChangesets for cs in changesets_from_cvslog(log, module): File /home/matt/projects/cvsync/vcpx/cvs.py, line 68, in changesets_from_cvslog collected = ChangeSetCollector(log, module) File /home/matt/projects/cvsync/vcpx/cvs.py, line 109, in __init__ self.__parseCvsLog() File /home/matt/projects/cvsync/vcpx/cvs.py, line 274, in __parseCvsLog cs = self.__parseRevision(entry) File /home/matt/projects/cvsync/vcpx/cvs.py, line 180, in __parseRevision revision = self.__readline() File /home/matt/projects/cvsync/vcpx/cvs.py, line 160, in __readline assert currentdir.startswith(self.module), \ AssertionError: Directory some cvs directory does not start with module name 3 out of 4 projects I was interested in use CVS modules so when I finally got this error I put less emphasis on trying to debug more of the problems from 1. Unfortunately, I cannot distribute the source of the repo, but I would be glad to test out new changes. And if I had more free time, I'd supply more fixes. Lele Gaifax wrote: Matt == Matt Lavin [EMAIL PROTECTED] writes: Matt If you are targeting monotone as the new SCM then cvssync is Matt heads above tailor.py in terms of speed and correct file Matt generation. No doubt on former, given the different overall goals of the two, but I'd like to better understand the latter. Matt I got exactly what I wanted from the cvssync branch in Matt almost the first try as compared to working the tailor.py Matt over the course of a week to finally just give up. I'm sad to hear that, sorry for the waste of time. Maybe I could have been of some help, knowing that. Matt Specifically, cvssync seemed to handle the use of modules Matt and deleted files/directories much better than tailor.py. Well, here I'd gladly accept some more context, and possibly some more details on the errors you found, expecially if they are related to how tailor failed to build up reasonable changesets from upstream CVS. If on the contrary the problem is related to monotone support, I'm sure somebody could be interested as well. Thank you, ciao, lele. ___ Monotone-devel mailing list Monotone-devel@nongnu.org
Re: [Monotone-devel] [PATCH] and RFC: binary files merging and hook
On Tuesday 24 May 2005 17:33, rghetta wrote: function binary_file(name) lowname=string.lower(name) -- some known binaries, return true if (string.find(lowname, %.gif$)) then return true end You base the text/binary decision on the name of the file. How hard would it be to base it on the contents of the file instead, the way the Unix 'file' command does? ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] [PATCH] and RFC: binary files merging and hook
On Wed, May 25, 2005 at 12:33:04AM +0200, rghetta wrote: If the hook returns nil, the file will be treated as binary if the monotone function guess_binary() returns true, i.e. if the files contains NUL bytes or a selection of other ASCII control chars (for example, STX and ETX). Another possible way to do binary support, for discussion: -- have the merger peek at .mt-attrs, and if a binary attribute is set on a file, consider it binary. (Currently nothing in .mt-attrs has hard-coded behavior, so this would be a change.) -- use the cool new attr_init hooks to automatically guess at add time whether each file is binary. -- never again automatically touch this attribute; let people set it to what they want, if they want Another possible way to do binary support, for discussion: -- just use guess_binary() on the data at merge time I don't tend to store binary files under VCS, so I don't have as much of an intuition about what the nicest way to do so would be; it'd be good to hear opinions from those actually affected by this :-) -- Nathaniel -- /* Tell the world that we're going to be the grim * reaper of innocent orphaned children. */ -- Linux kernel 2.4.5, main.c This email may be read aloud. ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
