Re: making CVS more convenient
The idea is to support a cache repository (the one copied to a local machine by CVSup or CTM) transparently. So that the reads from directory will go from the local cache repository (and won't overstrain the remote server, and will be fast too), while the commits and other changes will go into the remote master repository. Good stuff. I wanted something like this *years* ago when we first started using CVS in FreeBSD. The value specified in CVSROOTCACHE is the local path to the cache repository. All the check-outs, updates, diffs etc. will be obtained from there. All the check-ins, tagging etc. will go into the master repository specified by CVSROOT. Naturally, to see these changes in the cache repository, it needs to be updated by some outside means such as CVSup or CTM. So, the cache doesn't automagically update itself when commits are done? This is less useful, since often-times after a commit has been done the user is still working in the same general area, so a 'cvs update' would now give the user older files since the read-only cache is not up-to-date, thus making it a requirement that everytime you make a commit, you also sychronize the cache. If this could be done automagically, it would make the cache more 'coherent' and things much more useful. Nate To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
Nate Williams wrote: The value specified in CVSROOTCACHE is the local path to the cache repository. All the check-outs, updates, diffs etc. will be obtained from there. All the check-ins, tagging etc. will go into the master repository specified by CVSROOT. Naturally, to see these changes in the cache repository, it needs to be updated by some outside means such as CVSup or CTM. So, the cache doesn't automagically update itself when commits are done? This is less useful, since often-times after a commit has been done the user is still working in the same general area, so a 'cvs update' would now give the user older files since the read-only cache is not up-to-date, thus making it a requirement that everytime you make a commit, you also sychronize the cache. That's the plan for the next stage, provided that the first stage goes well. I'm yet to play with CVSup and see if it can be integrated there (as with system()) easily without making a lot of changes to CVS itself. Otherwise I'm aftarid it's going to be a large amount of work to duplicate this functionality :-( Yet another idea is to be able to make local commits with committing them to the central remote repository later. Now I have to use RCS locally for the temporary in-delevopment versions of file. Would be nice to have a kind of a local branch which can be later committed as a whole - in one commit per file, or by duplicating all the intermediate versions with their messages. -SB To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
The value specified in CVSROOTCACHE is the local path to the cache repository. All the check-outs, updates, diffs etc. will be obtained from there. All the check-ins, tagging etc. will go into the master repository specified by CVSROOT. Naturally, to see these changes in the cache repository, it needs to be updated by some outside means such as CVSup or CTM. So, the cache doesn't automagically update itself when commits are done? This is less useful, since often-times after a commit has been done the user is still working in the same general area, so a 'cvs update' would now give the user older files since the read-only cache is not up-to-date, thus making it a requirement that everytime you make a commit, you also sychronize the cache. That's the plan for the next stage, provided that the first stage goes well. I'm yet to play with CVSup and see if it can be integrated there (as with system()) easily without making a lot of changes to CVS itself. Otherwise I'm aftarid it's going to be a large amount of work to duplicate this functionality :-( Another choice is to have the commit be also made to the 'cache' if and only if the remote (master) respository has accepted the commit. That way, the commit is made in both repositories using the same algorithm, so in essence they should be in sync. This saves the overhead of running a complete synchronization of all the files. And, you have a safe 'fallback' of mirroring the remote tree which should cleanup any problems you had, which will still be for any non-local modifications. Yet another idea is to be able to make local commits with committing them to the central remote repository later. I'd do the reverse, since the possibility of synchronization problems are a huge deal. Imagine if someone 'snuck' in and made a commit in the master tree after your local commit was made, but before 'later' occurred and your cache pushed it out to the master tree. If you only allow the commit if it can occur locally, you're ensuring that the cache can't get out of date with local changes. I tried something like the above (cause it was easier to implement), and it worked most of the time. However, the times it didn't work it was a royal pain in the *ss to cleanup and get the original commit back out. Now I have to use RCS locally for the temporary in-delevopment versions of file. Would be nice to have a kind of a local branch which can be later committed as a whole - in one commit per file, or by duplicating all the intermediate versions with their messages. Agreed. The downside to the above method is that it requires network access to make a commit. However, it certainly simplifies the problem set. :) :) Nate To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
Yet another idea is to be able to make local commits with committing them to the central remote repository later. I'd do the reverse, since the possibility of synchronization problems are a huge deal. Imagine if someone 'snuck' in and made a commit in the master tree after your local commit was made, but before 'later' occurred and your cache pushed it out to the master tree. If you only allow the commit if it can occur locally, you're ensuring that the cache can't get out of date with local changes. I tried something like the above (cause it was easier to implement), and it worked most of the time. However, the times it didn't work it was a royal pain in the *ss to cleanup and get the original commit back out. It'd be cool to teach CVSup to ignore updating certain files that have been marked locally as dirty or in flux until they've been committed through to the master repo. Even better would be to have CVSup ignore making changes to designated branches (RELENG_5_SEANC). The corollary to that would be to teach cvs(1) to commit changes to the master repo that have been made to the local repo. Version number sync would be a problem, but it'd be really cool to be able to do that. With a branch mapping layer (RELENG_5_SEANC - HEAD), people could actually get back into the habit of using CVS as a development tool instead of just a way of publishing finalized work. Maybe the above changes could be rolled into the rewrite of CVSup in C. CVSup - C ld cvsup -lkse cvsup(1) - base system ::grin:: -sc -- Sean Chittenden pgp0.pgp Description: PGP signature
jail support for ping, traceroute, etc.. crude hack
so, i am working on building a super-server for me and several friends to collaborate with on the money front to put our machine in a colo location, etc.. and still have good access to networking resources. as a result, i needed to modify the FreeBSD kernel such that it will allow us to use ping, traceroute and other tools. obviously we know there will be some underlying security issues associated but we are sophisticated to understand the nature of these and they are an 'acceptable' situation. my diffs are available at http://puck.nether.net/~jared/fbsd-4.8-rc1-diff-jail-raw_ip.txt and are against the 4.8-rc1 /usr/src/sys tree yeah, they're crude but it gets the desired job done. there is a sysctl to control it, so if its not the desired operation it can be easily tweaked. send me comments. enjoy, - jared -- Jared Mauch | pgp key available via finger from [EMAIL PROTECTED] clue++; | http://puck.nether.net/~jared/ My statements are only mine. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
With the -s option, cvsup will not touch files that it believes are in sync until they are updated on the server. ^^^ not ? no, not not. cvsup will not touch files that it believes are in sync, the operative word here being believes - with -s cvsup will use the checkout list and won't even look at the actual file unless the server says it has a newer version than what is listed in the checkout list. I think we're having a mis-communication. I want cvsup to edit the files that are in sync with the master server. I want cvsup to ignore or leave alone files that are out of sync with the server (ie: don't re-transfer the file in the assumption that its been corrupted). We working from the same set of intentions and just colliding in the communication dept (most likely what it means to be in sync)? -s is a bit dicey to trust unless you grab an exclusive lock on the file and prevent other people from making a change to the file on the server. -sc You actually *want* the -s induced failure in this case. If -s induces a failure when there are local changes that haven't come down from the master repo, that'd be slick... it still sounds like you'd have to clear your ,v file if you commit your changes after someone had made a change to the file after you made a local commit. [local commit to file A ] [different developer commits to file A on master repo] [commit to file A on master repo] [cvsup local repo with master repo] Wouldn't you have to delete A,v before A,v would continue to pick up future changes? -sc -- Sean Chittenden To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
Sean Chittenden [EMAIL PROTECTED] writes: [local commit to file A ] [different developer commits to file A on master repo] [commit to file A on master repo] [cvsup local repo with master repo] Wouldn't you have to delete A,v before A,v would continue to pick up future changes? -sc No, cvsup would notice that the local file has the wrong size and re-fetch it (rather than just add the missing delta) so the local change would be replaced with the remote change. DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
[local commit to file A ] [different developer commits to file A on master repo] [commit to file A on master repo] [cvsup local repo with master repo] Wouldn't you have to delete A,v before A,v would continue to pick up future changes? -sc No, cvsup would notice that the local file has the wrong size and re-fetch it (rather than just add the missing delta) so the local change would be replaced with the remote change. Right, which is what I was trying to suggest a fix for in the first place: the ability to prevent the loss of work committed to a local repository when using cvsup to sync repositories with the master repo. -sc -- Sean Chittenden To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
RE: jail support for ping, traceroute, etc.. crude hack
Hello, This patch is interesting. To my understanding though, ipfw uses RAW sockets to communicate with the kernel. Therefore, it might be possible to edit the ipfw table from within the jail, which may be a bad thing. Just a thought. Thanks, -- Mooneer Salem GPLTrans: http://www.translator.cx/ lifeafterking.org: http://www.lifeafterking.org/ -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Jared Mauch Sent: Sunday, March 16, 2003 1:14 PM To: [EMAIL PROTECTED] Subject: jail support for ping, traceroute, etc.. crude hack so, i am working on building a super-server for me and several friends to collaborate with on the money front to put our machine in a colo location, etc.. and still have good access to networking resources. as a result, i needed to modify the FreeBSD kernel such that it will allow us to use ping, traceroute and other tools. obviously we know there will be some underlying security issues associated but we are sophisticated to understand the nature of these and they are an 'acceptable' situation. my diffs are available at http://puck.nether.net/~jared/fbsd-4.8-rc1-diff-jail-raw_ip.txt and are against the 4.8-rc1 /usr/src/sys tree yeah, they're crude but it gets the desired job done. there is a sysctl to control it, so if its not the desired operation it can be easily tweaked. send me comments. enjoy, - jared -- Jared Mauch | pgp key available via finger from [EMAIL PROTECTED] clue++; | http://puck.nether.net/~jared/ My statements are only mine. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
Sean Chittenden wrote: It'd be cool to teach CVSup to ignore updating certain files that have been marked locally as dirty or in flux until they've been committed through to the master repo. Even better would be to have CVSup ignore making changes to designated branches (RELENG_5_SEANC). This issue comes up at least once a year. I first suggested this nearly 10 years ago, right after CVSup was first introduced. Due to logistical problems, this actually won't work, as we discovered after the first such discussion, around that time. What has to happen is an ability to CVSup a remote project repository into a local vendor branch. Local modifications occur on the vendor branch, which can then be tagged and merged at will with the vendor branch head, to create a new local vendor branch. The closest that CVS/CVSup has come to this is the idea of a local vendor branch that implicitly does not generate conflicts on a reimport. The corollary to that would be to teach cvs(1) to commit changes to the master repo that have been made to the local repo. Version number sync would be a problem, but it'd be really cool to be able to do that. With a branch mapping layer (RELENG_5_SEANC - HEAD), people could actually get back into the habit of using CVS as a development tool instead of just a way of publishing finalized work. Nate's suggestion covers the version number issue... sort of. It assumes that the patches will be approved for commit to the main repo, and it assumes that the main repo will not get tagged in between. The main problem with that is pretty obvious, especially around code-freeze/code-slush, but also for anyone without a commit bit, or who is being mentored, and so lacks the ability to just commit. A more minor problem is that it replaced the version conflict with a $FreeBSD$ conflict that CVSup has to ignore. What it really comes down to is use of tools. Using the magic branch number, it's possible to do what you want, if you are willing to commit the code twice, and maintain two copies, a snapshot and an active CVSup, of the repository, and update the snapshot from the active, when and if your changes are rolled in. And for changes that are refused... you have to generate a patch set from the snapshot, and apply it to the new snapshot, each time you want to update. So you flip between 2/3/2/3/2/3/2 repository copies, to enable you to deal with patch application conflicts, when they arise. Maybe the above changes could be rolled into the rewrite of CVSup in C. That's probably best. It would make changing the code to CVSup onto a local vendor branch much more accessible, and doing that would make it much easier to deal with all these issues: just check in on the local HEAD, and don't worry about it. Submit patches via a cvs diff vs. the HEAD and the vendor branch. Patches generated this way will magically disappear as the vendor branch is updated by CVSup. A bonus would be that, if you were building a product based on FreeBSD and several other projects that supported CVSup, by doing this, you could integrate them all into the same local repository. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
Sergey Babkin wrote: Nate Williams wrote: [ ... CVS cache and cache coherency ... ] Yet another idea is to be able to make local commits with committing them to the central remote repository later. Now I have to use RCS locally for the temporary in-delevopment versions of file. Would be nice to have a kind of a local branch which can be later committed as a whole - in one commit per file, or by duplicating all the intermediate versions with their messages. Not really possible, without CVSup coming onto a vendor branch instead of verbatim copying of the repository. Incoherent: ,---. ,---. | Main | cvsup ---| Cache | | Repo | | Repo | `---' `---' ^ | |cvs co cvs ci | | V | ,---. | | Work | `---| Copy | `---' Coherent: ,---. ,---. | Main | cvsup ---| Cache | | Repo | | Repo | `---' / `---' ^ / (Vendor Branch) |(HEAD) / | ,---. cvs ci ---| Cache | ^ | Repo | | `---' | | |cvs co | | | V | ,---. | | Work | `---| Copy | `---' This also happens to solve the I do all my developement work for my company using -STABLE problem... -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
The corollary to that would be to teach cvs(1) to commit changes to the master repo that have been made to the local repo. Version number sync would be a problem, but it'd be really cool to be able to do that. With a branch mapping layer (RELENG_5_SEANC - HEAD), people could actually get back into the habit of using CVS as a development tool instead of just a way of publishing finalized work. Nate's suggestion covers the version number issue... sort of. It assumes that the patches will be approved for commit to the main repo This is easy to get around, b/c if the commit doesn't happen successfully on the repo, then the commit fails, and as such it also won't also be committed to the local branch (the remote commit failed). and it assumes that the main repo will not get tagged in between. For *most* users, this is not a problem. My solution is for the developer. However, it's not intended to make the local cache a complete mirror of the remote repository. That is a whole 'nother project. :) The main problem with that is pretty obvious, especially around code-freeze/code-slush, but also for anyone without a commit bit, or who is being mentored, and so lacks the ability to just commit. Even during code-freeze, it does allow you to everything you *need* to do safely. If you attempt a commit and your local cache isn't up-to-date, then the commit will fail, and the user will have to re-sychronize their repository. Fortunately, this is a mostly rare occurance, especially if there are regular scheduled synchronization occurances (aka; nightly cron jobs). A more minor problem is that it replaced the version conflict with a $FreeBSD$ conflict that CVSup has to ignore. See above. This is mostly a non-issue as long as the versions are kept up-to-date. No merges will be attempted what-so-ever, even if they would not necessarily cause conflicts. However, this is all a pipe-dream, although Sergey's work is making it less pie-in-the-sky. The other solution to the problem is the P4 route. Making things so darn effecient that there's little need to have a local mirror. Where this falls down is when the remote developer doesn't have a 24x7 connection to the main repository. From what I've been told ClearCase allows for 'mirrored read-only' repositories similar to what most of the open-source CVS developers have been doing with sup/CVSup for years, although it's nowhere near as effecient as CVSup at creating snapshots. Nate To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
RE: making CVS more convenient
Nate Williams wrote: The other solution to the problem is the P4 route. Making things so darn effecient that there's little need to have a local mirror. Where this falls down is when the remote developer doesn't have a 24x7 connection to the main repository. From what I've been told ClearCase allows for 'mirrored read-only' repositories similar to what most of the open-source CVS developers have been doing with sup/CVSup for years, although it's nowhere near as effecient as CVSup at creating snapshots. The current version of Perforce has p4proxy which caches a local copy of the depot files used. To the p4 client, it looks just like the server. The Perforce model makes this a bit easier with a significant amount of client state stored on the server. What is the status of Perforce in the FreeBSD project? Is the issue the absence of a p4up? Licensing? Inertia? Regards, Jan Mikkelsen To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Kernel trace
Yaoping Ruan wrote: Does any one know the implementation of ktrace in FreeBSD? I would like to hack the source code and have a relatively easy way to copy the kernel stack image when a certain of thing happens, such as page fault. It should work like the breakpoints in gdb. But kernel panic is too much trouble for just a single stack image, and kgdb is not simple enough. Which source file(s) I should look at? Look at: http://www.freebsd.org/cgi/cvsweb.cgi/src/usr.bin/ktrace/ktrace.c http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/kern_ktrace.c And from the checkins list on each of these, determine who is maintaining the code. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
RE: making CVS more convenient
The other solution to the problem is the P4 route. Making things so darn effecient that there's little need to have a local mirror. Where this falls down is when the remote developer doesn't have a 24x7 connection to the main repository. From what I've been told ClearCase allows for 'mirrored read-only' repositories similar to what most of the open-source CVS developers have been doing with sup/CVSup for years, although it's nowhere near as effecient as CVSup at creating snapshots. The current version of Perforce has p4proxy which caches a local copy of the depot files used. Does it still require a working net link to the master repository? When it was originally released, I remember it being useful for slow links, but not so good on non-existant links (ie; airplane rides, etc..) What is the status of Perforce in the FreeBSD project? Is the issue the absence of a p4up? Licensing? Inertia? See the archives for a more thorough discussion, but I believe the licensing is the biggest issue. If we moved to use commercial software, it would make our development much more difficult for the average developer to track our progress. Nate To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
RE: making CVS more convenient
Nate Williams wrote: The current version of Perforce has p4proxy which caches a local copy of the depot files used. Does it still require a working net link to the master repository? When it was originally released, I remember it being useful for slow links, but not so good on non-existant links (ie; airplane rides, etc..) Yes, it still requires a working link. Perforce depends on being able to keep its database of client state up to date. What is the status of Perforce in the FreeBSD project? See the archives for a more thorough discussion, but I believe the licensing is the biggest issue. If we moved to use commercial software, it would make our development much more difficult for the average developer to track our progress. I'll take a look. Presumably something like a p4up could get around that. Regards, Jan. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: increasing KVA_PAGES and broken pthreads
Andrew Kinney wrote: Really? I was under the impression that FreeBSD was capable of addressing 8TB of RAM if the hardware supports it. Don't remember which FreeBSD list archive I read that in, but it's not a topic that seems to come up often since most hardware is limited to 4GB of address space. I've got access to hardware that can address 32GB of RAM. Not sure of the exact details of how it works (multiple external memory managers?), but it's a quad Xeon board by SuperMicro. No, it can not access 32G of RAM. It can access a 4G window on 32G of RAM. If it's a question of is there any application that can ever use that much RAM, we're certainly testing the limits here. :-) We're not swapping at all with 4GB, but on several occasions we've gotten close or swapped a few hundred KB. Our two little 2GHz CPUs are humming right along, but most of the time they're better than 60% idle. I imagine that if we pushed the CPUs a bit harder or got hit with a big traffic spike, we'd probably start swapping and want to start thinking about a system that can handle more RAM. Suggest you examine: http://docs.freebsd.org/cgi/getmsg.cgi?fetch=1124056+0+archive/2003/freebsd-current/20030316.freebsd-current Specifically, search for Jake Burkholder. Likely, when this is committed, it will be treated as a different architecture (IMO), rather than Patches to i386. See the recent PC98 It's ISA!/It's not!/Is too!/Is not!/... discussion for details as to why. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: increasing KVA_PAGES and broken pthreads
Mark Tinguely wrote: Intel x86 hardware allows windows of 4GB of virtual memory even if you have the PSE-36 and the PAE extensions with more the 4GB of physical memory. Sound a little like Intel's 64KB windows, but if I remember correctly, the 4GB does not have to be contiguous. It would require a true MMU such as those in the 64 bit architectures (AMD opteron, Intel ia64, sparc64) to simulataneously be able to use more than 4GB of RAM. So far the thought it is better to go with a true 64 MMU than take and get a flat address space than take the performance hit (, plus the memory loss for the much larger page table without much benefit) than try to shuffle in the 4GB windows. Check the thread on this topic in the archives. Jake Burkholder is doin PAE work in the context of FreeBSD. IMO, it should be committed in pae386 instead of i386 (e.g. treated like pc98). I don't know if this is possible, unless the hardware/software VM handling partition is sufficiently opaque (it should be, but there may be some border cases). Other than that, everything you've said is true, I think. I would add that that still leaves you with 4G of space to split between KVA and UVA, so it will not solve Mr. Kinney's problem in the least, since his problem is available kernel address space, not actual physical RAM. The KVA space *could* be split from the UVA space, giving 4G each, rather than 4G total. But for this to happen, you would have to decouple the kernel from dealing with process data that is passed to it from UVA by simply establishing page mappings for it, rather than copying it around. If you did that, it would, I think, drastically increase the amount of copying that happens on reads, writes, etc., and vastly slow down the copyin, copyout, uiomove, and similar functions, as well as making mmap of physical device memory, such as the AGP aperture used by X11 for AGP-enhanced video cards, rather harder to implement (if you consider that to access particular memory from user space may require moving elelments of your 4G window around, in the trap handler for page faults, you will see the problem). Practically, PAE/PSE-36 are not really useful, in that they do not keep you from bumping your head in the kernel or in user space processes, when it comes to available address space, which is *much* more important that the amount of available physical memory. And you can get 4G for each (per the above), without going to PAE/PSE-36. Short version: no matter what you do on a 32 bit architecture, a char * is going to be 32 bits. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
On Sun, Mar 16, 2003 at 04:32:48PM -0700, Nate Williams wrote: What is the status of Perforce in the FreeBSD project? Is the issue the absence of a p4up? Licensing? Inertia? See the archives for a more thorough discussion, but I believe the licensing is the biggest issue. If we moved to use commercial software, it would make our development much more difficult for the average developer to track our progress. I think one only needs to take a look at the Linux community and the situation they have found themselves in wrt to BitKeeper to understand the risks associated with making a project dependent on commercial source control. Even if our license with Perforce were rather liberal, without access to the Perforce source code we are leaving a lot of things to chance. What happens if Perforce folds or discontinues their source control product? Are our bits forever trapped inside a p4 repo which is dependent on binaries which may eventually cease working with our ABI and/or APIs and require a compatibility layer? What if we port to new platforms which Perforce doesn't offer binaries for (or even worse, they've folded and we can no longer get new binaries)? I think we have an opportunity to learn from the mistakes Linux has made here and we'd be foolish not to. It is important to note that CVS and Perforce are nowhere near the only options available in this space. In fact, CVS is not the only open source product out there. I think FreeBSD would be wise to consider a move to Subversion[0] when it reaches release, as it fixes most of the bugs and complaints about CVS while following POLA. svn(1) works pretty much like cvs(1) and that's a Good Thingtm. For a full discussion of the various SCMs available, both open source and proprietary, see Rick Moen's listing of them[1]. [0] - http://subversion.tigris.org/ [1] - http://linuxmafia.com/~rick/linux-info/scm.html Brandon D. Valentine -- [EMAIL PROTECTED] http://www.geekpunk.net Pseudo-Random Googlism: valentine is currently undertaking an esrc funded research project into living on the edge To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
Nate Williams wrote: Nate's suggestion covers the version number issue... sort of. It assumes that the patches will be approved for commit to the main repo This is easy to get around, b/c if the commit doesn't happen successfully on the repo, then the commit fails, and as such it also won't also be committed to the local branch (the remote commit failed). I see how you are viewing this: as a means of avoiding a full CVSup. I think the reason the cache was wanted was not to avoid the CVSup, but to allow operations *other than CVSup* to proceed more quickly? If so, this kind of reduces the reason for having a local cache: attempt locally, and then, if successful, attempt remotely. and it assumes that the main repo will not get tagged in between. For *most* users, this is not a problem. My solution is for the developer. However, it's not intended to make the local cache a complete mirror of the remote repository. That is a whole 'nother project. :) Specifically, it's for the FreeBSD developer, not the developer who uses FreeBSD. 8-). A more minor problem is that it replaced the version conflict with a $FreeBSD$ conflict that CVSup has to ignore. See above. This is mostly a non-issue as long as the versions are kept up-to-date. No merges will be attempted what-so-ever, even if they would not necessarily cause conflicts. I think this is still an issue because of the date, and because of the committer name. Even if the committer name is the same (in your scenario where the FreeBSD developer is the issue, I'll concede it might be, except in the mentor case), the timestamp will still shoot you in the foot. However, this is all a pipe-dream, although Sergey's work is making it less pie-in-the-sky. Yes. As I said: 10 years and counting. 8-). The other solution to the problem is the P4 route. Making things so darn effecient that there's little need to have a local mirror. Where this falls down is when the remote developer doesn't have a 24x7 connection to the main repository. From what I've been told ClearCase allows for 'mirrored read-only' repositories similar to what most of the open-source CVS developers have been doing with sup/CVSup for years, although it's nowhere near as effecient as CVSup at creating snapshots. Yes, P4 solves a *lot* of the problems, except the mirroring in the first place. ClearCase is nice, in its way, but you are right about CVSup being a much better tool for the job; that's why all the people who complain about it continue running it anyway... 8-). -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
Nate's suggestion covers the version number issue... sort of. It assumes that the patches will be approved for commit to the main repo This is easy to get around, b/c if the commit doesn't happen successfully on the repo, then the commit fails, and as such it also won't also be committed to the local branch (the remote commit failed). I see how you are viewing this: as a means of avoiding a full CVSup. I think the reason the cache was wanted was not to avoid the CVSup, but to allow operations *other than CVSup* to proceed more quickly? Having a local 'CVS' tree mirrored already does this. However, it's a hassle since everytime you make a commit, you have to modify the parameters (or use an alias), and after the commit has completed, you have to resynchronize your mirrored tree otherwise your commit will be lost on your first 'cvs update'. If so, this kind of reduces the reason for having a local cache: attempt locally, and then, if successful, attempt remotely. See above. It reduces the 'hassle' factor immensely. and it assumes that the main repo will not get tagged in between. For *most* users, this is not a problem. My solution is for the developer. However, it's not intended to make the local cache a complete mirror of the remote repository. That is a whole 'nother project. :) Specifically, it's for the FreeBSD developer, not the developer who uses FreeBSD. 8-). I wasn't trying to imply that the CVS mods were specifically targeted at FreeBSD users. For what it's worth, I have *numerous* occasions outside of the project where this functionality would have been helpful A more minor problem is that it replaced the version conflict with a $FreeBSD$ conflict that CVSup has to ignore. See above. This is mostly a non-issue as long as the versions are kept up-to-date. No merges will be attempted what-so-ever, even if they would not necessarily cause conflicts. I think this is still an issue because of the date, and because of the committer name. If the files are the same version, the committer name would also be the same in the Id tag, even when it's committed. Even if the committer name is the same (in your scenario where the FreeBSD developer is the issue, I'll concede it might be, except in the mentor case), the timestamp will still shoot you in the foot. CVS doesn't use timestamps. The Id is also created at checkout time, so it's value in the database is mostly irrelevant. The other solution to the problem is the P4 route. Making things so darn effecient that there's little need to have a local mirror. Where this falls down is when the remote developer doesn't have a 24x7 connection to the main repository. From what I've been told ClearCase allows for 'mirrored read-only' repositories similar to what most of the open-source CVS developers have been doing with sup/CVSup for years, although it's nowhere near as effecient as CVSup at creating snapshots. Yes, P4 solves a *lot* of the problems, except the mirroring in the first place. ClearCase is nice, in its way, but you are right about CVSup being a much better tool for the job; that's why all the people who complain about it continue running it anyway... 8-). *grin* Nate To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: jail support for ping, traceroute, etc.. crude hack
On Sun, Mar 16, 2003 at 02:30:36PM -0800, Mooneer Salem wrote: Hello, This patch is interesting. To my understanding though, ipfw uses RAW sockets to communicate with the kernel. Therefore, it might be possible to edit the ipfw table from within the jail, which may be a bad thing. Just a thought. At least in this environment I do not expect to be using IPFW, but this could be something to watch out for in other environments.. When i was looking at this i was somewhat frustated with the way suser() doesn't really allow any sort of a context-of-check to happen easily that i was able to find. ie, was it for a networking check, filesystem, etc.. so my first stab at this ended up with every user being able to do raw ip packets which was bad.. i ended up doing the p-p_prison save hack instead to get the result then applied the prison policy there. Something that i would have no idea where to start on but am interested in doing is also extending the jail support to include IPv6 in addition to the IPv4 sockets but that can cause some issues in an eui-64 environment.. personally i dislike eui-64 as if you change the ethernet card (which i tend to swap out periodically as i need to take a card out of my desktop to match up with dual-nic cards in other machines) it renumbers you. - jared -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Jared Mauch Sent: Sunday, March 16, 2003 1:14 PM To: [EMAIL PROTECTED] Subject: jail support for ping, traceroute, etc.. crude hack so, i am working on building a super-server for me and several friends to collaborate with on the money front to put our machine in a colo location, etc.. and still have good access to networking resources. as a result, i needed to modify the FreeBSD kernel such that it will allow us to use ping, traceroute and other tools. obviously we know there will be some underlying security issues associated but we are sophisticated to understand the nature of these and they are an 'acceptable' situation. my diffs are available at http://puck.nether.net/~jared/fbsd-4.8-rc1-diff-jail-raw_ip.txt and are against the 4.8-rc1 /usr/src/sys tree yeah, they're crude but it gets the desired job done. there is a sysctl to control it, so if its not the desired operation it can be easily tweaked. send me comments. enjoy, - jared -- Jared Mauch | pgp key available via finger from [EMAIL PROTECTED] clue++; | http://puck.nether.net/~jared/ My statements are only mine. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message -- Jared Mauch | pgp key available via finger from [EMAIL PROTECTED] clue++; | http://puck.nether.net/~jared/ My statements are only mine. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
Nate Williams wrote: I see how you are viewing this: as a means of avoiding a full CVSup. I think the reason the cache was wanted was not to avoid the CVSup, but to allow operations *other than CVSup* to proceed more quickly? Having a local 'CVS' tree mirrored already does this. However, it's a hassle since everytime you make a commit, you have to modify the parameters (or use an alias), and after the commit has completed, you have to resynchronize your mirrored tree otherwise your commit will be lost on your first 'cvs update'. Yes. This is the main gripe you are representing here, in a nutshell, I think: I want to CVSup, get on an airplane to the U.S., and be able to work like I normally work, without having to worry about synchronization, because it will happen automatically next time I connect up to the net. In theory, application and theory are the same, but in application, they are not. 8-). If so, this kind of reduces the reason for having a local cache: attempt locally, and then, if successful, attempt remotely. See above. It reduces the 'hassle' factor immensely. I don't think it can. The commits you want to make are to the local (offline) repository copy, and you want them to be reflected back to the online repository, automatically. This really can't happin, unless the order is local, remote. You suggested that the order should be remote, local, which solves a different problem: I want to CVSup and get a local repository copy, make changes to the main repository, and have them reflected in my local copy without a time-consuming CVSup. In other words, I want my local repository treated as a cache with an explicit coherency protocol to save me invalidations on write-through instead of write-back. Making this work is substantially harder, but at least by doing the commit to the remote repository first, and stalling the local commit until after it succeeds, means that you have the remote repository information in hand when you go to do the local commit, so you can either invalidate the local copy if it's older than the repository copy (one file CVSup), or, as happens now, refuse the commit. Maybe the way to handle this *could* be a cache, but it would be a weird animal: you would need to batch pending commits to the remote repository locally, but checkouts from the local repository would be pending until later. This risks an intermediate commit on a file being lost, or a loss of modification history for multiple commits on the same file, but it *could* be made to work, though there would be a real PITA with regard to CVSROOT moving around, unless you used some way to suppress path... The way you would do this is... snapshots! ...TA DA! Snapshot the local CVS repository copy, and then make commits locally. By examining the deltas between the local repository and the snapshot of it, you could replay commits into the remote repository. The problem here is that in the case of a conflict, you would need to discard the new image in favor of the snapshot; you might also run into conflicts, but if the replay mechanism were able to deal with it, it could rewrite history, so that you don't lose modification history, and you don't lose any incremental state (e.g. I add a manifest constant and use it in another file, and then I change the name, and I get a checkin conflict between the two). But as you said before, this is probably wishful thinking: for it to work, you would need to fiddle the dates, which would open some huge holes, I think. For *most* users, this is not a problem. My solution is for the developer. However, it's not intended to make the local cache a complete mirror of the remote repository. That is a whole 'nother project. :) Specifically, it's for the FreeBSD developer, not the developer who uses FreeBSD. 8-). I wasn't trying to imply that the CVS mods were specifically targeted at FreeBSD users. For what it's worth, I have *numerous* occasions outside of the project where this functionality would have been helpful Yep, I expected so; I think everyone does... hence the 8-). A more minor problem is that it replaced the version conflict with a $FreeBSD$ conflict that CVSup has to ignore. See above. This is mostly a non-issue as long as the versions are kept up-to-date. No merges will be attempted what-so-ever, even if they would not necessarily cause conflicts. I think this is still an issue because of the date, and because of the committer name. If the files are the same version, the committer name would also be the same in the Id tag, even when it's committed. Nope. I commit locally as nwilliams, and then I commit on FreeBSD.org as nate. Then I try to update, and the versions match, but the tag data in the file itself doesn't. Even if the committer name is the same (in your
Re: making CVS more convenient
I see how you are viewing this: as a means of avoiding a full CVSup. I think the reason the cache was wanted was not to avoid the CVSup, but to allow operations *other than CVSup* to proceed more quickly? Having a local 'CVS' tree mirrored already does this. However, it's a hassle since everytime you make a commit, you have to modify the parameters (or use an alias), and after the commit has completed, you have to resynchronize your mirrored tree otherwise your commit will be lost on your first 'cvs update'. Yes. This is the main gripe you are representing here, in a nutshell, I think: I want to CVSup, get on an airplane to the U.S., and be able to work like I normally work, without having to worry about synchronization, because it will happen automatically next time I connect up to the net. In theory, application and theory are the same, but in application, they are not. 8-). Sort of. I want to be able to CVSup, get on an airplane, create a bunch of changes (all the while having access to logs, ability to do diffs, etc..), get off an airplane, dialup from my hotel, commit my changes, and have everything *just* work w/out having to re-synchronize my tree. Right now, I can do this, but it requires a lot of steps to get right, and there's room for mistakes being made the entire time. (Accidentally committing changes to the local tree which get lost upon re-synchronization, confusion on the part of the users, requiring additional tools such as CVSup, configuration, etc..) If so, this kind of reduces the reason for having a local cache: attempt locally, and then, if successful, attempt remotely. See above. It reduces the 'hassle' factor immensely. I don't think it can. The commits you want to make are to the local (offline) repository copy, and you want them to be reflected back to the online repository, automatically. See above. I'm not expecting to commit when offline. I'm expecting to commit online, but I don't want to have to setup the CVSupd at the remote end, ensure that anytime we add new modules it's kept up-to-date, ensure that the users don't accidentally commit to the wrong tree, etc.. I want to set things up *once* in CVS, and have it *just* work. If they attempt to commit but don't have a connection to the master, then CVS will fail to allow the commit. If they attempt to commit and their local version does not match the version in the master, it fails. You get the picture. What you described (and I deleted) was a description of what might be nice, but what I think is nearly impossible to do correctly 100% of the time. A more minor problem is that it replaced the version conflict with a $FreeBSD$ conflict that CVSup has to ignore. See above. This is mostly a non-issue as long as the versions are kept up-to-date. No merges will be attempted what-so-ever, even if they would not necessarily cause conflicts. I think this is still an issue because of the date, and because of the committer name. If the files are the same version, the committer name would also be the same in the Id tag, even when it's committed. Nope. I commit locally as nwilliams, and then I commit on FreeBSD.org as nate. Then I try to update, and the versions match, but the tag data in the file itself doesn't. I couldn't commit as 'nwilliams' on the master repository, since the master doesn't allow commits by nwilliams. Again, when commits are done, they are done *remotely* against the master, and then 'mirrored' in the local cache. However, if they master aborts the commit, it simply won't get done locally. Nate To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
Thus spake Terry Lambert [EMAIL PROTECTED]: Nope. I commit locally as nwilliams, and then I commit on FreeBSD.org as nate. Then I try to update, and the versions match, but the tag data in the file itself doesn't. So Terry has actually been writing code for the FreeBSD project all these years and nobody knew it! And here I thought Nate was doing all of his own work. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
David Schultz wrote: Thus spake Terry Lambert [EMAIL PROTECTED]: Nope. I commit locally as nwilliams, and then I commit on FreeBSD.org as nate. Then I try to update, and the versions match, but the tag data in the file itself doesn't. So Terry has actually been writing code for the FreeBSD project all these years and nobody knew it! And here I thought Nate was doing all of his own work. Ar ar. PS: Can you vouch for personally having met everyone with a commit bit? No telling _who_ has commit access to the tree... gotta wonder... -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: jail support for ping, traceroute, etc.. crude hack
Jail is irrelevant if an attacker can access the kernel. It sounds like you're looking for a secure solution that UNIX doesn't even have the capability to implement. The real solution in a BSD environment would be too elaborate for my taste. It would make more sense to me to move away from UNIX ;) D To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
Sean Chittenden [EMAIL PROTECTED] writes: Right, which is what I was trying to suggest a fix for in the first place: the ability to prevent the loss of work committed to a local repository when using cvsup to sync repositories with the master repo. if you *want* to keep the local changes (I thought you didn't because they'd be reflected in the master repo) there is a documented mechanism for committing local changes on a CVS branch which cvsup will ignore. RTFM. DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
Sergey Babkin [EMAIL PROTECTED] writes: A similar thing may be achieved by checking the files out from the local repository and doing any modification command with option -d. But that's troublesome and inconvenient. Read the manual page for the shell you're using, with particular emphasis on the 'alias' builtin command. DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
Sean Chittenden [EMAIL PROTECTED] writes: It'd be cool to teach CVSup to ignore updating certain files that have been marked locally as dirty or in flux until they've been committed through to the master repo. With the -s option, cvsup will not touch files that it believes are in sync until they are updated on the server. DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
It'd be cool to teach CVSup to ignore updating certain files that have been marked locally as dirty or in flux until they've been committed through to the master repo. With the -s option, cvsup will not touch files that it believes are in sync until they are updated on the server. ^^^ not ? -s is a bit dicey to trust unless you grab an exclusive lock on the file and prevent other people from making a change to the file on the server. -sc -- Sean Chittenden To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: making CVS more convenient
Sean Chittenden [EMAIL PROTECTED] writes: With the -s option, cvsup will not touch files that it believes are in sync until they are updated on the server. ^^^ not ? no, not not. cvsup will not touch files that it believes are in sync, the operative word here being believes - with -s cvsup will use the checkout list and won't even look at the actual file unless the server says it has a newer version than what is listed in the checkout list. -s is a bit dicey to trust unless you grab an exclusive lock on the file and prevent other people from making a change to the file on the server. -sc You actually *want* the -s induced failure in this case. DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
fgdg
: unix windows,! To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: fgdg
On Mon, Mar 17, 2003 at 12:39:22PM +0300, sergej wrote: mozno li ustanovit% odnovremenno na odin disk: unix i windows, i kak jto sdelat%! Get yourself a copy of the Complete FreeBSD by Greg Lehey, which covers this subject very well. This question also belongs to freebsd-questions, not -hackers. In short, the configuration option is there with FreeBSD as delivered, but you need to take care on making the right steps at the right time. Starting off with BSD first is the better way to proceed, adding Windows later. Joerg -- Joerg B. MicheelEmail: [EMAIL PROTECTED] NLANR MNA at SDSC/UCSD Page: [EMAIL PROTECTED] The University of Waikato, CompScience Phone: +64 7 8384794 Private Bag 3105Fax: +64 7 8585095 Hamilton, New Zealand Plan: PMA, TINE and the DAG's To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: jail support for ping, traceroute, etc.. crude hack
On Sun, Mar 16, 2003 at 02:30:36PM -0800, Mooneer Salem wrote: When i was looking at this i was somewhat frustated with the way suser() doesn't really allow any sort of a context-of-check to happen easily that i was able to find. ie, was it for a networking check, filesystem, etc.. so my first stab at this ended up with every user being able to do raw ip packets which was bad.. i ended up doing the p-p_prison save hack instead to get the result then applied the prison policy there. It is time to invent ping socket and traceroute socket in addition to tcp, udp, divert so on? To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: jail support for ping, traceroute, etc.. crude hack
On Mon, Mar 17, 2003 at 10:06:27AM +0300, .@babolo.ru wrote: It is time to invent ping socket and traceroute socket in addition to tcp, udp, divert so on? Whilst this might seem nice, actually implementing so that it is both useful and safe is not easy. For a ping socket, this is reasonably easy if all you want is the ability to send ICMP ECHO REQUEST packets and receive any ICMP ECHO REPLY packets associated with previous request packets. It's not totally trivial because the kernel has to keep the state for outgoing packets to ensure that only the correct incoming packets are forwarded. (This is a security issue - you don't want somone finding out hosts someone outside that jail is pinging). Remember to allow for multiple responses to a single request and for long delays. You might also want to implement resource restrictions to prevent someone flooding the system with request packets. A traceroute socket is harder: There's no ICMP TRACEROUTE packet. Instead, traceroute(8) sends outgoing IP packets with varying TTL sizes and monitors incoming ICMP looking for check for HOST UNREACHABLE - TIME EXCEEDED IN TRANSIT packets. Again, the kernel would need to validate the incoming packets against outgoing packets. In both cases, you also need to work out how to handle other random ICMP packets that be received as a result of the outgoing packets. Peter To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message