Re: Odd ACL question
On Sun, 8 Feb 2004, Tim Kientzle wrote: TK>On Sat, 7 Feb 2004, Tim Kientzle wrote: TK>>Joerg Schilling's "star" archives ACLs as follows: TK>> TK>>"user::rwx,group::r--,group:mail:rw-:6,mask::rw-,other::r--" TK>> TK>>Note the "group:mail:rw-:6" entry that contains a fourth TK>>field with the uid/gid number. ... TK>> TK>>Question: Is this a useful extension? TK> TK>Harti Brandt responded: TK>> It definitely is. Joerg and I had several hours of talk on this issue. TK>> If you, for example, restore on a system that usually gets its passwd from TK>> YP or LDAP and you don't have it available ... TK> TK>Ah. That's the example I needed. Now to figure out how to implement TK>such functionality; hacking the acl library functions may TK>not be the best approach, but I'm equally dismayed by the prospect TK>of duplicating the acl library functions in my code. ;-( TK> TK>> As far as I know there are options to star that let you select the exact TK>> behaviour in these cases. TK> TK>This is one difference between 'star' and my work: 'star' offers TK>a great deal of control over the archiving/dearchiving TK>process; my work tries to remove the need for such control TK>by using intelligent algorithms. For example, bsdtar/libarchive TK>doesn't require you to specify the compression when reading archives; TK>it determines it automatically. TK> TK>In this case, I'm considering: TK> * If the username exists, use that. TK> * If the username does not exist and the UID is not already in TK> use, issue a warning and use the UID. TK> * If the username exists and the UID conflicts with the local TK> system, ??? TK> TK>This last case is the tough one. My temptation: map it to TK>an unused UID, issue a warning about the remap, and keep going. That may cause the problem I described. This may leave a file in a user directory that the user cannot delete without intervention of the root user, but its probably the simplest solution. What about non-existing groups? I remember talking with Joerg about this, but cannot remember the outcome of this. You may want to ask him. TK>There are certainly rare cases where manual control is TK>needed. That's why I'm pleased that 'star' is available TK>in ports. ;-) Are you going to replace that horrible thing called GNU tar in the bases system? harti -- harti brandt, http://www.fokus.fraunhofer.de/research/cc/cats/employees/hartmut.brandt/private [EMAIL PROTECTED], [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: bktr video card - VBI
Danny Braniss wrote: > Looking at the driver, i see that somethings changed :-) but > can't find much documentation, (yes i know the source is the force) > > While im studying the driver, some program/documentation would speed up > things. Hi, did you already have a look at the following two applications that use the vbi interface? Perhaps you can find something usefull there... ports/multimedia/nxtvepg ports/misc/alevt Simon pgp0.pgp Description: PGP signature
Re: mostly-reentrant resolver/getaddrinfo(3)
[ Brian: I've noticed that in the recent past, I cannot receive email from you due to reject: RCPT from mx2.freebsd.org[216.136.204.119]: 450 <[EMAIL PROTECTED]>: Sender address rejected: Domain not found Requests for any green.bikeshed.org RRs results in SERVFAIL. ] On Sat, Feb 07, 2004 at 07:01:47PM -0500, Brian F. Feldman wrote: > BTW, a slightly more complete patch that has the diffs for > /usr/include/resolv.h and also should correctly close the sockets that each > thread opens for the resolver can be found here: > > http://green.homeunix.org/~green/mostly_reentrant_resolver.patch Cool! Use pthread_once for creating keys, not a mutex (referring to res_init_mutex/res_keys_inited). I prefer to see `_pthread_*' in libc source (rather than `thr_*'), but that's just me. Cheers, -- Jacques Vidrine NTT/Verio SME FreeBSD UNIX Heimdal [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: bktr video card - VBI
> Hi, > > did you already have a look at the following two applications that use > the vbi interface? > > Perhaps you can find something usefull there... > > ports/multimedia/nxtvepg > ports/misc/alevt thanks, i checked alevt - there was a comment about it in the driver :-) now i know that /dev/vbi is for teletext - not exactly what i was looking for. danny ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Can we extend VolumeGroup after creation?
Hi, Can u guys tel me, i have created a VolumeGroup(myvg) of 120GB size, which holds oraclefiles and these files will be access by 2nodes which are on cluster. So can you please tell that i can increase the size of VolumeGroup or i cannot? Will AIX allows us to increase size of VolumeGroup in the Future? Please Advise. Ahmed _ Marriage? [1]Join BharatMatrimony.com. References 1. http://g.msn.com/8HMAENIN/2728??PS= ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
[call for helpers!] Tuning for the Beaver Challenge
Hi Beaver Challenge 2004 is coming!. Details in http://osuosl.org/benchmarks/bc/ We are preparing the tuning guide. Definitely we need suggestions and comments. Please see this forum to view the latest tuning guide: http://osuosl.org/forums/viewforum.php?f=8 Attached is a ver0.4 of the tuning guide. Regards Patrick == FreeBSD tuning guide for the beaver challenge 2004 For "stock" class Draft ver 0.4 == = Changelog = 0.1 Initial release 0.2 Add some notes about the packages for the benchmarking 0.3 sysctl.conf, loader.conf, more tuning on Apache and other minor updates 0.4 Partition order, 2.2.1(cron), 2.2.3 (sysctl), 2.2.5(flags), 2.2.6, 2.2.7 = 1. Background information = The machine: Dell PowerEdge 2650 2 - 2.8Ghz Intel Xeon processors 2GB of RAM 5 - 36GB U320 disks @ 10,000 RPM configured as RAID0 stripe Adaptec AAC raid controller It's full spec can be downloaded in here: http://www.dell.com/downloads/global/products/pedge/en/2650_specs.pdf From http://osuosl.org/benchmarks/bc/methodology/ .They will run these benchmarks: Bonnie++ - file system performance. lmbench - bandwidth and latency performance. VolanoMark - a Java benchmark. mySQL server throughput Apache server throughput Their testing script is also avaliable on their web page. For Bonnie, there testing parameters is this: my $file_size = "4g"; my $file_num= "128"; my $small_min_size = "16"; my $small_max_size = "2"; my $large_min_size = "2"; my $large_max_size = "10"; my $num_dir = "512"; my $test_runs = 1; my $symlink_runs = 1; my $hardlink_runs = 1; = 2. Tuning section = In this section, we are going to tune the system to provide the best benchmark. 2.1 Installation Partitioning There will be about 180GB harddisk space and 2GB RAM. How are we going to partition the FreeBSD system? Suggested layout be like this (in order) /home >4GB (their bonnie++ script benchmark on this partition, if we do not create /home, then /home will be a symobilc link to /usr/home) / xxGB swap4GB (It have 2GB RAM) /usrxxGB (need some space to hold the ports, and possibly to build the jdk) The /home is the first because we want to use the outside part of the hard disk. (So /home will be a separate partition instead of a slide?) Use UFS1 for FBSD4 (no choice), and UFS2 for FBSD5. Turn on soft updates for all partitions. Some thought: 1) /home can be reformatted (to use different block/fragment size). By looking at their bonnie script, it seems increasing the blocksize (default is 16KB) would benefit. But I do not have actual testing or supporting materials. Unless we have any good reasons, we would stick to the default. 2) In the long run, will those /usr/src, /usr/ports, /usr/obj make the /usr partition becomes more and more fragments? Should we make separate partitions for them? -- 2.2 Tweaking (First stage) -- After the installation, we are going to tune in different areas. 2.2.1 Turn off all unneeded services Remember for "stock" class: The tweaks must reflect a system that could be placed into production. Turn off run these services: inetd portmap sendmail sshd (we use serial console access?) cron usbd? (I doubt it has any USB devices in the BC) Question: Turn these off for the sake of the bechmarking? (which should not be done in a production server) syslog 2.2.2 /etc/fstab By default, the file system will be mounted with atime. Let's change it. For more information, refer to man 8 mount. To do it without reboot # mount -o update,noatime / Then, add 'noatime' to the appropriate place in /etc/fstab. So that it will be done automatically in next reboot. 2.2.3 /etc/sysctl.conf -- This is a very critical file that many tweaking will be done in here. To be completed, refer to Tuning man page TCP man page More parameters to tune in http://sinuspl.net/txt/ (look for sysctl.txt) kern.ipc.somaxconn=8192 kern.ipc.maxsockbuf=2097152 net.inet.ip.portrange.last=3 net.inet.tcp.rfc1323=1 net.inet.tcp.delayed_ack=0 net.inet.tcp.sendspace=32768 net.inet.tcp.recvspace=32768 net.inet.udp.recvspace=32768 net.inet.icmp.icmplim=0 # We will stick to the defaults unless we have any good reasons #net.inet.udp.maxdgram= #net.local.stream.sendspace= #net.local.stream.recvspace= #net.local.dgram.maxdgram= #net.local.dgram.recvspace= # Disable features that we are not going to use in the BC net.inet.icmp.drop_redirect=1 net.inet.icmp.log_redirect=0 net.inet.ip.redirect=0 net.inet6.ip6.redirect=0 net.inet.ip.sourceroute=0 net.inet.ip.accept_sourceroute=0 # Turn off ARP wrong interface log messages net.link.ether.inet.log_arp_wrong_iface=0 Question:
Making inheritance of group ID configurable
New items created on an ufs normally inherit their group ID from
the parent directory. I have the need for making this configurable.
Since the set-group-ID bit is not used for directories on BSD,
I would like to use it to decide on this: If it is set, the group
ID of the newly created item corresponds to the one of the creating
process. (Yeah, this is exactly the wrong way around compared to
what "the Others" do but I don't want to change the BSD default).
I am currently using this patch. It seems to work great but I would
like to know if I have forgotten something ;-)
--- sys/ufs/ufs/ufs_vnops.c.ORI Fri Jan 3 08:41:47 2003
+++ sys/ufs/ufs/ufs_vnops.c Mon Feb 9 16:32:46 2004
@@ -1276,6 +1276,11 @@
if (error)
goto out;
ip = VTOI(tvp);
+#ifdef ANDRE
+ if( (dp->i_mode & ISGID) )
+ ip->i_gid = cnp->cn_cred->cr_gid;
+ else
+#endif
ip->i_gid = dp->i_gid;
#ifdef SUIDDIR
{
@@ -2070,6 +2075,11 @@
if (error)
return (error);
ip = VTOI(tvp);
+#ifdef ANDRE
+ if( (pdir->i_mode & ISGID) )
+ ip->i_gid = cnp->cn_cred->cr_gid;
+ else
+#endif
ip->i_gid = pdir->i_gid;
#ifdef SUIDDIR
{
Any hints are welcome.
Thanks,
-Andre
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: mostly-reentrant resolver/getaddrinfo(3)
"Jacques A. Vidrine" <[EMAIL PROTECTED]> wrote:
> > BTW, a slightly more complete patch that has the diffs for
> > /usr/include/resolv.h and also should correctly close the sockets that each
> > thread opens for the resolver can be found here:
> >
> > http://green.homeunix.org/~green/mostly_reentrant_resolver.patch
>
> Cool!
>
> Use pthread_once for creating keys, not a mutex (referring to
> res_init_mutex/res_keys_inited).
>
> I prefer to see `_pthread_*' in libc source (rather than `thr_*'), but
> that's just me.
If I try to use pthread_once(3) then I still need to have a variable which
stores whether the call to pthread_key_create(3) succeeded from the
pthread_once(3), right? So at the least, I'd have something like:
static once_t keyonce;
static int keyfailed;
static void
key_allocate(void)
{
if (thr_keycreate(foo, bar) != 0)
keyfailed = 1;
}
static void
allocate(void)
{
if (thr_once(&keyonce, key_allocate) != 0 || keyfailed)
return (&_res_bogus);
}
It would be so much nicer if pthread_once(once, func, retval) existed, but I
guess this would probably still work
--
Brian Fundakowski Feldman \'[ FreeBSD ]''\
<> [EMAIL PROTECTED] \ The Power to Serve! \
Opinions expressed are my own. \,,\
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Odd ACL question
In the last episode (Feb 09), Harti Brandt said: > On Sun, 8 Feb 2004, Tim Kientzle wrote: > TK>On Sat, 7 Feb 2004, Tim Kientzle wrote: > TK>>Joerg Schilling's "star" archives ACLs as follows: > TK>> > TK>>"user::rwx,group::r--,group:mail:rw-:6,mask::rw-,other::r--" > TK>> > TK>>Note the "group:mail:rw-:6" entry that contains a fourth > TK>>field with the uid/gid number. ... > TK> > TK> * If the username exists and the UID conflicts with the local > TK> system, ??? > TK> > TK>This last case is the tough one. My temptation: map it to > TK>an unused UID, issue a warning about the remap, and keep going. > > That may cause the problem I described. This may leave a file in a > user directory that the user cannot delete without intervention of > the root user, but its probably the simplest solution. What about > non-existing groups? Any file that a user creates, that user can delete. If you're talking about a root user extracting something into a user's directory, that's different, but you have the same problem even without ACLs. -- Dan Nelson [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Making inheritance of group ID configurable
On Mon, Feb 09, 2004 at 05:10:59PM +0100, Andre Albsmeier wrote: > New items created on an ufs normally inherit their group ID from > the parent directory. I have the need for making this configurable. > > Since the set-group-ID bit is not used for directories on BSD, > I would like to use it to decide on this: If it is set, the group > ID of the newly created item corresponds to the one of the creating > process. (Yeah, this is exactly the wrong way around compared to > what "the Others" do but I don't want to change the BSD default). Would making this a filesystem option be a good idea? You could have the default behaviour be as always and an option that makes the filesystem behave as you've described. David. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Can we extend VolumeGroup after creation?
In the last episode (Feb 08), ahmed mohiuddin said: >Can u guys tel me, i have created a VolumeGroup(myvg) of 120GB >size, which holds oraclefiles and these files will be access by >2nodes which are on cluster. > >So can you please tell that i can increase the size of VolumeGroup >or i cannot? > >Will AIX allows us to increase size of VolumeGroup in the Future? You're asking the wrong people. AIX does let you grow volume graoups and the filesystems mounted on them. Please see http://publib16.boulder.ibm.com/doc_link/en_US/a_doc_lib/aixbman/baseadmn/lvm_maint.htm -- Dan Nelson [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Odd ACL question
On Mon, 9 Feb 2004, Dan Nelson wrote: DN>In the last episode (Feb 09), Harti Brandt said: DN>> On Sun, 8 Feb 2004, Tim Kientzle wrote: DN>> TK>On Sat, 7 Feb 2004, Tim Kientzle wrote: DN>> TK>>Joerg Schilling's "star" archives ACLs as follows: DN>> TK>> DN>> TK>>"user::rwx,group::r--,group:mail:rw-:6,mask::rw-,other::r--" DN>> TK>> DN>> TK>>Note the "group:mail:rw-:6" entry that contains a fourth DN>> TK>>field with the uid/gid number. ... DN>> TK> DN>> TK> * If the username exists and the UID conflicts with the local DN>> TK> system, ??? DN>> TK> DN>> TK>This last case is the tough one. My temptation: map it to DN>> TK>an unused UID, issue a warning about the remap, and keep going. DN>> DN>> That may cause the problem I described. This may leave a file in a DN>> user directory that the user cannot delete without intervention of DN>> the root user, but its probably the simplest solution. What about DN>> non-existing groups? DN> DN>Any file that a user creates, that user can delete. If you're talking DN>about a root user extracting something into a user's directory, that's DN>different, but you have the same problem even without ACLs. Yes, the question was, what to do with a file whose UID does not exist on the system. And, yes, this is about the root user. If you restore a file server for a couple of hundereds or thousands of user you probably don't want to fix undeleteable (by the users) file handish. harti -- harti brandt, http://www.fokus.fraunhofer.de/research/cc/cats/employees/hartmut.brandt/private [EMAIL PROTECTED], [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [call for helpers!] Tuning for the Beaver Challenge
Bascially everything in section 2.2.3 (/etc/sysctl.conf) is wrong; some values may need tweaking, but changing them blindly will not be helpful. The one setting that I do suggest you keep is: kern.ipc.somaxconn=512 (128 may be too low for http testing) The settings from section 2.2.4 will _probably_ be ok, but you'll have to run netstat -m after a benchmarking run to really know. One setting which you'll need to change for the apache2 run is kern.ipc.nsfbufs. Unfortunately, -stable doesn't have any way to tell if you're running out, so you'll have to just guess there. Under -current, netstat -m will tell you what you need to know. Also, you'll probably want to increase KVA_PAGES from 256 to 512 so that the kernel does not run out of memory. Under section 2.2.7, take out the section talking about CPU_FASTER_5X86_FPU through CPU_UPGRADE_HW_CACHE - none of these options is going to help, and it will just increase the likelyhood that something goes wrong. In general, it appears that you can run most of the benchmarks locally, so I suggest that you do that when trying to decide how to tweak settings. For the threads-related benchmarks (volcanomark and apache2 with worker) start asking for help on the -threads mailing list. This is the one thing that is likely to benefit from tweaking. Mike "Silby" Silbersack On Tue, 10 Feb 2004, Dung Patrick wrote: > Hi > > Beaver Challenge 2004 is coming!. > Details in http://osuosl.org/benchmarks/bc/ > > We are preparing the tuning guide. Definitely we need suggestions and comments. > > Please see this forum to view the latest tuning guide: > http://osuosl.org/forums/viewforum.php?f=8 > > Attached is a ver0.4 of the tuning guide. > > Regards > Patrick > > ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [call for helpers!] Tuning for the Beaver Challenge
Hello, The one setting that I do suggest you keep is: kern.ipc.somaxconn=512 (128 may be too low for http testing) In our experience with Apache and clients that do not use Keep Alive (are short lived), 512 is also very low. It causes listen queue overflows and leads to a very low throughput. Another place that we had to look at was the IP sw interrupt queue length. # IP sw interrupt queue len, default 50; # check queue drops stats in net.inet.ip.intr_queue_drops net.inet.ip.intr_queue_maxlen=1000 Hope this helps Aniruddha ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Subversion/CVS experiment summary
This is a bit of a long email, so please skip unless you're into source code revision management :) This is an informal report on the viability of using Subversion to manage the FreeBSD source code repository. Some of this is generic and will be familiar to anyone who has looked at SVN before, some is more FreeBSD-specific. NOTE: I'm not trying to push one SCM over the other or suggest that CVS is wholly inadequate. This is merely the result of an evaluation for my personal use, and I thought I'd post it in case anyone was interested. CVS has been used by the FreeBSD project for a LONG time for good reasons. Despite its shortcomings, I suspect that it will be in use for quite a while longer. - Section the 1st - Motive - My main motivation for these tests was to bring my local modifications to FreeBSD into some semblance of order. It seems I've amassed a bit of a collection of local patches, 3rd party patches, and side projects -- some of which are mutually exclusive or apply to different branches. Simply keeping a working copy with my changes in it works fine for one project but becomes painful when there are several. I'd also like to be able to keep version history for my modifications. I've heard good things about Perforce, and its effortless merge functionality looks really slick. If I'm ever involved with a major commercial coding project, I'll definitely give it some consideration. For my "free-time" projects however it's not really an option. A couple of my local mods are in a bit of a grey area as far as the 'non-commercial' license goes, so I'd rather avoid that whole issue. - Section the 2nd - Setup and conversion - Most of my tests were performed on the src/sys portion of the repository. It seemed to be large enough that I could get a general idea of how well Subversion scales, but small enough that I wouldn't spend all week waiting for the import to complete. All tests were done on a Pentium 4 2.8 GHz system with 512MB RAM. I used a local repository on one disk and the working directories on another (for both CVS and SVN). These tests have been done over the course of the last week and a half, using subversion-0.35.1_1. I've heard of attempts to convert the repo for testing using the cvs2svn.py failing (for more details, see the thread at http://docs.freebsd.org/cgi/getmsg.cgi?fetch=640133+0+archive/2004/freebsd-hackers/20040111.freebsd-hackers). These problems seem to be fixed in the most recent version of the script -- I have been able to successfully import sys, bin, sbin, and lib so far. The next target for testing is contrib as it seems to be the most likely candidate for problems with all those vendor branches. Comments on importing: It's SLOOW. It took 43.9 hours just for src/sys, and this is a relatively speedy system! It starts out at a pretty good pace, but the more commits it processes, the slower each one seems to take. For my purposes I would also need some method of incrementally updating the repository with any new commits made to CVS. This doesn't exist yet, but I'm thinking about trying to hack cvs2svn to do this. Kind of an inverse vendor branch I guess. - Section the 3rd - Head to Head - Yeah, I know comparing Subversion and CVS isn't a fair test -- SVN is designed to be much more than CVS. But it's a comparison that will be inevitably made, so might as well get it out of the way. Bad points (for SVN): * Repo size: The src/sys part of the tree alone is 1.2GB. The same portion of the repo in CVS is only 313MB. I had to keep a script running to routeinly purge unused database logs to avoid running out of disk space during the import. * Working set size: SVN keeps a complete copy of every file that is checked out in a hidden directory analogous to "CVS" directories. This does have some advantages outlined below, but effectively doubles the size of your working directory. * Speed: 0.35 is considerably slower than CVS for some operations. svn checkout is on average about 6 times slower than cvs checkout. Interestingly, CVS seems to benefit from the buffer cache much more than SVN does -- nearly a 50% decrease in execution time for CVS once the cache was populated. Please note however that checking the same thing out over and over isn't a very useful thing to do, and SVN fares better with the more common operations. * Not as thouroughly tested with large repositories. One advantage CVS has is that
Re: Re: [call for helpers!] Tuning for the Beaver Challenge
So you suggested to use the default values for section 2.2.3. I think increasing maxsockbuf, and portrange would be helpful. Without testing, it is hard to choose a value for kern.ipc.nsfbufs. For those optionts CPU_FASTER_5X86_FPU through CPU_UPGRADE_HW_CACHE, it is suggested by an user in the forum. I would like to see the comments from the mailing list. If those options are dangerous, then don't use them. Yes, I can test things locally. But I don't have too much time... Patrick -Original Message- From: Mike Silbersack <[EMAIL PROTECTED]> To: Dung Patrick <[EMAIL PROTECTED]> Date: Mon, 9 Feb 2004 11:12:38 -0600 (CST) Subject: Re: [call for helpers!] Tuning for the Beaver Challenge Bascially everything in section 2.2.3 (/etc/sysctl.conf) is wrong; some values may need tweaking, but changing them blindly will not be helpful. The one setting that I do suggest you keep is: kern.ipc.somaxconn=512 (128 may be too low for http testing) The settings from section 2.2.4 will _probably_ be ok, but you'll have to run netstat -m after a benchmarking run to really know. One setting which you'll need to change for the apache2 run is kern.ipc.nsfbufs. Unfortunately, -stable doesn't have any way to tell if you're running out, so you'll have to just guess there. Under -current, netstat -m will tell you what you need to know. Also, you'll probably want to increase KVA_PAGES from 256 to 512 so that the kernel does not run out of memory. Under section 2.2.7, take out the section talking about CPU_FASTER_5X86_FPU through CPU_UPGRADE_HW_CACHE - none of these options is going to help, and it will just increase the likelyhood that something goes wrong. In general, it appears that you can run most of the benchmarks locally, sodsuggest that you do that when trying to decide how to tweak settings. For the threads-related benchmarks (volcanomark and apache2 with worker) start asking for help on the -threads mailing list. This is the one thing that is likely to benefit from tweaking. Mike "Silby" Silbersack On Tue, 10 Feb 2004, Dung Patrick wrote: > Hi > > Beaver Challenge 2004 is coming!. > Details in http://osuosl.org/benchmarks/bc/ > > We are preparing the tuning guide. Definitely we need suggestions and comments. > > Please see this forum to view the latest tuning guide: > http://osuosl.org/forums/viewforum.php?f=8 > > Attached is a ver0.4 of the tuning guide. > > Regards > Patrick > > ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [call for helpers!] Tuning for the Beaver Challenge
In section 2.2.3, /etc/sysctl.conf: To our experience, it also helps to tune inode cache behavior by setting: vfs.vmiodirenable="0"; (maybe) vfs.nameileafonly="-1" On a server with large memory, if apache 1.x is tested, maybe it is also necessary to increase: vm.max_proc_mmap In section 2.2.4, /boot/loader.conf: Do you need to increase "net.inet.tcp.tcbhashsize" ? - Yaoping Dung Patrick wrote: > Hi > > Beaver Challenge 2004 is coming!. > Details in http://osuosl.org/benchmarks/bc/ > > We are preparing the tuning guide. Definitely we need suggestions and comments. > > Please see this forum to view the latest tuning guide: > http://osuosl.org/forums/viewforum.php?f=8 > > Attached is a ver0.4 of the tuning guide. > > Regards > Patrick > > > Name: bc-fbsd-tune guide.txt >bc-fbsd-tune guide.txtType: Plain Text (text/plain) > Encoding: BASE64 > > > ___ > [EMAIL PROTECTED] mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "[EMAIL PROTECTED]" ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Subversion/CVS experiment summary
On Mon, Feb 09, 2004 at 11:30:05AM -0600, Craig Boston wrote: > This is an informal report on the viability of using Subversion to manage the > FreeBSD source code repository. Some of this is generic and will be familiar > to anyone who has looked at SVN before, some is more FreeBSD-specific. Wow, you have done the experiment I thought to do one of these days! Cool! > - > Section the 1st - Motive > - > My main motivation for these tests was to bring my local modifications to > FreeBSD into some semblance of order. It seems I've amassed a bit of a > collection of local patches, 3rd party patches, and side projects -- some of > which are mutually exclusive or apply to different branches. Simply keeping > a working copy with my changes in it works fine for one project but becomes > painful when there are several. I'd also like to be able to keep version > history for my modifications. That is my primary motivation as well -- sort of a private branch for mods / testing things. > - > Section the 2nd - Setup and conversion > - > I've heard of attempts to convert the repo for testing using the cvs2svn.py > failing (for more details, see the thread at > http://docs.freebsd.org/cgi/getmsg.cgi?fetch=640133+0+archive/2004/freebsd-hackers/20040111.freebsd-hackers). > These problems seem to be fixed in the most recent version of the script -- I > have been able to successfully import sys, bin, sbin, and lib so far. The > next target for testing is contrib as it seems to be the most likely > candidate for problems with all those vendor branches. Did you have to modify the script, or pass unusual options? I'd like to reproduce this, but I didn't get very far when I tried a few days ago with the 0.37.0 version of the tool. I also tried refinecvs (formerly cvs2svn.pl), found at http://lev.serebryakov.spb.ru/refinecvs/ but although it looks like it handles things much better (even vendor branches etc), it loads EVERYTHING into memory -- which means that it eventually grew to 1G of memory/swap at which point my memory was exhausted, and this was at pass 2 of 7... > Comments on importing: It's SLOOW. It took 43.9 hours just for > src/sys, and this is a relatively speedy system! It starts out at a pretty > good pace, but the more commits it processes, the slower each one seems to > take. That doesn't bode well. Is committing in the new SVN repository also slow? > For my purposes I would also need some method of incrementally updating the > repository with any new commits made to CVS. This doesn't exist yet, but I'm > thinking about trying to hack cvs2svn to do this. Kind of an inverse vendor > branch I guess. My thoughts were now going to do something like Tom Lord proposed for an Arch gateway -- just import a CVS working copy into SVN at a certain cut-off date, and setup a bi-directional gateway between the two. That way people can use either tool. The hard problem to solve is indeed getting the changeset from CVS (at least it is when you're not running when a commit is made, as is the case in my setup where I simply cvsup the repository and thus get lots of commits at once). But if the whole repository can be converted it's probably the way to go. > - > Section the 3rd - Head to Head > - Great summary of the pros/cons of either system. > * "Requires" Apache for the network server. There is a simpler CVS-like > network protocol, but it suffers from the same problems with access > control and locking and the like that CVS does. In order to overcome > those limitations, you pretty much have to use Apache/WebDAV. Some may > argue that this isn't really a negative, but it certainly doesn't go with > the K.I.S.S. philosophy. Actually, would a sort of access control wrapper that is now also used with the FreeBSD CVS repository not work? I do agree that it would be nice to have per-directory access control with the svnserve method. > * No cvsup equivalent yet. You can fairly easily use WebDAV to pull a copy > of the trunk or a particular branch, but it's not nearly as efficient as > the rsync algorithm. There's also no way to use WebDAV to grab a certain > date or revision like you can with cvsup -- you have to have the svn > client installed. In order to be even a contender to replace CVS, it > still needs a *FAST* and *SIMPLE* way to synchronize source with an > arbitrary tag or revision. Agreed. > * Still no solution for the repeated merge problem. This is supposed to b
Re: Making inheritance of group ID configurable
On Mon, 09-Feb-2004 at 16:45:11 +, David Malone wrote: > On Mon, Feb 09, 2004 at 05:10:59PM +0100, Andre Albsmeier wrote: > > New items created on an ufs normally inherit their group ID from > > the parent directory. I have the need for making this configurable. > > > > Since the set-group-ID bit is not used for directories on BSD, > > I would like to use it to decide on this: If it is set, the group > > ID of the newly created item corresponds to the one of the creating > > process. (Yeah, this is exactly the wrong way around compared to > > what "the Others" do but I don't want to change the BSD default). > > Would making this a filesystem option be a good idea? You could > have the default behaviour be as always and an option that makes > the filesystem behave as you've described. Possibly. I intentionally didn't suggest this since I wanted to attract people to the technical aspects (and the correctness) of the patch and not to the political opinions :-). -Andre > > David. -- In a world without walls and fences, who needs windows and gates? ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [call for helpers!] Tuning for the Beaver Challenge
Dung Patrick <[EMAIL PROTECTED]> writes:
> For those optionts CPU_FASTER_5X86_FPU through CPU_UPGRADE_HW_CACHE,
> it is suggested by an user in the forum. I would like to see the
> comments from the mailing list. If those options are dangerous, then
> don't use them.
CPU_FASTER_5X86_FPU is not likely to have any positive impact on
performance, and fairly likely to render the system unbootable.
CPU_UPGRADE_HW_CACHE has absolutely no effect on non-PC98 systems.
further comments -
- symlink /usr/{src,obj,ports} to /slow/{src,obj,ports} where /slow
is a filesystem placed on the outside edge of the disk (to free up
space near the spindle for the filesystems actually used by the
benchmark)
- find out what file system the Linux people are using. if they are
using ext2fs or ext3fs, mount your filesystems async.
- most of what you put in sysctl.conf is completely irrelevant. the
rest needs to be tuned according to the actual needs of the
benchmark.
- you should not tune kern.maxfiles etc. unless the benchmark
actually hits those limits. increasing these numbers reduces the
amount of kernel memory available for other purposes.
btw, maxfiles and maxfilesperproc are tunable at run time.
- most of what you put in make.conf is bogus. just use
CPUTYPE ?= pentiumpro
CFLAGS = -O -pipe
COPTFLAGS = -O -pipe
- NOPROFILE has absolutely no impact on performance (except that it
shortens 'make world' a little)
- you *must* use -CURRENT and not 5.2 as the latter has issues with
the aac driver.
- don't use apm or acpi on 4.x.
- regarding jdk 1.4.2, just use the linux version (and make sure to
mount linprocfs). I very much doubt you'll notice a difference in
performance.
- mysql buffer and cache sizes etc. should imho be the same on all
test systems.
- some of the "papers" you reference ([3] and [4]) contain more
incorrect and dangerous information than useful advice.
DES
--
Dag-Erling Smørgrav - [EMAIL PROTECTED]
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Odd ACL question
Harti Brandt wrote: On Sun, 8 Feb 2004, Tim Kientzle wrote: TK>In this case, I'm considering: TK> * If the username exists, use that. TK> * If the username does not exist and the UID is not already in TK> use, issue a warning and use the UID. TK> * If the username exists and the UID conflicts with the local TK> system, ??? TK> TK>This last case is the tough one. My temptation: map it to TK>an unused UID, issue a warning about the remap, and keep going. That may cause the problem I described. This may leave a file in a user directory that the user cannot delete without intervention of the root user, but its probably the simplest solution. This would only happen if you are restoring an archive onto a different system. If it's the same system, there should be no UID conflicts and thus no need to remap the UIDs. I would be very interested in hearing any alternative suggestions. What about non-existing groups? I think I would handle this the same way (for consistency). Are you going to replace that horrible thing called GNU tar in the bases system? Probably, yes. Tim Kientzle ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Subversion/CVS experiment summary
Stijn Hoop <[EMAIL PROTECTED]> writes: > I also tried refinecvs (formerly cvs2svn.pl), found at > > http://lev.serebryakov.spb.ru/refinecvs/ > > but although it looks like it handles things much better (even vendor > branches etc), it loads EVERYTHING into memory -- which means that it > eventually grew to 1G of memory/swap at which point my memory was exhausted, > and this was at pass 2 of 7... Unfortunately there's no good way to avoid this. CVS discards a lot of information about each commit, and in order to reconstruct that information you have to view the repo as a whole. That's not really a problem though, since this is a one-time operation. If / when we decide to switch to SVN, we can easily find a machine with enough RAM to do the job. DES -- Dag-Erling Smørgrav - [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Subversion/CVS experiment summary
On Mon, Feb 09, 2004 at 07:49:55PM +0100, Dag-Erling Sm?rgrav wrote: > Stijn Hoop <[EMAIL PROTECTED]> writes: > > I also tried refinecvs (formerly cvs2svn.pl), found at > > > > http://lev.serebryakov.spb.ru/refinecvs/ > > > > but although it looks like it handles things much better (even vendor > > branches etc), it loads EVERYTHING into memory -- which means that it > > eventually grew to 1G of memory/swap at which point my memory was exhausted, > > and this was at pass 2 of 7... > > Unfortunately there's no good way to avoid this. CVS discards a lot > of information about each commit, and in order to reconstruct that > information you have to view the repo as a whole. > > That's not really a problem though, since this is a one-time > operation. If / when we decide to switch to SVN, we can easily find a > machine with enough RAM to do the job. Very true, but... First of all it would make these kind of tests easier if the script implemented it's own sort of cache function. Then I could try and see the feasibility of converting to Subversion by myself, just like the OP has done. I tried reading the source to see if this is "easily" implemented, but it's still way beyond my meagre perl skills. Second, even after you get the initial conversion done, I think there is a need for resyncing the SVN repository with the CVS repository -- with a state dump from refinecvs this would be relatively easy (only examine the deltas since last time), but this sort of behaviour is impossible with the current script. It would also be useful to implement a bi-directional gateway between SVN and CVS repositories, analogous to the idea someone from the arch project had. See point 3 at http://wiki.gnuarch.org/moin.cgi/Arch_20and_20CVS_20in_20the_20same_20tree (and of course substitute Subversion for arch). That said, as with my comments on Subversion, I was actually pleasantly surprised with my findings about all the tools involved, and the above is certainly only meant as constructive criticism. I think Lev is actually using the FreeBSD repository for testing his script, isn't he? --Stijn -- Tact, n.: The unsaid part of what you're thinking. pgp0.pgp Description: PGP signature
Re: Subversion/CVS experiment summary
cvs-to-perforce scripts use DB files to keep an information on related commits while they scan CVS repo. I didn't try FreeBSD CVS, but whole SGI Linux tree with full history was processed quite effortlessly, without running out of memory. -- Alexander Kabaev ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [call for helpers!] Tuning for the Beaver Challenge
For your consideration, bearing in mind I haven't read all details of the challenge or the systems and therefore may suggest inappropriate, wrong or dangerous things: Custom kernel enabling only the hardware needed, all compiled in (no modules), but be careful with fancy looking options in LINT/NOTES. It may be prudent to hard code interrupts, flags and other parameters to avoid potential suboptimal automatic allocations (eg. all PCI devs on irq 9). Mount all filesystems with async, noatime and *enable softupdates*. Use carefully sized (ie. just big enough) mfs filesystems for any area with dynamically generated data such as temporary files, server scoreboards, etc. /tmp and /var/run, at least. If there's a foo_enable="NO" you put in rc.conf, then do so. Disable as much logging as possible at system and application level, at source or config or by logging to /dev/null. If you have to log, trim I/O to the minimum, maximise buffering and minimise overhead (eg. "-n" for syslog). If any DNS activity is needed, a local cache (eg. dnscache) can help, but so can a nicely populated /etc/hosts. Depending on the tests, it may be a good idea to have the hostname (fully and unqualified) associated with the loopback rather than the (primary) network interface. For the Java benchmarks, try all the implementations which work, including the emulated ones. If the storage is going to be striped over all five disks in hardware by the controller, then notion of putting data "on the outside of the disk" doesn't apply. In fact in this configuration it _may_ make sense to use one big filesystem and leave it to the OS to optimise the filesystem I/O. -Andrew- -- ___ | -Andrew J. Caines- Unix Systems Engineer [EMAIL PROTECTED] | | "They that can give up essential liberty to obtain a little temporary | | safety deserve neither liberty nor safety" - Benjamin Franklin, 1759 | ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Subversion/CVS experiment summary
On Monday 09 February 2004 11:53 am, Stijn Hoop wrote: > Did you have to modify the script, or pass unusual options? I'd like to > reproduce this, but I didn't get very far when I tried a few days ago with > the 0.37.0 version of the tool. No, I used the script as-is. The version I have is LastChangedRevision: 8527, with a date of Jan. 29. It looks like that version is slightly newer than the one included with 0.37 (Rev 8512). One thing that may have made a difference is that so far I've been importing things in chunks rather than trying to do the whole repo at once. > but although it looks like it handles things much better (even vendor > branches etc), it loads EVERYTHING into memory -- which means that it > eventually grew to 1G of memory/swap at which point my memory was > exhausted, and this was at pass 2 of 7... Does the Python version do the same thing? I didn't think to look at memory usage very closely while it was running :-/ > > Comments on importing: It's SLOOW. It took 43.9 hours just for > > src/sys, and this is a relatively speedy system! It starts out at a > > pretty good pace, but the more commits it processes, the slower each one > > seems to take. > > That doesn't bode well. Is committing in the new SVN repository also slow? No, creating a new branch and committing new and changed files to it seems to be just as quick as with an empty repository. I haven't delved into the script enough to know for sure, so this is a wild guess, but I think the speed issue has to do with the script itself. I'm guessing that the method it uses to track the status/branch/etc. of the RCS files is subject to a linear slowdown. I'm going to attempt to verify this by doing a dump / load cycle on the repo that everything was imported into. If it goes quickly then we can assume it's the conversion script. If not, then there are bigger problems... > My thoughts were now going to do something like Tom Lord proposed for an > Arch gateway -- just import a CVS working copy into SVN at a certain > cut-off date, and setup a bi-directional gateway between the two. If I'm reading that right, it sounds similar to a thought I had about just routinely checking out snapshots and committing them on a vendor branch. Of course you'd have to do that separately for each branch you're interested in. IMO, I find it immensely useful to have the entire history of a file at hand. > Actually, would a sort of access control wrapper that is now also used with > the FreeBSD CVS repository not work? I do agree that it would be nice to > have per-directory access control with the svnserve method. Yes, I think the same sort of access hooks (pre-commit?) can be used. The Subversion manual even mentions that, I just forgot about it... That method has always seemed a little... hackish to me. You still need write access (file permissions) to almost the entire repo so it does nothing against editing the files directly -- though with SVN this is a little more difficult as it's all bdb files rather than plain text. Maybe there's a more secure way to do it with a restricted shell that I just don't know about. > > [ repeated merges ] > Which is a shame, this would be a major selling point. On the other hand, > considering the amount of work done and the fact that it really works quite > well already (at least for my small repository) should make people want to > switch :) In the release notes for 0.37 there is a brief blurb about "'svn merge' now notices ancestry by default". I'm not sure exactly what that means or if it's related... > > It also looks as if Subversion 0.37 (aka 1.0-RC) has just been released. > > I'll have to take a look at it and see if any of the problems noted above > > have been resolved. > > Please let me know the results! Will do! My local Subversion server (one running Apache, not the machine I've been doing the tests on) had just finished upgrading the port. I don't think it needs a dump/load cycle but I'm doing one anyway just to be safe... Craig ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Subversion/CVS experiment summary
Alexander Kabaev <[EMAIL PROTECTED]> writes: > cvs-to-perforce scripts use DB files to keep an information on related > commits while they scan CVS repo. I didn't try FreeBSD CVS, but whole > SGI Linux tree with full history was processed quite effortlessly, > without running out of memory. Perforce uses CVS as backing store, so I expect matters are a little different than for SVN. DES -- Dag-Erling Smørgrav - [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [call for helpers!] Tuning for the Beaver Challenge
Andrew J Caines <[EMAIL PROTECTED]> writes: > Mount all filesystems with async, noatime and *enable softupdates*. async and softupdates are mutually exclusive. DES -- Dag-Erling Smørgrav - [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [call for helpers!] Tuning for the Beaver Challenge
Andrew J Caines <[EMAIL PROTECTED]> writes: > If any DNS activity is needed, a local cache (eg. dnscache) can help, but > so can a nicely populated /etc/hosts. Depending on the tests, it may be a > good idea to have the hostname (fully and unqualified) associated with the > loopback rather than the (primary) network interface. that won't make any difference, the host's own IP addresses are automatically routed through lo0. DES -- Dag-Erling Smørgrav - [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Zombie processes no clearing
Usually zombie processes clear when the controlling parent quits but I seem to have a issue with the latest bf1942 dedicated server under FreeBSD where this is not the case. Each map change creates 2 new zombie processes and even quitting the entire server doesn't clear them. Am I missing something obvious? Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone (023) 8024 3137 or return the E.mail to [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Subversion/CVS experiment summary
On Mon, Feb 09, 2004 at 01:26:45PM -0600, Craig Boston wrote: > On Monday 09 February 2004 11:53 am, Stijn Hoop wrote: > > Did you have to modify the script, or pass unusual options? I'd like to > > reproduce this, but I didn't get very far when I tried a few days ago with > > the 0.37.0 version of the tool. > > No, I used the script as-is. The version I have is > LastChangedRevision: 8527, with a date of Jan. 29. It looks like that > version is slightly newer than the one included with 0.37 (Rev 8512). Well, that explains a lot -- for some reason I tested using $LastChangedRevision: 7921 $. I'll try with an up-to-date one then. > One thing that may have made a difference is that so far I've been importing > things in chunks rather than trying to do the whole repo at once. Yes, I was afraid though that commits might have spanned subtrees. But then again, even if they did they would just get committed as separate revisions to the tree, and I suppose one could live with that. > > but although it looks like it handles things much better (even vendor > > branches etc), it loads EVERYTHING into memory -- which means that it > > eventually grew to 1G of memory/swap at which point my memory was > > exhausted, and this was at pass 2 of 7... > > Does the Python version do the same thing? I didn't think to look at memory > usage very closely while it was running :-/ As far as I understood it builds a disk cache instead of using malloc(). This might explain the slowness :) > > My thoughts were now going to do something like Tom Lord proposed for an > > Arch gateway -- just import a CVS working copy into SVN at a certain > > cut-off date, and setup a bi-directional gateway between the two. > > If I'm reading that right, it sounds similar to a thought I had about just > routinely checking out snapshots and committing them on a vendor branch. > Of course you'd have to do that separately for each branch you're interested > in. Yes, that's the idea. You 'just' need a tool that can determine changesets from a CVS repository to automate this. See http://wiki.gnuarch.org/moin.cgi/Arch_20and_20CVS_20in_20the_20same_20tree but substitute Subversion for arch :) > IMO, I find it immensely useful to have the entire history of a file at hand. But you do have all history of a file at hand; you just need to have a separate version system for the older history. Which is admittedly a bit unwieldy, but it certainly makes for a smooth transition in the distributed repository case... > > Actually, would a sort of access control wrapper that is now also used with > > the FreeBSD CVS repository not work? I do agree that it would be nice to > > have per-directory access control with the svnserve method. > > Yes, I think the same sort of access hooks (pre-commit?) can be used. The > Subversion manual even mentions that, I just forgot about it... > > That method has always seemed a little... hackish to me. It is, but it does work. Maybe I'll test and see if I can 'port' those scripts to Subversion :) --Stijn -- My server has more fans than Britney. -- Steve Warwick, from a posting at [EMAIL PROTECTED] pgp0.pgp Description: PGP signature
Re: Subversion/CVS experiment summary
On Mon, 09 Feb 2004 20:43:05 +0100 [EMAIL PROTECTED] (Dag-Erling Sm_rgrav) wrote: > Alexander Kabaev <[EMAIL PROTECTED]> writes: > > cvs-to-perforce scripts use DB files to keep an information on > > related commits while they scan CVS repo. I didn't try FreeBSD CVS, > > but whole SGI Linux tree with full history was processed quite > > effortlessly, without running out of memory. > > Perforce uses CVS as backing store, so I expect matters are a little > different than for SVN. > There are two versions of cvs2p4 script and one of them populates repository using p4 commands. Either way, both versions require full scan of source repository to locate related changes and group them together. So matters are not that different for SVN. -- Alexander Kabaev ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Subversion/CVS experiment summary
According to Craig Boston: > This is an informal report on the viability of using Subversion to manage the > FreeBSD source code repository. Some of this is generic and will be familiar > to anyone who has looked at SVN before, some is more FreeBSD-specific. Thanks for doing this. I tried the same around 0.12/0.14 and it was a complete disaster with svn crashing with an out-of-memory error after taking all of my 512 MB of RAM and the 2 GB of swap... In the meantime, I'm switching to Arch/tla instead of Perforce for my own projects, the distributed nature of arch makes it enormously useful before I have several machines with repos on them and I want to be able to commit and merge across all these repos and only Arch gives me that (well BK does but I don't like the license and is closed source anyway). I've thought about replacing CVS with Arch in the FreeBSD context, even organised a BoF about that at BSDCon '03, but the development is rather different and I'm not sure it could be applied w/o too much pain. On a side remark, I still don't trust svn way of having everything stored in a BDB. Makes repos far too large, generates lots of logs and makes recovery more complicated. -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- [EMAIL PROTECTED] Darwin snuadh.freenix.org Kernel Version 7.2.0: Thu Dec 11 16:20:23 PST 2003 ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Subversion/CVS experiment summary
On Monday 09 February 2004 03:06 pm, Stijn Hoop wrote: > Well, that explains a lot -- for some reason I tested using > $LastChangedRevision: 7921 $. I'll try with an up-to-date one then. I was looking through the change history for cvs2svn.py and it seems that the 0.37 version is almost exactly the same as the 0.35 version. For some reason it looks like they just re-tagged the old version rather than bring in the changes from HEAD... > > One thing that may have made a difference is that so far I've been > > importing things in chunks rather than trying to do the whole repo at > > once. > > Yes, I was afraid though that commits might have spanned subtrees. But then > again, even if they did they would just get committed as separate revisions > to the tree, and I suppose one could live with that. There probably are some commits that do. Only reason I did it like that was to try to trap failure cases more quickly without having to wait for it to get through stage 1 on the whole repo. My plan has always been to go back and try to convert the whole thing when I was sure it would import cleanly and had the resources to do it (the fastest CPU machine I have probably doesn't have enough disk space right now to handle it). > > Does the Python version do the same thing? I didn't think to look at > > memory usage very closely while it was running :-/ > > As far as I understood it builds a disk cache instead of using malloc(). > This might explain the slowness :) Ok, that's consistent with what I saw here. It looked like it created several large temporary bdb databases, but I don't remember any excessive swapping going on. > Yes, that's the idea. You 'just' need a tool that can determine changesets > from a CVS repository to automate this. See > > http://wiki.gnuarch.org/moin.cgi/Arch_20and_20CVS_20in_20the_20same_20tree > > but substitute Subversion for arch :) Makes sense. I believe you mentioned earlier that post-commit hooks could be used for this? But that of course requires assistance from the repomaster. It might also be possible to rig up a script to monitor the cvs-all mailing list and get its changesets from there... > It is, but it does work. Maybe I'll test and see if I can 'port' those > scripts to Subversion :) Yes, it does work as long as your users are relatively trusted and you keep good backups :). Still, it would probably be the most painless transition path to use that over ssh. In regards to the speed test: ARGH! svn dump died on me with this message: * Dumped revision 18576. * Dumped revision 18577. * Dumped revision 18578. * Dumped revision 18579. * Dumped revision 18580. svn: Invalid change ordering: non-add change on deleted path If it's really invalid I wonder how it ended up in the repo in the first place. Not good. I'll have to do some digging to find out what causes that. Craig ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [call for helpers!] Tuning for the Beaver Challenge
On Mon, Feb 09, 2004 at 07:27:08PM +0100 I heard the voice of Dag-Erling Sm?rgrav, and lo! it spake thus: > > CPU_FASTER_5X86_FPU is not likely to have any positive impact on > performance, and fairly likely to render the system unbootable. I would guess just from the name that this (and some similarly named options) apply only to Cyrix 5x86 processors. Somehow, I don't think you'll run into too many of them in benchmarks these days. Just a hunch. >CPUTYPE ?= pentiumpro I recall a thread somewhere recently about pentiumpro being decidedly suboptimal for some new CPUs. Although, on 4.x with the older version of gcc, it may not matter. -- Matthew Fuller (MF4839) | [EMAIL PROTECTED] Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ "The only reason I'm burning my candle at both ends, is because I haven't figured out how to light the middle yet" ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: mostly-reentrant resolver/getaddrinfo(3)
Alright, here we go! I simplified some things out a bit and used
pthread_once(3) to make things look a little cleaner. The RES_BOGUS flag
was unnecessary, and now single-threaded programs and the first thread of
multi-threaded programs do not incur the allocation of per-thread resolver
storage. I also allocated all the per-thread storage at once because the
user is not privy to that, anyway. I've gotten good feedback so far, and
the latest round of changes at the least "works for me" :) Try it in your
multi-tab Mozilla!
Index: include/resolv.h
===
RCS file: /usr/ncvs/src/include/resolv.h,v
retrieving revision 1.23
diff -u -r1.23 resolv.h
--- include/resolv.h7 Dec 2003 12:32:23 - 1.23
+++ include/resolv.h10 Feb 2004 00:55:35 -
@@ -200,7 +200,12 @@
char * humanname; /* Its fun name, like "mail exchanger" */
};
-extern struct __res_state _res;
+__BEGIN_DECLS
+extern struct __res_state *___res(void);
+extern struct __res_state_ext *___res_ext(void);
+__END_DECLS
+#define_res(*___res())
+#define_res_ext(*___res_ext())
/* for INET6 */
extern struct __res_state_ext _res_ext;
Index: lib/libc/include/reentrant.h
===
RCS file: /usr/ncvs/src/lib/libc/include/reentrant.h,v
retrieving revision 1.2
diff -u -r1.2 reentrant.h
--- lib/libc/include/reentrant.h1 Nov 2002 09:37:17 - 1.2
+++ lib/libc/include/reentrant.h10 Feb 2004 01:11:45 -
@@ -94,10 +94,12 @@
#define mutex_tpthread_mutex_t
#define cond_t pthread_cond_t
#define rwlock_t pthread_rwlock_t
+#define once_t pthread_once_t
#define thread_key_t pthread_key_t
#define MUTEX_INITIALIZER PTHREAD_MUTEX_INITIALIZER
#define RWLOCK_INITIALIZER PTHREAD_RWLOCK_INITIALIZER
+#define ONCE_INITIALIZER PTHREAD_ONCE_INIT
#define mutex_init(m, a) _pthread_mutex_init(m, a)
#define mutex_lock(m) if (__isthreaded) \
@@ -127,6 +129,7 @@
#define thr_getspecific(k) _pthread_getspecific(k)
#define thr_sigsetmask(f, n, o)_pthread_sigmask(f, n, o)
+#define thr_once(o, i) _pthread_once(o, i)
#define thr_self() _pthread_self()
#define thr_exit(x)_pthread_exit(x)
#define thr_main() _pthread_main_np()
Index: lib/libc/net/getaddrinfo.c
===
RCS file: /usr/ncvs/src/lib/libc/net/getaddrinfo.c,v
retrieving revision 1.48
diff -u -r1.48 getaddrinfo.c
--- lib/libc/net/getaddrinfo.c 30 Oct 2003 17:36:53 - 1.48
+++ lib/libc/net/getaddrinfo.c 6 Feb 2004 06:20:23 -
@@ -1511,6 +1511,7 @@
return 0;
}
+ THREAD_UNLOCK();
switch (_nsdispatch(&result, dtab, NSDB_HOSTS, "getaddrinfo",
default_dns_files, hostname, pai)) {
case NS_TRYAGAIN:
@@ -1524,20 +1525,20 @@
goto free;
case NS_SUCCESS:
error = 0;
+ THREAD_LOCK();
for (cur = result; cur; cur = cur->ai_next) {
GET_PORT(cur, servname);
/* canonname should be filled already */
}
+ THREAD_UNLOCK();
break;
}
- THREAD_UNLOCK();
*res = result;
return 0;
free:
- THREAD_UNLOCK();
if (result)
freeaddrinfo(result);
return error;
@@ -2037,6 +2038,7 @@
memset(&sentinel, 0, sizeof(sentinel));
cur = &sentinel;
+ THREAD_LOCK();
_sethtent();
while ((p = _gethtent(name, pai)) != NULL) {
cur->ai_next = p;
@@ -2044,6 +2046,7 @@
cur = cur->ai_next;
}
_endhtent();
+ THREAD_UNLOCK();
*((struct addrinfo **)rv) = sentinel.ai_next;
if (sentinel.ai_next == NULL)
@@ -2152,9 +2155,12 @@
memset(&sentinel, 0, sizeof(sentinel));
cur = &sentinel;
+ THREAD_LOCK();
if (!__ypdomain) {
- if (_yp_check(&__ypdomain) == 0)
+ if (_yp_check(&__ypdomain) == 0) {
+ THREAD_UNLOCK();
return NS_UNAVAIL;
+ }
}
if (__ypcurrent)
free(__ypcurrent);
@@ -2189,6 +2195,7 @@
cur = cur->ai_next;
}
}
+ THREAD_UNLOCK();
if (sentinel.ai_next == NULL) {
h_errno = HOST_NOT_FOUND;
Index: lib/libc/net/res_init.c
===
RCS file: /usr/ncvs/src/lib/libc/net/res_init.c,v
retrieving revision 1.31
diff -u -r1.31 res_init.c
--- lib/libc/net/res_init.c 7 Dec 2003 12:32:24 - 1.31
+++ lib/libc/net
kernel load address
Hi all,
When reading locore.s, I have the following question.
In the locore.s, there's a macro R defined as
#define R(foo) ((foo)-KERNBASE)
where KERNBASE equals to 0xC000.
But in the sys/conf/ldscript.i386, it reads
SECTIONS
{
. = kernbase + 0x0010 + SIZEOF_HEADERS;
.
}
The kernel is loaded at 0x0010 by boot2 or loader with program
header stripped. So, how the R macro can still work?
I think it should be defined as `((foo)-KERNBASE-SIZEOF_HEADERS)' or
the ldscript.i386 should be modified to
. = kernbase + 0x0010;
I refers to the ld script for Linux i386 kernel(vmlinux.lds), it reads
SECTIONS
{
. = 0xC000 + 0x10;
..
}
Thanks.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: sources needed - msdosfs merge from darwin
On Thu, Feb 05, 2004, Friedemann Becker wrote: > Hello, > > I want to take a look at the darwin msdosfs - merge is on the todo list. > I had trouble logging in to the cvs and wrote a mail to the > apple-support. I got an auto-answer containing: > > >Q: I've registered, but I can't login to CVS or the web site. What's > >wrong? > >A: We are currently experiencing intermittent failures during the login > >process. We are working to resolve this as quickly as possible. Please > >keep trying and we apologize for the inconvenience. > > So my question is, has someone a copy of the msdosfs part of the darwin > kernel, so i could use it while waiting for the cvs to come up again... > > And: has anyone begun to import the sources in question already? please > drop a short note, so i know what is already done. You can try the sources on opendarwin.org. However, it is my understanding that using Apple's msdosfs improvements presents licensing problems due to the viral nature of the APSL. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [call for helpers!] Tuning for the Beaver Challenge
This might also be an opportune time to test the benefits of compilers other than gcc, such as Intel's reputedly super-optimised-vs-gcc C/C++ compiler (lang/icc). I've not tried icc or any of the other non-gcc compilers and I'm not suggesting it will help, or even what one might try to compile, but maybe some others can offer their experiences? -Andrew- -- ___ | -Andrew J. Caines- Unix Systems Engineer [EMAIL PROTECTED] | | "They that can give up essential liberty to obtain a little temporary | | safety deserve neither liberty nor safety" - Benjamin Franklin, 1759 | ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [call for helpers!] Tuning for the Beaver Challenge
"Matthew D. Fuller" <[EMAIL PROTECTED]> writes: > >CPUTYPE ?= pentiumpro > I recall a thread somewhere recently about pentiumpro being decidedly > suboptimal for some new CPUs. Although, on 4.x with the older version > of gcc, it may not matter. I believe the reverse is true - pentiumpro proved to be superior to p3 and p4 even on p3s and p4s. DES -- Dag-Erling Smørgrav - [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
