Re: Need some advice regarding portable user IDs
On Tue, 17 Aug 1999, Brian C. Grayson wrote: > On Tue, Aug 17, 1999 at 07:17:45PM -0700, Wilfredo Sanchez wrote: > > A group of us at Apple are trying to figure out how to handle > > situations where a filesystem with "foreign" user ID's are present. > > Have you looked at mount_umap(8)? I (naively) think it would > solve most of your concerns. I don't think so. umap is for translating credentials between domains. I think what Fred wants to do is different, and that is to ignore the credentials on the system. Fred, right now what happens in MacOS when I take a disk which has sharing credentials set up, and hook it into another machine? How are the credentials handled there? Also, one of the problems which has been brought up in the thread is that umap needs to know what credentials to translate to. For that, we'd need to stash the credentails on the drive. Take care, Bill To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: Need some advice regarding portable user IDs
Wilfredo Sanchez wrote: > > A group of us at Apple are trying to figure out how to handle > situations where a filesystem with "foreign" user ID's are present. > The basic problem is that the user experience using Unix semantics > are not really pleasant. I think some examples would help: > > I'm working with Joe on a project, and I have some sources I want > him to take a look at, so I mount a floppy disk. Well, that's a bad > example, because floppies are "out"... So I mount a zip disk with UFS > on it, and I copy my source tree onto it, and hand this to Joe. Joe > takes the disk home, and sticks it in his computer, and he finds > that he can't read the files, because I have a lamer umask, and as a > bonus, I don't have an account on his machine, so the files are owned > by some random UID. Having run into this problem myself, both with tar'ed archives and zip disks I came up with the following dream world one day. In my world there are two magic uid's, one for a "read only" user and one for a "read write" user. If I give Joe a zip disk with my files on it all owned by my uid, by default when he puts it in his drive at his workstation it comes up with all files owned by the read only user. He can read the files, copy them to his work station, etc. but he can't make changes to the existing files. If I want to let Joe change the files on my disk I can set the rw flag, so when he pops it in his drive it comes up owned by the "read write" user. (Still assuming that Joe and I have mismatched uid's.) In a system like this I'd also like to see a flag that lets the owner require that the uid semantics are enforced. Also, the flags should be settable on the file, directory and disk levels. Finally in an ultra-paranoid environment you could set an option that requires that the uid matchup is for the actual person, similar to the way an ssh public key/private key pair works. Hope this is useful, Doug To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: Need some advice regarding portable user IDs
Wilfredo Sanchez wrote: > > A group of us at Apple are trying to figure out how to handle > situations where a filesystem with "foreign" user ID's are present. > The basic problem is that the user experience using Unix semantics > are not really pleasant. I think some examples would help: > > I'm working with Joe on a project, and I have some sources I want > him to take a look at, so I mount a floppy disk. Well, that's a bad > example, because floppies are "out"... So I mount a zip disk with UFS > on it, and I copy my source tree onto it, and hand this to Joe. Joe > takes the disk home, and sticks it in his computer, and he finds > that he can't read the files, because I have a lamer umask, and as a > bonus, I don't have an account on his machine, so the files are owned > by some random UID. Having run into this problem myself, both with tar'ed archives and zip disks I came up with the following dream world one day. In my world there are two magic uid's, one for a "read only" user and one for a "read write" user. If I give Joe a zip disk with my files on it all owned by my uid, by default when he puts it in his drive at his workstation it comes up with all files owned by the read only user. He can read the files, copy them to his work station, etc. but he can't make changes to the existing files. If I want to let Joe change the files on my disk I can set the rw flag, so when he pops it in his drive it comes up owned by the "read write" user. (Still assuming that Joe and I have mismatched uid's.) In a system like this I'd also like to see a flag that lets the owner require that the uid semantics are enforced. Also, the flags should be settable on the file, directory and disk levels. Finally in an ultra-paranoid environment you could set an option that requires that the uid matchup is for the actual person, similar to the way an ssh public key/private key pair works. Hope this is useful, Doug To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
In message <[EMAIL PROTECTED]>, Terry Lambert writes: >> >You would have to de-collapse several VOP lists that have been >> >pre-collapsed. >> >> You are talking gibberish here. Please show code where this is >> a problem. > >When you write a proxy stacking layer, such as John Heidemann's >network proxy stacking layer (an NFS alternative), VOP's which >would normally be handled by vfs_default have to be handled on >the other end of the proxy, instead, in the same way that they >would be handled by the vfs_default stuff. And what prevents you from taking over the default op ? -- Poul-Henning Kamp FreeBSD coreteam member [EMAIL PROTECTED] "Real hackers run -current on their laptop." FreeBSD -- It will take a long time before progress goes too far! To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Need some advice regarding portable user IDs
On Wed, 18 Aug 1999, Marc Ramirez wrote: > Oh! I was under the impression that it just didn't work, even with > correct perms, but I use FreeBSD. Lemme try it... Can't mount, even > with 0666 on /dev/fd0. Maybe I'm being stupid. Wouldn't be the first > time! It's controlled by a sysctl in FreeBSD which defaults to "off" - I forget what it is called. Kris To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
yp_mkdb
I am in the process of upgrading a NIS master using version 2.1.0 to version 3.2. The 'Makefile' has been customized to include automount maps for our IRIX machines as was the 'Makefile' in the old NIS Master. The problem is that for some reason the program 'yp_mkdb' in 3.2 is much more picky. It does not tolerate lines as such: nschein -rw,intrnister:/usr/home:& It complains when it encounters the '-' if removed it works fine. Also it has problems with the '+' and absence of a whitespace following the first ASCI string in the following line: +auto.home What has changed in yp_mkdb? Is there a way to escape certain symbols? I have already tried \ '' "". Or is there another way to make the database file? Nathaniel Schein To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
yp_mkdb
I am in the process of upgrading a NIS master using version 2.1.0 to version 3.2. The 'Makefile' has been customized to include automount maps for our IRIX machines as was the 'Makefile' in the old NIS Master. The problem is that for some reason the program 'yp_mkdb' in 3.2 is much more picky. It does not tolerate lines as such: nschein -rw,intrnister:/usr/home:& It complains when it encounters the '-' if removed it works fine. Also it has problems with the '+' and absence of a whitespace following the first ASCI string in the following line: +auto.home What has changed in yp_mkdb? Is there a way to escape certain symbols? I have already tried \ '' "". Or is there another way to make the database file? Nathaniel Schein To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-questions" in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Gigabit ethernet support?
Any supported cards in 3.2.x? The HCL pages don't list any:( Thanks, --- David To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: cvs commit: src/bin/test test.c
On Wed, Aug 18, 1999, Sheldon Hearn wrote: > > green 1999/08/17 17:18:53 PDT > > > > Modified files: > > bin/test test.c > > Log: > > The new test(1) did not use access() correctly. I don't know why, since > > supposedly it's ksh-derived, and it's not broken in pdksh. I've added > > a test for test running as root: if testing for -x, the file must be > > mode & 0111 to get "success", rather than just existant. > > > > Reviewed by: chris > > What were you actually trying to fix, here? I didn't see any discussion > of this on hackers, current or bugs, nor in response to my initial > commit message. He was "fixing" (though, as Bruce pointed out, it wasn't a valid fix) test -x. Apparently, access(2) will return 'success' on access(file, X_OK) if called by a program run by root. The patch partly solves the problem, but the euid-vs-ruid problem remains. -- |Chris Costello |Disc space, the final frontier! `-- To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
On Wed, 18 Aug 1999, Poul-Henning Kamp wrote: > Yes, but we need subsecond in the filesystems. Think about make(1) on > a blinding fast machine... Oh yes, I realize that. :-) It's just that I thought you were at one point suggesting having 128 bits to the left of the decimal point (128 bits worth of seconds). I was trying to say that'd be a bit much. :-) Take care, Bill To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
In message <199908181737.laa03...@mt.sri.com>, Nate Williams writes: >> > Both struct timespec and struct timeval are major mistakes, they >> > make arithmetic on timestamps an expensive operation. Timestamps >> > should be stored as integers using an fix-point notations, for >> > instance 64bits with 32bit fractional seconds (the NTP timestamp), >> > or in the future 128/48. >... >> >> > Extending from 64 to 128bits would be a cheap shift and increased >> > precision and range could go hand in hand. >> >> I doubt we need more than 64 bit times. 2^63 seconds works out to >> 292,279,025,208 years, or 292 (american) billion years. > >I think Poul's point is that in the future seconds is probably way too >coarse grained. Computer's are getting faster all the time, and in the >future we may need 64 seconds, plus an additional 64 bits for the >fractions of a second, which will be necessary for accurate timekeeping. No, 64bits of fractions will not be needed, at least until we start using FreeBSD as embedded computer in Heisenbergcompensators. I recall somebody saying that 100GHz was the highest realistic (or lowest unrealistic) clock frequency using digital logic, the argument was pretty convincing physically: light speed sets a size limit, that prescripes some voltage gradients which in turn produces EMC which in turn makes sure nothing works. Also various tunnel effects, and the general heisenberisms took their toll. State of the art time interval measuring equipment is into the "a few picosecond" territory (http://www.timing.com/). Based on that I would say that 40 to 48 bits will be OK for the fraction. As a sidebar: I had a kernel running which used 32i.32f timestamps and converted to timeval & timespec as needed and it actually made a lot of code look a lot more sane. I may go back and do it some day. -- Poul-Henning Kamp FreeBSD coreteam member p...@freebsd.org "Real hackers run -current on their laptop." FreeBSD -- It will take a long time before progress goes too far! To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
> >> > > I'm not familiar with the VFS_default stuff. All the vop_default_desc > >> > > routines in NetBSD point to error routines. > >> > > >> > In FreeBSD, they now point to default routines that are *not* error > >> > routines. This is the problem. I admit the change was very well > >> > intentioned, since it made the code a hell of a lot more readable, > >> > but choosing between readable and additional function, I take function > >> > over form (I think the way I would have "fixed" the readability is by > >> > making the operations that result in the descriptor set for a mounted > >> > FS instance be both discrete, and named for their specific function). > >> > >> As I recall most of FBSD's default routines are also error routines, if > >> the exceptions were a problem it would would be trivial to fix. > > > >You would have to de-collapse several VOP lists that have been > >pre-collapsed. > > You are talking gibberish here. Please show code where this is > a problem. When you write a proxy stacking layer, such as John Heidemann's network proxy stacking layer (an NFS alternative), VOP's which would normally be handled by vfs_default have to be handled on the other end of the proxy, instead, in the same way that they would be handled by the vfs_default stuff. Some VOP's, like advisory locking, need both local assertion and remote proxy of the VOP to avoid introducing race windows. The result of this is that, if you rely on the vfs_default stuff, then you can't proxy those VOP's into a different address space, either on another machine, or to a user space VFS stacking layer developement environment. This is the same problem that embedding VM references directly into any FS causes, and that vm_object_t aliases would exacerbate. John has, in the past, sent me a number of stacking layers done by various people, with the requirement that I not redistribute them, as they are not what he would consider to be properly representative of finished work. Since John himself did the network proxy, you could perhaps get him to send you a copy, so you could have direct access to code where this was a problem. Make sure that the system you are talking to over the proxy is not assumed to be a FreeBSD system (e.g. don't assume that the vfs_default stuff exists on the other end of the proxy, or that it would be functional). Terry Lambert te...@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
On Wed, 18 Aug 1999, Terry Lambert wrote: > > Right. That exported struct lock * makes locking down to the lowest-level > > file easy - you just feed it to the lock manager, and you're locking the > > same lock the lowest level fs uses. You then lock all vnodes stacked over > > this one at the same time. Otherwise, you just call VOP_LOCK below and > > then lock yourself. > > I think this defeats the purpose of the stacking architecture; I > think that if you look at an unadulterated NULLFS, you'll see what I > mean. Please be more precise. I have looked at an unadulterated NULLFS, and found it lacking. I don't see how this change breaks stacking. > Intermediate FS's should not trap VOP's that are not applicable > to them. True. But VOP_LOCK is applicable to layered fs's. :-) > One of the purposes of doing a VOP_LOCK on intermediate vnodes > that aren't backing objects is to deal with the global vnode > pool management. I'd really like FS's to own their vnode pools, > but even without that, you don't need the locking, since you > only need to flush data on vnodes that are backing objects. > > If we look at a stack of FS's with intermediate exposure into the > namespace, then it's clear that the issue is really only applicable > to objects that act as a backing store: > > > ---- > FSExposed in hierarchyBacking object > ---- > top yes no > intermediate_1no no > intermediate_2no yes > intermediate_3yes no > bottomno yes > ---- > > So when we lock "top", we only lock in intermediate_2 and in bottom. No. One of the things Heidemann notes in his dissertation is that to prevent deadlock, you have to lock the whole stack of vnodes at once, not bit by bit. i.e. there is one lock for the whole thing. > > Actually isn't the only problem when you have vnode fan-in (union FS)? > > i.e. a plain compressing layer should not introduce vnode locking > > problems. > > If it's a block compression layer, it will. Also a translation layer; > consider a pure Unicode system that wants to remotely mount an FS > from a legacy system. To do this, it needs to expand the pages from > the legacy system [only it can, since the legacy system doesn't know > about Unicode] in a 2:1 ratio. Now consider doing a byte-range lock > on a file on such a system. To propogate the lock, you have to do > an arithmetic conversion at the translation layer. This gets worse > if the lower end FS is exposed in the namespace as well. Wait. byte-range locking is different from vnode locking. I've been talking about vnode locking, which is different from the byte-range locking you're discussing above. > > Nope. The problem is that while stacking (null, umap, and overlay fs's) > > work, we don't have the coherency issues worked out so that upper layers > > can cache data. i.e. so that the lower fs knows it has to ask the uper > > layers to give pages back. :-) But multiple ls -lR's work fine. :-) > > With UVM in NetBSD, this is (supposedly) not an issue. UBC. UVM is a new memory manager. UBC unifies the buffer cache with the VM system. > You could actually think of it this way, as well: only FS's that > contain vnodes that provide backing should implement VOP_GETPAGES > and VOP_PUTPAGES, and all I/O should be done through paging. Right. That's part of UBC. :-) Take care, Bill To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
In message <199908181848.laa14...@usr02.primenet.com>, Terry Lambert writes: >> >You would have to de-collapse several VOP lists that have been >> >pre-collapsed. >> >> You are talking gibberish here. Please show code where this is >> a problem. > >When you write a proxy stacking layer, such as John Heidemann's >network proxy stacking layer (an NFS alternative), VOP's which >would normally be handled by vfs_default have to be handled on >the other end of the proxy, instead, in the same way that they >would be handled by the vfs_default stuff. And what prevents you from taking over the default op ? -- Poul-Henning Kamp FreeBSD coreteam member p...@freebsd.org "Real hackers run -current on their laptop." FreeBSD -- It will take a long time before progress goes too far! To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
The discussions between Kirk and matt over a glass of beer/drink at kirk's party at USENIX and at the Bay area User's group. On Wed, 18 Aug 1999, Nate Williams wrote: > > > Matt doesn't represent the FreeBSD project, and even if he rewrites > > > the VFS subsystem so he can understand it, his rewrite would face > > > considerable resistance on its way into FreeBSD. I don't think > > > there is reason to rewrite it, but there certainly are areas > > > that need fixing. > > > > You are misinformed as far as I know.. From discussions I saw, th > > main architect of a VFS rewrite would be Kirk, and Matt would be acting as > > Kirk's right-hand-man. > > Which discussions are these? Are they archived somewhere? > > > Nate > To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
:On Wed, 18 Aug 1999, Poul-Henning Kamp wrote: : :> Matt doesn't represent the FreeBSD project, and even if he rewrites :> the VFS subsystem so he can understand it, his rewrite would face :> considerable resistance on its way into FreeBSD. I don't think :> there is reason to rewrite it, but there certainly are areas :> that need fixing. : :You are misinformed as far as I know.. From discussions I saw, th :main architect of a VFS rewrite would be Kirk, and Matt would be acting as :Kirk's right-hand-man. Yes, this is correct. Kirk is going to be the main architect. I have been heavily involved and will continue to be. :> >> The use of the "vfs_default" to make unimplemented VOP's : :> I beg to differ. The only difference is that we pass through :> multiple layers before we hit the bottom of the stack. There is :... :Well I believe that Kirk considers them misguided too, but he stated that :he wasn't going to remove them without serious thought about the alternatives. The vfs op callout layering has not been on the radar screen. There are much too many other more serious problems. I really doubt that any changes will be made to this piece any time in the next year or even two, if at all. The main items on the radar screen are related to buffer management (struct buf stuff. For example, preventing VM blockages due to pages being wired by write I/O's), VFS locking and reference count issues (for example, namei lookups, blockages in the pager and syncer due to vnode locks held by blocked processes, etc...), and interactions between VFS and VM (for example: moving away from VOP_READ/VOP_WRITE and moving more towards a getpages/putpages model). None of the items have been set in stone yet. We're waiting for Kirk to get back from vacation and get back into the groove. -Matt To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
> > Matt doesn't represent the FreeBSD project, and even if he rewrites > > the VFS subsystem so he can understand it, his rewrite would face > > considerable resistance on its way into FreeBSD. I don't think > > there is reason to rewrite it, but there certainly are areas > > that need fixing. > > You are misinformed as far as I know.. From discussions I saw, th > main architect of a VFS rewrite would be Kirk, and Matt would be acting as > Kirk's right-hand-man. Which discussions are these? Are they archived somewhere? Nate To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
On Wed, 18 Aug 1999, Poul-Henning Kamp wrote: > Matt doesn't represent the FreeBSD project, and even if he rewrites > the VFS subsystem so he can understand it, his rewrite would face > considerable resistance on its way into FreeBSD. I don't think > there is reason to rewrite it, but there certainly are areas > that need fixing. You are misinformed as far as I know.. From discussions I saw, th main architect of a VFS rewrite would be Kirk, and Matt would be acting as Kirk's right-hand-man. > > >>The use of the "vfs_default" to make unimplemented VOP's > >>fall through to code which implements function, while well > >>intentioned, is misguided. > > I beg to differ. The only difference is that we pass through > multiple layers before we hit the bottom of the stack. There is > no loss of functionality but significant gain of clarity and > modularity. Well I believe that Kirk considers them misguided too, but he stated that he wasn't going to remove them without serious thought about the alternatives. To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
> > > > > > 2. Advisory locks are hung off private backing objects. > > > I'm not sure. The struct lock * is only used by layered filesystems, so > > > they can keep track both of the underlying vnode lock, and if needed their > > > own vnode lock. For advisory locks, would we want to keep track both of > > > locks on our layer and the layer below? Don't we want either one or the > > > other? i.e. layers bypass to the one below, or deal with it all > > > themselves. > > > > I think you want the lock on the intermediate layer: basically, on > > every vnode that has data associated with it that is unique to a > > layer. Let's not forget, also, that you can expose a layer into > > the namespace in one place, and expose it covered under another > > layer, at another. If you locked down to the backing object, then > > the only issue you would be left with is one or more intermediate > > backing objects. > > Right. That exported struct lock * makes locking down to the lowest-level > file easy - you just feed it to the lock manager, and you're locking the > same lock the lowest level fs uses. You then lock all vnodes stacked over > this one at the same time. Otherwise, you just call VOP_LOCK below and > then lock yourself. I think this defeats the purpose of the stacking architecture; I think that if you look at an unadulterated NULLFS, you'll see what I mean. Intermediate FS's should not trap VOP's that are not applicable to them. One of the purposes of doing a VOP_LOCK on intermediate vnodes that aren't backing objects is to deal with the global vnode pool management. I'd really like FS's to own their vnode pools, but even without that, you don't need the locking, since you only need to flush data on vnodes that are backing objects. If we look at a stack of FS's with intermediate exposure into the namespace, then it's clear that the issue is really only applicable to objects that act as a backing store: -- -- FS Exposed in hierarchyBacking object -- -- top yes no intermediate_1 no no intermediate_2 no yes intermediate_3 yes no bottom no yes -- -- So when we lock "top", we only lock in intermediate_2 and in bottom. Then we attempt to lock in intermediate_3, but it fails: not because there is a lock on the vnode in intermediate_3, but because there is a lock in bottom. It's unnecessary to lock the vnodes in the intermediate path, or even at the exposure level, unless they are vnodes that have an associated backing store. The need to lock in intermediate_2 exists because it is a translation layer or a namespace escape. It deals with compression, or it deals with file-as-a-directory folding, or it deals with file-hiding (perhaps for a quoata file), etc.. If it didn't, it wouldn't need backing store (and therefore wouldn't need to be locked). > > For a layer with an intermediate backing object, I'm prepared to > > declare it "special", and proxy the operation down to any inferior > > backing object (e.g. a union FS that adds files from two FS's > > together, rather than just directoriy entry lists). I think such > > layers are the exception, not the rule. > > Actually isn't the only problem when you have vnode fan-in (union FS)? > i.e. a plain compressing layer should not introduce vnode locking > problems. If it's a block compression layer, it will. Also a translation layer; consider a pure Unicode system that wants to remotely mount an FS from a legacy system. To do this, it needs to expand the pages from the legacy system [only it can, since the legacy system doesn't know about Unicode] in a 2:1 ratio. Now consider doing a byte-range lock on a file on such a system. To propogate the lock, you have to do an arithmetic conversion at the translation layer. This gets worse if the lower end FS is exposed in the namespace as well. You could make the same arguments for other types of translation or namespace escapes. > > I think that export policies are the realm of /etc/exports. > > > > The problem with each FS implementing its own policy, is that this > > is another place that copyinstr() gets called, when it shouldn't. > > Well, my thought was that, like with current code, most every fs would > just call vfs_export() when it's presented an export operation. But by > retaining the option of having the fs do its own thing, we can support > different export semantics if desired. I think this bears down on whether the NFS server VFS consumer is allowed access to the VFS stack at the particular intermediate layer. I think this is really an administrative policy decision, and not an option for the VFS. I think
Re: BSD XFS Port & BSD VFS Rewrite
In message , Bill Studenmund writes: >> >I doubt we need more than 64 bit times. 2^63 seconds works out to >> >292,279,025,208 years, or 292 (american) billion years. Current theories >> >put the age of the universe at I think 12 to 16 billion years. So 64-bit >> >signed times in seconds will cover from before the big bang to way past >> >any time we'll be caring about. :-) > >I was unclear. I was refering to the seconds side of things. Sub-second >resolution would need other bits. Yes, but we need subsecond in the filesystems. Think about make(1) on a blinding fast machine... -- Poul-Henning Kamp FreeBSD coreteam member p...@freebsd.org "Real hackers run -current on their laptop." FreeBSD -- It will take a long time before progress goes too far! To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
In message , Julian Elischer writes: >On Wed, 18 Aug 1999, Poul-Henning Kamp wrote: > >> Matt doesn't represent the FreeBSD project, and even if he rewrites >> the VFS subsystem so he can understand it, his rewrite would face >> considerable resistance on its way into FreeBSD. I don't think >> there is reason to rewrite it, but there certainly are areas >> that need fixing. > >You are misinformed as far as I know.. From discussions I saw, th >main architect of a VFS rewrite would be Kirk, and Matt would be acting as >Kirk's right-hand-man. I bet that Matt and Kirk uses "rewrite" for two very different concepts. The resulting reviews will be equally different. >> >> The use of the "vfs_default" to make unimplemented VOP's >> >> fall through to code which implements function, while well >> >> intentioned, is misguided. >> >> I beg to differ. The only difference is that we pass through >> multiple layers before we hit the bottom of the stack. There is >> no loss of functionality but significant gain of clarity and >> modularity. > >Well I believe that Kirk considers them misguided too, but he stated that >he wasn't going to remove them without serious thought about the alternatives. I'll be more than ready to discuss this with Kirk. -- Poul-Henning Kamp FreeBSD coreteam member p...@freebsd.org "Real hackers run -current on their laptop." FreeBSD -- It will take a long time before progress goes too far! To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Unable to locate function body: ip_nat_init()
Hi, I am going through the networking code for the stable release. In the file ip_input.c there is a call to a function ip_nat_init() in the function ip_init(). However I have been unable to locate the code for this function (ip_nat_init()). Could somebody please tell me in which file is this function defined? Also in the file ip_input.c a function pointer ip_nat_ptr is dereferenced in the function ip_input.c However I could not locate where this function pointer is being initialized. Could somebody please tell me where this pointer is being initialized? Thanks in advance. Ratnakarprasad Tiwari
Re: BSD XFS Port & BSD VFS Rewrite
> >> > > I'm not familiar with the VFS_default stuff. All the vop_default_desc > >> > > routines in NetBSD point to error routines. > >> > > >> > In FreeBSD, they now point to default routines that are *not* error > >> > routines. This is the problem. I admit the change was very well > >> > intentioned, since it made the code a hell of a lot more readable, > >> > but choosing between readable and additional function, I take function > >> > over form (I think the way I would have "fixed" the readability is by > >> > making the operations that result in the descriptor set for a mounted > >> > FS instance be both discrete, and named for their specific function). > >> > >> As I recall most of FBSD's default routines are also error routines, if > >> the exceptions were a problem it would would be trivial to fix. > > > >You would have to de-collapse several VOP lists that have been > >pre-collapsed. > > You are talking gibberish here. Please show code where this is > a problem. When you write a proxy stacking layer, such as John Heidemann's network proxy stacking layer (an NFS alternative), VOP's which would normally be handled by vfs_default have to be handled on the other end of the proxy, instead, in the same way that they would be handled by the vfs_default stuff. Some VOP's, like advisory locking, need both local assertion and remote proxy of the VOP to avoid introducing race windows. The result of this is that, if you rely on the vfs_default stuff, then you can't proxy those VOP's into a different address space, either on another machine, or to a user space VFS stacking layer developement environment. This is the same problem that embedding VM references directly into any FS causes, and that vm_object_t aliases would exacerbate. John has, in the past, sent me a number of stacking layers done by various people, with the requirement that I not redistribute them, as they are not what he would consider to be properly representative of finished work. Since John himself did the network proxy, you could perhaps get him to send you a copy, so you could have direct access to code where this was a problem. Make sure that the system you are talking to over the proxy is not assumed to be a FreeBSD system (e.g. don't assume that the vfs_default stuff exists on the other end of the proxy, or that it would be functional). Terry Lambert [EMAIL PROTECTED] --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
On Wed, 18 Aug 1999, Poul-Henning Kamp wrote: > Yes, but we need subsecond in the filesystems. Think about make(1) on > a blinding fast machine... Oh yes, I realize that. :-) It's just that I thought you were at one point suggesting having 128 bits to the left of the decimal point (128 bits worth of seconds). I was trying to say that'd be a bit much. :-) Take care, Bill To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
On Wed, 18 Aug 1999, Terry Lambert wrote: > > Right. That exported struct lock * makes locking down to the lowest-level > > file easy - you just feed it to the lock manager, and you're locking the > > same lock the lowest level fs uses. You then lock all vnodes stacked over > > this one at the same time. Otherwise, you just call VOP_LOCK below and > > then lock yourself. > > I think this defeats the purpose of the stacking architecture; I > think that if you look at an unadulterated NULLFS, you'll see what I > mean. Please be more precise. I have looked at an unadulterated NULLFS, and found it lacking. I don't see how this change breaks stacking. > Intermediate FS's should not trap VOP's that are not applicable > to them. True. But VOP_LOCK is applicable to layered fs's. :-) > One of the purposes of doing a VOP_LOCK on intermediate vnodes > that aren't backing objects is to deal with the global vnode > pool management. I'd really like FS's to own their vnode pools, > but even without that, you don't need the locking, since you > only need to flush data on vnodes that are backing objects. > > If we look at a stack of FS's with intermediate exposure into the > namespace, then it's clear that the issue is really only applicable > to objects that act as a backing store: > > > ---- > FSExposed in hierarchyBacking object > ---- > top yes no > intermediate_1no no > intermediate_2no yes > intermediate_3yes no > bottomno yes > ---- > > So when we lock "top", we only lock in intermediate_2 and in bottom. No. One of the things Heidemann notes in his dissertation is that to prevent deadlock, you have to lock the whole stack of vnodes at once, not bit by bit. i.e. there is one lock for the whole thing. > > Actually isn't the only problem when you have vnode fan-in (union FS)? > > i.e. a plain compressing layer should not introduce vnode locking > > problems. > > If it's a block compression layer, it will. Also a translation layer; > consider a pure Unicode system that wants to remotely mount an FS > from a legacy system. To do this, it needs to expand the pages from > the legacy system [only it can, since the legacy system doesn't know > about Unicode] in a 2:1 ratio. Now consider doing a byte-range lock > on a file on such a system. To propogate the lock, you have to do > an arithmetic conversion at the translation layer. This gets worse > if the lower end FS is exposed in the namespace as well. Wait. byte-range locking is different from vnode locking. I've been talking about vnode locking, which is different from the byte-range locking you're discussing above. > > Nope. The problem is that while stacking (null, umap, and overlay fs's) > > work, we don't have the coherency issues worked out so that upper layers > > can cache data. i.e. so that the lower fs knows it has to ask the uper > > layers to give pages back. :-) But multiple ls -lR's work fine. :-) > > With UVM in NetBSD, this is (supposedly) not an issue. UBC. UVM is a new memory manager. UBC unifies the buffer cache with the VM system. > You could actually think of it this way, as well: only FS's that > contain vnodes that provide backing should implement VOP_GETPAGES > and VOP_PUTPAGES, and all I/O should be done through paging. Right. That's part of UBC. :-) Take care, Bill To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
In message <[EMAIL PROTECTED]>, Nate Williams writes: >> > Both struct timespec and struct timeval are major mistakes, they >> > make arithmetic on timestamps an expensive operation. Timestamps >> > should be stored as integers using an fix-point notations, for >> > instance 64bits with 32bit fractional seconds (the NTP timestamp), >> > or in the future 128/48. >... >> >> > Extending from 64 to 128bits would be a cheap shift and increased >> > precision and range could go hand in hand. >> >> I doubt we need more than 64 bit times. 2^63 seconds works out to >> 292,279,025,208 years, or 292 (american) billion years. > >I think Poul's point is that in the future seconds is probably way too >coarse grained. Computer's are getting faster all the time, and in the >future we may need 64 seconds, plus an additional 64 bits for the >fractions of a second, which will be necessary for accurate timekeeping. No, 64bits of fractions will not be needed, at least until we start using FreeBSD as embedded computer in Heisenbergcompensators. I recall somebody saying that 100GHz was the highest realistic (or lowest unrealistic) clock frequency using digital logic, the argument was pretty convincing physically: light speed sets a size limit, that prescripes some voltage gradients which in turn produces EMC which in turn makes sure nothing works. Also various tunnel effects, and the general heisenberisms took their toll. State of the art time interval measuring equipment is into the "a few picosecond" territory (http://www.timing.com/). Based on that I would say that 40 to 48 bits will be OK for the fraction. As a sidebar: I had a kernel running which used 32i.32f timestamps and converted to timeval & timespec as needed and it actually made a lot of code look a lot more sane. I may go back and do it some day. -- Poul-Henning Kamp FreeBSD coreteam member [EMAIL PROTECTED] "Real hackers run -current on their laptop." FreeBSD -- It will take a long time before progress goes too far! To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
On Wed, 18 Aug 1999, Poul-Henning Kamp wrote: > Matt doesn't represent the FreeBSD project, and even if he rewrites > the VFS subsystem so he can understand it, his rewrite would face > considerable resistance on its way into FreeBSD. I don't think > there is reason to rewrite it, but there certainly are areas > that need fixing. You are misinformed as far as I know.. From discussions I saw, th main architect of a VFS rewrite would be Kirk, and Matt would be acting as Kirk's right-hand-man. > > >>The use of the "vfs_default" to make unimplemented VOP's > >>fall through to code which implements function, while well > >>intentioned, is misguided. > > I beg to differ. The only difference is that we pass through > multiple layers before we hit the bottom of the stack. There is > no loss of functionality but significant gain of clarity and > modularity. Well I believe that Kirk considers them misguided too, but he stated that he wasn't going to remove them without serious thought about the alternatives. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
The discussions between Kirk and matt over a glass of beer/drink at kirk's party at USENIX and at the Bay area User's group. On Wed, 18 Aug 1999, Nate Williams wrote: > > > Matt doesn't represent the FreeBSD project, and even if he rewrites > > > the VFS subsystem so he can understand it, his rewrite would face > > > considerable resistance on its way into FreeBSD. I don't think > > > there is reason to rewrite it, but there certainly are areas > > > that need fixing. > > > > You are misinformed as far as I know.. From discussions I saw, th > > main architect of a VFS rewrite would be Kirk, and Matt would be acting as > > Kirk's right-hand-man. > > Which discussions are these? Are they archived somewhere? > > > Nate > To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: network performance vs. linux on small transfers
According to kadal: > you may also try other MTA such as qmail, postfix, etc. Postfix (and qmail I think) support SMTP PIPELINING, which greatly reduce latency. It is very interesting for small messages. -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- robe...@keltia.freenix.fr FreeBSD keltia.freenix.fr 4.0-CURRENT #73: Sat Jul 31 15:36:05 CEST 1999 To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: Need some advice regarding portable user IDs
On Tue, 17 Aug 1999, Brian C. Grayson wrote: > On Tue, Aug 17, 1999 at 07:17:45PM -0700, Wilfredo Sanchez wrote: > > A group of us at Apple are trying to figure out how to handle > > situations where a filesystem with "foreign" user ID's are present. > > Have you looked at mount_umap(8)? I (naively) think it would > solve most of your concerns. I don't think so. umap is for translating credentials between domains. I think what Fred wants to do is different, and that is to ignore the credentials on the system. Fred, right now what happens in MacOS when I take a disk which has sharing credentials set up, and hook it into another machine? How are the credentials handled there? Also, one of the problems which has been brought up in the thread is that umap needs to know what credentials to translate to. For that, we'd need to stash the credentails on the drive. Take care, Bill To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
> > Matt doesn't represent the FreeBSD project, and even if he rewrites > > the VFS subsystem so he can understand it, his rewrite would face > > considerable resistance on its way into FreeBSD. I don't think > > there is reason to rewrite it, but there certainly are areas > > that need fixing. > > You are misinformed as far as I know.. From discussions I saw, th > main architect of a VFS rewrite would be Kirk, and Matt would be acting as > Kirk's right-hand-man. Which discussions are these? Are they archived somewhere? Nate To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
yp_mkdb
I am in the process of upgrading a NIS master using version 2.1.0 to version 3.2. The 'Makefile' has been customized to include automount maps for our IRIX machines as was the 'Makefile' in the old NIS Master. The problem is that for some reason the program 'yp_mkdb' in 3.2 is much more picky. It does not tolerate lines as such: nschein -rw,intrnister:/usr/home:& It complains when it encounters the '-' if removed it works fine. Also it has problems with the '+' and absence of a whitespace following the first ASCI string in the following line: +auto.home What has changed in yp_mkdb? Is there a way to escape certain symbols? I have already tried \ '' "". Or is there another way to make the database file? Nathaniel Schein To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
> > > > > > 2. Advisory locks are hung off private backing objects. > > > I'm not sure. The struct lock * is only used by layered filesystems, so > > > they can keep track both of the underlying vnode lock, and if needed their > > > own vnode lock. For advisory locks, would we want to keep track both of > > > locks on our layer and the layer below? Don't we want either one or the > > > other? i.e. layers bypass to the one below, or deal with it all > > > themselves. > > > > I think you want the lock on the intermediate layer: basically, on > > every vnode that has data associated with it that is unique to a > > layer. Let's not forget, also, that you can expose a layer into > > the namespace in one place, and expose it covered under another > > layer, at another. If you locked down to the backing object, then > > the only issue you would be left with is one or more intermediate > > backing objects. > > Right. That exported struct lock * makes locking down to the lowest-level > file easy - you just feed it to the lock manager, and you're locking the > same lock the lowest level fs uses. You then lock all vnodes stacked over > this one at the same time. Otherwise, you just call VOP_LOCK below and > then lock yourself. I think this defeats the purpose of the stacking architecture; I think that if you look at an unadulterated NULLFS, you'll see what I mean. Intermediate FS's should not trap VOP's that are not applicable to them. One of the purposes of doing a VOP_LOCK on intermediate vnodes that aren't backing objects is to deal with the global vnode pool management. I'd really like FS's to own their vnode pools, but even without that, you don't need the locking, since you only need to flush data on vnodes that are backing objects. If we look at a stack of FS's with intermediate exposure into the namespace, then it's clear that the issue is really only applicable to objects that act as a backing store: -- -- FS Exposed in hierarchyBacking object -- -- top yes no intermediate_1 no no intermediate_2 no yes intermediate_3 yes no bottom no yes -- -- So when we lock "top", we only lock in intermediate_2 and in bottom. Then we attempt to lock in intermediate_3, but it fails: not because there is a lock on the vnode in intermediate_3, but because there is a lock in bottom. It's unnecessary to lock the vnodes in the intermediate path, or even at the exposure level, unless they are vnodes that have an associated backing store. The need to lock in intermediate_2 exists because it is a translation layer or a namespace escape. It deals with compression, or it deals with file-as-a-directory folding, or it deals with file-hiding (perhaps for a quoata file), etc.. If it didn't, it wouldn't need backing store (and therefore wouldn't need to be locked). > > For a layer with an intermediate backing object, I'm prepared to > > declare it "special", and proxy the operation down to any inferior > > backing object (e.g. a union FS that adds files from two FS's > > together, rather than just directoriy entry lists). I think such > > layers are the exception, not the rule. > > Actually isn't the only problem when you have vnode fan-in (union FS)? > i.e. a plain compressing layer should not introduce vnode locking > problems. If it's a block compression layer, it will. Also a translation layer; consider a pure Unicode system that wants to remotely mount an FS from a legacy system. To do this, it needs to expand the pages from the legacy system [only it can, since the legacy system doesn't know about Unicode] in a 2:1 ratio. Now consider doing a byte-range lock on a file on such a system. To propogate the lock, you have to do an arithmetic conversion at the translation layer. This gets worse if the lower end FS is exposed in the namespace as well. You could make the same arguments for other types of translation or namespace escapes. > > I think that export policies are the realm of /etc/exports. > > > > The problem with each FS implementing its own policy, is that this > > is another place that copyinstr() gets called, when it shouldn't. > > Well, my thought was that, like with current code, most every fs would > just call vfs_export() when it's presented an export operation. But by > retaining the option of having the fs do its own thing, we can support > different export semantics if desired. I think this bears down on whether the NFS server VFS consumer is allowed access to the VFS stack at the particular intermediate layer. I think this is really an administrative policy decision, and not an option for the VFS. I think
Gigabit ethernet support?
Any supported cards in 3.2.x? The HCL pages don't list any:( Thanks, --- David To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
yp_mkdb
I am in the process of upgrading a NIS master using version 2.1.0 to version 3.2. The 'Makefile' has been customized to include automount maps for our IRIX machines as was the 'Makefile' in the old NIS Master. The problem is that for some reason the program 'yp_mkdb' in 3.2 is much more picky. It does not tolerate lines as such: nschein -rw,intrnister:/usr/home:& It complains when it encounters the '-' if removed it works fine. Also it has problems with the '+' and absence of a whitespace following the first ASCI string in the following line: +auto.home What has changed in yp_mkdb? Is there a way to escape certain symbols? I have already tried \ '' "". Or is there another way to make the database file? Nathaniel Schein To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-questions" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
:On Wed, 18 Aug 1999, Poul-Henning Kamp wrote: : :> Matt doesn't represent the FreeBSD project, and even if he rewrites :> the VFS subsystem so he can understand it, his rewrite would face :> considerable resistance on its way into FreeBSD. I don't think :> there is reason to rewrite it, but there certainly are areas :> that need fixing. : :You are misinformed as far as I know.. From discussions I saw, th :main architect of a VFS rewrite would be Kirk, and Matt would be acting as :Kirk's right-hand-man. Yes, this is correct. Kirk is going to be the main architect. I have been heavily involved and will continue to be. :> >> The use of the "vfs_default" to make unimplemented VOP's : :> I beg to differ. The only difference is that we pass through :> multiple layers before we hit the bottom of the stack. There is :... :Well I believe that Kirk considers them misguided too, but he stated that :he wasn't going to remove them without serious thought about the alternatives. The vfs op callout layering has not been on the radar screen. There are much too many other more serious problems. I really doubt that any changes will be made to this piece any time in the next year or even two, if at all. The main items on the radar screen are related to buffer management (struct buf stuff. For example, preventing VM blockages due to pages being wired by write I/O's), VFS locking and reference count issues (for example, namei lookups, blockages in the pager and syncer due to vnode locks held by blocked processes, etc...), and interactions between VFS and VM (for example: moving away from VOP_READ/VOP_WRITE and moving more towards a getpages/putpages model). None of the items have been set in stone yet. We're waiting for Kirk to get back from vacation and get back into the groove. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: several messages
Wayne Cuddy scribbled this message on Aug 24: > Thank you for your reply. At what point should I set this socket option? I > am assuming right after the socket is allocated?? > > I will try this and post my results tomorrow night. > > For those wondering, I cannot just execute Sendmail directly, there are many > architectural reasons for this design... sounds like you need to fork upon recieption of the message and send the message in a child process... don't forget to reap your children though... you don't want a lot of zombies laying around... if you do this, you really don't need to set the TCP_NODELAY option.. but you might want to anyways... -- John-Mark Gurney Voice: +1 541 684 8449 Cu Networking P.O. Box 5693, 97405 "The soul contains in itself the event that shall presently befall it. The event is only the actualizing of its thought." -- Ralph Waldo Emerson To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: cvs commit: src/bin/test test.c
On Wed, Aug 18, 1999, Sheldon Hearn wrote: > > green 1999/08/17 17:18:53 PDT > > > > Modified files: > > bin/test test.c > > Log: > > The new test(1) did not use access() correctly. I don't know why, since > > supposedly it's ksh-derived, and it's not broken in pdksh. I've added > > a test for test running as root: if testing for -x, the file must be > > mode & 0111 to get "success", rather than just existant. > > > > Reviewed by: chris > > What were you actually trying to fix, here? I didn't see any discussion > of this on hackers, current or bugs, nor in response to my initial > commit message. He was "fixing" (though, as Bruce pointed out, it wasn't a valid fix) test -x. Apparently, access(2) will return 'success' on access(file, X_OK) if called by a program run by root. The patch partly solves the problem, but the euid-vs-ruid problem remains. -- |Chris Costello <[EMAIL PROTECTED]> |Disc space, the final frontier! `-- To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Need some advice regarding portable user IDs
On Wed, 18 Aug 1999, Marc Ramirez wrote: > Oh! I was under the impression that it just didn't work, even with > correct perms, but I use FreeBSD. Lemme try it... Can't mount, even > with 0666 on /dev/fd0. Maybe I'm being stupid. Wouldn't be the first > time! It's controlled by a sysctl in FreeBSD which defaults to "off" - I forget what it is called. Kris To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
In message <[EMAIL PROTECTED]>, Julian Elischer writes: >On Wed, 18 Aug 1999, Poul-Henning Kamp wrote: > >> Matt doesn't represent the FreeBSD project, and even if he rewrites >> the VFS subsystem so he can understand it, his rewrite would face >> considerable resistance on its way into FreeBSD. I don't think >> there is reason to rewrite it, but there certainly are areas >> that need fixing. > >You are misinformed as far as I know.. From discussions I saw, th >main architect of a VFS rewrite would be Kirk, and Matt would be acting as >Kirk's right-hand-man. I bet that Matt and Kirk uses "rewrite" for two very different concepts. The resulting reviews will be equally different. >> >> The use of the "vfs_default" to make unimplemented VOP's >> >> fall through to code which implements function, while well >> >> intentioned, is misguided. >> >> I beg to differ. The only difference is that we pass through >> multiple layers before we hit the bottom of the stack. There is >> no loss of functionality but significant gain of clarity and >> modularity. > >Well I believe that Kirk considers them misguided too, but he stated that >he wasn't going to remove them without serious thought about the alternatives. I'll be more than ready to discuss this with Kirk. -- Poul-Henning Kamp FreeBSD coreteam member [EMAIL PROTECTED] "Real hackers run -current on their laptop." FreeBSD -- It will take a long time before progress goes too far! To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: network performance vs. linux on small transfers
On Tue, 24 Aug 1999, kadal wrote: > > > > On Tue, 24 Aug 1999, Wayne Cuddy wrote: > > > Date: Tue, 24 Aug 1999 00:38:21 -0400 (EDT) > > From: Wayne Cuddy > > To: FreeBSD Hackers List > > Subject: network performance vs. linux on small transfers > > > > I am involved in a messaging system at work in which we need to send/receive > > large amounts of small (one line messages) SMTP messages. We are currently > > using Sendmail 8.9.3 > > on HPUX. > > > > Our application sends messages down a FIFO to a daemon process that is > > reading from > > the FIFO. This process then connects to port 25 of the destination system > > and > > delivers the mail via SMTP. Currently the destination system is the local > > system so everything is done on one machine. > > > > Using HPUX we typically pass 5 messages a second. This system is a dual > > 180Mhz K class server so this is surprisingly low performance for this > > system. > > > > When testing on FreeBSD 3.1 we also got 5 messages a second. This system > > is a > > 500Mhz P3, this is also unacceptable performance. > > > > When we tested with Linux (kernel 2.2.5) we passed 15 messages a second > > consistently using the exact same P3 described above. > > > > Since the HPUX and FreeBSD numbers are so close I am wondering there is some > > performance tuning that I do not know about. Do you think the number might > > change if multiple hosts were used? > > > > The daemon that reads from the FIFO makes only one connection to the local > > Sendmail to deliver multiple messages in sequence. > > do you really have to deliver the messages sequentially ? SMTP > conversation is rather slow, especially for small messages. > > you may want to try to deliver them simultaneously, by creating multiple > SMTP conversations. > > you may also try other MTA such as qmail, postfix, etc. > > > I REALLY want to use FreeBSD over Linux on this one and need some major help > > to get the performance out of FreeBSD. > > how about profiling your program / system ? try to find where it spends > most time. it could be forking, disk I/O, SMTP conversation, etc. I > strongly suspect that it's SMTP conversation, but can't really sure before > you mesure it. Wild guess, the creation of spool files syncronously is killing performance you may give freebsd a signifigant boost by either: mount -o async -u /var (spool partition) or enabling softupdates, check out: /usr/src/sys/contrib/softupdates/README.softupdates. please tell me how it goes. good luck, -Alfred To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: BSD XFS Port & BSD VFS Rewrite
In message <[EMAIL PROTECTED]>, Bill Studenmund writes: >> >I doubt we need more than 64 bit times. 2^63 seconds works out to >> >292,279,025,208 years, or 292 (american) billion years. Current theories >> >put the age of the universe at I think 12 to 16 billion years. So 64-bit >> >signed times in seconds will cover from before the big bang to way past >> >any time we'll be caring about. :-) > >I was unclear. I was refering to the seconds side of things. Sub-second >resolution would need other bits. Yes, but we need subsecond in the filesystems. Think about make(1) on a blinding fast machine... -- Poul-Henning Kamp FreeBSD coreteam member [EMAIL PROTECTED] "Real hackers run -current on their laptop." FreeBSD -- It will take a long time before progress goes too far! To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Unable to locate function body: ip_nat_init()
Hi,  I am going through the networking code for the stable release.  In the file ip_input.c there is a call to a function ip_nat_init() in the function  ip_init(). However I have been unable to locate the code for this function (ip_nat_init()).   Could somebody please tell me in which file is this function defined?    Also in the file ip_input.c a function pointer ip_nat_ptr is dereferenced in the function ip_input.c  However I could not locate where this function pointer is being initialized. Could somebody please  tell me where this pointer is being initialized?  Thanks in advance. Ratnakarprasad Tiwari
Re: several messages
As an (former) implementer of fast TCP/IP peer-peer communications, I'd have to agree with Dave, and say that it is definitely the TCP_NODELAY option. You'll find that disabling the TCP-ACK delay will greatly increase your performace. The reason that it is so "slow" is because the TCP/IP stack is trying to wait to send a TCP "ACK" piggy-backed with data that you MAY BE SENDING SOON. So it waits for 1/5 of a second for you to send SOMETHING, then shrugs its shoulders when you don't, and sends the TCP ACK without further delay. This is a "standard" that most TCP/IP stacks implement. You'll find the same "problem" under Windows. In fact, when I was doing the peer-peer communications, from a Unix to a Windows '95 machine (and NT), the TCP_NODELAY was not yet implemented in the WinSock library. The workaround was to send garbage back as fast as possible, so the ACK could piggy-back itself on SOMETHING. You may want to set the transmit and recieve low-water marks as well. Look at the man page for "setsockopt". -Mark Taylor NetMAX Developer mtay...@cybernet.com http://www.netmax.com/ Wayne Cuddy wrote: > > Thank you for your reply. At what point should I set this socket option? I > am assuming right after the socket is allocated?? > > I will try this and post my results tomorrow night. > > For those wondering, I cannot just execute Sendmail directly, there are many > architectural reasons for this design... > > Thanks again, > > Wayne > > On Tue, 24 Aug 1999, Daniel O'Connor wrote: > > > Date: Tue, 24 Aug 1999 13:41:37 +0930 (CST) > > From: Daniel O'Connor > > To: Wayne Cuddy > > Cc: FreeBSD Hackers List > > Subject: RE: network performance vs. linux on small transfers > > > > > > On 24-Aug-99 Wayne Cuddy wrote: > > > I REALLY want to use FreeBSD over Linux on this one and need some major > > > help > > > to get the performance out of FreeBSD. > > > > Tried setsockopt and TCP_NODELAY? > > > > >From netinet/tcp.h > > #define TCP_NODELAY 0x01/* don't delay send to coalesce packets */ > > > > --- > > Daniel O'Connor software and network engineer > > for Genesis Software - http://www.gsoft.com.au > > "The nice thing about standards is that there > > are so many of them to choose from." > > -- Andrew Tanenbaum > > > > On Mon, 23 Aug 1999, David Greenman wrote: > > > Date: Mon, 23 Aug 1999 21:17:06 -0700 > > From: David Greenman > > To: wa...@crb-web.com > > Cc: FreeBSD Hackers List > > Subject: Re: network performance vs. linux on small transfers > > > > >I am involved in a messaging system at work in which we need to > > >send/receive > > >large amounts of small (one line messages) SMTP messages. We are > > >currently using Sendmail 8.9.3 > > >on HPUX. > > > > > >Our application sends messages down a FIFO to a daemon process that is > > >reading from > > >the FIFO. This process then connects to port 25 of the destination system > > >and > > >delivers the mail via SMTP. Currently the destination system is the local > > >system so everything is done on one machine. > > > > > >Using HPUX we typically pass 5 messages a second. This system is a dual > > >180Mhz K class server so this is surprisingly low performance for this > > >system. > > > > > >When testing on FreeBSD 3.1 we also got 5 messages a second. This system > > >is a > > >500Mhz P3, this is also unacceptable performance. > > > > > >When we tested with Linux (kernel 2.2.5) we passed 15 messages a second > > >consistently using the exact same P3 described above. > > > > > >Since the HPUX and FreeBSD numbers are so close I am wondering there is > > >some > > >performance tuning that I do not know about. Do you think the number might > > >change if multiple hosts were used? > > > > > >The daemon that reads from the FIFO makes only one connection to the local > > >Sendmail to deliver multiple messages in sequence. > > > > > > > > >I REALLY want to use FreeBSD over Linux on this one and need some major > > >help > > >to get the performance out of FreeBSD. > > > >Are you setting the TCP_NODELAY socket option on the SMTP connection? If > > not, then please do that and let me know if it fixes the problem or not. > > > > -DG > > > > David Greenman > > Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org > > Creator of high-performance Internet servers - http://www.terasolutions.com > > Pave the road of life with opportunities. > > > > To Unsubscribe: send mail to majord...@freebsd.org > with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: network performance vs. linux on small transfers
According to kadal: > you may also try other MTA such as qmail, postfix, etc. Postfix (and qmail I think) support SMTP PIPELINING, which greatly reduce latency. It is very interesting for small messages. -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- [EMAIL PROTECTED] FreeBSD keltia.freenix.fr 4.0-CURRENT #73: Sat Jul 31 15:36:05 CEST 1999 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: several messages
Wayne Cuddy scribbled this message on Aug 24: > Thank you for your reply. At what point should I set this socket option? I > am assuming right after the socket is allocated?? > > I will try this and post my results tomorrow night. > > For those wondering, I cannot just execute Sendmail directly, there are many > architectural reasons for this design... sounds like you need to fork upon recieption of the message and send the message in a child process... don't forget to reap your children though... you don't want a lot of zombies laying around... if you do this, you really don't need to set the TCP_NODELAY option.. but you might want to anyways... -- John-Mark Gurney Voice: +1 541 684 8449 Cu Networking P.O. Box 5693, 97405 "The soul contains in itself the event that shall presently befall it. The event is only the actualizing of its thought." -- Ralph Waldo Emerson To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: network performance vs. linux on small transfers
On Tue, 24 Aug 1999, Wayne Cuddy wrote: > Date: Tue, 24 Aug 1999 00:38:21 -0400 (EDT) > From: Wayne Cuddy > To: FreeBSD Hackers List > Subject: network performance vs. linux on small transfers > > I am involved in a messaging system at work in which we need to send/receive > large amounts of small (one line messages) SMTP messages. We are currently > using Sendmail 8.9.3 > on HPUX. > > Our application sends messages down a FIFO to a daemon process that is > reading from > the FIFO. This process then connects to port 25 of the destination system and > delivers the mail via SMTP. Currently the destination system is the local > system so everything is done on one machine. > > Using HPUX we typically pass 5 messages a second. This system is a dual > 180Mhz K class server so this is surprisingly low performance for this system. > > When testing on FreeBSD 3.1 we also got 5 messages a second. This system is a > 500Mhz P3, this is also unacceptable performance. > > When we tested with Linux (kernel 2.2.5) we passed 15 messages a second > consistently using the exact same P3 described above. > > Since the HPUX and FreeBSD numbers are so close I am wondering there is some > performance tuning that I do not know about. Do you think the number might > change if multiple hosts were used? > > The daemon that reads from the FIFO makes only one connection to the local > Sendmail to deliver multiple messages in sequence. do you really have to deliver the messages sequentially ? SMTP conversation is rather slow, especially for small messages. you may want to try to deliver them simultaneously, by creating multiple SMTP conversations. you may also try other MTA such as qmail, postfix, etc. > I REALLY want to use FreeBSD over Linux on this one and need some major help > to get the performance out of FreeBSD. how about profiling your program / system ? try to find where it spends most time. it could be forking, disk I/O, SMTP conversation, etc. I strongly suspect that it's SMTP conversation, but can't really sure before you mesure it. -k- To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: network performance vs. linux on small transfers
On Tue, 24 Aug 1999, kadal wrote: > > > > On Tue, 24 Aug 1999, Wayne Cuddy wrote: > > > Date: Tue, 24 Aug 1999 00:38:21 -0400 (EDT) > > From: Wayne Cuddy <[EMAIL PROTECTED]> > > To: FreeBSD Hackers List <[EMAIL PROTECTED]> > > Subject: network performance vs. linux on small transfers > > > > I am involved in a messaging system at work in which we need to send/receive > > large amounts of small (one line messages) SMTP messages. We are currently using >Sendmail 8.9.3 > > on HPUX. > > > > Our application sends messages down a FIFO to a daemon process that is reading from > > the FIFO. This process then connects to port 25 of the destination system and > > delivers the mail via SMTP. Currently the destination system is the local > > system so everything is done on one machine. > > > > Using HPUX we typically pass 5 messages a second. This system is a dual > > 180Mhz K class server so this is surprisingly low performance for this system. > > > > When testing on FreeBSD 3.1 we also got 5 messages a second. This system is a > > 500Mhz P3, this is also unacceptable performance. > > > > When we tested with Linux (kernel 2.2.5) we passed 15 messages a second > > consistently using the exact same P3 described above. > > > > Since the HPUX and FreeBSD numbers are so close I am wondering there is some > > performance tuning that I do not know about. Do you think the number might > > change if multiple hosts were used? > > > > The daemon that reads from the FIFO makes only one connection to the local > > Sendmail to deliver multiple messages in sequence. > > do you really have to deliver the messages sequentially ? SMTP > conversation is rather slow, especially for small messages. > > you may want to try to deliver them simultaneously, by creating multiple > SMTP conversations. > > you may also try other MTA such as qmail, postfix, etc. > > > I REALLY want to use FreeBSD over Linux on this one and need some major help > > to get the performance out of FreeBSD. > > how about profiling your program / system ? try to find where it spends > most time. it could be forking, disk I/O, SMTP conversation, etc. I > strongly suspect that it's SMTP conversation, but can't really sure before > you mesure it. Wild guess, the creation of spool files syncronously is killing performance you may give freebsd a signifigant boost by either: mount -o async -u /var (spool partition) or enabling softupdates, check out: /usr/src/sys/contrib/softupdates/README.softupdates. please tell me how it goes. good luck, -Alfred To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: several messages
Thank you for your reply. At what point should I set this socket option? I am assuming right after the socket is allocated?? I will try this and post my results tomorrow night. For those wondering, I cannot just execute Sendmail directly, there are many architectural reasons for this design... Thanks again, Wayne On Tue, 24 Aug 1999, Daniel O'Connor wrote: > Date: Tue, 24 Aug 1999 13:41:37 +0930 (CST) > From: Daniel O'Connor > To: Wayne Cuddy > Cc: FreeBSD Hackers List > Subject: RE: network performance vs. linux on small transfers > > > On 24-Aug-99 Wayne Cuddy wrote: > > I REALLY want to use FreeBSD over Linux on this one and need some major help > > to get the performance out of FreeBSD. > > Tried setsockopt and TCP_NODELAY? > > >From netinet/tcp.h > #define TCP_NODELAY 0x01/* don't delay send to coalesce packets */ > > --- > Daniel O'Connor software and network engineer > for Genesis Software - http://www.gsoft.com.au > "The nice thing about standards is that there > are so many of them to choose from." > -- Andrew Tanenbaum > On Mon, 23 Aug 1999, David Greenman wrote: > Date: Mon, 23 Aug 1999 21:17:06 -0700 > From: David Greenman > To: wa...@crb-web.com > Cc: FreeBSD Hackers List > Subject: Re: network performance vs. linux on small transfers > > >I am involved in a messaging system at work in which we need to send/receive > >large amounts of small (one line messages) SMTP messages. We are currently > >using Sendmail 8.9.3 > >on HPUX. > > > >Our application sends messages down a FIFO to a daemon process that is > >reading from > >the FIFO. This process then connects to port 25 of the destination system > >and > >delivers the mail via SMTP. Currently the destination system is the local > >system so everything is done on one machine. > > > >Using HPUX we typically pass 5 messages a second. This system is a dual > >180Mhz K class server so this is surprisingly low performance for this > >system. > > > >When testing on FreeBSD 3.1 we also got 5 messages a second. This system is > >a > >500Mhz P3, this is also unacceptable performance. > > > >When we tested with Linux (kernel 2.2.5) we passed 15 messages a second > >consistently using the exact same P3 described above. > > > >Since the HPUX and FreeBSD numbers are so close I am wondering there is some > >performance tuning that I do not know about. Do you think the number might > >change if multiple hosts were used? > > > >The daemon that reads from the FIFO makes only one connection to the local > >Sendmail to deliver multiple messages in sequence. > > > > > >I REALLY want to use FreeBSD over Linux on this one and need some major help > >to get the performance out of FreeBSD. > >Are you setting the TCP_NODELAY socket option on the SMTP connection? If > not, then please do that and let me know if it fixes the problem or not. > > -DG > > David Greenman > Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org > Creator of high-performance Internet servers - http://www.terasolutions.com > Pave the road of life with opportunities. > To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: network performance vs. linux on small transfers
>I am involved in a messaging system at work in which we need to send/receive >large amounts of small (one line messages) SMTP messages. We are currently >using Sendmail 8.9.3 >on HPUX. > >Our application sends messages down a FIFO to a daemon process that is reading >from >the FIFO. This process then connects to port 25 of the destination system and >delivers the mail via SMTP. Currently the destination system is the local >system so everything is done on one machine. > >Using HPUX we typically pass 5 messages a second. This system is a dual >180Mhz K class server so this is surprisingly low performance for this system. > >When testing on FreeBSD 3.1 we also got 5 messages a second. This system is a >500Mhz P3, this is also unacceptable performance. > >When we tested with Linux (kernel 2.2.5) we passed 15 messages a second >consistently using the exact same P3 described above. > >Since the HPUX and FreeBSD numbers are so close I am wondering there is some >performance tuning that I do not know about. Do you think the number might >change if multiple hosts were used? > >The daemon that reads from the FIFO makes only one connection to the local >Sendmail to deliver multiple messages in sequence. > > >I REALLY want to use FreeBSD over Linux on this one and need some major help >to get the performance out of FreeBSD. Are you setting the TCP_NODELAY socket option on the SMTP connection? If not, then please do that and let me know if it fixes the problem or not. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: network performance vs. linux on small transfers
Wayne Cuddy wrote: > > I am involved in a messaging system at work in which we need to send/receive > large amounts of small (one line messages) SMTP messages. We are currently > using Sendmail 8.9.3 > on HPUX. > > Our application sends messages down a FIFO to a daemon process that is > reading from > the FIFO. This process then connects to port 25 of the destination system and > delivers the mail via SMTP. Currently the destination system is the local > system so everything is done on one machine. > > Using HPUX we typically pass 5 messages a second. This system is a dual > 180Mhz K class server so this is surprisingly low performance for this system. > > When testing on FreeBSD 3.1 we also got 5 messages a second. This system is a > 500Mhz P3, this is also unacceptable performance. > > When we tested with Linux (kernel 2.2.5) we passed 15 messages a second > consistently using the exact same P3 described above. > > Since the HPUX and FreeBSD numbers are so close I am wondering there is some > performance tuning that I do not know about. Do you think the number might > change if multiple hosts were used? > > The daemon that reads from the FIFO makes only one connection to the local > Sendmail to deliver multiple messages in sequence. Why not just fork and exec sendmail instead? -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC http://softweyr.com/ w...@softweyr.com To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: several messages
As an (former) implementer of fast TCP/IP peer-peer communications, I'd have to agree with Dave, and say that it is definitely the TCP_NODELAY option. You'll find that disabling the TCP-ACK delay will greatly increase your performace. The reason that it is so "slow" is because the TCP/IP stack is trying to wait to send a TCP "ACK" piggy-backed with data that you MAY BE SENDING SOON. So it waits for 1/5 of a second for you to send SOMETHING, then shrugs its shoulders when you don't, and sends the TCP ACK without further delay. This is a "standard" that most TCP/IP stacks implement. You'll find the same "problem" under Windows. In fact, when I was doing the peer-peer communications, from a Unix to a Windows '95 machine (and NT), the TCP_NODELAY was not yet implemented in the WinSock library. The workaround was to send garbage back as fast as possible, so the ACK could piggy-back itself on SOMETHING. You may want to set the transmit and recieve low-water marks as well. Look at the man page for "setsockopt". -Mark Taylor NetMAX Developer [EMAIL PROTECTED] http://www.netmax.com/ Wayne Cuddy wrote: > > Thank you for your reply. At what point should I set this socket option? I > am assuming right after the socket is allocated?? > > I will try this and post my results tomorrow night. > > For those wondering, I cannot just execute Sendmail directly, there are many > architectural reasons for this design... > > Thanks again, > > Wayne > > On Tue, 24 Aug 1999, Daniel O'Connor wrote: > > > Date: Tue, 24 Aug 1999 13:41:37 +0930 (CST) > > From: Daniel O'Connor <[EMAIL PROTECTED]> > > To: Wayne Cuddy <[EMAIL PROTECTED]> > > Cc: FreeBSD Hackers List <[EMAIL PROTECTED]> > > Subject: RE: network performance vs. linux on small transfers > > > > > > On 24-Aug-99 Wayne Cuddy wrote: > > > I REALLY want to use FreeBSD over Linux on this one and need some major help > > > to get the performance out of FreeBSD. > > > > Tried setsockopt and TCP_NODELAY? > > > > >From netinet/tcp.h > > #define TCP_NODELAY 0x01/* don't delay send to coalesce packets */ > > > > --- > > Daniel O'Connor software and network engineer > > for Genesis Software - http://www.gsoft.com.au > > "The nice thing about standards is that there > > are so many of them to choose from." > > -- Andrew Tanenbaum > > > > On Mon, 23 Aug 1999, David Greenman wrote: > > > Date: Mon, 23 Aug 1999 21:17:06 -0700 > > From: David Greenman <[EMAIL PROTECTED]> > > To: [EMAIL PROTECTED] > > Cc: FreeBSD Hackers List <[EMAIL PROTECTED]> > > Subject: Re: network performance vs. linux on small transfers > > > > >I am involved in a messaging system at work in which we need to send/receive > > >large amounts of small (one line messages) SMTP messages. We are currently using >Sendmail 8.9.3 > > >on HPUX. > > > > > >Our application sends messages down a FIFO to a daemon process that is reading >from > > >the FIFO. This process then connects to port 25 of the destination system and > > >delivers the mail via SMTP. Currently the destination system is the local > > >system so everything is done on one machine. > > > > > >Using HPUX we typically pass 5 messages a second. This system is a dual > > >180Mhz K class server so this is surprisingly low performance for this system. > > > > > >When testing on FreeBSD 3.1 we also got 5 messages a second. This system is a > > >500Mhz P3, this is also unacceptable performance. > > > > > >When we tested with Linux (kernel 2.2.5) we passed 15 messages a second > > >consistently using the exact same P3 described above. > > > > > >Since the HPUX and FreeBSD numbers are so close I am wondering there is some > > >performance tuning that I do not know about. Do you think the number might > > >change if multiple hosts were used? > > > > > >The daemon that reads from the FIFO makes only one connection to the local > > >Sendmail to deliver multiple messages in sequence. > > > > > > > > >I REALLY want to use FreeBSD over Linux on this one and need some major help > > >to get the performance out of FreeBSD. > > > >Are you setting the TCP_NODELAY socket option on the SMTP connection? If > > not, then please do that and let me know if it fixes the problem or not. > > > > -DG > > > > David Greenman > > Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org > > Creator of high-performance Internet servers - http://www.terasolutions.com > > Pave the road of life with opportunities. > > > > To Unsubscribe: send mail to [EMAIL PROTECTED] > with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
network performance vs. linux on small transfers
I am involved in a messaging system at work in which we need to send/receive large amounts of small (one line messages) SMTP messages. We are currently using Sendmail 8.9.3 on HPUX. Our application sends messages down a FIFO to a daemon process that is reading from the FIFO. This process then connects to port 25 of the destination system and delivers the mail via SMTP. Currently the destination system is the local system so everything is done on one machine. Using HPUX we typically pass 5 messages a second. This system is a dual 180Mhz K class server so this is surprisingly low performance for this system. When testing on FreeBSD 3.1 we also got 5 messages a second. This system is a 500Mhz P3, this is also unacceptable performance. When we tested with Linux (kernel 2.2.5) we passed 15 messages a second consistently using the exact same P3 described above. Since the HPUX and FreeBSD numbers are so close I am wondering there is some performance tuning that I do not know about. Do you think the number might change if multiple hosts were used? The daemon that reads from the FIFO makes only one connection to the local Sendmail to deliver multiple messages in sequence. I REALLY want to use FreeBSD over Linux on this one and need some major help to get the performance out of FreeBSD. Much thanks in advance, Wayne Cuddy To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: network performance vs. linux on small transfers
On Tue, 24 Aug 1999, Wayne Cuddy wrote: > Date: Tue, 24 Aug 1999 00:38:21 -0400 (EDT) > From: Wayne Cuddy <[EMAIL PROTECTED]> > To: FreeBSD Hackers List <[EMAIL PROTECTED]> > Subject: network performance vs. linux on small transfers > > I am involved in a messaging system at work in which we need to send/receive > large amounts of small (one line messages) SMTP messages. We are currently using >Sendmail 8.9.3 > on HPUX. > > Our application sends messages down a FIFO to a daemon process that is reading from > the FIFO. This process then connects to port 25 of the destination system and > delivers the mail via SMTP. Currently the destination system is the local > system so everything is done on one machine. > > Using HPUX we typically pass 5 messages a second. This system is a dual > 180Mhz K class server so this is surprisingly low performance for this system. > > When testing on FreeBSD 3.1 we also got 5 messages a second. This system is a > 500Mhz P3, this is also unacceptable performance. > > When we tested with Linux (kernel 2.2.5) we passed 15 messages a second > consistently using the exact same P3 described above. > > Since the HPUX and FreeBSD numbers are so close I am wondering there is some > performance tuning that I do not know about. Do you think the number might > change if multiple hosts were used? > > The daemon that reads from the FIFO makes only one connection to the local > Sendmail to deliver multiple messages in sequence. do you really have to deliver the messages sequentially ? SMTP conversation is rather slow, especially for small messages. you may want to try to deliver them simultaneously, by creating multiple SMTP conversations. you may also try other MTA such as qmail, postfix, etc. > I REALLY want to use FreeBSD over Linux on this one and need some major help > to get the performance out of FreeBSD. how about profiling your program / system ? try to find where it spends most time. it could be forking, disk I/O, SMTP conversation, etc. I strongly suspect that it's SMTP conversation, but can't really sure before you mesure it. -k- To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Monday, 23 August 1999 at 23:27:27 -0400, Christopher Masto wrote: > On Mon, Aug 23, 1999 at 11:16:21PM -0400, Chuck Robey wrote: >> On Mon, 23 Aug 1999, Christopher Masto wrote: >> >>> Bleah.. I can't count the number of times I've seen idiotic code like: >>> >>> open file >>> read data >>> close file >>> open file for write >>> write data >>> close file >>> >>> Mandatory locking of the type above doesn't force such a thing to work. >> >> What has that code you show above got to do with mandatory locking? >> You completely missed the explicit locking calls that you have to make, >> to get and release the locks. If you don't make the call, and you have >> madatory locking, then your process will sleep until someone else >> releases the lock; > > Exactly. You said that mandatory locking means that user A's correct > use of locking means that user B doesn't have to be careful. That's > not the case, since A can step in between B's read and write. B doesn't have to be careful about messing up A's transaction, like he doesn't need to be careful not to overwrite A's address space. If he wants his own transactions protected, he needs to do something about that. Greg -- See complete headers for address, home page and phone numbers finger g...@lemis.com for PGP public key To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Monday, 23 August 1999 at 23:34:34 -0400, Christopher Masto wrote: > On Tue, Aug 24, 1999 at 12:52:10PM +0930, Greg Lehey wrote: >> No, I think you're confusing opening and locking. It's something like >> this: >> >> User 1 User 2 >> >> open fileopen file >> lock fileread file (blocks) >> diddle file >> unlock file >> read completes > > How about a little timing difference? > > User 1User 2 > > open file > read file > open file > lock file (blocks?) > close file > lock returnsopen file (blocks) > diddle file > unlock file > scribble over User 1's changes That's OK. In my scenario, user 2 can't know about what user 1 is doing. Typically there'll be multiple accesses. Since you mentioned mail, let's look at it, since that's where things really *do* use (advisory) locking. When I save my mail folder, I need to ensure that sendmail doesn't choose that moment to deliver a new message; otherwise it might disappear completely. Mailers and sendmail do have an agreement of some kind. But what if I want to merge the contents of another mail folder: cat oldmail >>/var/mail/grog That works, but it's playing with fire: if sendmail is delivering a message at the same time, it won't see me, and my cat doesn't get a lock beforehand, so both an incoming message and part of my mail folder could end up getting written to the same location. With mandatory locking, it would work, transparently. > What I'm getting at is that if User 2 has to do something special > anyway, it might as well be using advisory locking. What I'm getting at is that User 2 doesn't have to do anything different. >>> who knows how many scripts and programs will now be vulnerable to >>> hanging forever.. >> >> Why? There is a danger, of course, that user 1 will lock the file and >> not unlock it. That's a badly written program, so you stop it. End >> of hang. > > That's not what I meant. It hasn't been on FreeBSD, so FreeBSD is not > designed to deal with it. Obviously. > I mentioned a couple of examples.. if I lock a bunch of files in my > web space, does apache get a bunch of children stuck forever? Only as long as you keep them locked. And yes, you shouldn't go locking files longer than you need them. > Who knows what might get tripped up? Nobody. Do you know where your next SIGSEGV is coming from? Greg -- See complete headers for address, home page and phone numbers finger g...@lemis.com for PGP public key To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: several messages
Thank you for your reply. At what point should I set this socket option? I am assuming right after the socket is allocated?? I will try this and post my results tomorrow night. For those wondering, I cannot just execute Sendmail directly, there are many architectural reasons for this design... Thanks again, Wayne On Tue, 24 Aug 1999, Daniel O'Connor wrote: > Date: Tue, 24 Aug 1999 13:41:37 +0930 (CST) > From: Daniel O'Connor <[EMAIL PROTECTED]> > To: Wayne Cuddy <[EMAIL PROTECTED]> > Cc: FreeBSD Hackers List <[EMAIL PROTECTED]> > Subject: RE: network performance vs. linux on small transfers > > > On 24-Aug-99 Wayne Cuddy wrote: > > I REALLY want to use FreeBSD over Linux on this one and need some major help > > to get the performance out of FreeBSD. > > Tried setsockopt and TCP_NODELAY? > > >From netinet/tcp.h > #define TCP_NODELAY 0x01/* don't delay send to coalesce packets */ > > --- > Daniel O'Connor software and network engineer > for Genesis Software - http://www.gsoft.com.au > "The nice thing about standards is that there > are so many of them to choose from." > -- Andrew Tanenbaum > On Mon, 23 Aug 1999, David Greenman wrote: > Date: Mon, 23 Aug 1999 21:17:06 -0700 > From: David Greenman <[EMAIL PROTECTED]> > To: [EMAIL PROTECTED] > Cc: FreeBSD Hackers List <[EMAIL PROTECTED]> > Subject: Re: network performance vs. linux on small transfers > > >I am involved in a messaging system at work in which we need to send/receive > >large amounts of small (one line messages) SMTP messages. We are currently using >Sendmail 8.9.3 > >on HPUX. > > > >Our application sends messages down a FIFO to a daemon process that is reading from > >the FIFO. This process then connects to port 25 of the destination system and > >delivers the mail via SMTP. Currently the destination system is the local > >system so everything is done on one machine. > > > >Using HPUX we typically pass 5 messages a second. This system is a dual > >180Mhz K class server so this is surprisingly low performance for this system. > > > >When testing on FreeBSD 3.1 we also got 5 messages a second. This system is a > >500Mhz P3, this is also unacceptable performance. > > > >When we tested with Linux (kernel 2.2.5) we passed 15 messages a second > >consistently using the exact same P3 described above. > > > >Since the HPUX and FreeBSD numbers are so close I am wondering there is some > >performance tuning that I do not know about. Do you think the number might > >change if multiple hosts were used? > > > >The daemon that reads from the FIFO makes only one connection to the local > >Sendmail to deliver multiple messages in sequence. > > > > > >I REALLY want to use FreeBSD over Linux on this one and need some major help > >to get the performance out of FreeBSD. > >Are you setting the TCP_NODELAY socket option on the SMTP connection? If > not, then please do that and let me know if it fixes the problem or not. > > -DG > > David Greenman > Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org > Creator of high-performance Internet servers - http://www.terasolutions.com > Pave the road of life with opportunities. > To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Tue, Aug 24, 1999 at 12:52:10PM +0930, Greg Lehey wrote: > No, I think you're confusing opening and locking. It's something like > this: > > User 1User 2 > > open file open file > lock file read file (blocks) > diddle file > unlock file > read completes How about a little timing difference? User 1 User 2 open file read file open file lock file (blocks?) close file lock returnsopen file (blocks) diddle file unlock file scribble over User 1's changes What I'm getting at is that if User 2 has to do something special anyway, it might as well be using advisory locking. > > That seems extremely dangerous, given all the time that such a thing > > hasn't been around.. > > I've been using it for 22 years now. > > > who knows how many scripts and programs will now be vulnerable to > > hanging forever.. > > Why? There is a danger, of course, that user 1 will lock the file and > not unlock it. That's a badly written program, so you stop it. End > of hang. That's not what I meant. It hasn't been on FreeBSD, so FreeBSD is not designed to deal with it. I mentioned a couple of examples.. if I lock a bunch of files in my web space, does apache get a bunch of children stuck forever? Who knows what might get tripped up? -- Christopher Masto Senior Network Monkey NetMonger Communications ch...@netmonger.neti...@netmonger.nethttp://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Mon, Aug 23, 1999 at 11:16:21PM -0400, Chuck Robey wrote: > On Mon, 23 Aug 1999, Christopher Masto wrote: > > > Bleah.. I can't count the number of times I've seen idiotic code like: > > > > open file > > read data > > close file > > open file for write > > write data > > close file > > > > Mandatory locking of the type above doesn't force such a thing to work. > > What has that code you show above got to do with mandatory locking? > You completely missed the explicit locking calls that you have to make, > to get and release the locks. If you don't make the call, and you have > madatory locking, then your process will sleep until someone else > releases the lock; Exactly. You said that mandatory locking means that user A's correct use of locking means that user B doesn't have to be careful. That's not the case, since A can step in between B's read and write. A's mandatory lock doesn't help. I don't see the use for it. -- Christopher Masto Senior Network Monkey NetMonger Communications ch...@netmonger.neti...@netmonger.nethttp://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: network performance vs. linux on small transfers
>I am involved in a messaging system at work in which we need to send/receive >large amounts of small (one line messages) SMTP messages. We are currently using >Sendmail 8.9.3 >on HPUX. > >Our application sends messages down a FIFO to a daemon process that is reading from >the FIFO. This process then connects to port 25 of the destination system and >delivers the mail via SMTP. Currently the destination system is the local >system so everything is done on one machine. > >Using HPUX we typically pass 5 messages a second. This system is a dual >180Mhz K class server so this is surprisingly low performance for this system. > >When testing on FreeBSD 3.1 we also got 5 messages a second. This system is a >500Mhz P3, this is also unacceptable performance. > >When we tested with Linux (kernel 2.2.5) we passed 15 messages a second >consistently using the exact same P3 described above. > >Since the HPUX and FreeBSD numbers are so close I am wondering there is some >performance tuning that I do not know about. Do you think the number might >change if multiple hosts were used? > >The daemon that reads from the FIFO makes only one connection to the local >Sendmail to deliver multiple messages in sequence. > > >I REALLY want to use FreeBSD over Linux on this one and need some major help >to get the performance out of FreeBSD. Are you setting the TCP_NODELAY socket option on the SMTP connection? If not, then please do that and let me know if it fixes the problem or not. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Monday, 23 August 1999 at 23:11:30 -0400, Christopher Masto wrote: > On Mon, Aug 23, 1999 at 10:59:10PM -0400, Chuck Robey wrote: >>> Dunno about that.. if you're using advisory locking, you know to say >>> "lock the file, then read the data, do your calculation, write it out, >>> and unlock". This manditory locking sounds like an invitation for >>> disaster. "I don't need to pay attention to the details because >>> the kernel will take care of it for me." >>> >>> Actually, I don't really understand the paradigm. Two processes need >>> to safely update a file, so one of them aquires a mandatory lock, and >>> the other.. uh.. just blocks trying to open the file? How does it >>> know it's not the first one? >> >> It means that if user A puts data in (and follows locking procedure >> correctly) then he doesn't have to worry that user B might not be >> following correct locking procedure, because user B is mandatorily >> forced to follow the procedure. There isn't any added sloppiness, just >> a guarantee that if one user locks a file, no other rogues can get into >> it while the lock exists. > > Bleah.. I can't count the number of times I've seen idiotic code like: > > open file > read data > close file > open file for write > write data > close file > > Mandatory locking of the type above doesn't force such a thing to > work. I don't see a consistency problem in the code above, it's just inefficient. > Now that I've read the rest of the thread, I see that the meaning may > be that certain files are marked such that they can't be opened > without locking. No, I think you're confusing opening and locking. It's something like this: User 1 User 2 open file open file lock file read file (blocks) diddle file unlock file read completes In fact, fcntl locking is range locking, not file locking, so as long as the two users don't want to access the same part of the file. > That seems extremely dangerous, given all the time that such a thing > hasn't been around.. I've been using it for 22 years now. > who knows how many scripts and programs will now be vulnerable to > hanging forever.. Why? There is a danger, of course, that user 1 will lock the file and not unlock it. That's a badly written program, so you stop it. End of hang. Greg -- See complete headers for address, home page and phone numbers finger g...@lemis.com for PGP public key To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Mon, 23 Aug 1999, Christopher Masto wrote: > Bleah.. I can't count the number of times I've seen idiotic code like: > > open file > read data > close file > open file for write > write data > close file > > Mandatory locking of the type above doesn't force such a thing to work. What has that code you show above got to do with mandatory locking? You completely missed the explicit locking calls that you have to make, to get and release the locks. If you don't make the call, and you have madatory locking, then your process will sleep until someone else releases the lock; if you only have advisory locking, and you use the miscreant code you show, then indeed things will go awry. ---+--- Chuck Robey| Interests include any kind of voice or data chu...@picnic.mat.net | communications topic, C programming, and Unix. 213 Lakeside Drive Apt T-1 | Greenbelt, MD 20770| picnic.mat.net: FreeBSD/i386 (301) 220-2114 | jaunt.mat.net : FreeBSD/Alpha ---+--- To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: network performance vs. linux on small transfers
Wayne Cuddy wrote: > > I am involved in a messaging system at work in which we need to send/receive > large amounts of small (one line messages) SMTP messages. We are currently using >Sendmail 8.9.3 > on HPUX. > > Our application sends messages down a FIFO to a daemon process that is reading from > the FIFO. This process then connects to port 25 of the destination system and > delivers the mail via SMTP. Currently the destination system is the local > system so everything is done on one machine. > > Using HPUX we typically pass 5 messages a second. This system is a dual > 180Mhz K class server so this is surprisingly low performance for this system. > > When testing on FreeBSD 3.1 we also got 5 messages a second. This system is a > 500Mhz P3, this is also unacceptable performance. > > When we tested with Linux (kernel 2.2.5) we passed 15 messages a second > consistently using the exact same P3 described above. > > Since the HPUX and FreeBSD numbers are so close I am wondering there is some > performance tuning that I do not know about. Do you think the number might > change if multiple hosts were used? > > The daemon that reads from the FIFO makes only one connection to the local > Sendmail to deliver multiple messages in sequence. Why not just fork and exec sendmail instead? -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC http://softweyr.com/ [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Mon, Aug 23, 1999 at 10:59:10PM -0400, Chuck Robey wrote: > > Dunno about that.. if you're using advisory locking, you know to say > > "lock the file, then read the data, do your calculation, write it out, > > and unlock". This manditory locking sounds like an invitation for > > disaster. "I don't need to pay attention to the details because > > the kernel will take care of it for me." > > > > Actually, I don't really understand the paradigm. Two processes need > > to safely update a file, so one of them aquires a mandatory lock, and > > the other.. uh.. just blocks trying to open the file? How does it > > know it's not the first one? > > It means that if user A puts data in (and follows locking procedure > correctly) then he doesn't have to worry that user B might not be > following correct locking procedure, because user B is mandatorily > forced to follow the procedure. There isn't any added sloppiness, just > a guarantee that if one user locks a file, no other rogues can get into > it while the lock exists. Bleah.. I can't count the number of times I've seen idiotic code like: open file read data close file open file for write write data close file Mandatory locking of the type above doesn't force such a thing to work. Now that I've read the rest of the thread, I see that the meaning may be that certain files are marked such that they can't be opened without locking. That seems extremely dangerous, given all the time that such a thing hasn't been around.. who knows how many scripts and programs will now be vulnerable to hanging forever.. can I lock my maildrop? My web pages? My print spool? -- Christopher Masto Senior Network Monkey NetMonger Communications ch...@netmonger.neti...@netmonger.nethttp://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Mon, 23 Aug 1999, Christopher Masto wrote: > > The thing about well-intentioned but incorrect locking code is that > > it will appear to work fine, until it trips over the one code path > > where it forgets to lock some file that it should have locked. And > > even then, the code will "work" just fine, until multiple processes > > are accessing that file at the same time. > > > > I think it is appropriate for an operating system to provide an option > > such that *it* (the system) will enforce the locking, and not have to > > trust that all code-paths in all programs will do the right thing > > WRT advisory locking. > > Dunno about that.. if you're using advisory locking, you know to say > "lock the file, then read the data, do your calculation, write it out, > and unlock". This manditory locking sounds like an invitation for > disaster. "I don't need to pay attention to the details because > the kernel will take care of it for me." > > Actually, I don't really understand the paradigm. Two processes need > to safely update a file, so one of them aquires a mandatory lock, and > the other.. uh.. just blocks trying to open the file? How does it > know it's not the first one? It means that if user A puts data in (and follows locking procedure correctly) then he doesn't have to worry that user B might not be following correct locking procedure, because user B is mandatorily forced to follow the procedure. There isn't any added sloppiness, just a guarantee that if one user locks a file, no other rogues can get into it while the lock exists. ---+--- Chuck Robey| Interests include any kind of voice or data chu...@picnic.mat.net | communications topic, C programming, and Unix. 213 Lakeside Drive Apt T-1 | Greenbelt, MD 20770| picnic.mat.net: FreeBSD/i386 (301) 220-2114 | jaunt.mat.net : FreeBSD/Alpha ---+--- To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
network performance vs. linux on small transfers
I am involved in a messaging system at work in which we need to send/receive large amounts of small (one line messages) SMTP messages. We are currently using Sendmail 8.9.3 on HPUX. Our application sends messages down a FIFO to a daemon process that is reading from the FIFO. This process then connects to port 25 of the destination system and delivers the mail via SMTP. Currently the destination system is the local system so everything is done on one machine. Using HPUX we typically pass 5 messages a second. This system is a dual 180Mhz K class server so this is surprisingly low performance for this system. When testing on FreeBSD 3.1 we also got 5 messages a second. This system is a 500Mhz P3, this is also unacceptable performance. When we tested with Linux (kernel 2.2.5) we passed 15 messages a second consistently using the exact same P3 described above. Since the HPUX and FreeBSD numbers are so close I am wondering there is some performance tuning that I do not know about. Do you think the number might change if multiple hosts were used? The daemon that reads from the FIFO makes only one connection to the local Sendmail to deliver multiple messages in sequence. I REALLY want to use FreeBSD over Linux on this one and need some major help to get the performance out of FreeBSD. Much thanks in advance, Wayne Cuddy To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Monday, 23 August 1999 at 23:27:27 -0400, Christopher Masto wrote: > On Mon, Aug 23, 1999 at 11:16:21PM -0400, Chuck Robey wrote: >> On Mon, 23 Aug 1999, Christopher Masto wrote: >> >>> Bleah.. I can't count the number of times I've seen idiotic code like: >>> >>> open file >>> read data >>> close file >>> open file for write >>> write data >>> close file >>> >>> Mandatory locking of the type above doesn't force such a thing to work. >> >> What has that code you show above got to do with mandatory locking? >> You completely missed the explicit locking calls that you have to make, >> to get and release the locks. If you don't make the call, and you have >> madatory locking, then your process will sleep until someone else >> releases the lock; > > Exactly. You said that mandatory locking means that user A's correct > use of locking means that user B doesn't have to be careful. That's > not the case, since A can step in between B's read and write. B doesn't have to be careful about messing up A's transaction, like he doesn't need to be careful not to overwrite A's address space. If he wants his own transactions protected, he needs to do something about that. Greg -- See complete headers for address, home page and phone numbers finger [EMAIL PROTECTED] for PGP public key To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Monday, 23 August 1999 at 23:34:34 -0400, Christopher Masto wrote: > On Tue, Aug 24, 1999 at 12:52:10PM +0930, Greg Lehey wrote: >> No, I think you're confusing opening and locking. It's something like >> this: >> >> User 1 User 2 >> >> open fileopen file >> lock fileread file (blocks) >> diddle file >> unlock file >> read completes > > How about a little timing difference? > > User 1User 2 > > open file > read file > open file > lock file (blocks?) > close file > lock returnsopen file (blocks) > diddle file > unlock file > scribble over User 1's changes That's OK. In my scenario, user 2 can't know about what user 1 is doing. Typically there'll be multiple accesses. Since you mentioned mail, let's look at it, since that's where things really *do* use (advisory) locking. When I save my mail folder, I need to ensure that sendmail doesn't choose that moment to deliver a new message; otherwise it might disappear completely. Mailers and sendmail do have an agreement of some kind. But what if I want to merge the contents of another mail folder: cat oldmail >>/var/mail/grog That works, but it's playing with fire: if sendmail is delivering a message at the same time, it won't see me, and my cat doesn't get a lock beforehand, so both an incoming message and part of my mail folder could end up getting written to the same location. With mandatory locking, it would work, transparently. > What I'm getting at is that if User 2 has to do something special > anyway, it might as well be using advisory locking. What I'm getting at is that User 2 doesn't have to do anything different. >>> who knows how many scripts and programs will now be vulnerable to >>> hanging forever.. >> >> Why? There is a danger, of course, that user 1 will lock the file and >> not unlock it. That's a badly written program, so you stop it. End >> of hang. > > That's not what I meant. It hasn't been on FreeBSD, so FreeBSD is not > designed to deal with it. Obviously. > I mentioned a couple of examples.. if I lock a bunch of files in my > web space, does apache get a bunch of children stuck forever? Only as long as you keep them locked. And yes, you shouldn't go locking files longer than you need them. > Who knows what might get tripped up? Nobody. Do you know where your next SIGSEGV is coming from? Greg -- See complete headers for address, home page and phone numbers finger [EMAIL PROTECTED] for PGP public key To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Tue, Aug 24, 1999 at 12:52:10PM +0930, Greg Lehey wrote: > No, I think you're confusing opening and locking. It's something like > this: > > User 1User 2 > > open file open file > lock file read file (blocks) > diddle file > unlock file > read completes How about a little timing difference? User 1 User 2 open file read file open file lock file (blocks?) close file lock returnsopen file (blocks) diddle file unlock file scribble over User 1's changes What I'm getting at is that if User 2 has to do something special anyway, it might as well be using advisory locking. > > That seems extremely dangerous, given all the time that such a thing > > hasn't been around.. > > I've been using it for 22 years now. > > > who knows how many scripts and programs will now be vulnerable to > > hanging forever.. > > Why? There is a danger, of course, that user 1 will lock the file and > not unlock it. That's a badly written program, so you stop it. End > of hang. That's not what I meant. It hasn't been on FreeBSD, so FreeBSD is not designed to deal with it. I mentioned a couple of examples.. if I lock a bunch of files in my web space, does apache get a bunch of children stuck forever? Who knows what might get tripped up? -- Christopher Masto Senior Network Monkey NetMonger Communications [EMAIL PROTECTED][EMAIL PROTECTED]http://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
> The thing about well-intentioned but incorrect locking code is that > it will appear to work fine, until it trips over the one code path > where it forgets to lock some file that it should have locked. And > even then, the code will "work" just fine, until multiple processes > are accessing that file at the same time. > > I think it is appropriate for an operating system to provide an option > such that *it* (the system) will enforce the locking, and not have to > trust that all code-paths in all programs will do the right thing > WRT advisory locking. Dunno about that.. if you're using advisory locking, you know to say "lock the file, then read the data, do your calculation, write it out, and unlock". This manditory locking sounds like an invitation for disaster. "I don't need to pay attention to the details because the kernel will take care of it for me." Actually, I don't really understand the paradigm. Two processes need to safely update a file, so one of them aquires a mandatory lock, and the other.. uh.. just blocks trying to open the file? How does it know it's not the first one? -- Christopher Masto Senior Network Monkey NetMonger Communications ch...@netmonger.neti...@netmonger.nethttp://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Mon, Aug 23, 1999 at 11:16:21PM -0400, Chuck Robey wrote: > On Mon, 23 Aug 1999, Christopher Masto wrote: > > > Bleah.. I can't count the number of times I've seen idiotic code like: > > > > open file > > read data > > close file > > open file for write > > write data > > close file > > > > Mandatory locking of the type above doesn't force such a thing to work. > > What has that code you show above got to do with mandatory locking? > You completely missed the explicit locking calls that you have to make, > to get and release the locks. If you don't make the call, and you have > madatory locking, then your process will sleep until someone else > releases the lock; Exactly. You said that mandatory locking means that user A's correct use of locking means that user B doesn't have to be careful. That's not the case, since A can step in between B's read and write. A's mandatory lock doesn't help. I don't see the use for it. -- Christopher Masto Senior Network Monkey NetMonger Communications [EMAIL PROTECTED][EMAIL PROTECTED]http://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Monday, 23 August 1999 at 23:11:30 -0400, Christopher Masto wrote: > On Mon, Aug 23, 1999 at 10:59:10PM -0400, Chuck Robey wrote: >>> Dunno about that.. if you're using advisory locking, you know to say >>> "lock the file, then read the data, do your calculation, write it out, >>> and unlock". This manditory locking sounds like an invitation for >>> disaster. "I don't need to pay attention to the details because >>> the kernel will take care of it for me." >>> >>> Actually, I don't really understand the paradigm. Two processes need >>> to safely update a file, so one of them aquires a mandatory lock, and >>> the other.. uh.. just blocks trying to open the file? How does it >>> know it's not the first one? >> >> It means that if user A puts data in (and follows locking procedure >> correctly) then he doesn't have to worry that user B might not be >> following correct locking procedure, because user B is mandatorily >> forced to follow the procedure. There isn't any added sloppiness, just >> a guarantee that if one user locks a file, no other rogues can get into >> it while the lock exists. > > Bleah.. I can't count the number of times I've seen idiotic code like: > > open file > read data > close file > open file for write > write data > close file > > Mandatory locking of the type above doesn't force such a thing to > work. I don't see a consistency problem in the code above, it's just inefficient. > Now that I've read the rest of the thread, I see that the meaning may > be that certain files are marked such that they can't be opened > without locking. No, I think you're confusing opening and locking. It's something like this: User 1 User 2 open file open file lock file read file (blocks) diddle file unlock file read completes In fact, fcntl locking is range locking, not file locking, so as long as the two users don't want to access the same part of the file. > That seems extremely dangerous, given all the time that such a thing > hasn't been around.. I've been using it for 22 years now. > who knows how many scripts and programs will now be vulnerable to > hanging forever.. Why? There is a danger, of course, that user 1 will lock the file and not unlock it. That's a badly written program, so you stop it. End of hang. Greg -- See complete headers for address, home page and phone numbers finger [EMAIL PROTECTED] for PGP public key To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Mon, 23 Aug 1999, Christopher Masto wrote: > Bleah.. I can't count the number of times I've seen idiotic code like: > > open file > read data > close file > open file for write > write data > close file > > Mandatory locking of the type above doesn't force such a thing to work. What has that code you show above got to do with mandatory locking? You completely missed the explicit locking calls that you have to make, to get and release the locks. If you don't make the call, and you have madatory locking, then your process will sleep until someone else releases the lock; if you only have advisory locking, and you use the miscreant code you show, then indeed things will go awry. ---+--- Chuck Robey| Interests include any kind of voice or data [EMAIL PROTECTED] | communications topic, C programming, and Unix. 213 Lakeside Drive Apt T-1 | Greenbelt, MD 20770| picnic.mat.net: FreeBSD/i386 (301) 220-2114 | jaunt.mat.net : FreeBSD/Alpha ---+--- To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: setting up -STABLE for hack contest
On Mon, 23 Aug 1999 17:44:47 -0700 "Dave Walton" wrote: > Hm, just did that. While reading up on nmap, I saw this: > > "TCP Initial Window -- This simply involves checking the window >size on returned packets. [...] In their "completely rewritten" >TCP stack for NT5, Microsoft uses 0x402E. Interestingly, that is >exactly the number used by OpenBSD and FreeBSD." > > Gee, I wonder how that happened... It's not "interesting" at all. Lots of systems default to a 16k receive window. -- Jason R. Thorpe To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Mon, Aug 23, 1999 at 10:59:10PM -0400, Chuck Robey wrote: > > Dunno about that.. if you're using advisory locking, you know to say > > "lock the file, then read the data, do your calculation, write it out, > > and unlock". This manditory locking sounds like an invitation for > > disaster. "I don't need to pay attention to the details because > > the kernel will take care of it for me." > > > > Actually, I don't really understand the paradigm. Two processes need > > to safely update a file, so one of them aquires a mandatory lock, and > > the other.. uh.. just blocks trying to open the file? How does it > > know it's not the first one? > > It means that if user A puts data in (and follows locking procedure > correctly) then he doesn't have to worry that user B might not be > following correct locking procedure, because user B is mandatorily > forced to follow the procedure. There isn't any added sloppiness, just > a guarantee that if one user locks a file, no other rogues can get into > it while the lock exists. Bleah.. I can't count the number of times I've seen idiotic code like: open file read data close file open file for write write data close file Mandatory locking of the type above doesn't force such a thing to work. Now that I've read the rest of the thread, I see that the meaning may be that certain files are marked such that they can't be opened without locking. That seems extremely dangerous, given all the time that such a thing hasn't been around.. who knows how many scripts and programs will now be vulnerable to hanging forever.. can I lock my maildrop? My web pages? My print spool? -- Christopher Masto Senior Network Monkey NetMonger Communications [EMAIL PROTECTED][EMAIL PROTECTED]http://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Mon, 23 Aug 1999, Christopher Masto wrote: > > The thing about well-intentioned but incorrect locking code is that > > it will appear to work fine, until it trips over the one code path > > where it forgets to lock some file that it should have locked. And > > even then, the code will "work" just fine, until multiple processes > > are accessing that file at the same time. > > > > I think it is appropriate for an operating system to provide an option > > such that *it* (the system) will enforce the locking, and not have to > > trust that all code-paths in all programs will do the right thing > > WRT advisory locking. > > Dunno about that.. if you're using advisory locking, you know to say > "lock the file, then read the data, do your calculation, write it out, > and unlock". This manditory locking sounds like an invitation for > disaster. "I don't need to pay attention to the details because > the kernel will take care of it for me." > > Actually, I don't really understand the paradigm. Two processes need > to safely update a file, so one of them aquires a mandatory lock, and > the other.. uh.. just blocks trying to open the file? How does it > know it's not the first one? It means that if user A puts data in (and follows locking procedure correctly) then he doesn't have to worry that user B might not be following correct locking procedure, because user B is mandatorily forced to follow the procedure. There isn't any added sloppiness, just a guarantee that if one user locks a file, no other rogues can get into it while the lock exists. ---+--- Chuck Robey| Interests include any kind of voice or data [EMAIL PROTECTED] | communications topic, C programming, and Unix. 213 Lakeside Drive Apt T-1 | Greenbelt, MD 20770| picnic.mat.net: FreeBSD/i386 (301) 220-2114 | jaunt.mat.net : FreeBSD/Alpha ---+--- To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: anybody love qsort.c?
On Mon, Aug 23, 1999 at 12:28:32AM -0700, Christopher Seiwald wrote: > > The alteration that I've tried and tested is to have the isort bail > back to qsort if it does more than N swaps. I put N at 1024, which Perhaps a ratio: #comparisons : # swaps If the ratio gets too high, then bail. -- This is my .signature which gets appended to the end of my messages. To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
> The thing about well-intentioned but incorrect locking code is that > it will appear to work fine, until it trips over the one code path > where it forgets to lock some file that it should have locked. And > even then, the code will "work" just fine, until multiple processes > are accessing that file at the same time. > > I think it is appropriate for an operating system to provide an option > such that *it* (the system) will enforce the locking, and not have to > trust that all code-paths in all programs will do the right thing > WRT advisory locking. Dunno about that.. if you're using advisory locking, you know to say "lock the file, then read the data, do your calculation, write it out, and unlock". This manditory locking sounds like an invitation for disaster. "I don't need to pay attention to the details because the kernel will take care of it for me." Actually, I don't really understand the paradigm. Two processes need to safely update a file, so one of them aquires a mandatory lock, and the other.. uh.. just blocks trying to open the file? How does it know it's not the first one? -- Christopher Masto Senior Network Monkey NetMonger Communications [EMAIL PROTECTED][EMAIL PROTECTED]http://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Mon, Aug 23, 1999 at 10:12:38PM +0200, Mark Murray wrote: > > In process-space, this is the kernel. In file-space, this should > be root. Processes that require mandatory locking must revoke > superuser before attempting locks. I don't like restricting the breaking of mandatory locks to the superuser. It could be restricted to specific users (say file owner + root)... -- This is my .signature which gets appended to the end of my messages. To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Mon, Aug 23, 1999 at 03:28:01PM -0400, Garance A Drosihn wrote: > > Anyway, I am also puzzled as to why there would be much objection > to the option of mandatory locking. My initial systems-programming If you provide mandatory locks that can be broken, then many of the objections may disappear... Providing mandatory locks that can be broken would be rather useful, I think. Mandatory locks that cannot be broken are another matter... -- This is my .signature which gets appended to the end of my messages. To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: setting up -STABLE for hack contest
On Mon, 23 Aug 1999 17:44:47 -0700 "Dave Walton" <[EMAIL PROTECTED]> wrote: > Hm, just did that. While reading up on nmap, I saw this: > > "TCP Initial Window -- This simply involves checking the window >size on returned packets. [...] In their "completely rewritten" >TCP stack for NT5, Microsoft uses 0x402E. Interestingly, that is >exactly the number used by OpenBSD and FreeBSD." > > Gee, I wonder how that happened... It's not "interesting" at all. Lots of systems default to a 16k receive window. -- Jason R. Thorpe <[EMAIL PROTECTED]> To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: anybody love qsort.c?
On Mon, Aug 23, 1999 at 12:28:32AM -0700, Christopher Seiwald wrote: > > The alteration that I've tried and tested is to have the isort bail > back to qsort if it does more than N swaps. I put N at 1024, which Perhaps a ratio: #comparisons : # swaps If the ratio gets too high, then bail. -- This is my .signature which gets appended to the end of my messages. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
Hi Greg, hackers list, I don't want to express an opinion about the need or otherwise for mandatory locking, but I would appreciate a teensy clarification of the problem domain: On Mon, Aug 23, 1999 at 05:43:45PM +0930, Greg Lehey wrote: > To write a block to a RAID-5 device, you need to: > > 1. Read the old data into a temporary buffer. > 2. Read the old parity data corresponding to the data into a > temporary buffer. > 3. XOR the two, storing the result in one of the temporary buffers. > 4. XOR the result with the data which is to be written. > 5. Write the data block. > 6. Write the parity block. Are you suggesting that random user processes have to do all of this every time that they access a vinum drive? I thought that access was mediated by a driver or server process. Isn't this process in a position to ensure the integrity of the transaction without any help from locking mechanisms? If some major maintenance utility needs to run on the raw device, during which time the state is inconsistent as far as user processes are concerned, isn't it sufficient to remove their read priveliges? Sorry if these are dumb questions. I have had no experience at file system or data-base applications, but I do want to learn as much about them as I can. -- Andrew To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: setting up -STABLE for hack contest
Geoff Rehmet writes: > Also have a look at ports/security/nmap, and go to > www.insecure.org. Hm, just did that. While reading up on nmap, I saw this: "TCP Initial Window -- This simply involves checking the window size on returned packets. [...] In their "completely rewritten" TCP stack for NT5, Microsoft uses 0x402E. Interestingly, that is exactly the number used by OpenBSD and FreeBSD." Gee, I wonder how that happened... Dave -- Dave Walton Webmaster, Postmaster Nordic Entertainment Worldwide wal...@nordicdms.com http://www.nordicdms.com -- To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Mon, Aug 23, 1999 at 10:12:38PM +0200, Mark Murray wrote: > > In process-space, this is the kernel. In file-space, this should > be root. Processes that require mandatory locking must revoke > superuser before attempting locks. I don't like restricting the breaking of mandatory locks to the superuser. It could be restricted to specific users (say file owner + root)... -- This is my .signature which gets appended to the end of my messages. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: [freebsdcon] radisson reservation
>> > Unfortunately, you have to call the local hotel to get reservations, and >> > not the toll-free national hotline. The hotel in Berkeley doesn't have a >> > toll free number, so after sitting on hold with the Berkeley Radission for >> > 15 minutes, burning long distance money, I decided to call the national >> > reservations hotline. They told me I had to call the reservations desk at >> > the Berkeley Radission. >> >> I called the 800 number in the FreeBSDcon brochure, it worked fine. > >They should stick that magic 800 number on the FreeBSDcon web site. > >> Regretfully, I now won't be attending the conference so now I get to find >> out just how well they cancel reservations. > >Bummer. Thank you very much for all of you gave me the tip, I will FAX the hotel with mentioning "walnut creek cdrom". itojun To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Mon, Aug 23, 1999 at 03:28:01PM -0400, Garance A Drosihn wrote: > > Anyway, I am also puzzled as to why there would be much objection > to the option of mandatory locking. My initial systems-programming If you provide mandatory locks that can be broken, then many of the objections may disappear... Providing mandatory locks that can be broken would be rather useful, I think. Mandatory locks that cannot be broken are another matter... -- This is my .signature which gets appended to the end of my messages. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
Hi Greg, hackers list, I don't want to express an opinion about the need or otherwise for mandatory locking, but I would appreciate a teensy clarification of the problem domain: On Mon, Aug 23, 1999 at 05:43:45PM +0930, Greg Lehey wrote: > To write a block to a RAID-5 device, you need to: > > 1. Read the old data into a temporary buffer. > 2. Read the old parity data corresponding to the data into a > temporary buffer. > 3. XOR the two, storing the result in one of the temporary buffers. > 4. XOR the result with the data which is to be written. > 5. Write the data block. > 6. Write the parity block. Are you suggesting that random user processes have to do all of this every time that they access a vinum drive? I thought that access was mediated by a driver or server process. Isn't this process in a position to ensure the integrity of the transaction without any help from locking mechanisms? If some major maintenance utility needs to run on the raw device, during which time the state is inconsistent as far as user processes are concerned, isn't it sufficient to remove their read priveliges? Sorry if these are dumb questions. I have had no experience at file system or data-base applications, but I do want to learn as much about them as I can. -- Andrew To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Monday, 23 August 1999 at 15:28:01 -0400, Garance A Drosihn wrote: > At 3:28 PM +0930 8/23/99, Greg Lehey wrote: >> I'm a little surprised that there's any objection to the concept >> of mandatory locking. In transaction processing, locking is not >> optional, and if any process at all can access a file or set of... > > For what it's worth, I also like the idea of mandatory locking. > It's important to think through some of the implementation details, > but I would also like to see some way to specify mandatory locking > in at least some situations. > > I'm not particularly thrilled with the idea of keying it off chmod > bits, though. That seems like a recipe for disaster. Remember where that came from: System V. I don't like it either, but if we're going to do it, we should offer this method (maybe as an option) for compatibility reasons. > Anyway, I am also puzzled as to why there would be much objection to > the option of mandatory locking. My initial systems-programming > experience was on a mainframe OS where mandatory locking was the > NORM, and you had to go out of your way to avoid locking. It seemed > to work quite well in my experience. Ditto in all points. I think that the people who are objecting are seeing potential rather than real problems. Obviously you don't apply mandatory locking to every file, but there are many cases where it makes sense. On Monday, 23 August 1999 at 17:47:46 -0400, Garance A Drosihn wrote: > At 10:12 PM +0200 8/23/99, Mark Murray wrote: >> Folk are all skirting around a very convenient (and necessary) >> loophole; in cases where there _is_ mandatory locking, there >> is always some meta-user which is allowed to violate this. > > If we include non-unix systems in the discussion, this isn't > always true. In the mainframe OS that I used to work on, there was > no meta-user who could just ignore locks set by other processes. > The super-user could find which process had a file locked, and > kill that process (thus removing the lock). Or the super-user > could run a program which modified the in-core locking table > so the file did not appear to be locked. Exactly. The reason why mandatory locking isn't a problem is because it's possible to kill processes holding the locks. It should be possible to find out who these processes are. > However, ordinary programs run by that super user did not have > any magic power to ignore mandatory locks set by some other > process. > > It is true that nobody is running that mainframe OS anymore... :-) They're running similar OSs. Tandem's NSK, for example, has always had mandatory locking. > I'm just saying we COULD (and maybe "should"?) implement this > such that even root has to honor mandatory locks. Root (or some > thing) would have a way to get around or cancel the mandatory > lock, but "standard" programs run by root should not bypass the > mandatory locking. (IMO) I don't think that it should be necessary for root to have this override. The tendency in the last few years has been to lessen root's authority, not increase it. And remember, just because people on this forum haven't had much experience with mandatory locking doesn't mean that others don't: it's been in use for years, and people *don't* have (many) problems with deadlocks. The real issue here is that (mandatory) locking is standard on just about every operating system nowadays. I'd guess that even Microsoft has it. The lack of support on FreeBSD doesn't improve the system. I've done some searching on the web, and came up with a lot of stuff. http://homer.njit.edu:8000/cis332/flock.html and http://miller.cs.wm.edu/lxr3.linux/http/source/Documentation/mandatory.txt seem to be the most interesting. Greg -- See complete headers for address, home page and phone numbers finger g...@lemis.com for PGP public key To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: setting up -STABLE for hack contest
Geoff Rehmet writes: > Also have a look at ports/security/nmap, and go to > www.insecure.org. Hm, just did that. While reading up on nmap, I saw this: "TCP Initial Window -- This simply involves checking the window size on returned packets. [...] In their "completely rewritten" TCP stack for NT5, Microsoft uses 0x402E. Interestingly, that is exactly the number used by OpenBSD and FreeBSD." Gee, I wonder how that happened... Dave -- Dave Walton Webmaster, Postmaster Nordic Entertainment Worldwide [EMAIL PROTECTED] http://www.nordicdms.com -- To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: [freebsdcon] radisson reservation
>> > Unfortunately, you have to call the local hotel to get reservations, and >> > not the toll-free national hotline. The hotel in Berkeley doesn't have a >> > toll free number, so after sitting on hold with the Berkeley Radission for >> > 15 minutes, burning long distance money, I decided to call the national >> > reservations hotline. They told me I had to call the reservations desk at >> > the Berkeley Radission. >> >> I called the 800 number in the FreeBSDcon brochure, it worked fine. > >They should stick that magic 800 number on the FreeBSDcon web site. > >> Regretfully, I now won't be attending the conference so now I get to find >> out just how well they cancel reservations. > >Bummer. Thank you very much for all of you gave me the tip, I will FAX the hotel with mentioning "walnut creek cdrom". itojun To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
Chuck Robey wrote: > > I think Garrett's fears are of folks unwittingly wedging machines too > easily, so real mandatory locking ought to be restricted to programs > that root can set up. And those fears are well-founded, but your proposed solution just creates another set of bottlenecks. Making mandatory locks available to any process and giving root an avenue by which it can revoke the locks, by whatever means, is a better solution. SIGKILL seems like an ideal candidate to me. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC http://softweyr.com/ w...@softweyr.com To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Monday, 23 August 1999 at 15:28:01 -0400, Garance A Drosihn wrote: > At 3:28 PM +0930 8/23/99, Greg Lehey wrote: >> I'm a little surprised that there's any objection to the concept >> of mandatory locking. In transaction processing, locking is not >> optional, and if any process at all can access a file or set of... > > For what it's worth, I also like the idea of mandatory locking. > It's important to think through some of the implementation details, > but I would also like to see some way to specify mandatory locking > in at least some situations. > > I'm not particularly thrilled with the idea of keying it off chmod > bits, though. That seems like a recipe for disaster. Remember where that came from: System V. I don't like it either, but if we're going to do it, we should offer this method (maybe as an option) for compatibility reasons. > Anyway, I am also puzzled as to why there would be much objection to > the option of mandatory locking. My initial systems-programming > experience was on a mainframe OS where mandatory locking was the > NORM, and you had to go out of your way to avoid locking. It seemed > to work quite well in my experience. Ditto in all points. I think that the people who are objecting are seeing potential rather than real problems. Obviously you don't apply mandatory locking to every file, but there are many cases where it makes sense. On Monday, 23 August 1999 at 17:47:46 -0400, Garance A Drosihn wrote: > At 10:12 PM +0200 8/23/99, Mark Murray wrote: >> Folk are all skirting around a very convenient (and necessary) >> loophole; in cases where there _is_ mandatory locking, there >> is always some meta-user which is allowed to violate this. > > If we include non-unix systems in the discussion, this isn't > always true. In the mainframe OS that I used to work on, there was > no meta-user who could just ignore locks set by other processes. > The super-user could find which process had a file locked, and > kill that process (thus removing the lock). Or the super-user > could run a program which modified the in-core locking table > so the file did not appear to be locked. Exactly. The reason why mandatory locking isn't a problem is because it's possible to kill processes holding the locks. It should be possible to find out who these processes are. > However, ordinary programs run by that super user did not have > any magic power to ignore mandatory locks set by some other > process. > > It is true that nobody is running that mainframe OS anymore... :-) They're running similar OSs. Tandem's NSK, for example, has always had mandatory locking. > I'm just saying we COULD (and maybe "should"?) implement this > such that even root has to honor mandatory locks. Root (or some > thing) would have a way to get around or cancel the mandatory > lock, but "standard" programs run by root should not bypass the > mandatory locking. (IMO) I don't think that it should be necessary for root to have this override. The tendency in the last few years has been to lessen root's authority, not increase it. And remember, just because people on this forum haven't had much experience with mandatory locking doesn't mean that others don't: it's been in use for years, and people *don't* have (many) problems with deadlocks. The real issue here is that (mandatory) locking is standard on just about every operating system nowadays. I'd guess that even Microsoft has it. The lack of support on FreeBSD doesn't improve the system. I've done some searching on the web, and came up with a lot of stuff. http://homer.njit.edu:8000/cis332/flock.html and http://miller.cs.wm.edu/lxr3.linux/http/source/Documentation/mandatory.txt seem to be the most interesting. Greg -- See complete headers for address, home page and phone numbers finger [EMAIL PROTECTED] for PGP public key To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Mon, 23 Aug 1999, Garance A Drosihn wrote: > At 11:29 AM -0400 8/23/99, Chuck Robey wrote: > >I think mandatory locking should exist, but only be available to root. > >If a program needs this, it must run with root privs, so that ordinary > >users cannot wedge the machine, but (as usual) root can shoot himself > >in the foot (traditional Unix methodology). > > I don't think we want to force people into running their program as > root just to get mandatory locking. Perhaps there would be a program > with root-privs which would have to be run to register files which > will have mandatory locking, but the program which manipulates those > files shouldn't have to run as root. There are other ways to access the rights, such as sockets, pipes, etc. You write a server which runs as root and can lock, and the clients, running with clients privs, make service requests. If you restrict locking to root, then even if someone manages to wedge his machine, he's not doing anything that an idiot with root and the rm command can't do much worse. I think Garrett's fears are of folks unwittingly wedging machines too easily, so real mandatory locking ought to be restricted to programs that root can set up. > > > --- > Garance Alistair Drosehn = g...@eclipse.acs.rpi.edu > Senior Systems Programmer or dro...@rpi.edu > Rensselaer Polytechnic Institute > +--- Chuck Robey | Interests include any kind of voice or data chu...@picnic.mat.net | communications topic, C programming, and Unix. 213 Lakeside Drive Apt T-1 | Greenbelt, MD 20770 | I run picnic and jaunt, both FreeBSD-current. (301) 220-2114 | +--- To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
Chuck Robey wrote: > > I think Garrett's fears are of folks unwittingly wedging machines too > easily, so real mandatory locking ought to be restricted to programs > that root can set up. And those fears are well-founded, but your proposed solution just creates another set of bottlenecks. Making mandatory locks available to any process and giving root an avenue by which it can revoke the locks, by whatever means, is a better solution. SIGKILL seems like an ideal candidate to me. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC http://softweyr.com/ [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Mon, 23 Aug 1999, Garance A Drosihn wrote: > At 11:29 AM -0400 8/23/99, Chuck Robey wrote: > >I think mandatory locking should exist, but only be available to root. > >If a program needs this, it must run with root privs, so that ordinary > >users cannot wedge the machine, but (as usual) root can shoot himself > >in the foot (traditional Unix methodology). > > I don't think we want to force people into running their program as > root just to get mandatory locking. Perhaps there would be a program > with root-privs which would have to be run to register files which > will have mandatory locking, but the program which manipulates those > files shouldn't have to run as root. There are other ways to access the rights, such as sockets, pipes, etc. You write a server which runs as root and can lock, and the clients, running with clients privs, make service requests. If you restrict locking to root, then even if someone manages to wedge his machine, he's not doing anything that an idiot with root and the rm command can't do much worse. I think Garrett's fears are of folks unwittingly wedging machines too easily, so real mandatory locking ought to be restricted to programs that root can set up. > > > --- > Garance Alistair Drosehn = [EMAIL PROTECTED] > Senior Systems Programmer or [EMAIL PROTECTED] > Rensselaer Polytechnic Institute > +--- Chuck Robey | Interests include any kind of voice or data [EMAIL PROTECTED] | communications topic, C programming, and Unix. 213 Lakeside Drive Apt T-1 | Greenbelt, MD 20770 | I run picnic and jaunt, both FreeBSD-current. (301) 220-2114 | +--- To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
At 10:12 PM +0200 8/23/99, Mark Murray wrote: Folk are all skirting around a very convenient (and necessary) loophole; in cases where there _is_ mandatory locking, there is always some meta-user which is allowed to violate this. If we include non-unix systems in the discussion, this isn't always true. In the mainframe OS that I used to work on, there was no meta-user who could just ignore locks set by other processes. The super-user could find which process had a file locked, and kill that process (thus removing the lock). Or the super-user could run a program which modified the in-core locking table so the file did not appear to be locked. However, ordinary programs run by that super user did not have any magic power to ignore mandatory locks set by some other process. It is true that nobody is running that mainframe OS anymore... :-) I'm just saying we COULD (and maybe "should"?) implement this such that even root has to honor mandatory locks. Root (or some thing) would have a way to get around or cancel the mandatory lock, but "standard" programs run by root should not bypass the mandatory locking. (IMO) --- Garance Alistair Drosehn = g...@eclipse.acs.rpi.edu Senior Systems Programmer or dro...@rpi.edu Rensselaer Polytechnic Institute To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: L440GX+ Server Board
Has anyone else gotten this server board to work? I've got an N440BX and have been considering getting the L440GX+ but haven't because I don't know if it works.. Kevin On Mon, 23 Aug 1999, Luiz Morte da Costa Junior wrote: > > Hi list, > > About the problem bellow, I bought a 2940 Adaptec Ultra2 Wide SCSI > controller, but it didn't work too. > > I wrote to Justin T. Gibbs and he told me that my problem is not SCSI. > > Somebody has any idea? > > []s, > > Luiz Morte da Costa Junior > Analista de RedesE-mail: mo...@correionet.com.br > Telefone: +55 19 754-2532Fax: +55 19 255-7576 > CorreioNet - Correio Popular Campinas - SP - Brazil > > > On Sun, 22 Aug 1999, Luiz Morte da Costa Junior wrote: > > > > > Hi all, > > > > I have a problem with a server running a FreeBSD 3.2. > > > > My Server has a L440GX+ Serber Board (intel), with network card 10/100, > > SCSI AIC 7896 (80MB/s), 2 SCSI disk with 9GB (80MB/s), 2 pentium III > > 450Mhz (not overclocked). The NIC and SCSI are onboard. > > > > I recompiled a kernel to SMP, and it worked. The server is ok, but when I > > run a comand with disk access (whith vipw or mysql), the performance of > > server goes down. My server stays very very very slow. If I use pine to > > read my messages, it doesn't work. When the comand finishes, the server > > stays "ok" again. > > > > I recompiled the kernel with "maxusers 128", but it doesn't work. > > > > My SCSI cable has a terminator and the scsi setup is ok (I think :) ). > > > > The dmesg command output is in attchmnt. > > > > I appreciate any help. > > > > []s, > > > > Luiz Morte da Costa Junior > > Analista de RedesE-mail: mo...@correionet.com.br > > Telefone: +55 19 754-2532Fax: +55 19 255-7576 > > CorreioNet - Correio Popular Campinas - SP - Brazil > > > > []s, > > Luiz Morte da Costa Junior > Analista de RedesE-mail: mo...@correionet.com.br > Telefone: +55 19 754-2532Fax: +55 19 255-7576 > CorreioNet - Correio Popular Campinas - SP - Brazil > > > > To Unsubscribe: send mail to majord...@freebsd.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
At 10:12 PM +0200 8/23/99, Mark Murray wrote: >Folk are all skirting around a very convenient (and necessary) >loophole; in cases where there _is_ mandatory locking, there >is always some meta-user which is allowed to violate this. If we include non-unix systems in the discussion, this isn't always true. In the mainframe OS that I used to work on, there was no meta-user who could just ignore locks set by other processes. The super-user could find which process had a file locked, and kill that process (thus removing the lock). Or the super-user could run a program which modified the in-core locking table so the file did not appear to be locked. However, ordinary programs run by that super user did not have any magic power to ignore mandatory locks set by some other process. It is true that nobody is running that mainframe OS anymore... :-) I'm just saying we COULD (and maybe "should"?) implement this such that even root has to honor mandatory locks. Root (or some thing) would have a way to get around or cancel the mandatory lock, but "standard" programs run by root should not bypass the mandatory locking. (IMO) --- Garance Alistair Drosehn = [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Institute To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: proposed change for /etc/periodic/* scripts
On 23-Aug-99 Cillian Sharkey wrote: > > yes perhaps an /etc/periodic.conf would be good, to control the level > of verbosity and/or set options for each script ? I've hacked periodic here so that the scripts can be turned off with knobs in a periodic.conf file. This would simplify customizing new installitions - one no longer needs to add exit 0 to scripts. Duncan --- Duncan Barclay | God smiles upon the little children, d...@ragnet.demon.co.uk | the alcoholics, and the permanently stoned. To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
