Re: [lkml]Re: VM problems still in 2.2.18
- Original Message - From: "Alan Cox" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Cc: <[EMAIL PROTECTED]> Sent: Thursday, December 14, 2000 2:38 PM Subject: Re: [lkml]Re: VM problems still in 2.2.18 > > I think Andrea just earned his official God status ;) Just wanted to quickly report that after applying Andreas patch, the problem has been fixed here as well. I've got two machines here that were previously unable to stay up more than a day, been pounding on them since Friday without any problems. Great! :) -- Mark grep me no patterns and I'll tell you no lines. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [lkml]Re: VM problems still in 2.2.18
On Sat, Dec 16, 2000 at 02:49:40AM -0800, Chris Mason wrote: > GFP_BUFFER. As far as I know that should be safe, but the change is Yes that's ok. Andrea - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [lkml]Re: VM problems still in 2.2.18
On Fri, 15 Dec 2000, Alan Cox wrote: > > Yes, the same `current' context must run the down/up pair of calls and as you > > said it is legal to rely on it on all the places it's used. > > I assume thats not an issue to reiserfs ? > I don't think so. There are two places reiserfs calls down/up. In our file->release() callback, and in the ioctl we added for lilo so it can upack files with tails. Neither calls copy_from/to_user or does an allocation other than GFP_BUFFER. As far as I know that should be safe, but the change is trivial if we need to make it. -chris - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [lkml]Re: VM problems still in 2.2.18
> Now we know when we can block so we can run f_ops->write ourselfs that's also > more efficient in terms of both performance and also memory pressure during > swap of course. Yep > As said reiserfs AFIK didn't need any change, so only VFS is using > fs_down/fs_up from the point of view of reiserfs. Ok. Im not keen on adding fs_down but it does look like the right bandage (and a nice speed up) for 2.2. I hate that kind of bug - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [lkml]Re: VM problems still in 2.2.18
On Fri, Dec 15, 2000 at 06:46:32PM +, Alan Cox wrote: > so the actual problem is either - the returning 1 when it is the wrong answer > - or the failure to block somewhere else (where its safe) based on a kpiod > maintained semaphore ? The problem is not to find a safe place where to wait, the problem is to know "when" we can block. That's the only thing current->fs_locks tells us. Sometime we simply can't wait because I/O can't start until we return (it would deadlock regardless we wait on a kpiod semaphore or in down(i_sem) ourselfs). Once we know "if" we can't wait or not the whole point of kpiod is lost as kpiod exists only because we didn't know that and so we were not able to wait. Now we know when we can block so we can run f_ops->write ourselfs that's also more efficient in terms of both performance and also memory pressure during swap of course. > I assume thats not an issue to reiserfs ? As said reiserfs AFIK didn't need any change, so only VFS is using fs_down/fs_up from the point of view of reiserfs. Andrea - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [lkml]Re: VM problems still in 2.2.18
> o don't wait I/O completion to avoid deadlocking on the i_sem > o swap_out returns 1 and memory balancing code so thinks we did progress > in freeing memory and goes to allocate memory from the freelist > without waiting I/O completion > o repeat N times the above so the actual problem is either - the returning 1 when it is the wrong answer - or the failure to block somewhere else (where its safe) based on a kpiod maintained semaphore ? > Yes, the same `current' context must run the down/up pair of calls and as you > said it is legal to rely on it on all the places it's used. I assume thats not an issue to reiserfs ? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [lkml]Re: VM problems still in 2.2.18
On Fri, Dec 15, 2000 at 05:57:18PM +, Alan Cox wrote: > How hard is it to seperate losing kpiod (optimisation) from the MAP_SHARED > changes ? I am assuming they are two seperate issues, possibly wrongly Losing kpiod isn't an optimization ;(. Losing kpiod is the MAP_SHARED bugfix. The problem was: o swap_out o wants to flush a MAP_SHARED dirty page to disk o so allocate kpiod-struct o sumbit the page-flush request to kpiod o don't wait I/O completion to avoid deadlocking on the i_sem o swap_out returns 1 and memory balancing code so thinks we did progress in freeing memory and goes to allocate memory from the freelist without waiting I/O completion o repeat N times the above o in the meantime kpiod has a big queue but it's blocked slowly writing those pages to disk o while it writes a few pages swap_out floods again the queue without waiting and it empties the freelist (task killed) The problem was the lack of write throttling due the kpiod async-only nature. > Providing no inode semaphore is upped from a different task , which seems > currently quite a valid legal thing to do (ditto doing the up on completion of > something in bh or irq context) Yes, the same `current' context must run the down/up pair of calls and as you said it is legal to rely on it on all the places it's used. Andrea - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [lkml]Re: VM problems still in 2.2.18
> The changes in semaphore semantics are necessary to fix the spurious out of > memory with MAP_SHARED mappings and they came together with the removal of the > always-asynchronous kpiod. While it's certainly possible to remove it I don't > think removing the fix for MAP_SHARED stuff is a good idea. How hard is it to seperate losing kpiod (optimisation) from the MAP_SHARED changes ? I am assuming they are two seperate issues, possibly wrongly > Basically it's always safe to replace: > > down(&inode->i_sem); > /* critical section */ > up(&inode->isem); > > with the new fs-semaphore: > > fs_down(&inode->i_sem); > /* critical section */ > fs_up(&inode->i_sem); Providing no inode semaphore is upped from a different task , which seems currently quite a valid legal thing to do (ditto doing the up on completion of something in bh or irq context) Alan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [lkml]Re: VM problems still in 2.2.18
> figure out what else from this series can be put into 19pre. Believe the > major changes left in the aa series are bigmem and lvm. I would love to see > lvm officially in 2.2... lvm, 4Gb support, raid 0.90... to be honest by the time that sort of stuff would get integrated (except maybe the 4Gig support) 2.4 will probable be stable and a better option for such boxes - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [lkml]Re: VM problems still in 2.2.18
On Thu, Dec 14, 2000 at 11:17:11PM +, Alan Cox wrote: > Andrea - can we have the core VM changes you did without adopting the > change in semaphore semantics for file system locking which will give third > party fs maintainers headaches and doesnt match 2.4 behaviour either ? The changes in semaphore semantics are necessary to fix the spurious out of memory with MAP_SHARED mappings and they came together with the removal of the always-asynchronous kpiod. While it's certainly possible to remove it I don't think removing the fix for MAP_SHARED stuff is a good idea. Basically it's always safe to replace: down(&inode->i_sem); /* critical section */ up(&inode->isem); with the new fs-semaphore: fs_down(&inode->i_sem); /* critical section */ fs_up(&inode->i_sem); While it's always safe to use fs_down() in place of down(), it should be done only with inodes that can be memory mapped using MAP_SHARED if in the /* critical section */ there is any kind of not-GFP_ATOMIC or not-GFP_BUFFER allocation like any kind of user-copy (as in every f_ops->write callback). Most of those down are handled in the VFS and 99% of the time no changes are necessary in the lowlevel fs so it should not even generate headaches to third party fs maintainers (I think infact reiserfs patch didn't need any change). Note: directory i_sem doesn't need the fs_down since a directory can't be mapped in memory. 2.4.x doesn't need this per-process fs_locks information to avoid recursing on the i_sem while flushing MAP_SHARED mappings to disk because the lowlevel fs is required to implement a_ops->writepage to support MAP_SHARED, and the page-flush gets synchronized only via the per-page lock (while in 2.2.x we need the i_sem on the inode to call the non-zero-copy f_ops->write to flush the MAP_SHARED dirty pages). Andrea - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
[lkml]Re: VM problems still in 2.2.18
Alan Cox wrote: > > slrnpull --expire on a news-spool of about 600 Mb in 200,000 files gave > > a lot of 'trying_to_free..' errors. > > > > 2.2.18 + VM-global, booted with mem=32M: > > > > slrnpull --expire on the same spool worked fine. > > I think Andrea just earned his official God status ;) I have been using the aa series kernels through out the 18pre series (with reiserfs). They have worked very well. Suggest you and Andrea talk and figure out what else from this series can be put into 19pre. Believe the major changes left in the aa series are bigmem and lvm. I would love to see lvm officially in 2.2... Luck, Ed Tomlinson - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [lkml]Re: VM problems still in 2.2.18
> > I think Andrea just earned his official God status ;) > So, maybe his divine VM patches will make it into 2.2.19? The question is merely 'in what form' . I wanted to keep them seperate from the other large changes in 2.2.18 for obvious reasons. Andrea - can we have the core VM changes you did without adopting the change in semaphore semantics for file system locking which will give third party fs maintainers headaches and doesnt match 2.4 behaviour either ? Alan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [lkml]Re: VM problems still in 2.2.18
On 2000/12/14 Alan Cox wrote: > > slrnpull --expire on a news-spool of about 600 Mb in 200,000 files gave > > a lot of 'trying_to_free..' errors. > > > > 2.2.18 + VM-global, booted with mem=32M: > > > > slrnpull --expire on the same spool worked fine. > > I think Andrea just earned his official God status ;) > How about a 2.2.19-pre1 == 2.2.18-aa2 ? -- Juan Antonio Magallon Lacarta #> cd /pub mailto:[EMAIL PROTECTED] #> more beer Linux werewolf 2.2.18-aa2 #1 SMP Thu Dec 14 21:22:40 CET 2000 i686 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [lkml]Re: VM problems still in 2.2.18
Alan Cox wrote: > > slrnpull --expire on a news-spool of about 600 Mb in 200,000 files gave > > a lot of 'trying_to_free..' errors. > > > > 2.2.18 + VM-global, booted with mem=32M: > > > > slrnpull --expire on the same spool worked fine. > > I think Andrea just earned his official God status ;) So, maybe his divine VM patches will make it into 2.2.19? jjs - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [lkml]Re: VM problems still in 2.2.18
> slrnpull --expire on a news-spool of about 600 Mb in 200,000 files gave > a lot of 'trying_to_free..' errors. > > 2.2.18 + VM-global, booted with mem=32M: > > slrnpull --expire on the same spool worked fine. I think Andrea just earned his official God status ;) - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [lkml]Re: VM problems still in 2.2.18
On Thu, Dec 14, 2000 at 09:57:28AM +, Alan Cox wrote: > > bug was discovered. Ever since, I have two boxes here > > that keep falling over. Box A will randomly lock without > > warning and box B will die and start printing this message > > repeatedly on the screen until I physically hit reset: > > What are these two boxes doing ? > > > Is there a patch out there that I can apply to 2.2.14 > > against the security bug? The machines were very stable > > on that kernel. > > Andrea's VM-global patch seems to be a wonder cure for those who have tried > it. Give it a shot and let folks know. My experience: 2.2.18pre25 erroneously booted with mem=64M: slrnpull --expire on a news-spool of about 600 Mb in 200,000 files gave a lot of 'trying_to_free..' errors. 2.2.18 + VM-global, booted with mem=32M: slrnpull --expire on the same spool worked fine. Good luck, Jurriaan -- proof by reference to inaccessible literature: The author cites a simple corollary of a theorem to be found in a privately circulated memoir of the Slovenian Philological Society, 1883 (second edition). GNU/Linux 2.2.18 SMP 2x1117 bogomips load av: 0.56 0.15 0.05 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
