Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread lucio
> No, and it would be hard to do it because you'd need ways to compact > fragmented memory after a lot of mallocs and frees. And then, you'd > need a way to fix the pointers after compacting. Is it all localised, or is the code scattered across multiple kernel modules? Many years ago I put a lot

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread Charles Forsyth
> perhaps i've been unclear. i don't have any problem dealing with failed > alloc. malloc has always been able to return 0. > > dealing with a page fault due to overcommit is a different story. that's a slightly different aspect. the note should not be "page fault" but "out of memory" (or some

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread erik quanstrom
>> to any process. suppose i start a program that allocates 8k but between >> the malloc and the memset, another program uses the last available >> page in memory, then my original program faults. > > yes, and you'll always have to deal with that in some form or another. > i've started a program,

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread Charles Forsyth
> to any process. suppose i start a program that allocates 8k but between > the malloc and the memset, another program uses the last available > page in memory, then my original program faults. yes, and you'll always have to deal with that in some form or another. i've started a program, it alloc

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread erik quanstrom
On Mon Sep 3 16:48:59 EDT 2007, [EMAIL PROTECTED] wrote: > If your machines are regularly running out of VM, something is wrong > in your environment. I would argue that we'd be better off fixing > upas/fs to be less greedy with memory than contorting the system to > try to avoid overcommitting m

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread erik quanstrom
> One option for Erik: try changing the segment allocator so that it > faults in all segment pages on creation. Would this do what you want? > I will try this if I get time later today. Assuming it is as simple as > my simple-minded description makes it sound. grudgingly, i admit it would -- assum

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread Eric Van Hensbergen
I was thinking that this was probably what we wanted to do for HPC also, having the option of turning off zero-filling pages -eric On 9/3/07, ron minnich <[EMAIL PROTECTED]> wrote: > One option for Erik: try changing the segment allocator so that it > faults in all segment pages on

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread ron minnich
One option for Erik: try changing the segment allocator so that it faults in all segment pages on creation. Would this do what you want? I will try this if I get time later today. Assuming it is as simple as my simple-minded description makes it sound. If it would, maybe a simple echo faultall > /

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread Charles Forsyth
> if we allow overcomitted memory, *any* access of brk'd memory might page > fault. this seems like a real step backwards in error recovery as most > programs > assume that malloc either returns n bytes of valid memory or fails. since > this assumption is false, either we need to make it true or

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread geoff
venti/copy is just an example; programs may legitimately have large stacks. If your machines are regularly running out of VM, something is wrong in your environment. I would argue that we'd be better off fixing upas/fs to be less greedy with memory than contorting the system to try to avoid overc

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread geoff
If you don't want to swap, rename or delete any swap partitions you have and don't run swap in termrc nor cpurc.

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread erik quanstrom
> One might allocate at least 3.2GB of swap for a 4GB machine, but many > of our machines run with no swap, and we're probably not alone. And > 200 processes are not a lot. Would you really have over 32GB of swap > allocated for a 4GB machine with 2,000 processes? > > Programs can use a surprisi

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread erik quanstrom
>> The current swap just frustrates people who expect it to work, and >> then have their systems freeze randomly. Maybe by disabling/remove >> swap support, then if someone really needs swap he will fix it first >> and then we can add it back. > > i'm not sure all the random freezes are caused by

Re: [9fans] sources access from brazil

2007-09-03 Thread balaji
On 9/3/07, erik quanstrom <[EMAIL PROTECTED]> wrote: > >> there are? where are these available? > > > > ftp://ftp.apnic.net/pub/stats/arin/delegated-arin-latest > > arin only covers part of the world. i don't see any > entries for bazil in this file. all the rirs however are > required to implem

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread Uriel
Can't find out for sure until someone fixes it :)) One more reason IMHO why we are best off having it disabled, so when things freeze we know it is something else. Best wishes uriel On 9/3/07, Charles Forsyth <[EMAIL PROTECTED]> wrote: > > The current swap just frustrates people who expect it t

Re: [9fans] Re: everything is a directory

2007-09-03 Thread Uriel
Well, I expected people who wants extended attributes would want resource forks (double yuck) and who knows what else. Actually I think something like this was russ' suggestion at IWP9 to handle things like locking, symlinks and other PoSix junk. All that would be needed is a well defined convent

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread Charles Forsyth
> The current swap just frustrates people who expect it to work, and > then have their systems freeze randomly. Maybe by disabling/remove > swap support, then if someone really needs swap he will fix it first > and then we can add it back. i'm not sure all the random freezes are caused by swap. in

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread Charles Forsyth
>> Well, when I used it on an old 32 MB laptop (terminal) and a 64 MB >> desktop (cpu server), swap would seem to work all right until you >> hit about 30-40% usage. This was the case with both systems; when >> I asked about it, a couple other people mentioned the same behavior. >> The thing is, it

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread Uriel
This has been discussed before, but given how hard it is to debug this, and that nobody seems to have enough interest/motivation to do so, wouldn't it make more sense to totally remove swap support? I really can't understand why things like il are removed when they actually work, and swap is left a

Re: [9fans] sources access from brazil

2007-09-03 Thread erik quanstrom
>> there are? where are these available? > > ftp://ftp.apnic.net/pub/stats/arin/delegated-arin-latest arin only covers part of the world. i don't see any entries for bazil in this file. all the rirs however are required to implement whois. - erik

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread Charles Forsyth
> I don't actually need the swap partition, it's just there... ummm... not > sure why; I installed on this machine before I found out that swap is > broken. And it's not that I *think* swap is broken; it's been confirmed it worked adequately to cover minor shortfalls in memory, which could happe

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread geoff
One might allocate at least 3.2GB of swap for a 4GB machine, but many of our machines run with no swap, and we're probably not alone. And 200 processes are not a lot. Would you really have over 32GB of swap allocated for a 4GB machine with 2,000 processes? Programs can use a surprising amount of

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread john
>>> Also, it's broken, broken, broken on Plan 9 >> >> but could you describe what antisocial behavior it exhibits and how one >> could reproduce this behavior? i have never used to-disk paging on plan 9, >> so i don't know. >> > > Well, when I used it on an old 32 MB laptop (terminal) and a 64

Re: [9fans] sources access from brazil

2007-09-03 Thread balaji
> there are? where are these available? ftp://ftp.apnic.net/pub/stats/arin/delegated-arin-latest

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread john
>> Also, it's broken, broken, broken on Plan 9 > > but could you describe what antisocial behavior it exhibits and how one > could reproduce this behavior? i have never used to-disk paging on plan 9, > so i don't know. > Well, when I used it on an old 32 MB laptop (terminal) and a 64 MB desktop

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread Francisco J Ballesteros
> but could you describe what antisocial behavior it exhibits and how one > could reproduce this behavior? i have never used to-disk paging on plan 9, > so i don't know. Last time I tried the machine did freeze, like rock solid. It happen at some point after the swap partition was being used (sa

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread erik quanstrom
> Also, it's broken, broken, broken on Plan 9 but could you describe what antisocial behavior it exhibits and how one could reproduce this behavior? i have never used to-disk paging on plan 9, so i don't know. > and nobody wants to fix it. this has been a good discussion so far. let's not go o

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread john
> On 9/3/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: >> >> Also, it's broken, broken, broken on Plan 9 and nobody wants to fix it. >> The upside to this is that we can just say how we don't want it anyway, >> there's no conceivable reason anyone would want swap, and operating >> systems with

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread Francisco J Ballesteros
Fixing something nobody uses might be fun, but probably it's low on the stacks of most people because we don't consume too much memory. On 9/3/07, Gorka Guardiola <[EMAIL PROTECTED]> wrote: > On 9/3/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > > > Also, it's broken, broken, broken on Plan

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread Gorka Guardiola
On 9/3/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > Also, it's broken, broken, broken on Plan 9 and nobody wants to fix it. > The upside to this is that we can just say how we don't want it anyway, > there's no conceivable reason anyone would want swap, and operating > systems with working

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread john
[EMAIL PROTECTED] wrote: > also, why should i have to have swap? i really don't want it. it > introduces new failure modes and could introduce wide latency > variations. linux called, it wants it's choppy, laggy ui back. > Also, it's broken, broken, broken on Plan 9 and nobody wants to fix it.

Re: [9fans] Re: everything is a directory

2007-09-03 Thread Joel C. Salomon
On 9/3/07, Uriel <[EMAIL PROTECTED]> wrote: > Disgusting indeed, but it might even work with no changes to 9P; it > might confuse some clients, but doesn't sound easily doable given some > basic naming conventions (of course, then you lose the possibility to > cd into that dir and run ls there, whi

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread Sape Mullender
>> It would be cool to be able to get a handle on being able to shrink >> the memory occupied by an application dynamically. Malloc (through >> brk()) grows the memory footprint, but free does not shrink it. >> The same is true for the stack. Once allocated, it doesn't get freed >> until the proc

Re: [9fans] handling tabs in text frames

2007-09-03 Thread y i y u s
> I'm considering an alternative to the current tab behaviour in text frames. ... I just found this one, I think it is what you want to implement. It looks very useful to me: http://nickgravgaard.com/elastictabstops/ regards to the list (my first post after months quietly listening...), -- -

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread erik quanstrom
> Most applications probably use much less than 1 MB, but a lot depends > on who wrote the program. Our threaded programs typically have a 4K > or 8K (K, not M) fixed-size stack per thread and that works fine, > although you have to remember not to declare big arrays/structs as > local variables.

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread Sape Mullender
>> If system calls were the only way to change memory allocation, one >> could probably keep a strict accounting of pages allocated and fail >> system calls that require more VM than is available. But neither Plan >> 9 nor Unix works that way. The big exception is stack growth. The >> kernel aut

Re: [9fans] sources access from brazil

2007-09-03 Thread Charles Forsyth
>> of course, that's separate from the suggestion of using better databases etc. >> > > as long as we all know about sources' importance, i think the use of > better sources for determining request locations is of principal > matter since, for example, the actual scheme is so non-optimal that it >

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread erik quanstrom
>> would have to commit just for stacks. With 2,000 processes, that >> would rise to 32GB just for stacks. > > With 4GB RAM, wouldn't you allocate at least that much swap > no matter what? that's pretty expensive if you're booting from flash and not using a remote fileserver. 8GB flash is expen

Re: [9fans] thread(2) man page

2007-09-03 Thread Sape Mullender
> just to state it. We have int as return type for procexecl, procexec > in the thread(2) man page and void in /sys/src/libthread/exec.c If it returns at all, there was an error. If it doesn't return, there may or may not be an error, but you won't see it. Exec() probably has the int return valu

Re: [9fans] plan 9 overcommits memory?

2007-09-03 Thread erik quanstrom
> If system calls were the only way to change memory allocation, one > could probably keep a strict accounting of pages allocated and fail > system calls that require more VM than is available. But neither Plan > 9 nor Unix works that way. The big exception is stack growth. The > kernel automati

Re: [9fans] sources access from brazil

2007-09-03 Thread erik quanstrom
> 1) why would whois info matter to determine where a client is? There > are databases of ip country locations which would make much more > sense, depending on notoriously unreliable whois servers is silly both > from a security point of view and from a reliability point of view. there are? where

Re: [9fans] sources access from brazil

2007-09-03 Thread Iruata Souza
On 9/3/07, Charles Forsyth <[EMAIL PROTECTED]> wrote: > the export regulations might indeed be silly or unenforceable, or the > interpretation > of them overly pessimistic, but it would be an unwise person who would > ever willingly risk any contact whatsoever with that curious breed known as > U

Re: [9fans] sources access from brazil

2007-09-03 Thread Charles Forsyth
the export regulations might indeed be silly or unenforceable, or the interpretation of them overly pessimistic, but it would be an unwise person who would ever willingly risk any contact whatsoever with that curious breed known as US federal prosecutors. of course, that's separate from the sugg

Re: [9fans] Re: everything is a directory

2007-09-03 Thread Charles Forsyth
> Unix already made the > change a couple of decades ago. plan 9's definition of read on a directory, and the definition of the data thereby returned, is different from Unix's and there was no need to make a change. for instance, the data returned is unrelated to any underlying storage structure,

[9fans] thread(2) man page

2007-09-03 Thread Markus Sonderegger
Hi, just to state it. We have int as return type for procexecl, procexec in the thread(2) man page and void in /sys/src/libthread/exec.c Regards

Re: [9fans] Re: everything is a directory

2007-09-03 Thread Douglas A. Gwyn
"Joel C. Salomon" <[EMAIL PROTECTED]> wrote ... > Back on topic, I wasn't proposing a readdir() syscall but pointing out > that Douglas's suggestion would not in fact be painless or transparent > under Plan 9. Since I was unable to get Plan9 installed successfully when I set up my home systems, an

Re: [9fans] Re: everything is a directory

2007-09-03 Thread Uriel
Disgusting indeed, but it might even work with no changes to 9P; it might confuse some clients, but doesn't sound easily doable given some basic naming conventions (of course, then you lose the possibility to cd into that dir and run ls there, which is the main reason I thought an extension to the