Re: [9fans] security questions

[blstuart]

>> Principles of Operating Systems: Design and Applications
>> by Brian Stuart
>>
>> ( http://www.amazon.co.uk/exec/obidos/ASIN/1418837695 )
>>
>> I've only just started reading it, so can't really comment on how good
>> it is yet. Looks promising so far though.
> 
> I recently bought this book and have read most of it.  It's especially
> good at bridging the gap between OS theory and the gritty details of
> implementation with clear explanations of selected source code
> extracts from the Inferno and Linux kernels.  The chapter on Inferno
> process management and its scheduler is especially illuminating.
> 
> Although it focuses on the implementation of Inferno I've also found
> it helpful for understanding the Plan 9 kernel since it covers the
> Inferno device driver model, viz. embedded 9p/Styx servers.  It also
> reviews the Inferno implementation of kfs, which is written in Limbo,
> but the mental translation to C is easy.

Thank you.  I'm glad you're finding it useful.

BLS

Re: [9fans] security questions

[John Barham]

Robert Raschke wrote:
> Also note there's a new book out that includes Inferno as a major
> example, essentially explaining OS principles in general, in Inferno,
> and in Linux:
>
> Principles of Operating Systems: Design and Applications
> by Brian Stuart
>
> ( http://www.amazon.co.uk/exec/obidos/ASIN/1418837695 )
>
> I've only just started reading it, so can't really comment on how good
> it is yet. Looks promising so far though.

I recently bought this book and have read most of it.  It's especially
good at bridging the gap between OS theory and the gritty details of
implementation with clear explanations of selected source code
extracts from the Inferno and Linux kernels.  The chapter on Inferno
process management and its scheduler is especially illuminating.

Although it focuses on the implementation of Inferno I've also found
it helpful for understanding the Plan 9 kernel since it covers the
Inferno device driver model, viz. embedded 9p/Styx servers.  It also
reviews the Inferno implementation of kfs, which is written in Limbo,
but the mental translation to C is easy.

  John

Re: [9fans] security questions

[Bakul Shah]

On Fri, 17 Apr 2009 08:14:12 EDT "Devon H. O'Dell"   
wrote:
> 2009/4/17 erik quanstrom :
> >> What if each user can have a separate IP stack, separate
> >> (virtualized) interfaces and so on?
> >
> > already possible, but you do need 1 physical ethernet
> > per ip stack if you want to talk to the outside world.
> 
> I'm sure it wouldn't be hard to add a virtual ``physical'' interface,
> even though that seems a little bit pervasive, given the already
> semi-virtual nature due to namespaces. Not sure how much of a hassle
> it would be to make multiple stacks bindable to a single interface...
> but perhaps that's the better way to go?

You'd have to add a packet classifier of some sort.  Packets
to host A get delivered to logical interface #1, host B get
delivered to #2 and so on.  Going out is not a problem.

Alternatively put each virtual host on a different VLAN (if
your ethernet controller does VLANs).

> >> But you'd have to implement some sort of limits on
> >> oversubcribing (ratio of virtual to real resources). Unlike
> >> securitization in the hedge fund world.
> >
> > this would add a lot of code and result in the same problem
> > as today =97 you can be run out of a criticial resource.
> 
> Oversubscribing is the root of the problem. In fact, even if it was
> already done, on a terminal server, imagmem is also set to kpages. So
> if someone found a way to blow up the kernel's draw buffer, boom. I
> don't know how far reaching that is, as I've never really seen the
> draw code.

If you are planning to open up a system to the public, then
provisioning for the peak use of your system will result in a
lot of waste (even if you had the resources to so provision).
Even your ISP uses oversubscription (probably by a factor of
100, if not more. If his upstream data pipes give him N bps,
he will give out 100N bps of total bandwidth to his
customers.  If you want guaranteed bandwidth, you have to
shell out a lot more for a "gold" service level agreement).

What I meant is
a) you need to ensure that a single user can't exceed his resoucre limits,
b) enforce a sensible oversubscription limit (if you oversubscribe
   by a factor of 30, don't let in the 31st concurrent user), and
c) very likely you also want to put these users in different
   login classes (ala *BSD) and disallow each class to
   cumulatively exceed configured resource limit (*BSD
   doesn't do this) -- this is where I was thinking of CBQ.

Re: [9fans] security questions

[lucio]

> Very nice of you to go to lengths for describing Inferno to a non-techie. 
> Thank you. Just got the Fourth Edition ISO and will try it. Maybe even 
> learn some Limbo in long term.

My pleasure.  I just hope no one decides to confront me on all the
inaccuracies that are likely to have crept in :-)

++L

Re: [9fans] security questions

[Robert Raschke]

On Fri, Apr 17, 2009 at 2:08 PM, Eris Discordia
 wrote:
> Very nice of you to go to lengths for describing Inferno to a non-techie.
> Thank you. Just got the Fourth Edition ISO and will try it. Maybe even learn
> some Limbo in long term.

Also note there's a new book out that includes Inferno as a major
example, essentially explaining OS principles in general, in Inferno,
and in Linux:

Principles of Operating Systems: Design and Applications
by Brian Stuart

( http://www.amazon.co.uk/exec/obidos/ASIN/1418837695 )

I've only just started reading it, so can't really comment on how good
it is yet. Looks promising so far though.

Robby

Re: [9fans] security questions

[gdiaz]

hello 

you might want to take a look to vitanuova resources page for other inferno 
flavours than the official release.

inferno-os.googlecode.com
acme-sac.googlecode.com


slds.

gabi

Re: [9fans] security questions

[Eris Discordia]

Very nice of you to go to lengths for describing Inferno to a non-techie. 
Thank you. Just got the Fourth Edition ISO and will try it. Maybe even 
learn some Limbo in long term.


--On Friday, April 17, 2009 1:55 PM +0200 lu...@proxima.alt.za wrote:


what it is that Inferno does for a user or what a user can do
with it; what distinguishes it from other (operating?) systems. I've
decided to try it because documentation says it will readily run on
Windows.


Let's start with the fact that Inferno is a small-footprint, hosted
operating environment with its own, complete development tool set.  As
such it is strictly portable across many architectures with all the
advantages of such portability as well as all the useful features
Inferno inherited from Plan 9.  Not least of these is Limbo, a
programming language based on the mourned Alef and, conveniently,
interpreted by the Limbo virtual machine, not dissimilar from, but
much better thought out than the JAVA virtual machine.

You can pile on any number of additional great attributes of Inferno
and Limbo that make them highly useful.  There is also the option to
run Inferno natively on some architectures (I've never dug any deeper
than the PC for this, so off the top of my head I can provide no
exciting examples) with all the drawbacks of needing device drivers
for all sorts of inconsiderate platforms.

In a way, I guess Inferno is a slightly different Plan 9 with built-in
virtualisation for a wide range of platforms.  But the differences are
notable even if the philosophy is the same between the two
environment.

++L

Re: [9fans] security questions

[Steve Simon]

My understanding is that would prevent people listening and pretending to
offer services on my behalf, but would not stop them dialing SMTP ports
on other machines and sending them spam.

-Steve

Re: [9fans] security questions

[Devon H. O'Dell]

2009/4/17 erik quanstrom :
>> What if each user can have a separate IP stack, separate
>> (virtualized) interfaces and so on?
>
> already possible, but you do need 1 physical ethernet
> per ip stack if you want to talk to the outside world.

I'm sure it wouldn't be hard to add a virtual ``physical'' interface,
even though that seems a little bit pervasive, given the already
semi-virtual nature due to namespaces. Not sure how much of a hassle
it would be to make multiple stacks bindable to a single interface...
but perhaps that's the better way to go?

>> But you'd have to implement some sort of limits on
>> oversubcribing (ratio of virtual to real resources). Unlike
>> securitization in the hedge fund world.
>
> this would add a lot of code and result in the same problem
> as today — you can be run out of a criticial resource.

Oversubscribing is the root of the problem. In fact, even if it was
already done, on a terminal server, imagmem is also set to kpages. So
if someone found a way to blow up the kernel's draw buffer, boom. I
don't know how far reaching that is, as I've never really seen the
draw code.

Unfortunately, that's what you have to do unless you can afford to
invest in more hardware, or have a small userbase. Or find some middle
ground -- and maybe that's what the `virtualization' would address.

> - erik

--dho

Re: [9fans] security questions

[erik quanstrom]

> Dialing remote ports
> I don't become a spam relay so some restriction must be in place,
> I guess this would require a minor modification to the IP stack.

does ip/hogports solve your problem?

- erik

Re: [9fans] security questions

[maht]

If you want true isolation between the users you should give
them each a VM, not a Plan 9 account.

Russ

  

So we chose to use a VM, now we have two problems

*http://tinyurl.com/cuul2m

or
*
http://www.computerworld.com/action/article.do?command=viewArticleBasic&taxonomyName=operating_systems&articleId=9131647&taxonomyId=89&intsrc=kc_top

Re: [9fans] security questions

[Devon H. O'Dell]

2009/4/17 Bakul Shah :
> On Thu, 16 Apr 2009 22:19:21 EDT "Devon H. O'Dell"   
> wrote:
>> 2009/4/16 Bakul Shah :
>> > Why not give each user a virtual plan9? Not like vmware/qemu
>> > but more like FreeBSD's jail(8), "done more elegantly"[TM]!
>> > To deal with potentially malicious users you can virtualize
>> > resources, backed by limited/configurable real resources.
>>
>> I saw a talk about Mult at DCBSDCon. I think it's a much better idea
>> than FreeBSD jail(8), and its security is provable.
>>
>> See also: http://mult.bsd.lv/
>
> But is it elegant?

Rather.

> [Interviewer: What do you think the analog for software is?
>  Arthur Whiteny: Poetry.
>  Interviewer: Poetry captures the aesthetics, but not the precision.
>  Arthur Whiteny: I don't know, may be it does.
>  -- ACM Queue Feb/Mar 2009, page 18.
>    http://mags.acm.org/queue/20090203]
>
> Perhaps Plan9's model would be easier (and more fun) to
> extend to accomplish this. One can already have a private
> namespace.  How about changing proc(3) to show only your
> login process and its descendents? What if each user can have
> a separate IP stack, separate (virtualized) interfaces and so
> on?  But you'd have to implement some sort of limits on
> oversubcribing (ratio of virtual to real resources). Unlike
> securitization in the hedge fund world.
>
>

Re: [9fans] security questions

[lucio]

> what it is that Inferno does for a user or what a user can do 
> with it; what distinguishes it from other (operating?) systems. I've 
> decided to try it because documentation says it will readily run on Windows.

Let's start with the fact that Inferno is a small-footprint, hosted
operating environment with its own, complete development tool set.  As
such it is strictly portable across many architectures with all the
advantages of such portability as well as all the useful features
Inferno inherited from Plan 9.  Not least of these is Limbo, a
programming language based on the mourned Alef and, conveniently,
interpreted by the Limbo virtual machine, not dissimilar from, but
much better thought out than the JAVA virtual machine.

You can pile on any number of additional great attributes of Inferno
and Limbo that make them highly useful.  There is also the option to
run Inferno natively on some architectures (I've never dug any deeper
than the PC for this, so off the top of my head I can provide no
exciting examples) with all the drawbacks of needing device drivers
for all sorts of inconsiderate platforms.

In a way, I guess Inferno is a slightly different Plan 9 with built-in
virtualisation for a wide range of platforms.  But the differences are
notable even if the philosophy is the same between the two
environment.

++L

Re: [9fans] security questions

[erik quanstrom]

> The
> virtual memory management is too persuasive to be broken in any
> significant way.

do you mean pervasive?  if you do, i don't buy the argument.
it's easy to get lucky when doing concurrent programming with
locks, as in the plan 9 kernel.  it's easy to get lucky in many cases,
and yet have completely bogus locking.  (as i rediscovered this
morning.)

- erik

Re: [9fans] security questions

[lucio]

> Working swap would do me to fix this, but sadly rlimits would probably
> be easier to implement. 

There's an intrinsic belief that there cannot be anything wrong with
Plan 9's swap.  Having encountered the rather tightly embedded use of
swap/segmentation/etc.  in the Plan 9 kernel, but without having
explored it to any extent, I'm beginning to see where the faith
principle comes from.  Before anyone can be convinced to "fix" swap,
it is imperative to be able to supply a reproducible error case.  The
virtual memory management is too persuasive to be broken in any
significant way.

++L

Re: [9fans] security questions

[lucio]

> Erik's mod would help, but add a seccond threshold where after 15 secconds
> you kill the proc failed the most fork() calls - the danger here is a spam
> storm may cause listen(1) to be killed.

You could put the rate limiting in listen(8) first, you may have
noticed that inetd(8) has this feature, at least in NetBSD, enabled by
default, in contravention of the POLA.

++L

Re: [9fans] security questions

[erik quanstrom]

> What if each user can have a separate IP stack, separate
> (virtualized) interfaces and so on?

already possible, but you do need 1 physical ethernet
per ip stack if you want to talk to the outside world.

> But you'd have to implement some sort of limits on
> oversubcribing (ratio of virtual to real resources). Unlike
> securitization in the hedge fund world.

this would add a lot of code and result in the same problem
as today — you can be run out of a criticial resource.

- erik

Re: [9fans] security questions

[Eris Discordia]

I see. Thanks for the edification :-) I found--still find--it hard to 
understand what Inferno is/does. Actually read 
 but it isn't very 
direct about what it is that Inferno does for a user or what a user can do 
with it; what distinguishes it from other (operating?) systems. I've 
decided to try it because documentation says it will readily run on Windows.


As a side note, I found a short passage in the Inferno paper that confirmed 
something I had pointed out previously on this list in almost identical 
wording (and been ridiculed for):



The Styx protocol lies above and is independent of the communications
transport layer; it is readily carried over TCP/IP, PPP, ATM or various
modem transport protocols.


--On Friday, April 17, 2009 11:47 AM +0200 lu...@proxima.alt.za wrote:


I don't know what Inferno is but the phrase 'virtual machine' appears
somewhere in the product description. Isn't Inferno the 'it' you're
searching for?


No, Inferno resembles - very superficially, as you will discover if
you study the literature - a JAVA interpreter surrounded by its own
operating system.  There are so many clever things about Inferno, it
is hard to do it justice.  But it is not a virtualiser.  More's the
pity, of course.  A virtualiser with Inferno's good features would be
a very useful device.

Actually, I have long had a feeling that there is a convergence of
VNC, Drawterm, Inferno and the many virtualising tools (VMware, Xen,
Lguest, etc.), but it's one of these intuition things that I cannot
turn into anything concrete.

++L

Re: [9fans] security questions

[Mechiel Lukkien]

On Fri, Apr 17, 2009 at 11:29:47AM +0100, Steve Simon wrote:
> I am interested in the idea of adding some kind of resource limits
> to plan9. If they existsed I would probably open it up to external
> users, however different things would worry me:
> 
> CPU use
> Implement the Fair share scheduler
> 
> User memory
> Working swap would do me to fix this, but sadly rlimits would probably
> be easier to implement. 
> 
> Network bandwidth
> Again a FSS type algorithm delaying or dropping packets could rate
> control the network well I think.
> 
> Dialing remote ports
> I don't become a spam relay so some restriction must be in place,
> I guess this would require a minor modification to the IP stack.
> 
> Fork bombs
> Erik's mod would help, but add a seccond threshold where after 15 secconds
> you kill the proc failed the most fork() calls - the danger here is a spam
> storm may cause listen(1) to be killed.
> 
> Running out of kernel memory
> I don't perceive this as a problem, though this could be my lack of vision.

of all the resource capping on a public plan 9 server, i would say the
limits should be per user.  not per-process (group) limits or similar.
i don't know how feasable that (accounting) is.

e.g. make sure a single user gets at most e.g. 50% of all available
resources (memory, procs, cpu time).  seems fairest to me.  leftover cpu
time can be given to active users.  leftover memory should probably just
go unused (unless you want to start with swap, which lets you scale a
bit further but has limits too).  if the per-user memory is too low,
just add more memory so it won't be.  then at least multiple users can
use the system and a single one cannot lock it up.

dialing to the outside is perhaps easiest with an external firewall
(e.g. on adsl modem, they all have one nowadays).  same for bandwidth
limiting.  that won't fairly share the network bandwidth among the users
though of the cpu servers, but will leave your home connection usable.

then there is "none".  anyone can become none, and services run as
none (at least initially).  with per-user limits, anyone can hog none's
resources, leaving none left for network services (which other users
need to login).  perhaps this is the reason per-user limits won't work?
or what would be the impact of disallowing becoming none for
non-hostowners?  normal users might not need it?

mjl

Re: [9fans] security questions

[Steve Simon]

I am interested in the idea of adding some kind of resource limits
to plan9. If they existsed I would probably open it up to external
users, however different things would worry me:

CPU use
Implement the Fair share scheduler

User memory
Working swap would do me to fix this, but sadly rlimits would probably
be easier to implement. 

Network bandwidth
Again a FSS type algorithm delaying or dropping packets could rate
control the network well I think.

Dialing remote ports
I don't become a spam relay so some restriction must be in place,
I guess this would require a minor modification to the IP stack.

Fork bombs
Erik's mod would help, but add a seccond threshold where after 15 secconds
you kill the proc failed the most fork() calls - the danger here is a spam
storm may cause listen(1) to be killed.

Running out of kernel memory
I don't perceive this as a problem, though this could be my lack of vision.

My 2¢ worth.

-Steve

Re: [9fans] security questions

[lucio]

> Unlike
> securitization in the hedge fund world.

Actually, it is a lot safer to provide something like securitisation
(hm, make that "s" a "z", it is no doubt a native, American word) in a
virtualised environment, you're much less likely to bring down the
entire system's economy, then.

++L

Re: [9fans] security questions

[lucio]

> I don't know what Inferno is but the phrase 'virtual machine' appears 
> somewhere in the product description. Isn't Inferno the 'it' you're 
> searching for?

No, Inferno resembles - very superficially, as you will discover if
you study the literature - a JAVA interpreter surrounded by its own
operating system.  There are so many clever things about Inferno, it
is hard to do it justice.  But it is not a virtualiser.  More's the
pity, of course.  A virtualiser with Inferno's good features would be
a very useful device.

Actually, I have long had a feeling that there is a convergence of
VNC, Drawterm, Inferno and the many virtualising tools (VMware, Xen,
Lguest, etc.), but it's one of these intuition things that I cannot
turn into anything concrete.

++L

Re: [9fans] security questions

[Charles Forsyth]

>Conceptually, anyway. Why is everyone always so hell-bent on hair-splitting? :P

probably the other options suggested by the careers advisor were theology and 
hairdressing.

Re: [9fans] security questions

[Richard Miller]

> having the potential for running out of memory in an interrupt
> handler might be a sign that a little code reorg is in order, if you
> are worried about this sort of thing.  (and even if you're not.)

To begin with:

   grep -n '.((iallocb)|(qproduce))' /sys/src/9/^(port pc)^/*.c

Re: [9fans] security questions

[Bakul Shah]

On Thu, 16 Apr 2009 22:19:21 EDT "Devon H. O'Dell"   
wrote:
> 2009/4/16 Bakul Shah :
> > Why not give each user a virtual plan9? Not like vmware/qemu
> > but more like FreeBSD's jail(8), "done more elegantly"[TM]!
> > To deal with potentially malicious users you can virtualize
> > resources, backed by limited/configurable real resources.
> 
> I saw a talk about Mult at DCBSDCon. I think it's a much better idea
> than FreeBSD jail(8), and its security is provable.
>
> See also: http://mult.bsd.lv/

But is it elegant?
[Interviewer: What do you think the analog for software is?
 Arthur Whiteny: Poetry.
 Interviewer: Poetry captures the aesthetics, but not the precision.
 Arthur Whiteny: I don't know, may be it does.
 -- ACM Queue Feb/Mar 2009, page 18.
http://mags.acm.org/queue/20090203]

Perhaps Plan9's model would be easier (and more fun) to
extend to accomplish this. One can already have a private
namespace.  How about changing proc(3) to show only your
login process and its descendents? What if each user can have
a separate IP stack, separate (virtualized) interfaces and so
on?  But you'd have to implement some sort of limits on
oversubcribing (ratio of virtual to real resources). Unlike
securitization in the hedge fund world.

Re: [9fans] security questions

[Eris Discordia]

The other thought that comes to mind is to consider something
like class based queuing (from the networking world).  That
is, allow choice of different allocation/scheduling/resource
use policies and allow further subdivision.


As with jail, this is also present in FreeBSD, I believe. It's called 
'login classes.' Although it's probably not as flexible as you'd want it to 
be.


--On Thursday, April 16, 2009 7:07 PM -0700 Bakul Shah 
 wrote:



On Thu, 16 Apr 2009 21:25:06 EDT "Devon H. O'Dell"
  wrote:

That said, I don't disagree. Perhaps Plan 9's environment hasn't been
assumed to contain malicious users. Which brings up the question: Can
Plan 9 be safely run in a potentially malicious environment?  Based on
this argument, no, it cannot. Since I want to run Plan 9 in this sort
of environment (and thus move away from that assumption), I want to
address these problems, and I kind of feel like it's weird to be
essentially told, ``Don't do that.''


Why not give each user a virtual plan9? Not like vmware/qemu
but more like FreeBSD's jail(8), "done more elegantly"[TM]!
To deal with potentially malicious users you can virtualize
resources, backed by limited/configurable real resources.

The other thought that comes to mind is to consider something
like class based queuing (from the networking world).  That
is, allow choice of different allocation/scheduling/resource
use policies and allow further subdivision. Then you can give
preferential treatment to known good guys.  Other users can
still experiment to their heart's content within the
resources allowed them.

My point being think of a consistent high level model that
you like and then worry about implementation details.

Re: [9fans] security questions

[Eris Discordia]

Plan 9 itself makes a great platfrom on which to construct
virtualisation.


I don't know what Inferno is but the phrase 'virtual machine' appears 
somewhere in the product description. Isn't Inferno the 'it' you're 
searching for?


--On Friday, April 17, 2009 6:48 AM +0200 lu...@proxima.alt.za wrote:


One can indirectly (and more consistently) limit the number of
allocated resources in this fashion (indeed, the number of open file
descriptors) by determining the amount of memory consumed by that
resource as proportional to the size of the resource. If I as a user
have 64,000 allocations of type Foo, and struct Foo is 64 bytes, then
I hold 1,000 Foos.


And by this, I clearly mean 64,000 bytes of allocated Foos.


From purely a spectator's perspective, I believe that if one needs to
add considerable complexity to Plan 9 in the form of user-based kernel
resource management, one may as well look carefully at the option of
adding self-virtualisation to the Plan 9 kernel and manage resources
in the virtualisation layer.

Plan 9 has provided a wide range of sophisticated, yet simple
techniques to solve a wide range of computer/system problems, but I'm
of the opinion that it missed virtualisation as one of these
techniques.  I may be dreaming, but I've long been of the opinion that
Plan 9 itself makes a great platfrom on which to construct
virtualisation.

++L

Re: [9fans] security questions

[Bruce Ellis]

As a another data point I'll offer IW9P2009-Bondi - involved a lot of
beer and beach/camping but we wrote a shit-load of code. And it was
fun. Not much sleep. Had to eat too but time sharing coding and
cooking went well.

brucee

On Fri, Apr 17, 2009 at 3:52 PM, andrey mirtchovski
 wrote:
>> 5. No code is ever implemented by anyone
>
> extremely efficient, from a SLOC point of view, no?
>
> it  also leaves a lot of time for drinking belgian beer, which is nice.
>
>

Re: [9fans] security questions

[Bruce Ellis]

Not productive huh? That why not even Tiger reads the list anymore.
But I read mail from you.

brucee

On Fri, Apr 17, 2009 at 3:48 PM,   wrote:
>> On Thu, Apr 16, 2009 at 10:33 PM, Devon H. O'Dell  
>> wrote:
>>> 2009/4/16 erik quanstrom :
 On Thu Apr 16 22:18:35 EDT 2009, devon.od...@gmail.com wrote:
> > i just stated what i thought the historical situation was.  the
> > point was only that changing direction will be difficult.
>
> This thread certainly proves that :)

 a 9fans thread proves nothing.
>>>
>>> Conceptually, anyway. Why is everyone always so hell-bent on 
>>> hair-splitting? :P
>>
>> So much code, so much precision, so much specificity in dealing with
>> computers. It often spreads outside of the computational context and
>> into one's personality.
>>
>
> I'd think less that than the well-documented 9fans discussion process:
> 1. Someone brings up an idea
> 2. Half the list explains why it's bad
> 3. Trolling
> 4. Deciding on what is most "plan 9-ish"
> 5. No code is ever implemented by anyone
>
>
>
> John
>
>
>

Re: [9fans] security questions

[andrey mirtchovski]

> 5. No code is ever implemented by anyone

extremely efficient, from a SLOC point of view, no?

it  also leaves a lot of time for drinking belgian beer, which is nice.

Re: [9fans] security questions

[john]

> On Thu, Apr 16, 2009 at 10:33 PM, Devon H. O'Dell  
> wrote:
>> 2009/4/16 erik quanstrom :
>>> On Thu Apr 16 22:18:35 EDT 2009, devon.od...@gmail.com wrote:
 > i just stated what i thought the historical situation was.  the
 > point was only that changing direction will be difficult.

 This thread certainly proves that :)
>>>
>>> a 9fans thread proves nothing.
>>
>> Conceptually, anyway. Why is everyone always so hell-bent on hair-splitting? 
>> :P
> 
> So much code, so much precision, so much specificity in dealing with
> computers. It often spreads outside of the computational context and
> into one's personality.
> 

I'd think less that than the well-documented 9fans discussion process:
1. Someone brings up an idea
2. Half the list explains why it's bad
3. Trolling
4. Deciding on what is most "plan 9-ish"
5. No code is ever implemented by anyone



John

Re: [9fans] security questions

[lucio]

>> One can indirectly (and more consistently) limit the number of
>> allocated resources in this fashion (indeed, the number of open file
>> descriptors) by determining the amount of memory consumed by that
>> resource as proportional to the size of the resource. If I as a user
>> have 64,000 allocations of type Foo, and struct Foo is 64 bytes, then
>> I hold 1,000 Foos.
> 
> And by this, I clearly mean 64,000 bytes of allocated Foos.

>From purely a spectator's perspective, I believe that if one needs to
add considerable complexity to Plan 9 in the form of user-based kernel
resource management, one may as well look carefully at the option of
adding self-virtualisation to the Plan 9 kernel and manage resources
in the virtualisation layer.

Plan 9 has provided a wide range of sophisticated, yet simple
techniques to solve a wide range of computer/system problems, but I'm
of the opinion that it missed virtualisation as one of these
techniques.  I may be dreaming, but I've long been of the opinion that
Plan 9 itself makes a great platfrom on which to construct
virtualisation.

++L

Re: [9fans] security questions

[J.R. Mauro]

On Thu, Apr 16, 2009 at 10:33 PM, Devon H. O'Dell  wrote:
> 2009/4/16 erik quanstrom :
>> On Thu Apr 16 22:18:35 EDT 2009, devon.od...@gmail.com wrote:
>>> > i just stated what i thought the historical situation was.  the
>>> > point was only that changing direction will be difficult.
>>>
>>> This thread certainly proves that :)
>>
>> a 9fans thread proves nothing.
>
> Conceptually, anyway. Why is everyone always so hell-bent on hair-splitting? 
> :P

So much code, so much precision, so much specificity in dealing with
computers. It often spreads outside of the computational context and
into one's personality.

>
>> - erik
>>
>>
>
>

Re: [9fans] security questions

[Devon H. O'Dell]

2009/4/16 erik quanstrom :
> On Thu Apr 16 22:18:35 EDT 2009, devon.od...@gmail.com wrote:
>> > i just stated what i thought the historical situation was.  the
>> > point was only that changing direction will be difficult.
>>
>> This thread certainly proves that :)
>
> a 9fans thread proves nothing.

Conceptually, anyway. Why is everyone always so hell-bent on hair-splitting? :P

> - erik
>
>

Re: [9fans] security questions

[erik quanstrom]

On Thu Apr 16 22:18:35 EDT 2009, devon.od...@gmail.com wrote:
> > i just stated what i thought the historical situation was.  the
> > point was only that changing direction will be difficult.
> 
> This thread certainly proves that :)

a 9fans thread proves nothing.

- erik

Re: [9fans] security questions

[Devon H. O'Dell]

2009/4/16 Bakul Shah :
> On Thu, 16 Apr 2009 21:25:06 EDT "Devon H. O'Dell"   
> wrote:
>> That said, I don't disagree. Perhaps Plan 9's environment hasn't been
>> assumed to contain malicious users. Which brings up the question: Can
>> Plan 9 be safely run in a potentially malicious environment?  Based on
>> this argument, no, it cannot. Since I want to run Plan 9 in this sort
>> of environment (and thus move away from that assumption), I want to
>> address these problems, and I kind of feel like it's weird to be
>> essentially told, ``Don't do that.''
>
> Why not give each user a virtual plan9? Not like vmware/qemu
> but more like FreeBSD's jail(8), "done more elegantly"[TM]!
> To deal with potentially malicious users you can virtualize
> resources, backed by limited/configurable real resources.

I saw a talk about Mult at DCBSDCon. I think it's a much better idea
than FreeBSD jail(8), and its security is provable.

See also: http://mult.bsd.lv/

I do like this idea.

> The other thought that comes to mind is to consider something
> like class based queuing (from the networking world).  That
> is, allow choice of different allocation/scheduling/resource
> use policies and allow further subdivision. Then you can give
> preferential treatment to known good guys.  Other users can
> still experiment to their heart's content within the
> resources allowed them.
>
> My point being think of a consistent high level model that
> you like and then worry about implementation details.

That's also another interesting idea I hadn't considered.

Anybody else with thoughts on these?

--dho

Re: [9fans] security questions

[Devon H. O'Dell]

2009/4/16 erik quanstrom :
>> Right, we're saying the same thing backwards. I just am not sure why
>> smalloc was brought up. Yes, it is able to sleep until memory is
>> available for the operation, but it's not used *everywhere*.
>
> that's part of my point.  sometimes smalloc is appropriate,
> sometimes it is not.  it depends on other things than just
> what you are going to use the allocation for.
>
> do you have a particular, concrete example?

No, you brought it up first, asking if I had looked at it. I think
this particular thread of discussion is agreed upon and tangential.

>> > but what they aren't doing is writing fork bomb programs or programs
>> > that fuzz device drivers.
>>
>> Right, and that's a real threat.
>
> but you can't defend against it unless you decide you
> can have exactly n processes per system and each one can
> have 1/n * userlandsize bytes of memory.  you could divvy
> up memory by user to be a little bit more flexable, but
> providing hard guarantees is going to be very limiting.

Yeah, that's part of the reason I posted here, was to see if other
people had different ideas. This usually isn't a bad place to pull in
thoughts and solutions. And of course processes aren't the only
`potential evil.'

I agree that my solution may not be the best one. But I can't come up
with anything better right now, and at least this seems
semi-Plan-9-ish.

> don't forget about the stack overcommitment issue.

Heh.

>> Yes, there are, but it doesn't mean that it's an invalid assumption.
>> If you're arguing that my point is invalid because it's not a proper
>> application of Plan 9, I'd argue that Plan 9 isn't fit for the
>> Internet, where there are malicious users and script kiddies.
>
> i just stated what i thought the historical situation was.  the
> point was only that changing direction will be difficult.

This thread certainly proves that :)

> i don't think the second part of your argument holds.
> defending from a local threat (like a fork bomb) is much different
> from defending against script kiddies.  it's also much harder.
>
> also, script kiddies don't do a good job of targeting plan 9.

Point taken.

>> If you don't want to run Plan 9 there, ok. Maybe I'm the only one on
>> the list who does. Maybe someone will come out later who also wants
>> to.
>
> if i didn't think this could be useful, i wouldn't
> bother replying.

Point taken :). So are there other / better ideas? Did anybody else
read that paper about Viengoos?

> - erik

--dho

Re: [9fans] security questions

[Bakul Shah]

On Thu, 16 Apr 2009 21:25:06 EDT "Devon H. O'Dell"   
wrote:
> That said, I don't disagree. Perhaps Plan 9's environment hasn't been
> assumed to contain malicious users. Which brings up the question: Can
> Plan 9 be safely run in a potentially malicious environment?  Based on
> this argument, no, it cannot. Since I want to run Plan 9 in this sort
> of environment (and thus move away from that assumption), I want to
> address these problems, and I kind of feel like it's weird to be
> essentially told, ``Don't do that.''

Why not give each user a virtual plan9? Not like vmware/qemu
but more like FreeBSD's jail(8), "done more elegantly"[TM]!
To deal with potentially malicious users you can virtualize
resources, backed by limited/configurable real resources.

The other thought that comes to mind is to consider something
like class based queuing (from the networking world).  That
is, allow choice of different allocation/scheduling/resource
use policies and allow further subdivision. Then you can give
preferential treatment to known good guys.  Other users can
still experiment to their heart's content within the
resources allowed them.

My point being think of a consistent high level model that
you like and then worry about implementation details.

Re: [9fans] security questions

[Russ Cox]

> That said, I don't disagree. Perhaps Plan 9's environment hasn't been
> assumed to contain malicious users. Which brings up the question: Can
> Plan 9 be safely run in a potentially malicious environment?  Based on
> this argument, no, it cannot. Since I want to run Plan 9 in this sort
> of environment (and thus move away from that assumption), I want to
> address these problems, and I kind of feel like it's weird to be
> essentially told, ``Don't do that.''

If you were trying to run Plan 9 on systems that were allowed
to flip 1% of the bits in memory at random each day, we'd tell
you "don't do that" too.

Linux and FreeBSD and OS X can't be run in the kind of
environment you describe either.  If people are being malicious
and trying to take down the system, the right fix is to kick them off.

If you want true isolation between the users you should give
them each a VM, not a Plan 9 account.

Russ

Re: [9fans] security questions

[erik quanstrom]

> > interrupts are quite different.  there are lots of things that are
> > a bad idea in interrupt context.  but one can wakeup a kernel
> > proc that's sitting there waiting to deal with all the hair.
> 
> Right, we're saying the same thing backwards. I just am not sure why
> smalloc was brought up. Yes, it is able to sleep until memory is
> available for the operation, but it's not used *everywhere*.

that's part of my point.  sometimes smalloc is appropriate,
sometimes it is not.  it depends on other things than just
what you are going to use the allocation for.

do you have a particular, concrete example?

> > but what they aren't doing is writing fork bomb programs or programs
> > that fuzz device drivers.
> 
> Right, and that's a real threat.

but you can't defend against it unless you decide you
can have exactly n processes per system and each one can
have 1/n * userlandsize bytes of memory.  you could divvy
up memory by user to be a little bit more flexable, but
providing hard guarantees is going to be very limiting.

don't forget about the stack overcommitment issue.

> Yes, there are, but it doesn't mean that it's an invalid assumption.
> If you're arguing that my point is invalid because it's not a proper
> application of Plan 9, I'd argue that Plan 9 isn't fit for the
> Internet, where there are malicious users and script kiddies.

i just stated what i thought the historical situation was.  the
point was only that changing direction will be difficult.

i don't think the second part of your argument holds.
defending from a local threat (like a fork bomb) is much different
from defending against script kiddies.  it's also much harder.

also, script kiddies don't do a good job of targeting plan 9.

> If you don't want to run Plan 9 there, ok. Maybe I'm the only one on
> the list who does. Maybe someone will come out later who also wants
> to.

if i didn't think this could be useful, i wouldn't
bother replying.

- erik

Re: [9fans] security questions

[Devon H. O'Dell]

2009/4/16 erik quanstrom :
>>
>> My misunderstanding then, as smalloc is available in port/alloc.c,
>> which is also compiled into the kernel. I'm not concerned about oom
>> conditions in userland.
>
> smalloc is used in the kernel, but only when running with up (user
> process) and only when dealing with a "user's" data.  for example,
> when gathering up the response to a read on /proc/%d/regs or
> whatever.
>
> interrupts are quite different.  there are lots of things that are
> a bad idea in interrupt context.  but one can wakeup a kernel
> proc that's sitting there waiting to deal with all the hair.

Right, we're saying the same thing backwards. I just am not sure why
smalloc was brought up. Yes, it is able to sleep until memory is
available for the operation, but it's not used *everywhere*.

>> > having the potential for running out of memory in an interrupt
>> > handler might be a sign that a little code reorg is in order, if you
>> > are worried about this sort of thing.  (and even if you're not.)
>>
>> The potential for running out of memory in an interrupt handler exists
>> if a user has found a way to consume kernel resources from userland
>> and the interrupt needs to allocate that extra 1 byte.
>
> doctor, doctor it hurts when i do this!  there's a simple solution.
> never allocate in interrupt context.  i haven't ever felt the need
> to allocate in interrupt context.  in fact drivers like 82598 or
> the myricom driver never allocate during operation.

It was a point, not an argument. You're dismissing it altogether, I'm
saying that it's a potential issue. No, I haven't read the interrupt
code for every driver. It could be a complete non issue, and it
probably is. I'm just discussing it :).

>> > in any event, i think there is more code to deal with these problems
>> > in the kernel that first meets the eye.  much of it is small and, if you're
>> > not looking for it, easy to miss.
>>
>> I don't think so.
>
> are you telling me this code is not there, or that you've seen all of it?

I think we have a miscommunication, because you cut out the rest of
the context of my explanation, so I don't know how to address your
question. Neither; it wasn't the point I was making.

>> > i have had exactly 1 out-of-resource reboot in the last 18 months.
>> > without real data on what and where the problems are, i would
>> > think this would become a difficult issue.
>>
>> I don't know what your use case is, though I know that you probably
>> use the system more heavily than I. I think with people trying to find
>> issues, it could be a much easier endeavor, and I think it's a fun one
>> to address.
>
> i have 35 people working hard to kill the systems with 700MB mailboxes,
> and other tricks.

Which is more heavily than I, as I thought :)

> but what they aren't doing is writing fork bomb programs or programs
> that fuzz device drivers.

Right, and that's a real threat.

>> We're shielded in part by our small userbase and relative lack of
>> interest in examining code and auditing. But that's not security.
>
> i could argue that what you're looking into isn't the type of
> security that plan 9 has ever provided.  i think the assumption
> has been that the environment has been a work group and
> there aren't any hostile users.  there are lots of problems when
> you move away from that assumption.

Yes, there are, but it doesn't mean that it's an invalid assumption.
If you're arguing that my point is invalid because it's not a proper
application of Plan 9, I'd argue that Plan 9 isn't fit for the
Internet, where there are malicious users and script kiddies.

That said, I don't disagree. Perhaps Plan 9's environment hasn't been
assumed to contain malicious users. Which brings up the question: Can
Plan 9 be safely run in a potentially malicious environment?  Based on
this argument, no, it cannot. Since I want to run Plan 9 in this sort
of environment (and thus move away from that assumption), I want to
address these problems, and I kind of feel like it's weird to be
essentially told, ``Don't do that.''

If you don't want to run Plan 9 there, ok. Maybe I'm the only one on
the list who does. Maybe someone will come out later who also wants
to.

Either way, the code is in development, and I'm going to finish it.
Don't use it if you don't want to :)

> i would think the *first* order of business would be to encrypt
> all traffic to the fs so users can't use snoopy to captures other
> user's fs traffic

This is another serious issue. Perhaps I'll tackle it next.

> - erik

--dho

Re: [9fans] security questions

[erik quanstrom]

> > plan 9 doesn't have interrupt threads, but that's beside the point.
> >
> > interrupts are driven by the hardware, not users.  so smalloc, which
> > is used to allow user space to wait for memory if it is not currently
> > available doesn't make any sense.
> 
> My misunderstanding then, as smalloc is available in port/alloc.c,
> which is also compiled into the kernel. I'm not concerned about oom
> conditions in userland.

smalloc is used in the kernel, but only when running with up (user
process) and only when dealing with a "user's" data.  for example,
when gathering up the response to a read on /proc/%d/regs or
whatever.

interrupts are quite different.  there are lots of things that are
a bad idea in interrupt context.  but one can wakeup a kernel
proc that's sitting there waiting to deal with all the hair.

> > having the potential for running out of memory in an interrupt
> > handler might be a sign that a little code reorg is in order, if you
> > are worried about this sort of thing.  (and even if you're not.)
> 
> The potential for running out of memory in an interrupt handler exists
> if a user has found a way to consume kernel resources from userland
> and the interrupt needs to allocate that extra 1 byte.

doctor, doctor it hurts when i do this!  there's a simple solution.
never allocate in interrupt context.  i haven't ever felt the need
to allocate in interrupt context.  in fact drivers like 82598 or 
the myricom driver never allocate during operation.

> > in any event, i think there is more code to deal with these problems
> > in the kernel that first meets the eye.  much of it is small and, if you're
> > not looking for it, easy to miss.
> 
> I don't think so.

are you telling me this code is not there, or that you've seen all of it?

> > i have had exactly 1 out-of-resource reboot in the last 18 months.
> > without real data on what and where the problems are, i would
> > think this would become a difficult issue.
> 
> I don't know what your use case is, though I know that you probably
> use the system more heavily than I. I think with people trying to find
> issues, it could be a much easier endeavor, and I think it's a fun one
> to address.

i have 35 people working hard to kill the systems with 700MB mailboxes,
and other tricks.

but what they aren't doing is writing fork bomb programs or programs
that fuzz device drivers.

> We're shielded in part by our small userbase and relative lack of
> interest in examining code and auditing. But that's not security.

i could argue that what you're looking into isn't the type of
security that plan 9 has ever provided.  i think the assumption
has been that the environment has been a work group and
there aren't any hostile users.  there are lots of problems when
you move away from that assumption.

i would think the *first* order of business would be to encrypt
all traffic to the fs so users can't use snoopy to captures other
user's fs traffic

- erik

Re: [9fans] security questions

[Devon H. O'Dell]

2009/4/16 erik quanstrom :
> On Thu Apr 16 17:51:42 EDT 2009, devon.od...@gmail.com wrote:
>> 2009/4/16 erik quanstrom :
>> > have you taken a look at the protection measures already
>> > built into the kernel like smalloc?
>>
>> At least in FreeBSD, you can't sleep in an interrupt thread. I suppose
>> that's probably also the case in Plan 9 interrupt handlers, and this
>> would mitigate that situation.
>
> plan 9 doesn't have interrupt threads, but that's beside the point.
>
> interrupts are driven by the hardware, not users.  so smalloc, which
> is used to allow user space to wait for memory if it is not currently
> available doesn't make any sense.

My misunderstanding then, as smalloc is available in port/alloc.c,
which is also compiled into the kernel. I'm not concerned about oom
conditions in userland.

> having the potential for running out of memory in an interrupt
> handler might be a sign that a little code reorg is in order, if you
> are worried about this sort of thing.  (and even if you're not.)

The potential for running out of memory in an interrupt handler exists
if a user has found a way to consume kernel resources from userland
and the interrupt needs to allocate that extra 1 byte.

> in any event, i think there is more code to deal with these problems
> in the kernel that first meets the eye.  much of it is small and, if you're
> not looking for it, easy to miss.

I don't think so. You can get very specific about the problem or you
can get very generic. This is a generic implementation that would
allow for dealing with such problems as they arise, and actually
dealing with them in an easy, extensible fashion, without having to
add huge support code diffs for calculation of every resource you're
tracking (as is the case with rlimits, which require you to add
functions for each limit, hooks everywhere you want to track them,
shell support, and sysctls).

>From a codebase perspective, it's not a lot more code than you'd
think. It's very little code so far, and I'm about halfway finished
with the support part. Initial implementation will probably be
suboptimal, but I think useful for proof of concept.

Identifying the areas that need attention is the difficult part, but
I'll address that further down.

>> Depends again on the application. If you're talking about a terminal,
>> yes. If you're talking about a CPU server where someone is working on
>> code, someone else is writing a presentation, and yet another person
>> is in the middle of a video transcode, you're talking about a lot of
>> wasted time, potentially.
>
> and potentially, i could win the lottery.  ☺.

And potentially, this code would go into Plan 9. ☺ ☺ ☺.

I think those chances are a little smaller. Fileservers and terminal
servers, I don't think this is a big issue. CPU servers, I think it
is. I'm considering setting up a public Plan 9 cluster, though however
useless/useful it is to be remains to be seen. Part of the purpose of
this endeavor is to find exactly these kind of conditions. Whether
anybody is interested remains to be seen, but perhaps some incentives
can be provided. I don't know.

> i have had exactly 1 out-of-resource reboot in the last 18 months.
> without real data on what and where the problems are, i would
> think this would become a difficult issue.

I don't know what your use case is, though I know that you probably
use the system more heavily than I. I think with people trying to find
issues, it could be a much easier endeavor, and I think it's a fun one
to address.

The fact that there was one proves that these issues can occur
theoretically. All it needs is identification and reproduction.

We're shielded in part by our small userbase and relative lack of
interest in examining code and auditing. But that's not security.

> - erik

--dho

Re: [9fans] security questions

[erik quanstrom]

On Thu Apr 16 17:51:42 EDT 2009, devon.od...@gmail.com wrote:
> 2009/4/16 erik quanstrom :
> > have you taken a look at the protection measures already
> > built into the kernel like smalloc?
> 
> At least in FreeBSD, you can't sleep in an interrupt thread. I suppose
> that's probably also the case in Plan 9 interrupt handlers, and this
> would mitigate that situation.

plan 9 doesn't have interrupt threads, but that's beside the point.

interrupts are driven by the hardware, not users.  so smalloc, which
is used to allow user space to wait for memory if it is not currently
available doesn't make any sense.

having the potential for running out of memory in an interrupt
handler might be a sign that a little code reorg is in order, if you
are worried about this sort of thing.  (and even if you're not.)

in any event, i think there is more code to deal with these problems
in the kernel that first meets the eye.  much of it is small and, if you're
not looking for it, easy to miss.  

> Depends again on the application. If you're talking about a terminal,
> yes. If you're talking about a CPU server where someone is working on
> code, someone else is writing a presentation, and yet another person
> is in the middle of a video transcode, you're talking about a lot of
> wasted time, potentially.

and potentially, i could win the lottery.  ☺.

i have had exactly 1 out-of-resource reboot in the last 18 months.
without real data on what and where the problems are, i would
think this would become a difficult issue.

- erik

Re: [9fans] security questions

[Devon H. O'Dell]

2009/4/16 erik quanstrom :
> have you taken a look at the protection measures already
> built into the kernel like smalloc?

At least in FreeBSD, you can't sleep in an interrupt thread. I suppose
that's probably also the case in Plan 9 interrupt handlers, and this
would mitigate that situation.

>> While it may not be perfectly ideal, it allows the administrator to
>> maintain control over the system.
>
> being a system adminstrator, i dislike any ideas that require
> extra adminstration.  for the same reason, content-based email
> filters are impossible to maintain yourself.  you can't keep up
> with the threats.

There is that sentiment as well. I'm writing the code anyway, because
at least the API implementation is smaller than the amount of text in
emails I've already written about it (not to mention IRC discussion).
I tend to feel like there should be reasonably large defaults so that
you don't have to muck with it. Or maybe it's turned off by default. I
don't know. Either way, I'm not particularly expecting it to be
accepted as a patch, but I'm willing to be pleasantly surprised.

Besides, I haven't touched my sources tree for a good two years. It
needs something fresh :).

> otoh, if you panic because you run out of resources or whatever, the
> consequence should be a reboot.  at most you loose some editor
> state and need to spend a few minutes logging back in.

Depends again on the application. If you're talking about a terminal,
yes. If you're talking about a CPU server where someone is working on
code, someone else is writing a presentation, and yet another person
is in the middle of a video transcode, you're talking about a lot of
wasted time, potentially.

> if you allow users to log into your fileserver, you're asking
> for trouble.

Fileserver, I agree. I don't think a sane person would allow
`untrusted' user access, let alone anonymous access.. On a CPU server,
you kind of have to allow access to potentially untrusted users.
That's kind of the definition. (Hell, even paying users aren't
necessarily trustworthy. Tons of web hosts get exploited each year by
paying customers).

> the one resource that does need some fancier managment is
> procs.  the problem is that out-of-processes tends to act like
> livelock.  it would probablly be an improvement to set
> ooprocs variable when out of procs and clear it when some
> fraction (say 1/10) becomes free.  then one could panic if
> the out-of-processes condition persists for, say, 30 seconds.

Maybe I'll look at that after this.

> (i haven't had this problem since i added some better spam
> filtering.)
>
> - erik

Thanks for your insight so far, in any case :)

--dho

Re: [9fans] security questions

[erik quanstrom]

have you taken a look at the protection measures already
built into the kernel like smalloc?

> While it may not be perfectly ideal, it allows the administrator to
> maintain control over the system.

being a system adminstrator, i dislike any ideas that require
extra adminstration.  for the same reason, content-based email
filters are impossible to maintain yourself.  you can't keep up
with the threats.

otoh, if you panic because you run out of resources or whatever, the
consequence should be a reboot.  at most you loose some editor
state and need to spend a few minutes logging back in.

if you allow users to log into your fileserver, you're asking
for trouble.

the one resource that does need some fancier managment is
procs.  the problem is that out-of-processes tends to act like
livelock.  it would probablly be an improvement to set
ooprocs variable when out of procs and clear it when some
fraction (say 1/10) becomes free.  then one could panic if
the out-of-processes condition persists for, say, 30 seconds.

(i haven't had this problem since i added some better spam
filtering.)

- erik

Re: [9fans] security questions

[Devon H. O'Dell]

> One can indirectly (and more consistently) limit the number of
> allocated resources in this fashion (indeed, the number of open file
> descriptors) by determining the amount of memory consumed by that
> resource as proportional to the size of the resource. If I as a user
> have 64,000 allocations of type Foo, and struct Foo is 64 bytes, then
> I hold 1,000 Foos.

And by this, I clearly mean 64,000 bytes of allocated Foos.

--dho

Re: [9fans] security questions

[Devon H. O'Dell]

2009/4/16 Venkatesh Srinivas :
> Devlimit / Rlimit is less than ideal - the resource limits aren't
> adaptive to program needs and to resource availability. They would be
> describing resources that user programs have very little visible
> control over (kernel resources), except by changing their syscall mix
> or giving up a segment or so. Or failing outright.

Right, but that's part of the point. They have very little visible
control over said resources, but the kernel does need to impose
limitations on the allowable resources, more below. Either way, I
agree. This form of resource limitation sucks. My reasoning is
different, however: I feel that the number of places you need to add
new code and verify that any future changes respect these sorts of
limitations. This is much more difficult to do than when the
limitation is built into the allocator.

> Prohibitions per-user are kinda bad in general - what if you want to
> run a potentially hostile (or more likely buggy) program? You can
> already run it in its own ns, but having it be able to stab you via
> kernel resources, about which you can do nothing, is bad.

This depends on your perspective. If you are an administrator, and you
are running a system that provides access to a plethora of users, your
viewpoint is different than if you are a programmer, writing an
application with expensive needs. A user who wants to run a
potentially hostile program should not be able to affect the system to
the point that other users have their own negative experiences. A user
running a buggy program that hogs a ton of memory is not such a big
deal. Buy more memory. A user running a buggy program that runs the
kernel out of resources, causing it to halt is a big deal.

> The typed allocator is worth looking at for speed reasons - the slab
> allocator and customalloc have shown that its faster (from the
> perspective of allocation time, fragmentation) to do things that way.
> But I don't really see it addressing the problem? Now the constraints
> are per-resource, but they're still magic constraints.

This is the pitfall of all tunables. Unless someone pulls out a
calculator (or is mathematically brilliant), figuring out exact
numbers for X number of R resources spread between N users on a system
with Y amount of memory is silly.

Fortunately, an administrator makes educated best guesses based upon
these same factors:

1) The expected needs of the users of the system (and in other cases,
the absolute maximum needs of the users)
2) The limitations of the system itself. My laptop has 2GB of RAM, the
Plan 9 kernel only sits in 256MB of that. Now, assuming I have a
program that's able to allocate 10,000 64 byte structures a second, I
can panic the system in under two minutes. Does that mean I want to
limit my users to under 10,000 of those structures? Not necessarily,
but if I expect 40 users at a given time, I might want to make sure
that number is well under 100,000.

While it may not be perfectly ideal, it allows the administrator to
maintain control over the system. Additionally, there's typically
always a way to turn them off (echo 0 > /dev/constraint/resource,
hypothetically speaking). In this respect, I don't see how it doesn't
address the problem...

...Unless you consider that using Pools for granular limitations isn't
the best idea. In this light, perhaps additional pools aren't the
correct answer. While creating an individual pool per limited resource
serves to implement a hard limit on that resource, it's also required
to have a maximum amount of memory. So if you ditch that idea and just
make typed allocations, the problem is `solved':

When an allocation takes place, we check various heuristics to
determine whether or not the allocation is valid. Does the user have
over X Fids? Does the process hold open more than Y ports?

If these heuristics fail, the memory is not allocated, and the program
takes whatever direction it takes when it does not have resources.
(Crash, complain, happily move forward, whatever).

If they pass, the program gets what it needs, and {crashes, complains,
happily moves forward}.

One can indirectly (and more consistently) limit the number of
allocated resources in this fashion (indeed, the number of open file
descriptors) by determining the amount of memory consumed by that
resource as proportional to the size of the resource. If I as a user
have 64,000 allocations of type Foo, and struct Foo is 64 bytes, then
I hold 1,000 Foos.

The one unfortunate downside to this is that implementing this as an
allocation limit does not make it `provably safe.' That is to say, if
I create a kernel subsystem that allocates Foos, and I allocate
everything with T_MANAGED, there is no protection on the number of
Foos I allocate (assuming T_MANAGED means that this is a memory
allocation I manage myself. It's therefore not provably safer, in
terms of crashing the kernel, and I haven't been able to come up with
an idea that is provably safe (unless type de

Re: [9fans] security questions

[Venkatesh Srinivas]

Devlimit / Rlimit is less than ideal - the resource limits aren't
adaptive to program needs and to resource availability. They would be
describing resources that user programs have very little visible
control over (kernel resources), except by changing their syscall mix
or giving up a segment or so. Or failing outright.

Prohibitions per-user are kinda bad in general - what if you want to
run a potentially hostile (or more likely buggy) program? You can
already run it in its own ns, but having it be able to stab you via
kernel resources, about which you can do nothing, is bad.

The typed allocator is worth looking at for speed reasons - the slab
allocator and customalloc have shown that its faster (from the
perspective of allocation time, fragmentation) to do things that way.
But I don't really see it addressing the problem? Now the constraints
are per-resource, but they're still magic constraints.

Something that might be interesting would be for each primitive pgroup
to be born with a maximum percentage 'under pressure' associated with
each interesting resource. Child pgroups would allocate out of their
parent's pgroup limits. Until there is pressure, limits could be
unenforced, leading to higher utilization than magic constants in
rlimit.

To give a chance for a process to give up some of its resources
(caches, recomputable data) under pressure, there could be a
per-process /dev/halp device, which a program could read; reads would
block until a process was over its limit and something else needed
some more soup. If the app responds, great. Otherwise, something like
culling it or swapping it out (assuming that swap still worked :)) or
even slowing down its allocation rate artificially might be answers...

If people are interested, there is some work going on in a research
kernel called Viengoos
(http://www.gnu.org/software/hurd/microkernel/viengoos.html) (gasp!
the hurd!) trying to address pretty much the same problem...

-- vs

Re: [9fans] security questions

[erik quanstrom]

> The benefit to this approach is that we would have an extremely easy
> way to add new constraints as needed (simply create another tunable
> pool), without changing the API or interfering with multiple
> subsystems, outside of changing malloc calls if needed. The limits
> could be checked on a per-process or per-user (or both) basis.
> 
> We already have a pool for kernel memory, and a pool for kernel draw
> memory. Seems reasonable that we could have some for network buffers,
> files, processes and the like.

the network buffers are limited by the size of the network queues,
or they are pre-allocated.

you're right though.  there are a number of dynamicly allocated
resources in the kernel that are not managed.  in the old days,
the kernel didn't dynamicly allocate anything.  when things are
staticly allocated you don't have this problem, but everyone
complains about running out of $whatit structures.  if you
do dynamic allocation, then people complain about $unimportant
resource starving $important resource.

assuming a careful kernel and assuming that hostile users are
booted, resource contention is just about picking who loses.
and you're not going to get agreement that you're doing a good
job when you're picking losers.  ☺

- erik

[9fans] security questions

[Devon H. O'Dell]

In the interests of academia (and from the idea of setting up a public
Plan 9 cluster) comes the following mail. I'm sure people will brush
some of this off as a non-issue, but I'm curious what others think.

It doesn't seem that Plan 9 does much to protect the kernel from
memory / resource exhaustion. When it comes to kernel memory, the
standard philosophy of ``add more memory'' doesn't quite cut it:
there's a limited amount for the kernel, and if a user can exhaust
that, it's not a Good Thing. (Another argument I heard today was
``deal with the offending user swiftly,'' but that does little against
full disclosure). There are two potential ways to combat this (though
there are additional advantages to the existence of both):

1) Introduce more memory pools with tunable limits.

The idea here would be to make malloc() default to its current
behavior: just allocate allocate space from available arenas in
mainmem. An additional interface (talloc?) would be provided for
type-based allocations. These would be additional pools that serve to
store specific kernel data structures (Blocks, Chans, Procs, etc.).
This provides two benefits:

 o Protection against kernel memory starvation by exhaustion of a
specific resource
 o Some level of debugalloc-style memory information without all of the overhead

I suppose it would be possible to allow for tunable settings as well
by providing a FS to set e.g. minarea or maxsize.

The benefit to this approach is that we would have an extremely easy
way to add new constraints as needed (simply create another tunable
pool), without changing the API or interfering with multiple
subsystems, outside of changing malloc calls if needed. The limits
could be checked on a per-process or per-user (or both) basis.

We already have a pool for kernel memory, and a pool for kernel draw
memory. Seems reasonable that we could have some for network buffers,
files, processes and the like.

2) Introduce a `devlimit' device, which imposes limits on specific
kernel resources. The limits would be set on either a per-process or
per-user basis (or both, depending on the nature of the limit).

#2 seems more like the unixy rlimit model, and the more I think about
it, the less I like it. It's a bit more difficult to `get right', it
doesn't `feel' very Plan 9-ish, and adding new limits requires more
incestuous code. However, the limits are more finely tuned.

Just wondering any thoughts on this, which seems more feasible, if
anybody would feel it's a `good idea,' and the like. I got mixed
(though mostly positive from those who understood the issue) feedback
on IRC when I brought up the problem. I don't have any sample cases in
which it would be possible to starve the kernel of memory.

--dho