Re: [9fans] VMs, etc.

[Devon H. O'Dell]

2009/4/20 erik quanstrom :
>
> i'm not following along.  what would be the application?

Jukebox, perhaps?

> - erik
>
>

Re: [9fans] VMs, etc.

[maht]

erik quanstrom wrote:

what could we do today, but don't quite dare?


  
  
a Blue Ray writer does 50Gb per disk (we're supposed to be getting one 
soon, so maybe I can report back about this later)


ArcVault SCSI autoloading tape drives do from 9.6tb -  76tb

http://www.b2net.co.uk/overland/overland_arcvault_12_autoloader.htm
http://www.b2net.co.uk/overland/overland_arcvault_24_autoloader.htm
http://www.b2net.co.uk/overland/overland_arcvault_48_library.htm



i'm not following along.  what would be the application?

- erik

  

I realised after I posted, I snipped the thing about the Labs worm drive

Re: [9fans] VMs, etc.

[erik quanstrom]

> >> what could we do today, but don't quite dare?
> >> 
> >
> >   
> a Blue Ray writer does 50Gb per disk (we're supposed to be getting one 
> soon, so maybe I can report back about this later)
> 
> ArcVault SCSI autoloading tape drives do from 9.6tb -  76tb
> 
> http://www.b2net.co.uk/overland/overland_arcvault_12_autoloader.htm
> http://www.b2net.co.uk/overland/overland_arcvault_24_autoloader.htm
> http://www.b2net.co.uk/overland/overland_arcvault_48_library.htm

i'm not following along.  what would be the application?

- erik

Re: [9fans] VMs, etc.

[maht]

what could we do today, but don't quite dare?



  
a Blue Ray writer does 50Gb per disk (we're supposed to be getting one 
soon, so maybe I can report back about this later)


ArcVault SCSI autoloading tape drives do from 9.6tb -  76tb

http://www.b2net.co.uk/overland/overland_arcvault_12_autoloader.htm
http://www.b2net.co.uk/overland/overland_arcvault_24_autoloader.htm
http://www.b2net.co.uk/overland/overland_arcvault_48_library.htm

Re: [9fans] VMs, etc. (was: Re: security questions)

[blstuart]

> people's ideas about what's complicated or hard don't change
> as quickly as computing power and storage has increased.  i
> think there's currently a failure of imagination, at least on
> my part.  there must be problems that aren't considered
> because they were hard.
> 
> as an old example, i think that the lab's use of worm storage
> for the main file server was incredibly insightful.
> 
> what could we do today, but don't quite dare?

That's a very good question.  I'm afraid my imagintion
may be failing here as well.  When I think about how
to use cycles, my mind tends to gravitate toward simulation
tasks that need cycles by virtue of scale: things like
weather forcasting, protein folding, etc.  One other
thing that periodically gets my attention are some
ideas in the AI realm.  But none of those are particularly 
novel.  Maybe that's why I don't find myself longing
for more performance than the current crop of machines.

Ultimately, I think you're right though.  Someone
more clever than I will likely identify a use for the
cycles that is more original than my thoughts and
more intellectually interesting than eye candy.

BLS

Re: [9fans] VMs, etc. (was: Re: security questions)

[blstuart]

> The seesion would not be suspended, it would continue to operate as
> your agent and identity and, typically, accept mail on your behalf,
> perform "background" operations such as pay your accounts and in
> general represent you to the web to the extent that security (or lack
> thereof, for many unsophisticated users) permits.  Nothing wrong with
> me having a private search bot to look for particular pornography or
> art or documentation while I'm asleep, the trick is to run it on
> whatever platform(s) are suitable at the time.

Okay, I think I have the picture now.  The idea of logging
in and out kind of goes out the window.  It gets replaced
by connecting, disconnecting, and migrating your online
alter ego.  So when I fire up a terminal, I'm don't announce
my presence, but instead pull the alter ego to whatever
machine I'm using at the moment.  Shutting down the
terminal would amount to pushing it back out into the
cloud.

>   The rest of the time, I
> find it preferable to use GPRS (3G is not yet available) for on-demand
> connections because I pay per volume and not for connect time.
> Naturally, that makes my network a roaming one.

And in cases like this, you basically remotely connect to
the "console" of your alter ego.

> Do you get my drift?

I think I've got a clearer picture than I had before.

> In passing, a device that struck me as being extremely handy is the
> 3G, USB dongle that is highly popular here, you mey be more familiar
> with it than I: it contains a simulated CD-ROM that it uses to install
> its software.  I though that was particularly clever, specially if you
> transform it into a Plan 9 or Inferno boot device.

That does sound cool.  The only 3G interface I've worked
with was PCMCIA and basically just looked like a modem.

> I'm sorry if I'm throwing around too many ideas with too little flesh,
> I must confess that I find this particular discussion very exciting, I
> have never really had occasion to look at these ideas as carefully as
> I am doing now.

I'm still inclined to think Inferno would make a good
starting point.  Suppose we have an Inferno configuration
without #U.  Now we move the whole Inferno instance
around.  Because apps don't have access to #U, state
information is pretty much confined to Styx connections
and resources managed by the Inferno kernel that gets
moved along with the running apps.  I'm sure that as
with any other approach to migration, there be dragons
there.  But as an initial line of thinking, it seems to be
a new model that's worth investigating.

BLS

Re: [9fans] VMs, etc. (was: Re: security questions)

[Mechiel Lukkien]

On Sat, Apr 18, 2009 at 10:54:34AM -0400, erik quanstrom wrote:
> as an old example, i think that the lab's use of worm storage
> for the main file server was incredibly insightful.
> 
> what could we do today, but don't quite dare?

stop writing all programs in C, and start writing them in a
higher-level language that takes care of the tedious work that's
hard to get right.  like memory/fd accounting and preventing bugs
from compromising system security.  dare i say limbo?

just another old example though...

mjl

Re: [9fans] VMs, etc. (was: Re: security questions)

[erik quanstrom]

On Fri Apr 17 16:22:55 EDT 2009, blstu...@bellsouth.net wrote:
> >> I often tell my students that every cycle used by overhead
> >> (kernel, UI, etc) is a cycle taken away from doing the work
> >> of applications.  I'd much rather have my DNA sequencing
> >> application finish in 25 days instead of 30 than to have
> >> the system look pretty during those 30 days.
> > 
> > i didn't mean to imply we should not be frugal with cycles.
> > i ment to say that we simply don't have anything useful to
> > do with a vast majority of cycles, and that's just as wasteful
> > as doing bouncing icons.  we need to work on that problem.
> 
> I gotcha.  I guess it depends on what you're doing.  I remember
> years ago running a simulation on the 11/750 we had.  It
> simulated a DSP chip running 2 seconds of real time.  It
> ran for over a week.  (While it was running, I took the time
> to write another, faster simulator that was able to run
> the simulation in about 2 hours.)  For something like that,
> we can certainly use all the cycles we can get.  On the other
> hand, I might look for a faster way to compile a kernel
> a while back, but now it compiles fast enough on most
> any machine that I'm not too concerned about where to
> use the cycles.  (I'm speaking of a Plan 9 or Inferno kernel
> here; not a *BSD or Linux kernel.)  But I suspect that
> virtualization and Dis-style VMs are a pretty good use of
> cycles we have to spare.

people's ideas about what's complicated or hard don't change
as quickly as computing power and storage has increased.  i
think there's currently a failure of imagination, at least on
my part.  there must be problems that aren't considered
because they were hard.

as an old example, i think that the lab's use of worm storage
for the main file server was incredibly insightful.

what could we do today, but don't quite dare?

- erik

Re: [9fans] VMs, etc. (was: Re: security questions)

[lucio]

> I guess I'm a little slow; it's taken me a little while to get
> my head around this and understand it.  Let me see if I've
> got the right picture.  When I "login" I basically look up a
> previously saved session in much the same way that LISP
> systems would save a whole environment.  Then when I
> "log off" my session is suspended and saved.  Alternatively,
> I could always log into the same previously saved state.

The seesion would not be suspended, it would continue to operate as
your agent and identity and, typically, accept mail on your behalf,
perform "background" operations such as pay your accounts and in
general represent you to the web to the extent that security (or lack
thereof, for many unsophisticated users) permits.  Nothing wrong with
me having a private search bot to look for particular pornography or
art or documentation while I'm asleep, the trick is to run it on
whatever platform(s) are suitable at the time.

Take my situation, for example.  I am at the dial-up end of an ISDN
BRA connection (2 x 64kbps channels for all intents and purposes, one
of them reserved for voice calls) which costs me a nominal amount to
stay connected (when the powers that be allow it) from 19:00 to 07:00
each weekday and from Friday evening to Monday morning (and a fortune
during what the Telco calls "peak time").  The rest of the time, I
find it preferable to use GPRS (3G is not yet available) for on-demand
connections because I pay per volume and not for connect time.
Naturally, that makes my network a roaming one.  Having my mail
exchanger et al.  in Cape Town permanently on line at a client's
premises provides the visibility I need all the time, but it is not
something I will continue to be able to afford as my involvement with
that client will eventually stop.  I'm not sure I can afford hosting
thereafter, but that is a separate issue.

The other organisation I am associated with has a hosted Linux server
I may use, or I may piggyback on their hosting contract, but I get too
little choice of platform on which to operate and even the hosting
structure may not suit me for a number of technical and political
reasons.

My dream is to be able to virtualise not so much the platform as the
"application" where application means whatever I feel like using at
the time.  Including being able to access on a low speed line the
stuff that is, say, strictly text based.  Or, as I often do, download
big volume items overnight, while I sleep.  But most of all, I want to
walk away from the workstation and pick up where I left off anywhere
else, including accessing the "profile" using the local resources, not
necessarily the extraordinary features I may have built for myself in
my workshop (I wish!).

Most of all, it must be possible for me to enhance my "profile"
wherever I am, teach it new tricks whenever I discover them, make it
aware that they may only work in specific locations.

Do you get my drift?

The crucial bit is that it depends heavily on "being on the network"
insofar as having access to resources you cannot possibly be expected
to carry on your laptop (now you need Windows, just now you need
MacOS, say, or connection to your burglar alarm).  In fact, my idea of
security is to deploy my mobile phone as the key, GPRS allows me a
very inexpensive, always on-line tool to provide, say, encryption keys
that have my identity firmly attached to them, practically anywhere in
South Africa and in most places in Africa, nevermind Europe
(connectivity was superb in Italy and Greece, last October) or the
USA.  Given the access key, any "terminal" ought to be able to provide
at least part of the "experience" I'm likely to need.

In passing, a device that struck me as being extremely handy is the
3G, USB dongle that is highly popular here, you mey be more familiar
with it than I: it contains a simulated CD-ROM that it uses to install
its software.  I though that was particularly clever, specially if you
transform it into a Plan 9 or Inferno boot device.

I'm sorry if I'm throwing around too many ideas with too little flesh,
I must confess that I find this particular discussion very exciting, I
have never really had occasion to look at these ideas as carefully as
I am doing now.

I was going to address the issue of being disconnected and I note that
to some extent I have, because once you treat your mobile phone as a
factor, being disconnected becomes a non-issue.  But if you do land in
a dead spot, for real, then, sure, you need much of your profile on
your portable.  How much lives in your phone (no matter how, that has
to be connected to a computing device or _be_ a computing device) and
how much on, say, on your laptop, is not important, as both have to be
with you, ideally they ought to be the same device and most likely
will be.  In fact, in a Plan 9 paradigm, the phone is the
CPU/fileserver, the laptop is the terminal (now you got me thinking!).
Replication is another issue that needs careful thought, although once

Re: [9fans] VMs, etc. (was: Re: security questions)

[Mechiel Lukkien]

On Fri, Apr 17, 2009 at 04:25:40PM -0500, blstu...@bellsouth.net wrote:
> Again, that's not to say that there aren't other valid motivators
> for some centralized functionality.  It's just that in my opinion,
> we're at the point were if it's raw cycles we need, we'll have
> to be looking at a large cluster and not a simple CPU server.

exactly.

the main use of a cpu server for me (and many others i suspect) is running
network services.  it's still nice to have a machine that's always on for
that (my terminals are not stable/always on enough for providing services
to others).  perhaps "cpu server" is a wrong name name.  "service server"
anyone? ;)

mjl

Re: [9fans] VMs, etc. (was: Re: security questions)

[tlaronde]

On Fri, Apr 17, 2009 at 04:25:40PM -0500, blstu...@bellsouth.net wrote:
> 
> Again, that's not to say that there aren't other valid motivators
> for some centralized functionality.  It's just that in my opinion,
> we're at the point were if it's raw cycles we need, we'll have
> to be looking at a large cluster and not a simple CPU server.

Well there is perhaps a hint about what we disagree about. I'm not using
CPU with the strict present meaning in Plan 9 but as a _logical_
processing unit (this can actually be, in this scheme, a cluster or
whatever).

This does not invalidate the logical difference between a terminal and
"a" CPU. A node can be both a CPU (resp. member of a CPU) and a terminal
etc. The plan 9 distinction, on the usage side et on the topology,
between FileServer, CPU and Terminal is sound and fundamental IMHO.

Enough for me at the moment since, even if I have some things on the
application side, for the rest my discussion of "cloud computing" could
be a discussion about "vapor computing" ;)
-- 
Thierry Laronde (Alceste) 
 http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] VMs, etc. (was: Re: security questions)

[blstuart]

> I'd like to add to Brian Stuart's comments the point that previous 
> specialization of various "boxes" is mostly disappearing. At some point in 
> near future all boxes may contain identical or very similar powerful 
> hardware--even probably all integrated into one "black box." So cheap that 

The domination of the commodity reminds me a lot of the
parallel processing world.  At one time, the big honkin'
machines had very custom interconnect designs and often
custom CPUs as well.  But by the time commodity CPUs
got to the point where they were competitive with what
you could do custom, and Ethernet got to the point where
it was competitive with what you could do custom, it
became very rare that you could justify a custom machine.
It was much more cost-effective to build a large cluster
of commodity machines.

For me, personally, this is leading to a point where my
home network is converging on a collection of laptops,
some get used the way most laptops get used, and some
just sit closed on shelves in the rack.  The primary hardware
differences between servers and terminals is that servers
have bigger disks and the lids on terminals tend to stay
open where on servers they tend to stay closed.  It's
getting farther away from the blinkin lights I miss, but
it sure makes my office more comfortable in the summer
both in terms of heat and noise.  Now if I could just get
that Cisco switch to be quieter...

BLS

Re: [9fans] VMs, etc. (was: Re: security questions)

[blstuart]

>> Absolutly, but part of what has changed over the past 20
>> years is that the rate at which this local processing power
>> has grown has been faster than rate at which the processing
>> power of the rack-mount box in the machine room has
>> grown (large clusters not withstanding, that is).  So the
>> gap between them has narrowed.
> 
> This is a geek attitude ;) You say that since I can buy something more
> powerful (if I do not change the programs for fatter ones...) for the
> same amount of money or a little more, I have to find something to do
> with that.

I'm not sure I follow.  The point where I would do something
special to get a more powerful system are several years past.
For example, a little over a year ago, the hinges on my work
laptop broke.  When ordering a new one, there was no need
to get a quote for one more powerful than the coporate standard
ones.  The ones in the "catalog" were powerful enough to do
pretty much anything I needed.  This is partly because the
performance has grown faster than my need and because the
performance gap with larger systems has closed.  In '89, a
desktop box would be something along the lines of an early
SPARCstation.  There was a pretty large gap between its
power and that of a large SGI machine one might use for a
CPU server.  Today, the difference between a base-model
machine and a single machine CPU server isn't as big as it
once was.

> My point of view is: if my terminal works, I keep it. If not, I buy
> something cheaper, including in TCO, for happily doing the work that has
> to be done ;)

I don't disagree.  For that matter, pretty much all the machines
I use here at home are ones that were surplus and I rescued.
But once you get to the point where the cheapest one you can
find has more than enough capability, performance ceases to
be a motivator for a separate CPU server.

Again, that's not to say that there aren't other valid motivators
for some centralized functionality.  It's just that in my opinion,
we're at the point were if it's raw cycles we need, we'll have
to be looking at a large cluster and not a simple CPU server.

BLS

Re: [9fans] VMs, etc. (was: Re: security questions)

[blstuart]

> What struck me when first looking at Xen, long after I had decided
> that there was real merit in VMware, was that it allowed migration as
> well as checkpoint/restarting of guest OS images with the smallest
>...
> 
> The way I see it, we would progress from conventional utilities strung
> together with Windows' crappy glue to having a single "profile"
> application, itself a virtualiser's guest, which includes any
> activities you may find useful online.  It sits on the web and follows

I guess I'm a little slow; it's taken me a little while to get
my head around this and understand it.  Let me see if I've
got the right picture.  When I "login" I basically look up a
previously saved session in much the same way that LISP
systems would save a whole environment.  Then when I
"log off" my session is suspended and saved.  Alternatively,
I could always log into the same previously saved state.

> you around, wherever you go. ...
> 
> Do you not like it? 

If I understand it, I at least find it interesting.  (I think I'd
have to try using it before I decided on preference.)  I can
easily see different saved environments that I use depending
on whether I'm at home or at work or wherever.  But what
happens if I'm not on any network at all?  The more I think
about it, the more I think this could be handled with the
same mechanism that handles better integration of laptops
and file servers.

> It smacks of Inferno and o/mero on top of a
> virtualiser-enhanced Plan 9. 

Hmmm.  It might be pretty easy to whip up a prototype
based on Inferno.  I must give this some thought...

BLS

Re: [9fans] VMs, etc. (was: Re: security questions)

[blstuart]

>> I often tell my students that every cycle used by overhead
>> (kernel, UI, etc) is a cycle taken away from doing the work
>> of applications.  I'd much rather have my DNA sequencing
>> application finish in 25 days instead of 30 than to have
>> the system look pretty during those 30 days.
> 
> i didn't mean to imply we should not be frugal with cycles.
> i ment to say that we simply don't have anything useful to
> do with a vast majority of cycles, and that's just as wasteful
> as doing bouncing icons.  we need to work on that problem.

I gotcha.  I guess it depends on what you're doing.  I remember
years ago running a simulation on the 11/750 we had.  It
simulated a DSP chip running 2 seconds of real time.  It
ran for over a week.  (While it was running, I took the time
to write another, faster simulator that was able to run
the simulation in about 2 hours.)  For something like that,
we can certainly use all the cycles we can get.  On the other
hand, I might look for a faster way to compile a kernel
a while back, but now it compiles fast enough on most
any machine that I'm not too concerned about where to
use the cycles.  (I'm speaking of a Plan 9 or Inferno kernel
here; not a *BSD or Linux kernel.)  But I suspect that
virtualization and Dis-style VMs are a pretty good use of
cycles we have to spare.

>> > that, plus the fact that the the mhz wars are dead and
>> > gone.
>> 
>> Does that mean we're all playing core wars now? :)
> 
> yes it does.  i've got $50 that says that in 2011 we'll be
> saying that this one goes to eleven (cores).

Excellent.  I never expected to see core wars and Spinal
Tap in the same discussion about Plan 9.

BLS

Re: [9fans] VMs, etc. (was: Re: security questions)

[J.R. Mauro]

On Fri, Apr 17, 2009 at 2:59 PM, Eris Discordia
 wrote:
>> even today on an average computer one has this articulation: a CPU (with
>> a FPU perhaps) ; tightly or loosely connected storage (?ATA or SAN) ;
>> graphical capacities (terminal) : GPU.
>
> It happens so that a reversal of specialization has really taken place, as
> Brian Stuart suggests. These "terminals" you speak of, GPUs, contain such
> vast untapped general processing capabilities that new uses and a new
> framework for using them are being defined: GPGPU and OpenCL.
>
> 
> 
>
> Right now, the GPU on my low-end video card takes a huge burden off of the
> CPU when leveraged by the right H.264 decoder. Two high definition AVC
> streams would significantly slow down my computer before I began using a
> CUDA-enabled decoder. Now I can easily play four in parallel.
>
> Similarly, the GPUs in PS3 boxes are being integrated into one of the
> largest loosely-coupled clusters on the planet.
>
> 
>
> Today, even a mere cellphone may contain enough processing power to run a
> low-traffic web server or a 3D video game. This processing power comes cheap
> so it is mostly wasted.

I can't find the link, but a recent article described someone's
efforts at CMU to develop what he calls "FAWN" Fast Array of Wimpy
Nodes. He basically took a bunch of eeePC boards and turned them into
a single computer.

The performance per watt of such an array was staggeringly higher than
a monster computer with Xeons and disks.

So hopefully in the future, we will be able to have more fine-grained
control over such things and fewer cycles will be wasted. It's time
people realized that CPU cycles are a bit like employment. Sure
UNemployment is a problem, but so is UNDERemployment, and the latter
is sometimes harder to gauge.

>
> I'd like to add to Brian Stuart's comments the point that previous
> specialization of various "boxes" is mostly disappearing. At some point in
> near future all boxes may contain identical or very similar powerful
> hardware--even probably all integrated into one "black box." So cheap that
> it doesn't matter if one or another hardware resource is wasted. To put to
> good use such a computational environment system software should stop
> incorporating a role-based model of various installations. All boxes, except
> the costliest most special ones, shall be peers.
>
> --On Friday, April 17, 2009 7:11 PM +0200 tlaro...@polynum.com wrote:
>
>> On Fri, Apr 17, 2009 at 11:32:33AM -0500, blstu...@bellsouth.net wrote:
>>>
>>> - First, the gap between the computational power at the
>>> terminal and the computational power in the machine room
>>> has shrunk to the point where it might no longer be significant.
>>> It may be worth rethinking the separation of CPU and terminal.
>>> For example, I'm typing this in acme running in a 9vx terminal
>>> booted using using a combined fs/cpu/auth server for the
>>> file system.  But I rarely use the cpu server capability of
>>> that machine.
>>
>> I'm afraid I don't quite agree with you.
>>
>> The definition of a terminal has changed. In Unix, the graphical
>> interface (X11) was a graphical variant of the text terminal interface,
>> i.e. the articulation (link, network) was put on the wrong place,
>> the graphical terminal (X11 server) being a kind of dumb terminal (a
>> little above a frame buffer), leaving all the processing, including the
>> handling of the graphical interface (generating the image,
>> administrating the UI, the menus) on the CPU (Xlib and toolkits run on
>> the CPU, not the Xserver).
>>
>> A terminal is not a no-processing capabilities (a dumb terminal):
>> it can be a full terminal, that is able to handle the interface,
>> the representation of data and commands (wandering in a menu shall
>> be terminal stuff; other users have not to be impacted by an user's
>> wandering through the UI).
>>
>> More and more, for administration, using light terminals, without
>> software installations is a way to go (less ressources in TCO). "Green"
>> technology. Data less terminals for security (one looses a terminal, not
>> the data), and data less for safety (data is centralized and protected).
>>
>>
>> Secondly, one is accustomed to a physical user being several distinct
>> logical users (accounts), for managing different tasks, or accessing
>> different kind of data.
>>
>> But (to my surprise), the converse is true: a collection of individuals
>> can be a single logical user, having to handle concurrently the very
>> same rw data. Terminals are then just distinct views of the same data
>> (imagine in a CAD program having different windows, different views of a
>> file ; this is the same, except that the windows are on different
>> terminals, with different "instances" of the logical user in front of
>> them).
>>
>> The processing is then better kept on a single CPU, handling the
>

Re: [9fans] VMs, etc. (was: Re: security questions)

[Eris Discordia]

even today on an average computer one has this articulation: a CPU (with
a FPU perhaps) ; tightly or loosely connected storage (?ATA or SAN) ;
graphical capacities (terminal) : GPU.


It happens so that a reversal of specialization has really taken place, as 
Brian Stuart suggests. These "terminals" you speak of, GPUs, contain such 
vast untapped general processing capabilities that new uses and a new 
framework for using them are being defined: GPGPU and OpenCL.





Right now, the GPU on my low-end video card takes a huge burden off of the 
CPU when leveraged by the right H.264 decoder. Two high definition AVC 
streams would significantly slow down my computer before I began using a 
CUDA-enabled decoder. Now I can easily play four in parallel.


Similarly, the GPUs in PS3 boxes are being integrated into one of the 
largest loosely-coupled clusters on the planet.




Today, even a mere cellphone may contain enough processing power to run a 
low-traffic web server or a 3D video game. This processing power comes 
cheap so it is mostly wasted.


I'd like to add to Brian Stuart's comments the point that previous 
specialization of various "boxes" is mostly disappearing. At some point in 
near future all boxes may contain identical or very similar powerful 
hardware--even probably all integrated into one "black box." So cheap that 
it doesn't matter if one or another hardware resource is wasted. To put to 
good use such a computational environment system software should stop 
incorporating a role-based model of various installations. All boxes, 
except the costliest most special ones, shall be peers.


--On Friday, April 17, 2009 7:11 PM +0200 tlaro...@polynum.com wrote:


On Fri, Apr 17, 2009 at 11:32:33AM -0500, blstu...@bellsouth.net wrote:

- First, the gap between the computational power at the
terminal and the computational power in the machine room
has shrunk to the point where it might no longer be significant.
It may be worth rethinking the separation of CPU and terminal.
For example, I'm typing this in acme running in a 9vx terminal
booted using using a combined fs/cpu/auth server for the
file system.  But I rarely use the cpu server capability of
that machine.


I'm afraid I don't quite agree with you.

The definition of a terminal has changed. In Unix, the graphical
interface (X11) was a graphical variant of the text terminal interface,
i.e. the articulation (link, network) was put on the wrong place,
the graphical terminal (X11 server) being a kind of dumb terminal (a
little above a frame buffer), leaving all the processing, including the
handling of the graphical interface (generating the image,
administrating the UI, the menus) on the CPU (Xlib and toolkits run on
the CPU, not the Xserver).

A terminal is not a no-processing capabilities (a dumb terminal):
it can be a full terminal, that is able to handle the interface,
the representation of data and commands (wandering in a menu shall
be terminal stuff; other users have not to be impacted by an user's
wandering through the UI).

More and more, for administration, using light terminals, without
software installations is a way to go (less ressources in TCO). "Green"
technology. Data less terminals for security (one looses a terminal, not
the data), and data less for safety (data is centralized and protected).


Secondly, one is accustomed to a physical user being several distinct
logical users (accounts), for managing different tasks, or accessing
different kind of data.

But (to my surprise), the converse is true: a collection of individuals
can be a single logical user, having to handle concurrently the very
same rw data. Terminals are then just distinct views of the same data
(imagine in a CAD program having different windows, different views of a
file ; this is the same, except that the windows are on different
terminals, with different "instances" of the logical user in front of
them).

The processing is then better kept on a single CPU, handling the
concurrency (and not the fileserver trying to accomodate). The views are
multiplexed, but not the handling of the data.

Thirdly, you can have a slow/loose link between a CPU and a terminal
since the commands are only a small fraction of the processing done.
You must have a fast or tight link between the CPU and the fileserver.

In some sense, logically (but not efficiently: read the caveats in the
Plan9 papers; a processor is nothing without tightly coupled memory, so
memory is not a remote pool sharable---Mach!), even today on an
average computer one has this articulation: a CPU (with a FPU
perhaps) ; tightly or loosely connected storage (?ATA or SAN) ;
graphical capacities (terminal) : GPU.

--
Thierry Laronde (Alceste) 
 http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] VMs, etc. (was: Re: security questions)

[tlaronde]

On Fri, Apr 17, 2009 at 01:31:12PM -0500, blstu...@bellsouth.net wrote:
> 
> Absolutly, but part of what has changed over the past 20
> years is that the rate at which this local processing power
> has grown has been faster than rate at which the processing
> power of the rack-mount box in the machine room has
> grown (large clusters not withstanding, that is).  So the
> gap between them has narrowed.

This is a geek attitude ;) You say that since I can buy something more
powerful (if I do not change the programs for fatter ones...) for the
same amount of money or a little more, I have to find something to do
with that.

My point of view is: if my terminal works, I keep it. If not, I buy
something cheaper, including in TCO, for happily doing the work that has
to be done ;)

I don't have to buy expensive things and try to find something to do
with them.

I try to have hardware that matches my needs.

And I prefer to put money on a CPU, more powerful, "far" from average
user "creativity", and the only beast I have to manage.

> 
> > The processing is then better kept on a single CPU, handling the
> > concurrency (and not the fileserver trying to accomodate). The views are
> > multiplexed, but not the handling of the data
> 
> That is part of the conversation the question is meant
> to raise.  If cycles/second isn't as strong a justification
> for separate CPU servers, then are there other reasons
> we should still have the separation?  If so, do we need
> to think differently about the model?

The main point I have discovered very recently is that giving access to 
the "system" resources is a centralized thing, and that a logical user
can have several distinct sessions on several distinct terminals, but
these are just "views": the data opened, especially for random rw is
opened by a single program.

Fileservers have only to provide what they do provide :

1) Random read/write for an uniq user.

2) Append only for shared data. (In KerGIS for example, some attributes
can be shared among users. So distinct (logical) users can open a file
rw, but they only append/write and the semantics of the data is so that
appending the n+1 records doesn't invalidate the [0,n]---records are
partitions, there is no overlapping. Changing the records (random
access) is possible but the cases are rare, and the stuff is done by the
user manager (another logical user)).

So the semantics of the data and the handling of users is so that a user
can randomly read/write (not sharable). A group can append/write but
without modifying records. And others can only (perhaps) read.

-- 
Thierry Laronde (Alceste) 
 http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] VMs, etc. (was: Re: security questions)

[erik quanstrom]

> I often tell my students that every cycle used by overhead
> (kernel, UI, etc) is a cycle taken away from doing the work
> of applications.  I'd much rather have my DNA sequencing
> application finish in 25 days instead of 30 than to have
> the system look pretty during those 30 days.

i didn't mean to imply we should not be frugal with cycles.
i ment to say that we simply don't have anything useful to
do with a vast majority of cycles, and that's just as wasteful
as doing bouncing icons.  we need to work on that problem.

> > that, plus the fact that the the mhz wars are dead and
> > gone.
> 
> Does that mean we're all playing core wars now? :)

yes it does.  i've got $50 that says that in 2011 we'll be
saying that this one goes to eleven (cores).

- erik

Re: [9fans] VMs, etc. (was: Re: security questions)

[lucio]

> But the question in my mind for a while
> has been, is it time for another step back and rethinking
> the big picture?

Maybe, and maybe what we ought to look at is precisely what Plan 9
skipped, with good reason, in its infancy: distributed "core"
resources or "the platform as a filesystem".

What struck me when first looking at Xen, long after I had decided
that there was real merit in VMware, was that it allowed migration as
well as checkpoint/restarting of guest OS images with the smallest
amount of administration.  Today, to me, that means distributed
virtualisation.  So, back to my first impression: Plan 9 would make a
much better foundation for a virtualiser than any of the other OSes
currently in use (limited to my experience, there may be something in
the league of IBM's 1960s VMS (do I remember right?  sanctions made
IBM a little scarce in my formative years) out there that I don't know
about).  Given a Plan 9 based virtualiser, are we far from using
long-running applications and migrating them in flight from whichever
equipment may have been useful yesterday to whatever is handy today?

The way I see it, we would progress from conventional utilities strung
together with Windows' crappy glue to having a single "profile"
application, itself a virtualiser's guest, which includes any
activities you may find useful online.  It sits on the web and follows
you around, wherever you go.  It is engineered against any possible
failures, including security-related ones and is always there for you.
Add Venti to its persistent objects and you can also rewind to a
better past state.

Do you not like it?  It smacks of Inferno and o/mero on top of a
virtualiser-enhanced Plan 9.  Those who might prefer the conventional
Windows/Linux platforms may have to wait a little longer before they
figure out how to catch up :-)

++L

Re: [9fans] VMs, etc. (was: Re: security questions)

[erik quanstrom]

On Fri Apr 17 14:21:03 EDT 2009, tlaro...@polynum.com wrote:
> On Fri, Apr 17, 2009 at 01:29:09PM -0400, erik quanstrom wrote:
> > > In some sense, logically (but not efficiently: read the caveats in the
> > > Plan9 papers; a processor is nothing without tightly coupled memory, so
> > > memory is not a remote pool sharable---Mach!), 
> > 
> > if you look closely enough, this kind of breaks down.  numa
> > machines are pretty popular these days (opteron, intel qpi-based
> > processors).  it's possible with a modest loss of performance to
> > share memory across processors and not worry about it.
> 
> NUMA are, from my point of view, "tightly" connected.
> 
> By loosely, I mean a memory accessed by non dedicated processor 
> hardware means (if this makes sense). Moving data from different
> memories via some IP based protocol or worse. But all in all,
> finally a copy is put in the tightly connected memory, whether huge
> caches, or dedicated main memory.

why do you care what gives you the illusion of a large, flat
address space?  that is, what is special about having a quick
path network instead of, say, infiniband or ethernet?

why does networking imply ip networking?

my point is that i think we need to recognize that there vast differences
in performance between, say, local memory, memory across the quickpath
bus, memory on the the next machine, and these differences may vary
greatly between one set of machines and another.

then, the 64¢ question is, how does one use this to one's advantage
without assuming ahead of time what's faster than what.

(one could easily imagine a 40gbps ethernet connection being competitive
with a 3-hop numa connection.)

- erik

Re: [9fans] VMs, etc. (was: Re: security questions)

[blstuart]

>> Absolutly, but part of what has changed over the past 20
>> years is that the rate at which this local processing power
>> has grown has been faster than rate at which the processing
>> power of the rack-mount box in the machine room has
>> grown (large clusters not withstanding, that is).  So the
>> gap between them has narrowed.
> 
> or, we have miserably failed as of late in putting ever cycle we
> can dream about to good use; we'd care more about the cycles
> of a cpu server if we were better at using them up.

What?  Dancing icons and sound effects for menu selections
are good use of cycles? :)

>   every cycle's perfect, every cycle's great
>   if one cycle's wasted, god gets quite irate

I often tell my students that every cycle used by overhead
(kernel, UI, etc) is a cycle taken away from doing the work
of applications.  I'd much rather have my DNA sequencing
application finish in 25 days instead of 30 than to have
the system look pretty during those 30 days.

> that, plus the fact that the the mhz wars are dead and
> gone.

Does that mean we're all playing core wars now? :)

BLS

Re: [9fans] VMs, etc. (was: Re: security questions)

[blstuart]

> if you look closely enough, this kind of breaks down.  numa
> machines are pretty popular these days (opteron, intel qpi-based
> processors).  it's possible with a modest loss of performance to
> share memory across processors and not worry about it.

Way back in the dim times when hypercubes roamed
the earth, I played around a bit with parallel machines.
When I was writing my master's thesis, I tried to find
a way to dispell the idea that shared-memory vs interconnection
network was as bipolar as the terms multiprocessor and
multicomputer would suggest.  One of the few things
in that work that I think still makes sense is characterizing
the degree of coupling as a continuum based on the ratio
of bytes transferred between CPUs to bytes accessed in
local memory.  So C.mmp would have a very high degree
of coupling and s...@home would have a very low degree
of coupling.  The upshot is that if I have a fast enough
network, my degree of coupling is high enough that I
don't really care whether or how much memory is local
and how much is on the other side of the building.

Of course, until recently, the rate at which CPU fetches
must be to keep the pipeline full has grown much
faster than network speeds.  So the idea of remote
memory hasn't been all that useful.  However, I wouldn't
be surprised to see that change over the next 10 to 20
years.  So maybe my local CPU will gain access to most
of its memory by importing /dev/memctl from a memory
server (1/2 :))

BLS

Re: [9fans] VMs, etc. (was: Re: security questions)

[erik quanstrom]

> Absolutly, but part of what has changed over the past 20
> years is that the rate at which this local processing power
> has grown has been faster than rate at which the processing
> power of the rack-mount box in the machine room has
> grown (large clusters not withstanding, that is).  So the
> gap between them has narrowed.

or, we have miserably failed as of late in putting ever cycle we
can dream about to good use; we'd care more about the cycles
of a cpu server if we were better at using them up.

every cycle's perfect, every cycle's great
if one cycle's wasted, god gets quite irate

that, plus the fact that the the mhz wars are dead and
gone.

- erik

Re: [9fans] VMs, etc. (was: Re: security questions)

[blstuart]

> The definition of a terminal has changed. In Unix, the graphical

In the broader sense of terminal, I don't disagree.  I was
being somewhat clumsy in talking about terminals in
the Plan 9 sense of the processing power local to my
fingers.

> A terminal is not a no-processing capabilities (a dumb terminal):
> it can be a full terminal, that is able to handle the interface,
> the representation of data and commands (wandering in a menu shall
> be terminal stuff; other users have not to be impacted by an user's
> wandering through the UI).

Absolutly, but part of what has changed over the past 20
years is that the rate at which this local processing power
has grown has been faster than rate at which the processing
power of the rack-mount box in the machine room has
grown (large clusters not withstanding, that is).  So the
gap between them has narrowed.

> The processing is then better kept on a single CPU, handling the
> concurrency (and not the fileserver trying to accomodate). The views are
> multiplexed, but not the handling of the data

That is part of the conversation the question is meant
to raise.  If cycles/second isn't as strong a justification
for separate CPU servers, then are there other reasons
we should still have the separation?  If so, do we need
to think differently about the model?

> In some sense, logically (but not efficiently: read the caveats in the
> Plan9 papers; a processor is nothing without tightly coupled memory, so

The flip side is actually what intrigues me more, namely
machines where the connection to the file system is
even more loosly coupled than sharing Ethernet.  I'd
like to have my usage on the laptop sitting in Starbucks
to be as much a part of the model as using one of
the BlueGene machines as an enormous CPU server
while sitting in the next room.

BLS

Re: [9fans] VMs, etc. (was: Re: security questions)

[tlaronde]

On Fri, Apr 17, 2009 at 01:29:09PM -0400, erik quanstrom wrote:
> > In some sense, logically (but not efficiently: read the caveats in the
> > Plan9 papers; a processor is nothing without tightly coupled memory, so
> > memory is not a remote pool sharable---Mach!), 
> 
> if you look closely enough, this kind of breaks down.  numa
> machines are pretty popular these days (opteron, intel qpi-based
> processors).  it's possible with a modest loss of performance to
> share memory across processors and not worry about it.

NUMA are, from my point of view, "tightly" connected.

By loosely, I mean a memory accessed by non dedicated processor 
hardware means (if this makes sense). Moving data from different
memories via some IP based protocol or worse. But all in all,
finally a copy is put in the tightly connected memory, whether huge
caches, or dedicated main memory.

The disaster of Mach (I don't know if my bad english is responsible for
this, but in the Plan9 paper the "research" or "university" OS that is
implicitely gibed at is Mach) is a kind of example.

NUMA are sufficiently special beasts that the majority of huge computing
facilities have been done by clusters (because it was easier for
software only organizations).

This definitively doesn't mean NUMA has no raison d'être. On the
contrary, this is an argument supplementary to the distinction
between the UI (terminals) and the CPU.

-- 
Thierry Laronde (Alceste) 
 http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] VMs, etc. (was: Re: security questions)

[erik quanstrom]

> In some sense, logically (but not efficiently: read the caveats in the
> Plan9 papers; a processor is nothing without tightly coupled memory, so
> memory is not a remote pool sharable---Mach!), 

if you look closely enough, this kind of breaks down.  numa
machines are pretty popular these days (opteron, intel qpi-based
processors).  it's possible with a modest loss of performance to
share memory across processors and not worry about it.

there is such an enormous difference in network speeds
(4 orders of magnitude 1mbps dsl/wireless up to 10gbps)
that it's hard to generalize but i don't see why tightly
coupled memory is an absolutely necessary.  you could
think of the network as 1/10th to 1/1th speed quickpath.
it may still be a big win.

> even today on an
> average computer one has this articulation: a CPU (with a FPU
> perhaps) ; tightly or loosely connected storage (?ATA or SAN) ;
> graphical capacities (terminal) : GPU.

plan 9 can make the nas/dasd dichotomy disappear.

import -E ssl storage.coraid.com '#S' /n/bigdisks

- erik

Re: [9fans] VMs, etc. (was: Re: security questions)

[tlaronde]

On Fri, Apr 17, 2009 at 11:32:33AM -0500, blstu...@bellsouth.net wrote:
> - First, the gap between the computational power at the
> terminal and the computational power in the machine room
> has shrunk to the point where it might no longer be significant.
> It may be worth rethinking the separation of CPU and terminal.
> For example, I'm typing this in acme running in a 9vx terminal
> booted using using a combined fs/cpu/auth server for the
> file system.  But I rarely use the cpu server capability of
> that machine.

I'm afraid I don't quite agree with you.

The definition of a terminal has changed. In Unix, the graphical
interface (X11) was a graphical variant of the text terminal interface,
i.e. the articulation (link, network) was put on the wrong place,
the graphical terminal (X11 server) being a kind of dumb terminal (a
little above a frame buffer), leaving all the processing, including the
handling of the graphical interface (generating the image,
administrating the UI, the menus) on the CPU (Xlib and toolkits run on
the CPU, not the Xserver).

A terminal is not a no-processing capabilities (a dumb terminal):
it can be a full terminal, that is able to handle the interface,
the representation of data and commands (wandering in a menu shall
be terminal stuff; other users have not to be impacted by an user's
wandering through the UI).

More and more, for administration, using light terminals, without
software installations is a way to go (less ressources in TCO). "Green"
technology. Data less terminals for security (one looses a terminal, not
the data), and data less for safety (data is centralized and protected).


Secondly, one is accustomed to a physical user being several distinct
logical users (accounts), for managing different tasks, or accessing
different kind of data.

But (to my surprise), the converse is true: a collection of individuals
can be a single logical user, having to handle concurrently the very
same rw data. Terminals are then just distinct views of the same data
(imagine in a CAD program having different windows, different views of a
file ; this is the same, except that the windows are on different
terminals, with different "instances" of the logical user in front of
them).

The processing is then better kept on a single CPU, handling the
concurrency (and not the fileserver trying to accomodate). The views are
multiplexed, but not the handling of the data.

Thirdly, you can have a slow/loose link between a CPU and a terminal
since the commands are only a small fraction of the processing done.
You must have a fast or tight link between the CPU and the fileserver.

In some sense, logically (but not efficiently: read the caveats in the
Plan9 papers; a processor is nothing without tightly coupled memory, so
memory is not a remote pool sharable---Mach!), even today on an
average computer one has this articulation: a CPU (with a FPU
perhaps) ; tightly or loosely connected storage (?ATA or SAN) ;
graphical capacities (terminal) : GPU.

-- 
Thierry Laronde (Alceste) 
 http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

[9fans] VMs, etc. (was: Re: security questions)

[blstuart]

> Actually, I have long had a feeling that there is a convergence of
> VNC, Drawterm, Inferno and the many virtualising tools (VMware, Xen,
> Lguest, etc.), but it's one of these intuition things that I cannot
> turn into anything concrete.

This brings to mind something that's been rolling around
in the back of my head for a while.  It was about 20 years
between the earliest UNIX and early Plan 9, and it's been
about 20 years since early Plan 9.  One of the major drivers
of Plan 9 was the change in the computing landscape that
UNIX had adapted to at best clunkily.  In particular, unlike
1969, 1) networks were nearly universal, 2) significant computing
was done at the terminal rather than back at the mini
at the other end of the RS-232 line, and 3) graphical interfaces
were quite common.  None of these were part of the UNIX
model and the ways they were acommodated by 1989
were, in allusion to Kidder, like paper bags taped on the
side of the machine.  Don't get me wrong, given the constraints
and uncertainties of the times, the early networking, GUI,
and distributed techniques were pretty good first cuts.
But by 1989, they were well-understood enough that
it made sense to reconsider them from scratch.

So what about today?  It seems to me there are also three
major aspects of the milieu that have changed since 1989.
- First, the gap between the computational power at the
terminal and the computational power in the machine room
has shrunk to the point where it might no longer be significant.
It may be worth rethinking the separation of CPU and terminal.
For example, I'm typing this in acme running in a 9vx terminal
booted using using a combined fs/cpu/auth server for the
file system.  But I rarely use the cpu server capability of
that machine.
- Second, network access has since become both ubituitous
and sporadic.  In 1989 being on the network meant sitting
at a fixed machine tethered to the wall by Ethernet.  Today,
one of the most common modes of use is the laptop that
we use to carry our computing world around with us.  We
might be on the network at home, at work, at a hotel, at
Starbucks, or not at all, even all in the same day.  So how
can a laptop and a file server play nice?
- Third, virtualization is no longer the domain of IBM big
iron (VM) and low-performance experiments (e.g. P-machines).
The current multi-core CPUs practically beg for virtualized
environments.

Am I suggesting another start-from-scratch project?  Not
necessarily, but I don't want to reject that out of hand,
either.  I tend to think that Plan 9 and Inferno can be a
good base that can adapt well to these changes.  Though
I'm inclined to think that there's an opportunity to create
a better hypervisor, inspired by these systems we know
and love.  As an example of the kind of rumination that
would be part of this process, is it possible to create a
hypervisor where the resources of one VM can be imported
by another with minimal (or better, no) modification to
the mainstream guys?  This would allow any OS to
leverage the device drivers written for another.

I've gone on long enough.  Those of you who have not
recently been laid off don't need to spend too much time
on my musings.  But the question in my mind for a while
has been, is it time for another step back and rethinking
the big picture?

BLS