Re: [Haskell-cafe] OS design FP aesthetics
On Mon, 18 Jun 2007, Creighton Hogg wrote: Okay, but these don't seem to really be design flaws so much as the inevitable results of age and the need for backwards compatibility. I'm looking more for technical problems that you would want to see fixed in our magical UberOS. I don't think I'll have much to say about Haskell or FP in general here, so this is way off-topic. In particular, I'm skeptical about relying on language guarantees as the basis of OS security, because that tends to limit the scope of your platform - you still want to support legacy C applications. Perhaps you can use typed assembly language techniques to check machine code before running it... dunno. Anyway: (1) Security partitioning. Current OSs are based on timesharing security, where users are protected from each other, but any process owned by a user can do ANYTHING to any other process owned by that user. This makes you utterly vulnerable to viruses, since if an attacker has compromised the user of a single user machine, they've effectively compromised the machine, whether or not they have got root. Furthermore, users don't have enough privilege to create new security partitions (i.e. new users), which would allow them protect themselves against untrusted code. As others have mentioned, a capability-based system is the ideal to aim for. (2) Unix has an assumption that disks are fast and the network is slow, in that you can't do something else while waiting for data from the disk, but you can with data from the network (using select() etc.) There are some AIO APIs that fix this problem, but they are disgusting and don't fit in with the network APIs. There's also the problem of paging which means that memory can unexpectedly take 10ms instead of 10ns to access. (3) The network APIs assume that memory bandwidth is cheap and that it makes sense to copy data from userland buffers to kernel buffers. It would be faster if buffers were sharable. There are some zero-copy APIs (e.g. sendfile), but they are not general. Cunning tricks to retro-fit zero copy onto the existing API depend on page table hacks and often have unexpectedly bad performance in some situations. (4) The system-call-as-function-call model assumes that system calls take negligible time. This is not true for filesystem calls, e.g. stat, open, etc. especially ones that aren't supported by an AIO API. Even fast system calls uncur the cost of two context switches. A better model is system-api-as-network-protocol, in which userland can send a bunch of requests to the kernel in one system call, and get the results back as and when they are available. There are some APIs like this already in the event-handling area, e.g. kqueue and epoll, but they are not general. The Linux syslets idea is heading in this direction. Tony. -- f.a.n.finch [EMAIL PROTECTED] http://dotat.at/ HEBRIDES BAILEY FAIR ISLE: EAST OR NORTHEAST 4 OR 5, OCCASIONALLY 6. SLIGHT TO MODERATE, OCCASIONALLY ROUGH. SHOWERS. MODERATE OR GOOD, OCCASIONALLY POOR. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] OS design FP aesthetics
Creighton Hogg wrote: (Also, have you noticed that no large Haskell applications exist? It's very hard to convince people that Haskell is not a toy language when no large applications exist. Building an entire *OS* with it would rather satisfy that requirement...!) Well, I imagine that there are large applications being used at places like Galois or Credit Suisse that are just proprietary. As for open source, I guess the largest application is probably GHC itself. GHC is certainly the only large application that *I* can think of... I don't know. It seems that Erlang is slowly taking over the world and all the rest of it, but you never hear anything about Haskell. Shame, really... To me, C is the pinacle of everything that is wrong with computer programming. At this point I can only say that we disagree and bow out of this tangent. Tangent I will agree on. ;-) ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] OS design FP aesthetics
On 6/18/07, Andrew Coppin [EMAIL PROTECTED] wrote: Creighton Hogg wrote: Well, since we're on the subject and it's only the Cafe list, what is it that you find messy about Linux that you would want to be solved by some hypothetical Haskell OS? This is drifting off-topic again, but here goes... Yeah, so I'll just split this off into a different thread. There are lots of things to like about Linux. It doesn't cost money. It's fast. It's reliable. It's flexible. It's secure. Okay, I'm not sure if I'd agree with the reliable secure points. I mean, relative to what could be done. I'm a rank amateur when it comes to OS work but when I've looked at recent papers Linux really isn't that cutting edge. I mean, it may be reliable in comparison to Windows 98 has less known exploits than any Windows system, but in terms of how good it *could* be I think there's an awful lot of room for growth. However, unfortunately it's still Unix. In other words, it's a vast incoherant mess of largely incompatible ad-hoc solutions to individual problems implemented independently by unrelated hackers over the 40+ years of history that this software has been around. New software has to emulate quirks in old software, and client programs work around the emulated quirks in the new software to get the functionallity it actually wants. One vast tangled mess of complexity and disorder. Exhibit A: Package managers exist. Exhibit B: Autoconf exists. I rest my case. Okay, but these don't seem to really be design flaws so much as the inevitable results of age and the need for backwards compatibility. I'm looking more for technical problems that you would want to see fixed in our magical UberOS. An operating system should have a simple, clear, consistent design. Not unlike a certain programming language named after a dead mathematition, come to think of it... (Have you ever programmed in C? You can certainly see where Unix gets its features from - terse, cryptic and messy.) This is another thing we're just going to disagree on. I think C++ is a pretty messy language, but feel that straight up C is rather simple and elegant. I had only used C++ before, but a friend rather easily convinced me that C is in fact a very sexy language when used in its intended design space. Still, I don't have the skill to write a functioning operating system - much less one that's ready for the desktop - so that's that I suppose... (I did seriously investigate the task once. Indeed, I got as far as writing a bootloader. It worked too!) Would you mind sharing the code? I'd be interested. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] OS design FP aesthetics
Creighton Hogg wrote: There are lots of things to like about Linux. It doesn't cost money. It's fast. It's reliable. It's flexible. It's secure. Okay, I'm not sure if I'd agree with the reliable secure points. I mean, relative to what could be done. I'm a rank amateur when it comes to OS work but when I've looked at recent papers Linux really isn't that cutting edge. I mean, it may be reliable in comparison to Windows 98 has less known exploits than any Windows system, but in terms of how good it *could* be I think there's an awful lot of room for growth. Isn't there a lot of room for improvement in *any* product? Okay, but these don't seem to really be design flaws so much as the inevitable results of age and the need for backwards compatibility. I'm looking more for technical problems that you would want to see fixed in our magical UberOS. Technically, both Windoze and Linux work. It would just be nice to have an OS with a coherent design, that's all. Nothing revolutionary (after all, it's an OS), just something done properly. (Also, have you noticed that no large Haskell applications exist? It's very hard to convince people that Haskell is not a toy language when no large applications exist. Building an entire *OS* with it would rather satisfy that requirement...!) (Have you ever programmed in C? You can certainly see where Unix gets its features from - terse, cryptic and messy.) This is another thing we're just going to disagree on. I think C++ is a pretty messy language, but feel that straight up C is rather simple and elegant. I had only used C++ before, but a friend rather easily convinced me that C is in fact a very sexy language when used in its intended design space. To me, C is the pinacle of everything that is wrong with computer programming. How does the saying go? C combines the power and flexibility of machine code with the ease of use of machine code. (Which isn't quite fair - machine code is easier to read than C.) For some reason, there is this perverse correlation... There are wonderful, beautiful languages like Haskell, that nobody uses and that you can't do anything practical with. And then there are ugly, complex, messy, flabby languages like C, C++, Perl, VisualBasic, etc. which are what everybody uses, and which have the power to actually do things. As somebody who's spent their entire life obsessively programming computers, this state of affairs makes me really sad. I really wish to God there was a language like Haskell that was useable in the real world. :-( (I did seriously investigate the task once. Indeed, I got as far as writing a bootloader. It worked too!) Would you mind sharing the code? I'd be interested. Seriously - we're talking about 1 page of assembly. That's it. Write it to the boot sector of a floppy, reboot your PC and it says Hello World on the screen. That's the sum total of how far I got. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] OS design FP aesthetics
(someone else's quotes are scattered through here, my mailer lost the nested quoting) On Jun 18, 2007, at 16:46 , Creighton Hogg wrote: There are lots of things to like about Linux. It doesn't cost money. It's fast. It's reliable. It's flexible. It's secure. As someone who was involved with early Linux development but eventually bailed to FreeBSD for personal use, and who currently supports Linux in his day job, I'm somewhat dubious about reliable and secure. One vast tangled mess of complexity and disorder. Exhibit A: Package managers exist. Exhibit B: Autoconf exists. I rest my case. Erm, package managers should exist regardless. That *multiple* incompatible package managers exist can be argued as a flaw. The real problem indicated by your exhibits (assuming the preceding adjustment) is that Unix is not a monolithic entity; it's the result of dozens of companies extending the original research OS in multiple incompatible directions, then multiple standards committees harmonizing them more or less by legitimizing every incompatible spec for which they couldn't get people to agree on a single path. The development of the pax utility by the POSIX committee is a particularly horrifying example. That said, autoconf also deals with some monstrosities nobody should have to deal with these days. (Ancient Eunice, anyone? Domain/OS?) (Have you ever programmed in C? You can certainly see where Unix gets its features from - terse, cryptic and messy.) Heh. Actually, Unix is terse because it was developed and originally used on 110-baud teletypewriters. I've *used* a KSR33; you quickly come to appreciate the fact that Unix commands are terse. C has its issues, but the original KR C was quite simple and elegant. (And IMO preferable to its predecessor BCPL.) -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] [EMAIL PROTECTED] system administrator [openafs,heimdal,too many hats] [EMAIL PROTECTED] electrical and computer engineering, carnegie mellon universityKF8NH ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] OS design FP aesthetics
On 6/18/07, Creighton Hogg [EMAIL PROTECTED] wrote: On 6/18/07, Andrew Coppin [EMAIL PROTECTED] wrote: Creighton Hogg wrote: There are lots of things to like about Linux. It doesn't cost money. It's fast. It's reliable. It's flexible. It's secure. Okay, I'm not sure if I'd agree with the reliable secure points. I mean, relative to what could be done. I'm a rank amateur when it comes to OS work but when I've looked at recent papers Linux really isn't that cutting edge. I mean, it may be reliable in comparison to Windows 98 has less known exploits than any Windows system, but in terms of how good it *could* be I think there's an awful lot of room for growth. Isn't there a lot of room for improvement in *any* product? Well, I'm not just talking about improvement. I'm talking about things like capabilities, self-healing kernels, separation kernels, exo kernels, things that may have serious advantages but can't necessarily be strapped on to a preexisting kernel such as Linux. Bah. Of course after I say this I get a bad feeling, so I checked the interwebs and found out that there has been work on incorporating self-healing into Linux. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] OS design FP aesthetics
Well, since we're on the subject and it's only the Cafe list, what is it that you find messy about Linux that you would want to be solved by some hypothetical Haskell OS? The hypothetical Haskell OS, especially if it were targeted toward 64 bit machines, could keep processes from messing with each other by way of language based security, and run them all in a single memory space. (The first system to do this, I believe, was the MULTIPOP timesharing system, but there are other precedents, too.) This would eliminate or simplify lots of context switches and buffer copies and memory management and other nastiness that now goes into kernels. /jaap ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] OS design FP aesthetics
On Jun 18, 2007, at 19:51 , Creighton Hogg wrote: The hypothetical Haskell OS, especially if it were targeted toward 64 bit machines, could keep processes from messing with each other by way of language based security, and run them all in a single memory space. (The first system to do this, I believe, was the MULTIPOP timesharing system, but there are other precedents, too.) This would eliminate or simplify lots of context switches and buffer copies and memory management and other nastiness that now goes into kernels. Okay, I remember seeing an example of this before , but I'm not sure if I see what language based security Haskell's type system could provide in protecting address spaces from each other. Normally I've seen capabilities used so that you can't access anything you can't name. Can you elaborate a little? He's saying that the language itself prevents programs from writing outside their address spaces or (via things like STM or runST) accessing resources they don't own. Which is a nice theory, but is dependent on the runtime not being buggy (I think some problems have been demonstrated with large arrays in GHC...). -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] [EMAIL PROTECTED] system administrator [openafs,heimdal,too many hats] [EMAIL PROTECTED] electrical and computer engineering, carnegie mellon universityKF8NH ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] OS design FP aesthetics
On 6/19/07, Creighton Hogg [EMAIL PROTECTED] wrote: Okay, I remember seeing an example of this before , but I'm not sure if I see what language based security Haskell's type system could provide in protecting address spaces from each other. Normally I've seen capabilities used so that you can't access anything you can't name. Can you elaborate a little? Nothing new here. Haskell might be more elegant than some of the earlier proposals, but the idea is old. FWIW, my PhD supervisor, Zoltan Somogyi did his PhD thesis on exactly such a scheme, using logic programming, rather than lazy functional programming. I'm not sure if the thesis itself is online. cheers, T. -- Dr Thomas Conway [EMAIL PROTECTED] Silence is the perfectest herald of joy: I were but little happy, if I could say how much. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] OS design FP aesthetics
On 6/18/07, Thomas Conway [EMAIL PROTECTED] wrote: On 6/19/07, Creighton Hogg [EMAIL PROTECTED] wrote: Okay, I remember seeing an example of this before , but I'm not sure if I see what language based security Haskell's type system could provide in protecting address spaces from each other. Normally I've seen capabilities used so that you can't access anything you can't name. Can you elaborate a little? Nothing new here. Haskell might be more elegant than some of the earlier proposals, but the idea is old. FWIW, my PhD supervisor, Zoltan Somogyi did his PhD thesis on exactly such a scheme, using logic programming, rather than lazy functional programming. I'm not sure if the thesis itself is online. Actually it is! Thank you for the reference. http://www.cs.mu.oz.au/~zs/papers/papers.html in case anyone else is interested ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] OS design FP aesthetics
Normally I've seen capabilities used so that you can't access anything you can't name. Can you elaborate a little? He's saying that the language itself prevents programs from writing outside their address spaces Yep. Capabilities are usually not actually unforgeable, they are just picked from a largish key space. You can guess at them if you want to bother. Somewhere in the Exokernel papers, there is some discussion of this, and reference to the fact that a 64 bit capability is at least as secure as an 8 byte UNIX password, which I suppose is a fair assessment of the situation. With language based security, you can have unforgeable capabilities, though. An example of a system based on that idea, although expressed in different words, is http://mumble.net/~jar/pubs/secureos/secureos.html An alternative approach to unforgeable capabilities is to build them into the hardware using a tagged architecture. There is a paper by Tom Knight (mastermind of the original Lisp Machine) and Jeremy Brown that is relevant here, although it actually deals with the somewhat different problem of data dissemination control (think DRM but more for military applications and suchlike): http://www.ai.mit.edu/projects/aries/Documents/Memos/ARIES-15.pdf Note that earlier tagged architectures, including the Knight Machine and the K-machine, used tags mostly to speed up Lisp and not to enforce any sort of type discipline. Which is a nice theory, but is dependent on the runtime not being buggy (I think some problems have been demonstrated with large arrays in GHC...). There are three ways to deal with this. The first is the aforementioned tagged architecture in hardware. The second is running everything within a virtual machine that has a tagged architecture. In both of these cases, there would be a trusted security kernel, but you could make it pretty damn small. Unfortunately hardware is expensive and virtual machines are slow (although people keep claiming that they're getting better, and FPGAs are getting faster and cheaper all the time). The third way is to have a compiler that compiles Haskell, or any other typesafe language, into a typed assembly language with a sufficiently expressive type system. You can then feed TAL-annotated binaries to the (trusted) loader. This way, you can also support multiple programming languages while still maintaining language based security and a small trusted security kernel, viz. the TAL type checker. /jaap ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] OS design FP aesthetics
On Tue, Jun 19, 2007 at 02:36:31AM +0200, Jaap Weel wrote: Normally I've seen capabilities used so that you can't access anything you can't name. Can you elaborate a little? He's saying that the language itself prevents programs from writing outside their address spaces Yep. Capabilities are usually not actually unforgeable, they are just picked from a largish key space. You can guess at them if you want to bother. Somewhere in the Exokernel papers, there is some discussion of this, and reference to the fact that a 64 bit capability is at least as secure as an 8 byte UNIX password, which I suppose is a fair assessment of the situation. Every capability system I've seen works like Unix file descriptors. The kernel assigns capability numbers, and since the numbers are only valid in one process, and the only valid capability numbers are to capabilities your have, there is no danger caused by guessing. Stefan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] OS design FP aesthetics
Every capability system I've seen works like Unix file descriptors. The kernel assigns capability numbers, and since the numbers are only valid in one process, and the only valid capability numbers are to capabilities your have, there is no danger caused by guessing. You know, when I typed that, I knew I really ought to qualify it a bit, because the word capability is used in several ways. You are, of course, right to say that this is a common implementation of capabilities in operating systems with multiple memory spaces, but it does not work in a single memory space design without language security where user processes can access the kernel tables. /jaap ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] OS design FP aesthetics
On 6/18/07, Creighton Hogg [EMAIL PROTECTED] wrote: On 6/18/07, Andrew Coppin [EMAIL PROTECTED] wrote: Creighton Hogg wrote: Well, since we're on the subject and it's only the Cafe list, what is it that you find messy about Linux that you would want to be solved by some hypothetical Haskell OS? This is drifting off-topic again, but here goes... Yeah, so I'll just split this off into a different thread. There are lots of things to like about Linux. It doesn't cost money. It's fast. It's reliable. It's flexible. It's secure. Okay, I'm not sure if I'd agree with the reliable secure points. I mean, relative to what could be done. I'm a rank amateur when it comes to OS work but when I've looked at recent papers Linux really isn't that cutting edge. I mean, it may be reliable in comparison to Windows 98 has less known exploits than any Windows system, but in terms of how good it *could* be I think there's an awful lot of room for growth. However, unfortunately it's still Unix. In other words, it's a vast incoherant mess of largely incompatible ad-hoc solutions to individual problems implemented independently by unrelated hackers over the 40+ years of history that this software has been around. New software has to emulate quirks in old software, and client programs work around the emulated quirks in the new software to get the functionallity it actually wants. One vast tangled mess of complexity and disorder. Exhibit A: Package managers exist. Exhibit B: Autoconf exists. I rest my case. Okay, but these don't seem to really be design flaws so much as the inevitable results of age and the need for backwards compatibility. I'm looking more for technical problems that you would want to see fixed in our magical UberOS. An operating system should have a simple, clear, consistent design. Not unlike a certain programming language named after a dead mathematition, come to think of it... (Have you ever programmed in C? You can certainly see where Unix gets its features from - terse, cryptic and messy.) This is another thing we're just going to disagree on. I think C++ is a pretty messy language, but feel that straight up C is rather simple and elegant. I had only used C++ before, but a friend rather easily convinced me that C is in fact a very sexy language when used in its intended design space. Still, I don't have the skill to write a functioning operating system - much less one that's ready for the desktop - so that's that I suppose... (I did seriously investigate the task once. Indeed, I got as far as writing a bootloader. It worked too!) Would you mind sharing the code? I'd be interested. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe While this isn't an operating system written in a functional programming language it is quite an important part of one. NixOS is an experiment to see if we can build an operating system in which software packages, configuration files, boot scripts and the like are all managaed in a purely functional way, that is, they are all built by deterministic functions and they never change after they have been built., from http://nix.cs.uu.nl/nixos/index.html One thing microsoft has being doing which is interesting is singularity. It is a research OS done in .NET and is completely managed. It will be interesting to see the effects of a managed runtime environment and quite possibly open the door for a functional language to target the runtime. I think operating systems, and software design in general, will be headed towards integrating functional techniques from languages like Haskell into C and C++. Google's map/reduce paper is an excellent example but so is Tim Sweeney's talk on the future of video game design. I suppose more importantly.. would haskell kernel be done as a microkernel or a monolithic kernel ;-) Marketing it would be hard. Who would want to buy a lazy os? --Ryan Dickie ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] OS design FP aesthetics
software packages, configuration files, boot scripts and the like are all managaed in a purely functional way, that is, they are all built by deterministic functions and they never change after they have been built., from http://nix.cs.uu.nl/nixos/index.html One thing microsoft has being doing which is interesting is singularity. its just not clear what these projects are trying to fix. i am all for good research, indeed a lot of it manifests itself in linux, bsd etc...but chucking a decade+ of debugging, userland tools, docs, etc...its a nonstarter. who is downloading minix3? purity of design at the os level has few takers in the real world. people want high performance, small footprint, and security. linux is delivering on these as good as anything else actually in use by mass markets. there have been other softball comments here, like the offhand comment that linux is not reliable and secure, but i haven't seen any substantial commentary on why this is the case. try these comments out on the kernel mailing list if you want to be brave AND ontopic. i say all of this as a freebsd user, so don't construe my defense of linux as a political bias. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
