Re: vkernel GSoC, some questions
On Sun, Mar 16, 2008 at 11:42:57AM +0200, Jordan Gordeev wrote: vkernel is similar to User Mode Linux technology. You can boot vkernel as a user mode process. I think it will be good to have similar in FreeBSD. There are several links: http://leaf.dragonflybsd.org/mailarchive/users/2007-01/msg00237.html http://www.dragonflybsd.org/docs/articles/vkernel/vkernel.shtml The two links that Andrey posted are very good. I just want to add a short summary: A vkernel is a kernel running as a user process under a real kernel. The vkernel runs in the CPU's priviledge ring 3. It services its child processes like a normal kernel, but whenever a page table needs to be modified, context switched, or some other privileged operation needs to be executed, the vkernel asks the real kernel through a syscall interface. True, but ISTR that contrary to User-Mode Linux, the virtual memory operations are handled by the host kernel, which should increase speed. I have no pointer for this, I've got this information on #dragonflybsd. Regards, -- Jeremie Le Hen jeremie at le-hen dot org ttz at chchile dot org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Summer of Code 2008 Project Ideas
On Mon, 17 Mar 2008, Murray Stokely wrote: The FreeBSD Project was again accepted as a mentoring organization for the Google Summer of Code. The student application period will begin next week so if you have any ideas for great student projects, please send them to [EMAIL PROTECTED] or post them here for discussion. A good student project has a well defined purpose, some key FreeBSD developers that could be identified as potential mentors, and is feasible for a student to complete in a few months time. The existing ideas list is available here : http://www.freebsd.org/projects/ideas/ If you can suggest something (getting specific parts of valgrind working on FreeBSD?) then please provide information in the form of the other projects listed on the page as far as difficulty level, requirements, etc.. FYI, to students considering doing projects -- in the last day or two, we've made significant updates to the project ideas list. If you looked a few days ago, please look again. In particular, we've flagged a large number of potential SoC projects that were not there previously. We've also filtered out some that looked too big to be a good 3-month summer project, although you can still find many on the full ideas list. If you're going to work on a proposal for one of these projects, please directly contact the contacts listed for the project to get feedback before submitting your proposal. We will continue to update the project ideas page as new ideas come in, so do keep checking back. Robert N M Watson Computer Laboratory University of Cambridge ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: vkernel GSoC, some questions
Hi, Sorry for jumping in here, but I've seen several people talking about that 5 seconds to reboot thing ... Are you aware that a standard FreeBSD kernel also takes just 5 seconds to reboot within qemu? And that's even when _not_ using the kqemu accelerator module. I've used qemu a lot for debugging my /boot/loader stuff. It takes just about 1 second to get to the loader (i.e. from starting qemu on the command line to the loader menu popping up on the screen). Going single-user takes about 3 seconds. Going multi-user takes a little bit longer, depending on what stuff is enabled via rc.conf. The vkernel feature has certainly benefits, e.g. the fact that you can attach to it with standard gdb and use the familiar debugging facilities, which can attract more developers who wouldn't dare to touch the kernel otherwise. But the 5 seconds to reboot is not a unique feature; you can have that with qemu as well, and it's there today. Best regards Oliver -- Oliver Fromme, secnetix GmbH Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd I have stopped reading Stephen King novels. Now I just read C code instead. -- Richard A. O'Keefe ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: remote operation or admin
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jeremy Chadwick wrote: On Mon, Mar 17, 2008 at 08:43:49PM -0400, Chuck Robey wrote: I have 4 computers, 1 big FreeBSD-current (4 x86 procs), 2 GentooLinux (1 is a dial AMD Opteron, the other a dual older x86), and 1 MacOSX (dual PPC). I was thinking about looking for two items, I'm not sure if I want one or both of them: either some software to let me merely remotely manage them (public software, mind) or, even better, something to get these disparate hardwares to be able to work together, and (as much as possible) to be able to share work. What might be the best, in terms of ability, and especially the ability to make these work together? If they're not a FreeBSD port, as long as they're reasonably stable, I don't mind porting things, but it needs to be stable on all those CPUs. Could you reo\commend me something? I'll go chase each one down, I won't jump on you if you're wrong, gimme your guesses, ok? I don't understand your question. It's almost like you're asking two questions: 1) How can I manage all of these machines remotely? (E.g. ssh, VNC, serial console, KVM, etc.) 2) How can I harness the power of all of these machines simultaneously? (e.g. some form of CPU clustering) Can you elaborate? Sure. I suppose what was really driving me was a little daydreaming that I'd been having, about the direction of computing, in the coming few years. Not 20 years ahead, but more likely 2 or 4 years ahead. I was thinking that the present wave towards constructing single-ships with multiple cores on them could only grow in scale, so the real question would be, just how large could n get? Then, IF that were true, it begs another question ... right now, we rely upon our fine smp code to do a fairly good job of both managing the administration of those cores, AND the dispatching of jobs amongst them. By now, you've probably smelled the rat: I didn't allow for the difference in hardware that I'd painted: 2 x86 cores, 6 x86_64 cores, and 2 PPC cores. Well, I had to throw all that in if I were to reach for the sky, but I was asking for all that I could possibly get, but not, honestly, unwilling to consider something maybe a bit less than that, if it were easier to consider. What is most important in my considerations are, how might it to possible to stretch our present smp software to be able to extend the management domains to cover multiple computers? Some sort of a bridge here, because there is no software today (that I'm awarae of, and that sure leaves a huge set of holes) that lets you manage the cores as separate computers) so that maybe today I might be able to have an 8 or 10 core system, and maybe tomorrow look at the economic and software possibility of having a 256 core system. I figure that there would need to be some tight reins on latency, and you would want some BIGTIME comm links, I dunno, maybe not be able to use even Gigabit ethernet, maybe needing some sort of scsi bus linkage, something on that scale? Or, is Fiber getting to that range yet? Anyhow, is it even remotely posible for us to be able to strech our present SMP software (even with it's limitation on word size to limit the range to 32 processors) to be able to jump across machines? That would be one hell of a huge thing to consider, now wouldn't it? -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFH4Ub5z62J6PPcoOkRAl9qAJ0fROAdNq/QFFR+7cYix0BvwjGZhQCeIsUH fAWfi7eKgNogAF6uaKpgMoI= =ZIcm -END PGP SIGNATURE- ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: remote operation or admin
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Bill Moran wrote: In response to Chuck Robey [EMAIL PROTECTED]: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I have 4 computers, 1 big FreeBSD-current (4 x86 procs), 2 GentooLinux (1 is a dial AMD Opteron, the other a dual older x86), and 1 MacOSX (dual PPC). I was thinking about looking for two items, I'm not sure if I want one or both of them: either some software to let me merely remotely manage them (public software, mind) or, even better, something to get these disparate hardwares to be able to work together, and (as much as possible) to be able to share work. What might be the best, in terms of ability, and especially the ability to make these work together? If they're not a FreeBSD port, as long as they're reasonably stable, I don't mind porting things, but it needs to be stable on all those CPUs. Could you reo\commend me something? I'll go chase each one down, I won't jump on you if you're wrong, gimme your guesses, ok? Your question is extraordinarily vague. I manage multiple systems remotely using ssh, so I'm not sure what additional stuff you want. I left it vague on purpose, because I know I am speaking to a crowd thjat certainly includes folks who know more on the subject than i do, and I was curious as to what woud pop up. I filled in some more of the puzzle in a separate email i just sent (so i won't duplicate that all here), but I have already begun to learn, because, no, I didn't know off all these links. I was thinking (very, very quickly) of putting our smp and the Ganglia thing into a mixer, 2 minutes at high speed, and see what came out? Anyhow, thanks for the links, please don't hesitate to give me more on this subject (this fuzzy cloud?) However, here are some links that may provide some help. All of these are FreeBSD ports, and I'm fairly sure they'll work on Gentoo and OSX as well: http://www.webmin.com/ http://www.cfengine.org/ http://reductivelabs.com/trac/puppet/ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFH4UhHz62J6PPcoOkRAvwFAJ9Y3Q1osqjUJVrQrB89Ya9RczLIowCbBC8V xqihMxtA9lwvt2TEPl8xBN0= =6QlR -END PGP SIGNATURE- ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: remote operation or admin
On Wed, Mar 19, 2008 at 01:01:45PM -0400, Chuck Robey wrote: What is most important in my considerations are, how might it to possible to stretch our present smp software to be able to extend the management domains to cover multiple computers? Some sort of a bridge here, because there is no software today (that I'm awarae of, and that sure leaves a huge set of holes) that lets you manage the cores as separate computers) so that maybe today I might be able to have an 8 or 10 core system, and maybe tomorrow look at the economic and software possibility of having a 256 core system. I figure that there would need to be some tight reins on latency, and you would want some BIGTIME comm links, I dunno, maybe not be able to use even Gigabit ethernet, maybe needing some sort of scsi bus linkage, something on that scale? Or, is Fiber getting to that range yet? Anyhow, is it even remotely posible for us to be able to strech our present SMP software (even with it's limitation on word size to limit the range to 32 processors) to be able to jump across machines? That would be one hell of a huge thing to consider, now wouldn't it? Ahh, you're talking about parallel computing, clustering, or grid computing. The Linux folks often refer to an implementation called Beowulf: http://en.wikipedia.org/wiki/Beowulf_%28computing%29 I was also able to find these, more specific to the BSDs: http://www.freebsd.org/advocacy/myths.html#clustering http://lists.freebsd.org/pipermail/freebsd-cluster/2006-June/000292.html http://people.freebsd.org/~brooks/papers/bsdcon2003/fbsdcluster/ -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: remote operation or admin
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jeremy Chadwick wrote: On Wed, Mar 19, 2008 at 01:01:45PM -0400, Chuck Robey wrote: What is most important in my considerations are, how might it to possible to stretch our present smp software to be able to extend the management domains to cover multiple computers? Some sort of a bridge here, because there is no software today (that I'm awarae of, and that sure leaves a huge set of holes) that lets you manage the cores as separate computers) so that maybe today I might be able to have an 8 or 10 core system, and maybe tomorrow look at the economic and software possibility of having a 256 core system. I figure that there would need to be some tight reins on latency, and you would want some BIGTIME comm links, I dunno, maybe not be able to use even Gigabit ethernet, maybe needing some sort of scsi bus linkage, something on that scale? Or, is Fiber getting to that range yet? Anyhow, is it even remotely posible for us to be able to strech our present SMP software (even with it's limitation on word size to limit the range to 32 processors) to be able to jump across machines? That would be one hell of a huge thing to consider, now wouldn't it? Ahh, you're talking about parallel computing, clustering, or grid computing. The Linux folks often refer to an implementation called Beowulf: http://en.wikipedia.org/wiki/Beowulf_%28computing%29 I was also able to find these, more specific to the BSDs: http://www.freebsd.org/advocacy/myths.html#clustering http://lists.freebsd.org/pipermail/freebsd-cluster/2006-June/000292.html http://people.freebsd.org/~brooks/papers/bsdcon2003/fbsdcluster/ Well, I am, and I'm not, if you could answer me one quiestion, then I would probably know for sure. What is the difference between our SMP and the general idea of clustering, as typified by Beowulf? I was under the impression I was talking about seeing the possibility of moving the two closer together, but maybe I'm confused in the meanings? -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFH4VWKz62J6PPcoOkRAtxnAJoDKlh4EgbL17TLUuH3gEDCwt4u9ACePFGh dWFantC9QyS5g9FJh80wXNI= =viHY -END PGP SIGNATURE- ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: remote operation or admin
On Wed, Mar 19, 2008 at 02:03:54PM -0400, Chuck Robey wrote: Well, I am, and I'm not, if you could answer me one quiestion, then I would probably know for sure. What is the difference between our SMP and the general idea of clustering, as typified by Beowulf? I was under the impression I was talking about seeing the possibility of moving the two closer together, but maybe I'm confused in the meanings? SMP as an implementation is mainly intended for single systems with multiple processors (e.g. multiple physical CPUs, or multiple cores; same thing). It distributes kernel operations (kernel threads) across those processors, rather than only utilising a single processor. Clustering allows for the distribution of a task (a compile using gcc, running of certain disk I/O tasks, running multiple userland (or I suppose kernel, if the kernel had clustering support) threads) across multiple physical computers on a local network. The best example I have for real-world clustering is rendering (mostly 3D, but you can render anything; I'm referring to 3D in this case). A person doing modelling creates a model scene using 3D objects, applies textures to it, lighting, raytracing aspects, vertex/bones animation, and anything else -- all using their single workstation. Then the person wants to see what it all looks like -- either as a still frame (JPEG/PNG/TIFF), or as a rendered animation (AVI/MPG/MJPEG). Without any form of clustering, the workstation has to do all of the processing/rendering work by its lonesome self. This can take a very, very long time -- modellers aren't going to wait 2 hours for their work to render, only to find they messed up some bones vertexes half way into the animation. With clustering, the workstation has the capability to send the rendering request out onto the network to a series of what're called slaves (other computers set up to handle such requests). The workstation says I want this rendered. I want all of you to do it. Let's say there's 200 machines in the cluster as slaves, and let's say all 200 of those machines are dual-core (so 400 CPUs total). You then have 400 CPUs rendering your animation, versus just 2 on the workstation. The same concept can apply to compiling (gcc saying I want this C file compiled or whatever), or any other distributed computing computational desired. It all depends on if the software you want to support clustering can do it. Different clustering softwares run at different levels; some might act as virtual environments, thus underlying software may not need to know about clustering (e.g. it just works); others might require each program to be fully cluster-aware. Make sense? :-) -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: remote operation or admin
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jeremy Chadwick wrote: On Wed, Mar 19, 2008 at 02:03:54PM -0400, Chuck Robey wrote: Well, I am, and I'm not, if you could answer me one quiestion, then I would probably know for sure. What is the difference between our SMP and the general idea of clustering, as typified by Beowulf? I was under the impression I was talking about seeing the possibility of moving the two closer together, but maybe I'm confused in the meanings? SMP as an implementation is mainly intended for single systems with multiple processors (e.g. multiple physical CPUs, or multiple cores; same thing). It distributes kernel operations (kernel threads) across those processors, rather than only utilising a single processor. Clustering allows for the distribution of a task (a compile using gcc, running of certain disk I/O tasks, running multiple userland (or I suppose kernel, if the kernel had clustering support) threads) across multiple physical computers on a local network. The best example I have for real-world clustering is rendering (mostly 3D, but you can render anything; I'm referring to 3D in this case). A person doing modelling creates a model scene using 3D objects, applies textures to it, lighting, raytracing aspects, vertex/bones animation, and anything else -- all using their single workstation. Then the person wants to see what it all looks like -- either as a still frame (JPEG/PNG/TIFF), or as a rendered animation (AVI/MPG/MJPEG). Without any form of clustering, the workstation has to do all of the processing/rendering work by its lonesome self. This can take a very, very long time -- modellers aren't going to wait 2 hours for their work to render, only to find they messed up some bones vertexes half way into the animation. With clustering, the workstation has the capability to send the rendering request out onto the network to a series of what're called slaves (other computers set up to handle such requests). The workstation says I want this rendered. I want all of you to do it. Let's say there's 200 machines in the cluster as slaves, and let's say all 200 of those machines are dual-core (so 400 CPUs total). You then have 400 CPUs rendering your animation, versus just 2 on the workstation. The same concept can apply to compiling (gcc saying I want this C file compiled or whatever), or any other distributed computing computational desired. It all depends on if the software you want to support clustering can do it. Different clustering softwares run at different levels; some might act as virtual environments, thus underlying software may not need to know about clustering (e.g. it just works); others might require each program to be fully cluster-aware. Make sense? :-) Not completely yet (I tend to be stubborn, if I carry this too far, tell me in private mail and I will politely drop it). Your use cases show me the differences in size, and *because* of the size, the differences in how you'd use them, and that part I did already know. I'm perfectly well aware of the current differences in size, but what I'm after is what are the real differences, ignoring size, in what they actually accomplish, and how they go about doing it. I'm thinking of the possibility of perhaps finding it it might be possible to find some way to extend the work domain of an smp system to stretch across machine lines, to jump across motherboards. Maybe not to be global (huge latencies scare me away), but what about just going 3 feet, on a very high speed bus, like maybe a private pci bus? Not what is, what could be? And, I have experienced just as many looniees as you have, who ask for some gigantic task, then sit back and want to take the credit and act like a cheerleader, figuring they really got the thing going. Well, I DON'T really want the help, I have in mind a project for me and a friend, with small smallish machine resources, maybe a bunch of small ARM boards. I wouldn't turn down help, but I'm not really proselytizing. Something small, but with a bunch of bandwidth. So, in that case, what really are the differences between smp and clustering, besides the raw current size of the implementation? Are there huge basic differences between the clustering concept and by smp's actual tasks? -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFH4WUUz62J6PPcoOkRAoJNAJ4o4jpDeVlLG2X6Sk1ccggORyO2GwCgku/s U3w/oylwEgt8CufFZwccfw4= =zbNA -END PGP SIGNATURE- ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: remote operation or admin
Chuck Robey wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jeremy Chadwick wrote: On Wed, Mar 19, 2008 at 02:03:54PM -0400, Chuck Robey wrote: Well, I am, and I'm not, if you could answer me one quiestion, then I would probably know for sure. What is the difference between our SMP and the general idea of clustering, as typified by Beowulf? I was under the impression I was talking about seeing the possibility of moving the two closer together, but maybe I'm confused in the meanings? SMP as an implementation is mainly intended for single systems with multiple processors (e.g. multiple physical CPUs, or multiple cores; same thing). It distributes kernel operations (kernel threads) across those processors, rather than only utilising a single processor. Clustering allows for the distribution of a task (a compile using gcc, running of certain disk I/O tasks, running multiple userland (or I suppose kernel, if the kernel had clustering support) threads) across multiple physical computers on a local network. The best example I have for real-world clustering is rendering (mostly 3D, but you can render anything; I'm referring to 3D in this case). A person doing modelling creates a model scene using 3D objects, applies textures to it, lighting, raytracing aspects, vertex/bones animation, and anything else -- all using their single workstation. Then the person wants to see what it all looks like -- either as a still frame (JPEG/PNG/TIFF), or as a rendered animation (AVI/MPG/MJPEG). Without any form of clustering, the workstation has to do all of the processing/rendering work by its lonesome self. This can take a very, very long time -- modellers aren't going to wait 2 hours for their work to render, only to find they messed up some bones vertexes half way into the animation. With clustering, the workstation has the capability to send the rendering request out onto the network to a series of what're called slaves (other computers set up to handle such requests). The workstation says I want this rendered. I want all of you to do it. Let's say there's 200 machines in the cluster as slaves, and let's say all 200 of those machines are dual-core (so 400 CPUs total). You then have 400 CPUs rendering your animation, versus just 2 on the workstation. The same concept can apply to compiling (gcc saying I want this C file compiled or whatever), or any other distributed computing computational desired. It all depends on if the software you want to support clustering can do it. Different clustering softwares run at different levels; some might act as virtual environments, thus underlying software may not need to know about clustering (e.g. it just works); others might require each program to be fully cluster-aware. Make sense? :-) Not completely yet (I tend to be stubborn, if I carry this too far, tell me in private mail and I will politely drop it). Your use cases show me the differences in size, and *because* of the size, the differences in how you'd use them, and that part I did already know. I'm perfectly well aware of the current differences in size, but what I'm after is what are the real differences, ignoring size, in what they actually accomplish, and how they go about doing it. I'm thinking of the possibility of perhaps finding it it might be possible to find some way to extend the work domain of an smp system to stretch across machine lines, to jump across motherboards. Maybe not to be global (huge latencies scare me away), but what about just going 3 feet, on a very high speed bus, like maybe a private pci bus? Not what is, what could be? And, I have experienced just as many looniees as you have, who ask for some gigantic task, then sit back and want to take the credit and act like a cheerleader, figuring they really got the thing going. Well, I DON'T really want the help, I have in mind a project for me and a friend, with small smallish machine resources, maybe a bunch of small ARM boards. I wouldn't turn down help, but I'm not really proselytizing. Something small, but with a bunch of bandwidth. So, in that case, what really are the differences between smp and clustering, besides the raw current size of the implementation? Are there huge basic differences between the clustering concept and by smp's actual tasks? The difference is in the S in SMP. In SMP all memory (and other resources) is equally available to all processors and threads and processes can migrate around without special regard for location. File descriptors and data are equally available for all threadds of a process independenly of which CPU thye are on.. etc. etc. In clustering, you basically have to have an extra layer of indirectinb between the working processor and teh resources. Some resources are local. Some are not. threads running in a process can only do so when they have access to all the resources of the process. processes MAY be able to migrate, but it is a complex operation that
Re: remote operation or admin
On Wed, Mar 19, 2008 at 02:03:54PM -0400, Chuck Robey wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jeremy Chadwick wrote: On Wed, Mar 19, 2008 at 01:01:45PM -0400, Chuck Robey wrote: What is most important in my considerations are, how might it to possible to stretch our present smp software to be able to extend the management domains to cover multiple computers? Some sort of a bridge here, because there is no software today (that I'm awarae of, and that sure leaves a huge set of holes) that lets you manage the cores as separate computers) so that maybe today I might be able to have an 8 or 10 core system, and maybe tomorrow look at the economic and software possibility of having a 256 core system. I figure that there would need to be some tight reins on latency, and you would want some BIGTIME comm links, I dunno, maybe not be able to use even Gigabit ethernet, maybe needing some sort of scsi bus linkage, something on that scale? Or, is Fiber getting to that range yet? Anyhow, is it even remotely posible for us to be able to strech our present SMP software (even with it's limitation on word size to limit the range to 32 processors) to be able to jump across machines? That would be one hell of a huge thing to consider, now wouldn't it? Ahh, you're talking about parallel computing, clustering, or grid computing. The Linux folks often refer to an implementation called Beowulf: http://en.wikipedia.org/wiki/Beowulf_%28computing%29 I was also able to find these, more specific to the BSDs: http://www.freebsd.org/advocacy/myths.html#clustering http://lists.freebsd.org/pipermail/freebsd-cluster/2006-June/000292.html http://people.freebsd.org/~brooks/papers/bsdcon2003/fbsdcluster/ Well, I am, and I'm not, if you could answer me one quiestion, then I would probably know for sure. What is the difference between our SMP and the general idea of clustering, as typified by Beowulf? I was under the impression I was talking about seeing the possibility of moving the two closer together, but maybe I'm confused in the meanings? The short version is that software written for SMP and for clusters make very different assumptions on what operations are available and the relative costs between them. Software written for one of them will typically either not run at all, or run very inefficiently on the other. Longer version: SMP (Symmetric Multi Processing) refers to a situation where all the CPUs involved are 'equal' and all use the same shared physical memory. In an SMP system it does not really matter which CPU you run a program on, since they are all equal and all have the same access to memory. One important feature of such a system is that when one CPU writes to memory all the others can see that write. A close relative of SMP is NUMA (Non-Uniform Memory Access), with the most popular variant being ccNUMA (cache coherent NUMA). Here all CPUs still share the memory, but different parts of memory can be differently expensive to access depending on which CPU is involved. (For example: CPU 1 might have fast access to memory area A, and slow access to memory area B, while CPU 2 have fast access to B, and slow access to A.) (Many multi-CPU machines in use today are actually ccNUMA even if they are often called SMP. Software written for SMP will typically run unmodified on an ccNUMA system, although perhaps with somewhat suboptimal performance.) In a cluster on the other hand the CPUs do not share physical memory. They cannot automatically see each others memory operations. Any communication between the CPUs must take place over the network (which is much slower than the internal buses inside a computer.) So in an SMP system communication between CPUs is fast, switching which CPU is running a given program is a cheap and simple operation, and much of the work in synchronizing the CPUs is taken care of automatically by the hardware. In a cluster communication between nodes is expensive, transferring a program from one CPU to another is slow and complicated, and one needs to do extra work to keep each CPU aware of what the others are doing. In a clustered system one usually also has to take care of the case that one node crashes or the network connection to it is broken. In typical SMP systems one just ignores the possibility of one CPU crashing. (The exception being some mainframe systems with high redundancy where one can replace almost all components (including CPUs) while the machine is running. There is a reason why these cost lots and lots of money.) A system that is written to work in a clustered environment can fairly easily be moved to run on an SMP machine, but it will do a lot of work that is not necessary under SMP and thus not make very good use of the hardware. Moving from SMP to cluster is more difficult. One can emulate the missing hardware support in software, but this has a very high overhead. Or one
Asynchronous syscalls for massive concurrency
Hello, Reading the recent thread on -current regarding dropping KSE and the evolution of application design with respect to concurrency, prompted me to want to ask what people think regarding the desirability and/or feasability of making FreeBSD support an asynchronous interface to *all* system calls. In particular, I would love to hear from experienced kernel developers whether this is a total pipe dream in terms of practically implementing it semi-efficiently with a reasonable amount of work, in a real-world operating system like FreeBSD. It is something that I have secretly coveting for for a long time (generally, not for FreeBSD specifically). Let me explain my motivation. Some will probably disagree, others will think I am pointing out the obvious, but I still want to summarize it here in the hopes of triggering a fruitful and/or interesting discussion. If this has been discussed, feel free to point me in the right direction. Apologies if this reads like a rant (will se if anyone bothers to read it to begin with ;). State machines vs. concurrency in application design: It is my belief that, in general, from a design perspective there is a desire to model a fundamentally concurrent process in the form of concurrency, rather than a state machine. It is my belief that, ignoring efficiency, concurrency tends to lead to cleaner, more modularized and more easily maintainable code. Concurrency above does not necessarily refer to operating system threads using the pthread model. I will mention implementation issues later on. Right now, I am arguing strictly from the design perspective of how I want to structure my code. In the simplest cases, a state machine will be pretty small and easy to understand (let's say the state machine parsing simple line-based messages in an IRC server). As the problem grows more complex, you will invariable wish to modularize the code; introduce abstractions; moving details away from the core loop. This can be done to any extent; you might generalize it a bit and be fairly satisfied. Until you want to introduce even more features and you feel the need to further generalize. You end up reaching a point where the framework for writing a modularized state machine is approaching the featurefullness of actual concurrency. If your language supports continuations, you may very well simply implement the state machine in terms of these. In this case, you pretty much have userland co-operative threading. That is one end of the extreme; the other end is the simple core state machine loop without modularizations. In between, you have various levels of generality in explicit state management. In this way, I view true concurrency modelling as essentially being the perfect state machine, completely hidden from application code. From this perspective, manually writing explicit state management is essentially a less powerful and less expressive way of achieving the same goal, that I would only resort to if true concurrency is not possible or practical. I will readily admit that I have not written a lot of state machine style code, exactly because of my view expressed above. I have however written, and write on a daily basis, a pretty considerable amount of multi-threaded (or otherwise concurrent) code. Some of it could reasonable be re-done in the form of a state machine, while in other cases the complexity would be absolutely tremendous without using some form of state management framework that at least approaches true concurrency (and even then complexity would be considerably higher than now). This is my reason for fundamentally preferring a concurrent approach, and would love for it to be supported by the language/environment being used, to an extent such that there is no need to fall back to explicit state management. Enabling massive concurrency by asynchronous syscalls: Support for asynchronous syscalls, I believe, would help *a lot* in realizing practical massive concurrency in a variety of environments, from C/C++ to more higher level languages. C/C++ and similar w/ pthread concurrency model: One can already come *fairly* far by writing code in a stack conscious manner. Most of the C/C++ I do is written with this in mind, and having tens of thousands of threads or more is not a problem at all, because the stack size can be very very conservative. This is not to say that it is necessarily the most efficient approach (context switching overhead, cache localitly, etc), but it does get you pretty far - particularly when the application is extremely concurrent in relation to the actual amount of work done in total (that is, when your problem is not CPU efficiency, but the ability to e.g. handle 30 000 clients, most of which are mostly idle). That said, there are still issues. Regardless of whether explicit memory management is used, or garbage collection, there is a desire to reduce contention. Many mallocs have
Re: Building in /usr/src copied to $HOME
On Wednesday 19 March 2008 05:36:47 Giorgos Keramidas wrote:
On 2008-03-18 21:28, V??clav Haisman [EMAIL PROTECTED] wrote:
Hi,
I am trying to use /usr/src copied to my $HOME but the build process
doesn't want to work. For example when I try build /bin/cp I get the
following:
shell::wilx:~/freebsd/src/bin/cp make
/usr/home/users/wilx/freebsd/src/bin/cp/../Makefile.inc, line 9:
Malformed conditional (${MK_DYNAMICROOT} == no)
/usr/share/mk/bsd.init.mk, line 15: if-less endif
make: fatal errors encountered -- cannot continue
Looks like a problem with /usr/share/mk being out of date (not sure this
should be an issue, though). A simple fix should be to add -m
`pwd`/share/mk to make ... or -m `pwd`/../../share/mk in your example
above. The buildworld target seems to pick this automatically.
Is there any sort of tutorial/prescription anywhere how to use
/usr/src tree that does not reside in /usr/src?
Hmmm, that should work. I regularly build as a non-root user, at
`/home/build/src'.
The error about ``Malformed conditional'' seems a bit odd too. Are you
using /usr/bin/make? What version of FreeBSD is the build host
running, and what version of the source tree have you checked out?
--
/\ Best regards, | [EMAIL PROTECTED]
\ / Max Laier | ICQ #67774661
X http://pf4freebsd.love2party.net/ | [EMAIL PROTECTED]
/ \ ASCII Ribbon Campaign | Against HTML Mail and News
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Summer of Code 2008 Project Ideas
Robert Watson wrote: On Mon, 17 Mar 2008, Murray Stokely wrote: The FreeBSD Project was again accepted as a mentoring organization for the Google Summer of Code. The student application period will begin next week so if you have any ideas for great student projects, please send them to [EMAIL PROTECTED] or post them here for discussion. A good student project has a well defined purpose, some key FreeBSD developers that could be identified as potential mentors, and is feasible for a student to complete in a few months time. The existing ideas list is available here : http://www.freebsd.org/projects/ideas/ If you can suggest something (getting specific parts of valgrind working on FreeBSD?) then please provide information in the form of the other projects listed on the page as far as difficulty level, requirements, etc.. FYI, to students considering doing projects -- in the last day or two, we've made significant updates to the project ideas list. If you looked a few days ago, please look again. In particular, we've flagged a large number of potential SoC projects that were not there previously. We've also filtered out some that looked too big to be a good 3-month summer project, although you can still find many on the full ideas list. If you're going to work on a proposal for one of these projects, please directly contact the contacts listed for the project to get feedback before submitting your proposal. We will continue to update the project ideas page as new ideas come in, so do keep checking back. Also, keep in mind that you can submit more than one application if there are multiple projects you find interesting (or we add some later). You can also come up with your own project ideas if we have missed one :) Kris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Asynchronous syscalls for massive concurrency
Peter Schuller wrote: Hello, Reading the recent thread on -current regarding dropping KSE and the evolution of application design with respect to concurrency, prompted me to want to ask what people think regarding the desirability and/or feasability of making FreeBSD support an asynchronous interface to *all* system calls. In particular, I would love to hear from experienced kernel developers whether this is a total pipe dream in terms of practically implementing it semi-efficiently with a reasonable amount of work, in a real-world operating system like FreeBSD. It is something that I have secretly coveting for for a long time (generally, not for FreeBSD specifically). Let me explain my motivation. Some will probably disagree, others will think I am pointing out the obvious, but I still want to summarize it here in the hopes of triggering a fruitful and/or interesting discussion. If this has been discussed, feel free to point me in the right direction. Apologies if this reads like a rant (will se if anyone bothers to read it to begin with ;). State machines vs. concurrency in application design: It is my belief that, in general, from a design perspective there is a desire to model a fundamentally concurrent process in the form of concurrency, rather than a state machine. It is my belief that, ignoring efficiency, concurrency tends to lead to cleaner, more modularized and more easily maintainable code. Concurrency above does not necessarily refer to operating system threads using the pthread model. I will mention implementation issues later on. Right now, I am arguing strictly from the design perspective of how I want to structure my code. In the simplest cases, a state machine will be pretty small and easy to understand (let's say the state machine parsing simple line-based messages in an IRC server). As the problem grows more complex, you will invariable wish to modularize the code; introduce abstractions; moving details away from the core loop. This can be done to any extent; you might generalize it a bit and be fairly satisfied. Until you want to introduce even more features and you feel the need to further generalize. You end up reaching a point where the framework for writing a modularized state machine is approaching the featurefullness of actual concurrency. If your language supports continuations, you may very well simply implement the state machine in terms of these. In this case, you pretty much have userland co-operative threading. That is one end of the extreme; the other end is the simple core state machine loop without modularizations. In between, you have various levels of generality in explicit state management. In this way, I view true concurrency modelling as essentially being the perfect state machine, completely hidden from application code. From this perspective, manually writing explicit state management is essentially a less powerful and less expressive way of achieving the same goal, that I would only resort to if true concurrency is not possible or practical. I will readily admit that I have not written a lot of state machine style code, exactly because of my view expressed above. I have however written, and write on a daily basis, a pretty considerable amount of multi-threaded (or otherwise concurrent) code. Some of it could reasonable be re-done in the form of a state machine, while in other cases the complexity would be absolutely tremendous without using some form of state management framework that at least approaches true concurrency (and even then complexity would be considerably higher than now). This is my reason for fundamentally preferring a concurrent approach, and would love for it to be supported by the language/environment being used, to an extent such that there is no need to fall back to explicit state management. Enabling massive concurrency by asynchronous syscalls: Support for asynchronous syscalls, I believe, would help *a lot* in realizing practical massive concurrency in a variety of environments, from C/C++ to more higher level languages. C/C++ and similar w/ pthread concurrency model: One can already come *fairly* far by writing code in a stack conscious manner. Most of the C/C++ I do is written with this in mind, and having tens of thousands of threads or more is not a problem at all, because the stack size can be very very conservative. This is not to say that it is necessarily the most efficient approach (context switching overhead, cache localitly, etc), but it does get you pretty far - particularly when the application is extremely concurrent in relation to the actual amount of work done in total (that is, when your problem is not CPU efficiency, but the ability to e.g. handle 30 000 clients, most of which are mostly idle). That said, there are still issues. Regardless of whether explicit memory management is used, or garbage collection, there is a desire to reduce contention.
Re: remote operation or admin
On Wed, 19 Mar 2008 15:10:12 -0400 Chuck Robey [EMAIL PROTECTED] wrote: Not completely yet (I tend to be stubborn, if I carry this too far, tell me in private mail and I will politely drop it). Your use cases show me the differences in size, and *because* of the size, the differences in how you'd use them, and that part I did already know. I'm perfectly well aware of the current differences in size, but what I'm after is what are the real differences, ignoring size, in what they actually accomplish, and how they go about doing it. I'm thinking of the possibility of perhaps finding it it might be possible to find some way to extend the work domain of an smp system to stretch across machine lines, to jump across motherboards. Maybe not to be global (huge latencies scare me away), but what about just going 3 feet, on a very high speed bus, like maybe a private pci bus? Not what is, what could be? What you're describing is a classic multi-cpu system. From the software point of view, it's just a another variant on the whole multiprocessor thing. I believe most modern multi-cpu systems share all the resources, and are all SMP. You could build one with memory that wasn't equally accessible to all CPUs in the system, but in general you don't want to do that unless the CPUs in question are doing different things, such as a GPU, an I/O processor of some kind, etc. So the answer to your two central questions what are the real differences between what they actually accomplish and how they go about doing it - at least for tightly coupled, everything shared multi-cpu boxes - is there aren't any. There's been lots of work done in these areas, dating back to the 60s. Multics, Tandem and V are the ones that come to my mind, but there are others. There are schools of concurrent software design that hold that this should be true across the entire scale: that you design your system to work the same way on a single box with a quad-core CPU as you would on single-cpu boxes one each in Sao Paulo, Anchorage, Melbourne and Trondheim, as the differences are implementation details. In real life, those difference are huge. In particular, if you're staying inside the same room, the critical one is going from all memory shared to some memory shared, as the former means you can have a single memory location representing an external resource, and once you lock it you're set. With the latter, you can't do that - you have to bounce messages back and forth between the system, and get them all agree to let you have it before you work on it. This can easily push timing things out to as bad as the next level of sharing. For instance, I build ETL systems that shove thousands of multi-megabyte files/hour between boxes. It's actually faster to transfer the data over the network and let each box use an unshared file system for them than to rename the files on a shared disk from one directory to another, because doing the file rename requires locking both directories on every system that has access to them - which in this case means dozens of them. mike -- Mike Meyer [EMAIL PROTECTED] http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: remote operation or admin
On Wed, 19 Mar 2008 21:33:11 +0100 Erik Trulsson [EMAIL PROTECTED] wrote: A system that is written to work in a clustered environment can fairly easily be moved to run on an SMP machine, but it will do a lot of work that is not necessary under SMP and thus not make very good use of the hardware. Moving from SMP to cluster is more difficult. One can emulate the missing hardware support in software, but this has a very high overhead. Or one can rewrite the software completely, which is a lot of work. One way to think of such is that cluster software will typically consist of lots of processes talking to each other over sockets whereas SMP software will typically consist of one process with threads talking to each other through shared memory locations. On an SMP system the cluster software could be rewritten to use shared memory and hence improve performance. Likewise, moving the SMP program into a cluster environment means you have to replace shared objects with proxies that copy things around and coordinate actions. At least one concurrent software development system (Bertrand Meyer's SCOOP) is set up so the developer doesn't worry about those things; the location and communications channels of the objects in the program is specified at launch time. mike -- Mike Meyer [EMAIL PROTECTED] http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Asynchronous syscalls for massive concurrency
KSE's asynchronous system calls could be used to do what you suggest however it turns out that if you are going to do this you probably want to do it in a way where the program is aware of what is going on, so the ability to make all syscalls TRANSPARENTLY ASYNCHRONOUS is not really a requirement. I did not mean to imply it had to be transparent; in fact that was not my goal. But I see it as a problem that, right now, if a process wishes to make n concurrent system calls (concurrent in the sense of the time period between request to response, not actually concurrent in terms of execution in the kernel or in user space), it *must* have n threads (or n co-operating processes). In other words, there is an enforcement that logical concurrency be matched with actual machine/kernel concurrency. This makes userland threading non-transparent (from the perspective of application code written in a language or framework that attempts to provide light-weight concurrency) because you can suddenly not count on all operations playing well with it. Basically, disregarding tricks that do things like offloading certain operations to separate processes (which still does not remove the n process/n thread requirement), any light-weight concurrency implementation is not generally applicable. That is, you are forced to trade trade light weight against a restricted feature set. The async syscall part of KSE could be kept/reintroduced pretty seperatly from the thread support part, but it really requires someone to decide how to use it. The intended use would be to augment exiting code to take advantage of it. If it caught on among operating systems, it would be a big step forward in my opinion, from the perspective of application development. If a single or a few operating systems did it, they would have an advantage to the extent that someone actually implements the necessary support in $language_or_framework. In particular various higher level languages where it would make a huge difference to be able to support this in the interpreter/vm/core library. It would remove, or lessen, the need to differentiate between not quite threads and threads (witness stackless python tasks or ruby fibers, vs. the actual native threads in Python and Ruby 1.9; once again trading light-weight vs. generality). As far as I can see, it can be applied to the C runtime library as well, but the advantage would be less since you won't get around the need to maintain a per-thread native stack. This latter point is why I am mostly interested in it from the higher level language perspective, where higher level primitives (e.g., continuations, interpreter level stacks on the heap, etc) can be used for thread context instead of the native stack, thus removing the need to take an n kbyte hit per thread just for the stack alone. It is not even mutually exclusive to native code compilation. Oh and btw, regarding my comment on paging: While as far as I can tell it holds relative to native threading, it is still not worse (better in fact) than the single-threaded state machine model where a high-latency paging operation blocks the entire process. -- / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller [EMAIL PROTECTED]' Key retrieval: Send an E-Mail to [EMAIL PROTECTED] E-Mail: [EMAIL PROTECTED] Web: http://www.scode.org pgpssgMFqSbEJ.pgp Description: PGP signature
Re: Summer of Code 2008 Project Ideas
On Wed, Mar 19, 2008 at 10:17:21PM +0100, Kris Kennaway wrote: You can also come up with your own project ideas if we have missed one :) I've got one: Implement wake on lan support for every ethernet device driver in the tree. ifconfig in -CURRENT already supports configuring WOL, but there's not a single driver yet that makes use of this. There should be enough information to get interested people going at http://wiki.freebsd.org/WakeOnLan Apart from if_vr patches by Yongari (linked from the wiki page) and some other patches that sit in a few peoples' mailboxes, nothing has been happening around this in a while. I myself am busy working on Subversion atm and will start working on my Bachelor theses starting next semester (April-Juli), so I'm out :/ I'd be available to answer questions if necessary, however. -- stefan http://stsp.name PGP Key: 0xF59D25F0 pgpv2O4mHcYXj.pgp Description: PGP signature
Re: vkernel GSoC, some questions
:Matthew Dillon wrote: : :Matt, :... : :Don't you use something like VMWare for development and debugging? : : We use vkernel's for development and debugging. Pretty much everything : except hardware device driver development can be done using a vkernel... : :Does that include trying to get rid of the BGL, for example? Yah, the SMP stuff works the same in a vkernel as it does in a real kernel. If anything, the vkernel is even more sensitive to conflicts since vkernel 'cpus' are threads on the real system and can wind up being scheduled as time-share (for example, booting a 4-cpu vkernel on a single-cpu platform). -Matt Matthew Dillon [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Summer of Code 2008 Project Ideas
Stefan Sperling [EMAIL PROTECTED] writes: On Wed, Mar 19, 2008 at 10:17:21PM +0100, Kris Kennaway wrote: You can also come up with your own project ideas if we have missed one :) I've got one: Implement wake on lan support for every ethernet device driver in the tree. I like that idea. I miss being able to wake a few of my machines the way I could a couple of releases ago. -- Christian Laursen ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: remote operation or admin
On Wed, 19 Mar 2008 12:31:24 -0700 Julian Elischer [EMAIL PROTECTED] wrote: Chuck Robey wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jeremy Chadwick wrote: On Wed, Mar 19, 2008 at 02:03:54PM -0400, Chuck Robey wrote: Well, I am, and I'm not, if you could answer me one quiestion, then I would probably know for sure. What is the difference between our SMP and the general idea of clustering, as typified by Beowulf? I was under the impression I was talking about seeing the possibility of moving the two closer together, but maybe I'm confused in the meanings? SMP as an implementation is mainly intended for single systems with multiple processors (e.g. multiple physical CPUs, or multiple cores; same thing). It distributes kernel operations (kernel threads) across those processors, rather than only utilising a single processor. Clustering allows for the distribution of a task (a compile using gcc, running of certain disk I/O tasks, running multiple userland (or I suppose kernel, if the kernel had clustering support) threads) across multiple physical computers on a local network. The best example I have for real-world clustering is rendering (mostly 3D, but you can render anything; I'm referring to 3D in this case). A person doing modelling creates a model scene using 3D objects, applies textures to it, lighting, raytracing aspects, vertex/bones animation, and anything else -- all using their single workstation. Then the person wants to see what it all looks like -- either as a still frame (JPEG/PNG/TIFF), or as a rendered animation (AVI/MPG/MJPEG). Without any form of clustering, the workstation has to do all of the processing/rendering work by its lonesome self. This can take a very, very long time -- modellers aren't going to wait 2 hours for their work to render, only to find they messed up some bones vertexes half way into the animation. With clustering, the workstation has the capability to send the rendering request out onto the network to a series of what're called slaves (other computers set up to handle such requests). The workstation says I want this rendered. I want all of you to do it. Let's say there's 200 machines in the cluster as slaves, and let's say all 200 of those machines are dual-core (so 400 CPUs total). You then have 400 CPUs rendering your animation, versus just 2 on the workstation. The same concept can apply to compiling (gcc saying I want this C file compiled or whatever), or any other distributed computing computational desired. It all depends on if the software you want to support clustering can do it. Different clustering softwares run at different levels; some might act as virtual environments, thus underlying software may not need to know about clustering (e.g. it just works); others might require each program to be fully cluster-aware. Make sense? :-) Not completely yet (I tend to be stubborn, if I carry this too far, tell me in private mail and I will politely drop it). Your use cases show me the differences in size, and *because* of the size, the differences in how you'd use them, and that part I did already know. I'm perfectly well aware of the current differences in size, but what I'm after is what are the real differences, ignoring size, in what they actually accomplish, and how they go about doing it. I'm thinking of the possibility of perhaps finding it it might be possible to find some way to extend the work domain of an smp system to stretch across machine lines, to jump across motherboards. Maybe not to be global (huge latencies scare me away), but what about just going 3 feet, on a very high speed bus, like maybe a private pci bus? Not what is, what could be? And, I have experienced just as many looniees as you have, who ask for some gigantic task, then sit back and want to take the credit and act like a cheerleader, figuring they really got the thing going. Well, I DON'T really want the help, I have in mind a project for me and a friend, with small smallish machine resources, maybe a bunch of small ARM boards. I wouldn't turn down help, but I'm not really proselytizing. Something small, but with a bunch of bandwidth. So, in that case, what really are the differences between smp and clustering, besides the raw current size of the implementation? Are there huge basic differences between the clustering concept and by smp's actual tasks? The difference is in the S in SMP. I think you're grabbing the wrong difference. Loosing the symmetry in an SMP system leaves you with an asymmetric MP system, not a cluster. You can do asymmetric MP on a single box. Most early Unix MP implementations were such, as the easiest thing to implement. In SMP all memory (and other resources) is equally available to all processors and threads and processes can migrate around without special
