Re: [leaf-devel] 2.6.x kernel support?
Luis.F.Correia wrote: Hi Natanael! -Original Message- From: Natanael Copa [mailto:[EMAIL PROTECTED] Sent: Thursday, August 25, 2005 3:08 PM To: [EMAIL PROTECTED] Cc: Mike Noyes; [EMAIL PROTECTED]; leaf-devel@lists.sourceforge.net Subject: Re: [leaf-devel] 2.6.x kernel support? [EMAIL PROTECTED] wrote: Evolution is also (slowly) making things better by developers filling up gaps (like webconf), provide missing packages, making proof of concepts for new technology or creating a new package format like Nathaneal did. But in the end it would be nice to bring it all together instead of abbandon branches. To start it's 'Bering' not 'Bearing' :))) lol ... *blushes* :) whatever... Personally I think the bearing *concept* is very interesting. Load runtime modules (lrp's) and run from RAM. After the enlightening discussion about tmpfs I think it have become even more interesting. Some people that I work with is interesting in pushing this concept futher. What if we run squid from RAM (actually we already do this)? what if we run a mailserver from RAM? what if run LDAP, SQL etc etc, things you might want to do in bigger environments. There is an RFC that strictly forbids the use of mail queues that run in a ramdisk, search the ML archives for the proper document. Yes. I have been through that discussion (we are currently using mail-proxies from RAM with no spooling om RAM disk), but the idea with running mail/sql/ldap server from RAM is this: run the services from RAM. Run the database/spools on any kind of backen storage harddisk/raid/lvm/nfs/ifs/whatever. the server goes down. (cpu burns up, the power burns up, the building burns up (backend storage could be ouside building)) Now, how long time does it take to get that server up and run again? Well... grab the nearest available server hardware, burn a new cd image (or floppy or usb stick or whatever) and get a copy of the latest configuration of that server. (Since the server runs from RAM you need some kind of backup that survives reboots anyway) Boot it up and connect to the backend storage and you are up and running again. Since we also run from RAM there are no moving parts that increases risk for hardware failure. Since the executables are loaded into RAM during boot, instead of running from a loopdevice mouned on cdrom, there are never any delays caused by the cdrom spinning up. The run-from-RAM concept is interesting. Suddenly we are in the possition where we look for using this run-from-RAM concept for solving other problems, targeting other goals than LEAF/Bering is supposed to. No objections there, of course! Now, I agree completely that forking and abbandon branches is generally a bad thing, but sometimes the goals changes and sometimes it changes fast. I dont expect that the LEAF/Bearing team should change their goals just because the needs in my environment changes. That was not my intention at all, i guess a lot of things I wrote were misinterpreted. Maybe. Anyway.. I believe most of us agree on things. We just see things from different angles. So instead of pushing my needs/goals to the current projects, I start looking for other distros and projects. If there is no-one suitable, I create something new or fork and while doing so, I do everything that is in my power to make sure that others have the possibility to benefit from my stuff. You can join LEAF with your GNAP based distro, as it is not that far apart from what we have now. And maybe we in a not So distant future can benefit from the things you have discovered and fixed in the meantime. Thanks. Things are really floating here right now and I'm not sure if joing LEAF is the way to go. And I can asure you that you already have been benefited from things related to this project, even if it not is directly from me. btw its based on Gentoo not GNAP. Its more a sibling to GNAP than a child of it. -- Natanael Copa --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
On Mon, 2005-08-29 at 09:08, Mike Noyes wrote: On Tue, 2005-08-23 at 13:30, Natanael Copa wrote: Mike Noyes wrote: Would you like to join us, and make Alpine a LEAF branch? Thank you very much for the invitation. I have to think about it (talk with some other ppl too), but that is not impossible. Please let me know what you decide on. You should know that Alpine will sacrifice size for faster development. I am working on 3 branches of Alpine currently. (all uses uclibc but some uses newer version of gcc and some does not include SSP,PIE etc.) When I get time I will probably add a 4th too. Every branch has a build environement with all packages in tbz2 format (gentoo binary), then there is a apk repository with all apk's (you can load runtime apk's from http) and then there is all iso images that includes all the apks. So every package is stored atleast 3 times. I would prefer saving a couple of generations backwards for history. What are the requirements for your source code trees, and is cvs acceptable for version control system? I have been able to temporarly borrow 500MB space from lmdata.net but I have found out that it is only enough for uploading a demo. I guess every branch could easily use 1GB. I'm inquiring about this requirement with the SF staff. I'm not hopeful that they'll allocate the resources you need to leaf. :-( Natanael, I received a non committal positive response from the SF staff concerning my query. :-) It looks like we may be able to accommodate Alpine as a LEAF branch. :-) Are you sure that you still want Alpine under the leaf umbrella? I do, but I'm only one voice. I'd like our other project admins to express their opinions too. -- Mike Noyes mhnoyes at users.sourceforge.net http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
Hello Charles, I'm all for modular. Not being able to (easily) backup the initramfs image is (in my opinion) not a serious problem, assuming the boot process changes as well. I envision the initramfs as containing only that code required to build a root filesystem image, or roughly incorperating the functionality of the existing /linuxrc script. Like Martin I'm also a bit puzzled here. Are you saying that it should be possible to remove the modules needed for mounting the boot device from the initramfs and put them elsewhere? To make sense, these features would have to be coded in something very small (ie: no big C library required) so the extra space could be justified (I'd give up 10-50K to have a powerful scripting language like forth or LUA avaialble at runtime!). Me too in that case, because the extra space this costs can be gained back by removing the tools needed to backup such an initramfs. Eric --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
RE: [leaf-devel] 2.6.x kernel support?
Hi Mike, -Original Message- From: Mike Noyes [mailto:[EMAIL PROTECTED] On Wed, 2005-08-24 at 11:07, [EMAIL PROTECTED] wrote: Can you tell us what your intentions are? By reading your comments it seems like you are trying to force the creation of new branches. Eric, Nothing different than I've always espoused. Nor am I trying to force anything. However, I still believe what I said in my Evolution as a project development model post. I don't think anyone here is wishing to move to a monolithic development model. Why are you saying that? I interpreted Luis's comment as closing off discussion for leaf developers. Bering uClibc is the dominant leaf branch, and I thought the community might wish to coalesce around it. As I say in another email, that was not my intention at all. The discussion should be closed only as far as our uClibc branch was concerned. We hae already tried and tested the alternatives. Maybe I read to much into Luis's comment. It wouldn't be the first time I misinterpreted something. :-( Note: I'll never be what I was prior to my accident, and I don't contribute much. I don't wish to be a point of discord in the community. It may just be time for me to step aside. Luis, If I misinterpreted your comments, I apologize. Again, no need to apologise, all our work and all our comments are valid. We all do what we can, when we can, and we all have our limits, after all that's what define us Humans! Keep up your good work! Mike Noyes mhnoyes at users.sourceforge.net http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs Luis Correia Bering uClibc Team Member PGP Fingerprint: BC44 D7DA 5A17 F92A CA21 9ABE DFF0 3540 2322 21F6 Key Server: http://pgp.mit.edu --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
Hi Mike, Mike Noyes wrote: On Wed, 2005-08-24 at 15:51, Martin Hejl wrote: In the end, I tend to agree with Luis. I'm not going to tell anybody to stop discussing new possibilities - but discussion without the ability/willingness to actually do more than just discuss things is pretty much what I contribute to management these days (I spent a fair share of my work time in meetings with people who discuss things they neither know how to do, nor would they be willing to provide the manpower to get it done - they just like discussing things). Maybe that's why I'm very skeptical about the value of discussions all by themselves. Discussing how to solve a problem at hand is perfectly fine, and usually also very useful - but discussing things despite the fact that every participant of the discussion has a different idea of what the actual issue one might be discussing is something else... Martin, That excludes me from any comments regarding leaf development, and relegates me to irrelevance. I'm not a programmer, or knowledgeable compared to most of you. :-( that surely wasn't the point I was trying to make - and I'm pretty sure you know that. We all appreciate your work and input - leaf is a group effort, not something that's done by a couple of coders. People don't have to be programmers to be able to express their ideas on leaf-devel, or pass some sort of a test or anything like that, and that's a good thing. I was merely trying to point out that discussions like that often develop a life of their own and become disconnected from what's actually happening on the development front. Again, maybe I just see too much of that kind of thing in my day job, and are therefore unable to see the honest attempt to find the best technological solution for leaf. If that's the case, I sincerely apologize for my cynicism/sarcasm. No need for you to apologize. You've contributed lots of time, effort, and code to LEAF. True, but people who write code can misunderstand something too, or read too much into something. Martin --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
Hello Erich, LEAF runs in RAM, so it's not only a matter of 16 Mb storage but also the availability of internal RAM. This shouldn't be a problem in most cases, but if LEAF is ported to other architectures with limited RAM this could become a problem. Indeed, and I am surprised noone ever brought this up. Currently LEAF (and probably other RAM based systems) is wasting RAM for storage of binaries , libraries and scripts, some of which are loaded only once. If we want to keep the RAM based architecture of the LEAF systems and reduce the RAM footprint I believe we need to restructure our packages so they can be held in a small number of loop mounted cramfs images. I believe we could reduce the filesystem footprint significantly. LINCE does this already AFAIK. Only the configuable files need to be held in a R/W filesystem. I don't follow you here. Why do you wan't to use loop mounted cramfs? Does it use RAM or ROM (Flash/HD/Floppy/..). The advantage of full RAM based systems is that you can unmount the storage media. Besides the footprint of Bering-uClibc with base packages is only ~8Mb. Eric --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
Eric [EMAIL PROTECTED] wrote: I don't follow you here. Why do you wan't to use loop mounted cramfs? Does it use RAM or ROM (Flash/HD/Floppy/..). The advantage of full RAM based systems is that you can unmount the storage media. Besides the footprint of Bering-uClibc with base packages is only ~8Mb. Yes, but look at it when it holds ipsec, ssh, samba, squid... A loop mounted cramfs looks (for read_only operations) exactly like a part of the file system tree. The benefit of this is that, even on ram, it cannot be trivially modified and it takes a lot less RAM space because it compresses its contents. For example if we had all the /bin in a cramfs called bin.cfs which would sit at / we could mount mount -o loop /bin.cfs /bin and the space needed by bin.cfs would be a lot smaller than if the individual files in /bin would be installed normally. The same is true for all read_only components, like /lib /sbin /usr/bin cheers Erich --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
Erich Titl wrote: Eric [EMAIL PROTECTED] wrote: I don't follow you here. Why do you wan't to use loop mounted cramfs? Does it use RAM or ROM (Flash/HD/Floppy/..). The advantage of full RAM based systems is that you can unmount the storage media. Besides the footprint of Bering-uClibc with base packages is only ~8Mb. Yes, but look at it when it holds ipsec, ssh, samba, squid... A loop mounted cramfs looks (for read_only operations) exactly like a part of the file system tree. The benefit of this is that, even on ram, it cannot be trivially modified and it takes a lot less RAM space because it compresses its contents. Are you really sure about that? I am not really familiar with how the tmpfs or ramfs works, but to me sounds logical that when you execute something on a tmpfs dist, the memory is never copied from ramdisk to RAM, because it is already in ram. It should be possible to run it from 1 single copy in RAM. But if you run it from a mounted loopdevice in ram, the code needs to be uncompressed and then executed. That means you will need a copy of the compressed code and one copy of the uncompressed. So if the compression ratio is 50% a compressed cramfs image on a tmpfs would take 50% more ram to execute the same code than if you just runs it natively from tmpfs. Again, I don't know how cramfs or tmpfs works so I could be completely wrong about this but I think is sounds logical. Also, the nature of Bering uClibc is that you only need to unpack the needed packages to RAM, so you are probably using the packages that are extracted to RAM. On a normal disk based system, the linux kernel would cache the used executable data from disk to RAM so the memory will be used anyway - but the kernel has the option to free it if needed for other things. When running from RAM you will only prevent the kernel from releasing the memory to use on other things wich guarantees you an ultrafast system. Adding a swap disk is also an interesting topic... You will save RAM when mounting a cramfs image on cdrom or mount a disk, but the difference is maybe not so big as you might think. -- Natanael Copa --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
Natanael Copa wrote: Erich Titl wrote: Eric .. Are you really sure about that? I am not really familiar with how the tmpfs or ramfs works, but to me sounds logical that when you execute something on a tmpfs dist, the memory is never copied from ramdisk to RAM, because it is already in ram. It should be possible to run it from 1 single copy in RAM. If this is true, then the entire idea is wrong. I doubt though, because the program loader does more than just make a copy of the executable. It needs to dynamically link to libraries, e.t.c. I am not sure it meddles with the intricacies of a file system. But if you run it from a mounted loopdevice in ram, the code needs to be uncompressed and then executed. That means you will need a copy of the compressed code and one copy of the uncompressed. True, but see above, maybe someone can shed a light on this. One thing is sure, if you compress the binary it needs to be uncompressed before execution. ... You will save RAM when mounting a cramfs image on cdrom or mount a disk, but the difference is maybe not so big as you might think. This may defeat the read-only methods implemented by disabling ide drivers. cheers Erich --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
Eric [EMAIL PROTECTED] wrote: ... How would such a thing be implemented for all binaries? Cramfs is ro so you can't populate a loop mounted cramfs. This would mean that you either put all /bin, /sbin, ... binaries in seperate cfs files and don't have packages anymore, put a bin.cfs in every package containing the binaries and make tons of mount points or create a cramfs within your running system from the loaded packages and still need the initial amount of RAM ... IMHO LEAF should have some kind of a firmware which holds most libraries and binaries of a certain release. Maybe we need firmware sets to satisfy Joe Average. Long time ago I almost went crazy before I discovered that the ipsec package did _not_ contain ipsec.o. So there is no consistent package scheme anyway. Besides, you can also lzma your programs and have the same space savings. What about libraries? How do you see a way to create f.e. bin.cfs and still be able to install packages? Packages should consist mostly of configuration data. Backing up, for example, root.lrp is most of the time a pain in the butt. The same applies to initrd, ipsec, ssh and others. They are just big. Few people need to change the full content of a .lrp file. Most often we just configure /etc/network/interfaces and a small number of files in /etc/shorewall. Erich --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
Natanael Copa wrote: Erich Titl wrote: .. I dont have time for reading about memory management in Linux right now, but AFAIK the executables and libraries are mmap'ed. This would mean, that an executable would onle be mapped to memory, but copied as soon as the memory page it resides on is written to by anyone. This could produce some interrupts. ... I don't know if I should/can consider a RAMdisk simply RAM. It needs to behave like a disk in the sense of logical I/O. ... It would really not make any sense to copy a mmap'ed executable from RAM to another place in RAM, That would be true if RAMdisk does just mapping. but as I said, I'm not 100% sure about that and currently I dont have time to investigate it (so, strictly I should have kept my muth shut, but now its to late anyway...:) Same for me, still it is an interesting object. Maybe someone with deeper insight into RAMdisks on Linux could tell. cheers Erich --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Erich Titl wrote: | Natanael Copa wrote: | Erich Titl wrote: | | .. | | I dont have time for reading about memory management in Linux right now, | but AFAIK the executables and libraries are mmap'ed. | | This would mean, that an executable would onle be mapped to memory, but | copied as soon as the memory page it resides on is written to by anyone. | This could produce some interrupts. | ... | I don't know if I should/can consider a RAMdisk simply RAM. It needs to | behave like a disk in the sense of logical I/O. | | ... | | It would really not make any sense to copy a mmap'ed executable from RAM | to another place in RAM, | | That would be true if RAMdisk does just mapping. | | but as I said, I'm not 100% sure about that and | currently I dont have time to investigate it (so, strictly I should | have kept my muth shut, but now its to late anyway...:) | | Same for me, still it is an interesting object. Maybe someone with | deeper insight into RAMdisks on Linux could tell. The data in a tmpfs filesystem resides entirely in the page cache (and/or the swap file, if it exists). This is the same status code would have if it was loaded from (for example) an ext2 formatted partition on a HDD or some other 'conventional' filesystem (ie: cramfs, iso9660, FAT, etc.). Upon loading, each instance of an application will get it's own private memory area for stacks and variables, but the executable code is only loaded into memory once. The MMU is used to make the memory visible to more than one process at once (ie: you can have 50 apache process running at once, but they're all executing out of the same chunk of physical memory containing the code, although each process will also have it's own private memory area for state information). So...if you have a program in tmpfs, it can be run in-place, as the files are already in the page-cache. The special handling of tmpfs is due to the fact that there is *NO* filesystem as such on the tmpfs device...instead the virtual memory manager itself takes care of the filesystem details (hence, no formatting is required!). With pretty much any other filesystem, this is *NOT* the case, and you'll have two copies of any data (the copy on the filesystem and the copy in the page cache). This is because tmpfs is the only filesystem that's handled directly by the virtual memory manager, without using a filesystem driver. - -- Charles Steinkuehler [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.0 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFDDJXLLywbqEHdNFwRAsWEAJ926QQ2T6whnJYexsfZWcaZhD591gCgnjMC 8Ri6P6yOHL1Kx2gzk6VUWFg= =GEyf -END PGP SIGNATURE- --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
RE: [leaf-devel] 2.6.x kernel support?
Hi! -Original Message- From: Erich Titl [mailto:[EMAIL PROTECTED] Sent: Wednesday, August 24, 2005 4:25 PM To: Natanael Copa Cc: leaf-devel@lists.sourceforge.net Subject: Re: [leaf-devel] 2.6.x kernel support? Natanael Copa wrote: Erich Titl wrote: .. I dont have time for reading about memory management in Linux right now, but AFAIK the executables and libraries are mmap'ed. This would mean, that an executable would onle be mapped to memory, but copied as soon as the memory page it resides on is written to by anyone. This could produce some interrupts. ... I don't know if I should/can consider a RAMdisk simply RAM. It needs to behave like a disk in the sense of logical I/O. ... It would really not make any sense to copy a mmap'ed executable from RAM to another place in RAM, That would be true if RAMdisk does just mapping. but as I said, I'm not 100% sure about that and currently I dont have time to investigate it (so, strictly I should have kept my muth shut, but now its to late anyway...:) Same for me, still it is an interesting object. Maybe someone with deeper insight into RAMdisks on Linux could tell. Is this whole conversation about loading to ram, using initramfs or any other kind relevant to the current branches of LEAF? The way I see it, the team i'm part of has already analysed the implications of switching over to 2.6 kernels, including the need for a new initial filesystem. For several reasons we have already gave to the community, we have decided that for now, 2.6 and all that goes with it, is not really a great improvement. Which means that we (Bering-uClibc Team), are going to continue supporting a bootable floppy version, using a basic set of a complete, stable production grade router/firewall. For me personally that is a goal to maintain as long as possible, have a floppy based firewall. Altough I don't use floppy-based setup any longer, I still feel that if we make all efforts to keep supporting it, we will maintain focus. Having a larger boot media will lead to all sorts of excesses... Still, one idea from Erich was retained in my mind, and has also lurked in my ideas from time to time, which is the firmware thing... Discussing this will open a whole new can of worms, what should be considered basic and essential? Who will have the power to decide which stays in the firmware or not? We don't want to loose the modular design we have now. Not being able to backup the initramfs for example is not a very nice thing. I think we may be loosing too much time discussing stuff with little or no result... The central DB design comes to my mind for example... So, unless someone has a good small footprint design using the latest available techologies, providing the same capabilities we have now, lets leave the matter for now. p.s. please treat my opinions as my own and not as if I was talking on behalf of the Bering-uClibc Team! Luis Correia Bering uClibc Team Member PGP Fingerprint: BC44 D7DA 5A17 F92A CA21 9ABE DFF0 3540 2322 21F6 Key Server: http://pgp.mit.edu --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
RE: [leaf-devel] 2.6.x kernel support?
On Wed, 2005-08-24 at 08:45, Luis.F.Correia wrote: Is this whole conversation about loading to ram, using initramfs or any other kind relevant to the current branches of LEAF? Luis, No, but it may be relevant to future LEAF branches. The way I see it, the team i'm part of has already analysed the implications of switching over to 2.6 kernels, including the need for a new initial filesystem. For several reasons we have already gave to the community, we have decided that for now, 2.6 and all that goes with it, is not really a great improvement. That's a decision the LEAF Bering uClibc branch made. Which means that we (Bering-uClibc Team), are going to continue supporting a bootable floppy version, using a basic set of a complete, stable production grade router/firewall. Another decision by the LEAF Bering uClibc branch. WISP-Dist took another position. Altough I don't use floppy-based setup any longer, I still feel that if we make all efforts to keep supporting it, we will maintain focus. Having a larger boot media will lead to all sorts of excesses... Size is always a concern. From our project goals: Maintain as small a footprint as possible for release/branch target installations. We don't want to loose the modular design we have now. Not being able to backup the initramfs for example is not a very nice thing. Agreed. However, GNAP is much closer to a true embedded design. I think we may be loosing too much time discussing stuff with little or no result... The central DB design comes to my mind for example... I in no way think discussion is a wast of time. It's where ideas come from, and synergy is generated. So, unless someone has a good small footprint design using the latest available techologies, providing the same capabilities we have now, lets leave the matter for now. I've said this before, but it seems it's time to mention it again. Evolution as a project development model may not work for the community anymore. If the community wishes to move to a monolithic development model, I'll step aside. -- Mike Noyes mhnoyes at users.sourceforge.net http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
Hello Erich, How would such a thing be implemented for all binaries? Cramfs is ro so you can't populate a loop mounted cramfs. This would mean that you either put all /bin, /sbin, ... binaries in seperate cfs files and don't have packages anymore, put a bin.cfs in every package containing the binaries and make tons of mount points or create a cramfs within your running system from the loaded packages and still need the initial amount of RAM ... IMHO LEAF should have some kind of a firmware which holds most libraries and binaries of a certain release. Maybe we need firmware sets to satisfy Joe Average. We do have that: initrd and root contains most libraries and binaries of a certain release. Ofourse we can merge root and initrd but I don't see that as an advantage (it will cost more memory while booting). Long time ago I almost went crazy before I discovered that the ipsec package did _not_ contain ipsec.o. So there is no consistent package scheme anyway. That's not true, every module is placed in modules.lrp (repository) and packages are stand-alone. A module will change with every kernel upgrade, which isn't true for a package like ipsec. You will also get crazy if your package has to be updated with every release, besides from a maintanance viewpoint it's also a crime. Besides, you can also lzma your programs and have the same space savings. What about libraries? What about them? Libraries don't take much space and are shared with multiple programs. besides they are always needed so loaded in memory anyway. How do you see a way to create f.e. bin.cfs and still be able to install packages? Packages should consist mostly of configuration data. Backing up, for example, root.lrp is most of the time a pain in the butt. The same applies to initrd, ipsec, ssh and others. They are just big. Few people need to change the full content of a .lrp file. Most often we just configure /etc/network/interfaces and a small number of files in /etc/shorewall. I know you mentioned that before, but config files can change between versions, meaning that with an update of the binaries you can get very strange results. This is really a maintanance nightmare and will remove the coupling between a program and its config file. Now it's an entety (consistent package). There is hardly any need to backup initrd and root, and like you mention most often we only change interfaces and shorewall, so you only have to backup etc.lrp and shorwall.lrp. Eric --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
Erich Titl wrote: I just don't like to have all these binaries and libraries _unprotected_ and _uncompressed_ on RAM disk. mount -o ro ... should solve the unprotected part. -- Natanael Copa --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
Hello Erich, That's not true, every module is placed in modules.lrp (repository) and packages are stand-alone. A module will change with every kernel upgrade, which isn't true for a package like ipsec. You will also get crazy if your package has to be updated with every release, besides from a maintanance viewpoint it's also a crime. Mhhh... can't really agreee, a package that needs a certain kernel module has to provide that module unless you have some kind of dependency mechanism. Opinions are free :-) Why? In the case of ipsec an user can also add different ipsec modules like aes and the like for added functionality. Should they all be put in the base ipsec package and add a lot of size? The same for pppd/pppoe/pppoatm and there are numerous other examples. IMHO it's easier to just update modules.lrp/kernel itself with a kernel upgrade then to update numerous package which all contain kernel modules (which aren't compatible with a newer kernel) and reconfigure each package. Most users doesn't know if a package contains a module and probably don't know why a program doesn't work anymore. It's easier and more consistent with one modules.lrp. Also, Arne Bernin has an updgrade tool for modules.lrp he posted the link some time ago to this list. You can just copy your /etc/modules to it and a new modules.lrp (with correct dependencies) is created for the new kernel version. From a maintanance viewpoint it's also a crime to constantly recreate packages and put them in CVS when only a kernel version changes. Besides, you can also lzma your programs and have the same space savings. What about libraries? What about them? Libraries don't take much space and are shared with multiple programs. besides they are always needed so loaded in memory anyway. Why don't they take much space? They are on RAM disk and unless Nathanael is right and they are _not_ copied to memory They are copied to memory, programs use them (constantly) so they must be available in memory. How do you see a way to create f.e. bin.cfs and still be able to install packages? Packages should consist mostly of configuration data. Backing up, for example, root.lrp is most of the time a pain in the butt. The same applies to initrd, ipsec, ssh and others. They are just big. Few people need to change the full content of a .lrp file. Most often we just configure /etc/network/interfaces and a small number of files in /etc/shorewall. I know you mentioned that before, but config files can change between versions, meaning that with an update of the binaries you can get very strange results. This is really a maintanance nightmare and will remove the coupling between a program and its config file. Now it's an entety (consistent package). Mhhh maybe, but not so much of a maintenance problem but providing upgrade tools :-) An upgrade tool is perfectly capable of dealing with either option. So I don't see a need to split config from binaries. In the case of shorwall, where an upgrade tool would be of most help, the configuration isn't compatible between most releases so remove the coupling will make you very vulnerable for attacks (because of wrong config). But indeed an upgrade tool will be helpfull, but changing the packages (split) isn't necessary for such a tool. There is hardly any need to backup initrd and root, and like you mention most often we only change interfaces and shorewall, so you only have to backup etc.lrp and shorwall.lrp. I just don't like to have all these binaries and libraries _unprotected_ and _uncompressed_ on RAM disk. They are not unprotected ;) And now I stop ranting :)) cheers Erich Eric --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
RE: [leaf-devel] 2.6.x kernel support?
Hello Mike, Can you tell us what your intentions are? By reading your comments it seems like you are trying to force the creation of new branches. snip We don't want to loose the modular design we have now. Not being able to backup the initramfs for example is not a very nice thing. Agreed. However, GNAP is much closer to a true embedded design. Why? I think we may be loosing too much time discussing stuff with little or no result... The central DB design comes to my mind for example... I in no way think discussion is a wast of time. It's where ideas come from, and synergy is generated. Agree with that, but I don't think that's what Luis meant ;) So, unless someone has a good small footprint design using the latest available techologies, providing the same capabilities we have now, lets leave the matter for now. I've said this before, but it seems it's time to mention it again. Evolution as a project development model may not work for the community anymore. If the community wishes to move to a monolithic development model, I'll step aside. I don't think anyone here is wishing to move to a monolithic development model. Why are you saying that? Eric --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
RE: [leaf-devel] 2.6.x kernel support?
Hello Mike, It seems that my mails are currently bounced at the list, probably because of a wrong mail setting at my site. Please forward to the list. Can you tell us what your intentions are? By reading your comments it seems like you are trying to force the creation of new branches. Eric, Nothing different than I've always espoused. Nor am I trying to force anything. However, I still believe what I said in my Evolution as a project development model post. I also believe in your evolution model, I also believe in cooperation and discussion. I'm not sure if creating a lot of (incompatible?) branches is really helping or making things easier for an end user. Evolution is also (slowly) making things better by developers filling up gaps (like webconf), provide missing packages, making proof of concepts for new technology or creating a new package format like Nathaneal did. But in the end it would be nice to bring it all together instead of abbandon branches. I don't think anyone here is wishing to move to a monolithic development model. Why are you saying that? I interpreted Luis's comment as closing off discussion for leaf developers. Bering uClibc is the dominant leaf branch, and I thought the community might wish to coalesce around it. Maybe I read to much into Luis's comment. It wouldn't be the first time I misinterpreted something. :-( I think Luis meant something else, ofcourse I can't speak for him, but I think he means: please show how to make things better (he even mention it in the last lines). Note: I'll never be what I was prior to my accident, and I don't contribute much. I don't wish to be a point of discord in the community. It may just be time for me to step aside. No Mike, I think you are doing a wonderfull job. Luis, If I misinterpreted your comments, I apologize. -- Mike Noyes mhnoyes at users.sourceforge.net http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs Eric --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
On Wed, 2005-08-24 at 15:51, Martin Hejl wrote: In the end, I tend to agree with Luis. I'm not going to tell anybody to stop discussing new possibilities - but discussion without the ability/willingness to actually do more than just discuss things is pretty much what I contribute to management these days (I spent a fair share of my work time in meetings with people who discuss things they neither know how to do, nor would they be willing to provide the manpower to get it done - they just like discussing things). Maybe that's why I'm very skeptical about the value of discussions all by themselves. Discussing how to solve a problem at hand is perfectly fine, and usually also very useful - but discussing things despite the fact that every participant of the discussion has a different idea of what the actual issue one might be discussing is something else... Martin, That excludes me from any comments regarding leaf development, and relegates me to irrelevance. I'm not a programmer, or knowledgeable compared to most of you. :-( Almost anyone can do the website, docbook, and mailing list stuff. Again, maybe I just see too much of that kind of thing in my day job, and are therefore unable to see the honest attempt to find the best technological solution for leaf. If that's the case, I sincerely apologize for my cynicism/sarcasm. No need for you to apologize. You've contributed lots of time, effort, and code to LEAF. As they say, those that write the code determine its fate. -- Mike Noyes mhnoyes at users.sourceforge.net http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
On Tue, 2005-08-23 at 08:26 -0700, Mike Noyes wrote: Homer, It looks like Bering uClibc for kernel 2.6. The only bad thing I see is GNAPs 16MB footprint. This is probably caused by the kernel and lack of busybox in GNAP. Yea, but might give some ideas on the conversion. Though, asides from a floppy, what can you buy today that wouldn't hold a 16MB image? -- Homer Parker [EMAIL PROTECTED] A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Maybe because some people are too annoyed by top-posting. Q: Why do I not get an answer to my question(s)? --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mike Noyes wrote: | Charles, | Forth code is hard to find. The best I could locate is here: | | http://forth.sourceforge.net/ | http://cvs.sourceforge.net/viewcvs.py/forth/forth-repository/ Yep...the language is hard to search on (with forth being a very common english word), and most of the public code was around in the old printed listing and maybe online BBS days. Not a lot in a searchable archive form on the 'net. | I'll look for lua source next. Good luck! - -- Charles Steinkuehler [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.0 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFDCGsSLywbqEHdNFwRAvB9AJ0a58XwSS1XbpwX/OUeRHERdpPErACeOeqR iFo13QtaaUbZfpL/FVRiJqU= =Ppo9 -END PGP SIGNATURE- --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
On Sun, 2005-08-21 at 04:52, Charles Steinkuehler wrote: Yep...the language is hard to search on (with forth being a very common english word), and most of the public code was around in the old printed listing and maybe online BBS days. Not a lot in a searchable archive form on the 'net. Charles, Is this what you're looking for? Data Compression and Decompression in Tight Places http://www.programmersheaven.com/zone22/cat208/2252.htm I describe in this report an implementation of a general purpose data compression/decompression algorithm which is simple, achieves good compression, and uses few memory resources. File compression techniques for Forth programmers. | I'll look for lua source next. Good luck! It looks like lua programmers are using zlib. Lua is able to use C functions. 24 - An Overview of the C API http://www.lua.org/pil/24.html -- Mike Noyes mhnoyes at users.sourceforge.net http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
Mike, I think Charles means that we need tar and gzip equivalents in a interpreter language because that's the compression we use with every package (lrp = tar.gz). Those utilities are needed to install the packages with the linuxrc script inside the initramfs. Zlib is completely different and would also mean to include the zlib library next to the lua binary inside the initramfs, which is probably much bigger than using the native klibc compiled utilities. But we have some options to make things smaller and maybe even use the klibc utilities (if the needed ones are implemented yet). We (Bering-uClibc team) did some testing with read-only fs (romfs) in initrd, instead of minix and lzma compression of initrd and the kernel. It's also possible to use lzma on an initramfs It has some drawbacks, first with a read-only fs in the initial ramdisk (read-only seems to be the only option in initramfs anyway) it isn't possible to populate /dev before the root tmpfs is created. This means that you can't set tmpfs sizes in leaf.cfg anymore, this has to be done in syslinux.cfg (or an other bootloader config file). Secondly, if you use lzma you can't back-up the initrd anymore within a running system. Well you can but the utilities to do this are so big that you will loose the space savings enterily. This will probably also be the case in an initramfs anyway. I remember Charles talking about getting initial modules out of an initramfs so this may become a non issue. But even with those space savings it's possible that the size increase due to a 2.6 kernel and the pre-init utilities makes the image a lot bigger. Well, like I said before there are not much advantages right now but it's definitly something we look at. Eric On Sun, 2005-08-21 at 04:52, Charles Steinkuehler wrote: Yep...the language is hard to search on (with forth being a very common english word), and most of the public code was around in the old printed listing and maybe online BBS days. Not a lot in a searchable archive form on the 'net. Charles, Is this what you're looking for? Data Compression and Decompression in Tight Places http://www.programmersheaven.com/zone22/cat208/2252.htm I describe in this report an implementation of a general purpose data compression/decompression algorithm which is simple, achieves good compression, and uses few memory resources. File compression techniques for Forth programmers. | I'll look for lua source next. Good luck! It looks like lua programmers are using zlib. Lua is able to use C functions. 24 - An Overview of the C API http://www.lua.org/pil/24.html -- Mike Noyes mhnoyes at users.sourceforge.net http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/leaf-devel --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
On Sun, 2005-08-21 at 09:13, [EMAIL PROTECTED] wrote: Mike, I think Charles means that we need tar and gzip equivalents in a interpreter language because that's the compression we use with every package (lrp = tar.gz). Those utilities are needed to install the packages with the linuxrc script inside the initramfs. Zlib is completely different and would also mean to include the zlib library next to the lua binary inside the initramfs, which is probably much bigger than using the native klibc compiled utilities. But we have some options to make things smaller and maybe even use the klibc utilities (if the needed ones are implemented yet). Eric, There is a gcc wrapper (klcc) for compiling applications with klibc. Gzip, ash, and syslinux are already ported. Is tar the only missing item? http://www.kernel.org/pub/linux/libs/klibc/cvsroot/klibc/ klcc.1,v klcc.in,v -- Mike Noyes mhnoyes at users.sourceforge.net http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
Mike, Tar is not the only thing missing, f.e. there is no makedevs utility available and probably some other tools we need in linuxrc. Eric, Point taken. There are quite a few missing pieces. :-( Note: I believe udev replaced makedevs in linux 2.6. True, but udev is huge :-) Noted, and as I said before, I'm not recommending Bering uClibc move yet or ever. I still think we should evaluate and work up some prof-of-concept/Alpha linux_2.6+initramfs release for the community to play with. We will definitly move if new drivers are only available in kernel 2.6 or other advantages will show up. I'll stop now. :-( ;-) Eric --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
Hi Mike, Eric, We load and write to a compressed image currently. Initramfs is just a cpio image. I thought we could remove some complexity. We write to a gzip compressed loop mounted minix filesystem, we could indeed remove a bit of complexity. We could also use a gzip compressed romfs to have the same level of complexity as with a cpio image. But like romfs, initramfs is read only (AFAIK) so has the drawback of not being able to populate /dev before tmpfs in linuxrc. Besides, cpio is a seperate program (adding size) while we currently are using gzip for both initrd as the packages. e.g. initramfs -- ramfs/cramfs --backup-- cpio initramfs image Both ramfs and cramfs are read only and need big (especially cramfs) programs to make backup possible. We tested with romfs which userspace fs creation program is smaller than (c)ramfs. Probably just a bad WAG by me. Sorry I brought it up. :-( No problem, interesting discussion :-) Eric --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
On Fri, 2005-08-19 at 12:26, [EMAIL PROTECTED] wrote: But I agree, the initramfs approach would make a technical cleaner implementation. But it also means (because of redundancy and a bigger kernel (2.6)) a bigger base image. I also don't see a lot of real advantages for our case yet. Eric, There may not be at this time. However, I believe someone in the leaf community should evaluate it now, and work on prof-of-concept/Alpha. This would allow the rest of us to try it out, and see if it does have long term benefits that aren't obvious right now. Note: I'm not advocating Bering uClibc switch to initramfs. -- Mike Noyes mhnoyes at users.sourceforge.net http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
On Fri, 2005-08-19 at 15:01, Charles Steinkuehler wrote: I would, however, be in favor of using a very powerful, but very small 'shell-like' scripting language (ie: forth) in the initramfs, with the 'applications' being scripts rather than biaries, which is what would make this idea attractive (at least to me). The downside is utilities like tar and gunzip would have to be coded in forth, which I haven't been able to find (or had the spare time to write). Charles, Does the initramfs cpio newc/crc buffer format fulfill the compressed file support? I'll look for forth gzip and tar source. -- Mike Noyes mhnoyes at users.sourceforge.net http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mike Noyes wrote: | On Fri, 2005-08-19 at 15:01, Charles Steinkuehler wrote: | I would, however, be in favor of using a very powerful, but very small | 'shell-like' scripting language (ie: forth) in the initramfs, with the | 'applications' being scripts rather than biaries, which is what would make | this idea attractive (at least to me). The downside is utilities like tar | and gunzip would have to be coded in forth, which I haven't been able to | find (or had the spare time to write). | | Charles, | Does the initramfs cpio newc/crc buffer format fulfill the compressed | file support? It's not the initramfs system I'm worried about, particularly. It's the utilities we need to put INTO the initramfs in order to build a LEAF root filesystem image from the LRP files and configuration information (ie: LEAF.CFG and the kernel command line parameters). Several utilities are needed (look at /linuxrc for full details), but a lot of stuff could be worked around (ie: replace sed code with 'native' script) or are fairly trivial (ie: mount/umount and similar are pretty much just calls to the kernel with little user-mode function we'd have to duplicate), so it's stuff like gunzip and tar that I think will be hardest to replicate in a 'ground up' rework of the init scripts. | I'll look for forth gzip and tar source. Good luck! I've looked for these before, but have never been able to put much time into it. While you're on the prowl, it might also be good to keep an eye out for lua or (other small script interpreter) versions of these utilities as well... - -- Charles Steinkuehler [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.0 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFDB3jvLywbqEHdNFwRArfJAKDPpcmvOkiXhcOECxejh323Qjn85QCgiMNV kH5Jj9koHlcQeuV0dgkPTMg= =j3iv -END PGP SIGNATURE- --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
On Sat, 2005-08-20 at 11:39, Charles Steinkuehler wrote: | I'll look for forth gzip and tar source. Good luck! I've looked for these before, but have never been able to put much time into it. While you're on the prowl, it might also be good to keep an eye out for lua or (other small script interpreter) versions of these utilities as well... Charles, Forth code is hard to find. The best I could locate is here: http://forth.sourceforge.net/ http://cvs.sourceforge.net/viewcvs.py/forth/forth-repository/ I'll look for lua source next. -- Mike Noyes mhnoyes at users.sourceforge.net http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
RE: [leaf-devel] 2.6.x kernel support?
On Thu, 2005-08-18 at 23:45, [EMAIL PROTECTED] wrote: It's not very trivial to move to initramfs. Currently we use busybox in initrd which is compiled against uClibc, initramfs is using a method off pre-init programs which must be compiled against klibc. This means having the same type of programs compiled against klibc (which can't be busybox, because that won't be compiled against klibc) in initramfs and userspace programs (busybox) compiled against uClibc (or glibc in the case of plain Bering). Eric, I'm seeing people use initramfs with busybox. From what I understand either uClibc or klibc can be used with initramfs. Google string: initramfs busybox http://sourceforge.net/mailarchive/forum.php?thread_id=7934561forum_id=5348 Google string: initramfs uclibc klibc http://www.redhat.com/archives/dm-devel/2004-September/msg8.html Implementing initramfs would mean a lot of redundancy and added size. Besides not all programs we need in pre-init have a klibc version (like makedevs f.e.). I'm not understanding how this change would introduce redundancy. Of course, size is always a concern. We already use ramfs b.t.w. But what is currently the real benefit of initramfs to LEAF? The lwn article sums up the benefits, but I'm still looking for a better overview of the features/benefits. Initramfs arrives http://lwn.net/Articles/14776/ The combination initramfs and kernel 2.6 makes the distro 50% bigger. Have you tried it? I'm seeing boot images from other projects that are approximately 1.5M. Another interesting klibc and initramfs link: http://fxr.watson.org/fxr/source/Documentation/early-userspace/?v=linux-2.6.9 -- Mike Noyes mhnoyes at users.sourceforge.net http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
RE: [leaf-devel] 2.6.x kernel support?
On Fri, 2005-08-19 at 08:48, Mike Noyes wrote: On Thu, 2005-08-18 at 23:45, [EMAIL PROTECTED] wrote: We already use ramfs b.t.w. But what is currently the real benefit of initramfs to LEAF? The lwn article sums up the benefits, but I'm still looking for a better overview of the features/benefits. Initramfs arrives http://lwn.net/Articles/14776/ An initramfs howto http://www.vas.nu/pipermail/klibc/2005-August/00.html -- Mike Noyes mhnoyes at users.sourceforge.net http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mike Noyes wrote: | On Thu, 2005-08-18 at 23:45, [EMAIL PROTECTED] wrote: | We already use ramfs b.t.w. But what is currently the real benefit of | initramfs to LEAF? | | The lwn article sums up the benefits, but I'm still looking for a | better overview of the features/benefits. To me, it looks like the new initramfs would make it possible to do something like the old LRP patch (which allowed the kernel to use a tar.gz file as an initial ramdisk image) without requiring a kernel patch. It also looks like a handy way to solve various boot-strap problems (like getting a kernel with built-in RAID to look for RAID images *AFTER* some external module is loaded). A lot of this stuff is currently (at least circa 2.2/2.4, which I'm more familiar with) kind of 'hacked' into the kernel, and if your boot sequence is particularly odd, you might have to patch the kernel (or load an excessively large initial ramdisk). This feature might be useful in making a very small initial ramdisk image for leaf that 'bootstrapped' the full LEAF system, and would not require a C library of it's own (instead using klibc and essentailly making direct kernel calls). You'll never be able to run bind or sendmail this way, but being able to run a simple shell (or other script processor like forth, lua, etc.) and do things like extract tar files to build a root filesystem image would be all we'd need. This would eliminate the 'special' file that has existed in LRP/LEAF since the beginning (ie: root.lrp or initrd.lrp), replacing it with a (hopefully) standard chunk of init code that was simply linked with the kernel. - -- Charles Steinkuehler [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.0 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFDBhfRLywbqEHdNFwRAmVaAKC4ctI4urM+d2fudhAHPB6kPow07gCfeN8c 9COK9Mms7v+4FAAzqVYnw3k= =fP/3 -END PGP SIGNATURE- --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
Hello Mike, Charles, Still only have webmail, so I hope it will show up on the list This feature might be useful in making a very small initial ramdisk image for leaf that 'bootstrapped' the full LEAF system, and would not require a C library of it's own (instead using klibc and essentailly making direct kernel calls). You'll never be able to run bind or sendmail this way, but being able to run a simple shell (or other script processor like forth, lua, etc.) and do things like extract tar files to build a root filesystem image would be all we'd need. This is what I mean with redundant, you would need a shell (and other programs) in the initramfs (pre-init) and in userspace which isn't the same one. So you need the space for a klibc compiled shell in the initramfs and an other (uClibc/glibc) compiled shell in root.lrp (or so), while currently we use one shell which is both used for pre-init (linuxrc) and userspace. It isn't possible to use the klibc initramfs programs in userspace (AFAIK), which would be pointless anyway because the klibc versions are/should be very limited in functionality/size. For Mike: There are ofcourse images of ~1.5 Mb which contain an initramfs and 2.6 kernel, but they don't contain all the programs we have in such an image ;) But I agree, the initramfs approach would make a technical cleaner implementation. But it also means (because of redundancy and a bigger kernel (2.6)) a bigger base image. I also don't see a lot of real advantages for our case yet. Eric Spakman --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] 2.6.x kernel support?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 [EMAIL PROTECTED] wrote: | Hello Mike, Charles, | | Still only have webmail, so I hope it will show up on the list | | This feature might be useful in making a very small initial ramdisk image | for leaf that 'bootstrapped' the full LEAF system, and would not require | a C library of it's own (instead using klibc and essentailly making direct | kernel calls). You'll never be able to run bind or sendmail this way, | but being able to run a simple shell (or other script processor like | forth, lua, etc.) and do things like extract tar files to build a root | filesystem image would be all we'd need. | | This is what I mean with redundant, you would need a shell (and other | programs) in the initramfs (pre-init) and in userspace which isn't the | same one. So you need the space for a klibc compiled shell in the | initramfs and an other (uClibc/glibc) compiled shell in root.lrp (or so), | while currently we use one shell which is both used for pre-init (linuxrc) | and userspace. | It isn't possible to use the klibc initramfs programs in userspace | (AFAIK), which would be pointless anyway because the klibc versions | are/should be very limited in functionality/size. | | For Mike: There are ofcourse images of ~1.5 Mb which contain an initramfs | and 2.6 kernel, but they don't contain all the programs we have in such an | image ;) | | But I agree, the initramfs approach would make a technical cleaner | implementation. But it also means (because of redundancy and a bigger | kernel (2.6)) a bigger base image. I also don't see a lot of real | advantages for our case yet. I generally agree with your analysis. Using the initramfs system doesn't make sense for LEAF as it stands now. I would, however, be in favor of using a very powerful, but very small 'shell-like' scripting language (ie: forth) in the initramfs, with the 'applications' being scripts rather than biaries, which is what would make this idea attractive (at least to me). The downside is utilities like tar and gunzip would have to be coded in forth, which I haven't been able to find (or had the spare time to write). I think the entire extra 'footprint', including code to do tar, gunzip, and the forth interpreter itself could be squeezed into maybe 25K or so (assuming no re-use of the application scripts), meaning the 'redundancy' issue wouldn't be that bad, even for a floppy system. If you elected to reuse some of the scripted code, you'd only need a re-compiled interpreter for the appropriate libc, which for forth would probably mean 10-15K of 'redundancy'. ...of course, the probability of this actually happening is pretty close to zero (unless I somehow become independently wealthy!), but I still think it would be cool... - -- Charles Steinkuehler [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.0 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFDBlbULywbqEHdNFwRAnLCAKChLtlI9drIqDN9zgUebloC2L6g7gCg3F3L Mu/XxB1VWh7XxrRsKhulVbM= =ajCI -END PGP SIGNATURE- --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
RE: [leaf-devel] 2.6.x kernel support?
On Tue, 2005-07-12 at 14:09, Eric Spakman wrote: Currently we are not working on it, but we do look at kernel development ofcourse. Kernel 2.6 is much bigger than 2.4, it needs a lot of changes in the base packages (to make full use of the functions) and it has no real benefit for leaf now. Eric, Initramfs and ramfs may provide benefits to leaf branches. Re: Initrd and Initramfs http://www.ussg.iu.edu/hypermail/linux/kernel/0503.0/0739.html Initramfs arrives http://lwn.net/Articles/14776/ Ubuntu is using initramfs rather than initrd for Breezy Badger. Ubuntu Breezy Badger Colony 3 now available http://lwn.net/Articles/148200/ -- Mike Noyes mhnoyes at users.sourceforge.net http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
