Re: [Celinux-dev] CELF Project Proposal- Refactoring Qi, lightweight bootloader
On 12/27/09 23:15, Somebody in the thread at some point said: Hi Rob - I've also spent the last few years developing a project that produces native built environments for various QEMU targets and documents how to bootstrap various distros under them: http://impactlinux.com/fwl So I do have some firsthand experience here. I wasn't suggesting you don't have firsthand experience all over the place, eg, busybox, I know it. What I do suggest in the principles I have been bothering your Christmas with here can really pay off and are in the opposite direction to Qemu, bloated bootloaders, and buildroot style distro production. Fedora provides a whole solution there, with the restriction it's designed for native build, not cross. QEMU: it's not just for breakfast anymore. That's right Qemu often requires lunch, teatime and supper too to build anything :-) Which is why you hook it up to distcc so it can call out to the cross compiler, which speeds up the build and lets you take advantage of SMP. (Pages 217-226 of the above PDF.) Or you just do it native and you don't care about extreme rebuild case because you're using a distro that built it already. There's a critical advantage to build in specifically the execution environment you do cover in three words on slide 14 of your PDF I expand on at the end of the mail. That's also why my FWL project uses a statically linked version of busybox, because the static linking avoids the extra page retranslations on each exec and thus sped up the ./configure stage by 20%. (Pages 235-236 of PDF.) There's more things you can do to speed it up if you want to go down that rabbit hole (which the presentation does), and there's more work being done in qemu. (TCG was originally a performance hit but has improved since, although it varies widely by platform and is its own rabbit hole. Also switching to gigabit NIC emulation with jumbo frames helped distcc a lot.) Just saying people don't do distribution rebuilds for their desktop or server boxes unless they're Gentoo believers. So the need to consider this heavy duty stuff only exists at the distro. In Fedora case, guys at Marvell are running it and have access to lots of fast physical hardware they hooked up to Koji (you touch on this in your PDF). I don't think it's like they were waiting to hear about Qemu now they're gonna drop that and move to emulation. They're in the business of making fast ARM chips and they're going to outpace Qemu. But in general, Moore's Law says that qemu on current PC hardware is about the speed of current PC hardware seven years ago. (And obviously nobody ever built anything before 2003. :) The speed surely depends on the architecture being emulated. I did try Qemu on ARM on a nice desktop box here and it seemed unworkable to me. I am perfectly happy to trust Fedora to take care of disribution rebuilds for me. Newer ARM platforms like Cortex8+ and the Marvell Sheevaplug will outstrip emulated performance on a normal PC. There are 2GHz multi-core ARMs coming as well apparently. So I took the view I should ignore Qemu and get an early start on the true native build that will be the future of native build as opposed to cross due to that. Pages 24-34 of the above PDF go over this. The first two pages are on the advantages of native compiling on real hardware, the next eight pages are on the disadvantages. It can certainly be made to work, especially in a large corporation willing to spend a lot of money on hardware as a _prerequisite_ to choosing a deployment platform. This is what I have been calling buildroot thinking again. What do you think you will be rebuilding exactly out of Fedora when you put it on an ARM11+ system? Typically, it's going to be zero packages, nothing at all. What you will need to build typically is this: - bootloader (using Fedora's cross compiler package) - kernel (using Fedora's cross compiler package) - your own apps and libs (native build on device against libs there) In addition you will need to take evasive action around the boot flow, but to get started that's just init=/bin/my-init.sh For hobbyists, small businesses, and open source developers in general, there are significant advantages to emulation. (Page 208 comes to mind.) And if you _are_ going to throw money at hardware, x86-64 continues to have better price/performance ratio, which was always its thing. The point is you will definitely be throwing lots of money at hardware in this game, your ARM device platform. Since all products start with zero users, and we get to be the first, we had best spend all our time gaining experience with the hardware we plan to ship. If there are problems with upstream unless it's something really in the plumbing like cache behaviour which is a kernel issue, it can normally be shown in the code easy enough. Pages 68-71. If your definition of embedded development is using off the
Re: [Celinux-dev] CELF Project Proposal- Refactoring Qi, lightweight bootloader
On 12/28/09 00:21, Somebody in the thread at some point said: Hi - I started programming on a commodore 64. By modern standards, that system is so far down into embedded territory it's barely a computer. And yet people did development on it. My dear Rob I got started on a PET, I can understand your POV :-) kicked into the server space by the iPhone and such. I want to follow Moore's Law down into disruptive technology territory and find _out_ what it does. The big challenge I see is delivering highly complex Linux devices with insufficient developers in a way that won't disappear up its own ass and kill the project / customer with delays or failure to perform. It's nice if the device is efficient with every cycle, but that is a geek preoccupation. Many customers in suits will tell to spend an extra $1 to overcome it by hardware and gain back $5 from accelerated time to market, so long as they can depend on the high quality of the software basis to not kill that logic by delays. -Andy -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
How to store kernel pranic/oops
Hi, I know the open project proposal 2010 is closed, but it's just to start a discussion. It would be nice to save oops/panic automatically in a structure/file in ram. At the moment there are two way to save information: mtdoops (save the information in flash), with kdump/kexec (we can extract the dmesg from vmcore file). With these tools there are some drawbacks: 1) There are embedded systems without a flash where to save the information; 2) we could consider this kind of logs too volatile for a flash, I mean there's no reason to store it for a long time, it's important to recover and read them as soon as possible, at next boot for example. 3) kdump requires a lot of ram and resources for embedded systems. 4) kexec is available only for some archs. It would be nice to have a ramoops to save in a circular buffer in a persistent ram this kind of information. Any comments? Is there already anything similar out-of-tree? Marco -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to store kernel pranic/oops
On Mon, 2009-12-28 at 12:43 +0100, Marco Stornelli wrote: It would be nice to have a ramoops to save in a circular buffer in a persistent ram this kind of information. Any comments? Is there already anything similar out-of-tree? Can't it be done with what's in the tree already? Just create an MTD device using phram or something else, then point mtdoops at it. -- David WoodhouseOpen Source Technology Centre david.woodho...@intel.com Intel Corporation -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: CELF Project Proposal - Feasibility analisys of Android introduction in a completely tested industrial device
Raffaele Recalcati wrote: 2009/12/18 Raffaele Recalcati lamiapost...@gmail.com: Summary: Feasibility analisys of Android introduction in a completely tested industrial device. Description: By now Android has been ported to 600Mhz Cortex A8 cpu or similar. The declared Android requirements are instead lower, about 200Mhz Arm9 cpu with 100Mhz Ram bus. So I think the growing interest in this O.S. lacks some porting to less powerful cpus. The reasons to do this porting are commercial because of Google market power, but are also technnical, because Android debugging environment is very nice for not embedded developers. This could help the diffusion of opensource embedded Linux. This is interesting. Can you let me know if the focus of this work is to experiment with the lower bounds of Android scalability, or whether the focus is on Android use in industrial devices? If the latter, than it would be good to hear more about what might be needed to extend (or reduce :-) ) Android to fit this market. I'll add a proposal for this, but I'd like to hear more to clarify the proposal. Thanks, -- Tim = Tim Bird Architecture Group Chair, CE Linux Forum Senior Staff Engineer, Sony Corporation of America = Looking at http://git.kernel.org/?p=linux%2Fkernel%2Fgit%2Ftorvalds%2Flinux-2.6.gita=searchh=HEADst=commits=android my idea seems to be ... not very good at all !! What do you think? I'm not a kernel mainline developer so I don't understand the real meaning. Below there is my new proposal, but before accepting it please hear this new idea, that could be better and possible to be used also in 2.6.33 kernel: Adding to Maemo or Debian or Gentoo embedded systems a debugging environment similar to Android: I mean the possibility to connect to a single debugging server and, in a graphical very easy (Eclipse or QT Creator) way, connecting the gdb to the desired process, or, see rootfs, or tracing cpu usage, ... What I really like of Android is the fantastique environment, with the emulator that can do step by step debugging - Summary: Feasibility analisys of Android introduction in a completely tested industrial device. Description: Android is a new important graphic interface and a java virtual machine optimized for embedded devices. The Android breaking feature in respect of traditional GNU Linux embedded operating systems has discouraged the possibility to enhance an already existing GNU Linux embedded industrial device with at least the Android's graphic interface. By now Android has been fully ported to Cortex A8 cpus starting new mobile phone projects from scratch. The reasons to do this work are commercial because of Google market power, but are also technical, because Android debugging environment is very nice for not embedded developers. This could help the diffusion of opensource embedded GNU Linux in a wider point of view. The work should be maybe against the 2.6.31 kernel. The idea is to preserve industrial tested development but adding Android graphical interface and debugging features. The cpu dependency of the porting will be as small as possible. The most important kernel interfaces will be investigated, at least framebuffer for tft lcd, touchscreen, audio ac97 interface, usb host for usb pen. Related work: * Android Porting - http://www.kandroid.org/wbr/wbrandroid_pdk/ * Android Pxa270 - http://android-pxa270.sourceforge.net/ Scope: This should take more than 1 month for feasibility analysis. -- -- www.opensurf.it -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Re: How to store kernel panic/oops
David Woodhouse wrote: Can't it be done with what's in the tree already? Just create an MTD device using phram or something else, then point mtdoops at it Yes of course, if possible we shouldn't reinvent the wheel but I wondered if there was something more specific. To add mtdoops (more or less 1k) we have to add mtd subsys (more or less 14k) to the kernel to achieve this and it's all overhead. Marco -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Celinux-dev] CELF Project Proposal- Refactoring Qi, lightweight bootloader
Andy == Andy Green a...@warmcat.com writes: Hi, Andy This is what I have been calling buildroot thinking again. Thanks for an interesting discussion and all your free buildroot advertising ;) (buildroot maintainer) Andy This thread was meant to be about merits of Qi, it's kinda gone Andy off into embedded with distro rootfs because the philosophy is Andy related. In both cases burden on the developer is intended to be Andy removed and effort simplified to get the job done more reliably Andy and quicker. Sure, if your embedded device is very PC like (size/functionality) that makes sense. -- Bye, Peter Korsgaard -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Celinux-dev] CELF Project Proposal- Refactoring Qi, lightweight bootloader
On 12/28/09 19:57, Somebody in the thread at some point said: Hi Peter - Thanks for an interesting discussion and all your free buildroot advertising ;) (buildroot maintainer) Hey you are welcome... in the situations where you can't play the distro game then buildroot is the lifesaver. Same as U-Boot I have experienced in detail the amount of work involved in what buildroot delivers and it has to be respected for what it does. I'm really saying there's a region where buildroot is the right answer and a threshold beyond which a real distro is the right answer, we should not just keep on doing what we have being doing. (But you want to be careful you don't grow into OpenEmbedded's build system. For me it failed partway through a 1000+ package build on trying to build *host* dbus libs for me (yes, host dbus). I just wanted to package hello world on Openmoko. They did fix it in the end so it would build host dbus OK to be fair. But I felt that was a very long way from the point.) Andy This thread was meant to be about merits of Qi, it's kinda gone Andy off into embedded with distro rootfs because the philosophy is Andy related. In both cases burden on the developer is intended to be Andy removed and effort simplified to get the job done more reliably Andy and quicker. Sure, if your embedded device is very PC like (size/functionality) that makes sense. Thanks. As written the threshold seems to be at ARM11+ and SD particularly making it into enough of a PC you don't have to worry about bash vs ash or where a few 100MB of storage is going to come from. -Andy -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Celinux-dev] CELF Project Proposal- Refactoring Qi, lightweight bootloader
On Monday 28 December 2009 04:27:04 Andy Green wrote: I wasn't suggesting you don't have firsthand experience all over the place, eg, busybox, I know it. What I do suggest in the principles I have been bothering your Christmas with here can really pay off and are in the opposite direction to Qemu, bloated bootloaders, and buildroot style distro production. In the PDF I spent a dozen pages pointing out that I think buildroot is silly and how it diverged from its original goals. Why do you bring it up here? Building natively is nice. I think cross compiling extensive amounts of stuff is generally counterproductive, and you don't seem to disagree with that. (However, if there weren't cases where cross compiling _does_ make sense I wouldn't bother building the second stage cross compilers in my project and packaging them up for other people to use. I don't recommend it != people who do this are stupid.) You're conflating a bunch of issues. You can boot a full distro under qemu or under real hardware, so use a native distro is orthogonal to develop under emulation vs develop on target hardware. And building a hand-rolled system or assembling prebuilt distro bits can both be done natively (under either kind of native), so that's orthogonal too. I'm really not understanding what argument you're making. The one specific configuration I've chosen to use is better than any other possible configuration... Possibly there's a for your needs attached somewhere that you're missing? You seem to be confusing that's not an interesting option for me with that's not an interesting option. You're conflating a bunch of different issues into a One True Way, which is sad. Fedora provides a whole solution there, with the restriction it's designed for native build, not cross. QEMU: it's not just for breakfast anymore. That's right Qemu often requires lunch, teatime and supper too to build anything :-) Which is why you hook it up to distcc so it can call out to the cross compiler, which speeds up the build and lets you take advantage of SMP. (Pages 217-226 of the above PDF.) Or you just do it native and you don't care about extreme rebuild case because you're using a distro that built it already. I find reproducibility to be kind of nice. (Remember when ubuntu dropped Powerpc support? Or how Red hat Enterprise is finally dropping itanic? Even today can you build a full distro natively on new hardware like microblaze or blackfin?) I also don't see any difference between your let the distro handle everything with the vendor supplies a BSP, why would you need anything else? That's certainly a point of view, and for you it may be fine. Anything can be outsourced. But I don't have to do it, therefore nobody does is a questionable position. There's more things you can do to speed it up if you want to go down that rabbit hole (which the presentation does), and there's more work being done in qemu. (TCG was originally a performance hit but has improved since, although it varies widely by platform and is its own rabbit hole. Also switching to gigabit NIC emulation with jumbo frames helped distcc a lot.) Just saying people don't do distribution rebuilds for their desktop or server boxes unless they're Gentoo believers. Linux From Scratch does not exist? My friend Mark (who inherited a Slackware for PowerPC cross-build system) is in reality unemployed? Nobody ever actually works _on_ Fedora and needs to do a development build? It's interesting to see someone who can imply these things with a straight face on an embedded development list. So the need to consider this heavy duty stuff only exists at the distro. In Fedora case, guys at Marvell are running it and have access to lots of fast physical hardware they hooked up to Koji (you touch on this in your PDF). I don't think it's like they were waiting to hear about Qemu now they're gonna drop that and move to emulation. You also seem to be unaware that QEMU was only started in 2003 (first commit to the repository, Feb 18, 2003), and didn't even work on x86 all that well until the end of 2005. Decent arm support started showing up in 2006 but it was darn wonky, and the switch from dyngen to TCG didn't happen until Feb 1, 2008, before which you couldn't even build qemu with gcc 4.x: http://landley.net/qemu/2008-01-29.html#Feb_1,_2008_-_TCG Therefore, they didn't drop support, QEMU wasn't ready for prime time back in 2006 when my then-boss (Manas Saksena) quit Timesys to go launch Fedora for Arm. (The boss I had after that, David Mandala, is now in charge of Ubuntu Mobile. Timesys had some darn impressive alumni, pity the company screwed up so badly none of us stayed around. I myself stayed through four bosses, three CEOs, and outlasted 80% of my fellow engineers...) By the way: http://fedoraproject.org/wiki/Architectures/ARM/HowToQemu You continue to conflate
Re: Re: How to store kernel panic/oops
On Mon, Dec 28, 2009 at 07:03:48PM +0100, Marco Stornelli wrote: David Woodhouse wrote: Can't it be done with what's in the tree already? Just create an MTD device using phram or something else, then point mtdoops at it Yes of course, if possible we shouldn't reinvent the wheel but I wondered if there was something more specific. To add mtdoops (more or less 1k) we have to add mtd subsys (more or less 14k) to the kernel to achieve this and it's all overhead. panic/oops information is punted down to a kmsg dumper, which mtdoops just happens to be one of. If you wanted to do your own platform specific hack for size constraints you could easily just have your own special one instead. Take a look at linux/kmsg_dump.h. -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
