Re: busybox applet selection (again)
Hi, ext Clarence Risher wrote: On Jan 2, 2008 7:04 AM, Eero Tamminen [EMAIL PROTECTED] wrote: As to bloatedness, there's an apt-hook installed on the device that removes the extra docs when a package is installed (at least man info pages). Developer could also run something like Debian's localepurge. Most of the package sizes can come from documentation and localization (which Busybox tools are lacking), sometimes also from extra binaries. IMHO the actual binary size is significant only in fairly rare cases. Just my two cents... I have a guide for setting up localepurge on the 770 at http://sparrsstuff.com/localepurge It saves enough space to be worthwhile. Thanks! What are the Busybox grep command issues you mention in that page? Are they still issues with the Chinook Busybox grep? If yes, are they reported to Busybox bugzilla or maemo one? - Eero ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: busybox applet selection (again)
On 1/2/08, Eero Tamminen [EMAIL PROTECTED] wrote: This is fairly unlikely case, but IMHO reason enough not to have two busybox versions with different set of tools. Main thing is that the developer can get the required tool, not whether it's in Busybox I think. I agree that the important thing is access to the tools. I thought a repackaged busybox would be a cheap way to get those tools, but I'm equally happy to have the full versions of them if they are readily available (to date, they are not). This also suggests broadening the conversation to the availability and organization of tools that aren't available in busybox or in the developer repos but are generally useful for developers/advanced users. For example ssh and svn are available in some form in extras or in the garage, but having them in the developer/sdk repository seems more appropriate and will most likely mean they are better maintained (no offense to community maintainers). How arduous is the process for getting new tools into the developer repo? Will advanced users looking for command line tools think to look in the developer repo? Diff is both a Debian essential and in POSIX standard so I guess it's addition could be considered for maemo base system. If(?) Busybox diff is good enough for dpkg (for showing config file diffs on upgrade), it could be provided by Bysybox. If we have a full version of one, we may as well have a full version of the other (since dpkg apparently does fine without diff now) Other similar binaries would be clear, logname, nice, nohup, od, split and sum. Except for the first one, in Debian they are all in coreutils package. Maybe you could make a bug about these with me on CC, I can then add it to our internal system (they are API changes so they need to go through certain review process). I will do this sometime in the next week. I've added a project to the garage where I'll upload replacement versions of busybox with more options switched on for the community to test: http://garage.maemo.org/projects/busybox-test/ (it will take me a day or two to upload sources and get some releases up). Based on the conversation here, it seems it makes the most sense to just switch on more of the core utils and leave most other things out (networking tools, archive utilities etc) ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: busybox applet selection (again)
ext Damien Moore [EMAIL PROTECTED] writes: https://bugs.maemo.org/show_bug.cgi?id=989 the bug was marked WONTFIX Eero Tamminen's resolution was to not add any additional applets to BusyBox because in his opinion those needs can best be met by creating full versions of the tools in separate packages. I don't think this is a good idea because it creates a proliferation of unnecessarily bloated packages with the attendant problems of maintaining multiple packages (keeping in mind that the target hardware is a capacity constrained tablet). The benefit of busybox is that most appplets add just a few kb to the binary size and all of them sit inside a single binary. I have to agree with Eero here. It's much more useful to have the original tools available instead of (too) simple busybox variants. For example, you need to be root to run busybox ping and it does not support flood ping. Having iputils-ping would fix both of these problems. -- Kalle Valo ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: busybox applet selection (again)
Hi, I think it's good you raised this issue as it needs some discussion. Busybox is a base part of maemo, but we don't yet list it as an official SDK API although most packages need/use it when they are installed. ext Damien Moore wrote: I'd like to bring up the busybox applet selection issue again. This was previously discussed here: http://lists.maemo.org/pipermail//maemo-developers/2007-April/009738.html with associated bug here: https://bugs.maemo.org/show_bug.cgi?id=989 the bug was marked WONTFIX Eero Tamminen's resolution was to not add any additional applets to BusyBox because in his opinion those needs can best be met by creating full versions of the tools in separate packages. I don't think this is a good idea because it creates a proliferation of unnecessarily bloated packages with the attendant problems of maintaining multiple packages (keeping in mind that the target hardware is a capacity constrained tablet). The maintenance wouldn't be a problem. Then those packages can come directly from Debian. Also, then trying to install a properly working replacement for a buggy/incomplete Busybox functionality doesn't mean that one would need to remove first Busybox and half the device software depending on what else Busybox provides. As to bloatedness, there's an apt-hook installed on the device that removes the extra docs when a package is installed (at least man info pages). Developer could also run something like Debian's localepurge. Most of the package sizes can come from documentation and localization (which Busybox tools are lacking), sometimes also from extra binaries. IMHO the actual binary size is significant only in fairly rare cases. The benefit of busybox is that most appplets add just a few kb to the binary size and all of them sit inside a single binary. If they are not on the device, but separately installable packages in a tools repository, they take even less space on most users' devices. :-) N8x0 devices have more Flash available than 770, so the disk usage isn't anymore as severe issue as it was. This differs from package to package, so the decision about this needs to be done case by case. My proposal is to create an alternative essential busybox package that optionally replaces the default busybox, say busybox-max. busybox-max would be built with many more applets (more of the archive tools, more shell support (e.g. longer command histories and color coded ls), I think improving the already configured Busybox functionality is a good idea. We're providing an xterm with the device, so it makes sense to have what's already installed as usable as possible. It would be nice to have a bug about that with details about what Busybox options should/could be enabled/disabled and what will be their effect to Busybox size RAM usage. What are your propositions for enhancing the currently configured Busybox tools? more networking tools etc. This package would not be installed by default, it would just be an optional package for developers/hackers. I don't know how well dpkg would handle the package replacement but it is worth exploring. However, I don't think it's a good idea to add more tools to Busybox. Busybox is an essential package, so having incompatible versions of it or trying to replace some of the binaries with the real versions will end up with packaging conflicts. IMHO only good reasons for adding a tool to Busybox would be compatibility to Debian (derived distributions) from which most of the other maemo tools come from. I.e. if Busybox claims to provide a certain Debian package, it should try to provide as many binaries from that package as possible. And even this only if: - The real package (without docs and localization) is significantly larger than the corresponding Busybox binaries size - The package in question is an essential in Debian and at least some of its binaries are used by the preinstalled device software What's the problem of using the real packages / functionality instead (specific example, please)? E.g. if you want a good interactive shell, why not add that as a separate package (Busybox POSIX shell would then be used only by the shell scripts)? - Eero ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: busybox applet selection (again)
On 1/1/08, sebastian maemo [EMAIL PROTECTED] wrote: What would happen with an order like this?... # apt-get -o APT::Force-LoopBreak=1 install busybox replacement Maybe a broken system? I would think almost certainly a broken system. I think dpkg scripts depend on busybox tools being there, so they can't be removed (even briefly) just replaced. There must be a way to do it. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: busybox applet selection (again)
On 1/2/08, Kalle Valo [EMAIL PROTECTED] wrote: I have to agree with Eero here. It's much more useful to have the original tools available instead of (too) simple busybox variants. what I'm suggesting would be an optional package that, if set up correctly, won't break anything (but will block the installation of overlapping tools in other packages). Users who want original tool sets could take the standard busybox package and install separate tools packages. If having the original tools is always better perhaps busybox shouldn't be used at all? Yet another alternative: a package that replaces busybox with original tools For example, you need to be root to run busybox ping and it does not support flood ping. for good or bad the root problem is overcome with # chmod 4777 /bin/busybox Having iputils-ping would fix both of these problems. why not take all of iputils? On 1/2/08, Kalle Valo [EMAIL PROTECTED] wrote: ext Damien Moore [EMAIL PROTECTED] writes: https://bugs.maemo.org/show_bug.cgi?id=989 the bug was marked WONTFIX Eero Tamminen's resolution was to not add any additional applets to BusyBox because in his opinion those needs can best be met by creating full versions of the tools in separate packages. I don't think this is a good idea because it creates a proliferation of unnecessarily bloated packages with the attendant problems of maintaining multiple packages (keeping in mind that the target hardware is a capacity constrained tablet). The benefit of busybox is that most appplets add just a few kb to the binary size and all of them sit inside a single binary. I have to agree with Eero here. It's much more useful to have the original tools available instead of (too) simple busybox variants. For example, you need to be root to run busybox ping and it does not support flood ping. Having iputils-ping would fix both of these problems. -- Kalle Valo ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: busybox applet selection (again)
ext Damien Moore [EMAIL PROTECTED] writes: On 1/2/08, Kalle Valo [EMAIL PROTECTED] wrote: I have to agree with Eero here. It's much more useful to have the original tools available instead of (too) simple busybox variants. what I'm suggesting would be an optional package that, if set up correctly, won't break anything (but will block the installation of overlapping tools in other packages). Users who want original tool sets could take the standard busybox package and install separate tools packages. If having the original tools is always better perhaps busybox shouldn't be used at all? That's what Eero is proposing here: disable certain tools, for example ping, from busybox and install the full versions, for example iputils-ping. example, you need to be root to run busybox ping and it does not support flood ping. for good or bad the root problem is overcome with # chmod 4777 /bin/busybox I'm not even going to comment on that. Having iputils-ping would fix both of these problems. why not take all of iputils? Sure, if there is a need. ping was just an example. -- Kalle Valo ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: busybox applet selection (again)
On 1/2/08, Eero Tamminen [EMAIL PROTECTED] wrote: The maintenance wouldn't be a problem. Then those packages can come directly from Debian. Also, then trying to install a properly working replacement for a buggy/incomplete Busybox functionality doesn't mean that one would need to remove first Busybox and half the device software depending on what else Busybox provides. Shouldn't need to remove any device software if REPLACES is used and if any dependencies in the system respect treat the replacement as equivalent to the original. The only dangerous thing is that dpkg scripts presumably rely on busybox being there: at some point during install the old busybox is replaced by the new one, old symlinks are replaced by new and we have make sure that this won't create problems. As to bloatedness, there's an apt-hook installed on the device that removes the extra docs when a package is installed (at least man info pages). good to know. although sometimes I find the absence of docs an obstacle (would be nice if they could be saved to one of the mmcs) Developer could also run something like Debian's localepurge. Most of the package sizes can come from documentation and localization (which Busybox tools are lacking), sometimes also from extra binaries. IMHO the actual binary size is significant only in fairly rare cases. N8x0 devices have more Flash available than 770, so the disk usage isn't anymore as severe issue as it was. This differs from package to package, so the decision about this needs to be done case by case. I'll think about this. If it is really true that binary size doesn't matter, then that's a strong case for not using busybox at all (or at least offering a full versioned drop in toolset replacement for busybox) However, I don't think it's a good idea to add more tools to Busybox. Busybox is an essential package, so having incompatible versions of it or trying to replace some of the binaries with the real versions will end up with packaging conflicts. that's why I advocated making it optional. to resolve the conflict, reinstall the slimmed down one. Can busybox be configured to use shared libraries for classes of toolsets? that way, optional packages could be provide distinct functionality in a shared object with accompanying symlinks without the need to overwrite the busybox binary. IMHO only good reasons for adding a tool to Busybox would be compatibility to Debian (derived distributions) from which most of the other maemo tools come from. I.e. if Busybox claims to provide a certain Debian package, it should try to provide as many binaries from that package as possible. And even this only if: - The real package (without docs and localization) is significantly larger than the corresponding Busybox binaries size - The package in question is an essential in Debian and at least some of its binaries are used by the preinstalled device software It would be nice to have a bug about that with details about what Busybox options should/could be enabled/disabled and what will be their effect to Busybox size RAM usage. What are your propositions for enhancing the currently configured Busybox tools? ok, at the very least I think all of the coreutils should be available with more options switched on (personal favorites: diff, patch, color coded ls etc). I'd also like to have all of the archive tools available but maybe I'm in the minority. Even a busybox with virtually everything switched on is only 740kb binary. I can't imagine RAM usage would be an issue. I'll add more later, when I file the bug What's the problem of using the real packages / functionality instead (specific example, please)? E.g. if you want a good interactive shell, why not add that as a separate package (Busybox POSIX shell would then be used only by the shell scripts)? the problem isn't a specific one, it's a general one. 1. it requires a package hunt to create a working system (perhaps this is resolved by the creation of a super package that installs all of the others). 2. I can't help but think there will inevitably be small patches to the debian packages that makes maintenance of many sets of packages a nightmare. Maybe I'm wrong about that. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: busybox applet selection (again)
Hi, ext Damien Moore wrote: On 1/2/08, Eero Tamminen [EMAIL PROTECTED] wrote: The maintenance wouldn't be a problem. Then those packages can come directly from Debian. Also, then trying to install a properly working replacement for a buggy/incomplete Busybox functionality doesn't mean that one would need to remove first Busybox and half the device software depending on what else Busybox provides. Shouldn't need to remove any device software if REPLACES is used and if any dependencies in the system respect treat the replacement as equivalent to the original. The only dangerous thing is that dpkg scripts presumably rely on busybox being there: at some point during install the old busybox is replaced by the new one, old symlinks are replaced by new and we have make sure that this won't create problems. The problematic use-case would be trying to install a new package that works with the default busybox but depends/requires something that is in a Debian package from which only some binaries were included into busybox-max, but not that required one. With default Busybox the co. package would be automatically installed (as its contents don't conflict with busybox) and everything is fine. With busybox-max developer would need to replace busybox-max with the default busybox to be able to install the new package and its deps. This is fairly unlikely case, but IMHO reason enough not to have two busybox versions with different set of tools. Main thing is that the developer can get the required tool, not whether it's in Busybox I think. As to bloatedness, there's an apt-hook installed on the device that removes the extra docs when a package is installed (at least man info pages). good to know. although sometimes I find the absence of docs an obstacle (would be nice if they could be saved to one of the mmcs) I think you can just do apt-get remove docpurge if you don't like this. Unfortunately it doesn't restore the docs that were removed before this. :-) Developer could also run something like Debian's localepurge. Most of the package sizes can come from documentation and localization (which Busybox tools are lacking), sometimes also from extra binaries. IMHO the actual binary size is significant only in fairly rare cases. N8x0 devices have more Flash available than 770, so the disk usage isn't anymore as severe issue as it was. This differs from package to package, so the decision about this needs to be done case by case. I'll think about this. If it is really true that binary size doesn't matter, It matters, but less. And it depends on whether that functionality needs to be pre-installed. If it doesn't need to be pre-installed size is not really a concern with most packages. Developers can install only the tools they care about and e.g. keep rest on an MMC card. then that's a strong case for not using busybox at all (or at least offering a full versioned drop in toolset replacement for busybox) Busybox is pre-installed. However, I don't think it's a good idea to add more tools to Busybox. Busybox is an essential package, so having incompatible versions of it or trying to replace some of the binaries with the real versions will end up with packaging conflicts. that's why I advocated making it optional. to resolve the conflict, reinstall the slimmed down one. Can busybox be configured to use shared libraries for classes of toolsets? that way, optional packages could be provide distinct functionality in a shared object with accompanying symlinks without the need to overwrite the busybox binary. They would be pretty small libraries and not shared by any other binaries. Shared libraries include both startup (symbol resolving), and RAM cost (please read Depper's dsohowto), and don't really solve the problem of different busybox versions having different conflicts. Do all Busybox target platforms even support shared libraries (I doubt the chances of this patch getting to upstream Busybox)? IMHO only good reasons for adding a tool to Busybox would be compatibility to Debian (derived distributions) from which most of the other maemo tools come from. I.e. if Busybox claims to provide a certain Debian package, it should try to provide as many binaries from that package as possible. And even this only if: - The real package (without docs and localization) is significantly larger than the corresponding Busybox binaries size - The package in question is an essential in Debian and at least some of its binaries are used by the preinstalled device software It would be nice to have a bug about that with details about what Busybox options should/could be enabled/disabled and what will be their effect to Busybox size RAM usage. What are your propositions for enhancing the currently configured Busybox tools? ok, at the very least I think all of the coreutils should be available with more options switched on (personal favorites: diff,
Re: busybox applet selection (again)
On Jan 2, 2008 7:04 AM, Eero Tamminen [EMAIL PROTECTED] wrote: As to bloatedness, there's an apt-hook installed on the device that removes the extra docs when a package is installed (at least man info pages). Developer could also run something like Debian's localepurge. Most of the package sizes can come from documentation and localization (which Busybox tools are lacking), sometimes also from extra binaries. IMHO the actual binary size is significant only in fairly rare cases. Just my two cents... I have a guide for setting up localepurge on the 770 at http://sparrsstuff.com/localepurge It saves enough space to be worthwhile. (sorry to Eero about the double reply) ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
busybox applet selection (again)
I'd like to bring up the busybox applet selection issue again. This was previously discussed here: http://lists.maemo.org/pipermail//maemo-developers/2007-April/009738.html with associated bug here: https://bugs.maemo.org/show_bug.cgi?id=989 the bug was marked WONTFIX Eero Tamminen's resolution was to not add any additional applets to BusyBox because in his opinion those needs can best be met by creating full versions of the tools in separate packages. I don't think this is a good idea because it creates a proliferation of unnecessarily bloated packages with the attendant problems of maintaining multiple packages (keeping in mind that the target hardware is a capacity constrained tablet). The benefit of busybox is that most appplets add just a few kb to the binary size and all of them sit inside a single binary. My proposal is to create an alternative essential busybox package that optionally replaces the default busybox, say busybox-max. busybox-max would be built with many more applets (more of the archive tools, more shell support (e.g. longer command histories and color coded ls), more networking tools etc. This package would not be installed by default, it would just be an optional package for developers/hackers. I don't know how well dpkg would handle the package replacement but it is worth exploring. thanks, Damien Moore PS: Can someone please tell me where I can obtain the most recent busybox binary and source packages (v1.6.x, I think) that ships with OS2008. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: busybox applet selection (again)
@Clarence: I agree that dpkg should be able to handle this. Would REPLACE: busybox(##version##) be needed instead of PROVIDES? Because busybox is an essential package, you can't uninstall the existing one to reinstall the new one, which makes me suspect that the new one would need to REPLACE the old one. BTW, I found the missing busybox-1.6.1 package in the maemo repos -- brain meltdown on my part. On 1/1/08, Clarence Risher [EMAIL PROTECTED] wrote: apt can handle a situation like this just fine, and dpkg can too with a little work, and does so for many types of libraries and binaries already in debian and ubuntu. I believe you just make the new package PROVIDES the same things as the old package, with the same version number, and CONFLICTS the old package. On Jan 1, 2008 12:00 PM, Damien Moore [EMAIL PROTECTED] wrote: I'd like to bring up the busybox applet selection issue again. This was previously discussed here: http://lists.maemo.org/pipermail//maemo-developers/2007-April/009738.html with associated bug here: https://bugs.maemo.org/show_bug.cgi?id=989 the bug was marked WONTFIX Eero Tamminen's resolution was to not add any additional applets to BusyBox because in his opinion those needs can best be met by creating full versions of the tools in separate packages. I don't think this is a good idea because it creates a proliferation of unnecessarily bloated packages with the attendant problems of maintaining multiple packages (keeping in mind that the target hardware is a capacity constrained tablet). The benefit of busybox is that most appplets add just a few kb to the binary size and all of them sit inside a single binary. My proposal is to create an alternative essential busybox package that optionally replaces the default busybox, say busybox-max. busybox-max would be built with many more applets (more of the archive tools, more shell support (e.g. longer command histories and color coded ls), more networking tools etc. This package would not be installed by default, it would just be an optional package for developers/hackers. I don't know how well dpkg would handle the package replacement but it is worth exploring. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: busybox applet selection (again)
2008/1/2, Damien Moore [EMAIL PROTECTED]: @Clarence: I agree that dpkg should be able to handle this. Would REPLACE: busybox(##version##) be needed instead of PROVIDES? Because busybox is an essential package, you can't uninstall the existing one to reinstall the new one, which makes me suspect that the new one would need to REPLACE the old one. What would happen with an order like this?... # apt-get -o APT::Force-LoopBreak=1 install busybox replacement Maybe a broken system? ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
