[PATCH 1/3] Support both custom scripts and scripted applets
BusyBox has support for embedded shell scripts. Two types can be
distinguished: custom scripts and scripts implementing applets.
All embedded scripts can be run by name and are subject to tab
completion in standalone shell mode.
Custom scripts are otherwise unsupported by BusyBox and are intended
as a simple means for application-specific code to be included in
the binary. Such scripts should be placed in the 'embed' directory
at build time.
Applet scripts are integrated with the BusyBox build system and
are intended to be used to ship standard applets that just happen
to be implemented as scripts. They can be configured at build time
and appear just like native applets.
Such scripts should be placed in the 'applets_sh' directory. A stub
C program should be written to provide the usual applet configuration
details and placed in a suitable subsystem directory. It may be
helpful to have a configuration option to enable any dependencies the
script requires: see the 'nologin' applet for an example.
Of course, there's nothing stopping developers from including their
own custom scripts as applets if they so choose.
function old new delta
scripted_main - 53 +53
packed_usage 32951 32997 +46
.rodata 168516 168561 +45
applet_names26952703 +8
applet_main 31283136 +8
script_names 9 - -9
find_script_by_name 57 26 -31
run_applet_and_exit 782 728 -54
--
(add/remove: 1/1 grow/shrink: 4/2 up/down: 160/-94)Total: 66 bytes
Signed-off-by: Ron Yorston
---
.gitignore| 5 ++
applets/busybox.mkscripts | 16 ++
{embed => applets_sh}/nologin | 0
include/applets.src.h | 15 ++
include/libbb.h | 1 +
libbb/appletlib.c | 55 +++--
libbb/lineedit.c | 10 ++--
scripts/embedded_scripts | 91 ++-
shell/ash.c | 6 +++
util-linux/nologin.c | 27 +++
10 files changed, 182 insertions(+), 44 deletions(-)
create mode 100755 applets/busybox.mkscripts
rename {embed => applets_sh}/nologin (100%)
create mode 100644 util-linux/nologin.c
diff --git a/.gitignore b/.gitignore
index c03c2e8a6..becd9bf6d 100644
--- a/.gitignore
+++ b/.gitignore
@@ -56,3 +56,8 @@ cscope.po.out
#
tags
TAGS
+
+#
+# user-supplied scripts
+#
+/embed
diff --git a/applets/busybox.mkscripts b/applets/busybox.mkscripts
new file mode 100755
index 0..935685cba
--- /dev/null
+++ b/applets/busybox.mkscripts
@@ -0,0 +1,16 @@
+#!/bin/sh
+# Make busybox scripted applet list file.
+
+# input $1: full path to Config.h
+# input $2: full path to applets.h
+# output (stdout): list of pathnames that should be linked to busybox
+
+export LC_ALL=POSIX
+export LC_CTYPE=POSIX
+
+CONFIG_H=${1:-include/autoconf.h}
+APPLETS_H=${2:-include/applets.h}
+$HOSTCC -E -DMAKE_SCRIPTS -include $CONFIG_H $APPLETS_H |
+ awk '/^[ \t]*SCRIPT/{
+ print $2
+ }'
diff --git a/embed/nologin b/applets_sh/nologin
similarity index 100%
rename from embed/nologin
rename to applets_sh/nologin
diff --git a/include/applets.src.h b/include/applets.src.h
index 2ddf120ad..60968cec7 100644
--- a/include/applets.src.h
+++ b/include/applets.src.h
@@ -27,36 +27,49 @@ s - suid type:
# define APPLET_ODDNAME(name,main,l,s,help) int main##_main(int argc, char
**argv) MAIN_EXTERNALLY_VISIBLE;
# define APPLET_NOEXEC(name,main,l,s,help) int main##_main(int argc, char
**argv) MAIN_EXTERNALLY_VISIBLE;
# define APPLET_NOFORK(name,main,l,s,help) int main##_main(int argc, char
**argv) MAIN_EXTERNALLY_VISIBLE;
+# define APPLET_SCRIPTED(name,main,l,s,help)
#elif defined(NAME_MAIN)
# define APPLET(name,l,s)name name##_main
# define APPLET_ODDNAME(name,main,l,s,help) name main##_main
# define APPLET_NOEXEC(name,main,l,s,help) name main##_main
# define APPLET_NOFORK(name,main,l,s,help) name main##_main
+# define APPLET_SCRIPTED(name,main,l,s,help) name scripted_main
#elif defined(MAKE_USAGE) && ENABLE_FEATURE_VERBOSE_USAGE
# define APPLET(name,l,s)MAKE_USAGE(#name,
name##_trivial_usage name##_full_usage)
# define APPLET_ODDNAME(name,main,l,s,help) MAKE_USAGE(#name,
help##_trivial_usage help##_full_usage)
# define APPLET_NOEXEC(name,main,l,s,help) MAKE_USAGE(#name,
help##_trivial_usage help##_full_usage)
# define APPLET_NOFORK(name,main,l,s,help) MAKE_USAGE(#name,
help##_trivial_usage help##_full_usage)
+# define APPLET_SCRIPTED(name,main,l,s,help) MAKE_USA
Re: [PATCH 1/3] Support both custom scripts and scripted applets
Apologies for late reply. On Fri, Nov 9, 2018 at 12:50 PM Ron Yorston wrote: > BusyBox has support for embedded shell scripts. Two types can be > distinguished: custom scripts and scripts implementing applets. > > All embedded scripts can be run by name and are subject to tab > completion in standalone shell mode. > > Custom scripts are otherwise unsupported by BusyBox and are intended > as a simple means for application-specific code to be included in > the binary. Such scripts should be placed in the 'embed' directory > at build time. > > Applet scripts are integrated with the BusyBox build system and > are intended to be used to ship standard applets that just happen > to be implemented as scripts. They can be configured at build time > and appear just like native applets. Yes, conceptually there are two distinct scenarios as you described - applets which implement general utility function and "just happen to be written in shell", and custom scripts added by the people compiling the binary for their uses. But there is no technical distinction between them. Both are commands. I propose to treat both of these types scripts the same way. I have a feeling this will result in simpler code. Let me know if you foresee difficulties with this approach. ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH 1/3] Support both custom scripts and scripted applets
Denys Vlasenko wrote: >I propose to treat both of these types scripts the same way. > >I have a feeling this will result in simpler code. > >Let me know if you foresee difficulties with this approach. I suppose my concern is that it risks losing the distinctive features of the two types of script. My view is that 'scripted applets' should require configuration in the same way as native applets: - it should be possible to enable and disable them individually - they should be listed (in alphabetical order!) by 'busybox --help' and 'busybox --list' - they should be installed by 'busybox --install' - they should respond to 'busybox --help name' and 'name --help' The infrastructure for all of this is already present. Custom scripts on the other hand should require no configuration, apart from just dropping them in the 'embed' directory. If 'embed' is empty the code to support custom scripts won't be present in the binary. I'd prefer to maintain this distinction and use the same 'embedded scripts' machinery to support deployment of both types of script. But if all embedded scripts are to be treated alike I think the implementation should tend towards that of native applets, even if that raises the bar for people who want to deploy custom scripts. Ron ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH 1/3] Support both custom scripts and scripted applets
On 11/14/18 1:54 PM, Ron Yorston wrote: > Denys Vlasenko wrote: >> I propose to treat both of these types scripts the same way. >> >> I have a feeling this will result in simpler code. >> >> Let me know if you foresee difficulties with this approach. > > I suppose my concern is that it risks losing the distinctive features > of the two types of script. > > My view is that 'scripted applets' should require configuration in the > same way as native applets: > > - it should be possible to enable and disable them individually > - they should be listed (in alphabetical order!) by 'busybox --help' and > 'busybox --list' > - they should be installed by 'busybox --install' > - they should respond to 'busybox --help name' and 'name --help' > > The infrastructure for all of this is already present. > > Custom scripts on the other hand should require no configuration, apart > from just dropping them in the 'embed' directory. If 'embed' is empty > the code to support custom scripts won't be present in the binary. > > I'd prefer to maintain this distinction and use the same 'embedded > scripts' machinery to support deployment of both types of script. > > But if all embedded scripts are to be treated alike I think the > implementation should tend towards that of native applets, even if > that raises the bar for people who want to deploy custom scripts. What about not supporting "official" scripts at all? If they're exactly the same anyway except for being outright endorsed as a good idea, then just don't endorse them. Custom scripts seems like a useful use case. Scripted applets does not seem like a useful use case. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH 1/3] Support both custom scripts and scripted applets
On Wed, Nov 14, 2018 at 7:54 PM Ron Yorston wrote: > Denys Vlasenko wrote: > >I propose to treat both of these types scripts the same way. > > > >I have a feeling this will result in simpler code. > > > >Let me know if you foresee difficulties with this approach. > > I suppose my concern is that it risks losing the distinctive features > of the two types of script. > > My view is that 'scripted applets' should require configuration in the > same way as native applets: > > - it should be possible to enable and disable them individually > - they should be listed (in alphabetical order!) by 'busybox --help' and > 'busybox --list' > - they should be installed by 'busybox --install' > - they should respond to 'busybox --help name' and 'name --help' > > The infrastructure for all of this is already present. > > Custom scripts on the other hand should require no configuration, apart > from just dropping them in the 'embed' directory. If 'embed' is empty > the code to support custom scripts won't be present in the binary. > > I'd prefer to maintain this distinction and use the same 'embedded > scripts' machinery to support deployment of both types of script. > > But if all embedded scripts are to be treated alike I think the > implementation should tend towards that of native applets, even if > that raises the bar for people who want to deploy custom scripts. I basically agree. How about this? - embed/ applets can have configuration. If it exists, then they are visible in "make menuconfig" and can be selected or deselected. But if config is _absent_, they are included unconditionally. This seems to cover both cases. If you are ok with this, willing to cook a patch? ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH 1/3] Support both custom scripts and scripted applets
On Sat, Nov 17, 2018 at 6:56 AM Denys Vlasenko wrote: > I basically agree. > > How about this? - embed/ applets can have configuration. > If it exists, then they are visible in "make menuconfig" > and can be selected or deselected. > But if config is _absent_, they are included unconditionally. > > This seems to cover both cases. A compromise approach? That sounds good. However, I'm still in the position that BusyBox shouldn't be a supplier of scripted applets (as I said, that would complicate the configuration too much). If you downstream are okay to deploy scripted applets, then go ahead. I just don't wish dependency hell that's essentially bikeshedding (i.e. script applets too trivial to implement) to come up here. On Wed, Nov 14, 2018 at 7:54 PM Ron Yorston wrote: > My view is that 'scripted applets' should require configuration in the > same way as native applets: > > - it should be possible to enable and disable them individually > - they should be listed (in alphabetical order!) by 'busybox --help' and > 'busybox --list' > - they should be installed by 'busybox --install' > - they should respond to 'busybox --help name' and 'name --help' > > The infrastructure for all of this is already present. > > Custom scripts on the other hand should require no configuration, apart > from just dropping them in the 'embed' directory. If 'embed' is empty > the code to support custom scripts won't be present in the binary. > > I'd prefer to maintain this distinction and use the same 'embedded > scripts' machinery to support deployment of both types of script. > How about this: For the 'scripted applets' use case, you provide Kbuild and Config files just like other applets, except they just happen to reside in embed directory. For custom scripts, require at least a "list file" to indicate which should be built and embedded into the binary, and also in what order they would present in the 'busybox --help' output (they need not be in alphabetical order). Scripts in the embed directory but without any _directive_ on how they should be built would be ignored. ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH 1/3] Support both custom scripts and scripted applets
Denys Vlasenko wrote: >How about this? - embed/ applets can have configuration. >If it exists, then they are visible in "make menuconfig" >and can be selected or deselected. >But if config is _absent_, they are included unconditionally. > >This seems to cover both cases. So we have two types of script, those with and without configuration, but they all reside in the 'embed' directory? That seems to increase complexity at build-time without any change in complexity at run-time. And having all the scripts in the same place reduces the distinction between them (at build-time). >If you are ok with this, willing to cook a patch? I'm not really OK with it, for the above reasons, but I can have a go at making a patch anyway, just to see how horrible it looks ;-) >From your previous comments: >I propose to treat both of these types scripts the same way. >I have a feeling this will result in simpler code. I got the impression that your concern was the complexity of the run-time code. One way to reduce this is to treat custom scripts as if they were applets. This could be achieved by generating fake applet data for them at build-time. (Though, again, this will increase complexity at build-time.) The run-time code would then handle all embedded scripts as applets, thus requiring only one code path. The downside is that the end user would see all embedded scripts as applets. There would be no distinction between BusyBox-supplied scripted applets and custom scripts. Maybe I'll make a patch for that too. I still think the *approach* of the current patch best serves all parties: BusyBox developers, custom script developers and end users. The *implementation* may, of course, be subject to improvement. Ron ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH 1/3] Support both custom scripts and scripted applets
Kang-Che Sung wrote: >However, I'm still in the position that BusyBox shouldn't be a supplier of >scripted applets (as I said, that would complicate the configuration too much). >If you downstream are okay to deploy scripted applets, then go ahead. I just >don't wish dependency hell that's essentially bikeshedding (i.e. script applets >too trivial to implement) to come up here. Scripted applets provided by BusyBox will have been subject to review on this list and by the maintainers. We can trust Denys not to accept substandard scripts just as he does with native code. >For the 'scripted applets' use case, you provide Kbuild and Config files just >like other applets, except they just happen to reside in embed directory. Sounds like what Denys was proposing. To me it seems simpler to keep the different types of script in different directories. I don't see any advantage in putting them all in the same place. >For custom scripts, require at least a "list file" to indicate which should be >built and embedded into the binary, and also in what order they would present >in the 'busybox --help' output (they need not be in alphabetical order). > >Scripts in the embed directory but without any _directive_ on how they should >be built would be ignored. The developer controls what they put in the embed directory. The presence (or non-presence) of a script is sufficient to specify whether (or not) it should be embedded. I don't see the need for a list. Ron ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH 1/3] Support both custom scripts and scripted applets
OK, I've made a patch that uses the embed directory for custom scripts and scripted applets. We now have to track three kinds of script in embed: - custom scripts - enabled applet scripts - disabled applet scripts Since applets.h didn't include details of disabled applets I had to modify gen_build_files.sh to add the information required. The other two patches in the series can still be applied on top of this one. As I've already stated, I don't think it's a good idea to keep all the scripts in the same directory. Firstly because it loses the distinction between the two types of script and secondly because it adds build-time complexity without reducing run-time complexity. I still recommend '[PATCH 1/3 v2] Support both custom scripts and scripted applets' in preference to this new one. Ron ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH 1/3] Support both custom scripts and scripted applets
Right, I threatened to make another patch, and here it comes. This one goes back to the idea of keeping applet scripts and custom scripts in separate directories. Once again 'embed' is the place to put custom scripts and it isn't managed by BusyBox. Any scripts in 'embed' are given a fake applet configuration: - always enabled - no SUID, NOFORK, NOEXEC flags - install to /usr/bin - no help Since all embedded scripts are now treated as applets there's no need for special code to handle custom scripts: - libbb/lineedit.c doesn't need to tab-complete them - ash doesn't need to encode script numbers as applet numbers - all embedded scripts are accessed by applet number, not name This doesn't lead to much change in the size of the binary, though. The default build doesn't have any custom scripts so the code to support them isn't built. The size increase when a single custom script is added is reduced compared to [PATCH 1/3 v2], down by about 30 bytes. The second patch in the series isn't required. Since custom scripts now appear as applets there's no need for additional code to list them. That saves 152 bytes, according to [PATCH 2/3]. The third patch, to allow a profile script to be embedded, will need to be modified. I'll look into that some other time. I like this patch more than the earlier 'Support custom and applet scripts in embed directory'. It retains the distinction between custom and applet scripts at build-time. The build scripts and the code are both simpler. On the other hand it loses the distinction at run-time: all embedded scripts now look like applets. I haven't given this patch much testing. Be careful. Ron ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
