Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
On Wed, Mar 27, 2024 at 6:47 AM Rob Landley wrote: > > On 3/25/24 20:24, enh wrote: > > But "dpkg-query -S $(which $NAME)" is pretty easy to do the mapping > > yourself on > > debian... > > > > > > (yeah, though i suspect anyone trying to do this hypothetical "swap package > > $X > > for toybox" would want the _opposite_ mapping, from package name to all the > > commands. and i don't know of a way to ask apt that question? > > $ dpkg-query -L tar | grep bin/ > /bin/tar > /usr/sbin/rmt-tar > /usr/sbin/tarcat ah, interesting. > > other than > > brute-forcing all of the executables in all of the directories in $PATH, > > anyway.) > > Checking the $PATH would be clever but the above covers it for me. > > There are some insane packages which crap binaries under /usr/lib, such as > /usr/lib/libreoffice/program/oosplash or /usr/lib/man-db/manconv and > generally I > consider these packages to be maintained by madmen. > > I mean honestly: > > $ cat /usr/bin/7z > #! /bin/sh > exec /usr/lib/p7zip/7z "$@" > > Why would you do that? Why would ANYONE voluntarily DO that? weird. it's not even update-alternatives' fault, because that's just symlinks... > Rob ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
On 3/25/24 20:24, enh wrote: > But "dpkg-query -S $(which $NAME)" is pretty easy to do the mapping > yourself on > debian... > > > (yeah, though i suspect anyone trying to do this hypothetical "swap package $X > for toybox" would want the _opposite_ mapping, from package name to all the > commands. and i don't know of a way to ask apt that question? $ dpkg-query -L tar | grep bin/ /bin/tar /usr/sbin/rmt-tar /usr/sbin/tarcat > other than > brute-forcing all of the executables in all of the directories in $PATH, > anyway.) Checking the $PATH would be clever but the above covers it for me. There are some insane packages which crap binaries under /usr/lib, such as /usr/lib/libreoffice/program/oosplash or /usr/lib/man-db/manconv and generally I consider these packages to be maintained by madmen. I mean honestly: $ cat /usr/bin/7z #! /bin/sh exec /usr/lib/p7zip/7z "$@" Why would you do that? Why would ANYONE voluntarily DO that? Rob ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
On 3/25/24 20:20, enh wrote: > On Sun, Mar 24, 2024 at 12:45 AM Rob Landley wrote: > On 3/22/24 10:24, enh wrote: > > On Thu, Mar 21, 2024 at 8:45 PM Rob Landley wrote: > >> Anyway, toys/android basically meant (to me), "commands that come from > and are > >> maintained by Elliott which I can't even test because they don't apply > to a > >> vanilla linux system that isn't running the full android environment". > Although > >> that's a personally idiosyncratic definition because I lumped selinux > in with > >> that; > > > > (heh. you beat me to it :-) ) > > If the new kconfig greyed out unavailable entries and had a status line > saying > "depends on TOYBOX_ON_ANDROID" or similar when you cursored over a greyed > out > entry... > > ah, as the kind of lunatic who only ever edits these files by hand with vi, > i'd > actually just assumed that was kind of the whole point of the _existing_ > kconfig > stuff? To me half the point is it's the same UI as configuring the linux kernel, busybox, and buildroot. Meaning A) a bunch of people out there are familiar with it already, B) presumably the worst sharp edges have been filed off over the past 15 years. > (to be fair, i did launch it once, but saw it was a ridiculously deeply > nested > ui [and not expanded by default?], and thought "i don't understand the purpose > of this", couldn't see how to search, It literally has help text at the top of the screen. Forward slash is search, cursor up and down, space to toggle the highlighted thingy, enter to go into a menu, ESC to back out again, ESC from the top level to exit (it prompts you whether or not you want to save), ESC twice from _that_ to abort the exit. There's also a menu at the bottom, where if you cursor left and right it highlights different things, and the ENTER will do that thing instead. (The default is "select". I cursor right to "help" and hit enter because I never remember that ? is the hotkey for that.) Mostly I'm assuming "same UI as linux kernel" is like 2/3 of the userbase though. > and immediately went back to editing by > hand. at least that way i only need to know how to use my editor, which i need > to know regardless :-) ) Dependency resolution comes to mind. > If we really wanted to rush this, I could make a TOYBOX_UNFINISHED symbol > that > the pending stuff could depend on, and then the blocker is the kconfig > replacement... > > no, i've been cursing the broken tab-complete for -- wow, almost a decade now! > -- so i think i can survive :-) I admit I sometimes do "ls toys/*/skel* when I can't remember whether I called it "example" or "examples". > Not THIS release though. Working on release notes! (And lowering my > standards on > the todo list.) > > indeed... something that benefits the handful of folks working on toybox isn't > worth much compared to something that benefits the users! Working on it... Rob ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
On 3/24/24 10:15, Oliver Webb wrote:
> On Sunday, March 24th, 2024 at 04:09, Rob Landley wrote:
>> On 3/24/24 01:00, Oliver Webb wrote:
>
>> This isn't the hard part. To me, the hard part is wanting to share lib/.c
>> code
>> with this new binary, which implies it would live in toys/example/.c, which
>> means in the NEW design it would be a normal command that's "default n"...
>> and
>> maybe depends on TOYBOX_BUILD or some such? Except moving stuff from
>> scripts/.c
>> to toys/.c is conceptually ugly. But if we're getting rid of the
>> subdirectories... Maybe make.sh needs to be able to build commands that DON'T
>> live in toys/ but then...
>
> There is a chicken and egg problem with the build infrastructure and kconfig
> being a toy,
Yes, I know. That's why I've avoided it up until now.
> We need a .config file to build toys, and parsing the help text requires some
> kconfig
> parser, But we can't make a .config file until we have kconfig.
You don't need a .config file to build lib/*.c (policy, and why lib/lib.h is
separate from toys.h).
I'm talked before about doing packaged minimal headers to build "sed" and "sh"
standalone, as part of toybox airlock stuff. (Possibly the full airlock command
list needed to build toybox.) Ones that assume all the config probes failed and
$LIBRARIES is empty and so on.
The EASY way to do that is to have a scripts/shipped/generated with handcrafted
headers files, and then stick -I scripts/shipped at the start of $CFLAGS.
The hard way involves more cleanup so there are fewer header entry points, and I
could have a single "hairball.h".
The individual toys/*/*.c files only #include toys.h and generated/flags.h
(which could be a re-include of toys.h with a little #ifdef cleverness). The
rest are all included from toys.h and main.c.
The above #ifdef cleverness could wrap the generated/ includes in toys.h in a
__has_include("hairball.h") or similar, so I could provide a single replacement
file with the collated stuff I need for specific commands to build in a way that
assumes the host system has no brain, and then generated/build.sh it. The
problem is the includes are in two places: "generated/config.h" comes before
lib/portability.h (which comes before everything else so it can override
standard header #includes), and then the rest of the generated/*.h are after a
#define NEWTOY() and OLDTOY() so there's some reordering to do to combine them
all into one header.
(I don't think any of the generated/*.h files care about stuff in portability.h?
There would be various structs used before they were defined in
generated/globals.h if it got moved up, but that _should_ be ok? Nothing takes
the sizeof() them or similar that early. Eh, I should be able to work it out,
just haven't sat down to try yet. Far too many already open cans of worms...)
And then there's the #includes in main.c, the other half of the
declaration/definition pair for various global data: that needs newtoys.h,
help.h, and zhelp.h. One of which is already chopped out by a config option, the
second of which just needs a way to stub it to "", and which leaves newtoy.h to
address...
> The solution I thought of was to use the infrastructure that we will have to
> have to remove
> bash and gsed dependencies to build kconfig as a early step in the process.
No.
> But then we will
> still need to extract the help text.
config TOYBOX_HELP
bool "Help messages"
default y
help
Include help text for each command.
You can configure help out entirely. (This isn't CONFIG_HELP the command, this
is "the help subsystem" in the toybox general settings menu.) This was
intentional.
> Do you plan on not keeping 2 different kconfig parsers or moving scripts/*.c
> to toys/example
Look up at the first paragraph of mine you quoted in this email.
It's an open question, but stripping down a "cc -I scripts/prebuilts main.c
lib/*.c toys/*/{abc,def,ghi,jkl}.c" build so it could provide commands with
nothing but a compiler would be a step towards that.
Modulo that "cc *.c" doesn't parallelize across processors because C++
developers took over compiler development about 2 years after the Core Duo hit
the market and brought SMP to the cheap retail mainstream, at which point making
compilers better rather than merely more complicated hit a sudden brick wall.
And thus even on my ancient 4x laptop:
$ time make clean defconfig toybox
...
real0m16.170s
$ time generated/build.sh
...
real0m27.474s
I don't want to significatly slow down the build by compiling prerequisites? In
theory:
$ time gcc -I . main.c lib/*.c -o blah
...
real0m1.780s
(Yeah exits with a link error but that's not the point.)
And I mean yeah, 2 seconds, not that big a deal. But I'd prefer it be
optional...
> (We could
> make toys/build tho, which would make slightly more sense),
See the recent discussion of removing the toys/subdirectories once pending's
cleared out, rather than adding more of them.
>
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
On Sun, Mar 24, 2024 at 1:12 AM Rob Landley wrote: > On 3/22/24 10:26, enh wrote: > > On Fri, Mar 22, 2024 at 8:24 AM enh wrote: > >> (tbh, just merging "lsb" into "other" would be a step forwards. wtf > >> is/was "lsb" anyway? and while i can _usually_ guess "POSIX or not?" > >> correctly, "lsb or other" is impossible by virtue of being > >> meaningless.) > > > > (and to be clear, although "lsb" is particularly obscure, i think this > > is the same problem busybox's organization has: why do i have to care > > whether something is in coreutils or linux-utils or procps? how is > > that relevant to me? > > There's a reason I didn't use that as an organizing method. Although I did > try > to map them at the end of the roadmap, and need to redo that analysis now > since > it's been a while... > > > the best answer i can think of is "because i want > > to only use toybox/busybox to replace _that_ package", but i don't > > think the _directory structure_ helps there, right? that hypothetical > > person actually wants more metadata in the kconfig part of the comment > > inside each file?) > > That's the theoretical use, yes. So distros (and system builders like > gentoo, > buildroot, yocto, etc) can annotate package alternatives so if you want to > install busybox's tar instead of gnu tar your package management system > could > cope. yeah, "equivalent package" might be another field to add to your kconfig replacement --- then folks could configure toybox as "coreutils" or "procps" or both or whatever. (but, like i say, _i've_ never used this, and can't really imagine why i'd not just want "all the things", so...) > In practice, making something like dpkg handle that was near impossible, > and buildroot only did it because the maintainer of busybox created > buildroot. I > tried to add toybox to buildroot years ago and... > > https://lists.buildroot.org/pipermail/buildroot/2014-September/409298.html > > People still try from time to time: > > https://lists.buildroot.org/pipermail/buildroot/2017-January/181960.html > http://lists.busybox.net/pipermail/buildroot/2022-September/652474.html > > But even a build system that ALREADY lets you swap in/out buildroot vs gnu > versions of packages accomplished that by hardwiring busybox support deep > into > its build system. > > Getting something like debian to do that on the fly is... it's not really > designed for it. > > I can think of better ways to do it (and am studying debian's build system > in my > copious free time), but I've been busy with other things and most people > aren't > motivated to try... > > I note that I did it by hand back when creating aboriginal linux, which is > what > led me to maintaining busybox in the first place, ala: > > https://landley.net/aboriginal/old/ > > > When the Firmware Linux project started, busybox applets like sed and > sort > > weren't powerful enough to handle the "./configure; make; make install" > of > > packages like binutils or gcc. Busybox was usable in an embedded router > or > > rescue floppy, but trying to get real work done with it revealed numerous > > bugs and limitations. > > > > Busybox has now been fixed, and in Firmware Linux Busybox functions as an > > effective replacement for bzip2, coreutils, e2fsprogs, file, findutils, > gawk, > > grep, inetutils, less, modutils, net-tools, patch, procps, sed, shadow, > > sysklogd, sysvinit, tar, util-linux, and vim. (Eventually, it should be > > capable of replacing bash and diffutils as well, but it's not there yet.) > > That's the old page from before I restarted the project and renamed it > Aboriginal Linux (based on QEMU instead of User Mode Linux, ala > https://landley.net/notes-2005.html#27-10-2005). Before that I was going > though > the Linux From Scratch package list and _disposing_ of gnu packages, one > by one, > as I got busybox to replace them. > > But "dpkg-query -S $(which $NAME)" is pretty easy to do the mapping > yourself on > debian... > (yeah, though i suspect anyone trying to do this hypothetical "swap package $X for toybox" would want the _opposite_ mapping, from package name to all the commands. and i don't know of a way to ask apt that question? other than brute-forcing all of the executables in all of the directories in $PATH, anyway.) > Rob > ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
On Sun, Mar 24, 2024 at 12:45 AM Rob Landley wrote: > On 3/22/24 10:24, enh wrote: > > On Thu, Mar 21, 2024 at 8:45 PM Rob Landley wrote: > >> Anyway, toys/android basically meant (to me), "commands that come from > and are > >> maintained by Elliott which I can't even test because they don't apply > to a > >> vanilla linux system that isn't running the full android environment". > Although > >> that's a personally idiosyncratic definition because I lumped selinux > in with > >> that; > > > > (heh. you beat me to it :-) ) > > If the new kconfig greyed out unavailable entries and had a status line > saying > "depends on TOYBOX_ON_ANDROID" or similar when you cursored over a greyed > out > entry... > ah, as the kind of lunatic who only ever edits these files by hand with vi, i'd actually just assumed that was kind of the whole point of the _existing_ kconfig stuff? (to be fair, i did launch it once, but saw it was a ridiculously deeply nested ui [and not expanded by default?], and thought "i don't understand the purpose of this", couldn't see how to search, and immediately went back to editing by hand. at least that way i only need to know how to use my editor, which i need to know regardless :-) ) > There _is_ a way to collapse everything together into one directory and > make it > manage-ish-able. But there are currently 52 command files in pending, and > "ip.c" > alone is 6 commands and 3000 lines of "we already have route and ifconfig > and > iptables and so on as separate commands, why did they do it again?" > > >> It's been the status quo for a dozen years now (commit 3a9241add947 in > 2012) and > >> moving everything AGAIN would have costs, so I'd want a reason and > assurance > >> that we're not going to change our minds again. > > > > for me the holy grail is "tab complete works and i don't have to think > > about arbitrary partitions". > > It's a good point. > > > i think "not yet default 'y'" is pretty > > defensible (though the reason we're having this discussion is because > > people _don't_ read "pending" as "danger, keep out!"), but the rest > > seem so arbitrary. > > I'd like there to not BE "danger, keep out" in the tree, but a certain > large > korean company wanted their contributions checked in, I fell behind, and it > snowballed from there. > > >> Collapsing the directories > >> together when the last command is promoted (or deleted) out of pending > might > >> make sense, figuring out what to do about example/ (trusting to the > demo_ prefix > >> to annotate the example commands is nice, but hello.c hostid.c > logpath.c and > >> skeleton.c would need... something). > > > > no, i think example/ is defensible too. (i'd argue you're only ever > > going to look in there if you have a _reason_ to. or you've done a > > `grep -r` for something you're changing/checking all references to. > > the reason i completely forgot about example/ is that it never causes > > me the "where the fuck is _mount_?!" annoyance :-) ) > > Right now everything is at the same level. Having files at two different > levels > is not a simplification. > > Designing a way to have toys/*.c with no subdirectories and make it > manageable > seems a reasonable goal, if tricky to get to. Having toys/*.c _and > toys/*/*.c > does not smell like an improvement? > > We've got: android example lsb net other pending posix > > Pending needs everything cleaned up and prompted or deleted. Posix can be a > defconfig file. Example can be commands that "default n". Android isn't > necessary if a kconfig replacement greys things out instead of hiding them > and > displays WHY they're greyed out when you cursor over them (and the rewrite > is > needed to address pull request 332). Other, net, and lsb aren't sufficient > distinction to persist in the absence of other directories. > > And that's all of them, I think? > > If we really wanted to rush this, I could make a TOYBOX_UNFINISHED symbol > that > the pending stuff could depend on, and then the blocker is the kconfig > replacement... > no, i've been cursing the broken tab-complete for -- wow, almost a decade now! -- so i think i can survive :-) > Not THIS release though. Working on release notes! (And lowering my > standards on > the todo list.) > indeed... something that benefits the handful of folks working on toybox isn't worth much compared to something that benefits the users! > Rob > ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
On Sunday, March 24th, 2024 at 04:09, Rob Landley wrote: > On 3/24/24 01:00, Oliver Webb wrote: > This isn't the hard part. To me, the hard part is wanting to share lib/.c code > with this new binary, which implies it would live in toys/example/.c, which > means in the NEW design it would be a normal command that's "default n"... and > maybe depends on TOYBOX_BUILD or some such? Except moving stuff from > scripts/.c > to toys/.c is conceptually ugly. But if we're getting rid of the > subdirectories... Maybe make.sh needs to be able to build commands that DON'T > live in toys/ but then... There is a chicken and egg problem with the build infrastructure and kconfig being a toy, We need a .config file to build toys, and parsing the help text requires some kconfig parser, But we can't make a .config file until we have kconfig. The solution I thought of was to use the infrastructure that we will have to have to remove bash and gsed dependencies to build kconfig as a early step in the process. But then we will still need to extract the help text. Which implies we'd have to keep config2help.c, we'd have 2 seperate kconfig parsers, so maybe the solution is to build more upon config2help.c, we could "#include toys.h" (Although that includes generated/config.h, so "#ifndef TOYBOX_BUILD" and -D it in the CFLAGS for building this) and build with lib/*, Then again... Do you plan on not keeping 2 different kconfig parsers or moving scripts/*.c to toys/example (We could make toys/build tho, which would make slightly more sense), There's circular dependency nightmares that would HAVE to be worked around at one point or another, a lot of stuff in the infrastructure relies on kconfig so making kconfig rely on the infrastructure would require bootstrapping it in a weird way, but there are also problems with that (main.c includes generated/*help.h files, toys.h includes generated/config.h), So if the actual solution is to not make it a toy and build it with lib/* and toys.h to codeshare with lib/, that creates problems too... Does any of lib/ depend on those functions in toys.h that live in main, grepping through, of course there are parts, which means more ifdefs if we wanna pry it off the infrastructure, sigh. - Oliver Webb ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
On 3/24/24 01:00, Oliver Webb wrote: > I've done some research on this too, we have no "select" statements in any of > our config symbols, for a definition of "we" that is "I have intentionally not merged any", since I review and approve all the kconfig command sections in the headers and have been tracking that. (At one point the config2help.c stuff was trying to stitch dependencies together to merge help text, and didn't understand complicated syntax.) That said, forking the kconfig language definition is not something I do lightly. Ours has fallen way behind the kernel's, and thus looks like something else but is only compatible with a subset of it. We are about to _shrink_ that subset. This needs a FAQ entry at least. > but we do have a fair amount of that ""SYMBOL && (SYMBOL||SYMBOL)"" > expression processing that's > annoying to deal with. I was referring to that, yes. I need to implement processing for it. I've already implemented such processing in find, test, and twice in toysh (both command && command and $((math&)) ). > Also a "choice" block and a few number ranges in the main Config.in we will > need to deal with in some way, the depends/selects stuff seems easy but with > that expr evaluating probably isn't Yes, I know. > I tried to write a kconfig parser (As a toy to make the codesharing easier) I've written at a bunch, and mostly thrown them away again. There's a simple one in scripts/config2help.c and wrote one in python at https://landley.net/hg/kdocs/file/tip/make/menuconfig2html.py which generated https://landley.net/kdocs/menuconfig/ way back when. (Those are the only two published ones that come to mind, but I've written more over the years.) > and got absolutely nowhere. The approach I took to it was... This isn't the hard part. To me, the hard part is wanting to share lib/*.c code with this new binary, which implies it would live in toys/example/*.c, which means in the NEW design it would be a normal command that's "default n"... and maybe depends on TOYBOX_BUILD or some such? Except moving stuff from scripts/*.c to toys/*.c is conceptually ugly. But if we're getting rid of the subdirectories... Maybe make.sh needs to be able to build commands that DON'T live in toys/ but then... Unanswered design questions looming here, have not been jigsawed into an elegant picture yet. (How much of that is assembling pieces and how much is SAWING THEM UP I don't know yet...) Anyway, it seems like config2help.c should also share this plumbing if it's parsing the kconfig input anyway, which is convenient since I've been meaning to rewrite all that too (and yes THAT has a motivating "somebody is waiting for me to fix this", ala https://github.com/landley/toybox/issues/458 ), but there's also the usage: line regularization (https://landley.net/notes-2023.html#06-11-2023) and fixing the remaining sub-options with maybe some sort of help text include syntax for inserting other help texts at controllable points (as either blogged about or mentioned here on the list, I'd have to check my notes to see where I left off on that)... Once I've got the design worked out, coding it is usually the easy part. Rob ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
On 3/22/24 10:26, enh wrote: > On Fri, Mar 22, 2024 at 8:24 AM enh wrote: >> (tbh, just merging "lsb" into "other" would be a step forwards. wtf >> is/was "lsb" anyway? and while i can _usually_ guess "POSIX or not?" >> correctly, "lsb or other" is impossible by virtue of being >> meaningless.) > > (and to be clear, although "lsb" is particularly obscure, i think this > is the same problem busybox's organization has: why do i have to care > whether something is in coreutils or linux-utils or procps? how is > that relevant to me? There's a reason I didn't use that as an organizing method. Although I did try to map them at the end of the roadmap, and need to redo that analysis now since it's been a while... > the best answer i can think of is "because i want > to only use toybox/busybox to replace _that_ package", but i don't > think the _directory structure_ helps there, right? that hypothetical > person actually wants more metadata in the kconfig part of the comment > inside each file?) That's the theoretical use, yes. So distros (and system builders like gentoo, buildroot, yocto, etc) can annotate package alternatives so if you want to install busybox's tar instead of gnu tar your package management system could cope. In practice, making something like dpkg handle that was near impossible, and buildroot only did it because the maintainer of busybox created buildroot. I tried to add toybox to buildroot years ago and... https://lists.buildroot.org/pipermail/buildroot/2014-September/409298.html People still try from time to time: https://lists.buildroot.org/pipermail/buildroot/2017-January/181960.html http://lists.busybox.net/pipermail/buildroot/2022-September/652474.html But even a build system that ALREADY lets you swap in/out buildroot vs gnu versions of packages accomplished that by hardwiring busybox support deep into its build system. Getting something like debian to do that on the fly is... it's not really designed for it. I can think of better ways to do it (and am studying debian's build system in my copious free time), but I've been busy with other things and most people aren't motivated to try... I note that I did it by hand back when creating aboriginal linux, which is what led me to maintaining busybox in the first place, ala: https://landley.net/aboriginal/old/ > When the Firmware Linux project started, busybox applets like sed and sort > weren't powerful enough to handle the "./configure; make; make install" of > packages like binutils or gcc. Busybox was usable in an embedded router or > rescue floppy, but trying to get real work done with it revealed numerous > bugs and limitations. > > Busybox has now been fixed, and in Firmware Linux Busybox functions as an > effective replacement for bzip2, coreutils, e2fsprogs, file, findutils, gawk, > grep, inetutils, less, modutils, net-tools, patch, procps, sed, shadow, > sysklogd, sysvinit, tar, util-linux, and vim. (Eventually, it should be > capable of replacing bash and diffutils as well, but it's not there yet.) That's the old page from before I restarted the project and renamed it Aboriginal Linux (based on QEMU instead of User Mode Linux, ala https://landley.net/notes-2005.html#27-10-2005). Before that I was going though the Linux From Scratch package list and _disposing_ of gnu packages, one by one, as I got busybox to replace them. But "dpkg-query -S $(which $NAME)" is pretty easy to do the mapping yourself on debian... Rob ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
On 3/22/24 10:24, enh wrote: > On Thu, Mar 21, 2024 at 8:45 PM Rob Landley wrote: >> Anyway, toys/android basically meant (to me), "commands that come from and >> are >> maintained by Elliott which I can't even test because they don't apply to a >> vanilla linux system that isn't running the full android environment". >> Although >> that's a personally idiosyncratic definition because I lumped selinux in with >> that; > > (heh. you beat me to it :-) ) If the new kconfig greyed out unavailable entries and had a status line saying "depends on TOYBOX_ON_ANDROID" or similar when you cursored over a greyed out entry... There _is_ a way to collapse everything together into one directory and make it manage-ish-able. But there are currently 52 command files in pending, and "ip.c" alone is 6 commands and 3000 lines of "we already have route and ifconfig and iptables and so on as separate commands, why did they do it again?" >> It's been the status quo for a dozen years now (commit 3a9241add947 in 2012) >> and >> moving everything AGAIN would have costs, so I'd want a reason and assurance >> that we're not going to change our minds again. > > for me the holy grail is "tab complete works and i don't have to think > about arbitrary partitions". It's a good point. > i think "not yet default 'y'" is pretty > defensible (though the reason we're having this discussion is because > people _don't_ read "pending" as "danger, keep out!"), but the rest > seem so arbitrary. I'd like there to not BE "danger, keep out" in the tree, but a certain large korean company wanted their contributions checked in, I fell behind, and it snowballed from there. >> Collapsing the directories >> together when the last command is promoted (or deleted) out of pending might >> make sense, figuring out what to do about example/ (trusting to the demo_ >> prefix >> to annotate the example commands is nice, but hello.c hostid.c logpath.c and >> skeleton.c would need... something). > > no, i think example/ is defensible too. (i'd argue you're only ever > going to look in there if you have a _reason_ to. or you've done a > `grep -r` for something you're changing/checking all references to. > the reason i completely forgot about example/ is that it never causes > me the "where the fuck is _mount_?!" annoyance :-) ) Right now everything is at the same level. Having files at two different levels is not a simplification. Designing a way to have toys/*.c with no subdirectories and make it manageable seems a reasonable goal, if tricky to get to. Having toys/*.c _and toys/*/*.c does not smell like an improvement? We've got: android example lsb net other pending posix Pending needs everything cleaned up and prompted or deleted. Posix can be a defconfig file. Example can be commands that "default n". Android isn't necessary if a kconfig replacement greys things out instead of hiding them and displays WHY they're greyed out when you cursor over them (and the rewrite is needed to address pull request 332). Other, net, and lsb aren't sufficient distinction to persist in the absence of other directories. And that's all of them, I think? If we really wanted to rush this, I could make a TOYBOX_UNFINISHED symbol that the pending stuff could depend on, and then the blocker is the kconfig replacement... Not THIS release though. Working on release notes! (And lowering my standards on the todo list.) Rob ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
On Saturday, March 23rd, 2024 at 20:41, Rob Landley wrote: > On 3/21/24 23:59, Oliver Webb wrote: > > On Thursday, March 21st, 2024 at 22:45, Rob Landley r...@landley.net wrote: > > > On 3/17/24 14:52, Oliver Webb wrote: > > > > > > > Same here, I can remember the posix commands. > > > > > > Can you? I still have to check some from time to time, and the definition > > > of > > > whether "tar" is a posix command or not is outright eldrich bordering on > > > quantum. > > > > I can certainly remember them better then the LSB commands. Most of the > > time I can > > remember if a command is in posix, which is what matters when trying to > > find it usually. > > Congratulations? Me (In response to enh): "Same here, I can remember the posix commands [...]" you: "Can You?" Me: [Response that boils down to "yes"] you: "Congratulations?" (With a "How is this relevant" question-mark because I answered a question you asked) ... > Bikeshedding is vaguely related to the Dunning-Kruger effect, in which the > question "how hard can it be?" requiring some expertise to actually answer > gets > people in trouble. [...] > I think we have a different definition of the term. I always thought it was "Technical discussion that gets in the way of more important things" > And partly that the effort put into writing up plans for external consumption > is > NOT effort put into implementing those plans. (Ala "here's how I could > simplify > the kconfig design so a rewrite is less work, although that still doesn't > entirely address depends/selects resolution with "SYMBOL && > (SYMBOL||SYMBOL)". I > blog, and go into quick stream of consciousness asides on the mailing list, > but that's not remotely complete coverage.) I've done some research on this too, we have no "select" statements in any of our config symbols, but we do have a fair amount of that ""SYMBOL && (SYMBOL||SYMBOL)"" expression processing that's annoying to deal with. Also a "choice" block and a few number ranges in the main Config.in we will need to deal with in some way, the depends/selects stuff seems easy but with that expr evaluating probably isn't I tried to write a kconfig parser (As a toy to make the codesharing easier) and got absolutely nowhere. The approach I took to it was storing the entire menu structure as a tree of linked lists, each having a "type" (Menu, comment, selection, etc), A parent item, and a pointer to the next and previous item. The entire tree being stored in one root item who's parent is itself. After parsing the Config.in and generating this. (This is the point where I gave up) The screen drawing routine navigates this and lets you go back/forward/up/down and change config symbols, storing the result in a hash table or array before generating a .config (Idea: Maybe we could generate a C header file as a option instead of a .config file that has to be parsed.) What approach do you plan to take with the data structures? they matter a _lot_ less on oldconfig from my knowledge, but menu navigation is convenient when you have 250 items to go through > Sigh, lemme hit the old damaged .flv file with ffmpeg to convert it to mp4, > upload that, and here's the outline I gave the talk from: > > https://landley.net/talks/prototypeandfanclub.mp4 > https://landley.net/talks/flourish-2010.txt Just wget'ed both things, will watch and read through when I find the time. Thanks [I cut the summary of your talk out, but I read it, interesting stuff even tho I don't have anything to respond to it with, thank you] ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free > FYI lsb
On 3/22/24 18:09, scsijon wrote: > Date: Fri, 22 Mar 2024 08:24:18 -0700 > >> From: enh >> To: Rob Landley >> Cc: Oliver Webb , toybox >> >> Subject: Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free >> Message-ID: >> >> Content-Type: text/plain; charset="UTF-8" >> >> On Thu, Mar 21, 2024 at 8:45?PM Rob Landley wrote: >>> On 3/17/24 14:52, Oliver Webb wrote: >>>> On Thursday, March 14th, 2024 at 12:04, enh wrote: >>>>> at a high level, it does seem like many/most people interpret "pending" >>>>> as "almost done" (he says, being part of the problem himself, having >>>>> several pending things building and shipping on all Android devices) >>>>> whereas in actual fact it can mean anything from "yeah, actually pretty >>>>> much done" to "will be completely rewritten" via "still just trying >>>>> random experiments trying to work out _how_ this should be rewritten". >>>>> sadly i don't have a better suggestion... >>>> pending/experimental and pending/functional maybe, or something along that >>>> gist? >>> That would be my "not adding more complexity to manage transient clutter >>> that >>> should instead go away" objection, already made. >>> >>>> Then again it'd make it harder to track the history of pending commands, >>>> adding only new ones >>>> to those 2 directories would fix that, but would make the organizational >>>> problem for the old >>>> ones worse. >>> https://en.wikipedia.org/wiki/Fundamental_theorem_of_software_engineering >>> >>> Stop. No. Halt. Wait. Hold it. Woah. Cease. Desist. Caution severe tire >>> damage. >>> Klatu barata nikto. Subcalifragilisticexpialidocious. >>> >>>>> a branch would be the usual git option, but that would probably mean "no >>>>> pending stuff in the main branch" >>>> Also a problem if you want to switch Version Control systems or distribute >>>> tarballs without a .git/ directory. >>> I already DID switch version control systems (from mercurial to git), and I >>> already distribute release tarballs. Why do you think these are new issues? >>> >>>> It'd hide these commands too, >>> I want to close tabs. I am not creating additional scaffolding for clutter >>> management: >>> >>> $ ls -d */toys >>> clean3/toys clean8/toys github/toys kl4/toys kl9/toys >>> toybox/toys >>> clean5/toys clean.old/toys kl10/toyskl6/toys kleen/toys >>> clean6/toys clean/toys kl2/toys kl7/toys kl/toys >>> clean7/toys debian/toys kl3/toys kl8/toys release/toys >>> >>> I already try not to publish quite as much clutter as accumulates locally. >>> >>> There's some real fossils checked into the tree. I started work on gene2fs >>> back >>> under busybox, checked in what I had to the toybox repo in 055cfcbe5b05 in >>> 2007 >>> and haven't LOOKED at it this decade because I just haven't gotten back >>> around >>> to it. Since then they INVENTED EXT4. (I still hope to get back to it, but >>> at >>> the moment I'm answering email.) >>> >>>> For the first time I checked if there were any special branches in the >>>> repo because >>>> I didn't bother to think about that in the months I spent working on it. >>>> >>>>> i still struggle between "other" and "lsb" in particular. >>>> Same here, I can remember the posix commands. >>> Can you? I still have to check some from time to time, and the definition of >>> whether "tar" is a posix command or not is outright eldrich bordering on >>> quantum. >>> >>>> But I don't care about LSB enough to >>>> memorize everything in wants. And keeping all completed commands that >>>> aren't in poisx, >>>> lsb, networking or android >>> The "example" directory is important because it's the only other directory >>> of >>> commands that should not "default y" in defconfig. It has a policy >>> distinction. >>> >>> Back in 2012, when the number of commands was growing fast and having one >>> big >>> directory of them all was getting a bit busy, the alternative of sorting >>> them &g
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
On 3/21/24 23:59, Oliver Webb wrote: > On Thursday, March 21st, 2024 at 22:45, Rob Landley wrote: >> On 3/17/24 14:52, Oliver Webb wrote: >> > Same here, I can remember the posix commands. >> >> Can you? I still have to check some from time to time, and the definition of >> whether "tar" is a posix command or not is outright eldrich bordering on >> quantum. > > I can certainly remember them better then the LSB commands. Most of the time > I can > remember if a command is in posix, which is what matters when trying to find > it usually. Congratulations? >> Collapsing the directories together when the last command is >> promoted (or deleted) out of pending might make sense, > > What would happen when a new command shows up and we need to evaluate it then? Presumably once caught up there wouldn't usually be a dozen of them submitted the same month, so I wouldn't fall far enough behind to need a dedicated waiting room. > Or glibc does a new release and yet another thing breaks we need to demote and > re-promote eventually? I don't de-promote commands because glibc does something stupid each new release. That's just normal gnu/braindamage: https://github.com/landley/toybox/issues/450 https://github.com/landley/toybox/pull/364 https://github.com/landley/toybox/issues/362 I de-promoted a command since last release because I rewrite lib/password.c in a way that broke stuff and didn't want people poking me about it, which was me being lazy/whelmed. Not having the option to do that is fine too, and would have made that stay higher on the todo list. (I could also have "default n" it without moving it, I do that locally all the time when in-progress changes break stuff. The difference this time was I'd checked IN the stuff that broke a command, and didn't want to revert it.) >> I also note I think I've figured out how to replace kconfig: I can just make >> a >> list that scrolls up and down with a highlighted entry you hit space on, >> handle >> help text, search, exit/save, resolve selects and depends and have "menus" >> be a >> label line with its contents nested two spaces further to the right. > > [Some paragraphs bikeshedding about kconfig use to be here, may they rest in > a text file > until we get around to doing the kconfig rewrite] Technically a project's maintainer explaining upcoming design issues he actually plans to implement isn't "bikeshedding". Bikeshedding is vaguely related to the Dunning-Kruger effect, in which the question "how hard can it be?" requiring some expertise to actually answer gets people in trouble. Cyril Parkinson is mostly known for Parkinson's Law (work expands to fill available time) but he also came up with the bike shed example, where a committee approving plans for a nuclear reactor defers to the experts enough that at least its budget approval gets discussed quickly, but a committee approving plans for a bike shed will argue far longer about every detail because they think they could do it themselves and have strongly held opinions. Everybody has an opinion on building the bike shed, and thinks their opinion is equally valid as everyone else's with no deference to authority, experience, or expertise. But the thing about a committee approving plans is they STARTED with a viable plan for the thing, which they then ignore because they know better. If you feel like I'm "bikeshedding" about a kconfig replacement when I was involved in https://lkml.indiana.edu/hypermail/linux/kernel/0202.1/2037.html and argued at length with Roman Zippel about https://lwn.net/Articles/160497/ and dug rather a lot through busybox's fork of it back around https://git.busybox.net/busybox/log/scripts/config?id=7a43bd07e64e and already implemented scroll up/down/left right list logic like I'm describing in the "top" command... I think we have a different definition of the term. >> > A possible solution is to... >> >> ... >> >> > Then again... >> >> I need to stop checking email every time I sit down at my laptop, because >> bikeshedding can eat an endless amount of time and I've got other stuff to >> do. >> >> For one thing, I promised to look at >> https://github.com/landley/toybox/issues/486 tonight. > > Sorry for getting in the way of that, the technical discussion about it was > interesting enough to me to respond to. Recently found something to run off to > and do while still benefiting toybox, so I'll stop bikeshedding about stuff > like this. I'm complaining about my own insufficient time management skills, I'm not trying to discourage people from taking an interest in the project. I do find "why is it like this" easier to deal with than "let me tell you what you did wrong, and/or provide the obvious solution I'm sure you never thought about over the past 10 years, without needing to know the backstory". You can't get up to speed without learning stuff, and I should do MORE support for that. (Talks, youtube videos, design.html and faq.html and so on...) To be
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free > FYI lsb
Date: Fri, 22 Mar 2024 08:24:18 -0700 From: enh To: Rob Landley Cc: Oliver Webb , toybox Subject: Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free Message-ID: Content-Type: text/plain; charset="UTF-8" On Thu, Mar 21, 2024 at 8:45?PM Rob Landley wrote: On 3/17/24 14:52, Oliver Webb wrote: On Thursday, March 14th, 2024 at 12:04, enh wrote: at a high level, it does seem like many/most people interpret "pending" as "almost done" (he says, being part of the problem himself, having several pending things building and shipping on all Android devices) whereas in actual fact it can mean anything from "yeah, actually pretty much done" to "will be completely rewritten" via "still just trying random experiments trying to work out _how_ this should be rewritten". sadly i don't have a better suggestion... pending/experimental and pending/functional maybe, or something along that gist? That would be my "not adding more complexity to manage transient clutter that should instead go away" objection, already made. Then again it'd make it harder to track the history of pending commands, adding only new ones to those 2 directories would fix that, but would make the organizational problem for the old ones worse. https://en.wikipedia.org/wiki/Fundamental_theorem_of_software_engineering Stop. No. Halt. Wait. Hold it. Woah. Cease. Desist. Caution severe tire damage. Klatu barata nikto. Subcalifragilisticexpialidocious. a branch would be the usual git option, but that would probably mean "no pending stuff in the main branch" Also a problem if you want to switch Version Control systems or distribute tarballs without a .git/ directory. I already DID switch version control systems (from mercurial to git), and I already distribute release tarballs. Why do you think these are new issues? It'd hide these commands too, I want to close tabs. I am not creating additional scaffolding for clutter management: $ ls -d */toys clean3/toys clean8/toys github/toys kl4/toys kl9/toys toybox/toys clean5/toys clean.old/toys kl10/toyskl6/toys kleen/toys clean6/toys clean/toys kl2/toys kl7/toys kl/toys clean7/toys debian/toys kl3/toys kl8/toys release/toys I already try not to publish quite as much clutter as accumulates locally. There's some real fossils checked into the tree. I started work on gene2fs back under busybox, checked in what I had to the toybox repo in 055cfcbe5b05 in 2007 and haven't LOOKED at it this decade because I just haven't gotten back around to it. Since then they INVENTED EXT4. (I still hope to get back to it, but at the moment I'm answering email.) For the first time I checked if there were any special branches in the repo because I didn't bother to think about that in the months I spent working on it. i still struggle between "other" and "lsb" in particular. Same here, I can remember the posix commands. Can you? I still have to check some from time to time, and the definition of whether "tar" is a posix command or not is outright eldrich bordering on quantum. But I don't care about LSB enough to memorize everything in wants. And keeping all completed commands that aren't in poisx, lsb, networking or android The "example" directory is important because it's the only other directory of commands that should not "default y" in defconfig. It has a policy distinction. Back in 2012, when the number of commands was growing fast and having one big directory of them all was getting a bit busy, the alternative of sorting them into directories was annotating them with tags, and THAT was a nightmare (of the "this command has three tags" variety). And also implied future pressure to extend the existing kconfig implementation to USE the tags, which would be worse. Moving them into subdirectories, with each command in ONE directory, and a README explaining what the directory was for, with kconfig automatically displaying them in menus and using the first line of the README as the menu's title, seemed the least bad crowd control option at the time. in a massive "other" folder sorta defeats the purpose of these directories which are supposed to reduce clutter. It wasn't really about reducing clutter. I mean yeas, back then some web viewers wouldn't display more than 250 files in a directory, the way github truncates at 1000 today: https://github.com/landley/linux/tree/master/arch/arm/boot/dts But the goal was annotating command categories. Posix and pending are obvious, and I mentioned example. Back when I split them up, LSB was still a viable standard (the Linux Foundation hadn't destroyed it yet), and it STILL kind of means "this command existed back in Y2K and was considered part of the base system back then, even if posix never caught up". Several commands in pending get pr
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
On Fri, Mar 22, 2024 at 8:24 AM enh wrote: > > On Thu, Mar 21, 2024 at 8:45 PM Rob Landley wrote: > > > > On 3/17/24 14:52, Oliver Webb wrote: > > > On Thursday, March 14th, 2024 at 12:04, enh wrote: > > >> at a high level, it does seem like many/most people interpret "pending" > > >> as "almost done" (he says, being part of the problem himself, having > > >> several pending things building and shipping on all Android devices) > > >> whereas in actual fact it can mean anything from "yeah, actually pretty > > >> much done" to "will be completely rewritten" via "still just trying > > >> random experiments trying to work out _how_ this should be rewritten". > > >> sadly i don't have a better suggestion... > > > > > > pending/experimental and pending/functional maybe, or something along > > > that gist? > > > > That would be my "not adding more complexity to manage transient clutter > > that > > should instead go away" objection, already made. > > > > > Then again it'd make it harder to track the history of pending commands, > > > adding only new ones > > > to those 2 directories would fix that, but would make the organizational > > > problem for the old > > > ones worse. > > > > https://en.wikipedia.org/wiki/Fundamental_theorem_of_software_engineering > > > > Stop. No. Halt. Wait. Hold it. Woah. Cease. Desist. Caution severe tire > > damage. > > Klatu barata nikto. Subcalifragilisticexpialidocious. > > > > >> a branch would be the usual git option, but that would probably mean "no > > >> pending stuff in the main branch" > > > > > > Also a problem if you want to switch Version Control systems or > > > distribute tarballs without a .git/ directory. > > > > I already DID switch version control systems (from mercurial to git), and I > > already distribute release tarballs. Why do you think these are new issues? > > > > > It'd hide these commands too, > > > > I want to close tabs. I am not creating additional scaffolding for clutter > > management: > > > > $ ls -d */toys > > clean3/toys clean8/toys github/toys kl4/toys kl9/toys > > toybox/toys > > clean5/toys clean.old/toys kl10/toyskl6/toys kleen/toys > > clean6/toys clean/toys kl2/toys kl7/toys kl/toys > > clean7/toys debian/toys kl3/toys kl8/toys release/toys > > > > I already try not to publish quite as much clutter as accumulates locally. > > > > There's some real fossils checked into the tree. I started work on gene2fs > > back > > under busybox, checked in what I had to the toybox repo in 055cfcbe5b05 in > > 2007 > > and haven't LOOKED at it this decade because I just haven't gotten back > > around > > to it. Since then they INVENTED EXT4. (I still hope to get back to it, but > > at > > the moment I'm answering email.) > > > > > For the first time I checked if there were any special branches in the > > > repo because > > > I didn't bother to think about that in the months I spent working on it. > > > > > >> i still struggle between "other" and "lsb" in particular. > > > > > > Same here, I can remember the posix commands. > > > > Can you? I still have to check some from time to time, and the definition of > > whether "tar" is a posix command or not is outright eldrich bordering on > > quantum. > > > > > But I don't care about LSB enough to > > > memorize everything in wants. And keeping all completed commands that > > > aren't in poisx, > > > lsb, networking or android > > > > The "example" directory is important because it's the only other directory > > of > > commands that should not "default y" in defconfig. It has a policy > > distinction. > > > > Back in 2012, when the number of commands was growing fast and having one > > big > > directory of them all was getting a bit busy, the alternative of sorting > > them > > into directories was annotating them with tags, and THAT was a nightmare > > (of the > > "this command has three tags" variety). And also implied future pressure to > > extend the existing kconfig implementation to USE the tags, which would be > > worse. > > > > Moving them into subdirectories, with each command in ONE directory, and a > > README explaining what the directory was for, with kconfig automatically > > displaying them in menus and using the first line of the README as the > > menu's > > title, seemed the least bad crowd control option at the time. > > > > > in a massive "other" folder sorta defeats > > > the purpose of these directories which are supposed to reduce clutter. > > > > It wasn't really about reducing clutter. I mean yeas, back then some web > > viewers > > wouldn't display more than 250 files in a directory, the way github > > truncates at > > 1000 today: > > > > https://github.com/landley/linux/tree/master/arch/arm/boot/dts > > > > But the goal was annotating command categories. Posix and pending are > > obvious, > > and I mentioned example. Back when I split them up, LSB was still a viable > > standard (the Linux Foundation hadn't destroyed
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
On Thu, Mar 21, 2024 at 8:45 PM Rob Landley wrote: > > On 3/17/24 14:52, Oliver Webb wrote: > > On Thursday, March 14th, 2024 at 12:04, enh wrote: > >> at a high level, it does seem like many/most people interpret "pending" as > >> "almost done" (he says, being part of the problem himself, having several > >> pending things building and shipping on all Android devices) whereas in > >> actual fact it can mean anything from "yeah, actually pretty much done" to > >> "will be completely rewritten" via "still just trying random experiments > >> trying to work out _how_ this should be rewritten". > >> sadly i don't have a better suggestion... > > > > pending/experimental and pending/functional maybe, or something along that > > gist? > > That would be my "not adding more complexity to manage transient clutter that > should instead go away" objection, already made. > > > Then again it'd make it harder to track the history of pending commands, > > adding only new ones > > to those 2 directories would fix that, but would make the organizational > > problem for the old > > ones worse. > > https://en.wikipedia.org/wiki/Fundamental_theorem_of_software_engineering > > Stop. No. Halt. Wait. Hold it. Woah. Cease. Desist. Caution severe tire > damage. > Klatu barata nikto. Subcalifragilisticexpialidocious. > > >> a branch would be the usual git option, but that would probably mean "no > >> pending stuff in the main branch" > > > > Also a problem if you want to switch Version Control systems or distribute > > tarballs without a .git/ directory. > > I already DID switch version control systems (from mercurial to git), and I > already distribute release tarballs. Why do you think these are new issues? > > > It'd hide these commands too, > > I want to close tabs. I am not creating additional scaffolding for clutter > management: > > $ ls -d */toys > clean3/toys clean8/toys github/toys kl4/toys kl9/toys toybox/toys > clean5/toys clean.old/toys kl10/toyskl6/toys kleen/toys > clean6/toys clean/toys kl2/toys kl7/toys kl/toys > clean7/toys debian/toys kl3/toys kl8/toys release/toys > > I already try not to publish quite as much clutter as accumulates locally. > > There's some real fossils checked into the tree. I started work on gene2fs > back > under busybox, checked in what I had to the toybox repo in 055cfcbe5b05 in > 2007 > and haven't LOOKED at it this decade because I just haven't gotten back around > to it. Since then they INVENTED EXT4. (I still hope to get back to it, but at > the moment I'm answering email.) > > > For the first time I checked if there were any special branches in the repo > > because > > I didn't bother to think about that in the months I spent working on it. > > > >> i still struggle between "other" and "lsb" in particular. > > > > Same here, I can remember the posix commands. > > Can you? I still have to check some from time to time, and the definition of > whether "tar" is a posix command or not is outright eldrich bordering on > quantum. > > > But I don't care about LSB enough to > > memorize everything in wants. And keeping all completed commands that > > aren't in poisx, > > lsb, networking or android > > The "example" directory is important because it's the only other directory of > commands that should not "default y" in defconfig. It has a policy > distinction. > > Back in 2012, when the number of commands was growing fast and having one big > directory of them all was getting a bit busy, the alternative of sorting them > into directories was annotating them with tags, and THAT was a nightmare (of > the > "this command has three tags" variety). And also implied future pressure to > extend the existing kconfig implementation to USE the tags, which would be > worse. > > Moving them into subdirectories, with each command in ONE directory, and a > README explaining what the directory was for, with kconfig automatically > displaying them in menus and using the first line of the README as the menu's > title, seemed the least bad crowd control option at the time. > > > in a massive "other" folder sorta defeats > > the purpose of these directories which are supposed to reduce clutter. > > It wasn't really about reducing clutter. I mean yeas, back then some web > viewers > wouldn't display more than 250 files in a directory, the way github truncates > at > 1000 today: > > https://github.com/landley/linux/tree/master/arch/arm/boot/dts > > But the goal was annotating command categories. Posix and pending are obvious, > and I mentioned example. Back when I split them up, LSB was still a viable > standard (the Linux Foundation hadn't destroyed it yet), and it STILL kind of > means "this command existed back in Y2K and was considered part of the base > system back then, even if posix never caught up". Several commands in pending > get promoted into LSB (such as most of the password stuff, although oddly > mkpasswd is NOT in lsb 4.1). > > Hmmm,
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
On Thursday, March 21st, 2024 at 22:45, Rob Landley wrote: > On 3/17/24 14:52, Oliver Webb wrote: > > Also a problem if you want to switch Version Control systems or distribute > > tarballs without a .git/ directory. > > I already DID switch version control systems (from mercurial to git), and I > already distribute release tarballs. Why do you think these are new issues? I... Never said that those were new issues? Ones that would happen if we were to hide these in a new branch yes, but not new ones. > > Same here, I can remember the posix commands. > > Can you? I still have to check some from time to time, and the definition of > whether "tar" is a posix command or not is outright eldrich bordering on > quantum. I can certainly remember them better then the LSB commands. Most of the time I can remember if a command is in posix, which is what matters when trying to find it usually. > Back in 2012, when the number of commands was growing fast and having one big > directory of them all was getting a bit busy, the alternative of sorting them > into directories was annotating them with tags, and THAT was a nightmare (of > the > "this command has three tags" variety). And also implied future pressure to > extend the existing kconfig implementation to USE the tags, which would be > worse. > > Moving them into subdirectories, with each command in ONE directory, and a > README explaining what the directory was for, with kconfig automatically > displaying them in menus and using the first line of the README as the menu's > title, seemed the least bad crowd control option at the time. I did recognize that there was probably technical discussion behind this in 2012/2013 when they were moved. Asked about it later in the email too. Nice to know. > Collapsing the directories together when the last command is > promoted (or deleted) out of pending might make sense, What would happen when a new command shows up and we need to evaluate it then? Or glibc does a new release and yet another thing breaks we need to demote and re-promote eventually? > I also note I think I've figured out how to replace kconfig: I can just make a > list that scrolls up and down with a highlighted entry you hit space on, > handle > help text, search, exit/save, resolve selects and depends and have "menus" be > a > label line with its contents nested two spaces further to the right. [Some paragraphs bikeshedding about kconfig use to be here, may they rest in a text file until we get around to doing the kconfig rewrite] > > A possible solution is to... > > ... > > > Then again... > > I need to stop checking email every time I sit down at my laptop, because > bikeshedding can eat an endless amount of time and I've got other stuff to do. > > For one thing, I promised to look at > https://github.com/landley/toybox/issues/486 tonight. Sorry for getting in the way of that, the technical discussion about it was interesting enough to me to respond to. Recently found something to run off to and do while still benefiting toybox, so I'll stop bikeshedding about stuff like this. > P.S. I never mind people asking "what's the status of" or "why didn't you", > and > when people say they don't personally like stuff they can never actually be > WRONG about that. But suggesting what I "should" do is... tiring. I never thought about it in a "what should I make the maintainer do?" way, more in a "what benefits the project and makes it more usable?" way, considering possible solutions to issues. Sorry if it seems like I'm trying to order you around. I also stated in the email that reorganizing commands would be unfeasible, I talked about a possible way it could've been done "better", but I didn't say you should do anything. - Oliver Webb ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
On 3/17/24 14:52, Oliver Webb wrote: > On Thursday, March 14th, 2024 at 12:04, enh wrote: >> at a high level, it does seem like many/most people interpret "pending" as >> "almost done" (he says, being part of the problem himself, having several >> pending things building and shipping on all Android devices) whereas in >> actual fact it can mean anything from "yeah, actually pretty much done" to >> "will be completely rewritten" via "still just trying random experiments >> trying to work out _how_ this should be rewritten". >> sadly i don't have a better suggestion... > > pending/experimental and pending/functional maybe, or something along that > gist? That would be my "not adding more complexity to manage transient clutter that should instead go away" objection, already made. > Then again it'd make it harder to track the history of pending commands, > adding only new ones > to those 2 directories would fix that, but would make the organizational > problem for the old > ones worse. https://en.wikipedia.org/wiki/Fundamental_theorem_of_software_engineering Stop. No. Halt. Wait. Hold it. Woah. Cease. Desist. Caution severe tire damage. Klatu barata nikto. Subcalifragilisticexpialidocious. >> a branch would be the usual git option, but that would probably mean "no >> pending stuff in the main branch" > > Also a problem if you want to switch Version Control systems or distribute > tarballs without a .git/ directory. I already DID switch version control systems (from mercurial to git), and I already distribute release tarballs. Why do you think these are new issues? > It'd hide these commands too, I want to close tabs. I am not creating additional scaffolding for clutter management: $ ls -d */toys clean3/toys clean8/toys github/toys kl4/toys kl9/toys toybox/toys clean5/toys clean.old/toys kl10/toyskl6/toys kleen/toys clean6/toys clean/toys kl2/toys kl7/toys kl/toys clean7/toys debian/toys kl3/toys kl8/toys release/toys I already try not to publish quite as much clutter as accumulates locally. There's some real fossils checked into the tree. I started work on gene2fs back under busybox, checked in what I had to the toybox repo in 055cfcbe5b05 in 2007 and haven't LOOKED at it this decade because I just haven't gotten back around to it. Since then they INVENTED EXT4. (I still hope to get back to it, but at the moment I'm answering email.) > For the first time I checked if there were any special branches in the repo > because > I didn't bother to think about that in the months I spent working on it. > >> i still struggle between "other" and "lsb" in particular. > > Same here, I can remember the posix commands. Can you? I still have to check some from time to time, and the definition of whether "tar" is a posix command or not is outright eldrich bordering on quantum. > But I don't care about LSB enough to > memorize everything in wants. And keeping all completed commands that aren't > in poisx, > lsb, networking or android The "example" directory is important because it's the only other directory of commands that should not "default y" in defconfig. It has a policy distinction. Back in 2012, when the number of commands was growing fast and having one big directory of them all was getting a bit busy, the alternative of sorting them into directories was annotating them with tags, and THAT was a nightmare (of the "this command has three tags" variety). And also implied future pressure to extend the existing kconfig implementation to USE the tags, which would be worse. Moving them into subdirectories, with each command in ONE directory, and a README explaining what the directory was for, with kconfig automatically displaying them in menus and using the first line of the README as the menu's title, seemed the least bad crowd control option at the time. > in a massive "other" folder sorta defeats > the purpose of these directories which are supposed to reduce clutter. It wasn't really about reducing clutter. I mean yeas, back then some web viewers wouldn't display more than 250 files in a directory, the way github truncates at 1000 today: https://github.com/landley/linux/tree/master/arch/arm/boot/dts But the goal was annotating command categories. Posix and pending are obvious, and I mentioned example. Back when I split them up, LSB was still a viable standard (the Linux Foundation hadn't destroyed it yet), and it STILL kind of means "this command existed back in Y2K and was considered part of the base system back then, even if posix never caught up". Several commands in pending get promoted into LSB (such as most of the password stuff, although oddly mkpasswd is NOT in lsb 4.1). Hmmm, possibly instead of a dead standard the linux foundation killed, I should instead check the $PATH of my old red hat 9 install from the dawn of time... Hah, it's still on busybox's website: https://busybox.net/downloads/qemu/rh-9-shrike.img.bz2 Login as user
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
On Thursday, March 14th, 2024 at 12:04, enh wrote: > at a high level, it does seem like many/most people interpret "pending" as > "almost done" (he says, being part of the problem himself, having several > pending things building and shipping on all Android devices) whereas in > actual fact it can mean anything from "yeah, actually pretty much done" to > "will be completely rewritten" via "still just trying random experiments > trying to work out _how_ this should be rewritten". > sadly i don't have a better suggestion... pending/experimental and pending/functional maybe, or something along that gist? Then again it'd make it harder to track the history of pending commands, adding only new ones to those 2 directories would fix that, but would make the organizational problem for the old ones worse. > a branch would be the usual git option, but that would probably mean "no > pending stuff in the main branch" Also a problem if you want to switch Version Control systems or distribute tarballs without a .git/ directory. It'd hide these commands too, For the first time I checked if there were any special branches in the repo because I didn't bother to think about that in the months I spent working on it. > i still struggle between "other" and "lsb" in particular. Same here, I can remember the posix commands. But I don't care about LSB enough to memorize everything in wants. And keeping all completed commands that aren't in poisx, lsb, networking or android in a massive "other" folder sorta defeats the purpose of these directories which are supposed to reduce clutter. > `vi toys/foo.c` would save me a lot of thinking/calls to find. Eh, keeping all the commands in a massive toys/ directory makes it harder to tell what we have. A possible solution is to organize commands by their purpose. Like "net/", One directory for text-processing commands like head and grep and sort, one for programming languages like awk or bc (This becomes a lot more sensible if you want a compiler in toybox post-1.0), etc. This would mean that you could find a command by just knowing it's name and general purpose. Then again, it would take away from our ability to manage commands based on standards, And there are edge cases (Not like we are completely free of it with our current model: Does git.c go into net/ or other/ once it's done? Network access is a pretty key part of a read only git implementation). Does this outweigh the organizational usefulness of organizing commands on purpose? I realized this has probably already been discussed before back in 2012/2013 (When the commands were moved into posix/ and lsb/ and other/ and such), But haven't read the discussion on it if there was one. In any event, a complete reorganization of the commands would make the history of them a lot harder to keep track of. So it's probably not very viable until post-1.0. I don't know though, what does the project manager think? - Oliver Webb ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
> Not "I have a problem", but "it IS a problem". Well it's good that you're here
> to define objective reality. Without such a universal arbiter we would all be
> lost a world of opinion and nuance.
It is if you are trying to audit it to understand what it is doing so you
can fix 80 broken test cases. My interests align with the project and I've
already tested "does this matter" by trying to read through sh.c, if you want
auditable code
(The type someone could read through and fix bugs in) This seems like your
best course of action, at least from my perspective.
> Me, I tend to have a gap between being able to tell "this isn't ideal" and
> knowing what ideal would look like.
> Yelling at me
The original message in this thread was pretty civil, at the very least
civil compared to what you wrote in response to it. You are the one who
responded to it with one of the most histrionic emails I've ever seen
on a serious mailing list.
> "this is a mess, clean it up, how top it NOW because I'm in charge of you!"
How many feature requests could be represented as this? My first email on this
thread
was comparatively civil compared to yours ("You think there is a problem and
wanna
clean stuff up? Here take this 1000 line todo list and implement it yourself
because
you think you are CLEARLY BETTER THEN ME")
Oh, but your ego was hurt, or I stepped on your territory or something like
that...
> > and when breaking/mixing the few conventions there are happens constantly,
> > with local variables that interact with each other. That becomes a much
> > larger problem.
>
> "There are only two hard things in Computer Science: cache invalidation and
> naming things." -- Phil Karlton
>
> Do I need to tap the sign?
And you fall back to agreeing with me that "it's hard and I can't come up with
better ones now".
To which the response to "Can I help come up with better ones?" is (Ctrl-C
Ctrl-V)
"Here take this 1000 line todo list and implement it yourself because
you think you are CLEARLY BETTER THEN ME"
> > "i" is usually a counter used for at most a few dozen lines, "ss" and
> > friends span regularly for
> > hundreds of lines.
>
> They didn't span hundreds of lines when I started writing the function. The
> function changed over time. The function is not finished yet. Gratuitous
> refactoring makes "git annotate" and friends useless.
Clean code to track bugs down in eventually outweighs familiar code once it
gets bad enough. At least from my experience trying to fix bugs
> > I'm sure to someone who wrote 99% of it everything makes so much more sense.
>
> Tron: If you are a User, then everything you've done has been according to a
> plan.
>
> Kevin Flynn: Ha! You wish! Well, you know what it was like. You just keep
> doin'
> what it looks like what you're supposed to be doin', no matter how crazy it
> seems.
What am I supposed to get from this again? It's just a tron quote, which seems
to
be agreeing with me (code makes more sense to the people who understand it).
Here,
I have a more appropriate one:
Linus Torvalds: You know you’re brilliant,
but maybe you’d like to understand what you did 2 weeks from now.
> > Just like the people who maintain GCC and the coreutils and IOCCC entries.
>
> Carefully calculated to flatter me into agreeing with your opinions, I see.
[...]
> "Even ones written by" me. How nice.
Sorry, I looked for my red carpet to roll out but I couldn't find it.
I'm not sure if you were expecting me to be civil to your email or...
Yknow, In the follow up email you ignored I compared you Indirectly to Greg KH.
Though it was about territory too (Now it just seems like I hurt your ego,
sorry)
because this had a uncanny resemblance to the
Fairly minor and unimportant Greg/Sievers sysfs Documentation Shenanigans of
2007.
Felt bad because it was sorta inaccurate after taking another lay of the land,
Was even thinking about apologizing in this email, but then, out of the corner
of my
eye, I saw it. The last trick in the playbook:
> > Make it look like whatever you want, I honestly couldn't care.
>
> Oh good, I can stop reading then.
>
> Rob
Act insanely abrasive to a civil discussion because your territory or ego or
something
was hurt, then run away because (Who could've ever possibly predicted:) "He's
being MEAN to me!!!"
(Or whatever excuse you can find, if I didn't say "I honestly couldn't care."
you probably would've pulled another reason out to stop talking about this)
Half these arguments could be responded to by things in the email you just
"https://lkml.iu.edu/hypermail/linux/kernel/0707.2/3051.html"-d to.
And I'll let you run away from the argument and I'll go back to bikeshedding
whatever I can find
to make the project at least somewhat better (readelf -n segfault on 64 bit
ELF's, csplit regressions, etc)
(Assuming you still want patches from me). This flamewar has been as fun for me
as I guess it was for you.
> P.S. Are you aware that you have opinions? That
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
at a high level, it does seem like many/most people interpret "pending" as "almost done" (he says, being part of the problem himself, having several pending things building and shipping on all Android devices) whereas in actual fact it can mean anything from "yeah, actually pretty much done" to "will be completely rewritten" via "still just trying random experiments trying to work out _how_ this should be rewritten". sadly i don't have a better suggestion... a branch would be the usual git option, but that would probably mean "no pending stuff in the main branch", which would have been a net loss for Android. coming up with a clever name to distinguish the "keep out!" end of the "pending" spectrum might prevent some of these arguments^Wdiscussions, but i'm not sure and i don't know what that name should be... would even something like "experimental" be enough of a warning? dunno. anecdotally as an observer, it does seem like it is mostly just sh that causes all the arguments ... _i've_ been happily ignoring toybox sh because i'm fine with mksh for the time being (other than "can we have the bash math exponentiation?" i think _every_ mksh bug i've ever had is "why isn't the command-line editing better?", which isn't even a focus of toybox sh anyway!), but it seems like everyone looking at toybox for some kind of minimal distro wants to use the toybox sh. and those folks don't realize that "pending" in this case really does mean "NOT A STEP" :-) funnily enough, _personally_ i'd rather have _fewer_ directories --- toybox is significantly less confusing than busybox was by virtue of having significantly fewer directories, but i still struggle between "other" and "lsb" in particular. `vi toys/foo.c` would save me a lot of thinking/calls to find. (i've been trying to train myself to type `toys/*/foo.c` instead, so far without much success. i usually only remember after i've _already_ done the find(1) :-) ) On Thu, Mar 14, 2024 at 9:22 AM Rob Landley wrote: > On 3/13/24 18:19, Oliver Webb wrote: > >> If you want to complain, I do tend to have "s", "ss", and "sss" as an > > > > I did complain about that in the email. > > I noticed. > > >> You're basically complaining there was an "i" variable that wasn't a > loop index. > > > > That IS a problem, > > Not "I have a problem", but "it IS a problem". Well it's good that you're > here > to define objective reality. Without such a universal arbiter we would all > be > lost a world of opinion and nuance. > > Me, I tend to have a gap between being able to tell "this isn't ideal" and > knowing what ideal would look like. Yelling at me "this is a mess, clean > it up, > how top it NOW because I'm in charge of you!" doesn't really help the > second part. > > > and when breaking/mixing the few conventions there are happens > _constantly_, > > with local variables that interact with each other. That becomes a much > larger problem. > > "There are only two hard things in Computer Science: cache invalidation and > naming things." -- Phil Karlton > > Do I need to tap the sign? > > > "i" is usually a counter used for at most a few dozen lines, "ss" and > friends span regularly for > > hundreds of lines. > > They didn't span hundreds of lines when I started writing the function. The > function changed over time. The function is not finished yet. Gratuitous > refactoring makes "git annotate" and friends useless. > > (I remember once when I was maintaining a tinycc fork, I introduced a > subtle bug > that took me days to track down by REINDENTING the code because I screwed > up a > curly bracket placement. I do not yet have what I consider a load bearing > test > suite for this command.) > > >> In multiple places, f stands for function, and there's more than one > aspect to > >> functions. > > > > I'm sure to someone who wrote 99% of it everything makes so much more > sense. > > Tron: If you are a User, then everything you've done has been according to > a plan. > > Kevin Flynn: Ha! You wish! Well, you know what it was like. You just keep > doin' > what it looks like what you're supposed to be doin', no matter how crazy > it seems. > > > Just like > > the people who maintain GCC and the coreutils and IOCCC entries. > > Carefully calculated to flatter me into agreeing with your opinions, I see. > > >> > The problem isn't the length as I said, the problem is that there is > no convention for the naming > >> > of these. > >> > >> Maybe I should move the file to "pending"? > > > > This is the most important command in the entire project (you can't _do > anything_ on a machine without > > a shell), No other pending command I've ever seen is this in-auditable, > even ones written by you. > > "Even ones written by" me. How nice. > > >> And as I said: if I were to I apply an aesthetic patch which does > nothing but > >> make the code smell like you, toysh would be all yours and I would > never touch > >> it again. > > > > Make it look like whatever you want, I honestly couldn't care.
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
On 3/13/24 18:19, Oliver Webb wrote: >> If you want to complain, I do tend to have "s", "ss", and "sss" as an > > I did complain about that in the email. I noticed. >> You're basically complaining there was an "i" variable that wasn't a loop >> index. > > That IS a problem, Not "I have a problem", but "it IS a problem". Well it's good that you're here to define objective reality. Without such a universal arbiter we would all be lost a world of opinion and nuance. Me, I tend to have a gap between being able to tell "this isn't ideal" and knowing what ideal would look like. Yelling at me "this is a mess, clean it up, how top it NOW because I'm in charge of you!" doesn't really help the second part. > and when breaking/mixing the few conventions there are happens _constantly_, > with local variables that interact with each other. That becomes a much > larger problem. "There are only two hard things in Computer Science: cache invalidation and naming things." -- Phil Karlton Do I need to tap the sign? > "i" is usually a counter used for at most a few dozen lines, "ss" and friends > span regularly for > hundreds of lines. They didn't span hundreds of lines when I started writing the function. The function changed over time. The function is not finished yet. Gratuitous refactoring makes "git annotate" and friends useless. (I remember once when I was maintaining a tinycc fork, I introduced a subtle bug that took me days to track down by REINDENTING the code because I screwed up a curly bracket placement. I do not yet have what I consider a load bearing test suite for this command.) >> In multiple places, f stands for function, and there's more than one aspect >> to >> functions. > > I'm sure to someone who wrote 99% of it everything makes so much more sense. Tron: If you are a User, then everything you've done has been according to a plan. Kevin Flynn: Ha! You wish! Well, you know what it was like. You just keep doin' what it looks like what you're supposed to be doin', no matter how crazy it seems. > Just like > the people who maintain GCC and the coreutils and IOCCC entries. Carefully calculated to flatter me into agreeing with your opinions, I see. >> > The problem isn't the length as I said, the problem is that there is no >> > convention for the naming >> > of these. >> >> Maybe I should move the file to "pending"? > > This is the most important command in the entire project (you can't _do > anything_ on a machine without > a shell), No other pending command I've ever seen is this in-auditable, even > ones written by you. "Even ones written by" me. How nice. >> And as I said: if I were to I apply an aesthetic patch which does nothing but >> make the code smell like you, toysh would be all yours and I would never >> touch >> it again. > > Make it look like whatever you want, I honestly couldn't care. Oh good, I can stop reading then. Rob P.S. Are you aware that you have opinions? That you're not an ordained judge of what is and is not correct? No, why should I ask questions: you didn't. To answer some questions you didn't ask: A _better_ name for a variable is not always immediately obvious to me as I'm writing. The uses of a given variable tend to change as the function develops. I tend to perform cleanup passes to make the code more intelligible AFTER it's otherwise working (and has a full regression test suite it passes so I can catch bugs introduced by refactoring). Things that are used a LOT tend to have short names in my code, because BigLongCamelCaseName repeated 400 times in a function does not make code easier for _me_ to manage or understand, and variable names get out of date the exact same way comments get out of date. Obviously, your experience differs, and I look forward to the shell you write from scratch that renders toysh unnecessary. It should be easy for you, given you can dictate empirical truth in a way that obviates judgement. Me, I'm muddling through as best I can... ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
Added P.S.:
If this is territorial, just admit it. I won't bring up the
"is keeping territory a good thing in a open-source project?" argument. I never
mentioned territory because I don't think like that (I've waited for and
welcomed cleanup passes
on code I'm actively developing). You mentioned territory ("I'll never touch
this again if other
people are doing stylistic changes on it for auditability"). Like my goal was
"WORLD DOMINATION!"
instead of "Make this more readable so I and other people can build off of and
improve this".
And now, a link with no meaning or parallels to this discussion:
https://landley.net/notes-2018.html#06-09-2018
...
I'll leave it here unless you wanna bring up a counterargument, I'd like
to hear reasons why I'm wrong on this.
- Oliver Webb aquahobby...@proton.me
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
> On 3/8/24 21:22, Oliver Webb via Toybox wrote: > > Reading through sh.c, most of the variable names are 2 letters long > > (repeating the same letter), > > I switched from single character local variables to double character ones > because they're easier to search for without an elaborate IDE trying to parse > the code. /\ searches for specific words in vim > If you want to complain, I do tend to have "s", "ss", and "sss" as an I did complain about that in the email. > would otherwise be just "string". That be like naming _every_ integer "integer", (No wait, strings are _worse_ because they can hold a arbitrary string of anything) > > When a name > > like "in_source", "inputsrc", or just "inname" would tell you that without > > having to look > > over the code and make sure that's the case. > > No, you still have to look at the code. Especially when the code is not > finished, as evidenced by it still being in pending/ and not passing half its > test suite. (Let alone my giant pile of "turn these into proper shell tests", > attached...) Not if you are skimming over a section, that is mostly unimportant but interacts with a certain thing you are looking for bugs in. > And "c" holding the argument of "-c" is still probably a bad example to base > your rant on. I couldn't get far enough down from sh_main() to see any better examples, sorry. > And a command that's flagrantly not finished yet may also be a bad example... This is the only one I've seen which does stuff like this. No other command in pending does stuff like this (At least not one I've looked at). > > "ff" is a common name for file descriptors. > > In this idiom it's "f, but more searchable". > > You're basically complaining there was an "i" variable that wasn't a loop > index. That IS a problem, and when breaking/mixing the few conventions there are happens _constantly_, with local variables that interact with each other. That becomes a much larger problem. "i" is usually a counter used for at most a few dozen lines, "ss" and friends span regularly for hundreds of lines. > In multiple places, f stands for function, and there's more than one aspect to > functions. I'm sure to someone who wrote 99% of it everything makes so much more sense. Just like the people who maintain GCC and the coreutils and IOCCC entries. Unfortunately, I never got the chance to look at it when it was somewhat auditable. So when I try to remove warnings/ fix test cases > > The problem isn't the length as I said, the problem is that there is no > > convention for the naming > > of these. > > Maybe I should move the file to "pending"? This is the most important command in the entire project (you can't _do anything_ on a machine without a shell), No other pending command I've ever seen is this in-auditable, even ones written by you. > And as I said: if I were to I apply an aesthetic patch which does nothing but > make the code smell like you, toysh would be all yours and I would never touch > it again. Make it look like whatever you want, I honestly couldn't care. I am someone who's looked over the infrastructure and cares about toybox enough to work on and learn about it in my free time. And even binary searched sh.c for a segfault once. And I can barely understand it without taking hours upon hours to extensively look over every part because variable names don't _tell you anything_. The person who is trying to build a minimal system for whatever reason is not gonna sit down for 12 hours to debug-printf their way into understanding it so they can get whatever the bug of the day is fixed. If you wanna do it all yourself and don't care how obfuscated it is to an outside perspective without the testing and development knowledge gained along the way and internal thought process at the time. I honestly won't complain, anything that gets test cases fixed and missing builtins included. I can treat it like a black box until a changing number of bash features (what _are_ we implementing?) from a changing specification of a 84,000 line shell is added, then refactored into a state where it's readable. And I'll keep my hands off your territory until then. > Clearly I have been too slow to finish this, and it's time for you to take > over. > Your aesthetic sense is empirically superior to mine, (Because I want variable names that say something, and you want ones that say nothing because "less bytes" and "sh.c is my territory and I don't expect anyone else to even look at it until it's done because clearly I don't need any help"?) > I stand chastized. I'm sorry I offended you, but this code is easily the hardest code in the project to audit. I've never had issues figuring out how the other code works, this is the exception, _not_ the norm. Part of this is behemoth functions that set out to implement entire subsystems, which allows for spaghetti code, but a lot of this is also variable names saying _nothing_. You want
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
On 3/8/24 21:22, Oliver Webb via Toybox wrote:
> TL;DR: Rant about sh.c's variable names I forgot to include in main email, I
> have a patch to start fixing it but it conflicts with other stuff and I have
> to re-do it
>
> Reading through sh.c, most of the variable names are 2 letters long
> (repeating the same letter),
I switched from single character local variables to double character ones
because they're easier to search for without an elaborate IDE trying to parse
the code.
> for short functions (<20 lines) this isn't a problem. For example, I don't
> need a more
> descriptive name for the variable "c" in getvar() to understand it because I
> can look at the uses
> of it and infer from context it's about variable types. Longer functions
> where there are a lot of variables
> interacting with each other are a lot harder to keep track of.
>
> For example, what does "cc" do in sh_main?
Hold the -c argument. The first assignment to it is:
cc = TT.sh.c;
> From the looks of it. It's the name
> of command/file being processed, but to _know_ that's the case I have to audit
> 50 lines of code, and check over seeing if that's what it really does.
Variables sometimes get reused to mean slightly different things, and their use
can change over time when a lot of editing is done to a function. Especially for
code that isn't finished yet, there can be unpolished bits.
Although in this case, "-c is held in the variable c but I generally double such
variable names to give vi's text search at least SOME chance of finding it"
seems reasonable to me?
If you want to complain, I do tend to have "s", "ss", and "sss" as an
alternative to "s1", "s2", and "s3" when the long descriptive name for the
variable would otherwise be just "string". That gets a bit awkward at times and
I've looked at renaming some but not come up with better names.
> When a name
> like "in_source", "inputsrc", or just "inname" would tell you that without
> having to look
> over the code and make sure that's the case.
No, you still have to look at the code. Especially when the code is not
finished, as evidenced by it still being in pending/ and not passing half its
test suite. (Let alone my giant pile of "turn these into proper shell tests",
attached...)
And "c" holding the argument of "-c" is still probably a bad example to base
your rant on. And a command that's _flagrantly_ not finished yet may also be a
bad example...
> The problem isn't the length, the problem is that there isn't any convention
> to it and a lot
> of names convey little to no information ("ss" is usually a string, but "what
> does the string do?"
> is a question you need to commonly ask to find out what it's doing). It'd be
> like if run_lines()
> was called "rr()" or "expand_arg" was called "ea()", we name functions just
> fine (which makes
> this code somewhat readable, at least it's general structure)
Would you like me to stay out of your way while you take over toysh and refactor
it to your heart's content? I note that I will never touch it again if I hand it
over.
> "ff" is a common name for file descriptors.
In this idiom it's "f, but more searchable".
You're basically complaining there was an "i" variable that wasn't a loop index.
> So working down from sh_main, One could assume that
> "TT.ff" must have something to do with file management, right?... No, it's a
> doubly linked list
> of overhead for shell functions,
In this case it sounds like f stands for function, yes. Many words start with f.
> and "struct sh_fcall" is often called "ff" in functions that use
> it, we have 2 common names that do completely different things...
In multiple places, f stands for function, and there's more than one aspect to
functions.
> The problem isn't the length as I said, the problem is that there is no
> convention for the naming
> of these.
Maybe I should move the file to "pending"?
> No pattern that you can follow, you usually have to go to the start of the
> function
> (which can be hundreds of lines up, depending on what you are looking
> through) and see how it gets
> assigned (And interacts with other variables that have the exact same problem)
>
> I'd try to replace a lot of these names in a patch (I have already done so
> for sh_main in a patch
> that I didn't send because getting the compiler to shut up was more
> important, will send the patch
> when I'm sure it won't conflict with anything)
And as I said: if I were to I apply an aesthetic patch which does nothing but
make the code smell like you, toysh would be all yours and I would never touch
it again.
> but to be accurate in renaming I have to understand
> what they are, (The problem I'm trying to fix) which requires either better
> names to begin with or
> extensive auditing and debug printf-ing of the code that takes up a lot of
> time.
Clearly I have been too slow to finish this, and it's time for you to take over.
Your aesthetic sense is empirically superior to mine, I
Re: [Toybox] [PATCH] toysh: fix -Wuse-after-free
TL;DR: Rant about sh.c's variable names I forgot to include in main email, I
have a patch to start fixing it but it conflicts with other stuff and I have
to re-do it
Reading through sh.c, most of the variable names are 2 letters long (repeating
the same letter),
for short functions (<20 lines) this isn't a problem. For example, I don't need
a more
descriptive name for the variable "c" in getvar() to understand it because I
can look at the uses
of it and infer from context it's about variable types. Longer functions where
there are a lot of variables
interacting with each other are a lot harder to keep track of.
For example, what does "cc" do in sh_main? From the looks of it. It's the name
of command/file being processed, but to _know_ that's the case I have to audit
50 lines of code, and check over seeing if that's what it really does. When a
name
like "in_source", "inputsrc", or just "inname" would tell you that without
having to look
over the code and make sure that's the case.
The problem isn't the length, the problem is that there isn't any convention to
it and a lot
of names convey little to no information ("ss" is usually a string, but "what
does the string do?"
is a question you need to commonly ask to find out what it's doing). It'd be
like if run_lines()
was called "rr()" or "expand_arg" was called "ea()", we name functions just
fine (which makes
this code somewhat readable, at least it's general structure)
"ff" is a common name for file descriptors. So working down from sh_main, One
could assume that
"TT.ff" must have something to do with file management, right?... No, it's a
doubly linked list
of overhead for shell functions, and "struct sh_fcall" is often called "ff" in
functions that use
it, we have 2 common names that do completely different things...
The problem isn't the length as I said, the problem is that there is no
convention for the naming
of these. No pattern that you can follow, you usually have to go to the start
of the function
(which can be hundreds of lines up, depending on what you are looking through)
and see how it gets
assigned (And interacts with other variables that have the exact same problem)
I'd try to replace a lot of these names in a patch (I have already done so for
sh_main in a patch
that I didn't send because getting the compiler to shut up was more important,
will send the patch
when I'm sure it won't conflict with anything) but to be accurate in renaming I
have to understand
what they are, (The problem I'm trying to fix) which requires either better
names to begin with or
extensive auditing and debug printf-ing of the code that takes up a lot of time.
- Oliver Webb aquahobby...@proton.me
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
