On 7/5/21 1:50 PM, Wolfgang Denk wrote:
Dear Sean,
In message <5a967151-94f0-6037-2d02-0114c43b8...@gmail.com> you wrote:
AIUI hush has diverged significantly from what U-Boot has. This would
not be an "update" moreso than a complete port in the style of the
current series.
Agreed. However, as you write this it sounds like a big problem
only. I disagree here - it is also a chance to do a few things
different than with the original port.
Please keep in mind when this original port of the hush shell was
done: it was a time when many systems came with a total of 4 MB
flash memory, and not only U-Boot, but also the Linux kernel and a
ram-disk image or such had to fit into that. At that time 40 or 60 kB
code size for just a fancy shell was immense!
Under such restrictions (and the need to complete the task with a
given budget) many things were just commented out which from today's
point of view would be nice to have.
This is not a problem of the code complexity or resources, but only
of different requirements and different resources.
I don't think sh-style shells are a good match for U-Boot's execution
environment in the first place. The fundamental idea of an sh-style
shell is that the output of one command can be redirected to the input
(or arguments) of another command. This cannot be done (or rather would
be difficult to do) in U-Boot for a few reasons
There is an old saying:
The loser says: "It might be possible, but it is too difficult."
The winner says: "It might be difficult, but it is possible."
And yet, the winner may also say "this is too difficult to be worth it"
instead of laboring inefficiently :)
Apparently you take another position here than me.
First, I disagree that pipes are the "fundamental idea" of a shell.
It is a fundamental idea of the UNIX operating systems, indeed.
Well, I think all programming languages at their core are about
transferring information from one part of the program to another.
Without pipes, how do bourne-style shells communicate? Sure you can
transfer data into functions using arguments, but what about the other
way around? Numeric return values? Global variables? I think these are
rather anemic compared to proper return values.
But please keep in mind that we are here in a boot loader, not in a
full-blown OS context.
* U-Boot does not support multithreading. Existing shells tend to depend
strongly on this feature of the enviromnent. Many of the changes to
U-Boot's hush are solely to deal with the lack of this feature.
I disagree to both parts of your statement.
Multithreading (or rather, a concept of separate processes which can
eventually even run in parallel) is very nice to have, but it is not
mandatory in most cases. Command pipes can almost always be
strictly sequentialized and thus run in a single-task environment,
too. I'm not sure, but I think I even remember pipes in the "shell"
of the CPM "OS" on Z80 processors a number of decades ago...
The point here is that many Hush features were written with fork. For
example, to create a new scope (such as for a subshell), Hush just
fork()s. When parsing, if it encounters a construct like $(), it fork()s
and then modifies the map variable, (correctly) assuming that the
"original" map will remain unmodified. These things are all over and
make doing a port difficult; especially when they crop up as unforseen
bugs. Though I suppose the real issue here is a lack of virtual memory.
And the fact that our current version of hush did not attempt to
keep these features was much more driven by strict memory footprint
limitations that anything else. I claim it would have been
possible, and still is.
You also don't mention (and I guess you oversee) another critical
fact: so far, U-Boot has no concept of files. The classic design
principle "everything is a file" is missing even more than the
concept of processes.
IMO "everything is a file" is more of an API thing than a shell thing.
E.g. the idea that you can use open/read/write/close to access any
resource. But without pipes, I don't think such an abstraction is very
useful.
* Existing commands do not read from stdin, nor do they print useful
information to stdout. Command output is designed for human
consumption and is substantially more verbose than typical unix
commands.
This is a statement which is correct, but it does not contain any
pro or con for the discussion here.
In order for pipes to be useful, programs should generally follow the
Unix philosophy. In particular, many commands in U-Boot fall afoul of
rule 2:
Expect the output of every program to become the input to another, as
yet unknown, program. Don't clutter output with extraneous
information. Avoid stringently columnar or binary input formats.
Don't insist on interactive input.
Which makes it difficult to compose them.
And if we had the features, it would be easy to fix.
I don't know about that. I think making U-Boot commands emit output
designed for machine consumption would require substantial effort.
* Tools such as grep, cut, tr, sed, sort, uniq, etc. which are extremely
useful when working with streams are not present in U-Boot.
You are talking about OS environments here. But we are a boot
loader.
These programs are just as much part of the shell as the builtin
commands. They allow one to take input from one program and transform it
into the appropriate input to another program. This is especially
important for extracting information from programs which were not
designed to interface well.
Also, this argument is not fair, as the suggested LIL does not
provide grep, sed, awk, sort, uniq etc. functionality either, or
does it?
Of course it is fair. These programs are just as much part of the *nix
programming environment as the bourne shells which invoke them.
And of course, this feature is currently not present in U-Boot. To get
Correct, and for very good reasons.
which will set my_uuid to the uuid of the selected partition. My issue
with this is threefold: every command must add new syntax to do this,
that syntax is inconsistent, and it prevents easy composition. Consider
a script which wants to iterate over partitions. Instead of doing
for p in $(part list mmc 0); do
# ...
done
it must instead do
part list mmc 0 partitions
for p in $partitions; do
# ...
done
which unnecessarily adds an extra step. This overhead accumulates with
each command which adds something like this.
Apparently you fail to understand that this "extra step" is primarily
due to the fact that we are single tasking. Even if we has command
substitution (like we could have with hush) the execution sequence
would be serialized in exactly the same way under the hood - it's
just not as directly visible.
Please don't assume what I do and do not understand ;)
My point here is that instead of having a consistent interface for
passing data from a command to its caller, we instead have an
inconsistent one which is different for each command.
Both of these workarounds are natural consequences of using a sh-tyle
shell in an environment it is not suited for. If we are going to go to
No, they are not. They are much more the consequence of no strict
design guidelines for commands, so everybody implements what fits his
purposes best. A cleaner approach does not require any of the
additional features you've asked for - it would be possible as is.
So you would have every command have a new parameter at the end
specifying which environmental variable to put a return value in? IMO
such things should be done at the language level so that commands can
just call a function like set_result() and let the shell handle the
rest.
the effort of porting a new language (which must be done no matter if
we use Hush or some other language), we should pick one which has better
support for single-threaded programming.
In which way do you think "a new language" needs to be ported when
switching from an old to a new version of hush? It would be still a
(mostly) POSIX compatible shell, with some restrictions.
Modern Hush is completely unrecognizable compared to what we have in
U-Boot today. Any "updating" effort would be akin to going over the
entire upstream codebase and porting it from scratch.
I can fully understand that you defend your proposal, but let's be
fair and stick with the facts. Thanks.
And I'd appreciate if we could presume good faith of our peers.
--Sean
