Re: [PATCH v2 5/8] builtins/source: parse the -i option
On Sun, May 19, 2024 at 10:58 AM Matheus Afonso Martins Moreira < math...@matheusmoreira.com> wrote: > > This is not a problem that's introduced by this patch. > People can already do that today. Anyone could write > `alias source='rm -rf --no-preserve-root /*'` right now, > nothing stops them. So I don't see why this would be > a reason against inclusion of this feature. > I was not talking about breakage of a script, anybody knows you can stick an rm -rf anywhere in your script (BTW protection/containers,... mitigate the damages). I was talking of two incompatible package manager, where one can break the other, yours can break mine, mine can't break yours, I was asking for a way to secure the old (current) way for mine. So far I saw typeset -r BASH_SOURCE_PATH='' in my init path, but not even sure it will be enough, beside I don't control my users who may miss the init update but still can mix'n'match the package managers. Here, we are not talking adding one more feature into bash, we are talking about one feature predate (intentionally or not) another one, it is normal that some head's up occurs. > I understand your concern though. Perhaps we can find > ways to work around it or fully address it. > Not too sure about that. > > > Another things that bugs me, why not implementing your invention in a > > dynamically loaded builtin, advertise it on github, and let the one who > > really needs this great thing be their choice download it, and use it. > > I don't really want to maintain such a thing. I'm struggling to find time > to work on my personal projects as it is. We are lucky to have someone > who maintains bash and who is considering this feature for addition. > Strange argument, you want to forward the hot potatoes to someone else regarding maintenance. First of all you claim the feature is tiny, almost bug free in its inception, secondly it is vastly needed. If those assertion are true, then your maintenance workload is almost nil for your feature you can always access on top of bash with dynload. If there is a vast need for it, and if unfortunately a bug shows up, then in this vast population, there will be a good soul providing a pull request for it, your load would be minimal. That's the magic of open software :-) So I think a dynloadable builtin of yours would be a perfect use case of dynload, and a good way to challenge your invention, i.e measure the vast population, bug rate, etc... I think it is a good 'preview' way, before eventually move from dyn load to static load one day. Dynloadable builtins exist, use it. > I don't want to fork bash. I want to be able to download the latest version > with my package manager and get access to this. It is perfectly doable, clone, configure, build, stick you patch and provide it to you (or your users) not a big deal of effort. You can have that today, all that hidden in... bingo a library :-) > If I had to fork bash to > get this, I would simply give up and continue working on my own > programming language instead. > I think this is a great valuable idea! PS : On the citations side, I remember one, a software is bug free when you trash it :-)
Re: [PATCH v2 5/8] builtins/source: parse the -i option
At 2024-05-20T03:42:04+0700, Robert Elz wrote: > My memory is hopeless, and I'm sure someone on the list can supply the > correct attribution, but one of the big name CS people once said > something along the lines of "perfection isn't when there's nothing > left to add, but when there's nothing left to remove". That would be Antoine de Saint-Exupéry, author of _La Petit Prince_, which you may have read as a child, or read to children at some point. Wikipedia says he was a "writer, poet, and aviator". A reminder, were one needed, that intellectual silos are costly! :) (I only had this factoid ready at hand because Peter Schaffter, author of the mom macro package for groff, uses it in his .signature file.) Regards, Branden signature.asc Description: PGP signature
Re: ${var/\#} no longer works
On Sun, May 19, 2024, 14:05 konsolebox wrote:
> Here's an output from devel branch bash and 5.2.26:
>
> $ bash-9 -c 'echo "$BASH_VERSION"; x="abc#xyz"; echo "${x/\#}"'
> 5.3.0(1)-alpha
> abc#xyz
> $ bash-5.2.26 -c 'echo "$BASH_VERSION"; x="abc#xyz"; echo "${x/\#}"'
> 5.2.26(1)-release
> abcxyz
This is a bug caused by one of my patches, just reported last week:
https://lists.gnu.org/archive/html/bug-bash/2024-05/msg00174.html
Re: [PATCH v2 5/8] builtins/source: parse the -i option
Date:Sun, 19 May 2024 10:31:30 +
From:Matheus Afonso Martins Moreira
Message-ID:
<5LZgqrwhk0KEXRJqJIfPJzZvaJA8FVWbs_hnJHSPrPnzAzkjo7oA926voEMtU_3FGuZdylouhgpNXciyEJZPFdsScZ4pK0i4GLqv7ttR0nI=@matheusmoreira.com>
| > the real work of the source ('.' really) utility
| > is to read the file
|
| That's the work of read.
|
| > parse it, and execute it.
|
| That's the work of eval.
It could be.
| The point is they probably could.
| So why not do it?
Because there's no need, '.' exists, and is never going away,
so rather than doing lots of work, just use it. If functions
had been implemented initially, and '.' not, then perhaps it
might be done that way, and we wouldn't need a '.' command, and
any suggestion to add one could be defeated by an argument
like that. But that's all in the past, and can no longer be changed.
| A builtin is far more convenient, isn't it?
| Much more ergonomic.
Only when it is really needed.
| Not at all. If one solution is capable of
| accepting more inputs than another
| without sacrificing any functionality,
| then it's the better solution.
That's not true at all. My memory is hopeless, and I'm sure someone
on the list can supply the correct attribution, but one of the big name
CS people once said something along the lines of "perfection isn't
when there's nothing left to add, but when there's nothing left to
remove".And that's right - but it works only during initial design,
once something has been made available, people use it, and it is never
possible after that to remove it. (Unless you're python, and don't care
about breaking people's code.) But it does apply when adding
anything new (or considering that) - new stuff should be added only
when absolutely essential, not just because it is convenient for someone
(even many someones).
| > And a better solution to that would be to remove that restriction,
| Is removing things from bash and breaking users a valid choice here?
No, of course not, but removing that restriction wouldn't break
anything I can see.
| > which has no purpose that I can imagine.
|
| I didn't even know this restricted shell thing existed
| until about a week ago. The fact remains that it's in
| bash and that someone out there is probably using it.
Of course. Oh, I see, you think I was suggesting removing the
restricted shell. No, I wouldn't suggest that, no matter how
stupid the whole idea is. All I suggested removing (for the
purposes relevant here anyway) was the restriction that the '.'
command cannot read files with a '/' in their names. That
restriction seems pointless. Removing the restriction cannot
break any existing rbash setups (they can't be using '.' with
a name containing a '/') unless it created a vulnerability,
which I don't see as happening here.
| If the builtin works with the restricted shells
| while the pure shell function does not,
| the builtin is the better solution.
"If" yes, perhaps (though restricted shells work now without your
proposed changes) - which is why I suggested removing the restriction,
then the pure shell function would work with restricted shells too,
and no bash modifications would be needed (except to remove what seems
to be a pointless restriction.)
| Restricted shells do not allow command names with slashes.
That's fine, but they don't prohibit args to (most) commands
containing slashes, and in any case, using shell rules to provide
access control is silly, we have file and directory permissions,
and sometimes ACLs as well, to do that.
| > Just remove that restriction and this point goes away
| That would be a breaking change.
Breaking what?
| Not long after I sent the patches, a list of about
| a dozen projects implementing module loading
| and management in bash was posted.
| Clearly this is something people want.
I don't see the connection. All you're proposing is a different
way to convert a relative pathname in a '.' command into a
full one (in effect - that is, to locate it precisely in the
filesystem). Since that can be done with a function, which
can then operate according to whatever rules the writer of that
function wants (things relative to the script location, relative
to $PWD relative to $PWD when the script started running, relative
to $HOME, relative to some other fixed point in the filesystem, or ...)
and applying whatever other name transformations it desires.
Much more flexible, and better suited to varying desires.
| Well all those components exist now.
| Why not use them instead of source?
Because that would be stupid - one uses the best tools that
are available, but every new job does not require the invention
of a new tool just to handle that job. If you start doing that
you end up with so many tools, that no-one knows what exists
any more.
| If I just started deleting things I couldn't care less about,
| there wouldn't be much of bash left
Re: printf -u "$fd"?
On Sun, May 19, 2024, 20:11 Kerin Millar wrote: > On Sun, 19 May 2024, at 5:08 PM, alex xmb sw ratchev wrote: > > On Sat, May 18, 2024, 04:54 Zachary Santer wrote: > > > >> Was «difference between read -u fd and read <&"$fd"» on > help-b...@gnu.org > >> > >> On Thu, May 16, 2024 at 12:51 AM Kerin Millar > wrote: > >> > > >> > On Thu, 16 May 2024, at 3:25 AM, Peng Yu wrote: > >> > > Hi, > >> > > > >> > > It appears to me that read -u fd and read <&"$fd" achieve the same > >> > > result. But I may miss corner cases when they may be different. > >> > > > >> > > Is it true that they are exactly the same? > >> > > >> > They are not exactly the same. To write read -u fd is to instruct the > >> read builtin to read directly from the specified file descriptor. To > write > >> read <&"$fd" entails one invocation of the dup2 syscall to duplicate the > >> specified file descriptor to file descriptor #0 and another invocation > to > >> restore it once read has concluded. That's measurably slower where > looping > >> over read. > >> > >> So here's another tangent, but has it been considered to add an option > >> to the printf builtin to print to a given file descriptor, rather than > >> stdout? If printing to a number of different file descriptors in > >> succession, such an option would appear to have all the same benefits > >> as read's -u option. > >> > > > > id like a printf -u > > > > Zack > > I understand that you have pledged to use broken email software for the > rest of time but could you at least refrain from posting emails that look > as though they were signed off by someone other than yourself? It's not > cricket. > what , sorry ? is it the case everyone tells me i code obfuscated while at all i never did ? just cause i shortened varnames ? u speak about ' zack ' at end of mail display ? ye buggy email sw .. i del < and do 3 newlines and in the middle i enter text .. -- > Kerin Millar >
Re: printf -u "$fd"?
On Sun, 19 May 2024, at 5:08 PM, alex xmb sw ratchev wrote: > On Sat, May 18, 2024, 04:54 Zachary Santer wrote: > >> Was «difference between read -u fd and read <&"$fd"» on help-b...@gnu.org >> >> On Thu, May 16, 2024 at 12:51 AM Kerin Millar wrote: >> > >> > On Thu, 16 May 2024, at 3:25 AM, Peng Yu wrote: >> > > Hi, >> > > >> > > It appears to me that read -u fd and read <&"$fd" achieve the same >> > > result. But I may miss corner cases when they may be different. >> > > >> > > Is it true that they are exactly the same? >> > >> > They are not exactly the same. To write read -u fd is to instruct the >> read builtin to read directly from the specified file descriptor. To write >> read <&"$fd" entails one invocation of the dup2 syscall to duplicate the >> specified file descriptor to file descriptor #0 and another invocation to >> restore it once read has concluded. That's measurably slower where looping >> over read. >> >> So here's another tangent, but has it been considered to add an option >> to the printf builtin to print to a given file descriptor, rather than >> stdout? If printing to a number of different file descriptors in >> succession, such an option would appear to have all the same benefits >> as read's -u option. >> > > id like a printf -u > > Zack I understand that you have pledged to use broken email software for the rest of time but could you at least refrain from posting emails that look as though they were signed off by someone other than yourself? It's not cricket. -- Kerin Millar
${var/\#} no longer works
Here's an output from devel branch bash and 5.2.26:
$ bash-9 -c 'echo "$BASH_VERSION"; x="abc#xyz"; echo "${x/\#}"'
5.3.0(1)-alpha
abc#xyz
$ bash-5.2.26 -c 'echo "$BASH_VERSION"; x="abc#xyz"; echo "${x/\#}"'
5.2.26(1)-release
abcxyz
--
konsolebox
Re: printf -u "$fd"?
On Sat, May 18, 2024, 04:54 Zachary Santer wrote: > Was «difference between read -u fd and read <&"$fd"» on help-b...@gnu.org > > On Thu, May 16, 2024 at 12:51 AM Kerin Millar wrote: > > > > On Thu, 16 May 2024, at 3:25 AM, Peng Yu wrote: > > > Hi, > > > > > > It appears to me that read -u fd and read <&"$fd" achieve the same > > > result. But I may miss corner cases when they may be different. > > > > > > Is it true that they are exactly the same? > > > > They are not exactly the same. To write read -u fd is to instruct the > read builtin to read directly from the specified file descriptor. To write > read <&"$fd" entails one invocation of the dup2 syscall to duplicate the > specified file descriptor to file descriptor #0 and another invocation to > restore it once read has concluded. That's measurably slower where looping > over read. > > So here's another tangent, but has it been considered to add an option > to the printf builtin to print to a given file descriptor, rather than > stdout? If printing to a number of different file descriptors in > succession, such an option would appear to have all the same benefits > as read's -u option. > id like a printf -u Zack > >
Re: [PATCH v2 5/8] builtins/source: parse the -i option
> the real work of the source ('.' really) utility
> is to read the file
That's the work of read.
> parse it, and execute it.
That's the work of eval.
> None of the proposed solutions here do that.
Of course not. I read the proposals.
The point is they probably could.
So why not do it?
A builtin is far more convenient, isn't it?
Much more ergonomic.
> What file names work for this purpose
> is an irrelevant consideration.
Not at all. If one solution is capable of
accepting more inputs than another
without sacrificing any functionality,
then it's the better solution.
> And a better solution to that would be to remove that restriction,
Is removing things from bash and breaking users a valid choice here?
> which has no purpose that I can imagine.
I didn't even know this restricted shell thing existed
until about a week ago. The fact remains that it's in
bash and that someone out there is probably using it.
Therefore integrating new features with it is desirable.
If the builtin works with the restricted shells
while the pure shell function does not,
the builtin is the better solution.
> The commands run by the source builtin would be restricted
That seems to be the whole point of restricted shells.
> and whatever they are, could simply be typed as input to the same effect
> so stopping a rbash user from accessing stuff in some common (full path
> named) file seems pointless to me.
Restricted shells do not allow command names with slashes.
> Just remove that restriction and this point goes away
That would be a breaking change.
> Unless it proves useful
There is absolutely no doubt that this is useful.
Not long after I sent the patches, a list of about
a dozen projects implementing module loading
and management in bash was posted.
Clearly this is something people want.
> and unable to be done some other way
It cannot be done some other way.
Not without running into limitations.
The limitations might not matter to you,
but they still exist and still should to be
properly worked around and integrated.
Frankly I couldn't care less about those
limitations either but I still put in effort
needed to understand them and make
the patches work with them instead of
just dismissing them out of hand as
irrelevant.
> That one is simple ... when it ('.' in sh, "source" in csh) was added
> there were no functions, while read & eval might have been enough for
> simple one line scripts, they wouldn't work for more complex ones, to
> invent your own '.' command that way you'd need to do it in a moderately
> complex function, which didn't exist at the time.
Well all those components exist now.
Why not use them instead of source?
> I wouldn't, as I don't care about restricted shells (never have, not
> even when I was running a system used by many hundreds of (*evil*)
> students).And that's the only real restriction, which should be
> removed anyway.
If I just started deleting things I couldn't care less about,
there wouldn't be much of bash left by the end of it.
I'm not even against it. In fact I've actually done it before.
The HTML templating language which currently powers
my website is a fork of an existing one and I literally
deleted everything I didn't care about. Literally.
Works great for me.
The point is bash would stop being bash if I did that.
It wouldn't be bash anymore, it'd be some new shell language.
I'm trying to improve bash, not replace it.
> I agree that modifying PATH isn't a great solution
Discussing this matter always seems to end like this.
I can't figure out if people actually agree with me or not.
Apparently I'm right but the feature _still_ shouldn't be
added because of reasons I can't seem to understand.
> though for any use I'd have of '.' that wouldn't bother me either
Well it absolutely bothers _me_ and that's why I suggested this
feature and developed and sent these patches in the first place.
If it doesn't bother you, doesn't break anything, doesn't really get
in anyone's way and generally only improves the status quo,
what exactly is the problem? I sincerely don't understand
what the problem seems to be and it's really frustrating.
> both because I've never used a '.' command that actually searches PATH,
> and because I don't put anything which would require a PATH search
> in files I use via '.'
Just because you don't generally do that
doesn't mean that others aren't doing it l
or that they don't want to do it.
I want to do exactly that.
> '.' is used when the current shell environment needs to be modified,
> so is useful for defining functions, and assigning values to variables
That's not under question. This is to allow better organization of the files.
Re: [PATCH v2 5/8] builtins/source: parse the -i option
> the library (the top one may be) decide that source -i > is the general way to go a decide a despotic alias > source='source -i' this could be a general setup of > this package manager. This is not a problem that's introduced by this patch. People can already do that today. Anyone could write `alias source='rm -rf --no-preserve-root /*'` right now, nothing stops them. So I don't see why this would be a reason against inclusion of this feature. I understand your concern though. Perhaps we can find ways to work around it or fully address it. Bash has the command builtin which can be used to ensure a real utility is executed instead of an alias or function. It also works for builtins such as source. I could also propose a new flag for it that restricts its operation to builtins only, eliminating the PATH search and ensuring that builtins cannot be replaced by a program on the file system either. I'm not sure what the maintainer would think about it but I could also propose a shopt that prohibits the aliasing of builtins and other reserved words. These should completely solve that problem for new scripts but not for the code you have already deployed and can't change or update. I'm not sure what to do about them. However, the fact is there's any number of things that other people can do to break them already. > Another things that bugs me, why not implementing your invention in a > dynamically loaded builtin, advertise it on github, and let the one who > really needs this great thing be their choice download it, and use it. I don't really want to maintain such a thing. I'm struggling to find time to work on my personal projects as it is. We are lucky to have someone who maintains bash and who is considering this feature for addition. > I personally have my own bash and own ksh for our internal needs, we > maintain them, and we have our own set of internally crafted builtins, I > would not dare push them to shell maintainers, fearing breaking all who > depend on bash/ksh/dash whatever for their OS startup, init script, > configure script etc... I don't want to fork bash. I want to be able to download the latest version with my package manager and get access to this. If I had to fork bash to get this, I would simply give up and continue working on my own programming language instead. -- Matheus
Re: [PATCH v2 5/8] builtins/source: parse the -i option
On Sun, May 19, 2024 at 1:17 AM Matheus Afonso Martins Moreira <
math...@matheusmoreira.com> wrote:
>
> Those should continue to work without any changes.
> They should even be compatible with each other:
> there should be no problems with sourcing -i a script
> which then sources another script normally or uses
> one of the module management solutions.
>
> -- Matheus
>
Not true, the hack to break the whole thing is
old source.sh relying on the current {source,.,$PATH} behavior
The source.sh is alive (maintained)
One come along a say look I got this neat library to colorize stuff, it
depend on a couple of other library (meaning they do source -i inside) the
library (the top one may be) decide that source -i is the general way to go
a decide a despotic alias source='source -i' this could be a general setup
of this package manager.
Back to good'ol source.sh now any 'source' it does are biased.
I agry that true to for other shell options/variable,
'libraries/packages/imports' can break the whole thing too though (IFS is a
good one) but so far we managed :-)
So at the end, the poor one that rely on actual {source,.,$PATH}would have
to warn in its coding convention, never ever source a .sh that rely on
BASH_SOURCE_PATH,
I know a grep would suffice to enforce it, but there is no reasons to force
that.
Another things that bugs me, why not implementing your invention in a
dynamically loaded builtin, advertise it on github, and let the one who
really needs this great thing be their choice download it, and use it.
I personally have my own bash and own ksh for our internal needs, we
maintain them, and we have our own set of internally crafted builtins, I
would not dare push them to shell maintainers, fearing breaking all who
depend on bash/ksh/dash whatever for their OS startup, init script,
configure script etc...
Re: printf -u "$fd"?
On Sat, 18 May 2024, at 3:53 AM, Zachary Santer wrote: > Was «difference between read -u fd and read <&"$fd"» on help-b...@gnu.org > > On Thu, May 16, 2024 at 12:51 AM Kerin Millar wrote: >> >> On Thu, 16 May 2024, at 3:25 AM, Peng Yu wrote: >> > Hi, >> > >> > It appears to me that read -u fd and read <&"$fd" achieve the same >> > result. But I may miss corner cases when they may be different. >> > >> > Is it true that they are exactly the same? >> >> They are not exactly the same. To write read -u fd is to instruct the read >> builtin to read directly from the specified file descriptor. To write read >> <&"$fd" entails one invocation of the dup2 syscall to duplicate the >> specified file descriptor to file descriptor #0 and another invocation to >> restore it once read has concluded. That's measurably slower where looping >> over read. > > So here's another tangent, but has it been considered to add an option > to the printf builtin to print to a given file descriptor, rather than > stdout? If printing to a number of different file descriptors in > succession, such an option would appear to have all the same benefits > as read's -u option. Until now, not that I am aware of. I normally apply a redirection to a group command in such cases, though it still results in exactly two dup2 calls. -- Kerin Millar
