Re: [PATCH] ash: make bash_source command search current directory after PATH
On Mon, Jan 29, 2018 at 6:43 AM, Mike Frysinger wrote:
> On 28 Jan 2018 19:17, Denys Vlasenko wrote:
>> On Fri, Jan 26, 2018 at 7:50 PM, Mike Frysinger wrote:
>> > On 26 Jan 2018 15:15, Denys Vlasenko wrote:
>> >> On Fri, Jan 26, 2018 at 9:34 AM, Paul Otto wrote:
>> >> > This patch restores, and improves upon, expected behavior to BASH
>> >> > compatibility which was lost beginning with 1.27.0. This was pulled into
>> >> > Alpine 3.7 which, in turn was pulled into official Docker images
>> >> > beginning
>> >> > with docker:17.12. As a result, a large number of CICD builds that use
>> >> > "source filename" have broken everywhere.
>> >> >
>> >> > According to the BASH documentation, the source command should:
>> >> > Read and execute commands from filename in the current shell
>> >> > environment
>> >> > and return the exit status of the last command executed from filename.
>> >> > If
>> >> > filename does not contain a slash, filenames in PATH are used to
>> >> > find the
>> >> > directory containing filename. The file searched for in PATH need
>> >> > not be
>> >> > executable. When bash is not in posix mode, the current
>> >> > directory is
>> >> > searched if no file is found in PATH.
>> >>
>> >> I wish bash wouldn't introduce gratuitous standard violations.
>> >
>> > bash is its own shell. why can't it introduce its own extensions as it
>> > sees
>> > fit ?
>>
>> Because a divergence is not a good thing, especially if it's gratuitous
>
> this position makes no sense. are you saying that no shell should be allowed
> to extend the syntax however it wants ? every shell out there should only be
> allowed to implement POSIX and nothing else ?
The extensions which add some significantly useful functionality are ok.
Example: arrays.
Changes which don't really add such functionality do more harm than good.
Example: "function" keyword in bash made it possible to do this:
f() { echo "Good"; }
function g { echo "Your script is now not posix compatible,
congratu-effing-lations"; }
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH] ash: make bash_source command search current directory after PATH
On Sun, Jan 28, 2018 at 10:43 PM, Mike Frysinger wrote: > On 28 Jan 2018 19:17, Denys Vlasenko wrote: > > On Fri, Jan 26, 2018 at 7:50 PM, Mike Frysinger wrote: > > > On 26 Jan 2018 15:15, Denys Vlasenko wrote: > > >> On Fri, Jan 26, 2018 at 9:34 AM, Paul Otto wrote: > > >> > This patch restores, and improves upon, expected behavior to BASH > > >> > compatibility which was lost beginning with 1.27.0. This was pulled > into > > >> > Alpine 3.7 which, in turn was pulled into official Docker images > beginning > > >> > with docker:17.12. As a result, a large number of CICD builds that > use > > >> > "source filename" have broken everywhere. > > >> > > > >> > According to the BASH documentation, the source command should: > > >> > Read and execute commands from filename in the current shell > environment > > >> > and return the exit status of the last command executed from > filename. If > > >> > filename does not contain a slash, filenames in PATH are used to > find the > > >> > directory containing filename. The file searched for in PATH > need not be > > >> > executable. When bash is not in posix mode, the current > directory is > > >> > searched if no file is found in PATH. > > >> > > >> I wish bash wouldn't introduce gratuitous standard violations. > > > > > > bash is its own shell. why can't it introduce its own extensions as > it sees > > > fit ? > > > > Because a divergence is not a good thing, especially if it's gratuitous > > this position makes no sense. are you saying that no shell should be > allowed > to extend the syntax however it wants ? every shell out there should only > be > allowed to implement POSIX and nothing else ? > > Based on my understanding of the history of these shells, BASH came up with its syntax before the POSIX standard was ratified. So, while BASH came out after sh, it came out *before* this "standard" was established that some hold to so religiously. > > Making bash "source" behavior non-standard had nothing useful in it. > > "source" is already non-standard and not specified in POSIX. so simply by > using it, your script is not POSIX compliant. That is why, incidentally, I wrote my proposed contribution the way I did initially. In my view, while BASH treats "source" and "." the same, POSIX doesn't allow for "source" so why not have "source" hold to the BASH standard and "." hold to the POSIX standard? I definitely caved too quickly on that point, and wound up with my contribution being swallowed up into a patch that did the exact opposite of my intent. ¯\_(ツ)_/¯ ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH] ash: make bash_source command search current directory after PATH
I really want to argue that there is a quite simple workaround to get the
bash 'source' behavior you need: Just append ":." to the PATH environment
variable before you run the "." command.
Or this:
source () {
PATH="$PATH:." . "$@"
}
Was there any difficulty of doing it?
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH] ash: make bash_source command search current directory after PATH
Hello, On Sat, Jan 27, 2018 at 3:05 PM, Paul Otto wrote: > Thanks for taking care of this, Denys. It didn't wind up the way I'd hoped, > but at least it is predictable both ways now. I will see if there is a way > to get Alpine Linux to build with that config option set, to resolve the > regression introduced by busybox 1.27. This is done in 3.7 and edge as busybox-1.27.2-r8. Relevant commit: https://git.alpinelinux.org/cgit/aports/commit/?id=fae8e63b4df5c1052fc79ed551d442263ddcb523 William ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH] ash: make bash_source command search current directory after PATH
Hello, On Mon, Jan 29, 2018 at 4:42 PM, Paul Otto wrote: > > > >> > Making bash "source" behavior non-standard had nothing useful in it. >> >> "source" is already non-standard and not specified in POSIX. so simply by >> using it, your script is not POSIX compliant. > > > That is why, incidentally, I wrote my proposed contribution the way I did > initially. In my view, while BASH treats "source" and "." the same, POSIX > doesn't allow for "source" so why not have "source" hold to the BASH > standard and "." hold to the POSIX standard? I definitely caved too quickly > on that point, and wound up with my contribution being swallowed up into a > patch that did the exact opposite of my intent. ¯\_(ツ)_/¯ > > This is probably due to the fact that many (for instance, I) expect . and source to behave the same way :) Having them behaving differently would somewhat violate the principle of least surprise :) BR, -- Emmanuel Deloget ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH] ash: make bash_source command search current directory after PATH
Thanks for the heads up, William! Great to hear! Thanks, Paul On Jan 29, 2018 8:49 AM, "William Pitcock" wrote: > Hello, > > On Sat, Jan 27, 2018 at 3:05 PM, Paul Otto wrote: > > Thanks for taking care of this, Denys. It didn't wind up the way I'd > hoped, > > but at least it is predictable both ways now. I will see if there is a > way > > to get Alpine Linux to build with that config option set, to resolve the > > regression introduced by busybox 1.27. > > This is done in 3.7 and edge as busybox-1.27.2-r8. Relevant commit: > https://git.alpinelinux.org/cgit/aports/commit/?id= > fae8e63b4df5c1052fc79ed551d442263ddcb523 > > William > ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH] ash: make bash_source command search current directory after PATH
On 29 Jan 2018 14:41, Denys Vlasenko wrote:
> On Mon, Jan 29, 2018 at 6:43 AM, Mike Frysinger wrote:
> > On 28 Jan 2018 19:17, Denys Vlasenko wrote:
> >> On Fri, Jan 26, 2018 at 7:50 PM, Mike Frysinger wrote:
> >> > On 26 Jan 2018 15:15, Denys Vlasenko wrote:
> >> >> On Fri, Jan 26, 2018 at 9:34 AM, Paul Otto wrote:
> >> >> > This patch restores, and improves upon, expected behavior to BASH
> >> >> > compatibility which was lost beginning with 1.27.0. This was pulled
> >> >> > into
> >> >> > Alpine 3.7 which, in turn was pulled into official Docker images
> >> >> > beginning
> >> >> > with docker:17.12. As a result, a large number of CICD builds that use
> >> >> > "source filename" have broken everywhere.
> >> >> >
> >> >> > According to the BASH documentation, the source command should:
> >> >> > Read and execute commands from filename in the current shell
> >> >> > environment
> >> >> > and return the exit status of the last command executed from
> >> >> > filename. If
> >> >> > filename does not contain a slash, filenames in PATH are used to
> >> >> > find the
> >> >> > directory containing filename. The file searched for in PATH need
> >> >> > not be
> >> >> > executable. When bash is not in posix mode, the current
> >> >> > directory is
> >> >> > searched if no file is found in PATH.
> >> >>
> >> >> I wish bash wouldn't introduce gratuitous standard violations.
> >> >
> >> > bash is its own shell. why can't it introduce its own extensions as it
> >> > sees
> >> > fit ?
> >>
> >> Because a divergence is not a good thing, especially if it's gratuitous
> >
> > this position makes no sense. are you saying that no shell should be
> > allowed
> > to extend the syntax however it wants ? every shell out there should only
> > be
> > allowed to implement POSIX and nothing else ?
>
> The extensions which add some significantly useful functionality are ok.
> Example: arrays.
>
> Changes which don't really add such functionality do more harm than good.
> Example: "function" keyword in bash made it possible to do this:
in general, i think you're applying 20/20 hindsight to things. it's easy to
say something is a bad idea years after the fact with experience in hand.
also, i don't think the function keyword originated in bash. the internet
suggests it originated in the korn shell and was picked up by other shells.
also, i know you know this, but POSIX isn't a spec that leads the real world.
it exists based on the corpus of existing implementations, including new
functions and features. so unless people try out new things and get others
to adopt them, then POSIX isn't going to change either. that includes trying
things that fail.
> f() { echo "Good"; }
> function g { echo "Your script is now not posix compatible,
> congratu-effing-lations"; }
some people like that extra syntatic sugar because it makes scanning code
and identifying functions easier (to their eyes). i've met a few people
now who argue you should always keep the "function" keyword.
-mike
signature.asc
Description: Digital signature
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH] ash: make bash_source command search current directory after PATH
On 29 Jan 2018 08:42, Paul Otto wrote: > On Sun, Jan 28, 2018 at 10:43 PM, Mike Frysinger wrote: > > On 28 Jan 2018 19:17, Denys Vlasenko wrote: > > > Making bash "source" behavior non-standard had nothing useful in it. > > > > "source" is already non-standard and not specified in POSIX. so simply by > > using it, your script is not POSIX compliant. > > That is why, incidentally, I wrote my proposed contribution the way I did > initially. In my view, while BASH treats "source" and "." the same, POSIX > doesn't allow for "source" so why not have "source" hold to the BASH > standard and "." hold to the POSIX standard? I definitely caved too quickly > on that point, and wound up with my contribution being swallowed up into a > patch that did the exact opposite of my intent. ¯\_(ツ)_/¯ of course POSIX allows for "source". it would be crazy to have it specify "anything not explicitly permitted in this specification is banned". it does what it does everywhere else: everything not specified is "unspecified" and thus implementations are free to do what they want. although that is a generality. in the specific case of "source", POSIX has carved out that keyword as both reserved and unspecified: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_01_01 meaning that POSIX recognizes that some shells implement it in a semi-consistent manner, but not enough (yet) to standardize it like it has with other builtins. -mike signature.asc Description: Digital signature ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH 0/4] Add SCHED_BATCH and SCHED_IDLE support to chrt
Am 28.01.2018 19:23, schrieb Denys Vlasenko: > On Sun, Jan 28, 2018 at 11:10 AM, walter harms wrote: >>> On Wed, Jan 24, 2018 at 1:28 AM, Povilas Kanapickas >>> wrote: >> The following patches add SCHED_BATCH and SCHED_IDLE support to chrt. >> The priority limits are fixed to follow the specification. The last >> patch avoids hardcoding the values of SCHED_* macros as array indices. >> Perhaps counter-intuitively, this leads to binary size reduction of >> 90 bytes on x86-64. However the patch series as a whole still increase >> the binary size by 173 bytes on x86-64. > > Applied patches 1 and 2. Fixed a problem of patch 3 a bit differently. > Patch 4 increases the code, not applied. Thanks a lot. I'd like to know what I missed with regards to code size increase. Which architectures do you check the code size on? Is there anything else I should've known? Maybe the size of the text segment carries more weight than e.g. rodata or something similar? >>> >>> This may depend on toolchain and libc. In my test, with my toolchain, >>> it was not a win. >> >> Hi Denys, >> maybe i missed it somewhere but did you provide some numbers ? >> Did it grow for one byte or kilobytes ? > > ~40 bytes out of ~850 bytes in chrt.o > > function old new delta > get_policy_name- 57 +57 > show_min_max 59 64 +5 > chrt_main432 429 -3 > policies 72 - -72 > -- > (add/remove: 1/1 grow/shrink: 1/1 up/down: 62/-75)Total: -13 bytes >text databssdechexfilename > 933375481 6856 940712 e5aa8busybox_old > 933417481 6856 940754 e5ad2busybox_unstripped > thx for Info ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH] ash: make bash_source command search current directory after PATH
On Mon, Jan 29, 2018 at 5:55 PM, Mike Frysinger wrote: >> >> >> I wish bash wouldn't introduce gratuitous standard violations. >> >> > >> >> > bash is its own shell. why can't it introduce its own extensions as it >> >> > sees >> >> > fit ? >> >> >> >> Because a divergence is not a good thing, especially if it's gratuitous >> > >> > this position makes no sense. are you saying that no shell should be >> > allowed >> > to extend the syntax however it wants ? every shell out there should only >> > be >> > allowed to implement POSIX and nothing else ? >> >> The extensions which add some significantly useful functionality are ok. >> Example: arrays. >> >> Changes which don't really add such functionality do more harm than good. >> Example: "function" keyword in bash made it possible to do this: > > in general, i think you're applying 20/20 hindsight to things. it's easy to > say something is a bad idea years after the fact with experience in hand. The general observation that "compatibility is a good thing and should not be broken just for the fun of it" is very old. It's _good_ when you move your stuff from system to system, from distro to distro, from one OS to another and things not break. Or at least break not too severely. Unfortunately, every new generation of comp-sci students needs to learn it the hard way: by breaking it and then feeling the pain. I still wonder how on Earth Android filesystem ended up not having /bin/sh. No one at Google realized that people are using shell scripts all over the Unix universe? Or they seriously did not want people to run non-Android specific stuff on Android, and *wanted* non-android-tailored shell scripts to fail to execute? ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH] ash: make bash_source command search current directory after PATH
On 29 Jan 2018 18:52, Denys Vlasenko wrote: > On Mon, Jan 29, 2018 at 5:55 PM, Mike Frysinger wrote: > >> >> >> I wish bash wouldn't introduce gratuitous standard violations. > >> >> > > >> >> > bash is its own shell. why can't it introduce its own extensions as > >> >> > it sees > >> >> > fit ? > >> >> > >> >> Because a divergence is not a good thing, especially if it's gratuitous > >> > > >> > this position makes no sense. are you saying that no shell should be > >> > allowed > >> > to extend the syntax however it wants ? every shell out there should > >> > only be > >> > allowed to implement POSIX and nothing else ? > >> > >> The extensions which add some significantly useful functionality are ok. > >> Example: arrays. > >> > >> Changes which don't really add such functionality do more harm than good. > >> Example: "function" keyword in bash made it possible to do this: > > > > in general, i think you're applying 20/20 hindsight to things. it's easy to > > say something is a bad idea years after the fact with experience in hand. > > The general observation that "compatibility is a good thing > and should not be broken just for the fun of it" is very old. > It's _good_ when you move your stuff from system to system, > from distro to distro, from one OS to another and things not break. > Or at least break not too severely. again, 20/20 hindsight. you're applying this to a time when there was no real thing as compatibility: compatible with who ? this other random project that just started a year or two ago ? why are they better ? there were no distros to move between, and every system was custom built. it's why POSIX is such a mess of permission "implementation defined" cases as it tried to bring some level of sanity to different systems. > Unfortunately, every new generation of comp-sci students > needs to learn it the hard way: by breaking it and then feeling the pain. or they come out with something so much better that people move away from the previous thing entirely and it dies. > I still wonder how on Earth Android filesystem ended up not having /bin/sh. > No one at Google realized that people are using shell scripts > all over the Unix universe? Or they seriously did not want people > to run non-Android specific stuff on Android, and *wanted* > non-android-tailored shell scripts to fail to execute? i think it's more like you shouldn't be running shell the vast majority of the time. systemd showed the numbers to back this up. so on a resource constrained system (like Android phones, especially the first ones from over a decade ago), no shell is better as it forces people to use a constrained language. in this case, the init "exec" stuff, or a compiled language like C or C++. https://android.googlesource.com/platform/system/core/+/master/init/README.md -mike signature.asc Description: Digital signature ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH] ash: make bash_source command search current directory after PATH
On Mon, Jan 29, 2018 at 7:50 PM, Mike Frysinger wrote: >> I still wonder how on Earth Android filesystem ended up not having /bin/sh. >> No one at Google realized that people are using shell scripts >> all over the Unix universe? Or they seriously did not want people >> to run non-Android specific stuff on Android, and *wanted* >> non-android-tailored shell scripts to fail to execute? > > i think it's more like you shouldn't be running shell the vast majority of > the time. systemd showed the numbers to back this up. We should remove all shell scripts from e.g. kernel, gcc and busybox build machinery. Because systemd. Right. ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: ASH: why ash_ptr_to_globals_misc is declared as const pointer
On Fri, Jan 26, 2018 at 8:39 PM, Yunlian Jiang wrote:
> I will use a similar issue in lineedit.c to explain the problem. (it is
> easier to reproduce).
> The problem is that
> clang thinks lineedit_ptr_to_statics is a constant pointer, so the pointer
> should be unchanged. As a result, inside a function, it
> loads the value once, and it can use it as many as it want without worrying
> anthing changes.
> In the macro
>
> #define INIT_S() do { \
> (*(struct lineedit_statics**)&lineedit_ptr_to_statics) =
> xzalloc(sizeof(S)); \
> barrier(); \
> cmdedit_termw = 80; \
> IF_USERNAME_OR_HOMEDIR(home_pwd_buf = (char*)null_str;) \
> IF_FEATURE_EDITING_VI(delptr = delbuf;) \
> } while (0)
>
> (*(struct lineedit_statics**)&lineedit_ptr_to_statics) =
> xzalloc(sizeof(S));
> actually changes value of the pointer (from 0x0 to the return value of
> xzalloc). But clang did not realize that, otherwise, it
> should omit a compiler time error, so clang thinks it has nothing to do with
> lineedit_ptr_to_statics
>
> in the statement
> IF_FEATURE_EDITING_VI(delptr = delbuf;), clang still assumes the value of
> the pointer is '0x0', so the segfaults happens.
How clang knows that pointer was NULL? It's an extern, clang can't
infer its value
because it doesn't see it, right?
Can you post "make libbb/lineedit.s" result?
Can you experiment with better barriers?
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH] ash: make bash_source command search current directory after PATH
On 30 Jan 2018 04:00, Denys Vlasenko wrote: > On Mon, Jan 29, 2018 at 7:50 PM, Mike Frysinger wrote: > >> I still wonder how on Earth Android filesystem ended up not having /bin/sh. > >> No one at Google realized that people are using shell scripts > >> all over the Unix universe? Or they seriously did not want people > >> to run non-Android specific stuff on Android, and *wanted* > >> non-android-tailored shell scripts to fail to execute? > > > > i think it's more like you shouldn't be running shell the vast majority of > > the time. systemd showed the numbers to back this up. > > We should remove all shell scripts from e.g. kernel, gcc and busybox > build machinery. Because systemd. Right. i don't know what point you're trying to make, but drop the useless hyperbole. you asked about runtime Android, not about random dev packages. plus i was referring to hard general data (replacing shell scripts with compiled programs is proven benefit to boot time), not advocating for systemd. -mike signature.asc Description: Digital signature ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: ASH: why ash_ptr_to_globals_misc is declared as const pointer
On 30 Jan 2018 04:10, Denys Vlasenko wrote:
> On Fri, Jan 26, 2018 at 8:39 PM, Yunlian Jiang wrote:
> > I will use a similar issue in lineedit.c to explain the problem. (it is
> > easier to reproduce).
> > The problem is that
> > clang thinks lineedit_ptr_to_statics is a constant pointer, so the pointer
> > should be unchanged. As a result, inside a function, it
> > loads the value once, and it can use it as many as it want without worrying
> > anthing changes.
> > In the macro
> >
> > #define INIT_S() do { \
> > (*(struct lineedit_statics**)&lineedit_ptr_to_statics) =
> > xzalloc(sizeof(S)); \
> > barrier(); \
> > cmdedit_termw = 80; \
> > IF_USERNAME_OR_HOMEDIR(home_pwd_buf = (char*)null_str;) \
> > IF_FEATURE_EDITING_VI(delptr = delbuf;) \
> > } while (0)
> >
> > (*(struct lineedit_statics**)&lineedit_ptr_to_statics) =
> > xzalloc(sizeof(S));
> > actually changes value of the pointer (from 0x0 to the return value of
> > xzalloc). But clang did not realize that, otherwise, it
> > should omit a compiler time error, so clang thinks it has nothing to do with
> > lineedit_ptr_to_statics
> >
> > in the statement
> > IF_FEATURE_EDITING_VI(delptr = delbuf;), clang still assumes the value of
> > the pointer is '0x0', so the segfaults happens.
>
> How clang knows that pointer was NULL? It's an extern, clang can't
> infer its value
> because it doesn't see it, right?
i think he means is that clang loaded the pointer before the assignment/barrier
and it wasn't reloaded or delayed after the barrier, so it started out NULL and
stayed that way.
but i'm guessing ... the .s output would help as you suggested.
-mike
signature.asc
Description: Digital signature
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
