Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On 2017-03-02 11:45, Eric Blake wrote: > On 03/02/2017 11:28 AM, Brian Inglis wrote: >> summary of checkbashisms -f run on 140 POSIX shell scripts as >> identified by file in my Cygwin 64 /bin/ with 784 packages (about >> 20% of cygwin-pkg-maint, 8% of setup.ini) installed, 70 possible >> issues (some from wrapped scripts because of -f): > > Looking through some of the output, I already spot a few interesting > things: > >> possible bashism in /bin/autoconf-2.69 line 55 ($BASH_SOMETHING): >> if test -z "$BASH_VERSION$ZSH_VERSION" \ >> && (test "X`print -r -- $as_echo`" = "X$as_echo") 2>/dev/null; then > > Not a problem - the script is probing whether it needs to do > something based on being bash or not, but is written with all shells > in mind. > >> error: /bin/autoconf-2.69: Unterminated quoted string found, EOF >> reached. Wanted: <'>, opened in line 429 > > bug in checkbashisms; looks like it happens in several places in the > output Because of -f checks shell wrappers which it would otherwise just ignore - may not be helpful - but -px (POSIX + eXtra) may be more useful, as checkbashisms defaults to lintian checks where Debian Policy 10.4 overrides POSIX. Also running with -f on bash scripts and checking for exit 4 identifies candidates for changing shebang from bash to sh. >> possible bashism in /bin/autopoint line 57 ($RANDOM): >> tmp=$TMPDIR/gt$$-$RANDOM > Not a problem - the script is using $RANDOM for an additional level > of safety, but works safely even when $RANDOM is the empty string (as > on dash). Those are where I suggested mktemp would be a better solution to provide a random string - most scripts using this open multiple temp files and are part of or used by autotools which may be run on MPs. >> possible bashism in /bin/bzgrep line 66 (${parm/?/pat[/str]}): >> j=${i//\\/} > > First real problem. Except that 'grep j= /bin/bzgrep' has no hits on my > system with bzip2-1.0.6-2 installed; what version was this problem in? Cygwin patch release 1.0.6-3 installed 2017-02-24 but unchanged since 2007: $ ls -ghlo bzip2-1.0.6-3.src/bzip2-1.0.6/bzgrep -rw-r--r--+ 1 1.7K Jan 2 2007 bzip2-1.0.6-3.src/bzip2-1.0.6/bzgrep $ file /bin/bz{cmp,diff,less,more,*grep} # scripts from /etc/setup/bzip2.lst.gz /bin/bzcmp: symbolic link to bzdiff /bin/bzdiff: POSIX shell script, ASCII text executable /bin/bzless: symbolic link to bzmore /bin/bzmore: POSIX shell script, ASCII text executable /bin/bzegrep: symbolic link to bzgrep /bin/bzfgrep: symbolic link to bzgrep /bin/bzgrep: POSIX shell script, ASCII text executable $ fgrep -C9 j= bzip2-1.0.6-3.src/bzip2-1.0.6/bzgrep for i do if test -f "$i"; then :; else if test -f "$i.bz2"; then i="$i.bz2"; fi; fi if test $list -eq 1; then bzip2 -cdfq "$i" | $grep $opt "$pat" 2>&1 > /dev/null && echo $i r=$? elif test $# -eq 1 -o $silent -eq 1; then bzip2 -cdfq "$i" | $grep $opt "$pat" r=$? else j=${i//\\/} j=${j//|/\\|} j=${j//&/\\&} j=`printf "%s" "$j" | tr '\n' ' '` bzip2 -cdfq "$i" | $grep $opt "$pat" | sed "s|^|${j}:|" r=$? fi test "$r" -ne 0 && res="$r" done exit $res Judgement call whether to change shebang or spawn sed or other? Debian release is 1.0.6-7, bzgrep dated 2014, and bash expressions are gone, thanks to lintian and changes made to support more newer grep options. >> possible bashism in /bin/messagebus-config line 26 (should be '.', >> not 'source'): >> source ${CSIH_SCRIPT} > > Ah - now we get to an example file that is Cygwin-only (part of the > Cygwin port of dbus, but reading that file makes it obvious that it > is not from upstream) - where changing /bin/sh will cause breakage > that we can't rely on Debian to have caught first. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On 03/02/2017 11:28 AM, Brian Inglis wrote: > summary of checkbashisms -f run on 140 POSIX shell scripts as > identified by file in my Cygwin 64 /bin/ with 784 packages (about 20% > of cygwin-pkg-maint, 8% of setup.ini) installed, 70 possible issues > (some from wrapped scripts because of -f): Looking through some of the output, I already spot a few interesting things: > possible bashism in /bin/autoconf-2.69 line 55 ($BASH_SOMETHING): > if test -z "$BASH_VERSION$ZSH_VERSION" \ > && (test "X`print -r -- $as_echo`" = "X$as_echo") 2>/dev/null; then Not a problem - the script is probing whether it needs to do something based on being bash or not, but is written with all shells in mind. > error: /bin/autoconf-2.69: Unterminated quoted string found, EOF reached. > Wanted: <'>, opened in line 429 bug in checkbashisms; looks like it happens in several places in the output > possible bashism in /bin/autopoint line 57 ($RANDOM): > tmp=$TMPDIR/gt$$-$RANDOM Not a problem - the script is using $RANDOM for an additional level of safety, but works safely even when $RANDOM is the empty string (as on dash). > possible bashism in /bin/bzgrep line 66 (${parm/?/pat[/str]}): > j=${i//\\/} First real problem. Except that 'grep j= /bin/bzgrep' has no hits on my system with bzip2-1.0.6-2 installed; what version was this problem in? > possible bashism in /bin/messagebus-config line 26 (should be '.', not > 'source'): > source ${CSIH_SCRIPT} Ah - now we get to an example file that is Cygwin-only (part of the Cygwin port of dbus, but reading that file makes it obvious that it is not from upstream) - where changing /bin/sh will cause breakage that we can't rely on Debian to have caught first. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
Eric Blake writes: > To reiterate my answer in different terms: > > If you can convince Fedora to switch /bin/sh to dash, then I will > immediately follow in Cygwin. Until then, I'm worried that there are > enough scripts in the wild that use bashisms and will therefore break if > /bin/sh is not bash, even though that number has reduced somewhat since > Debian made their switch. Trying to make Cygwin the guinea pig, instead > of Fedora, is going about it backwards (you WANT the change to be done > in a place where there is plenty of manpower to deal with the fallout, > and Fedora has more manpower than Cygwin). Ack. I want to put forward that we should get the postinstall scripts moved from bash to dash as a first step even then. > I'm still toying with the idea of doing a test release of both bash and > dash that flips /bin/sh between them; but I'm still stuck on the problem > that a user MUST upgrade (or downgrade) both packages in tandem, or else > risk being left without a /bin/sh at all. Help would be appreciated in > figuring out the problem (telling me that "dash is faster than bash" is > not help, nor is telling me that "portable shell scripts don't care if > /bin/sh is bash or dash" - I already know those points. What I don't > know is how many non-portable scripts are out there, so how much > breakage would I be causing by forcing those non-portable scripts to > deal with their non-portability, and how to minimize the even-worse > breakage of an upgrade scenario that leaves no /bin/sh at all). I'd test that in production. I don't really need handholding w.r.t. update and potential downgrade process, so if you'd put the two packages somewhere into an unofficial repo that would work for me. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Q+, Q and microQ: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On 2017-03-02 07:29, Eric Blake wrote: > On 03/02/2017 07:36 AM, Marco Atzeri wrote: >> On 02/03/2017 13:36, Steven Penny wrote: >>> On Wed, 1 Mar 2017 23:31:24, Vince Rice wrote: Then you haven't been paying attention. And I didn't even attempt to make an argument one way or the other, except to say stop arguing. The horse is dead. >>> >>> Perhaps you could link to a constructive, concrete idea against >>> the change that someone has made besides Eric. Even better, you >>> could post your own constructive idea; surely you havent emailed >>> twice now with nothing constructive to add? >> >> He was constructive, but you seems biased in understanding the answer. > > To reiterate my answer in different terms: > > If you can convince Fedora to switch /bin/sh to dash, then I will > immediately follow in Cygwin. Until then, I'm worried that there are > enough scripts in the wild that use bashisms and will therefore break if > /bin/sh is not bash, even though that number has reduced somewhat since > Debian made their switch. Trying to make Cygwin the guinea pig, instead > of Fedora, is going about it backwards (you WANT the change to be done > in a place where there is plenty of manpower to deal with the fallout, > and Fedora has more manpower than Cygwin). I can see distros like *BSD or Debian making decisions like that or systemd, but not vendors like RH or Canonical: too little upside with too much downside for paying customers. > I'm still toying with the idea of doing a test release of both bash > and dash that flips /bin/sh between them; but I'm still stuck on the > problem that a user MUST upgrade (or downgrade) both packages in > tandem, or else risk being left without a /bin/sh at all. Help would > be appreciated in figuring out the problem (telling me that "dash is > faster than bash" is not help, nor is telling me that "portable shell > scripts don't care if /bin/sh is bash or dash" - I already know those > points. What I don't know is how many non-portable scripts are out > there, so how much breakage would I be causing by forcing those > non-portable scripts to deal with their non-portability, and how to > minimize the even-worse breakage of an upgrade scenario that leaves > no /bin/sh at all). summary of checkbashisms -f run on 140 POSIX shell scripts as identified by file in my Cygwin 64 /bin/ with 784 packages (about 20% of cygwin-pkg-maint, 8% of setup.ini) installed, 70 possible issues (some from wrapped scripts because of -f): 14 /bin/cronbug 7 /bin/procmail-config 5 /bin/ca-legacy 4 /bin/pcdovtoppm 3 /bin/ppmshadow 3 /bin/ppmfade 3 /bin/fig2ps2tex 3 /bin/bzgrep 2 /bin/ppmrainbow 2 /bin/pnmquant 2 /bin/pnmflip 2 /bin/install-catalog 1 /bin/zgrep 1 /bin/zdiff 1 /bin/xzdiff 1 /bin/updatedb 1 /bin/texi2dvi 1 /bin/ppmquant 1 /bin/pnmquantall 1 /bin/pic2graph 1 /bin/pgmcrater 1 /bin/pdfroff 1 /bin/messagebus-config 1 /bin/gzexe 1 /bin/grap2graph 1 /bin/gettextize 1 /bin/gettext.sh 1 /bin/eqn2graph 1 /bin/clisp-link 1 /bin/ccmakedep 1 /bin/autopoint 1 /bin/autoconf-2.69 with diagnostics (some from wrapped scripts because of -f): 16 echo -e 12 '((' should be '$((' 7 alternative test command ([[ foo ]] should be [ foo ]) 7 $RANDOM 6 unsafe echo with backslash 5 'function' is useless 4 type 4 bash arrays, ${name[0|*|@]} 3 ${parm/?/pat[/str]} 3 $_ 1 should be '.', not 'source' 1 $BASH_SOMETHING 1 $(OS|MACH)TYPE So about 20% of packages have scripts installed, with potential issues ranging from 5-12*70 == 350-840. Some potential fixes are obvious from my sample (attached, hopefully as inline text which will pass thru the filters): - change these scripts to shebang #!/bin/bash - replace echo -e with printf in most cases - replace [[ ... ]] with [ ... ] or case if patterns are used - strip leading function - use mktemp to eliminate $RANDOM but most require judgement in application. Generated scripts like libtool{,ize} may require complex patches submitted upstream to fix what may be sh dependent code generation or execution; not included in the attached sample, as file categorized these as: /bin/libtool:POSIX shell script, ASCII text executable, with very long lines, with escape sequences /bin/libtoolize: POSIX shell script, ASCII text executable, with very long lines, with escape sequences > Hmm, maybe I could create a NEW package, 'sh', which packages /bin/sh as > however I want it (probably bash to begin with, to at least give people > time to upgrade and pick up the packaging change before also having to > deal with any shell changing). New releases of both bash and dash would > depend on the new package, to guarantee that if you upgrade one shell, > you pick up the dependency. And by not
RE: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
From: Eric Blake > ... > Until then, I'm worried that there are > enough scripts in the wild that use bashisms and will therefore break if > /bin/sh is not bash, even though that number has reduced somewhat since > Debian made their switch. > ... I was thinking of testing my scripts by changing sh to be a link to dash, but ran into the following: $ which sh bash dash /usr/bin/sh /usr/bin/bash /usr/bin/dash $ ls -ld /usr/bin/*sh lrwxrwxrwx 1 knellis Domain Users8 Dec 1 11:38 /usr/bin/csh -> tcsh.exe -rwxr-xr-x 1 knellis Domain Users 4287 Jan 15 07:53 /usr/bin/instmodsh lrwxrwxrwx 1 knellis Domain Users 12 May 26 2015 /usr/bin/tclsh -> tclsh8.5.exe $ Why don't I see sh, bash, or dash? But then I tried: $ ls -ld /usr/bin/*sh.exe -rwxr-xr-x 2 knellis Domain Users 100883 Feb 17 08:47 /usr/bin/ash.exe -rwxr-xr-x 2 knellis Domain Users 739859 Jan 27 14:13 /usr/bin/bash.exe -rwxr-xr-x 2 knellis Domain Users 100883 Feb 17 08:47 /usr/bin/dash.exe -rwxr-xr-x 1 knellis Domain Users 11795 Aug 18 2013 /usr/bin/rsh.exe -rwxr-xr-x 2 knellis Domain Users 739859 Jan 27 14:13 /usr/bin/sh.exe -rwxr-xr-x 1 knellis Domain Users 650771 Dec 20 06:57 /usr/bin/ssh.exe -rwxr-xr-x 1 knellis Domain Users 372755 Nov 24 11:07 /usr/bin/tcsh.exe $ Another unexpected result from Cygwin's .exe magic. --Ken Nellis
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On 2017-03-02 06:36, Marco Atzeri wrote: > On 02/03/2017 13:36, Steven Penny wrote: >> On Wed, 1 Mar 2017 23:31:24, Vince Rice wrote: >>> Then you haven't been paying attention. And I didn't even attempt >>> to make an argument one way or the other, except to say stop >>> arguing. The horse is dead. >> >> Perhaps you could link to a constructive, concrete idea against >> the change that someone has made besides Eric. Even better, you >> could post your own constructive idea; surely you havent emailed >> twice now with nothing constructive to add? > > He was constructive, but you seems biased in understanding the > answer. > > ~ 70% of (weighted*) package managers rejected the proposal > already here: > > https://cygwin.com/ml/cygwin/2017-02/msg00310.html > https://cygwin.com/ml/cygwin/2017-02/msg00295.html > > And I agree with them; we have no bandwith/resources > to handle it. That makes it over 75% and we can appreciate your efforts and viewpoint. I will just have to continue using my Cygwin setup check and fixup profile script. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On 03/02/2017 07:36 AM, Marco Atzeri wrote: > On 02/03/2017 13:36, Steven Penny wrote: >> On Wed, 1 Mar 2017 23:31:24, Vince Rice wrote: >>> Then you haven't been paying attention. >>> And I didn't even attempt to make an argument one way or the other,=20 >>> except to say stop arguing. The horse is dead.= >> >> Perhaps you could link to a constructive, concrete idea against the >> change that >> someone has made besides Eric. Even better, you could post your own >> constructive >> idea; surely you havent emailed twice now with nothing constructive to >> add? > > He was constructive, but you seems biased in understanding the answer. To reiterate my answer in different terms: If you can convince Fedora to switch /bin/sh to dash, then I will immediately follow in Cygwin. Until then, I'm worried that there are enough scripts in the wild that use bashisms and will therefore break if /bin/sh is not bash, even though that number has reduced somewhat since Debian made their switch. Trying to make Cygwin the guinea pig, instead of Fedora, is going about it backwards (you WANT the change to be done in a place where there is plenty of manpower to deal with the fallout, and Fedora has more manpower than Cygwin). I'm still toying with the idea of doing a test release of both bash and dash that flips /bin/sh between them; but I'm still stuck on the problem that a user MUST upgrade (or downgrade) both packages in tandem, or else risk being left without a /bin/sh at all. Help would be appreciated in figuring out the problem (telling me that "dash is faster than bash" is not help, nor is telling me that "portable shell scripts don't care if /bin/sh is bash or dash" - I already know those points. What I don't know is how many non-portable scripts are out there, so how much breakage would I be causing by forcing those non-portable scripts to deal with their non-portability, and how to minimize the even-worse breakage of an upgrade scenario that leaves no /bin/sh at all). Hmm, maybe I could create a NEW package, 'sh', which packages /bin/sh as however I want it (probably bash to begin with, to at least give people time to upgrade and pick up the packaging change before also having to deal with any shell changing). New releases of both bash and dash would depend on the new package, to guarantee that if you upgrade one shell, you pick up the dependency. And by not having /bin/sh in either the bash or dash package, then we would at least avoid the current situation where upgrading/downgrading in the wrong order could leave a user without /bin/sh at all. You might still be in a situation where the wrong version of the 'sh' package leaves you with an outdated version of a shell, or the wrong flavor in relation to the current distro choice, but that's less of a problem than having no /bin/sh at all. In fact, having a separate 'sh' package may make it even easier to pick which shell flavor you prefer (if I always keep the 'curr' and 'test' versions pointed to different shells, then you make the choice of which sh flavor you want by which version you install). -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On 02/03/2017 13:36, Steven Penny wrote: On Wed, 1 Mar 2017 23:31:24, Vince Rice wrote: Then you haven't been paying attention. And I didn't even attempt to make an argument one way or the other,=20 except to say stop arguing. The horse is dead.= Perhaps you could link to a constructive, concrete idea against the change that someone has made besides Eric. Even better, you could post your own constructive idea; surely you havent emailed twice now with nothing constructive to add? He was constructive, but you seems biased in understanding the answer. ~ 70% of (weighted*) package managers rejected the proposal already here: https://cygwin.com/ml/cygwin/2017-02/msg00310.html https://cygwin.com/ml/cygwin/2017-02/msg00295.html And I agree with them; we have no bandwith/resources to handle it. Regards Marco * https://www.cygwin.com/cygwin-pkg-maint -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On Wed, 1 Mar 2017 23:31:24, Vince Rice wrote: Then you haven't been paying attention. And I didn't even attempt to make an argument one way or the other,=20 except to say stop arguing. The horse is dead.= Perhaps you could link to a constructive, concrete idea against the change that someone has made besides Eric. Even better, you could post your own constructive idea; surely you havent emailed twice now with nothing constructive to add? -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
> On Mar 1, 2017, at 11:27 PM, Steven Penny wrote: > > On Wed, 1 Mar 2017 21:46:38, Vince Rice wrote: >> There are valid reasons (which several others have made) not to force others >> to use it. > > Have they though? I have not seen anyone save Eric (including you) make a > valid > argument beyond "I like it the old way". Then you haven't been paying attention. And I didn't even attempt to make an argument one way or the other, except to say stop arguing. The horse is dead. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On Wed, 1 Mar 2017 21:46:38, Vince Rice wrote: There are valid reasons (which several others have made) not to force others to use it. Have they though? I have not seen anyone save Eric (including you) make a valid argument beyond "I like it the old way". -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
> On Mar 1, 2017, at 6:22 PM, Steven Pennywrote: > > On Wed, 1 Mar 2017 09:42:27, cyg Simple wrote: >> Oh but it is. We're discussing the bike shed named /bin/sh. >> One of the color names is bash the other is dash; it's still the same >> bike shed. > > I see, when you realize your argument does not hold water, you resort to name > calling. Here are concrete arguments: > > 1. Debian uses Dash as /bin/sh > 2. Ubuntu uses Dash as /bin/sh > 3. The POSIX standard defines sh, and by definition Dash is closer to this > than > Bash > 4. Dash is 3-5 times faster than Bash > 5. While it might be upsetting to some users, _they_ have made an error in > assuming Bash is /bin/sh, it would not be our error in switching to a shell > that is closer to POSIX sh > > Now, perhaps you can add a constructive post, or do you want to start with > some > "yo momma" jokes? He didn't call anyone names. He said that was a bike shed argument. Which is getting more accurate the longer this "discussion" goes on. You've argued your point. If you like dash, then use it. There are valid reasons (which several others have made) not to force others to use it. The decision isn't yours. Or mine. And the ones whose decision it is will switch or they won't. Let's move on. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On Wed, 1 Mar 2017 09:42:27, cyg Simple wrote: Oh but it is. We're discussing the bike shed named /bin/sh. One of the color names is bash the other is dash; it's still the same bike shed. I see, when you realize your argument does not hold water, you resort to name calling. Here are concrete arguments: 1. Debian uses Dash as /bin/sh 2. Ubuntu uses Dash as /bin/sh 3. The POSIX standard defines sh, and by definition Dash is closer to this than Bash 4. Dash is 3-5 times faster than Bash 5. While it might be upsetting to some users, _they_ have made an error in assuming Bash is /bin/sh, it would not be our error in switching to a shell that is closer to POSIX sh Now, perhaps you can add a constructive post, or do you want to start with some "yo momma" jokes? -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
RE: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
Personally, I would invoke a variation of Linus's attitude to breaking user space apps: the distribution should not break existing user's scripts, even if they are not following "the rules". If you want to speed up the scripts distributed by Cygwin, then I would suggest modifying those scripts to use #!/bin/dash. Or give the user an option to change "/bin/sh" to point to dash, but don't make that the default setting. But breaking who-knows-how-many user scripts without the user's informed consent seems like an undesirable solution. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On 2/28/2017 4:43 PM, Steven Penny wrote: > On Tue, 28 Feb 2017 15:52:15, cyg Simple wrote: >> Ironic that *you* should make the same argument for using #!/bin/bash as >> I've made to you about using #!/bin/dash. > > Its not the same argument: > Oh but it is. We're discussing the bike shed named /bin/sh. > - You are talking about people assuming Dash is /bin/sh > - I am talking about people assuming Bash is /bin/sh > One of the color names is bash the other is dash; it's still the same bike shed. -- cyg Simple -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On Tue, 28 Feb 2017 15:52:15, cyg Simple wrote: Ironic that *you* should make the same argument for using #!/bin/bash as I've made to you about using #!/bin/dash. Its not the same argument: - You are talking about people assuming Dash is /bin/sh - I am talking about people assuming Bash is /bin/sh Both are wrong, but assuming Dash is /bin/sh is leagues more innocuous: - Dash doesnt have arrays - Dash doesnt have herestrings - Dash doesnt have process substitution - etc So even if someone does mess up and include a Dashism with /bin/sh, it should be trivial to fix. If you include heavy Bashisms with /bin/sh, it can be very hard to undo that. Dash is not interchangable with POSIX sh, but it is certainly closer than Bash. But that doesn't mean you should expect /bin/sh to be dash. I certainly dont expect that. I expect /bin/sh to be POSIX sh, and try to write my scripts accordingly. Dash helps with this because it forces you to lose Bashisms when writing scripts. The idea is only sound because it uses less resources and when we're talking about Windows a little resource makes a big difference. Exactly. Even with Dash as /bin/sh, we are still about 5x slower than Linux /bin/sh. People who are using /bin/sh with the assumption that it is Bash can just change their shebang, or remove the Bashisms from their /bin/sh scripts. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On 2/27/2017 7:51 PM, Steven Penny wrote: > On Tue, 28 Feb 2017 10:12:47, Duncan Roe wrote: >> "we" being you and who else? >> /bin/sh has been bash for a long time and I would prefer it stays that >> way. > > “That’s the way it’s always been done” is not a good reason to keep doing > something. /bin/sh has been defined for 20 years, and it is not Bash: > > http://pubs.opengroup.org/onlinepubs/7908799/xcu/sh.html > > If you want to write a Bash script, you should be using #!/bin/bash, not > #!/bin/sh. If we are to continue using Bash, it should be for a good > reason, > like the one Eric presented, not just because you want to save 2 > characters on > your shebang line. > Ironic that *you* should make the same argument for using #!/bin/bash as I've made to you about using #!/bin/dash. If you want to ensure that #!/bin/sh is a certain resemblance of a shell then you must test for it and take other actions if not. You cannot portably rely on /bin/sh having certain qualities but you know that already. Your argument is that dash is "more" POSIX conforming than bash as sh and faster because its footprint is smaller. Should we change, maybe give it a test before the next release. It really doesn't matter other than the work required to make it so and the work required to help those who get surprised. But that doesn't mean you should expect /bin/sh to be dash. The idea is only sound because it uses less resources and when we're talking about Windows a little resource makes a big difference. -- cyg Simple -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On Tue, 28 Feb 2017 10:12:47, Duncan Roe wrote: "we" being you and who else? /bin/sh has been bash for a long time and I would prefer it stays that way. “That’s the way it’s always been done” is not a good reason to keep doing something. /bin/sh has been defined for 20 years, and it is not Bash: http://pubs.opengroup.org/onlinepubs/7908799/xcu/sh.html If you want to write a Bash script, you should be using #!/bin/bash, not #!/bin/sh. If we are to continue using Bash, it should be for a good reason, like the one Eric presented, not just because you want to save 2 characters on your shebang line. On Fri, 24 Feb 2017 08:43:30, Eric Blake wrote: But we are severely limited in volunteer manpower compared to Debian, and I suspect that 1) there won't be enough testers (we won't know the real impact until it is no longer experimental, but that is too late), and 2) even if testers are diligent, we will be unable to patch all the fallout in any sort of timely manner. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On Sat, Feb 25, 2017 at 09:08:55AM -0800, Steven Penny wrote: > On Sat, 25 Feb 2017 11:46:08, cyg Simple wrote: > > If *your* script has a dependency to run using *dash* instead of *sh* > > then you _must_ use #!/bin/dash anyway. > > I think you have a fundamental misunderstanding of what Dash is. Dash is a > minimal shell, similar to the "sh" defined by POSIX: > > http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html > > So saying "a dependency to run using *dash*" makes about as much sense as > saying > "Cygwin has a dependency to run using Cygwin". > > > And what I _*I*_ don't want dash as /bin/sh? You see not everyone in the > > community will agree. > > With respect to the community, yes of course you get a say. If the community > doesn???t agree on this we will continue to use Bash. However you do not get > a say > with respect to the standard. The POSIX standard specified that /bin/sh is the > shell I have linked above. Bash is a superset of that, so by definition those > extra features are not defined or guaranteed to exist. When writing /bin/sh > scripts, you should be assuming the users /bin/sh only has those features > defined by POSIX. Example: > > #!/bin/bash -> script that follows might have Bashisms > #!/bin/dash -> script that follows might have Dashisms (echo -n, local) > #!/bin/sh -> script that follows should be a POSIX script > > > You can make the suggestion to those using your script to do so otherwise > > but > > forcing dash as /bin/sh worldwide isn't something you can actually do. > > Yes we certainly can. Debian and Ubuntu have already done this. > "we" being you and who else? /bin/sh has been bash for a long time and I would prefer it stays that way. OTOH, I have started prefacing scripts with /bin/bash -p to be on the safe side. "-p" stops bash from importing functions from the environment BTW Cheers ... Duncan. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On Mon, 27 Feb 2017 11:20:54, Csaba Raduly wrote: Nice strawman argument. You seem to assume that Dash implements nothing beyond POSIX (and hence anything that runs on dash must run on every other shell). This is not the case. Perhaps you should have read my full post. Here, let me quote myself and link for you: On Sat, 25 Feb 2017 09:08:55, Steven Penny wrote: #!/bin/bash -> script that follows might have Bashisms #!/bin/dash -> script that follows might have Dashisms (echo -n, local) #!/bin/sh -> script that follows should be a POSIX script http://cygwin.com/ml/cygwin/2017-02/msg00320.html So, as I said before and now again, Dash is a minimal shell, similar to POSIX shell. Yes, it is possible to be in a situation like this: - script has #!/bin/sh - script has Dashisms, which can cause problem on POSIX shell However being that "echo -n" and "local" are probably the only Dashisms, that is trivial to fix. Compare to a Bash problem: - script has #!/bin/sh - script has Bashisms, which can cause problem on POSIX shell Even switching from a Bash script to Dash script can be a herculean task, if the script heavily relies on Bashisms like arrays, herestrings, process substitution, etc. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On Sat, Feb 25, 2017 at 6:08 PM, Steven Penny wrote: > On Sat, 25 Feb 2017 11:46:08, cyg Simple wrote: >> >> If *your* script has a dependency to run using *dash* instead of *sh* >> then you _must_ use #!/bin/dash anyway. > > > I think you have a fundamental misunderstanding of what Dash is. Dash is a > minimal shell, similar to the "sh" defined by POSIX: > > http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html > > So saying "a dependency to run using *dash*" makes about as much sense as > saying > "Cygwin has a dependency to run using Cygwin". Nice strawman argument. You seem to assume that Dash implements nothing beyond POSIX (and hence anything that runs on dash must run on every other shell). This is not the case. >From the dash man page: dash is the standard command interpreter for the Linux system. The current version of dash is in the process of being changed to conform with the POSIX 1003.2 and 1003.2a specifications for the shell. This version has many features which make it appear similar in some respects to the Korn shell, but it is not a Korn shell clone. Only features designated by POSIX, plus a few Berkeley extensions, are being incorporated into this shell. Csaba -- GCS a+ e++ d- C++ ULS$ L+$ !E- W++ P+++$ w++$ tv+ b++ DI D++ 5++ The Tao of math: The numbers you can count are not the real numbers. Life is complex, with real and imaginary parts. "Ok, it boots. Which means it must be bug-free and perfect. " -- Linus Torvalds "People disagree with me. I just ignore them." -- Linus Torvalds -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On Sat, 25 Feb 2017 11:46:08, cyg Simple wrote: If *your* script has a dependency to run using *dash* instead of *sh* then you _must_ use #!/bin/dash anyway. I think you have a fundamental misunderstanding of what Dash is. Dash is a minimal shell, similar to the "sh" defined by POSIX: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html So saying "a dependency to run using *dash*" makes about as much sense as saying "Cygwin has a dependency to run using Cygwin". And what I _*I*_ don't want dash as /bin/sh? You see not everyone in the community will agree. With respect to the community, yes of course you get a say. If the community doesn’t agree on this we will continue to use Bash. However you do not get a say with respect to the standard. The POSIX standard specified that /bin/sh is the shell I have linked above. Bash is a superset of that, so by definition those extra features are not defined or guaranteed to exist. When writing /bin/sh scripts, you should be assuming the users /bin/sh only has those features defined by POSIX. Example: #!/bin/bash -> script that follows might have Bashisms #!/bin/dash -> script that follows might have Dashisms (echo -n, local) #!/bin/sh -> script that follows should be a POSIX script You can make the suggestion to those using your script to do so otherwise but forcing dash as /bin/sh worldwide isn't something you can actually do. Yes we certainly can. Debian and Ubuntu have already done this. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On Fri, 24 Feb 2017 08:43:30, Eric Blake wrote: But we are severely limited in volunteer manpower compared to Debian, and I suspect that 1) there won't be enough testers (we won't know the real impact until it is no longer experimental, but that is too late), and 2) even if testers are diligent, we will be unable to patch all the fallout in any sort of timely manner. Are you really prepared to force the Cygwin community through that much growing pain? This is a fantastic question, thank you for asking. For me to answer it, I need to quantify how much pain is actually involved. To truly answer that I would need to download an check every single Cygwin package. That is probably overkill because surely some packages are rarely used. I used this methodology to come up with a list of packages to test: 1. Base category and recursive dependencies 2. Search my own repositories for "sage install", and got recursive dependencies for those too 3. Add checkbashisms package and recursive dependencies This yields the list at the bottom of this post. Probably some packages need to be added to this list, but it is a start. Next, I put all files from those packages through this Awk selector: FNR == 1 && $0 ~ "^#!.*/sh" To get potential scripts that need to be fixed. Then I ran checkbashism on those files. The results are here: http://github.com/svnpenn/a/blob/0a85bf7/docs/cygwin-bash-to-dash.txt I am willing to correct these, if you are willing to proceed. _autorebase alternatives autoconf autoconf2.1 autoconf2.5 automake automake1.10 automake1.11 automake1.12 automake1.13 automake1.14 automake1.15 automake1.4 automake1.5 automake1.6 automake1.7 automake1.8 automake1.9 base-cygwin base-files bash binutils bison bzip2 ca-certificates checkbashisms cmake coreutils csih curl cygrunsrv cygutils cygwin cygwin-devel dash dejavu-fonts desktop-file-utils dialog diffutils dri-drivers ed editrights file findutils flex gamin gawk gcc-core getent ghostscript ghostscript-fonts-other ghostscript-fonts-std git gnupg grep groff gsettings-desktop-schemas gzip hostname info ipc-utils less libarchive13 libargp libatomic1 libattr1 libblkid1 libbz2_1 libcairo2 libcom_err2 libcrypt0 libcurl4 libdb5.3 libdialog13 libedit0 libEGL1 libexpat1 libfam0 libffi6 libfontconfig1 libfontconfig-common libfreetype6 libgcc1 libgd3 libgdbm4 libGL1 libglapi0 libglib2.0_0 libgmp10 libgnutls28 libgomp1 libgraphite2_3 libgs9 libgssapi_krb5_2 libguile17 libharfbuzz0 libharfbuzz-icu0 libhogweed2 libICE6 libiconv libiconv2 libicu57 libicu58 libidn11 libidn2_0 libintl8 libisl13 libjbig2 libjpeg8 libk5crypto3 libkpathsea6 libkrb5_3 libkrb5support0 liblcms2_2 libllvm3.8 libllvm3.9 libltdl7 liblz4_1 liblzma5 liblzo2_2 libmetalink3 libmpc3 libmpfr4 libncurses-devel libncursesw10 libnettle4 libnghttp2_14 libnspr4 libnss3 libopenjpeg1 libopenldap2_4_2 libopenssl100 libp11-kit0 libpaper1 libpaper-common libpcre1 libpipeline1 libpixman1_0 libpng16 libpoppler62 libpopt0 libpopt-common libpsl5 libptexenc1 libquadmath0 libreadline7 libsasl2_3 libsigsegv2 libSM6 libsmartcols1 libsqlite3_0 libssh2_1 libssp0 libstdc++6 libsynctex1 libtasn1_6 libtexlua52_5 libtexluajit2 libtiff6 libtool libtxc_dxtn libunistring2 libusb0 libuuid1 libuuid-devel libvtv0 libwebp5 libX11_6 libX11-xcb1 libXau6 libXaw7 libxcb1 libxcb-glx0 libxcb-render0 libxcb-shm0 libXdmcp6 libXext6 libXft2 libXi6 libxml2 libXmu6 libXpm4 libXrender1 libXt6 libzzip0.13 login m4 make man-db mingw64-x86_64-binutils mingw64-x86_64-curl mingw64-x86_64-gcc-core mingw64-x86_64-gcc-g++ mingw64-x86_64-headers mingw64-x86_64-libgnurx mingw64-x86_64-libssh2 mingw64-x86_64-nghttp2 mingw64-x86_64-openssl mingw64-x86_64-pkg-config mingw64-x86_64-runtime mingw64-x86_64-windows-default-manifest mingw64-x86_64-win-iconv mingw64-x86_64-winpthreads mingw64-x86_64-zlib mintty ncurses openssh openssl p11-kit p11-kit-trust perl perl_autorebase perl_base perl-Carp perl-Digest-SHA perl-Encode-Locale perl-Error perl-File-Listing perl-HTML-Parser perl-HTML-Tagset perl-HTTP-Cookies perl-HTTP-Daemon perl-HTTP-Date perl-HTTP-Message perl-HTTP-Negotiate perl-IO-HTML perl-IO-String perl-libwww-perl perl-LWP-MediaTypes perl-Net-HTTP perl-Pod-Simple perl-Socket perl-TermReadKey perl-Tk perl-Tk-Pod perl-Try-Tiny perl-Unicode-Normalize perl-URI perl-WWW-RobotRules perl-XML-Parser pkg-config poppler-data publicsuffix-list-dafsa python rebase rsync run sed shared-mime-info tar terminfo texinfo texlive texlive-collection-basic texlive-collection-fontsrecommended texlive-collection-latex tzcode tzdata unzip util-linux vim-minimal w32api-headers w32api-runtime wget which windows-default-manifest words xz yasm zip zlib0 -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On 2/23/2017 2:46 PM, Steven Penny wrote: > On Thu, 23 Feb 2017 20:38:08, Andrey Repin wrote: >> $ cat /etc/postinstall/zp_dash-as-bin-sh.dash >> #!/bin/dash > >> test ! -h /bin/sh.exe && { >> _f="$(mktemp /bin/sh.XX)" >> { >> mv /bin/sh.exe "$_f" >> ln -s /bin/dash.exe /bin/sh.exe >> } || mv "$_f" /bin/sh.exe >> rm "$_f" >> } >> exit 0 > > This is a bad idea. > > If script succeeds, then yes sh is now a symlink to Dash, but only for my > system. I still have to use #!/bin/dash with my scripts, because otherwise > people with Bash -> sh will not get the speed boost. The whole point of > this > thread is that it should be changed for everyone. If *your* script has a dependency to run using *dash* instead of *sh* then you _must_ use #!/bin/dash anyway. And what I _*I*_ don't want dash as /bin/sh? You see not everyone in the community will agree. You can make the suggestion to those using your script to do so otherwise but forcing dash as /bin/sh worldwide isn't something you can actually do. -- cyg Simple -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On 2017-02-24 07:32, Eric Blake wrote: > On 02/23/2017 05:44 PM, Steven Penny wrote: >> On Thu, 23 Feb 2017 12:59:27, Brian Inglis wrote: >>> Some people may continue to need or prefer using bash in POSIX >>> mode. >> You might know this already, but "Bash in POSIX mode" is not quite >> POSIX. For example, arrays are not defined by POSIX: > You may not know this, but POSIX doesn't forbid extensions. Bash in > POSIX mode does NOT cripple all extensions, merely those that are > incompatible with POSIX. That's why so many people end up writing > #!/bin/sh scripts with bashisms, then get surprised when they break > on a different shell. > If you want a STRICT shell, Debian's posh is even better than dash > at ferreting out use of non-portable extensions. For those interested, Cygwin packages are available for posh, a "stripped down version of pdksh", and checkbashisms, a Perl script for flagging any "shell feature that is not required to be supported by POSIX". Neither dash nor posh have dependencies other than cygwin1.dll, but posh is not exactly lightweight at 598KB compared to dash at 99KB. While posh may be good for checking scripts when installed as /bin/sh, dash is a better choice for running scripts as a production /bin/sh. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On 2/24/2017 9:43 AM, Eric Blake wrote: On 02/23/2017 10:57 PM, Steven Penny wrote: Or more likely, many people likely have pre-existing scripts wrongly written as #!/bin/sh but which use bash-ism rather than portable POSIX-specified shell However, I think it is worth the trouble. If you'd like, I can post experimental versions of both bash and dash, which MUST be upgraded (or downgraded) in lockstep, where I move /bin/sh over to the dash package (do it wrong, and you could be left with no /bin/sh at all, which is not a good idea - although maybe I can use some postinstall scripts so that at least the upgrade side tries to play nice even when someone only does a partial upgrade). If people will then test with those experimental versions installed, and report breakage, we could get a feel for how many scripts installed by default are broken. But we are severely limited in volunteer manpower compared to Debian, and I suspect that 1) there won't be enough testers (we won't know the real impact until it is no longer experimental, but that is too late), and 2) even if testers are diligent, we will be unable to patch all the fallout in any sort of timely manner. Are you really prepared to force the Cygwin community through that much growing pain? I agree that /bin/sh as dash is much faster at executing configure scripts. But configure scripts aren't the only scripts in the wild. We do have checkbashisms ported to Cygwin, and that can help, but it is not a panacea. If dash doesn't support all the features of bash (which is incorrectly assumed in some cases for sh), aren't you just asking for trouble by breaking things? -lee -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On 02/23/2017 10:57 PM, Steven Penny wrote: >> Or more likely, many people likely have pre-existing scripts wrongly >> written as #!/bin/sh but which use bash-ism rather than portable >> POSIX-specified shell > > However, I think it is worth the trouble. If you'd like, I can post experimental versions of both bash and dash, which MUST be upgraded (or downgraded) in lockstep, where I move /bin/sh over to the dash package (do it wrong, and you could be left with no /bin/sh at all, which is not a good idea - although maybe I can use some postinstall scripts so that at least the upgrade side tries to play nice even when someone only does a partial upgrade). If people will then test with those experimental versions installed, and report breakage, we could get a feel for how many scripts installed by default are broken. But we are severely limited in volunteer manpower compared to Debian, and I suspect that 1) there won't be enough testers (we won't know the real impact until it is no longer experimental, but that is too late), and 2) even if testers are diligent, we will be unable to patch all the fallout in any sort of timely manner. Are you really prepared to force the Cygwin community through that much growing pain? I agree that /bin/sh as dash is much faster at executing configure scripts. But configure scripts aren't the only scripts in the wild. We do have checkbashisms ported to Cygwin, and that can help, but it is not a panacea. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On 02/23/2017 05:44 PM, Steven Penny wrote: > On Thu, 23 Feb 2017 12:59:27, Brian Inglis wrote: >> Some people may continue to need or prefer using bash in POSIX mode. > > You might know this already, but "Bash in POSIX mode" is not quite > POSIX. For > example, arrays are not defined by POSIX: You may not know this, but POSIX doesn't forbid extensions. Bash in POSIX mode does NOT cripple all extensions, merely those that are incompatible with POSIX. That's why so many people end up writing #!/bin/sh scripts with bashisms, then get surprised when they break on a different shell. If you want a STRICT shell, Debian's posh is even better than dash at ferreting out use of non-portable extensions. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On 2017-02-23 14:41, Eric Blake wrote: Or more likely, many people likely have pre-existing scripts wrongly written as #!/bin/sh but which use bash-ism rather than portable POSIX-specified shell (if it is portable, then bash vs. dash executing the script shouldn't matter - but as Debian found out when they switched to dash, there are a lot of non-portable scripts). On the other hand, Debian has already fixed much of the pre-packaged badness out there and pushed fixes upstream where feasible, so these days it's not as hairy to switch as when Debian first did it. But part of me is still reluctant to make the switch as long as Fedora still favors bash over dash. I concur; we simply don't have the manpower to handle this ourselves. -- Yaakov -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On Thu, 23 Feb 2017 22:15:37, "Larry Hall (Cygwin)" wrote: Since making a change of this magnitude is going to be an undertaking, we should be sure we're going to see the intended benefits before enduring the pain such a change would bring, though hopefully that pain would be short-lived and/or minor. :-) I am open to others debunking my previous test with other/better tests, but from where I sit the benefit can be clearly seen. Not just with script times, but startup time will see a 2-3 fold increase as well. This can be hugely helpful with situations like using Autotools (configure), where "sh" is spawned hundreds of times. I have done heavy testing "living" with Dash. Between my repos I have over 80 Dash scripts, with usage going back to 2015: http://github.com/svnpenn/a/commit/b4908b7 I think Eric said it well: On Thu, 23 Feb 2017 14:41:52, Eric Blake wrote: Or more likely, many people likely have pre-existing scripts wrongly written as #!/bin/sh but which use bash-ism rather than portable POSIX-specified shell However, I think it is worth the trouble. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On 02/23/2017 06:01 PM, Tony Kelman wrote: The big question remains, where this speed boost coming from? Is this a startup time? Or some internal slowness? Because in latter case, given your STC, this is a bash issue and should be reported upstream. Dunno what you meant by STC, but upstream is well aware: $ man bash | tail -n 21 | head -n 2 BUGS It's too big and too slow. Forgive me but this entire thread has been giving me a serious case of deja-vu. We've been down a similar path to this before, the last time around with "ash" rather than "dash" but the arguments sound very familiar. Brian Dessent has a nice summary a paragraph into his response below: https://sourceware.org/ml/cygwin/2008-03/msg00369.html Now this was "ash" and not "dash" and the actual transition from "bash" to "ash" and back again occurred well over 10 years ago, so allot of things have changed for sure. But I bring this up because I think the parallels are there. Since making a change of this magnitude is going to be an undertaking, we should be sure we're going to see the intended benefits before enduring the pain such a change would bring, though hopefully that pain would be short-lived and/or minor. :-) -- Larry _ A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting annoying in email? -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On 02/23/2017 06:01 PM, Tony Kelman wrote: The big question remains, where this speed boost coming from? Is this a startup time? Or some internal slowness? Because in latter case, given your STC, this is a bash issue and should be reported upstream. Dunno what you meant by STC, but upstream is well aware: $ man bash | tail -n 21 | head -n 2 BUGS It's too big and too slow. Forgive me but this entire thread has been giving me a serious case of deja-vu. We've been down a similar path to this before, the last time around with "ash" rather than "dash" but the arguments sound very familiar. Brian Dessent has a nice summary a paragraph into his response below: https://sourceware.org/ml/cygwin/2008-03/msg00369.html Now this was "ash" and not "dash" and the actual transition from "bash" to "ash" and back again occurred well over 10 years ago, so allot of things have changed for sure. But I bring this up because I think the parallels are there. Since making a change of this magnitude is going to be an undertaking, we should be sure we're going to see the intended benefits before enduring the pain such a change would bring, though hopefully that pain would be short-lived and/or minor. :-) -- Larry _ A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting annoying in email? -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On Thu, 23 Feb 2017 12:59:27, Brian Inglis wrote: Some people may continue to need or prefer using bash in POSIX mode. You might know this already, but "Bash in POSIX mode" is not quite POSIX. For example, arrays are not defined by POSIX: http://mywiki.wooledge.org/Bashism#Arrays Yes Bash even in POSIX mode handles them: $ bash --posix -c 'z=(2 3 4); echo ${z[2]}' 4 Dash rightfully does not handle them: $ dash -c 'z=(2 3 4); echo ${z[2]}' dash: 1: Syntax error: "(" unexpected -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
Greetings, Tony Kelman! >> The big question remains, where this speed boost coming from? >> Is this a startup time? Or some internal slowness? >> Because in latter case, given your STC, this is a bash issue and should be >> reported upstream. > Dunno what you meant by STC, https://cygwin.com/acronyms/#STC > but upstream is well aware: > $ man bash | tail -n 21 | head -n 2 > BUGS >It's too big and too slow. That's not answering my question. -- With best regards, Andrey Repin Friday, February 24, 2017 02:18:52 Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On Fri, 24 Feb 2017 00:50:08, Andrey Repin wrote: In fact, you should use specific shell name, if you want specific behavior. That is not at issue here. Both before and after this proposed change, I can use #!/bin/dash to explicitly call Dash and #!/bin/bash to explicitly call Bash. What _is_ at issue here is what is called with #!/bin/sh for _all_ users, not just myself or those whom have run your homebrewed script. In that regard, since we are giving deference to Debian by using their shell, I say we continue that pattern and use Dash as /bin/sh. The big question remains, where this speed boost coming from? Is this a startup time? Or some internal slowness? Because in latter case, given your STC, this is a bash issue and should be reported upstream. Dash is faster than Bash. This is not a question, it is a fact. It is not a _bug_ any more than a Ducati being faster than an 18-wheeler is a bug. Dash by design has less features than Bash _so_ that it can be faster than Dash. It would suit you well to do some research of your own on this topic before posting further. Cheers -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
> On Feb 23, 2017, at 5:04 PM, Eliot Mosswrote: > > On 2/23/2017 6:01 PM, Tony Kelman wrote: >>> The big question remains, where this speed boost coming from? >>> Is this a startup time? Or some internal slowness? >>> Because in latter case, given your STC, this is a bash issue and should be >>> reported upstream. >> >> Dunno what you meant by STC, > > Indeed, this gives around 20 possibilities, none of which seem to fit: > > http://www.acronymfinder.com/Slang/STC.html If it's used here, then the first place to look is on the Cygwin acronym page. https://cygwin.com/acronyms/#STC -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On Thu, Feb 23, 2017 at 3:04 PM, Eliot Mosswrote: > On 2/23/2017 6:01 PM, Tony Kelman wrote: >>> >>> The big question remains, where this speed boost coming from? >>> Is this a startup time? Or some internal slowness? >>> Because in latter case, given your STC, this is a bash issue and should >>> be >>> reported upstream. >> >> >> Dunno what you meant by STC, > > > Indeed, this gives around 20 possibilities, none of which seem to fit: > > http://www.acronymfinder.com/Slang/STC.html > > Cheers! Eliot Moss Small Test Case. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On 2/23/2017 6:01 PM, Tony Kelman wrote: The big question remains, where this speed boost coming from? Is this a startup time? Or some internal slowness? Because in latter case, given your STC, this is a bash issue and should be reported upstream. Dunno what you meant by STC, Indeed, this gives around 20 possibilities, none of which seem to fit: http://www.acronymfinder.com/Slang/STC.html Cheers! Eliot Moss -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
> The big question remains, where this speed boost coming from? > Is this a startup time? Or some internal slowness? > Because in latter case, given your STC, this is a bash issue and should be > reported upstream. Dunno what you meant by STC, but upstream is well aware: $ man bash | tail -n 21 | head -n 2 BUGS It's too big and too slow. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
Greetings, Steven Penny! > On Thu, 23 Feb 2017 20:38:08, Andrey Repin wrote: >> $ cat /etc/postinstall/zp_dash-as-bin-sh.dash >> #!/bin/dash >> test ! -h /bin/sh.exe && { >> _f="$(mktemp /bin/sh.XX)" >> { >> mv /bin/sh.exe "$_f" >> ln -s /bin/dash.exe /bin/sh.exe >> } || mv "$_f" /bin/sh.exe >> rm "$_f" >> } >> exit 0 > This is a bad idea. This is only bad idea for people assuming too much. > If script succeeds, then yes sh is now a symlink to Dash, but only for my > system. I still have to use #!/bin/dash with my scripts, In fact, you should use specific shell name, if you want specific behavior. > because otherwise > people with Bash -> sh will not get the speed boost. The whole point of this > thread is that it should be changed for everyone. The big question remains, where this speed boost coming from? Is this a startup time? Or some internal slowness? Because in latter case, given your STC, this is a bash issue and should be reported upstream. -- With best regards, Andrey Repin Friday, February 24, 2017 00:47:31 Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On 02/23/2017 01:59 PM, Brian Inglis wrote: >> If script succeeds, then yes sh is now a symlink to Dash, but only >> for my system. I still have to use #!/bin/dash with my scripts, >> because otherwise people with Bash -> sh will not get the speed >> boost. The whole point of this thread is that it should be changed >> for everyone. > > And should be done using update_alternatives so it and associated > man pages can easily be changed if anyone suffers breakage. > Some people may continue to need or prefer using bash in POSIX mode. Or more likely, many people likely have pre-existing scripts wrongly written as #!/bin/sh but which use bash-ism rather than portable POSIX-specified shell (if it is portable, then bash vs. dash executing the script shouldn't matter - but as Debian found out when they switched to dash, there are a lot of non-portable scripts). On the other hand, Debian has already fixed much of the pre-packaged badness out there and pushed fixes upstream where feasible, so these days it's not as hairy to switch as when Debian first did it. But part of me is still reluctant to make the switch as long as Fedora still favors bash over dash. One problem with using update_alternatives: the alternatives system relies on symlinks, but Windows is not so smart about executing through symlinks; are there people that will have a broken setup if /bin/sh is a symlink rather than an actual executable (hard links are fine, but symlinks is where I worry). -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On 2017-02-23 12:46, Steven Penny wrote: > On Thu, 23 Feb 2017 20:38:08, Andrey Repin wrote: >> $ cat /etc/postinstall/zp_dash-as-bin-sh.dash >> #!/bin/dash > >> test ! -h /bin/sh.exe && { >> _f="$(mktemp /bin/sh.XX)" >> { >> mv /bin/sh.exe "$_f" >> ln -s /bin/dash.exe /bin/sh.exe >> } || mv "$_f" /bin/sh.exe >> rm "$_f" >> } >> exit 0 > > This is a bad idea. > > If script succeeds, then yes sh is now a symlink to Dash, but only > for my system. I still have to use #!/bin/dash with my scripts, > because otherwise people with Bash -> sh will not get the speed > boost. The whole point of this thread is that it should be changed > for everyone. And should be done using update_alternatives so it and associated man pages can easily be changed if anyone suffers breakage. Some people may continue to need or prefer using bash in POSIX mode. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On Thu, 23 Feb 2017 20:38:08, Andrey Repin wrote: $ cat /etc/postinstall/zp_dash-as-bin-sh.dash #!/bin/dash test ! -h /bin/sh.exe && { _f="$(mktemp /bin/sh.XX)" { mv /bin/sh.exe "$_f" ln -s /bin/dash.exe /bin/sh.exe } || mv "$_f" /bin/sh.exe rm "$_f" } exit 0 This is a bad idea. If script succeeds, then yes sh is now a symlink to Dash, but only for my system. I still have to use #!/bin/dash with my scripts, because otherwise people with Bash -> sh will not get the speed boost. The whole point of this thread is that it should be changed for everyone. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
Greetings, Steven Penny! > On Fri, 17 Feb 2017 07:50:34, "Eric Blake (cygwin)" wrote: >> For now, there are no immediate plans of replacing /bin/sh >> with dash, but the possibility remains for the future. > I am a fan of this. $ cat /etc/postinstall/zp_dash-as-bin-sh.dash #!/bin/dash test ! -h /bin/sh.exe && { _f="$(mktemp /bin/sh.XX)" { mv /bin/sh.exe "$_f" ln -s /bin/dash.exe /bin/sh.exe } || mv "$_f" /bin/sh.exe rm "$_f" } exit 0 > Debian does this already, and it would dramatically speed up > shell scripts: > $ cat alfa.sh > x=40 > until [ "$((x -= 1))" = 0 ] > do : > done > $ time bash alfa.sh > real0m5.735s > $ time dash alfa.sh > real0m1.112s > Dash can handle large scripts; I am already using it with one (545 SLOC): > http://github.com/svnpenn/sage/blob/master/sage -- With best regards, Andrey Repin Thursday, February 23, 2017 20:36:35 Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1
On Fri, 17 Feb 2017 07:50:34, "Eric Blake (cygwin)" wrote: For now, there are no immediate plans of replacing /bin/sh with dash, but the possibility remains for the future. I am a fan of this. Debian does this already, and it would dramatically speed up shell scripts: $ cat alfa.sh x=40 until [ "$((x -= 1))" = 0 ] do : done $ time bash alfa.sh real0m5.735s $ time dash alfa.sh real0m1.112s Dash can handle large scripts; I am already using it with one (545 SLOC): http://github.com/svnpenn/sage/blob/master/sage -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple