Hi Paul, * Paul Eggert wrote on Wed, May 24, 2006 at 01:32:18AM CEST: > Ralf Wildenhues <[EMAIL PROTECTED]> writes: > > > I hope we get away with this. > > I don't think we will, since the bug occurs every 1024 bytes, and many > define.sed scripts are longer than that. > > I installed this patch, which works around this particular problem by > not using shell expansion at all in the here-documents used to create > defines.sed.
We've further analyzed this off-list now, thanks to Stepan and Tim for insistence and help! The original post about this issue I found here: http://lists.gnu.org/archive/html/bug-autoconf/2002-03/msg00056.html The corresponding patch here: http://lists.gnu.org/archive/html/autoconf-patches/2002-03/msg00052.html Let's look at this closely. The script below (first attachment) tries different types of substitutions in here documents. The output of the script for Solaris 2.6 dtksh (second attachment), and, with different bounds, for UnixWare 7.1.1 /usr/bin/posix/sh (third attachment) can be seen below. What do we see? - The failure is always connected with the position of the closing brace `}' of the substitution: avoiding parameter substitutions without braces helps. - Some failures are silent, some come with shell errors: using one-byte variable names prevent silent failures. - Command substitutions `cmd` work fine. - The UnixWare shell exposes this bug only at a later stage (we tried ranges of 1000 to 3100, and 4000 to 5000 for $i in the script). Further, the same bug also happens for dtksh on a UnixWare 7.1.1 box (with the lower limit already). So I propose the following doc patch (last attachment). And I propose that your patch be reverted and replaced by something mildly less ugly (Stepan, did you want to propose a patch to this extent?). Cheers, Ralf
x.sh
Description: Bourne shell script
ts[1]: {}: bad substitution
Breakage at i=1014 with ${long_var}
Silent breakage at i=1015 with ${long_var}
ts[1]: {}: bad substitution
Breakage at i=1015 with ${foo-XXX}
Silent breakage at i=1016 with ${long_var}
Silent breakage at i=1016 with ${foo-XXX}
Silent breakage at i=1017 with ${long_var}
Silent breakage at i=1017 with ${foo-XXX}
Silent breakage at i=1018 with ${long_var}
Silent breakage at i=1018 with ${foo-XXX}
Silent breakage at i=1019 with ${long_var}
Silent breakage at i=1019 with ${foo-XXX}
Silent breakage at i=1020 with ${long_var}
ts[1]: {}: bad substitution
Breakage at i=1021 with ${x}
Silent breakage at i=1021 with ${long_var}
ts[1]: {}: bad substitution
Breakage at i=2038 with ${long_var}
Silent breakage at i=2039 with ${long_var}
ts[1]: {}: bad substitution
Breakage at i=2039 with ${foo-XXX}
Silent breakage at i=2040 with ${long_var}
Silent breakage at i=2040 with ${foo-XXX}
Silent breakage at i=2041 with ${long_var}
Silent breakage at i=2041 with ${foo-XXX}
Silent breakage at i=2042 with ${long_var}
Silent breakage at i=2042 with ${foo-XXX}
Silent breakage at i=2043 with ${long_var}
Silent breakage at i=2043 with ${foo-XXX}
Silent breakage at i=2044 with ${long_var}
ts[1]: {}: bad substitution
Breakage at i=2045 with ${x}
Silent breakage at i=2045 with ${long_var}
ts[1]: {}: bad substitution
Breakage at i=3062 with ${long_var}
Silent breakage at i=3063 with ${long_var}
ts[1]: {}: bad substitution
Breakage at i=3063 with ${foo-XXX}
Silent breakage at i=3064 with ${long_var}
Silent breakage at i=3064 with ${foo-XXX}
Silent breakage at i=3065 with ${long_var}
Silent breakage at i=3065 with ${foo-XXX}
Silent breakage at i=3066 with ${long_var}
Silent breakage at i=3066 with ${foo-XXX}
Silent breakage at i=3067 with ${long_var}
Silent breakage at i=3067 with ${foo-XXX}
Silent breakage at i=3068 with ${long_var}
ts[1]: {}: bad substitution
Breakage at i=3069 with ${x}
Silent breakage at i=3069 with ${long_var}
...
ts: line 1: {}: bad substitution
Breakage at i=4086 with ${long_var}
Silent breakage at i=4087 with ${long_var}
ts: line 1: {}: bad substitution
Breakage at i=4087 with ${foo-XXX}
Silent breakage at i=4088 with ${long_var}
Silent breakage at i=4088 with ${foo-XXX}
Silent breakage at i=4089 with ${long_var}
Silent breakage at i=4089 with ${foo-XXX}
Silent breakage at i=4090 with ${long_var}
Silent breakage at i=4090 with ${foo-XXX}
Silent breakage at i=4091 with ${long_var}
Silent breakage at i=4091 with ${foo-XXX}
Silent breakage at i=4092 with ${long_var}
ts: line 1: {}: bad substitution
Breakage at i=4093 with ${x}
Silent breakage at i=4093 with ${long_var}
...
* doc/autoconf.texi (Here-Documents): We now know more about
the variable expansion in here documents bug.
Thanks to Tim Rice and Stepan Kasal.
Index: doc/autoconf.texi
===================================================================
RCS file: /cvsroot/autoconf/autoconf/doc/autoconf.texi,v
retrieving revision 1.1021
diff -u -r1.1021 autoconf.texi
--- doc/autoconf.texi 22 May 2006 17:27:50 -0000 1.1021
+++ doc/autoconf.texi 25 May 2006 05:58:44 -0000
@@ -10749,12 +10751,13 @@
Many older shells (including the Bourne shell) implement here-documents
inefficiently. And some shells mishandle large here-documents: for
-example, Solaris @command{dtksh}, which is derived from Korn shell
-version M-12/28/93d, mishandles variable expansion that occurs on
-1024-byte buffer boundaries within a here-document. Users can generally
-fix these problems by using a faster or more reliable shell, e.g., by
-using the command @samp{CONFIG_SHELL=/bin/bash /bin/bash ./configure} rather
-than plain @samp{./configure}.
+example, Solaris @command{dtksh} and the UnixWare Posix shell, which are
+derived from Korn shell version M-12/28/93d, mishandle braced variable
+expansion @[EMAIL PROTECTED]@}} that crosses a 1024- or 4096-byte buffer
boundary
+within a here-document. If the closing brace does not lie on the boundary,
+the failure is silent and the variable expansion will be empty, otherwise
+the shell will report a bad substitution. This bug can usually be worked
+around by omitting the braces: @code{$var}.
Some shells can be extremely inefficient when there are a lot of
here-documents inside a single statement. For instance if your
