Re: Problems with latest Cygwin make patch; builds OK but binary segfaults
Date: Fri, 8 Sep 2006 18:14:38 -0400 From: Christopher Faylor [EMAIL PROTECTED] On Fri, Sep 08, 2006 at 11:47:22PM +0300, Eli Zaretskii wrote: Date: Fri, 8 Sep 2006 13:51:21 -0400 From: Christopher Faylor So, again, I'd like to go slowly and carefully with this change. I agree, and said I would wait, but I can't wait indefinitely. Eventually, if no one else cares to try these changes, they will go in. Personally, I think it sets a bad precedent to suggest that you'll apply a patch over the potential objections of the project lead for the affected system. If you have objections that are based on actual use cases, i.e. if you (or someone else) can come up with a situation where this patch does, or could potentially do, some harm, then of course such objections will never be overruled, not by me, anyway. Any such objections in the history of this patch development were always taken in consideration and acted upon immediately. Otherwise, it sounds unfair to me to encourage me and others to do some non-trivial amount of work, only to hold the results back after the job is done and approved by everyone involved, including yourself. However, you might as well go ahead and check in the patch. If someone notes a problem then presumably it can be reported back to this mailing list. But, at least, you won't have to suffer with an indefinite wait before the patch is checked in. And in the meantime, since this is free software, if I don't agree that this change should be in the Cygwin release, I can always back it out in the next release. True on both accounts. However, I personally am in no hurry; when I said ``indefinitely'', I really meant a VERY long time. As no Make release is planned any time soon (AFAIK), I don't see any reason to interpret ``indefinite'' as meaning ``a few days''. If you don't think the Cygwin user community needs the patched version ASAP, I certainly won't push that against your opinion. In the meantime, I see we have another positive experience reported on the Cygwin list. ___ Make-w32 mailing list Make-w32@gnu.org http://lists.gnu.org/mailman/listinfo/make-w32
Re: Problems with latest Cygwin make patch; builds OK but binary segfaults
On Fri, Sep 08, 2006 at 11:47:22PM +0300, Eli Zaretskii wrote: Date: Fri, 8 Sep 2006 13:51:21 -0400 From: Christopher Faylor So, again, I'd like to go slowly and carefully with this change. I agree, and said I would wait, but I can't wait indefinitely. Eventually, if no one else cares to try these changes, they will go in. Personally, I think it sets a bad precedent to suggest that you'll apply a patch over the potential objections of the project lead for the affected system. However, you might as well go ahead and check in the patch. If someone notes a problem then presumably it can be reported back to this mailing list. But, at least, you won't have to suffer with an indefinite wait before the patch is checked in. And in the meantime, since this is free software, if I don't agree that this change should be in the Cygwin release, I can always back it out in the next release. cgf ___ Make-w32 mailing list Make-w32@gnu.org http://lists.gnu.org/mailman/listinfo/make-w32
Re: Problems with latest Cygwin make patch; builds OK but binary segfaults
From: William Sheehan [EMAIL PROTECTED] Date: Tue, 5 Sep 2006 11:03:22 -0700 Thanks for catching this; your patch works perfectly and does not demonstrate any of the problems I was encountering before. Thanks for testing. Paul, I recommend this patch (reproduced below) for inclusion in the development sources. There's one more problem with HAVE_DOS_PATHS: $abspath doesn't support it. I already submitted a patch for that, but it will not DTRT for Cygwin, now that Cygwin supports HAVE_DOS_PATHS. I will rework that patch and resubmit it soon. 2006-08-18 Eli Zaretskii [EMAIL PROTECTED] * config/dospaths.m4 ac_cv_dos_paths: Define to yes on Cygwin as well. * make.h (PATH_SEPARATOR_CHAR): Define only if still undefined. Normally, it is defined in config.h. * configure.in (PATH_SEPARATOR_CHAR): Define to the value of $PATH_SEPARATOR. * job.c (construct_command_argv_internal) [HAVE_DOS_PATHS]: Define sh_chars_sh for Windows platforms that emulate Unix. --- configure.in~0 2006-04-01 12:36:40.0 +0300 +++ configure.in2006-08-18 21:12:32.828125000 +0300 @@ -384,6 +384,8 @@ ;; esac +AC_DEFINE_UNQUOTED(PATH_SEPARATOR_CHAR,'$PATH_SEPARATOR',[Define to the character that separates directories in PATH.]) + # Include the Maintainer's Makefile section, if it's here. MAINT_MAKEFILE=/dev/null --- make.h~02006-02-16 03:54:43.0 +0200 +++ make.h 2006-08-18 21:12:32.859375000 +0300 @@ -347,12 +347,14 @@ #define S_(msg1,msg2,num) ngettext (msg1,msg2,num) /* Handle other OSs. */ -#if defined(HAVE_DOS_PATHS) -# define PATH_SEPARATOR_CHAR ';' -#elif defined(VMS) -# define PATH_SEPARATOR_CHAR ',' -#else -# define PATH_SEPARATOR_CHAR ':' +#ifndef PATH_SEPARATOR_CHAR +# if defined(HAVE_DOS_PATHS) +# define PATH_SEPARATOR_CHAR ';' +# elif defined(VMS) +# define PATH_SEPARATOR_CHAR ',' +# else +# define PATH_SEPARATOR_CHAR ':' +# endif #endif /* This is needed for getcwd() and chdir(). */ --- config/dospaths.m4~02006-03-10 06:20:45.0 +0200 +++ config/dospaths.m4 2006-08-18 21:12:32.859375000 +0300 @@ -22,7 +22,7 @@ AC_CACHE_CHECK([whether system uses MSDOS-style paths], [ac_cv_dos_paths], [ AC_COMPILE_IFELSE([ -#if !defined _WIN32 !defined __WIN32__ !defined __MSDOS__ !defined __EMX__ !defined __MSYS__ +#if !defined _WIN32 !defined __WIN32__ !defined __MSDOS__ !defined __EMX__ !defined __MSYS__ !defined __CYGWIN__ neither MSDOS nor Windows nor OS2 #endif ], --- job.c~0 2006-03-20 07:03:04.0 +0200 +++ job.c 2006-08-19 09:25:07.68750 +0300 @@ -2307,6 +2307,12 @@ construct_command_argv_internal (char *l login, logout, read, readonly, set, shift, switch, test, times, trap, umask, wait, while, 0 }; +# ifdef HAVE_DOS_PATHS + /* This is required if the MSYS/Cygwin ports (which do not define + WINDOWS32) are compiled with HAVE_DOS_PATHS defined, which uses + sh_chars_sh[] directly (see below). */ + static char *sh_chars_sh = sh_chars; +# endif /* HAVE_DOS_PATHS */ #endif register int i; register char *p; ___ Make-w32 mailing list Make-w32@gnu.org http://lists.gnu.org/mailman/listinfo/make-w32
Re: Problems with latest Cygwin make patch; builds OK but binary segfaults
On Tue, Sep 05, 2006 at 09:23:41PM +0300, Eli Zaretskii wrote: From: William Sheehan [EMAIL PROTECTED] Date: Tue, 5 Sep 2006 11:03:22 -0700 Thanks for catching this; your patch works perfectly and does not demonstrate any of the problems I was encountering before. Thanks for testing. Paul, I recommend this patch (reproduced below) for inclusion in the development sources. There's one more problem with HAVE_DOS_PATHS: $abspath doesn't support it. I already submitted a patch for that, but it will not DTRT for Cygwin, now that Cygwin supports HAVE_DOS_PATHS. I will rework that patch and resubmit it soon. We've only had one success report for this so far, and even that seemed to have generated some confusion. Given all of the hoopla on the Cygwin list about this functionality, I'd feel much more comfortable if a few of the complainers there tested this also. I'm not willing to go through this amount of pain very often so I'd like to make sure we got things right the first time. So, I'd prefer holding off until a couple more people try this out. cgf ___ Make-w32 mailing list Make-w32@gnu.org http://lists.gnu.org/mailman/listinfo/make-w32
Re: Problems with latest Cygwin make patch; builds OK but binary segfaults
Date: Tue, 5 Sep 2006 14:29:02 -0400 From: Christopher Faylor [EMAIL PROTECTED] We've only had one success report for this so far, and even that seemed to have generated some confusion. Actually, we had 2 success reports. Given all of the hoopla on the Cygwin list about this functionality, I'd feel much more comfortable if a few of the complainers there tested this also. Me too, but I cannot force any of you to promote the patch on the Cygwin mailing list. You refused to do that, while William Hoffman didn't say no, but didn't post anything, either. I don't think it's reasonable to ask me do more than I already did. ___ Make-w32 mailing list Make-w32@gnu.org http://lists.gnu.org/mailman/listinfo/make-w32
Re: Problems with latest Cygwin make patch; builds OK but binary segfaults
Date: Tue, 05 Sep 2006 15:37:19 -0400 From: William A. Hoffman [EMAIL PROTECTED] I have sent out a new message, and cross posted to this list so all can see it. Thanks. Let's wait for a couple more weeks. ___ Make-w32 mailing list Make-w32@gnu.org http://lists.gnu.org/mailman/listinfo/make-w32
Re: Problems with latest Cygwin make patch; builds OK but binary segfaults
From: William Sheehan [EMAIL PROTECTED] Date: Fri, 25 Aug 2006 10:52:00 -0700 Please rune make check on the version of Make you think fixes the problem. If all the tests pass, please post here a patch that you used to fix the recursion problem. Thanks. Done; the patch is attached. Running make check came back with zero failures and a nice smiley face. The patch has the same content as Bill Hoffman's patch, plus my modifications for job.c. I'm looking at the patch you sent, and I'm confused: I don't understand how could it solve the recursion problem. It seems you started from a patch that is different from the last one I posted. My last patch defines sh_chars_sh so that its value is _identical_ to sh_chars. Thus, any changes that replace sh_chars with sh_chars_sh do not change anything at all. Could you please try the original patch I posted here (reproduced below for your convenience), and see if the recursion problem happens with it? If it does, please try to find what needs to be changed after the patches below. Thanks. Here's the patch I posted after Chris suggested to modify the original one. Please apply it to the original unmodified sources of Make 3.81. --- configure.in~0 2006-04-01 12:36:40.0 +0300 +++ configure.in2006-08-18 21:12:32.828125000 +0300 @@ -384,6 +384,8 @@ ;; esac +AC_DEFINE_UNQUOTED(PATH_SEPARATOR_CHAR,'$PATH_SEPARATOR',[Define to the character that separates directories in PATH.]) + # Include the Maintainer's Makefile section, if it's here. MAINT_MAKEFILE=/dev/null --- make.h~02006-02-16 03:54:43.0 +0200 +++ make.h 2006-08-18 21:12:32.859375000 +0300 @@ -347,12 +347,14 @@ #define S_(msg1,msg2,num) ngettext (msg1,msg2,num) /* Handle other OSs. */ -#if defined(HAVE_DOS_PATHS) -# define PATH_SEPARATOR_CHAR ';' -#elif defined(VMS) -# define PATH_SEPARATOR_CHAR ',' -#else -# define PATH_SEPARATOR_CHAR ':' +#ifndef PATH_SEPARATOR_CHAR +# if defined(HAVE_DOS_PATHS) +# define PATH_SEPARATOR_CHAR ';' +# elif defined(VMS) +# define PATH_SEPARATOR_CHAR ',' +# else +# define PATH_SEPARATOR_CHAR ':' +# endif #endif /* This is needed for getcwd() and chdir(). */ --- config/dospaths.m4~02006-03-10 06:20:45.0 +0200 +++ config/dospaths.m4 2006-08-18 21:12:32.859375000 +0300 @@ -22,7 +22,7 @@ AC_CACHE_CHECK([whether system uses MSDOS-style paths], [ac_cv_dos_paths], [ AC_COMPILE_IFELSE([ -#if !defined _WIN32 !defined __WIN32__ !defined __MSDOS__ !defined __EMX__ !defined __MSYS__ +#if !defined _WIN32 !defined __WIN32__ !defined __MSDOS__ !defined __EMX__ !defined __MSYS__ !defined __CYGWIN__ neither MSDOS nor Windows nor OS2 #endif ], --- job.c~0 2006-03-20 07:03:04.0 +0200 +++ job.c 2006-08-19 09:25:07.68750 +0300 @@ -2307,6 +2307,12 @@ construct_command_argv_internal (char *l login, logout, read, readonly, set, shift, switch, test, times, trap, umask, wait, while, 0 }; +# ifdef HAVE_DOS_PATHS + /* This is required if the MSYS/Cygwin ports (which do not define + WINDOWS32) are compiled with HAVE_DOS_PATHS defined, which uses + sh_chars_sh[] directly (see below). */ + static char *sh_chars_sh = sh_chars; +# endif /* HAVE_DOS_PATHS */ #endif register int i; register char *p; ___ Make-w32 mailing list Make-w32@gnu.org http://lists.gnu.org/mailman/listinfo/make-w32
RE: Problems with latest Cygwin make patch; builds OK but binary segfaults
It could be an infinite recursion in construct_command_argv_internal, but it could also be something else. First, is it reasonable to get a stack overflow in a Cygwin program after only 11 recursive invocations of construct_command_argv_internal? Can you verify directly that the stack overflowed? If not, we need to know what caused the crash. I am not familiar with the limits of the Cygwin internals. The reason I suggested a stack overflow was that when I run the patched binary from cmd.exe, I get the following output verbatim: 28518 [main] make 2200 _cygtls::handle_exceptions: Exception: STATUS_STACK_OVERFLOW 30469 [main] make 2200 open_stackdumpfile: Dumping stack trace to make.exe.stackdump Second, do I understand correctly that you were running make clean in the top-level directory of make-3.81 source tree? You didn't mention that explicitly anywhere (sorry if I missed something), and I wouldn't want to guess. I need to know what part of the Makefile caused the crash. If possible, could you please craft a minimal Makefile that has only the necessary bits to recreate the problem? Sorry, that was a bad example that was added as a convenience. I was under the impression that the generated binary crashed on any Makefile, and since I was already in a directory with a makefile, I just ran it there with the clean target. It turns out the Makefiles I was generating/selected were lucky guesses, however I have narrowed down the pattern for a failed Makefile. The basic cause of the problems are the ~ and ! characters defined in sh_chars, line 2304 of job.c in the untouched make 3.81 sources. Or rather, that a Cygwin build uses sh_chars when (I think) it should be using sh_chars_sh. While the patch does try to address this by adding a new #ifdef HAVE_DOS_PATHS for sh_chars_sh around line 2309, it doesn't seem to account for the use of sh_chars on lines 2462 and 2751. On those two lines (2462 and 2751), I did a quick #ifdef HAVE_DOS_PATHS that substituted sh_chars_sh instead of sh_chars, and all of the problems I had before went away. Whether this is a proper fix or not I am not sure; I'm starting to grasp the logic used in the function, but not quite to zen yet :) My crashing problem only affects Makefiles whose actions contain the ~ or ! characters. Here are two quick tests that should crash a patched make under Cygwin: $ echo 'all : ; @echo Fun!' Makefile $ make $ echo 'squiggle : ; @echo creating C:/PROGRA~1/some.file' Makefile $ make The latter example is very common when using 8.3 pathnames, which are still in use with some older compilation toolchains and sometimes seen on Windows 2000 with its 2047 character command line limit. This is a standard feature of GNU Patch: if the patch file names the file config/dospaths.m4, Patch will look in the config/ subdirectory. What command did you use to invoke Patch, and what version of Patch is that? I just used the latest patch package from the Cygwin installer. The patch commands were in my attached .script file. I figured it was a version issue; thanks for the heads up. $ patch --version patch 2.5.8 Copyright (C) 1988 Larry Wall Copyright (C) 2002 Free Software Foundation, Inc. This program comes with NO WARRANTY, to the extent permitted by law. You may redistribute copies of this program under the terms of the GNU General Public License. For more information about these matters, see the file named COPYING. written by Larry Wall and Paul Eggert Are you saying that the original unpatched binary of Make 3.81 also crashes with the same Makefile? If so, that doesn't seem to say anything about the patch itself, right? No, I was simply clarifying the output in my attached cygcheck.out file. The cygcheck program says I have version 3.81, however in my .script file, you will see that /usr/bin/make comes back as 3.80. I was just describing why that discrepency was there. I am able to build the vanilla make 3.81 sources and everything works just fine, final binary included. The only difference between the two builds is that I apply the patch(es) and run autoheader/conf before the typical ./configure make. William Sheehan Builds Engineer / Network Administrator Open Interface North America ___ Make-w32 mailing list Make-w32@gnu.org http://lists.gnu.org/mailman/listinfo/make-w32
RE: Problems with latest Cygwin make patch; builds OK but binary segfaults
Please rune make check on the version of Make you think fixes the problem. If all the tests pass, please post here a patch that you used to fix the recursion problem. Thanks. Done; the patch is attached. Running make check came back with zero failures and a nice smiley face. The patch has the same content as Bill Hoffman's patch, plus my modifications for job.c. No, it wasn't a version issue: your Patch command lacks the -p0 switch, which I think is the cause of your trouble. Thank you! Adding -p0 did indeed allow for a single patch file. William Sheehan Builds Engineer / Network Administrator Open Interface North America cygwin-make.patch Description: Binary data ___ Make-w32 mailing list Make-w32@gnu.org http://lists.gnu.org/mailman/listinfo/make-w32
RE: Problems with latest Cygwin make patch; builds OK but binary segfaults
At 01:52 PM 8/25/2006, William Sheehan wrote: Please rune make check on the version of Make you think fixes the problem. If all the tests pass, please post here a patch that you used to fix the recursion problem. Thanks. Done; the patch is attached. Running make check came back with zero failures and a nice smiley face. The patch has the same content as Bill Hoffman's patch, plus my modifications for job.c. The patch is really not mine, Eli, myself and Chris came up with it. And now you. :) Thanks for finding the problem. -Bill ___ Make-w32 mailing list Make-w32@gnu.org http://lists.gnu.org/mailman/listinfo/make-w32
RE: Problems with latest Cygwin make patch; builds OK but binary segfaults
The patch is really not mine, Eli, myself and Chris came up with it. And now you. :) Thanks for finding the problem. I stand corrected; thanks to all involved! William Sheehan Builds Engineer / Network Administrator Open Interface North America ___ Make-w32 mailing list Make-w32@gnu.org http://lists.gnu.org/mailman/listinfo/make-w32
Re: Problems with latest Cygwin make patch; builds OK but binary segfaults
From: William Sheehan [EMAIL PROTECTED] Date: Fri, 25 Aug 2006 10:52:00 -0700 Please rune make check on the version of Make you think fixes the problem. If all the tests pass, please post here a patch that you used to fix the recursion problem. Thanks. Done; the patch is attached. Running make check came back with zero failures and a nice smiley face. The patch has the same content as Bill Hoffman's patch, plus my modifications for job.c. Thanks. I'm traveling at the moment, but I promise to review the patch and the original problem as soon as I return home, in a few days. ___ Make-w32 mailing list Make-w32@gnu.org http://lists.gnu.org/mailman/listinfo/make-w32
Re: Problems with latest Cygwin make patch; builds OK but binary segfaults
From: William Sheehan [EMAIL PROTECTED] Date: Thu, 24 Aug 2006 17:23:04 -0700 I was interested in trying out Bill Hoffman's patch that would add DOS path support to the Cygwin make binary. Thank you for taking time to check this. This process seems to work in that I don't receive any obvious problems during. However, when I try to use the final binary, it segfaults (or stack overflow when invoked from cmd.exe). I have attached a zip file containing relevant files that will (hopefully) determine if I'm doing something bizarre or if this is a legitimate problem. Here are the contents: It could be an infinite recursion in construct_command_argv_internal, but it could also be something else. First, is it reasonable to get a stack overflow in a Cygwin program after only 11 recursive invocations of construct_command_argv_internal? Can you verify directly that the stack overflowed? If not, we need to know what caused the crash. Second, do I understand correctly that you were running make clean in the top-level directory of make-3.81 source tree? You didn't mention that explicitly anywhere (sorry if I missed something), and I wouldn't want to guess. I need to know what part of the Makefile caused the crash. If possible, could you please craft a minimal Makefile that has only the necessary bits to recreate the problem? Third, could you please step through the code in construct_command_argv_internal and see what part of the code causes it to recurse so many times, then compare that with the unpatched source of Make for the same command, and see what is causing the endless recursion? I don't have Cygwin installed, so I cannot do this myself. cygwin-make-config.patch -- Patch for dospaths.m4 (as an aside, what patch process recognizes subdirectories as a patch target? This is a standard feature of GNU Patch: if the patch file names the file config/dospaths.m4, Patch will look in the config/ subdirectory. What command did you use to invoke Patch, and what version of Patch is that? Replacing the binary with the original make 3.81-1 binary did not change the results. Are you saying that the original unpatched binary of Make 3.81 also crashes with the same Makefile? If so, that doesn't seem to say anything about the patch itself, right? My own analysis of the problem is that there is an infinite recursion call to construct_command_argv_internal with the line parameter; it keeps getting called with more and more backslash characters. It is normal for Make to recurse once in construct_command_argv_internal, when the shell is called to run the command. But the recursion should stop after the first level, not go on further. To unlock this riddle, we need to understand why it doesn't stop. TIA ___ Make-w32 mailing list Make-w32@gnu.org http://lists.gnu.org/mailman/listinfo/make-w32
Re: Problems with latest Cygwin make patch; builds OK but binary segfaults
At 08:23 PM 8/24/2006, William Sheehan wrote: First, I know that this build is under a Cygwin environment, however it appears that the problem I encountered was in the patched make sources themselves. Please let me know if there is a more appropriate venue for this problem. With that said... I was interested in trying out Bill Hoffman's patch that would add DOS path support to the Cygwin make binary. I have a good amount of experience building under Linux, however not as much building packages under Cygwin. Here are the steps I followed to try out this latest patch (from http://article.gmane.org/gmane.comp.gnu.make.windows/2136): 1) Download and untar the make-3.81 release sources 2) Apply the patch 3) Run autoheader and autoconf 4) ./configure CFLAGS=-g (afterthought to troubleshoot the segfault) 5) make This process seems to work in that I don't receive any obvious problems during. However, when I try to use the final binary, it segfaults (or stack overflow when invoked from cmd.exe). I have attached a zip file containing relevant files that will (hopefully) determine if I'm doing something bizarre or if this is a legitimate problem. Here are the contents: What happens if you run make check after the build? Do all the tests fail? -Bill ___ Make-w32 mailing list Make-w32@gnu.org http://lists.gnu.org/mailman/listinfo/make-w32
