Re: OSError: [WinError 193] %1 is not a valid Win32 application
Hi Collin, > I can create > testdirs on Cygwin and MSYS2. Perhaps it is best to clarify > gnulib-tool.py is only supported there on Windows? Yes, feel free to add a bit of text about it to the DEPENDENCIES file. Other environments than Cygwin and MSYS2 don't need to be considered, since the user will also need sh, autoconf, automake, perl, gcc, and so on. > In this case shebang lines cause the issue. But there is also other > considerations. Windows shells use a different way of quoting, not > sure about encoding. Also gnulib-tool.py executes sed, find, patch, so > I'm not too sure how that works in Windows shells. You can assume that 'sh' (sh.exe) will be found in PATH. You don't need to think about cmd.exe or PowerShell — this would be a waste of time. Just rely on 'sh'. Bruno
Re: OSError: [WinError 193] %1 is not a valid Win32 application
On 5/1/24 7:26 AM, Jeffrey Walton wrote: > The shebang is wrong. I merely pointed out the portable way to write one. I think the issue that you brought up '/usr/bin' vs. '/usr/local/bin', as the BSDs use, is solved by Autoconf and Automake's installation process. They require the normal './configure && make && make install' and in that process they replace: #!@PERL@ with #!/usr/bin/perl or wherever it locates the system installation. We can't do that with gnulib scripts which don't get installed. Therefore, it is assumed /bin/sh exists and is a normal-ish POSIX shell. In the case of prefix-gnulib-mk, a perl script, perl is then executed from the users $PATH. #!/bin/sh eval 'exec perl -wSx "$0" "$@"' if 0; I think that should do the same as what you mentioned using '#!/usr/bin/env perl'. Collin
Re: OSError: [WinError 193] %1 is not a valid Win32 application
On 5/1/24 3:50 AM, Bruno Haible wrote: > Indeed, that is likely the explanation: All 3 programs (autoconf, > autoheader, automake) start with '#!/usr/bin/perl', and while native > Windows does not interpret the shebang line, the Cygwin and MSYS2 shells > apparently do. I created a Windows VM last night to test things. I can create testdirs on Cygwin and MSYS2. Perhaps it is best to clarify gnulib-tool.py is only supported there on Windows? I can create a seperate thread about some issues I see with running gnulib-tool.py on native Windows. I don't have much experience with it so maybe someone can chime in there. In this case shebang lines cause the issue. But there is also other considerations. Windows shells use a different way of quoting, not sure about encoding. Also gnulib-tool.py executes sed, find, patch, so I'm not too sure how that works in Windows shells. Collin
Re: OSError: [WinError 193] %1 is not a valid Win32 application
Hi Jeffrey, We don't need to discuss how "make install" in Autoconf and Automake should work. Let's keep focused on the current question: how to invoke the Autoconf and Automake programs, assuming they already exist on the user's disk. Thanks. Bruno
Re: OSError: [WinError 193] %1 is not a valid Win32 application
On Wed, May 1, 2024 at 10:11 AM Bruno Haible wrote: > Hi Jeffrey, > > We don't need to discuss how "make install" in Autoconf and Automake should > work. Let's keep focused on the current question: how to invoke the > Autoconf and Automake programs, assuming they already exist on the user's > disk. > I'm not sure what you mean. The shebang is wrong. I merely pointed out the portable way to write one. Jeff
Re: OSError: [WinError 193] %1 is not a valid Win32 application
On Wed, May 1, 2024 at 6:51 AM Bruno Haible wrote: > [...] > > I assume that windows doesn't do anything about the '#!/bin/sh' line. > > Indeed, that is likely the explanation: All 3 programs (autoconf, > autoheader, automake) start with '#!/usr/bin/perl', and while native > Windows does not interpret the shebang line, the Cygwin and MSYS2 shells > apparently do. > '#!/usr/bin/env perl' is probably a better choice since it is Posix. It will also make a difference on OpenBSD, where stuff in /usr is provided by the OS; and everything else gets installed /usr/local, like Bash after it is installed. (I think it is OpenBSD, but I know it is one of the BSDs). Jeff
Re: OSError: [WinError 193] %1 is not a valid Win32 application
Hi Collin, > > * We have basically three ways to deal with this (that I can see): > > a) Update the requirements and say that a Cygwin environment with a > > Cygwin-based Python build is required. (I tested that; so, we know > > that works.) > > b) Change the gnulib-tool entry-point script to prefer the shell-based > > implementation if the host OS, as viewed from Python, is native > > Windows (not Cygwin). > > c) Port gnulib-tool.py to native Windows. > > Your choice. > > Ideally, it would probably be best to get gnulib-tool.py to work on > Windows. Since Python should do all the heavy lifting in that area. OK. > I assume that windows doesn't do anything about the '#!/bin/sh' line. Indeed, that is likely the explanation: All 3 programs (autoconf, autoheader, automake) start with '#!/usr/bin/perl', and while native Windows does not interpret the shebang line, the Cygwin and MSYS2 shells apparently do. > Maybe placing the shell as argv[0] in sp.run would fix it? I think that's already taken into account by the Python implementation of shell=True. Since the reporter says that this fixes the problem (and it is more future-proof than setting argv[0] to "perl"), I would do that — but on native Windows only. That is, test os.name or sys.platform first, so that we avoid a slowdown on Unix. Bruno
Re: OSError: [WinError 193] %1 is not a valid Win32 application
On 4/30/24 6:44 AM, Bruno Haible wrote: > Collin: Before you just apply this patch, two notes: > * The patch is a pessimization on Unix systems, since executing with > shell=True is slower than without. > * We have basically three ways to deal with this (that I can see): > a) Update the requirements and say that a Cygwin environment with a > Cygwin-based Python build is required. (I tested that; so, we know > that works.) > b) Change the gnulib-tool entry-point script to prefer the shell-based > implementation if the host OS, as viewed from Python, is native > Windows (not Cygwin). > c) Port gnulib-tool.py to native Windows. > Your choice. Ideally, it would probably be best to get gnulib-tool.py to work on Windows. Since Python should do all the heavy lifting in that area. However, you mentioned previously that it has it's own issues with Autotools and such. Since it isn't free, I assume that anything there is "best effort" for the most part. You make a good point about shell=True being slower. I wonder if it is even necessary here. When I see this error message: OSError: [WinError 193] %1 is not a valid Win32 application I assume that windows doesn't do anything about the '#!/bin/sh' line. Maybe placing the shell as argv[0] in sp.run would fix it? I'm not too sure where that is typically installed on Windows though... Collin
Re: OSError: [WinError 193] %1 is not a valid Win32 application
Hi, Jianshan Jiang wrote: > I use the newest version of gnulib from git. when I run the > ./autogen.sh where I create for sed-4.9. gnulib-tool will throw an > exception like below: > ... > File "C:\Python312\Lib\subprocess.py", line 1538, in _execute_child > hp, ht, pid, tid = _winapi.CreateProcess(executable, args, >^^^^^^^^^^^^^^^^^^^ > OSError: [WinError 193] %1 is not a valid Win32 application Two questions and one suggestion: 1) You're apparently using Windows and can execute shell scripts. Which environment are you using for doing so? Cygwin? MSYS2? Other? 2) Why are you spending time to write an autogen.sh script for sed, when the README-hacking file recommends to use the 'bootstrap' script for the same purpose? And a suggestion: What if you set the environment variable GNULIB_TOOL_IMPL=sh ? Does your autogen.sh work then? > Finally, I have found the solution. I have add the 'shell=True' into > function sp.run in GLImport.py. You can see the detail > in 001-gnulib-fix-WinError-193.diff. Collin: Before you just apply this patch, two notes: * The patch is a pessimization on Unix systems, since executing with shell=True is slower than without. * We have basically three ways to deal with this (that I can see): a) Update the requirements and say that a Cygwin environment with a Cygwin-based Python build is required. (I tested that; so, we know that works.) b) Change the gnulib-tool entry-point script to prefer the shell-based implementation if the host OS, as viewed from Python, is native Windows (not Cygwin). c) Port gnulib-tool.py to native Windows. Your choice. Bruno
OSError: [WinError 193] %1 is not a valid Win32 application
Hello, I use the newest version of gnulib from git. when I run the ./autogen.sh where I create for sed-4.9. gnulib-tool will throw an exception like below: top/README-release top/maint.mk Traceback (most recent call last): File "E:\Githubs\msvc-pkgs\pkgs\gnulib\.gnulib-tool.py", line 30, in main.main_with_exception_handling() File "E:\Githubs\msvc-pkgs\pkgs\gnulib\pygnulib\main.py", line 1367, in main_with_exception_handling main() File "E:\Githubs\msvc-pkgs\pkgs\gnulib\pygnulib\main.py", line 953, in main importer.execute(filetable, transformers) File "E:\Githubs\msvc-pkgs\pkgs\gnulib\pygnulib\GLImport.py", line 1222, in execute emit = sp.run([joinpath(DIRS['root'], 'build-aux/prefix-gnulib-mk'), '--from-gnulib-tool', ^^^ File "C:\Python312\Lib\subprocess.py", line 548, in run with Popen(*popenargs, **kwargs) as process: ^^^ File "C:\Python312\Lib\subprocess.py", line 1026, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "C:\Python312\Lib\subprocess.py", line 1538, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, ^^^^^^^^^^^ OSError: [WinError 193] %1 is not a valid Win32 application Finally, I have found the solution. I have add the 'shell=True' into function sp.run in GLImport.py. You can see the detail in 001-gnulib-fix-WinError-193.diff. #!/bin/sh # Convenience script for regenerating all autogeneratable files that are # omitted from the version control repository. In particular, this script # also regenerates all aclocal.m4, config.h.in, Makefile.in, configure files # with new versions of autoconf or automake. # # This script requires autoconf-2.64..2.71 and automake-1.13..1.16 in the PATH. # Copyright (C) 2003-2023 Free Software Foundation, Inc. # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see <https://www.gnu.org/licenses/>. # Prerequisite (if not used from a released tarball): either # - the GNULIB_SRCDIR environment variable pointing to a gnulib checkout, or # - a preceding invocation of './autopull.sh'. # # Usage: ./autogen.sh [--skip-gnulib] # # Options: # --skip-gnulib Avoid fetching files from Gnulib. # This option is useful # - when you are working from a released tarball (possibly # with modifications), or # - as a speedup, if the set of gnulib modules did not # change since the last time you ran this script. # Nuisances. (unset CDPATH) >/dev/null 2>&1 && unset CDPATH skip_gnulib=false skip_gnulib_option= while :; do case "$1" in --skip-gnulib) skip_gnulib=true; skip_gnulib_option='--skip-gnulib'; shift;; *) break ;; esac done # The tests in gettext-tools/tests are not meant to be executable, because # they have a TESTS_ENVIRONMENT that specifies the shell explicitly. if ! $skip_gnulib; then if test -n "$GNULIB_SRCDIR"; then test -d "$GNULIB_SRCDIR" || { echo "*** GNULIB_SRCDIR is set but does not point to an existing directory." 1>&2 exit 1 } else GNULIB_SRCDIR=`dirname $(pwd)`/gnulib test -d "$GNULIB_SRCDIR" || { echo "*** Subdirectory 'gnulib' does not yet exist. Use './gitsub.sh pull' to create it, or set the environment variable GNULIB_SRCDIR." 1>&2 exit 1 } fi # Now it should contain a gnulib-tool. GNULIB_TOOL="$GNULIB_SRCDIR/gnulib-tool" test -f "$GNULIB_TOOL" || { echo "*** gnulib-tool not found." 1>&2 exit 1 } # https://www.gnu.org/software/gnulib/MODULES.html GNULIB_MODULES_POSIX_SUPPORT=' acl alloca binary-io btowc c-ctype closeout dfa eloop-threshold extensions fwriting getdelim gettext-h git-version-gen gitlog-to-changelog idx ignore-value localcharset manywarnings mbrlen mbrtowc mbsinit memchr memrchr minmax mkostemp obstack perl progname readlink readme-release regex rename selinux-h ssize_t stat-m