Re: Misleading error when attempting to run foreign executable

[Alex fxmbsw7 Ratchev]

also enoexist with its description of the ld parser not available

On Wed, Oct 13, 2021, 16:06 Andrea Monaco 
wrote:

>
>   > [ENOENT] A component of path or file does not name an existing file
>   > or path or file is an empty string.
>
>
> Then ENOENT is clearly not an appropriate error in this case.
>
>
>   > [EINVAL] The new process image file has appropriate privileges and
>   > has a recognized executable binary format, but the system does not
>   > support execution of a file with this format.
>
>
> I think the most fitting is EINVAL, as Robert said.  Did someone tell
> the Linux kernel people, so they can fix this?  Otherwise I will.
>
>
>
> Andrea Monaco
>
>

Re: Misleading error when attempting to run foreign executable

[Andrea Monaco]

  > [ENOENT] A component of path or file does not name an existing file
  > or path or file is an empty string.


Then ENOENT is clearly not an appropriate error in this case.


  > [EINVAL] The new process image file has appropriate privileges and
  > has a recognized executable binary format, but the system does not
  > support execution of a file with this format.


I think the most fitting is EINVAL, as Robert said.  Did someone tell
the Linux kernel people, so they can fix this?  Otherwise I will.



Andrea Monaco

Re: Misleading error when attempting to run foreign executable

[Ángel]

On 2021-10-12 at 00:48 +0700, Robert Elz wrote:
>   | When execve returns this error for what is a clearly invalid
> reason,
> 
> The problem is that sometimes it will probably be an invalid reason
> (that's where the race conditions can matter, that the file was there
> when it was checked doesn't mean it is still there now, so the reason
> might be valid, and I'm not sure it is really worth another stat() to
> find out - and even if the stat() is done and that says the file
> exists,

Note I did NOT add a stat() call. It was already there in bash, *after*
execve() fails (so it can do a manual script run, etc.). I only changed
the error message it outputs in this case.

> the exec attempt might just have happened in the middle of a
>  mv a a.old; mv a.new a
> sequence).

Sure, that could happen.¹ Still, I think the error message saying that
execve() failed but the program exists is not that misguided. And, more
importantly, fixes the general issue of the misleading "No such file or
directory" for files requesting an ELF interpreter which is not
available in the current system.

If this race condition was such a big deal, then we should also be
concerned about a program being replaced with a directory. Or a shell
script with a binary file once it decides it can be executed as a
script.

If someone changes the binary between execve() time and when bash
examines it (as it already does), the diagnostic can be wrong. But
that's still no worse than the alternative of letting the user inspect
it manually.


Best regards



¹ And the obvious solution if it is expected a can be run at any point
would have been to instead do: ln a a.old; mv a.new a

Re: Misleading error when attempting to run foreign executable

[Eli Schwartz]

On 10/11/21 07:09, Robert Elz wrote:
> This is clearly an OS problem, not one in bash.
> 
> POSIX says of ENOENT as it applies to the exec*() set of functions:
> 
> [ENOENT]  A component of path or file does not name an existing file
>   or path or file is an empty string.
> 
> "path" and "file" (which are in italics in the text, that just doesn't
> make it to this ascii e-mail) are args to (different instances of) the
> exec*() functions (ie: they are not generic words).
> 
> That is, ENOENT is only applicable if the file (or path) named in the
> arg to the sys call does not exist.


Indeed, but the kernel.org "Linux man pages" project distributes
manpages which freely "reinterpret" this:

[ENOENT]
The file pathname or a script or ELF interpreter does not exist.

[ENOEXEC]
An executable is not in a recognized format, is for the wrong
architecture, or has some other format error that means it cannot be
executed.

[EINVAL]
An ELF executable had more than one PT_INTERP segment (i.e., tried
to name more than one interpreter).


> This should be fixed at OS level, bash isn't the only shell that exists,
> and shells aren't the only programs that attempt to exec binaries which
> look as if they should be executable.


This would no doubt be a fascinating conversation to have with the Linux
kernel developers, but to be honest it looks like an intentional POSIX
"deviation", shall we say.

In the meantime I think that the most likely response will be "but this
is behaving exactly as documented, just add a Linux case to your
application to handle Linux error codes".


> ps: as an aside, no-one cares about any race conditions here, even if bash
> (or some other process) were to attempt to examine the file after an exec
> failure to generate a nicer message ... the only thing affected would be
> the error message, and if you get weird messages when attempting to run
> binaries which are changing as you're doing it, it should be no surprise.


Indeed, that was part of why I suggested doing so was harmless.


-- 
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User


OpenPGP_signature
Description: OpenPGP digital signature

Re: Misleading error when attempting to run foreign executable

[Alex fxmbsw7 Ratchev]

the mv a old new looks like it can be achieved by cgroups freeze [i only
have hears so] and ye its not portable

if choosen to somehow freeze it, i dunno, transparently somehow, for
letting changes happen

btw on my secondlast bash.git.devel build it ended up with some binary that
also produced via debian chroot on aarch64 phone on termux no such file
when executing, on ldd the ld linux aarch doesnt show up, in debians ldd it
appears

well devel bugs

On Mon, Oct 11, 2021, 19:49 Robert Elz  wrote:

> Date:Mon, 11 Oct 2021 11:00:54 -0400
> From:Chet Ramey 
> Message-ID:  <45ecd950-9e9b-553b-132b-04d1dcfad...@case.edu>
>
>   | When execve returns this error for what is a clearly invalid reason,
>
> The problem is that sometimes it will probably be an invalid reason
> (that's where the race conditions can matter, that the file was there
> when it was checked doesn't mean it is still there now, so the reason
> might be valid, and I'm not sure it is really worth another stat() to
> find out - and even if the stat() is done and that says the file exists,
> the exec attempt might just have happened in the middle of a
>  mv a a.old; mv a.new a
> sequence).
>
> Other times (like exec of a /path/to/file) you won't have any idea at
> all in advance that the file apparently exists, so the same underlying
> issue would generate different messages in different circumstances if you
> try and handle the case where you believe error to be invalid (unless you
> check after every ENOENT -- even in the /path/to/file case).
>
> Better to get the OS fixed (those kinds of fixes tend to seem less
> urgent when applications work around them instead of simply complaining).
>
> But that said:
>
>   | it's not unreasonable that the shell help.
>
> it isn't unreasonable - as I said (in the postscript) "even if bash
> (or some other process) were to attempt to examine the file after an exec
> failure to generate a nicer message...", kind of indicating that it
> wouldn't be a wrong thing to do, just certainly not required.
>
> kre
>
>
>

Re: Misleading error when attempting to run foreign executable

[Robert Elz]

Date:Mon, 11 Oct 2021 11:00:54 -0400
From:Chet Ramey 
Message-ID:  <45ecd950-9e9b-553b-132b-04d1dcfad...@case.edu>

  | When execve returns this error for what is a clearly invalid reason,

The problem is that sometimes it will probably be an invalid reason
(that's where the race conditions can matter, that the file was there
when it was checked doesn't mean it is still there now, so the reason
might be valid, and I'm not sure it is really worth another stat() to
find out - and even if the stat() is done and that says the file exists,
the exec attempt might just have happened in the middle of a
 mv a a.old; mv a.new a
sequence).

Other times (like exec of a /path/to/file) you won't have any idea at
all in advance that the file apparently exists, so the same underlying
issue would generate different messages in different circumstances if you
try and handle the case where you believe error to be invalid (unless you
check after every ENOENT -- even in the /path/to/file case).

Better to get the OS fixed (those kinds of fixes tend to seem less
urgent when applications work around them instead of simply complaining).

But that said:

  | it's not unreasonable that the shell help.

it isn't unreasonable - as I said (in the postscript) "even if bash
(or some other process) were to attempt to examine the file after an exec
failure to generate a nicer message...", kind of indicating that it
wouldn't be a wrong thing to do, just certainly not required.

kre

Re: Misleading error when attempting to run foreign executable

[Chet Ramey]

On 10/11/21 7:09 AM, Robert Elz wrote:
> Date:Sun, 10 Oct 2021 21:09:53 -0400
> From:Eli Schwartz 
> Message-ID:  <4a362385-066d-0795-9a02-ff8bbb920...@archlinux.org>
> 
>   | So I wonder, if bash already in this exact case attempts to open() the
>   | file and read() it to look for a shebang, what's the harm in assuming
>   | (or checking) that it exists in this patch?
> 
> This is clearly an OS problem, not one in bash.

This is true. And yet:

> POSIX says of ENOENT as it applies to the exec*() set of functions:
> 
> [ENOENT]  A component of path or file does not name an existing file
>   or path or file is an empty string.

When execve returns this error for what is a clearly invalid reason, it's
not unreasonable that the shell help. It's a quality of implementation
issue at that point. If execve ever returns a different error for this
situation, the 5 lines of code will no longer be needed.


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: Misleading error when attempting to run foreign executable

[Robert Elz]

Date:Sun, 10 Oct 2021 21:09:53 -0400
From:Eli Schwartz 
Message-ID:  <4a362385-066d-0795-9a02-ff8bbb920...@archlinux.org>

  | So I wonder, if bash already in this exact case attempts to open() the
  | file and read() it to look for a shebang, what's the harm in assuming
  | (or checking) that it exists in this patch?

This is clearly an OS problem, not one in bash.

POSIX says of ENOENT as it applies to the exec*() set of functions:

[ENOENT]  A component of path or file does not name an existing file
or path or file is an empty string.

"path" and "file" (which are in italics in the text, that just doesn't
make it to this ascii e-mail) are args to (different instances of) the
exec*() functions (ie: they are not generic words).

That is, ENOENT is only applicable if the file (or path) named in the
arg to the sys call does not exist.

On the other hand:

[EINVAL]  The new process image file has appropriate privileges and has
  a recognized executable binary format, but the system does not
  support execution of a file with this format.

looks like it would be an acceptable error for the situation described
(which would also cover when the file is an executable for a different
architecture, and cases like that).

[ENOEXEC]  The new process image file has the appropriate access permission
   but has an unrecognized format.

might almost qualify (but not really, as I believe the file was
understood to be a known executable format).

This should be fixed at OS level, bash isn't the only shell that exists,
and shells aren't the only programs that attempt to exec binaries which
look as if they should be executable.

kre

ps: as an aside, no-one cares about any race conditions here, even if bash
(or some other process) were to attempt to examine the file after an exec
failure to generate a nicer message ... the only thing affected would be
the error message, and if you get weird messages when attempting to run
binaries which are changing as you're doing it, it should be no surprise.

Re: Misleading error when attempting to run foreign executable

[Dmitry Goncharov via Bug reports for the GNU Bourne Again SHell]

On Sun, Oct 10, 2021 at 9:11 PM Eli Schwartz  wrote:
> So I wonder, if bash already in this exact case attempts to open() the
> file and read() it to look for a shebang, what's the harm in assuming
> (or checking) that it exists in this patch?

i guess, you are right.
i, on the other hand wonder, if exec could return ELIBBAD.

regards, Dmitry

Re: Misleading error when attempting to run foreign executable

[Eli Schwartz]

On 10/10/21 20:24, Dmitry Goncharov via Bug reports for the GNU Bourne
Again SHell wrote:
> On Sun, Oct 10, 2021 at 7:52 PM Ángel  wrote:
>> Looking at the code, it doesn't even need an extra stat(), it already
>> knows the file exists
> 
> Attempts to know ahead of time or check afterwards that the file
> exists are subject to a race condition.


Immediately above the patch diff you are responding to, is:



#if defined (HAVE_HASH_BANG_EXEC)
  READ_SAMPLE_BUF (command, sample, sample_len);
  if (sample_len > 0)
sample[sample_len - 1] = '\0';
  if (sample_len > 2 && sample[0] == '#' && sample[1] == '!')
{
  char *interp;
  int ilen;

  interp = getinterp (sample, sample_len, (int *)NULL);
  ilen = strlen (interp);
  errno = i;
  if (interp[ilen - 1] == '\r')
{
  interp = xrealloc (interp, ilen + 2);
  interp[ilen - 1] = '^';
  interp[ilen] = 'M';
  interp[ilen + 1] = '\0';
}
  sys_error (_("%s: %s: bad interpreter"), command, interp ? interp
: "");
  FREE (interp);
  return (EX_NOEXEC);
}
#endif



So I wonder, if bash already in this exact case attempts to open() the
file and read() it to look for a shebang, what's the harm in assuming
(or checking) that it exists in this patch?



-- 
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User


OpenPGP_signature
Description: OpenPGP digital signature

Re: Misleading error when attempting to run foreign executable

[Dmitry Goncharov via Bug reports for the GNU Bourne Again SHell]

On Sun, Oct 10, 2021 at 7:52 PM Ángel  wrote:
> Looking at the code, it doesn't even need an extra stat(), it already
> knows the file exists

Attempts to know ahead of time or check afterwards that the file
exists are subject to a race condition.

regards, Dmitry

Re: Misleading error when attempting to run foreign executable

[Ángel]

I agree with Ilkka and Alex: it would be helpful to show a different
error message in this case.

I know about ELF format and what leads to this, and it's still slightly
confusing when faced to it. You have to double-check that the file is
indeed there (not a mistyped command), and then locate the right
readelf incantation to find the requested interpreter and see that,
yes, it happens to not to be available on this host.

Looking at the code, it doesn't even need an extra stat(), it already
knows the file exists and only providing a different the error message
is needed. I'm attaching such patch against devel.

Best regards

diff --git a/execute_cmd.c b/execute_cmd.c
index 782ae678..0d3e8979 100644
--- a/execute_cmd.c
+++ b/execute_cmd.c
@@ -5992,8 +5992,20 @@ shell_execve (command, args, env)
 	  return (EX_NOEXEC);
 	}
 #endif
-	  errno = i;
-	  file_error (command);
+	  if (i == ENOENT)
+	{
+	  /* execve returned ENOENT, but since executable_file()
+	 returned non-zero, we know the file actually exists.
+	 It probably requests an ELF interpreter which is missing.
+	 Provide a helpful message noting that the program is
+	 actually there. */
+	  internal_error (_("%s: execve failed with `%s' albeit program exists. Needs a missing ELF interpreter?"), command, strerror (ENOENT));
+	}
+	  else
+	{
+	  errno = i;
+	  file_error (command);
+	}
 	}
   return (last_command_exit_value);
 }

Re: Misleading error when attempting to run foreign executable

[Alex fxmbsw7 Ratchev]

id apprechiate such a fmt update
thanks

On Fri, Oct 8, 2021, 17:44 Chet Ramey  wrote:

> On 10/4/21 10:06 AM, Ilkka Virta wrote:
> > On Mon, Oct 4, 2021 at 4:46 PM Chet Ramey  > > wrote:
> >
> > Bash reports the error it gets back from execve. In this case, it's
> > probably that the loader specified for the executable isn't present
> on your
> > system.
> >
> >
> > OTOH, for a script, Bash checks to see if the problem is with the
> > interpreter and reports accordingly:
> >
> >   $ ./foo.sh
> > bash: ./foo.sh: /bin/noexist: bad interpreter: No such file or directory
> >
> > The shell does go on to stat() the file after getting ENOENT from
> execve(),
> > so I suppose it could
> > add some clarifying note to the error message for the case of a binary
> file
> > too.
>
> About the only other thing it could say would be "cannot execute" or
> "exists but cannot execute" in addition to "no such file or directory."
> The shell doesn't know anything else. I'm not sure that would be a
> significant improvement: you already know you can't execute the file and
> you can easily check yourself whether or not the file exists.
>
> Chet
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>  ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
>
>

Re: Misleading error when attempting to run foreign executable

[Chet Ramey]

On 10/4/21 10:06 AM, Ilkka Virta wrote:
On Mon, Oct 4, 2021 at 4:46 PM Chet Ramey > wrote:


Bash reports the error it gets back from execve. In this case, it's
probably that the loader specified for the executable isn't present on your
system.


OTOH, for a script, Bash checks to see if the problem is with the 
interpreter and reports accordingly:


  $ ./foo.sh
bash: ./foo.sh: /bin/noexist: bad interpreter: No such file or directory

The shell does go on to stat() the file after getting ENOENT from execve(), 
so I suppose it could
add some clarifying note to the error message for the case of a binary file 
too.


About the only other thing it could say would be "cannot execute" or
"exists but cannot execute" in addition to "no such file or directory."
The shell doesn't know anything else. I'm not sure that would be a
significant improvement: you already know you can't execute the file and
you can easily check yourself whether or not the file exists.

Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: Misleading error when attempting to run foreign executable

[Dennis Williamson]

On Mon, Oct 4, 2021, 9:09 AM Ilkka Virta  wrote:

> On Mon, Oct 4, 2021 at 4:46 PM Chet Ramey  wrote:
>
> > Bash reports the error it gets back from execve. In this case, it's
> > probably that the loader specified for the executable isn't present on
> your
> > system.
> >
>
> OTOH, for a script, Bash checks to see if the problem is with the
> interpreter and reports accordingly:
>
>  $ ./foo.sh
> bash: ./foo.sh: /bin/noexist: bad interpreter: No such file or directory
>
> The shell does go on to stat() the file after getting ENOENT from execve(),
> so I suppose it could
> add some clarifying note to the error message for the case of a binary file
> too.
>

The error is produced by the kernel. Bash merely passes it along.

>

Re: Misleading error when attempting to run foreign executable

[Ilkka Virta]

On Mon, Oct 4, 2021 at 4:46 PM Chet Ramey  wrote:

> Bash reports the error it gets back from execve. In this case, it's
> probably that the loader specified for the executable isn't present on your
> system.
>

OTOH, for a script, Bash checks to see if the problem is with the
interpreter and reports accordingly:

 $ ./foo.sh
bash: ./foo.sh: /bin/noexist: bad interpreter: No such file or directory

The shell does go on to stat() the file after getting ENOENT from execve(),
so I suppose it could
add some clarifying note to the error message for the case of a binary file
too.

Re: Misleading error when attempting to run foreign executable

[Chet Ramey]

On 10/3/21 6:05 PM, Andrea Monaco wrote:
> 
> Hello,
> 
> 
> I'm using bash 5.0.3 on GNU/Linux. When I mount a GNU/Hurd filesystem and
> try to run an executable found there, I get for example:
> 
>   bash: ./ext2fs: No such file or directory
> 
> This seems a misleading error to me. The file is there and is
> executable, I guess it can't be run on my system but I think that
> another error message would be more appropriate.
> 
> Is this an issue with bash or maybe with the kernel?

Bash reports the error it gets back from execve. In this case, it's
probably that the loader specified for the executable isn't present on your
system.

Chet


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: Misleading error when attempting to run foreign executable

[Alex fxmbsw7 Ratchev]

it is trying to execute its ld.so parser which linux doesnt have of hurd (
or so ) , thats why u get no file

On Mon, Oct 4, 2021, 02:20 Andrea Monaco 
wrote:

>
> Hello,
>
>
> I'm using bash 5.0.3 on GNU/Linux. When I mount a GNU/Hurd filesystem and
> try to run an executable found there, I get for example:
>
>   bash: ./ext2fs: No such file or directory
>
> This seems a misleading error to me. The file is there and is
> executable, I guess it can't be run on my system but I think that
> another error message would be more appropriate.
>
> Is this an issue with bash or maybe with the kernel?
>
>
>
> Let me know,
>
> Andrea Monaco
>
>
>

Misleading error when attempting to run foreign executable

[Andrea Monaco]

Hello,


I'm using bash 5.0.3 on GNU/Linux. When I mount a GNU/Hurd filesystem and
try to run an executable found there, I get for example:

  bash: ./ext2fs: No such file or directory

This seems a misleading error to me. The file is there and is
executable, I guess it can't be run on my system but I think that
another error message would be more appropriate.

Is this an issue with bash or maybe with the kernel?



Let me know,

Andrea Monaco