[bug #56855] find -printf %h gives an empty string for the root directory '/'

[Eric Blake]

Follow-up Comment #3, bug #56855 (project findutils):

POSIX is clear in
https://pubs.opengroup.org/onlinepubs/9699919799/functions/dirname.html that
the dirname(3) function returns a non-empty string for input of "/".  However,
POSIX is also clear that the string composed by
'dirname(str)+"/"+basename(str)' need NOT refer to the same file as 'str';
case in point: when str is "//", POSIX says you could end up with any of
"///", "", or "/" depending on choices made in basename and dirname,
but all of those results are equivalent to to "/" while POSIX is explicit that
"//" may be distinct from "/".

Similar arguments can be made about the behavior of the basename(1) utility.

However, since -printf is not defined by POSIX, there is no requirement that
%h be defined by the POSIX functionality of basename(3) (the function), nor of
basename(1) (the utility).  However, if we intend to return the empty string,
it IS a bug if our documentation is not explicit that we intentionally differ
in behavior from the POSIX functions, and give examples why we think that is
the right thing to do.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

Re: [PATCH 3/3] maint: avoid warnings from sparse tool

[Eric Blake]

On Thu, Jan 06, 2022 at 04:47:40PM +0100, Bernhard Voelker wrote:
> https://sparse.docs.kernel.org/
> Running the tool against unviled the following warnings:

unveiled

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

POSIX conformance question on -perm

[Eric Blake]

In today's Austin Group meeting (the folks responsible for the POSIX
standard), the following question was raised:

https://austingroupbugs.net/view.php?id=1392

"The below proposed wording change would make gnu find non-conforming.
Their behavior seems to be intentional
(https://git.savannah.gnu.org/cgit/findutils.git/commit/?id=0da0554e49);
would they be willing to change their implementation?"

Here's the proposed change, either we can express a willingness to
change findutils to match (changing commit 0da0554e49), or we should
propose alternative wording that permits the difference in behavior for
GNU find vs. other find implementations along with an explanation why
the difference is worth preserving.

On page 2797 lines 91951-91959 change:

The mode argument is used to represent file mode bits.
It shall be identical in format to the symbolic_mode operand
described in chmod, and shall be interpreted as follows. To
start, a template shall be assumed with all file mode bits cleared. An
op symbol of '+' shall set the appropriate mode bits in the
template; '?' shall clear the appropriate bits; '=' shall set the
appropriate mode bits, without regard to the contents of the file mode
creation mask of the process. The op symbol of '?' cannot be the
first character of mode; this avoids ambiguity with the optional leading
. Since the initial mode is all bits off, there are not
any symbolic modes that need to use '?' as the first character.

to:

The mode argument is used to represent file mode
bits. It shall be processed in an identical matter to the
symbolic_mode operand described in chmod, except that:



The changes to file mode bits shall be applied to a template
instead of to any files.  The template shall initially have all file
mode bits cleared.

The op symbol '-' cannot be the first character of
mode; this avoids ambiguity with the optional leading
. Since the initial mode is all bits off, there are not
any symbolic modes that need to use '-' as the first character.




On page 2801 lines 92128-92130 (find example #3), change:

find . -perm ?o+w,+s

prints (-print is assumed) the names of all files in or below
the current directory, with all of the file permission bits S_ISUID,
S_ISGID, and S_IWOTH set.

to:

find . -perm -o+w,+s

prints (-print is assumed) the names of all files in or below
the current directory, with all of the file permission bits S_ISUID,
S_ISGID, and S_IWOTH set, regardless of the value of the file creation
mask. (Note that S_ISUID and S_ISGID cannot be present in the
mask.)


On page 2801 after line 92130 insert a new example #4 and renumber the
remaining examples:

find . -perm -+w

prints (-print is assumed) the names of all files in or below
the current directory, with S_IWUSR set if the file creation mask does
not have S_IWUSR set (otherwise the S_IWUSR bit is ignored), S_IWGRP set
if the file creation mask does not have S_IWGRP set (otherwise S_IWGRP
is ignored), and S_IWOTH set if the file creation mask does not have
S_IWOTH set (otherwise S_IWOTH is ignored).

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: Is there a way to let find return non-zero when nothing is found?

[Eric Blake]

On 11/11/19 3:45 AM, Peng Yu wrote:

Hi,

Sometimes, I'd like to know whether there is nothing found. Is there a
way to let find return none-zero when nothing is found? Thanks.


Not built in to find directly, but you can emulate it via a number of 
means, such as having find touch a witness file in addition to whatever 
else you were having it do when it finds any file, then testing if the 
witness file exists afterwards.  If the action you are having find 
perform is printing a list of files, it's even simpler: check whether 
output was produced.


Maybe it's worth adding something into find to do what you're asking, 
but I don't know that I would be the one to write such a patch.  We'd 
also have to be very clear about what the semantics are (is it a command 
line option, or would it be a predicate that you can place alongside 
other predicates where position matters).


--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: xargs prompts with the same string, but executes differently

[Eric Blake]

On 7/29/19 1:49 PM, 積丹尼 Dan Jacobson wrote:
> P.S.,
> $ xargs --help
>   -I R same as --replace=R
>   -i, --replace[=R]replace R in INITIAL-ARGS with names read
>  from standard input; if R is unspecified,
>  assume {}
> ...
> 
> Don't you want to mention one is deprecated? Or even remove the
> deprecated one from --help entirely?

Just because POSIX deprecated -i doesn't mean we have to.  We fully
support both -i and -I, with no plans to deprecate either one.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: xargs prompts with the same string, but executes differently

[Eric Blake]

On 7/24/19 4:20 PM, Bernhard Voelker wrote:

> I agree that another new option is way too much.  That question to the user
> is meant for interactive use only anyway, so what about simply:
> 
> diff --git a/xargs/xargs.c b/xargs/xargs.c
> index 913328b8..85084123 100644
> --- a/xargs/xargs.c
> +++ b/xargs/xargs.c
> @@ -1093,7 +1093,7 @@ print_args (bool ask)
> 
>for (i = 0; i < bc_state.cmd_argc - 1; i++)
>  {
> -  if (fprintf (stderr, "%s ", bc_state.cmd_argv[i]) < 0)
> +  if (fprintf (stderr, "%s ", quotearg_n_style (0, 
> shell_escape_quoting_style, bc_state.cmd_argv[i])) < 0)
> die (EXIT_FAILURE, errno, _("Failed to write to stderr"));

Looks reasonable to me (other than line wrapping).

>  }
> 
> Example:
> 
>   $ printf '%s\0' 000 '0 0' '0"0' "0'0" \
>   | xargs/xargs -0p '-I{}' echo hello '{}' world
>   echo hello 000 world ?...y
>   hello 000 world
>   echo hello '0 0' world ?...y
>   hello 0 0 world
>   echo hello '0"0' world ?...y
>   hello 0"0 world
>   echo hello "0'0" world ?...y
>   hello 0'0 world
> 
> Well, this also changes plain -t output (i.e., without -p), but it's not
> that bad actually: it doesn't seem to be specified anywhere how -t output
> should look like.  Just in case, we could still guard quotearg_n_style
> by "if (ask)".

POSIX merely says that -p implies -t, so I argue that consistency
between the two is better than only doing it for -p. I also agree that
POSIX doesn't seem to have any hard rules about what the format should
be, so doing the quoting unconditionally looks compliant.

> 
> Do you see any drawback / corner case?

Not so far.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: xargs prompts with the same string, but executes differently

[Eric Blake]

On 7/24/19 6:28 AM, James Youngman wrote:
> On Tue, Jul 23, 2019 at 7:51 PM 積丹尼 Dan Jacobson  wrote:
>>
>> OK.
>> But I now "demand" that xargs add a --show-quoting or whatever argument,
> 
> Your demand must be presented in the form of a patch.

Is a new option necessary, or can we just use gnulib's quotearg() module
unconditionally?  I guess the only reason to make it a new command-line
option would be to allow the user to specify which of the quotearg
styles they prefer to see (for reference, 'ls --quoting-style=WORD' is a
good example of producing various representations of the output to avoid
ambiguities).  If we do use gnulib's quotearg module, we also benefit
from QUOTING_STYLE in the environment regardless of whether we add a
command-line option.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: xargs prompts with the same string, but executes differently

[Eric Blake]

On 7/23/19 1:31 PM, Assaf Gordon wrote:

> 
> This is not about "-p", but simply that "{}" puts the entire input line
> ("0 0") as a single parameter (as if quoted).
> 
> The correct comparison is:
> 
>   echo | xargs -p xdotool mousemove 0 0 click 1 mousemove restore
> vs
>   echo | xargs -p xdotool mousemove "0 0" click 1 mousemove restore

Still, it would be nice if xargs could add appropriate quoting in its -p
output to avoid the ambiguity.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: find usage question

[Eric Blake]

On 3/19/19 10:56 AM, Budi wrote:
> How to instruct 'find' to perform -exec with its argument is from
> -printf, not from the default find output.
> tried so far in vain:
> find ~+ -type d -printf '%p /d/data/%p_%s\n' -exec cp \{\} \

You can't. But what you can do is:

find ... -printf 'cp %p /d/data/%p_%s\n' | xargs -L1

where you instruct the -printf form to output the entire command you
want executed (rather than trying to piece together the command with
-exec), and instead have xargs execute it.


-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: Problem with "." and ","

[Eric Blake]

On 3/15/19 5:45 AM, Michal wrote:
> Hi,
> I terrible sorry if I'm wasting you time, but probably I found a bug. In
> example like this:
>  find /home/certget/ssl/example.pem -mtime -0,1
> This will work on server with locales changed to "pl_PL.UTF-8" and on
> servers with "en_US.UTF-8" I need to use "-mtime -0.1". It's very
> irritating when I need to check if this same script works on every server.

Not a bug, but by design - the behavior of strtod() is locale-dependent.
In locales where ',' is treated as a decimal point (such as
pl_PL.UTF-8), strtod("0,1") returns 0.1 (the value 1/10); in locales
where '.' is decimal point (such as en_US.UTF-8), it returns 0 with ",1"
as trailing garbage.

> Is it possible to unify this and use for example always ".".

strtod() always parses "." as a decimal separator regardless of what
other characters it might ALSO parse depending on locale.  Beyond that,
it is your responsibility to be aware of what locale your script is using.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re:

[Eric Blake]

On 3/14/19 9:49 PM, Budi wrote:
> How do we make findutil search in ply by ply depth instead of binary
> search tree, ie. how to make it search on a ply depth thoroughly
> without getting inside any directory there before all par member is
> read and retrieved

I'm not exactly sure what your question is, but POSIX already requires
'find' to default to breadth-first traversal (find visits all files in
the current directory before descending to a subdirectory) so that the
-prune predicate works; but if the -depth predicate is supplied, then
'find' executes in depth-first traversal (subdirectories are visited
before files in the parent directory).

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [PATCH] find -samedev NAME: true for files on the same device as NAME

[Eric Blake]

On 11/1/18 10:56 AM, Kamil Dudka wrote:

There is no easy way to exclude mount points from the output of `find`.
The options -xdev or -mount prevent `find` from traversing over the
mount points but do not exclude the mount points themselves.  It could
be scripted by AWK on Linux as in the following example:

$ find / -xdev -maxdepth 1 $(awk '$2 != "/" { printf("! -path %s ", $2) }' 
/proc/mounts)

Would it make sense to introduce a new predicate similar to -samefile
that takes into account device numbers only?

With the proposed predicate implemented in `find`, the above command
could be encoded in a more user-friendly way:

$ find / -xdev -maxdepth 1 -samedev /

Originally reported at: https://bugzilla.redhat.com/1607772


Duplicate of 
https://lists.gnu.org/archive/html/bug-findutils/2018-09/msg00050.html


POSIX wants -mount and  -xdev to be slightly different, where -xdev 
stays unchanged from current behavior and -mount excludes the mount 
point itself.


--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [bug #33724] Find command is changing the access time of directory

[Eric Blake]

On 9/29/18 7:01 PM, Paul Eggert wrote:

Bernhard Voelker wrote:

So regarding gnulib, there are 2 alternatives:

a) rename FTS_NOATIME to FTS_NOATIME, and add a retry to open/openat
in each place when that was using O_NOATIME and we got EPERM.


I didn't quite get the point of the rename; those two identifiers look 
identical to me.


With the followup explanation of renaming to FTS_TRY_NOATIME,



The original commit said that FTS_NOATIME was intended to be best 
effort, and that FTS_NOATIME would be ineffective in some cases. If 
that's the intent, retrying would implement it.



b) remove FTS_NOATIME - the attached would do that ... in the naive
hope that no other project has picked it up yet (which I doubt).


If nobody's using FTS_NOATIME it'd probably be better to remove it, yes.

It'd be nice to hear Eric's opinion.


I could go either way. Coreutils has several utilities that do 
traversal, and a 'du --noatime' that makes a best effort might be 
interesting to someone. But, like you said, neither coreutils nor 
findutils has actually incorporated use of FTS_NOATIME, so removing it 
(on the grounds that it doesn't always do what the user wants, even if 
we implement the fallback to retrying the open without O_NOATIME on 
EPERM) is okay by me.



--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[bug #42318] find with xdev traverses other filesystem when mountpiont is subdirectory

[Eric Blake]

Follow-up Comment #12, bug #42318 (project findutils):

The initial POSIX interpretation has come out, and recommends standardizing
-mount to behave like nftw(FTW_MOUNT) (which does NOT include the mount point
- Solaris style), and adding nftw(FTW_XDEV) to match the existing behavior of
-xdev (which DOES include the mount point - GNU style). There is a 30-day
window for public comments on the bug report, but as it currently stands, it
DOES recommend that -mount and -xdev have slightly different semantics.

http://austingroupbugs.net/view.php?id=1133

I also didn't notice this bug when I created
https://savannah.gnu.org/bugs/?54745

___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[bug #54745] -mount needs to be slightly different from -xdev

[Eric Blake]

URL:
  

 Summary: -mount needs to be slightly different from -xdev
 Project: findutils
Submitted by: ericb
Submitted on: Thu 27 Sep 2018 09:31:35 AM MDT
Category: None
Severity: 3 - Normal
  Item Group: None
  Status: None
 Privacy: Public
 Assigned to: None
 Originator Name: 
Originator Email: 
 Open/Closed: Open
 Discussion Lock: Any
 Release: None
   Fixed Release: None

___

Details:

Today during the Austin Group meeting (the folks responsible for POSIX), it
was decided that -mount and -xdev should have a subtle difference in behavior,
in order to reconcile some differences between implementations and against
some other file-traversal programs like pax (the POSIX replacement to tar).
The difference stems on the question of whether, when requesting that a
cross-device file system entry (whether a mount point, or whether a symlink
that crosses mount points when -L follows symlinks), should be visited or
omitted during the traversal.  Both styles have their merits.  Quoting from
the discussion results:

http://austingroupbugs.net/view.php?id=1133
"
When restricting the search to files on one file system, it can sometimes
be desirable for the crossing points themselves to be acted on and sometimes
for them not to be acted on. (Crossing points are mount points and, if the -L
option is specified, symbolic links to directories on other file systems.) The
-xdev primary acts on them and the -mount primary does not. However, -mount
also does not act on symbolic links to non-directory files on other file
systems (if -L is specified). If there is a need for an application to exclude
crossing points but include symbolic links to non-directory files on other
file systems, this can be achieved by using two find commands as follows:

find -L dir -mount -type d -print
find -L dir -xdev ! -type d -print

(in a subshell whose output is piped to sort, if the order matters).

If both -mount and -xdev are specified, find obeys both primaries but the
end result is the same as if -xdev were not specified.
"

We'll need to patch GNU findutils to match, in order to make -mount skip
crossing points (while keeping -xdev with its behavior of visiting crossing
points).





___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

Re: posix_spawn_file_actions_addchdir

[Eric Blake]

On 09/07/2018 04:53 PM, Bruno Haible wrote:


Implemented as follows. The prototype is as in [3], except that I dropped
the suffix '_np', because
   - Functions with suffix '_np' are in the system's namespace, that is,
 there is no requirement that different systems implement them with the
 same semantics or same prototype.
   - Gnulib has fewer backward compatibility requirements, in case we need
 to change the prototype in order to conform to some future POSIX standard.


Did you also read my POSIX request (which also drops _np)? 
http://austingroupbugs.net/view.php?id=1208


There, I argue that we also need posix_spawn_file_actions_addfchdir(), 
in part because it lets us get the same power as openat() in determining 
how relative file names are resolved in relation to an fd.


--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: Introduce posix_spawn

[Eric Blake]

On 09/07/2018 09:44 AM, Eric Blake wrote:

Adding gnulib:

On 09/06/2018 11:37 PM, Barath Aron wrote:

On 9/7/18 4:54 AM, Dale R. Worley wrote:

Couldn't you change the w.d. to the correct target, spawn, and then
change the w.d. back?

Yes, one could do this.


Although it gets prohibitively expensive in a multi-threaded process to 
ensure proper locking between all threads that might want to use 
posix_spawn, since fchdir() affects global process state.


Actually, this particular problem can be easily solved on Threos, 
because the posix_spawn() API offers a function for this:
int posix_spawn_file_actions_addchdir_np(posix_spawn_file_actions_t* 
__restrict file_actions, const char* __restrict path);


Note the "np" at the end, it means "non portable". I also found 
references on Oracle's webpage [1].


It's useful enough that we may want to add support for this in gnulib, 
whether or not other systems pick up on it.  (gnulib is already able to 
overcome a lot of shortcomings in various platforms' posix_spawn)


In fact, it's so useful that I just proposed that POSIX standardize it, 
as well as fchdir:


http://austingroupbugs.net/view.php?id=1208

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: Introduce posix_spawn

[Eric Blake]

Adding gnulib:

On 09/06/2018 11:37 PM, Barath Aron wrote:

On 9/7/18 4:54 AM, Dale R. Worley wrote:

Couldn't you change the w.d. to the correct target, spawn, and then
change the w.d. back?

Yes, one could do this.

Alternatively, one could direct the spawned process to run a program
that would set the working directory to, say, the first argument, and
then use the remaining arguments to run/fork/spawn the target program.
Do you know systems that have this program? What is the name of this 
utility program?

Indeed, I would expect Threos to have a system program to do exactly
this, since other people must have seen the same problem.


Actually, this particular problem can be easily solved on Threos, 
because the posix_spawn() API offers a function for this:
int posix_spawn_file_actions_addchdir_np(posix_spawn_file_actions_t* 
__restrict file_actions, const char* __restrict path);


Note the "np" at the end, it means "non portable". I also found 
references on Oracle's webpage [1].


It's useful enough that we may want to add support for this in gnulib, 
whether or not other systems pick up on it.  (gnulib is already able to 
overcome a lot of shortcomings in various platforms' posix_spawn)




Aron


[1] 
https://docs.oracle.com/cd/E86824_01/html/E54766/posix-spawn-file-actions-addchdir-np-3c.html 







--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [PATCH] Significant performance improvements to locate, struct order optimizations

[Eric Blake]

On 05/16/2018 02:35 AM, Bernhard Voelker wrote:

1. By default, I replaced the strstr implementation with the
   fast_strstr implementation by Raphael Javaux (BSD 3-Clause,
   https://github.com/RaphaelJ/fast_strstr).


Which states:

"Its worst case complexity (O(n × m) where n is the length of the string 
and m the length of the searched sub-string) is the same as the naive 
brute-force algorithm but it mostly runs with a linear complexity (O(n)) 
on most strings."


However, gnulib's strstr() implementation has a worst-case complexity of 
O(n+m) (linear, not quadratic).  Thus, fast_strstr() is NOT always 
faster, given pathological input.


There may be ways to speed up gnulib's strstr().  gnulib was the first 
widely-available linear implementation of strstr() that I'm aware of (at 
the time I contributed it, I could not find any other open source 
implementation that was linear) - but that does not mean it is the 
fastest.  At one point, gnulib's version was imported to glibc (as 
linear is better than quadratic for large string searches, although the 
startup penalties are noticeable on small strings); later, glibc added 
heuristics to further speed up their implementation to dynamically 
switch between algorithms to avoid the penalties on small strings while 
still remaining linear on large strings.  I'm not sure if all of the 
subsequent glibc improvements have been ported back to gnulib.  gnulib 
already rejects libc strstr() that are quadratic (which I suspect is 
still the case with MacOS) - but that becomes a topic for gnulib to 
decide if we should also be replacing libc strstr() when we can prove 
that our implementation is faster, without forcing the replacement (and 
risking speed penalties) on other systems.




First of all, performance improvements should *never* be done in
one patch,


Make that: s/performance/multiple performance/

But yes, a good rule of thumb is that you should always have each patch 
do one thing only, rather than cramming multiple actions into a single 
patch.  You want to make the reviewer's life easier, even if a series of 
well-divided patches adds up to more cumulative changes (due to churn on 
some lines) than a single patch that combines all the changes at once.


--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: strange behaviour of find -iname

[Eric Blake]

On 03/27/2018 05:28 AM, r...@gommes.eu wrote:

Hello!

I am not sure this is a bug, but I fail to understand why find -iname
works differently according to the directory where I want to find
something...


Not a bug in find, but in your usage.


root@ZB17 /home # find -iname *astrill*


Insufficiently quoted.  Try
 echo find -iname *astrill*
to see what you are really executing.  If your current directory has 
nothing that satisfies the glob, then find sees a single argument, and 
does what you want.



root@ZB17 / # find -iname *astrill*
find: paths must precede expression: 20180327_astrill_before_update


But if you execute it in a directory where the glob expands to more than 
one file, then you really executing something like:

 find -iname ':astrill:' '_astrill_'
(I'm guessing here as to what the glob actually expanded to, but you can 
use echo to see for sure).


What you WANTED to type was:
 find -name '*astrill*'
which forces the shell to treat the argument as a literal rather than a 
glob, so that find can then match the glob.



--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[bug #52890] `find --name` ignores files with non-printable character in the filename

[Eric Blake]

Follow-up Comment #3, bug #52890 (project findutils):

You've discovered a fundamental flaw in the specification of the glob()
function - it is only required to have well-defined behavior on byte strings
that are valid encodings in the current locale.  There is no portable way to
make glob patterns match file names that contain encoding errors in a
multibyte locale.  But what you CAN do is specify 'LC_ALL=C find' for any
situation where you want to ensure that globs cannot fail due to encoding
errors, because the C locale is guaranteed by POSIX to be a single-byte
8-bit-clean locale (thus, no encoding errors are possible in that locale, and
glob() no longer has to worry about failure to match a byte string that is not
a valid character string).


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: Bizzare bug in find, potential security implications

[Eric Blake]

On 12/19/2017 03:31 PM, Bernhard Voelker wrote:



The test case in your attachment is a bit different, but also shows
the problem.  It seems that gnulib's regex does not find a match for
the pattern '.*\.exe$' for the files in the following directory:

   $ LC_ALL=C /usr/bin/ls -log htdocs
   ...
   drwxr-xr-x 2 4096 Dec 18 20:45 'Zielona G'$'\363''ra'
   ...

I'm not an expert on UTF and regex, but it seems that the $'\363'
character is not matched by the dot '.' meta character in your
locale.


POSIX says that regex only has to match characters (in particular, the 
glob '.' matches characters, not encoding errors).  If you pick a locale 
with multibyte characters that are subject to encoding errors when 
processing random bytes (as is the case when using a UTF-8 locale to 
process single-byte ISO filenames), then POSIX says regex behavior is 
undefined.  So while it is indeed annoying that find can't match files 
with encoding errors, it is somewhat expected behavior, because there's 
no sane way to make regex well-specified on encoding errors.


--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: selinux

[Eric Blake]

On 11/27/2017 12:18 PM, Gennadiy Krivdyuk wrote:

bug-findutils@gnu.org
 


Testsuite summary for GNU findutils 4.6.0
 


# TOTAL: 225
# PASS:  196
# SKIP:  28
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
 


See tests/test-suite.log


Thanks for the report.  However, without attaching test-suite.log, we 
have no details on what failed or why, so we can't do much 
troubleshooting of the failure.


--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: Need to know technical support for GnuWin32: FindUtils 4.2.20-2 versions

[Eric Blake]

On 08/30/2017 10:30 AM, Bernhard Voelker wrote:
> On 08/30/2017 11:19 AM, Ravinder wrote:
>> On Thu, Aug 24, 2017 at 4:58 PM, Ravinder <yugandharbho...@gmail.com> wrote:
>>> We are using GnuWin32: FindUtils 4.2.20-2 versions in our project, so
>>> requesting you to know whether the technical support is still giving for
>>> these versions, so that we will plan our project to use the latest version
>>> of GnuWin32: FindUtils.
> 
> You reached the upstream GNU findutils project.  We do not do maintenance on
> older versions here; we just develop/maintain the latest version in source 
> format.
> Maintenance is subject to downstream projects which build and provide the
> binaries/packages of the GNU findutils.  In your case, the software you
> are using is a downstream Windows port which is maintained here [0].
> 
> Actually, it is the same situation as for GNU coreutils you sent the same
> request for several times [1] ...
> 
> [0] http://gnuwin32.sourceforge.net/
> [1] http://debbugs.gnu.org/28224

If you want pre-compiled GNU software that runs on Windows, you may be
better off investigating the Cygwin project (https://www.cygwin.com),
which at least has more up-to-date ports than GnuWin32.  But again,
support questions about using Cygwin would be properly directed to the
Cygwin list, not here.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: Several changes made to fts.c in Gnulib

[Eric Blake]

On 07/27/2017 06:23 AM, James Youngman wrote:
> Looking at the patch, it looks not terribly burdensome to maintain
> "downtream" except for the possibility that some fts option could
> subsequently be introduced in gnulib which shares a value with
> FTS_NOLEAF.
> 
> Paul, since you don't plan to apply this patch upstream, do you have
> any suggestions for ensuring that we avoid collision or maintenance
> problems with the downstream patch?

As long as we maintain the patch in diff, then gnulib-tool will tell us
(any time we update git submodule for gnulib) whether we have to
regenerate the diff, at which point we can cater to a new option by
bumping our value for FTS_NOLEAF.  Here's an example of coreutils
storing its project-local diffs to upstream gnulib:
http://git.sv.gnu.org/cgit/coreutils.git/tree/gl/lib/tempname.c.diff

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: findutils 4.6.0 v. Tru64 (strftime() v. "%F"?)

[Eric Blake]

On 07/23/2017 04:22 PM, James Youngman wrote:
> Please try the attached patch, which should fix the problem with %F on
> Tru64 and on HP-UX.

Why not use gnulib's nstrftime module instead, then you can rely not
only on %F, but also on extensions like %s, regardless of libc?

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

[bug #51506] Better support for data processing with basenames

[Eric Blake]

Follow-up Comment #15, bug #51506 (project findutils):

Submit patches, and we'll review them.  Until then, you are wasting more
developer time arguing for the theoretical, than you are saving by optimizing
any programs.

I agree with James - there's no need to add yet more knobs to findutils'
configure. If you submit a patch that makes a difference, we'll enable it by
default; if your patch doesn't make a difference, we'll let you know why it
doesn't make the cut for inclusion upstream. But as this is free software,
you'll still be free to use your patch locally.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug #51506] Better support for data processing with basenames

[Eric Blake]

Follow-up Comment #1, bug #51506 (project findutils):

What resources are you envisioning that '-printf %f\\n' uses that would not be
used by adding a new option, when balanced against the pain of adding,
documenting, and maintaining a new option, for something that is already such
a corner case that you had to lead out with "I image that there is a class of
programs" instead of actually documenting a command line sequence or script
that demonstrates such a need?

You are asking for a premature optimization.  Until you can demonstrate a
concrete use case where find's -printf %f is not fast enough for your needs, I
see no reason to even try to bloat find with something that merely repeats
what -printf %f already supports.

-exec does NOT omit leading directory names, only -execdir (and -okdir) do
that.  But there really isn't any much reason to add more than those actions.

I don't see how this is a valid request for change, so much as an idle musing
of wondering if it is even possible to optimize something that wouldn't get
used in the first place.  The first rule of optimization: present a benchmark
showing that the existing approach has a bottleneck.  You haven't done that
yet.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: findutils 4.6.0 v. Tru64 (strftime() v. "%F"?)

[Eric Blake]

On 05/25/2017 05:43 PM, Steven M. Schweda wrote:
>I've subscribed to the bug-findutils list, so I don't need a separate
> copy of these messages.

It's still list policy to use the Reply-to-all button, so we don't have
to pay attention to who is subscribed.  If you don't want duplicates,
then the filtering should be done by you (not by the sender) - either
you should write your mail with a Followup-to: header that lists all the
addressees but omits yourself, or you should use mailman's option to
suppress messages going to you via the list when you are listed in to or
cc (but be aware that mailman2 has a bug that when you enable the
duplicate suppression, mailman rewrites the list version of the email to
strip you as an addressee, so followups by anyone else that uses
reply-all will not know that you were an original recipient, and you
will still have to play games with threads that are not either
fully-to-you or fully-to-the-list).


>On AIX, "man strftime" doesn't mention "%F", and the build failed:
> 
> gcc -std=gnu99  -g -O2   -o find ftsfind.o ./libfindtools.a
> ../lib/libfind.a ../gl/lib/libgnulib.a -lintl  -lm -lm
> ld: 0711-317 ERROR: Undefined symbol: .pthread_mutex_lock
> ld: 0711-317 ERROR: Undefined symbol: .pthread_mutex_unlock
> ld: 0711-317 ERROR: Undefined symbol: .pthread_mutex_destroy
> ld: 0711-317 ERROR: Undefined symbol: .pthread_mutex_init
> ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more
>  information.
> collect2: ld returned 8 exit status
> 
>Manually (re-)linking with "-lpthread" worked:
> 
> blue# cd find
> blue# gcc -std=gnu99  -g -O2   -o find ftsfind.o ./libfindtools.a
> ../lib/libfind.a ../gl/lib/libgnulib.a -lintl  -lm -lm -lpthread
>

Weird. I don't know if fixing findutils to use gnulib's nstrftime() will
still need to pull in -lpthread on that platform; if so, that will be a
gnulib patch to write. I guess we'll find out, once someone contributes
the fix to use nstrftime in the first place.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: findutils 4.6.0 v. Tru64 (strftime() v. "%F"?)

[Eric Blake]

[adding bug-gnulib]

On 05/25/2017 12:43 AM, Steven M. Schweda wrote:
>Tru64 is not alone.  On an old HP-UX system, it's different but still
> sub-ideal:
> 
> dy# findutils-4.6.0/find/find . -name fred -printf '%C+\n'
> May+16:18:11.00
> 
>There, "man strftime" says:
> 
> [...]
>   The following directives are provided for backward compatibility with
>   the directives supported by date(1) and the ctime(3C) functions.
>   These directives may be removed in a future release.  It is
>   recommended that the directives above be used in preference to those
>   below.
> [...]
>%FLocale's full month name (use %B instead).
> [...]
> 
>So, again, trusting the non-GNU strftime() seems unwise.

Gnulib just recently had a cleanup where we realized that the 'strftime'
module was misnamed:
https://lists.gnu.org/archive/html/bug-gnulib/2017-04/msg00167.html
https://lists.gnu.org/archive/html/bug-gnulib/2017-05/msg00022.html

So yes, either findutils should be using nstrftime() and not strftime()
(which will guarantee that these sequences work), or it is indeed time
to patch gnulib to provide a replacement strftime() on platforms that
are not POSIX-compliant (and then still patch findutils to use the newer
gnulib).

It's actually probably easier it findutils just starts using
nstrftime(), regardless of what else gnulib does, but it's at least
pointing out that gnulib should be documenting the known pitfalls in
native strftime() implementations.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

[bug #51069] find changes access time on directories

[Eric Blake]

Follow-up Comment #2, bug #51069 (project findutils):

Linux has an extension open(O_NOATIME) that allows one to pick-and-choose when
to NOT adjust the atime of a directory, without being as heavy-handed as a
mount option for all operations on the entire tree.  As an extension to POSIX,
we could offer an option to find to turn on noatime mode.  But it would first
need support in gnulib's fts code.  Several other GNU utilities (like
coreutils' du and friends) could also benefit from exposing such an option, on
operating systems where it is feasible.


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug #50835] -print0 causes all files to be output

[Eric Blake]

Update of bug #50835 (project findutils):

  Item Group:Wrong result => None   
  Status:None => Duplicate  
 Open/Closed:Open => Closed 

___

Follow-up Comment #1:

Your complaint is the same as in bug 50780. Quoting from my response there,
with a tweak of -printf changed to -print0:

When two expressions are written as 'A B', find behaves if you had instead
written 'A -a B'. The -a operator is short-circuiting: if expression A returns
false, expression B is not attempted; but if expression A returns true, then
expression B is used.

When you write -print0 first, it always executes (everything gets printed),
then you filter out the files, but do nothing further with the files you
filtered.

When you write -print0 second, it only executes on the files that got past the
-type filter. 


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug #50780] -type option after printf is ignored

[Eric Blake]

Update of bug #50780 (project findutils):

  Status:None => Invalid
 Open/Closed:Open => Closed 

___

Follow-up Comment #1:

And that behavior of -printf is expected.  It is comparable to the POSIX
behavior of -print.

The '-printf' expression (like POSIX -print) will print the file name being
visited, then return true.

The '-type f' expression will check whether the file being visited as a
regular file, and return true or false accordingly.

When two expressions are written as 'A B', find behaves if you had instead
written 'A -a B'.  The -a operator is short-circuiting: if expression A
returns false, expression B is not attempted; but if expression A returns
true, then expression B is used.

When you write -printf first, it always executes (everything gets printed),
then you filter out the files, but do nothing further with the files you
filtered.

When you write -printf second, it only executes on the files that got past the
-type filter.


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug #50606] Garbage printed with "no such file or directory"

[Eric Blake]

Follow-up Comment #1, bug #50606 (project findutils):

Most likely, that is not garbage, but a mismatch in your terminal and your
locale.  find is trying to use UTF-8 quote characters around the file name
that is missing, but your terminal doesn't understand how to display those
characters.  If 'LC_ALL=C find /nonexistent -print' doesn't have the same
problems, then the bug is that you are using a locale that pretends to have
UTF-8 support when your setup does not really honor that locale due to your
inferior terminal, and not a bug in find itself.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: RFE: allowing "" as a path specification for 'current dir' w/o prepending './' ?

[Eric Blake]

On 02/28/2017 11:28 AM, James Youngman wrote:

>> (in the sense that the find command line is a little language with
>>> specified semantics and the user is communicating their wishes by using
>>> that language).  POSIX specifies no semantics for the case where no path
>>> name argument is specified, so the current behaviour is compatible with
>>> POSIX.
>>>
>> 
>>POSIX is irrelevant to what users want -- it's what computers want.
>> Forcing users to adjust to what computers want is the opposite of
>> "user friendly".
>>
> 
> POSIX is a way of ensuring that someone's expectations for how programs
> behave (given some input, shell, C, or whatever) are reasonably consistent
> between systems claiming compliance.
> 
> The relevance, here, of POSIX is that find should behave consistently with
> POSIX in order to avoid unwelcome surprises for users and changes from
> previously-compliant behaviours to no-longer-compliant behaviours.   Given
> that the use of "find" without an argument is not specified by POSIX,
> implementations are free to specify what (their version of) find does in
> this case.   GNU has some long-established behaviour for this usage.  I
> don't think the usefulness of that behaviour is at issue here, though, and
> my only point was that GNU find's behaviour for "find" by itself is not
> forbidden by POSIX.

For the record, POSIX leaves 'find' up to the implementation (some
implementations give an error, GNU treats it as 'find .'), but requires
'find ""' to report an error because there is no such file or directory
(the empty path name has to fail with ENOENT).

So while I'm okay with something like:

find --trim-leading-dot

printing 'foo' instead of './foo', (and where you can wrap a function or
alias around it to always turn --trim-leading-dot on for your own
interactive dot), I am NOT okay with:

find ''

magically starting to produce successful output where POSIX requires it
to fail.

> But I've not said that you shouldn't remove the "./"  or that you can't
> have the feature.  You certainly can, you can change the code of GNU find
> any way you like.  I will even try to help you do it, time allowing.   I
> just won't be including this feature in the released version.

And I'm merely adding that such a change should be done via a
command-line option, not an environment variable.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: RFE: allowing "" as a path specification for 'current dir' w/o prepending './' ?

[Eric Blake]

On 03/02/2017 04:24 PM, L A Walsh wrote:

>> Please, do NOT make the behavior depend on an environment variable.

>Please do not make blanket statements: A foolish consistency is
> the Hobgoblin of small minds.", and please don't come into the
> middle of a discussion without reading the previous conversation:

Please don't assume that I haven't been following the conversation.

> 
>> >   ... I find the addition of "./" -- for interactive
>> >  use, to be *Visual Clutter*.  I was hoping for a simple 'short-cut'
>> >  for interactive use to get rid of the visual clutter ...

And I find the exact opposite - lack of ./ is visual clutter, because
with a leading ./, I am guaranteed that it resolves to a filename, but
without a leading ./, a file name that starts with '-' (for example) is
indistinguishable from what would normally be a command-line option.  In
other words, I think that always outputting ./ by default is a GOOD
thing, and that if you want to strip it, you should put in extra effort
(whether that be post-processing, or a new option that strips it by
default, is irrelevant, as long as it is something that you can request
to change what is otherwise the default).  And since I've now stated
clearly that I'm okay with a way to change the default, I'm also
reiterating that such a change SHOULD be done via a command-line option,
and NOT via an environment variable.

> 
>This is not about a POSIX specified feature (it's undefined), and
> is not about something that would normally be used with scripts, so
> your argument doesn't seem applicable.

On the contrary, my argument IS applicable - the mere fact that it is
NOT specified by POSIX what 'find' should do is all the more reason that
the default behavior should be as unambiguous as possible, and therefore
should include leading './' the same way that 'find .' does (it also
helps that we've consistently documented that 'find' behaves the same as
'find .', and I don't want to break that consistency - and it is for the
same reason that 'ls' and 'ls .' have the same output that 'find' and
'find .' should have the same output).  If a mere environment variable
is enough to change things, then that does risk breaking scripts, even
if such scripts are already relying non-POSIX extensions.  So GNU
programs in general should NOT use new environment variables to change
default behavior.

> 
>This topic is about a default, interactive behavior.  Not only
> does that not apply to scripts, but, by definition, precludes use
> of command-line options.

Ah, but that's where you're missing the point.  Changing interactive
behavior does NOT require that you are forced to always type the new
command-line option, because you are free to set up a wrapper function,
alias, or script on $PATH, which will supply that option on your behalf
for all your interactive use, without penalizing the default for
everyone else.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: RFE: allowing "" as a path specification for 'current dir' w/o prepending './' ?

[Eric Blake]

On 03/02/2017 01:25 AM, L A Walsh wrote:

> However given that some like the current behavior, a way to
> allow users a choice of of no-path being the same as "." or
> not-prepending "dot" would be the addition of an ENV var
> along the lines of "FIND_NOPATH_PREPENDS_NODOT" (or whatever!).

Please, do NOT make the behavior depend on an environment variable.
That is a sure way to break scripts that are not expecting the
environment to cause a change in behavior.  If you want a different
behavior, the right way to do it is via a new command-line option.


-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: Potential linux find utility bug / misdocumentation

[Eric Blake]

On 02/06/2017 08:31 AM, David Jamroga wrote:

> [fedora21@localhost ~]$ find . -iname Bon*
> 
> ./BonAirPainting
> 
> [fedora21@localhost ~]$ find . -iname bon*
> 
> ./MiscNetflix/bonbon.txt
> 
> ./BonAirPainting

Not a bug in find, but in your usage.  You forgot to use proper shell
quoting.  So you ended up executing:

find . -iname BonAirPainting

vs.

find . -iname 'bon*'

when you REALLy wanted to execute:

find . -iname 'Bon*'

in the first instance.  To prove this, rewrite your command lines to:

echo find . -iname Bon*
echo find . -iname bon*

and also play with what happens when you do:

touch bon1 bon2

and watch the behavior change yet again into a syntax error when you use
insufficient quoting.  Remember, find is one of the (few) programs where
you WANT to let 'find' do glob expansion, rather than the shell, making
it important to always quote the arguments so that the shell doesn't
prematurely expand things.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: Explaining +/-n

[Eric Blake]

On 11/22/2016 06:57 PM, Dale R. Worley wrote:
> I started working out the manual changes to address
> http://savannah.gnu.org/bugs/?49640
> 
> Because my setup is old, I figured I'd start trying out the new wording
> using commit 1a009ccb from 2015-12-20.  But the structure of that commit
> is peculiar:  the find.1 manual page is installed from
> findutils/find/find.1, but that file is *not* generated from
> findutils/doc/find.texi like you'd expect, it's a source file in the Git
> archive.  (Though that may be different from the structure of the latest
> commit.)
> 
> Could someone fill me in on what is going on here?  I'm reluctant to
> believe that find.1 and find.texi are maintained separately, but that's
> what it looks like.

Yes, findutils is one of the few pages that actually maintains a
separate man page at the moment.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Findutils-patches] [PATCH] find: handle more readdir(3) errors

[Eric Blake]

On 11/01/2016 04:32 PM, Bernhard Voelker wrote:

> 
>> However,
> 
>>> +++ b/lib/fdleak.c
> 
>>> -  while ((dent=readdir (dir)) != NULL) -   { +  while (1) +
>>> { +   errno = 0;
> 
> __^^

Uggh. Mail clients botched the formatting.

> 
>>> + dent = readdir (dir);

> 
> 
> errno _is_ initialized right before each readdir() call, see above,
> so I don't see the problem here.

Whoops - blame a late-day review on my part.

You're right - the first file did readdir() inside the for loop; the
second does readdir() inside a while(1) loop.  Okay, I stand corrected,
the second file is indeed fine (because it DOES reset errno each time);
but you are right that adding a comment to the first file will help.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Findutils-patches] [PATCH] find: handle more readdir(3) errors

[Eric Blake]

On 11/01/2016 02:37 PM, Bernhard Voelker wrote:
> Similar to the FTS readdir fix in v4.6.0-72-g155c9d1, handle the last
> two unhandled readdir(3) errors.
> 
> * find/pred.c (pred_empty): Do the above.
> * lib/fdleak.c (get_proc_max_fd): Likewise.  While at it, fix the
> condition to only skip "." and ".."; previously, also other files
> beginning with ".." would have been skipped - that was theoretically,
> as we only expect the FD files in "/proc/self/fd".
> ---
>  find/pred.c  |  8 
>  lib/fdleak.c | 21 +
>  2 files changed, 25 insertions(+), 4 deletions(-)
> 
> diff --git a/find/pred.c b/find/pred.c
> index f7e9b59..93f82b6 100644
> --- a/find/pred.c
> +++ b/find/pred.c

[expanding context]

> @@ -380,6 +380,7 @@ pred_empty (const char *pathname, struct stat *stat_buf, 
> struct predicate *pred_
> state.exit_status = 1;
> return false;
>   }
> +  errno = 0;
>for (dp = readdir (d); dp; dp = readdir (d))

Generally wrong; you have to clear errno before EVERY call to readdir(),
not just the first.  Either with 'for(..;..; errno = 0, dp = readdir
(d))', or by...

>   {
> if (dp->d_name[0] != '.'
  || (dp->d_name[1] != '\0'
  && (dp->d_name[1] != '.' || dp->d_name[2] != '\0')))
{
  empty = false;
> break;
>   }
>   }

...clearing errno at the end of the loop.  But for _this particular
loop_, the body does not touch errno, which means later iterations are
left with it still 0 from the earlier iterations,

> +  if (errno)
> + {
> +   /* Handle errors from readdir(3). */
> +   error (0, errno, "%s", safely_quote_err_filename (0, pathname));
> +   state.exit_status = 1;
> +   return false;
> + }

so I can agree to this part.  However,

> +++ b/lib/fdleak.c

> -  while ((dent=readdir (dir)) != NULL)
> - {
> +  while (1)
> +{
> +   errno = 0;
> +   dent = readdir (dir);
> +   if (NULL == dent)
> + {
> +   if (errno)
> + {
> +   error (0, errno, "%s", quotearg_n_style (0, 
> locale_quoting_style, path));
> +   good = 0;
> + }
> +   break;
> + }
> +
> if (dent->d_name[0] != '.'
> -   || (dent->d_name[0] != 0
> -   && dent->d_name[1] != 0 && dent->d_name[1] != '.'))
> +   || (dent->d_name[1] != 0
> +   && (dent->d_name[1] != '.' || dent->d_name[2] != 0)))
>       {
> const int fd = safe_atoi (dent->d_name, literal_quoting_style);

THIS loop is a demonstration of how not to use readdir; safe_atoi() can
clobber errno, so you are no longer guaranteed that each loop iteration
is starting with errno == 0.  You'll have to tweak the patch.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: putc(-1) == EOF on IRIX in frcode

[Eric Blake]

[adding bug-gnulib]

On 06/12/2016 03:47 PM, Rainer M. Canavan wrote:
> I've noticed that updatedb has consistently failed since upgrading
> to findutils 4.6.0. I'm using IRIX 6.5.30 and the mipspro compilers.

Thanks for the report.

> 
> It turns out that frcode assumes that a write error has occurred, 
> although in reality everything went well, because 
> putc(-1, stdout) returns -1 on success _and_ failure, since 
> EOF == -1, and therefore in frcode.c, line 335:
> 
> if (EOF == putc (diffcount, stdout))
> 
> exits with "write error". I'm not sure why this isn't always triggered 
> if (diffcount == -1), but I haven't checked if the same codepath is 
> always taken. 

That's a bug in IRIX putc(); POSIX is clear that putc() is supposed to
cast its argument to unsigned char before writing to the stream, and
then return the character just written.

Gnulib should be taught to work around the bug.

> 
> A possible fix would be to cast diffcount to unsigned char, this
> appears to work (and results in a valid locatedb) on both 
> linux and irix:
> 
> putc ((unsigned char)diffcount, stdout))

This workaround should be embedded in gnulib, so that findutils can
continue to use putc() without the cast.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [6 PATCHES] New feature added to find: -type xyz

[Eric Blake]

On 03/02/2016 01:44 PM, Dale R. Worley wrote:
> Eric Blake <ebl...@redhat.com> writes:
>>> The instructions to build is in the README-hacking file. Perhaps we
>>> should add INSTALL file. 
>>
>> INSTALL is generated as part of bootstrapping the package.
>> README-hacking tells how to bootstrap the package.  Building from git is
>> different than building from a tarball (which has already been
>> bootstrapped).
> 
> It would be useful if the Git repository had a simple INSTALL file
> saying something like:
> 
> This finutils source needs to be "bootstrapped" before the usual build
> and install procedures.  See README-hacking for instructions.
> 
> That would tell new users what to do.  The bootstrapping procedure would
> overwrite the initial INSTALL file with the normal INSTALL information.

It's amazingly hard to override INSTALL when using automake to generate
it automatically during bootstrap.  Submit a patch, and we'll consider
it, but I won't be the one writing such a patch.

> 
> BTW, in README-hacking, item 3 says:
> 
> 3. Run "configure" and "make" in the normal way.
> 
>  If you have GNU libintl installed, you can just run "configure".
>   ^^
>  Otherwise, run "configure --disable-nls".
>  ^^
> 
> At the two points I've marked, that should say "./configure" -- there's
> no guarantee that "." is in the user's path.

Indeed, and that's a much simpler thing to patch.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: __attribute__ and timeval problems building findutils on IRIX

[Eric Blake]

adding gnulib

On 03/01/2016 04:08 PM, Rainer M. Canavan wrote:

Thanks for the report.  I'm focusing on just the gnulib bug:

> A few tests cannot be compiled because struct timeval isn't 
> know. Shifting around the include statements or adding
> time.h fixes these.
> 
> 
> rainer
>  
> diff -ur ./tests/test-gettimeofday.c 
> ../../findutils-4.6.0//tests/test-gettimeofday.c
> --- ./tests/test-gettimeofday.c   Tue Dec 22 18:32:53 CET 2015
> +++ ../../findutils-4.6.0//tests/test-gettimeofday.c  Tue Mar 01 22:56:07 CET 
> 2016
> @@ -17,6 +17,7 @@
>  
>  #include 
>  
> +#include 
>  #include 

Rather, we should be fixing gnulib's replacement  header to
be self-contained.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[bug #47261] find produces two different results for the same command

[Eric Blake]

Follow-up Comment #5, bug #47261 (project findutils):

Could it be a case of automounting kicking in? If your file is hidden behind a
mount point, maybe the first run triggers the mount to automount but doesn't
descend, and the second one is able to descend into the (now-mounted)
location?


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: find -f -, please?

[Eric Blake]

On 02/15/2016 02:44 PM, Marc Chantreux wrote:
> 
> because the line exceed the max size of a command so i wrote
> 
> a-very-long-list-of-files |
> xargs -IX find X -prune -type f -name '*scm'
> 
> which works fine regardless the fact the find command was started for
> all single file (-I implies -L1).

It would also be possible to do this with fewer processes spawned,
insofar as none of the output of a-very-long-list-of-files can be
ambiguous with a find expression, by doing:

a-very-long-list-of-files |
  xargs sh -c 'find "$@" -prune -type f -name "*scm"' sh

> 
> I finally wrote
> 
> a-very-long-list-of-files | perl -lnE 'print if -f && /[.]scm$/'
> 
> this isn't a problem on a BSD system as find has the wonderfull -f.
> According to http://netbsd.gw.com/cgi-bin/man-cgi?find+1+NetBSD-current
> 
> -f  Specifies a file hierarchy for find to traverse.
> File hierarchies may also be specified as the
> operands immediately following the options.

Could you show the actual command line you envision writing with -f in
use?  I think you have confused two different feature requests, but
seeing your intended usage would make that easier to tell.

As far as I can tell, the BSD -f is mostly unnecessary fluff, but easy
to add.  That is, for an unambiguous name:

find -f foo expression

is identical to:

find foo expression

Meanwhile, if you want to start your search on a name that is ambiguous
with an expression element, you can use:

find -f -name expression

which can always be safely written as:

find ./-name expression

except that any output is now prefixed by ./ where with -f the extra ./
is not needed.  That is, BSD's -f is a way to say "treat the next
argument as a start point no matter what, even if without the -f it
would be ambiguous with the start of an expression".

> 
> It would be nice to get this feature into the gnu tools. I hope you like
> the idea.

The biggest argument in favor of adding the BSD -f feature is the fact
that it already exists in other implementations, thus adding it would
let GNU find still be a drop-in replacement for systems that were
relying on the non-free extension.  So yes, I'm interested in that idea.

> 
> I'm not a C expert but i get a naive overview of the source but my start
> should start with the patch at the end of the body then
> 
> * add the -f support in the code that read the flags
> * write a function process_traverse_path inspired by process_all_startpoints
>   that could be (pseudocode)
> 
>   foreach name in fflags
> handle = name eq '-' ? stdin : fopen(name)
> while getline handle, l
> find(l)

Ouch. You're proposing a completely different behavior than what the BSD
-f performs.  That is NOT a good idea.  If we add -f, it should have
identical semantics to BSD.  If we want new semantics, it should not be
named -f.

Your proposal is the ability to teach find to read a list of filenames
from an input file, as if those files had been independently specified
on the command line.  Typical GNU spelling for such a feature already
exists: see, for example, 'du --files0-from=F'.  So we could easily use
the same code (assuming coreutils has shared du's --files0-from in
gnulib), as a similar starting point for findutils.  Whether we should
burn a short option letter is a different question (the long option
alone is all the more coreutils provided, and I already stated that we
will NOT burn '-f' for a meaning different from BSD's).

Your idea also has merit (let me supply a list of names as if they had
been separate command line arguments), particularly since I could
probably come up with some pathological case where a single find
invocation on ALL of those start points behaves differently than
concatenating the results of multiple find invocations on a subset of
the start points (not even that hard to envision such a scenario - it
seems like any expression including an '-exec {} +' would be an easy
place to start).  So I'm interested in that too.

> 
> so if it is correct and there is no maintainer to do the job, i'll be
> happy to try to submit a real live patch. 

Your patch will probably be non-trivial (it would need documentation and
testsuite additions, alongside the actual code), but it does sound
interesting.  If you are interested in pursuing copyright assignment to
the FSF, then we'd love to have you submit a patch.

At any rate, even if you don't supply a patch, you have proposed not
one, but two potentially useful enhancements.  It may be worth recording
them in the Savannah bug tracker so we don't forget about this mail.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: Hello, maybe not the right mail address...

[Eric Blake]

On 01/21/2016 07:20 PM, Aurélien Gendron wrote:
> ... but after a fresh install of ubuntu 16.04 :I've done apt-get update, 
> apt-get dist-upgrade and apt-get autoremove, I get this error :
> Suppression de python3.4 (3.4.4-1) ...
> xargs : l'option -s contient un nombre non valide
> Suppression de libpython3.4-stdlib:amd64 (3.4.4-1) ...

Sounds like https://bugs.debian.org/809079, which is not findutils'
fault, and for which the fix is already known.

Also, checking the archives would show that you're not the first to
report it:
https://lists.gnu.org/archive/html/bug-findutils/2016-01/msg00074.html


-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: Comunicar errores a <bug-findutils@gnu.org>.Desinstalando libpython3.4-stdlib:amd64 (3.4.4-1) ...

[Eric Blake]

On 01/19/2016 01:13 PM, Javier Fernandez wrote:
> test@test-MS-9A55:~$ sudo apt autoremove
> Leyendo lista de paquetes... Hecho

Thanks for the transcript, but it is rather vague. You have not
specified what happened differently from what you were expecting, making
it hard to compose any response other than asking for more details.
Also, more readers of this list understand English than Spanish, so it
may help to ask your question in English.

> Desinstalando liblouis2:amd64 (2.6.2-2) ...
> Desinstalando python3.4 (3.4.4-1) ...
> xargs: número inválido para la opción -s
> Usage: xargs [OPTION]... COMMAND [INITIAL-ARGS]...
> Run COMMAND with arguments INITIAL-ARGS and more arguments read from input.

If your question has to do with why the xargs usage message appears in
the middle of the larger 'sudo apt autoremove' output, it is more likely
a bug in 'apt' than in xargs (that is, some bug in the apt script is
invoking xargs with incorrect arguments), in which case you aren't even
asking on the best list.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [PATCH] updatedb: run in the C locale, don't do case-folding.

[Eric Blake]

On 01/09/2016 02:18 PM, James Youngman wrote:
> * locate/updatedb.sh: Set LC_ALL to C to avoid unexpected character
> encodings in path names causing sort to fail (idea from Clarence
> Risher).  Don't do case-folding, since the character set in now C,
> which is likely inconsistent with the user's expectations anyway.
> Honour $TMPDIR. Correct the error message you get if you specify
> both --old-format and --dbformat.
> * NEWS: Explain these changes.
> ---
>  NEWS   |  7 +++
>  locate/updatedb.sh | 33 -
>  2 files changed, 31 insertions(+), 9 deletions(-)
> 

> +++ b/locate/updatedb.sh
> @@ -31,6 +31,19 @@ There is NO WARRANTY, to the extent permitted by law.
>  Written by Eric B. Decker, James Youngman, and Kevin Dalley.
>  '
>  
> +# File path names are not actually text, anyway (since there is no
> +# mechanism to enforce any constraint that the basename of a
> +# subdirectory has the same character encoding as the basename of its
> +# parent).  The practical effect is that, depending on the way a
> +# oarticular system is configured and the content of its filesystem,
> +# passing all the file names in the system through "sort" may generate
> +# character encoding errors in text-based tools like "sort".  To avoid
> +# this, we set LC_ALL=C.  This will, presumably, not work perfectly on
> +# systems where LC_ALL is not the way to do locale configuration or
> +# some other seting can override this.
> +LC_ALL=C
> +export LC_ALL
> +
>  
>  usage="\
>  Usage: $0 [--findoptions='-option1 -option2...']
> @@ -75,7 +88,7 @@ done
>  
>  case "${dbformat:+yes}_${old}" in
>  yes_yes)
> - echo "The --dbformat and --old cannot both be specified." >&2
> + echo "The --dbformat and --old-format cannot both be specified." >&2

Do we ever want to allow translation of error messages spat out by
updatedb?  If so, we can't globally set LC_ALL=C, but instead have to do
it piecemeal at a time on any operations we are doing that should not
leak to the user, while honoring the user's locale for operations that
produce text back to the user.  Which is obviously trickier to do.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: feature proposal/request: find -crtime

[Eric Blake]

On 12/03/2015 07:17 AM, Bozhidar Zhelev wrote:
> Hello All,
> 
> Kudos for this awesome piece of software and many thanks for the support
> and improvements you are providing! :)
> 
> I would want to propose what i think would be very useful feature for FIND
> 
> Can we have argument -crtime, that is similar in behaviour to atime, mtime
> & ctime?

We already support -newerBB for comparing birth times of files, where
that is known.  So such an option should be named -Btime for consistency
(not -crtime).

> Since there is no kernel API to get that info right away 

Then you're using the wrong kernel :)  BSD and Cygwin already provide
birthtime information as part of stat().

Yes, it would be nice for Linux to either enhance stat(), or add xstat()
(there has been a proposal in the past, but so far it hasn't made it
into the tree).

But until Linux makes it easy to get at birthtime, even the existing
-newerBB predicate is hard to implement.  If you would like to patch
gnulib to provide birthtime for Linux software, then multiple software
would benefit (GNU coreutils' stat(1), for example).  Until then,
teaching just findutils how to do a Linux-specific hack to get at
birthtime feels a bit counter-productive, when it would be better to fix
the kernel or at least gnulib to provide the information easily for
everyone to benefit.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: find / -iregex not working as expected

[Eric Blake]

On 11/12/2015 08:41 AM, Michael Convey wrote:
> I'm trying to find the following files with a single 'find' statement:
> 
> /var/lib/postgresql/9.4/main/postgresql.auto.conf
> /usr/lib/tmpfiles.d/postgresql.conf
> /etc/postgresql/9.4/main/postgresql.conf
> 
> I tried the following (and more), but I can't figure it out:
> 
> find / -iregex "*postgresql.*.conf"

This says find all files that match the case-insensitive regular
expression "*postgresql.*.conf".  A regex of ".*." is identical to a
regex of ".+" (1 or more arbitrary characters), and NOT the same as a
regular expression for a literal '.'.  A leading "*" in a regular
expression is unspecified behavior, but many regex engines treat it
identically to a regex of "\*" (matching only a literal '*').  As your
desired filenames do not start with a literal star, the regex won't
match them.

> find / -iregex "*/postgresql*.conf"

Still a literal star issue.

> find / -iregex *postgresql*.conf

Here, you lack shell quoting, so your shell tries to glob the '*' before
even passing the argument to find.  Generally not what you want.

> find / -iregex '*postgresql.*.conf'

This is no different than your first attempt (changing between '' and ""
shell quoting doesn't affect the string seen by find).

What you probably want is to use '-iname', not '-regex', for a
case-insensitive glob comparison.  As in:

find / -iname 'postgresql.*conf'

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: a minor bug

[Eric Blake]

On 11/06/2015 10:31 PM, Kyle Sallee wrote:
> By xargs a file descriptor is leaked.
> Within find; the omission might or might not exist.
> A patch is attached.
> 
> 
> findutils-4.4.2.o_cloexec.patch
> 
> 
> diff -r -U2 findutils-4.4.2/xargs/xargs.c findutils-4.4.2/xargs/xargs.c
> --- findutils-4.4.2/xargs/xargs.c -00-00 00:00:00.0 -
> +++ findutils-4.4.2/xargs/xargs.c -00-00 00:00:00.0 -
> @@ -647,5 +647,5 @@
>  {
>keep_stdin = 1;/* see prep_child_for_exec() */
> -  input_stream = fopen (input_file, "r");
> +  input_stream = fopen (input_file, "re");

fopen(,"re") is not (yet) POSIX; we'd have to guarantee that it is first
supported by gnulib before we could rely on it.  But the idea of marking
fds close-on-exec in xargs so that they are not leaked to the child
process makes sense.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [bug #45445] AIX make check test-getdelim.c:72: assertion failed Abort(coredump)

[Eric Blake]

[adding gnulib]

On 08/31/2015 02:50 PM, John F. Trudeau wrote:
> Follow-up Comment #1, bug #45445 (project findutils):
> 
> AIX7.1 libc.a contains getdelim/getline.  I cannot find any AIX documentation
> on theses functions.  Apparently the AIX getdelim does not expand the output
> buffer.  I was able to eliminate the test error by editing configure to
> force:
> 
> ac_cv_func_getdelim=no
> ac_cv_func_getline=no
> 
> Thus, causing the gnulib version of theses function to be used.

That means that gnulib should be testing whether the system
getdelim/getline matches the POSIX and GNU version, or whether it is an
unrelated symbol (as appears to be the case on AIX), so that configure
can get the right result from the beginning instead of you having to
override a cache value.

> Reply to this item at:
> 
>   <http://savannah.gnu.org/bugs/?45445>

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: find --help :: typo

[Eric Blake]

On 07/02/2015 10:18 AM, James Youngman wrote:
 2015-07-01 10:58 GMT+01:00 Conrad Koenitz conrad.koen...@napster.de:
 Hi,

 There is a small typo on the German version of ‚find —help‘. In the last 
 sentence it says:
 
 Thanks for the bug report!
 
 You didn't indicate which version you were using, but the contact
 information for the message translation teams is at
 http://translationproject.org/domain/findutils.html
 
 (I don't maintain the translation text myself)

You may also want to report another bug to the German translators
visible in their --help text.  Per po/de.po:

#. TRANSLATORS: The placeholder indicates the bug-reporting address
#. for this package.  Please add _another line_ saying
#. Report translation bugs to ...\n with the address for translation
#. bugs (typically your translation team's web or email address).
#: gl/lib/version-etc.c:245
#, c-format
msgid 
\n
Report bugs to: %s\n
msgstr 
\n
Melden Sie Fehler an: %s\n

what they SHOULD be doing is providing their translation bug address in
addition to bug-findutils.

Observe the difference to other translations which do it correctly:

$ LC_ALL=de_DE.UTF-8 ls --help | tail -n4
GNU coreutils Onlinehilfe: http://www.gnu.org/software/coreutils/
Melden Sie Übersetzungsfehler für ls an
translation-team...@lists.sourceforge.net
Für die vollständige Dokumentation starten Sie:
info coreutils 'ls invocation'

$ LC_ALL=de_DE.UTF-8 find --help | tail -n3
Fehler (und Informationen über deren Abarbeitungsstand) können
auf der Fehlerberichtseite der findutils auf http://savannah.gnu.org/
eingegeben oder per E-Mail an bug-findutils@gnu.org gesendet werden.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: Filename Expansion: Find Utility Pattern Matching vs Bash Shell Pattern Matching

[Eric Blake]

On 06/16/2015 01:02 PM, Michael Convey wrote:
 For filename expansion, the 'find' utility's '-name' option seems to
 function similarly, but not exactly the same as the bash shell's builtin
 pattern matching.

Correct. bash's builtin pattern matching is implemented as if by using
fnmatch(), and find's -name pattern matching is also implemented as if
by using fnmatch().  However, the two programs likely pass different
flags to fnmatch() (for example, POSIX requires that find -name '?'
include ., while in bash, the glob '?' does not match '.' unless you
have 'shopt -s dotglob' turned on).  There's also the matter that if
configure detects that your libc's fnmatch() is missing or otherwise
broken, then a replacement fnmatch is compiled in, and the replacement
may differ in functionality between bash and find (or even the configure
test deciding if the system fnmatch is broken).

 
From this, it sounds like it is not the shell that is performing the
 pattern matching, but the find utility using the fnmatch library.

If you write:

find -name *.foo

then the shell is using something equivalent to fnmatch to expand *.foo;
and depending on the expansion, you will (usually) end up with a find
syntax error; but (sometimes) end up with a valid command like:

find -name a.foo

If you write:

find -name '*.foo'

then you bypass the shell's use of fnmatch, and find itself sees a
literal pattern to expand using fnmatch.

 
 Here are my questions:
 
1. Is the bash shell's default filename expansion and pattern matching
(extglob shell option disabled) different from that of the find utility
using the -name option?

extglob is NOT part of the requirements of fnmatch, so that's a case
where bash is doing things itself (whether that involves breaking the
pattern down into something native fnmatch can handle, or whether the
complete fnmatch replacement comes from bash, is an implementation
detail of bash).  But if extglob is turned on, that's a case of bash
using different flags to fnmatch than find does (as find does not have a
notion of extglob).

2. If so, what are those differences?

As mentioned above, there are at least differences with extglob (which
is a bash extension) and dotglob (where matching a leading dot behaves
differently as required by POSIX).

3. Does bash also use the fnmatch library or some other mechanism for
filename expansion and pattern matching?

That may be a question better asked on the bash list, or by looking in
the bash source code.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: How to search for symlinks pointing to directories?

[Eric Blake]

On 04/18/2015 10:02 AM, Peng Yu wrote:
 Hi,
 
 I don't see there is a way to search for symlinks pointing to
 directories natively in find.
 
 This can be done by using -exec and test. But I want to avoid using
 -exec. Is there a way to such a search natively in find? Thanks.

find -xtype d \! -type d

does what you want. It finds all files that are directories after
symlink resolution, but which are not directories to begin with.

In an empty directory:
$ touch a
$ mkdir b
$ ln -s a c
$ ln -s b d
$ find -xtype d \! -type d
./d

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: How to search for symlinks pointing to directories?

[Eric Blake]

On 04/18/2015 11:41 AM, Peng Yu wrote:

 This can be used to check if a symbolic link is pointing a file, a
 directory, etc. Is there a way to check if a symbolic link is broken
 without enumerating all the possibilities?

find -L -type l

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[bug #44569] find newermt cannot figure out how to interpret some dates.

[Eric Blake]

Update of bug #44569 (project findutils):

  Status:None = Invalid
 Open/Closed:Open = Closed 

___

Follow-up Comment #1:

This is not a bug in find.

https://www.gnu.org/software/coreutils/faq/coreutils-faq.html#The-date-command-is-not-working-right_002e

The find command shares the same date parser as used in coreutils' date(1) -
and that parser honors daylight savings rules of your current locale.  As you
apparently live in a locale where that specific date in October has no
midnight (jumping straight from 23:59 on the previous day to 1:00 on that
day), then the date you have specified really is bogus.  Try specifying a time
that includes a safer hour, such as noon, that is unlikely to not exist due to
daylight savings jumps, or try using TZ=UTC to force a time zone that doesn't
have daylight savings issues.


___

Reply to this item at:

  http://savannah.gnu.org/bugs/?44569

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug #44570] Wrong find -type f -not -name *txt result for files with weird names

[Eric Blake]

Follow-up Comment #1, bug #44570 (project findutils):

Try again, using: LC_ALL=C find -type f -not -name '*.txt'

My guess is that you are in a non-C multibyte locale, and that the filenames
being printed are ones that are invalid when interpreted in the multibyte
encoding rules of your current locale.  globbing is only required to work on
valid encoded content, so the fact that you have a name that is not a valid
encoding means that the glob cannot match it, and therefore it gets printed
even though it ends in a .txt ending.  Switching to a locale where there are
no encoding errors (such as LC_ALL=C) will avoid the problem.

Oh, and don't use: -name *.txt.  You WANT to use -name '*.txt' (so that the
shell is not expanding the glob prior to calling find, but rather that find is
being given a literal glob to expand itself).

___

Reply to this item at:

  http://savannah.gnu.org/bugs/?44570

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: bug in find?

[Eric Blake]

On 10/24/2014 12:57 PM, Jan Meyer wrote:

 */xxx $ find /media/jm/Windows -name *.rar*find: paths must precede
 expression: logport.rar

Not a bug in find, but in your usage pattern.  Try this in each
directory where you see different behavior:

echo find /media/jm/Windows -name *.rar

to see what command you are really executing.  I'm betting that in the
directory where you had a failure, you had more than one *.rar file,
which resulted in an invalid command line for find; and in the directory
where it worked but could only find one name in subdirectories, it was
because the glob has already been expanded into that one name.  Only in
a directory that had no rar files at all does the shell leave the glob
*.rar untouched for find to then be able and look for all possible rar
files.

Next, try:

echo find /media/jm/Windows -name '*.rar'

to see the command line you WANTED to type, then finally try:

find /media/jm/Windows -name '*.rar'

to see it work.  Remember, you want * globbing to be quoted so that the
shell doesn't expand it early, because find also does globbing.

 Do I get a prize? Or a reprimand? :)

Which do you prefer? :)  You've made a classing quoting blunder, and
didn't find any problem in find itself.

By the way, your mail came through rather botched.  It is very hard to
see what you typed vs. what your terminal was displaying, when you had
line wraps that munged lines together.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: xargs - how to get it to execute input args as the command?

[Eric Blake]

On 01/05/2014 04:09 AM, Linda A. Walsh wrote:

 (tried exec, but I guess that is a shell builtin...sigh)...

How about 'env' instead of 'exec'?

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[bug #35753] Completion of error handling

[Eric Blake]

Follow-up Comment #3, bug #35753 (project findutils):

Patch 4 is pointless.  We use atexit(close_stdin) for a reason: all reads from
stdin and all writes to stdout/stderr will cause the ferror() flag for the
standard descriptor to be set if there is a failure, and close_stdin will then
guarantee non-zero exit status in that case.  The point of the atexit
installation is to avoid the churn of having to check every single fprintf or
other operation on stdout, and instead let the atexit hook take care of it for
us.

___

Reply to this item at:

  http://savannah.gnu.org/bugs/?35753

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: Regression: find dir/. -type d -empty -delete claims 'unsuccessful', breaking scripts.

[Eric Blake]

On 11/18/2013 04:55 PM, Linda A. Walsh wrote:
 In coreutils 8.21-7.1.3.
 
 It has been standard to use . in a directory to mean it's contents
 on a recursive or copy  (compare cp -al src/. dst/.).
 
 However, find dir/. -type d -empty -delete  works, but
 exits with a failure code causing scripts to break.

This behavior is required by POSIX.  Sorry.

 POSIX is dead

Ranting against POSIX won't buy you any friends.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: Regression: find dir/. -type d -empty -delete claims 'unsuccessful', breaking scripts.

[Eric Blake]

On 11/18/2013 06:20 PM, Linda Walsh wrote:

 However, find dir/. -type d -empty -delete  works, but
 exits with a failure code causing scripts to break.
 

 This behavior is required by POSIX.  Sorry.

POSIX requires that whatever you pass on the command line (dir/.) be
passed as the prefix to all paths traversed by find; therefore, find
must traverse dir/..  POSIX also requires that remove(dir/.) must fail.

 It sure looks like POSIX does NOT require it.  -delete is a gnu
 extension,
 so it wouldn't be covered either.

Right, but it makes no sense to handle -delete as though it were
visiting dir instead of dir/., nor to be inconsistent with what you
would get with the POSIX behavior of find dir/. | xargs rmdir (at
least, when there are no directories with space/newline issues in that
tree).

 So I ask where you get the idea that it is required by POSIX.

POSIX requires what gets visited, and the GNU extension of -delete
determines the actions attempted on the places visited; since -delete
implies remove(), and since remove(dir/.) is required to fail, it is
the only logical behavior, even if the extension itself is not required
by POSIX.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: Regression: find dir/. -type d -empty -delete claims 'unsuccessful', breaking scripts.

[Eric Blake]

On 11/18/2013 05:25 PM, Linda Walsh wrote:
 They don't require it for the case of find .?
 
 or cp -al a/. b/.
 
 ?

I couldn't parse this question.

 
 
 Are you saying we need an alternate core utils?

Nothing is stopping you from writing your own software that behaves the
way you want.  But I don't see anything in your report that points out
anything wrong with either GNU coreutils or GNU findutils with regards
to attempting to delete a directory with a trailing . as the final
component, and then correctly reporting the failure to you.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: xargs bugs

[Eric Blake]

On 11/01/2013 01:54 AM, Xiao Wu DAI wrote:
 
 hi,
 
 it seems `echo ..` doesn't work in GNU xargs version 4.2.27 like below
 | xargs -t -i mv {}`echo {} |sed 's/mode/dai/g'`

'echo' is your friend.  Try:

echo xargs -t -i mv {} `echo {} | sed 's/mode/dai/g'`

to see that you were executing:

xargs -t -i mv {} {}

which is NOT what you wanted.  You need to use proper quoting; and since
you want to do programmatic shell actions on {}, you need your xargs to
spawn an instance of sh.  Try:

xargs -t -i sh -c 'mv $1 $(echo $1 | sed 's/mode/dai/g')' sh {}

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[bug #40339] mountlist.c:936]: (error) Resource leak: dirp

[Eric Blake]

Follow-up Comment #1, bug #40339 (project findutils):

Already fixed in upstream gnulib commit 98171ec2; just a matter of pulling the
latest gnulib into the next findutils build.

___

Reply to this item at:

  http://savannah.gnu.org/bugs/?40339

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: [bug #34976] find: Failed to save working directory in order to [...]: Too many open files

[Eric Blake]

On 09/20/2013 04:01 PM, James Youngman wrote:
 Update of bug #34976 (project findutils):
 
 Severity:  3 - Normal = 4 - Important  
   Item Group:None = Wrong result   
   Status:None = Fixed  
 
 ___
 
 Follow-up Comment #8:
 
 I've applied and pushed Bernhard's change (after modifying it to update the
 ChangeLog).
 
 The applied patch is
 http://git.savannah.gnu.org/cgit/findutils.git/commit/?id=183115d0484816336f9c4d2a3198b5eae2ad4b92

Looking at that link, I see a (pre-existing) typo:

  /* For -execdir/-okdir predicates, the parser did not fill in
 - the wd_for_exec member of sturct exec_val. So for those
 - predicates, we do so now.
 + the wd_for_exec member of sturct exec_val. So for those
 + predicates, we do so now.

s/sturct/struct/

as well as some non-portable shell:

 + for i in $(seq 0 100) ; do

seq is not POSIX; and I don't know that you can assume that GNU
coreutils are installed everywhere.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: xargs -p operations out of order!

[Eric Blake]

On 07/31/2013 11:01 AM, jida...@jidanni.org wrote:
 What good is this?
 
 $ seq 111|xargs -n 5 -p
 echo 1 2 3 4 5 ?...
 echo 6 7 8 9 10 ?...
 echo 11 12 13 14 15 ?...y
 echo 16 17 18 19 20 ?...11 12 13 14 15
 
 You need to do the operation first before issuing the next prompt!

Sorry, but this is not xargs' fault.  You asked xargs to spawn multiple
processed in parallel, and it is up to the kernel which of those
processes gets scheduled first (they are competing with one another).
Furthermore, since each of those processes all want access to the same
terminal, there is no sane way to guarantee which of the processes will
get there first.  This is not an xargs bug.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: xargs -p operations out of order!

[Eric Blake]

On 07/31/2013 04:00 PM, Bernhard Voelker wrote:
 Hi Eric,
 
 On 07/31/2013 07:10 PM, Eric Blake wrote:
 On 07/31/2013 11:01 AM, jida...@jidanni.org wrote:
 $ seq 111|xargs -n 5 -p
 
 Sorry, but this is not xargs' fault.  You asked xargs to spawn multiple
 processed in parallel, [...]
 
 I'm not sure:
 
 the OP used lower-case -p, so xargs is spawning only _one_ process:

Oh, I totally misread the report, and thought it was complaining about
upper-case -P.

 
   $ xargs --help
   [...]
 -n, --max-args=MAX-ARGS  use at most MAX-ARGS arguments per command 
 line
 -P, --max-procs=MAX-PROCSrun at most MAX-PROCS processes at a time
 -p, --interactiveprompt before running commands
 
 But the effect is the same: with today's implementation, the 'echo'
 process is in parallel to the 'GNU xargs' process asking if it
 should spawn the next 'echo' process.

so the real race here is between xargs prompting about the next command
to be run, vs. the execution of the echo command.

 
 Hmm, POSIX says:
 http://pubs.opengroup.org/onlinepubs/9699919799/utilities/xargs.html
 
   The xargs utility shall then invoke the constructed command line
   and wait for its completion. This sequence shall be repeated until
   one of the following occurs: [...]

Indeed, I think you have a real bug after all - xargs should not be
prompting for the fourth command until after it reaps the echo process
spawned by the affirmative answer to the third prompt.

 
 Do I read it correct?

Yes, I think your reading of POSIX shows we have a bug in xargs.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[bug #38474] Unintended (?) behaviour change of -perm +mode predicate

[Eric Blake]

Follow-up Comment #9, bug #38474 (project findutils):

Paul's patch, as modified in commit 9040c5d, is not quite right.

The bool havekind variable is now useless; it is set to true by all branches
of the switch statement.  It can safely be deleted, along with a later switch
statement when havekind is still false.

The code doesn't reject -perm /+0111.  These lines:
  if (NULL == change
  || (perm_expr[0] == '+'  '0' = perm_expr[1]  perm_expr[1]  '8'))
should probably instead be:
  if (NULL == change
  || (perm_expr[mode_start] == '+'  c_isdigit(perm_expr[mode_start +
1]))

(Note that although c_isdigit will allow '8' and '9', those bogus digits will
have already caused the 'NULL == change' branch to be true; so it is a
reasonable simplification).


___

Reply to this item at:

  http://savannah.gnu.org/bugs/?38474

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug #38474] Unintended (?) behaviour change of -perm +mode predicate

[Eric Blake]

Follow-up Comment #5, bug #38474 (project findutils):

By my reading, '-perm +0100' is indeed unspecified by POSIX (POSIX only calls
out symbolic mode, not numeric mode), so we get to choose how to handle it
(the confusion with leading + is applicable to only things like '-perm +u+x').
 I see two possible choices that make the most sense for how we should handle
it: 1. make it behave the way it used to (no regression), or 2. make it return
an error.  Having it silently change behavior from the old way is not nice. 
I'm 60:40 towards changing it to be an error instead of fixing it to behave
the way it used to.


___

Reply to this item at:

  http://savannah.gnu.org/bugs/?38474

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug #38696] -exec didn't print error message if no '; ' delimiter found and redirection used

[Eric Blake]

Update of bug #38696 (project findutils):

  Status:None = Invalid
 Open/Closed:Open = Closed 


___

Reply to this item at:

  http://savannah.gnu.org/bugs/?38696

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: please cut up help text of xargs into manageable translatable chunks

[Eric Blake]

On 02/08/2013 02:16 PM, Bernhard Voelker wrote:
 
 Kidding aside, another difference is that secondary lines of
 a description are indented by 2 character. I added this to my
 new patch.

Indeed, I missed the importance of indenting subsequent lines, but a
quick glance at 'ls --help' shows that it is the right thing to do.


 
 Clarify -l option: separate from -L to emphasize that MAX-LINES
 is optional for the -l option.

Optional arguments to short options cannot be separated by whitespace.

 +  HTL (_(  -E END   if END occurs as a line of input, 
 the rest of\n
 +  the input is ignored\n));
 +  HTL (_(  -e [END], --eof[=END]equivalent to -E END if END is 
 specified;\n
 +  otherwise, there is no 
 end-of-file string\n));

-e has an optional option-argument, so it must be written '-e[END]', not
'-e [END]'.  That is, 'xargs -e/' is different than 'xargs -e /'.

 +  HTL (_(  --help   print a summary of the options to 
 xargs\n));
 +  HTL (_(  -I R same as --replace=R (R must be 
 specified)\n));
 +  HTL (_(  -i, --replace=[R]replace R in initial arguments with 
 names read\n
 +  from standard input; if R is 
 unspecified,\n
 +  assume {}\n));

-i is another argument with optional argument; it needs to be written
'-i[R]' to make it obvious.

 +  HTL (_(  -L, --max-lines=MAX-LINESuse at most MAX-LINES nonblank 
 input lines per\n
 +  command line\n));
 +  HTL (_(  -l [MAX-LINES]   similar to -L but defaults to at 
 most one\n
 +  nonblank input line if MAX-LINES 
 not specified\n));

And again, needs to be written '-l[MAX-LINES]'.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: Hi small improvement to find command.

[Eric Blake]

On 01/29/2013 01:24 AM, Prashant Sharma wrote:

[Please don't top-post on technical lists]

 The need for this arose with following scenario.
 I have stream processing engine doing some random stuff and creating a lot
 of files in the process. And since I did not wrote it myself and people who
 wrote it do not have clearing up mechanism for the files that accumulate
 and tend to either fill Inodes or File-system.
 
 So needed a way for clearing things up older than 30 seconds. Ofcourse I
 can touch a dummy file and then sleep for 30 seconds and then use find
 -newer.

Why not just touch the file 30 seconds into the past to begin with?
There's no need to sleep, since touch is for more than just the current
timestamp.

 But that sounds like so much for so less and for that matter, one
 can parse the output of stat in perl or even in bash and do the same thing.
 In this case a single line command might sound like an improvement.

Consider - we gave you a solution (with touch and -newer) that works now
(and complies with POSIX), vs. the delay of waiting for adding a new
extension feature and percolating it into your distro (probably a
process of a couple of years, even if we do decide to enhance find, and
even then, limits you to GNU find instead of POSIX).  It's hard to
justify adding a new feature if existing standard features already work
without too much additional effort.

However, you are correct that we can add suffix parsing to -mtime
without violating POSIX, so it might make a nice syntactic sugar
addition.  If you are willing to write the patch, including
documentation and testsuite additions, your proposal will get a lot
further.  But for now, it appears that no one else is interested in
implementing it on your behalf.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: Bug or changed behaviour in GNU find?

[Eric Blake]

 $ find . -name *.h -o -name *.cc -o -name *.tcc

This is equivalent to:

find . \( -name *.h -o -name *.cc -o -name *.tcc \) -print

 ./foo.h
 ./foo.cc
 ./foo.tcc
 
 $ find . -name *.h -o -name *.cc -o -name *.tcc -print0 | tr
 '\0' '\n'

This is equivalent to:

find . -name *.h -o -name *.cc -o \( -name *.tcc -print0 \) | tr '\0' '\n'

 ./foo.tcc
 
 IIRC this used to work with older version of find. Has something
 changed
 in find's behaviour, or am I missing something else here?

You're missing parenthesis, and you are mis-remembering, because find
has never worked the way you seem to be wanting.  Remember that the
implicit -print acts on the entire expression, but that the moment you
use an explicit -print or -print0, then parentheses are necessary to
make the operation have the same scope.

[bug #37423] Duplicate definition of program_name in test-canonicalize

[Eric Blake]

Follow-up Comment #1, bug #37423 (project findutils):

Thanks for the report.  This is an upstream bug in gnulib, and will be
resolved by having findutils update to a newer version of gnulib.

___

Reply to this item at:

  http://savannah.gnu.org/bugs/?37423

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug #37201] Colourized output

[Eric Blake]

Follow-up Comment #1, bug #37201 (project findutils):

And what exactly do you propose coloring?  Are you suggesting that 'find
-printf' should honor LS_COLORS to colorize file names according to file type,
in the same manner as ls?  Patches are certainly welcome, although it would be
best by starting by moving the ls coloring from coreutils into gnulib so that
it would be common code shared by both projects.

___

Reply to this item at:

  http://savannah.gnu.org/bugs/?37201

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: find: sort files by date modified

[Eric Blake]

On 07/20/2012 11:18 AM, Enda wrote:
 find -mtime 0 -printf %f\n
 
 
 I use find to list files that were modified in the last 24 hours, how
 do I sort them in order of when they were modified?

find -mtime 0 -exec ls -d --sort=time {} +

-- 
Eric Blake   ebl...@redhat.com+1-919-301-3266
Libvirt virtualization library http://libvirt.org





signature.asc
Description: OpenPGP digital signature

[bug #36539] -size flag seems to be broken

[Eric Blake]

Update of bug #36539 (project findutils):

  Status:None = Duplicate  
 Open/Closed:Open = Closed 

___

Follow-up Comment #1:

This is a duplicate of bug 12162.  The issue is that the current syntax is
interpreted as 'treat the suffix of the -size argument as block size, then
round the files up to that block size, then filter based on larger or smaller
than the boundary'.  Thus, -size -50k says to find files less than 50 * (1024)
when rounded to 1024 blocks, while -size -1M says to find files less than 1 *
(1024*1024) when rounded to (1024*1024) blocks.  50k rounds up to 1M, so it is
not less than 1 block of size 1M per block.

This is not intuitive, but changing it might break existing syntax. 
Suggestions on how to proceed are welcome.

___

Reply to this item at:

  http://savannah.gnu.org/bugs/?36539

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[bug #36452] xargs needs -I (eye) last ...

[Eric Blake]

Follow-up Comment #1, bug #36452 (project findutils):

POSIX states that mixing -I and -L is not portable:

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/xargs.html

You are using a non-portable command line, and so it should come as no
surprise that BSD and GNU xargs differ in behavior.

That said, you are not the first to request having -I and -L work together,
and presumably with semantics similar to BSD, or even at a bare minimum to
error out instead of silently changing behavior depending on argument
ordering; but no one has bothered to write and submit such a patch.

___

Reply to this item at:

  http://savannah.gnu.org/bugs/?36452

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: how to do findutils cross-compilation for ARM platform?

[Eric Blake]

[adding bug-gnulib]

On 12/25/2011 09:43 PM, chunrong lai wrote:
 I tried to build a ARM version of findutils in my ubuntu 10.10 (x86_64)
 with below commands
   CC=arm-linux-gnueabi-gcc CFLAGS=-g ./configure --build
 arm-cross-linux-gnueabi --host i686-pc-linux-gnu

In general, you should get in the habit of passing CC=... and CFLAGS= as
arguments to configure, rather than via the environment, so that
configure can properly record things so that a later run of
./config.status with a different set of environment variables will still
remember your configure choices.

   make
 
 I met errors as
 arm-linux-gnueabi-gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I../..
  -I../../intl   -g -MT areadlink-with-size.o -MD -MP -MF $depbase.Tpo -c -o
 areadlink-with-size.o areadlink-with-size.c \
 mv -f $depbase.Tpo $depbase.Po
 In file included from ./stdio.h:31,
   from areadlink-with-size.c:25:
 /usr/arm-linux-gnueabi/include/stdio.h:742: error: expected
 declaration specifiers or ‘...’ before ‘(’ token
 /usr/arm-linux-gnueabi/include/stdio.h:742: error: conflicting
 types for ‘rpl_fseeko’
./stdio.h:275: note: previous declaration of ‘rpl_fseeko’ was
 here
 but do not quite understand what happened. Any helps are appreciated.

This sounds like a gnulib problem.  Gnulib thinks that your ARM fseeko()
is deficient (perhaps because it is a pessimistic cross-compilation
guess, rather than an actual bug), but its attempts to replace things
via rpl_fseeko are triggering some sort of circular header inclusion
cycle where the gnulib replacement gets defined prior to the system
declaration.  A proper fix will probably require knowledge of which
header files are involved in the circular inclusion, and/or updating the
cross-compilation logic to guess correctly for your platform in the
first place.

Meanwhile, a quick workaround would be to prime the cache to avoid the
replacement in the first place (assuming that gnueabi's fseeko() is not
deficient, after all); try:

./configure CC=arm-linux-gnueabi-gcc CFLAGS=-g \
  --build arm-cross-linux-gnueabi --host i686-pc-linux-gnu \
  gl_cv_func_fflush_stdin=yes

to see if that is enough to bypass the pessimistic cross-compilation
guess (or post more of config.log so that we know the full set of
cross-compilation guesses in effect).

-- 
Eric Blake   ebl...@redhat.com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[bug #35141] [feature_request] -prune0 action, or a -0 option

[Eric Blake]

Update of bug #35141 (project findutils):

  Status:None = Invalid
 Open/Closed:Open = Closed 

___

Follow-up Comment #1:

Such an option already exists; it is named -print0.

Remember, if you don't specify any of the -ok, -exec, or -print family of
outputs, then find defaults to -print (not -print0).  -prune does not print,
rather, the lack of any other action implies the default of -print.  Your
example was thus parsed as:

find . ( -name 'bin*' -type d -prune ) -print | xargs -r rm -r -- 

when it sounds like you wanted something like:

find . -name 'bin*' -type d -prune -o -print0 | xargs -r rm -r --

or even more efficiently:

find -name 'bin*' -type d -prune -o -delete

I'm closing this as not a bug; there is no need to add -prune0 as a
NUL-printing counterpart of -prune, since -prune doesn't print in the first
place.


___

Reply to this item at:

  http://savannah.gnu.org/bugs/?35141

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: Dowload area shows still 4.4.2

[Eric Blake]

On 12/08/2011 07:04 AM, Voelker, Bernhard wrote:
 Andreas Metzler wrote:
 
 4.4.2 is still the latest stable release. Development versions (4.5.x)
 are available on http://alpha.gnu.org/pub/gnu/findutils/.
 
 Thank you. Then it's okay.
 
 But ... alpha?
 Cygwin is at 4.5.9.

Yes, that is an intentional choice of the cygwin distro to ship an
alpha-quality product.  Distros are free to pick stability or bleeding
edge, according to what they are willing to support, and according to
whether bleeding edge fixes issues that the distro packager (aka me for
cygwin) is not willing to cherry-pick back into stable.

-- 
Eric Blake   ebl...@redhat.com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: How to find by birth time?

[Eric Blake]

On 11/20/2011 10:10 AM, Peng Yu wrote:
 Hi,
 
 stat can give birth time. Is there an option search by birth time in
 find? (I find -ctime -atime -mtime but not -btime in find.)

It would be spelled -Btime, not -btime.  But be aware that this requires
kernel support - BSD and Cygwin support birthtime as part of stat(), but
Linux does not (so it will NOT work on Linux until the kernel developers
ever give us the xstat() interface).

-- 
Eric Blake   ebl...@redhat.com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: How to only return the 1 match then quit?

[Eric Blake]

On 11/19/2011 08:44 AM, Peng Yu wrote:
 Hi,
 
 I'm not able to find an argument so that find will only return the
 first match then quit. Could you please let me know if there is such
 an option?

find ... -print -quit

-- 
Eric Blake   ebl...@redhat.com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Findutils-patches] [PATCH] This change adds a sleep between invocations of cmd.

[Eric Blake]

On 09/20/2011 12:33 PM, ed wrote:

Gave that a go today on a vanilla xargs and it doesn't seem to work:

   $ perl -e 'for(my $a=0;$a100;$a++){print $a,\n;}' | xargs -l2 sh -c
   'sleep 1exec /bin/echo $@'
   1
   3
   5
   ^C

For whatever reason the shell seems to throw the first argument away.


The first argument becomes $0 to the shell, so you need to inject a 
dummy.  You meant:


perl -e 'for(my $a=0;$a100;$a++){print $a,\n;}' \
 | xargs -l2 sh -c 'sleep 1  exec /bin/echo $@' sh

so that your dummy 'sh' provides the proper $0, and the rest of your 
arguments are used as you meant.


--
Eric Blake   ebl...@redhat.com+1-801-349-2682
Libvirt virtualization library http://libvirt.org

[bug #34159] -path and -prune with -print and without have different result.

[Eric Blake]

Update of bug #34159 (project findutils):

  Status:None = Invalid
 Assigned to:None = ericb  
 Open/Closed:Open = Closed 

___

Follow-up Comment #2:

Not a bug.  -prune is documented as only having an action on directories, not
regular files - so -prune does not trim files.


$ find . -path '*/file' -prune -o -type f


is strictly equivalent to:


$ find . ( -path '*/file' -prune -o -type f ) -print


-path '*/file' is true for dir/file, -prune is always true but has no effect,
so the -o does not kick in, then the implicit -print prints the file name. 
Meanwhile:


$ find . -path '*/file' -prune -o -type f -print


is strictly equivalent to:


$ find . ( -path '*/file' -prune ) -o ( -type f -print )


-path '*/file' is true, -prune is true and has no effect, so the -o does not
kick in, and there is nothing left to -print the file name.


___

Reply to this item at:

  http://savannah.gnu.org/bugs/?34159

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: How to only find the first hit then return?

[Eric Blake]

On 08/25/2011 06:46 PM, Peng Yu wrote:

Hi,

By default, find will look for all the matches. Is there a way to let
find only get the first match and then return?


find ... -print -quit

does your normal ... find, prints the first answer, then quits.

--
Eric Blake   ebl...@redhat.com+1-801-349-2682
Libvirt virtualization library http://libvirt.org

Re: [PATCH] fts: introduce FTS_NOATIME

[Eric Blake]

On 07/08/2011 10:55 AM, Jim Meyering wrote:
 Eric Blake wrote:
 This gives clients the option to try a non-invasive traversal,
 where merely visiting a directory does not update its timestamp,
 where such is supported by the kernel.

 Note that whiteout support and O_NOATIME support are orthogonal:
 there is no way to get O_NOATIME behavior when using __opendir2
 to visit whiteouts on BSD systems.

 * lib/fts_.h (FTS_NOATIME): New bit flag.
 (FTS_OPTIONMASK): Adjust.
 * lib/fts.c (diropen, fts_open, fts_build): Honor it.
 (fd_ring_check): Debug code unconditionally uses O_NOATIME.
 Needed for findutils bug http://savannah.gnu.org/bugs/?33724
 
 This looks fine.  Thanks!

Now pushed.

I'll start looking into adding a --noatime into the various recursive
coreutils.  Gnulib's replacement fcntl.h guarantees that O_NOATIME is
defined; however it might be 0 on systems with no support, and even
where it is non-zero, there are older Linux kernels that silently ignore
the flag.  Therefore, my thoughts are that:

if O_NOATIME is 0, --noatime should be rejected as an outright
impossibility.

if O_NOATIME is non-zero, --noatime should enable FTS_NOATIME, but only
as a best-effort (that is, the option is silently ignored where the
kernel is too old), as there is no easy interface to tell at runtime
whether O_NOATIME will actually be honored.

-- 
Eric Blake   ebl...@redhat.com+1-801-349-2682
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[bug #33724] Find command is changing the access time of directory

[Eric Blake]

Follow-up Comment #1, bug #33724 (project findutils):

POSIX requires this behavior.  However, Linux provides the O_NOATIME flag to
open() to work around this behavior - so the obvious conclusion is that find
should support the option of requesting the use of the O_NOATIME flag.   It
can't be on by default, but certainly makes sense to support it where
possible.

___

Reply to this item at:

  http://savannah.gnu.org/bugs/?33724

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

fts and O_NOATIME

[Eric Blake]

http://savannah.gnu.org/bugs/?33724 raised the issue that findutils is
updating directory atime as part of traversing a hierarchy (and,
according to POSIX, this behavior is the correct default).  However, it
would be nice to support the use of fts in a mode where O_NOATIME is
used, so that directories don't appear to have an atime change merely
because fts called readdir.

It looks to me like the only way to do this is to use open(O_NOATIME)
followed by fdopendir() - but we are already using this approach for
other reasons.  So would it be worth a patch that adds a new FTS_NOATIME
flag, and when that flag is set, pass in the O_NOATIME flag to all
directories opened by fts?

-- 
Eric Blake   ebl...@redhat.com+1-801-349-2682
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[PATCH] fts: introduce FTS_NOATIME

[Eric Blake]

This gives clients the option to try a non-invasive traversal,
where merely visiting a directory does not update its timestamp,
where such is supported by the kernel.

Note that whiteout support and O_NOATIME support are orthogonal:
there is no way to get O_NOATIME behavior when using __opendir2
to visit whiteouts on BSD systems.

* lib/fts_.h (FTS_NOATIME): New bit flag.
(FTS_OPTIONMASK): Adjust.
* lib/fts.c (diropen, fts_open, fts_build): Honor it.
(fd_ring_check): Debug code unconditionally uses O_NOATIME.
Needed for findutils bug http://savannah.gnu.org/bugs/?33724

Signed-off-by: Eric Blake ebl...@redhat.com
---

Does this look correct?

 ChangeLog  |6 ++
 lib/fts.c  |   17 ++---
 lib/fts_.h |4 +++-
 3 files changed, 19 insertions(+), 8 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index dbf0bea..1dfa2b2 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,11 @@
 2011-07-07  Eric Blake  ebl...@redhat.com

+   fts: introduce FTS_NOATIME
+   * lib/fts_.h (FTS_NOATIME): New bit flag.
+   (FTS_OPTIONMASK): Adjust.
+   * lib/fts.c (diropen, fts_open, fts_build): Honor it.
+   (fd_ring_check): Debug code unconditionally uses O_NOATIME.
+
getopt: avoid compiler warning during configure
* m4/getopt.m4 (gl_GETOPT_CHECK_HEADRS): Avoid problems with
assigning string literals to non-const pointer.
diff --git a/lib/fts.c b/lib/fts.c
index d3ad1cd..7210c1b 100644
--- a/lib/fts.c
+++ b/lib/fts.c
@@ -349,7 +349,8 @@ internal_function
 diropen (FTS const *sp, char const *dir)
 {
   int open_flags = (O_SEARCH | O_DIRECTORY | O_NOCTTY | O_NONBLOCK
-| (ISSET (FTS_PHYSICAL) ? O_NOFOLLOW : 0));
+| (ISSET (FTS_PHYSICAL) ? O_NOFOLLOW : 0)
+| (ISSET (FTS_NOATIME) ? O_NOATIME : 0));

   int fd = (ISSET (FTS_CWDFD)
 ? openat (sp-fts_cwd_fd, dir, open_flags)
@@ -406,7 +407,8 @@ fts_open (char * const *argv,
early, doing it here saves us the trouble of ensuring
later (where it'd be messier) that . can in fact
be opened.  If not, revert to FTS_NOCHDIR mode.  */
-int fd = open (., O_SEARCH);
+int fd = open (.,
+   O_SEARCH | (ISSET (FTS_NOATIME) ? O_NOATIME : 0));
 if (fd  0)
   {
 /* Even if `.' is unreadable, don't revert to FTS_NOCHDIR mode
@@ -1241,10 +1243,11 @@ fts_build (register FTS *sp, int type)
 opendirat((! ISSET(FTS_NOCHDIR)  ISSET(FTS_CWDFD) \
? sp-fts_cwd_fd : AT_FDCWD),\
   file, \
-  ((ISSET(FTS_PHYSICAL) \
- ! (ISSET(FTS_COMFOLLOW)  \
-   cur-fts_level == FTS_ROOTLEVEL))  \
-   ? O_NOFOLLOW : 0),   \
+  (((ISSET(FTS_PHYSICAL)\
+  ! (ISSET(FTS_COMFOLLOW) \
+cur-fts_level == FTS_ROOTLEVEL)) \
+? O_NOFOLLOW : 0)   \
+   | (ISSET (FTS_NOATIME) ? O_NOATIME : 0)),\
   dir_fd)
 #endif
if ((dirp = __opendir2(cur-fts_accpath, oflag)) == NULL) {
@@ -1653,7 +1656,7 @@ fd_ring_check (FTS const *sp)
   int fd = i_ring_pop (fd_w);
   if (0 = fd)
 {
-  int parent_fd = openat (cwd_fd, .., O_SEARCH);
+  int parent_fd = openat (cwd_fd, .., O_SEARCH | O_NOATIME);
   if (parent_fd  0)
 {
   // Warn?
diff --git a/lib/fts_.h b/lib/fts_.h
index 6a22bec..fe92cfd 100644
--- a/lib/fts_.h
+++ b/lib/fts_.h
@@ -142,7 +142,9 @@ typedef struct {
  dirent.d_type data.  */
 # define FTS_DEFER_STAT 0x0400

-# define FTS_OPTIONMASK 0x07ff  /* valid user option mask */
+# define FTS_NOATIME0x0800  /* use O_NOATIME during traversal */
+
+# define FTS_OPTIONMASK 0x0fff  /* valid user option mask */

 # define FTS_NAMEONLY   0x1000  /* (private) child names only */
 # define FTS_STOP   0x2000  /* (private) unrecoverable error */
-- 
1.7.4.4

Re: [PATCH] fts: introduce FTS_NOATIME

[Eric Blake]

On 07/07/2011 03:02 PM, Paul Eggert wrote:
 On 07/07/11 10:41, Eric Blake wrote:
 This gives clients the option to try a non-invasive traversal,
 
 Thanks for doing that; a couple of minor comments:
 
 -int fd = open (., O_SEARCH);
 +int fd = open (.,
 +   O_SEARCH | (ISSET (FTS_NOATIME) ? O_NOATIME : 
 0));
 
 Shouldn't this use diropen rather than open?  Then you don't need
 to worry about checking the flag here.  (This comment applies to
 the existing code, too.)

Possibly.  Jim?

 
 +  int parent_fd = openat (cwd_fd, .., O_SEARCH | O_NOATIME);
 
 This uses O_NOATIME even when the user hasn't asked for it.

Yeah, because it is in debug-only code.  Debug code shouldn't interfere
with normal operation, whether or not FTS_NOATIME is in effect in normal
operation.

-- 
Eric Blake   ebl...@redhat.com+1-801-349-2682
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [PATCH 1/2] Eliminate some compiler warnings.

[Eric Blake]

On 06/14/2011 04:46 PM, James Youngman wrote:
 * find/find.c: Remove definition of SAFE_CHDIR, which we don't
 use.
 * find/defs.h: Remove redundant declaration of launch.
 * find/parser.c (parse_false): Cast unused arguments to void.
 (parse_print0): Likewise.
 (pred_context): Likewise.
 (parse_newerXY): Add some parens for slightly greater clarity.
 (make_segment): Avoid switch-missing-default-case warning by
 turning it into an if statement.
 (check_path_safety): Remove unused argument.
 (insert_exec_ok): Don't pass the unwanted arugment to

s/arugment/argument/

 check_path_safety.
 (get_relative_timestamp): silence compiler warning by adding a
 case for the remaining enumberation value rather than using

s/enumberation/enumeration/

 @@ -1614,8 +1617,8 @@ parse_newerXY (const struct parser_table* entry, char 
 **argv, int *arg_ptr)
  
/* -newertY (for any Y) is invalid. */
if (x == 't'
 -   || 0 == strchr (validchars, x)
 -   || 0 == strchr ( validchars, y))
 +   || (0 == strchr (validchars, x))
 +   || (0 == strchr ( validchars, y)))

strchr() returns a pointer, so I prefer to compare to NULL rather than
0.  Or even:

!strchr (validchars, x)

-- 
Eric Blake   ebl...@redhat.com+1-801-349-2682
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [PATCH 2/2] Fix further compiler warnings.

[Eric Blake]

On 06/14/2011 04:46 PM, James Youngman wrote:
 * find/ftsfind.c (show_outstanding_execdirs): Now that
 execp-state.cmd_argc is a size_t, we can't print it with %d.  So
 print it with PRIuMAX (and include inttypes.h to define that).

Or suck in the fprintf-posix module, and use %zu instead of PRIuMAX, to
save a cast.  But that's pretty heavy-weight, so your approach is fine.

 @@ -31,6 +42,7 @@
   (prec_name): Likewise!  Also remove spurious parentheses around
   return value.
   (prec_name): Remove spurious parentheses around return value.
 +
   * lib/buildcmd.h (buildcmd_state): change types of several fields

Was this hunk intentional?

-- 
Eric Blake   ebl...@redhat.com+1-801-349-2682
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[bug #33459] Find -exec not working correctly with binary matching

[Eric Blake]

Follow-up Comment #2, bug #33459 (project findutils):


find . -name test1 -or -name test2 -exec echo {} ;
find . -name test1 -or -name test2 -print


are identical to:


find . -name test1 -or ( -name test2 -exec echo {} ; )
find . -name test1 -or ( -name test2 -print )


while


find . -name test1 -or -name test2


is identical to:


find . ( -name test1 -or -name test2 ) -print


Notice that the absence of -print or -exec results in implicit () around the
rest of the expressions.

but the first 2 commands give the following output
./test2
the last command outputs properly with
./test1
./test2 

In all three commands, the results you saw are correct.  I think the only bug
here is that you misunderstood how the implicit -print also implies (), which
changes precedence on the rest of the line.


___

Reply to this item at:

  http://savannah.gnu.org/bugs/?33459

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Re: findutils on interix

[Eric Blake]

On 05/26/2011 01:10 AM, Markus Duft wrote:
 2. modify the configure script to refuse to build findutils at all on
 Interix unless suacomp is installed [i.e. make things worse].

 
 I'd be more for 2, as i'm currently contaminating a whole lot of packages 
 with suacomp anyway (gnulib, coreutils, etc.).


I would also strongly favor option 2, since gnulib recently documented
that it will only target Interix with suacomp installed:

http://lists.gnu.org/archive/html/bug-gnulib/2011-05/msg00368.html

In other words, suacomp is not deemed a burden for that platform, and
Interix without suacomp is unsupported by the very gnulib code you are
trying to use to work around Interix deficiencies.

-- 
Eric Blake   ebl...@redhat.com+1-801-349-2682
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: A bug in findutils?

[Eric Blake]

On 03/24/2011 02:05 AM, Cao, Da-Shi (EB-Presales-ZTE/HW-GZ) wrote:
 The compilation of findutils 4.4.2 failed when I did a cross compile on i386 
 to x86_64. The error is that type wint_t didn't get defined so the 
 compilation of mbchar.c in gnulib/lib failed.
 The type wint_t is defined in system wchar.h under some conditions. The 
 system wchar.h is included by gnulib/lib/wchar.h.
 The cross compilation error can be demonstrated by:
 #include stdio.h
 #include wchar.h
 
 It is supposed that wint_t should get defined after this sequence, but it 
 is not in this particular cross compilation.
 stdio.h includes _G_config.h, which in turn include wchar.h.
 But in the cross compilation there is a wchar.h in gunlib/lib. This header 
 file is guarded by _GL_WCHAR_H.
 This gnulib/lib/wchar.h includes system wchar.h. But when included from 
 _G_config.h, type wint_t is not defined. This is the expected behavior.
 But the #include wchar.h in _G_config.h gets _GL_WCHAR_H defined, so the 
 #include wchar.h as in the demonstration is bypassed.
 I cannot figure out how to make _GL_WCHAR_H not got defined when 
 gnulib/lib/wchar.h is included by _G_config.h.

Thanks for the report.  In reality, this is a gnulib issue, so I have
added bug-gnulib.  Also, I'm not sure if gnulib has fixed the problem in
the meantime, and findutils is just suffering from a stale gnulib
release, or if it still exists in upstream gnulib.

Are you in a position where you could help test building from git?  With
a recent gnulib.git checkout, it would be nice to know the results of:
./gnulib-tool --create-testdir --with-tests --dir=foo wchar stdio
followed by configuring in the just-created foo directory and running
'make check' for your cross compiler setup.

-- 
Eric Blake   ebl...@redhat.com+1-801-349-2682
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[bug #32519] Back-quoted `uuidgen` in -exec part is not executed everytime

[Eric Blake]

Update of bug #32519 (project findutils):

  Status:None = Invalid
 Assigned to:None = ericb  
 Open/Closed:Open = Closed 

___

Follow-up Comment #1:

Your problem is that you want to run multiple commands on each -execdir (both
unzip and uuidgen), but -execdir can only run a single command.  The solution
is to wrap things in yet another command - a shell that can then exec your
multiple commands.


find -name '*.zip' -execdir sh -c 'unzip $@ -d `uuidgen -r`' sh {} ;



___

Reply to this item at:

  http://savannah.gnu.org/bugs/?32519

___
  Message sent via/by Savannah
  http://savannah.gnu.org/