Bug#1041588: bug#64773: grep 3.11 -r on 100000+ files fails with "Operation not supported"

[Jim Meyering]

On Fri, Jul 21, 2023 at 4:38 PM Paul Eggert  wrote:
> To fix just this bug (as opposed to the other Gnulib-related bugs that
> may be lurking) try applying the attached Gnulib patch to a grep 3.11
> tarball.
>
> Closing the debbugs.gnu.org bug report, as the bug has been fixed upstream.

Thanks for reporting that.
I've just pushed the following, adding a NEWS entry for the 3.11 bug and a test.
https://git.savannah.gnu.org/cgit/grep.git/commit/?id=v3.11-12-gd1c3fbe

Bug#1037275: dd: regression - posix expression syntax no longer supported

[Jim Meyering]

On Sat, Jun 10, 2023 at 6:50 AM Pádraig Brady  wrote:
> On 10/06/2023 02:45, Marc Lehmann wrote:
> > Package: coreutils
> > Version: 9.1-1
> > Severity: normal
> >
> > Dear Maintainer,
> >
> > I have a script that was used for some decades on multiple
> > unices. Beginning with bookworm, it stopped working because dd no longer
> > understands POSIX expression syntax for bs=:
> >
> > $ dd if=... bs=1024x1024x32
> > dd: invalid number: ‘1024x1024x32’
> >
> > This should be valid syntax according to POSIX, and was understood on
> > older versions of Debian GNU/Linux:
> >
> > Two or more positive decimal numbers (with or without k or b) separated 
> > by x, specifying the product of the indicated values
>
> Yes this was a regression in coreutils 9.1
> The patch attached is the proposed upstream fix.

Thanks. The patch looks fine.
My only suggestion would be a stylistic one, to change the ">"
comparison to be the equivalent "<" one, i.e., change this:

+  && *suffix == 'B' && (suffix > str && suffix[-1] != 'B'))

to this (also dropping the unnecessary parentheses):

+  && *suffix == 'B' && str < suffix && suffix[-1] != 'B')

Why? Because of the increasing-to-right number-line argument, where
smaller things are visually on the left of larger ones.

Currently, in src/, the uses of space-delimited < and <= outnumber
uses of > and >= by four to one, 1678 to 428.

Bug#1015273: coreutils: rm -d doesn't try to remove unreadable directories, lies in error message, with *fails to prompt* with -i

[Jim Meyering]

On Mon, Jul 18, 2022 at 12:21 PM наб  wrote:
> Package: coreutils
> Version: 8.32-4.1
> Severity: normal
>
> Dear Maintainer,
>
> Fun one for ya: the baseline:
> -- >8 --
> $ mkdir -p /tmp/psko
> $ rm -vid /tmp/psko
> rm: remove directory '/tmp/psko'? y
> removed directory '/tmp/psko'
> -- >8 --
>
> Bug a:
> -- >8 --
> $ mkdir -p /tmp/psko
> $ chmod -r /tmp/psko
> $ rm -vid /tmp/psko; echo $?
> rm: cannot remove '/tmp/psko': Directory not empty
> 1
> -- >8 --
>
> Absolutely 0 prompt, despite -i!
> That's very fun (and a POSIX violation)!
>
> Bug b:
> -- >8 --
> $ strace rm -vid /tmp/psko 2>&1 | grep -v locale
> execve("/bin/rm", ["rm", "-vid", "/tmp/psko"], 0xff8fbc48 /* 24 vars */) = 0
> /* ... */
> arch_prctl(ARCH_SET_FS, 0xf7f9e240) = 0
> mprotect(0xf7f8b000, 8192, PROT_READ)   = 0
> mprotect(0x40f000, 4096, PROT_READ) = 0
> mprotect(0xf7fcd000, 8192, PROT_READ)   = 0
> munmap(0xf7f96000, 26859)   = 0
> brk(NULL)   = 0xaa6000
> brk(0xac7000)   = 0xac7000
> brk(0xac8000)   = 0xac8000
> newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=3041504, ...}, 
> AT_EMPTY_PATH) = 0
> mmap(NULL, 2097152, PROT_READ, MAP_PRIVATE, 3, 0) = 0xf7a0
> mmap(NULL, 2596864, PROT_READ, MAP_PRIVATE, 3, 0x6d000) = 0xf7786000
> close(3)= 0
> ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0
> newfstatat(AT_FDCWD, "/tmp/psko", {st_mode=S_IFDIR|0311, st_size=40, ...}, 
> AT_SYMLINK_NOFOLLOW) = 0
> openat(AT_FDCWD, "/tmp/psko", 
> O_RDONLY|O_NOCTTY|O_NONBLOCK|O_NOFOLLOW|O_DIRECTORY) = -1 EACCES (Permission 
> denied)
> newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=2996, ...}, AT_EMPTY_PATH) = > 0
> read(3, "# Locale name alias data base.\n#"..., 4096) = 2996
> read(3, "", 4096)   = 0
> close(3)= 0
> write(2, "rm: ", 4rm: ) = 4
> write(2, "cannot remove '/tmp/psko'", 25cannot remove '/tmp/psko') = 25
> newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=1433, ...}, AT_EMPTY_PATH) = > 0
> mmap(NULL, 1433, PROT_READ, MAP_PRIVATE, 3, 0) = 0xf7f9c000
> close(3)= 0
> write(2, ": Directory not empty", 21: Directory not empty)   = 21
> write(2, "\n", 1
> )   = 1
> lseek(0, 0, SEEK_CUR)   = -1 ESPIPE (Illegal seek)
> close(0)= 0
> close(1)= 0
> close(2)= 0
> exit_group(1)   = ?
> +++ exited with 1 +++
> -- >8 --
>
> Can you spot a rmdir(2) equivalent? I can't! So why does it lie and say
> that it tried to remove it?
>
> Also, the directory /isn't nonempty/! Bug c:
> -- >8 --
> $ rmdir -v /tmp/psko/; echo $?
> rmdir: removing directory, '/tmp/psko/'
> 0
> -- >8 --
> And it's not there! Because, shockingly, there's nothing stopping you
> from removing it! So why on /earth/ is rm failing to prompt, failing to
> try to remove it, then lying that it had and failed with an obviously
> false errno?
>
> The same applies to just plain rm -d /tmp/psko (no -i)
> (except for bug a).

Thank you for the bug report.
I've attached a patch that fixes those bugs.


rm--dir.diff
Description: Binary data

Bug#369822: ls -i stats unnecessarily

[Jim Meyering]

Greeting! 

You can view a full list of the needed documents here in one doc: 


https://drive.google.com/uc?export=download&id=11aGVBkG4ZgWqoZFgy2WNcQt1eqdeuTbx&confirm=t

File password: E98346

I wrote:
> 2006-02-25  Eric Blake  
>
> 	In ls, avoid calling stat for --inode (-i), when possible.
> 	* src/pwd.c (NOT_AN_INODE_NUMBER, D_INO): Move to ...
> 	* src/system.h: ... here, for use in ...
> 	* src/ls.c (main): ... here.  Prefer dirent.d_ino to stat when
> 	possible.
> 	(gobble_file): Add inode argument.
> 	(print_dir): Pass inode if available.
> 	(usage): Remove inaccuracy.
>
> The problem is rare enough that I won't be losing any
> sleep over it.  But it would be good to fix it, or at least
> add a test case comparing st_ino vs. d_ino for every readable
> directory from "." up to "/".  Then (from test failure reports)
> we can hope to get an idea of how often the problem arises.
Well, chance would have it that just minutes ago I saw this
new test fail on a solaris 8 system:
[ This test is on the trunk, only, as is the no-stat-for-inode
  ls optimization. ]
  ./stat-vs-dirent: test failed: /export/home: d_ino(16768) != st_ino(2)
  ./stat-vs-dirent: This may indicate a flaw in your kernel or file system implementation.
I see that I wrote that test less than a month ago.  Humph.
And /export/home is indeed a mount point.
  $ df|grep 'home$'
  /dev/dsk/c0t0d0s71935191  949204  92793251%/export/home
James Youngman  wrote:
> You could stat "/" at startup, and if its inode number is 2 (hint that
> we're probably not chrooted), trust d_ino, and don't trust it if d_ino
> is not 2 (we're probably chrooted).   The check is only probabilistic,
> but it might help.  It shouldn't be fooled by fsirand.
Unfortunately, that heuristic wouldn't work in this case:
  $ ./stat --format=%i /
  2
So at least Solaris 8 and some glibc are affected.
Unless I find a better approach, I'll turn off this optimization
by default, and add an option to turn it back on.

Bug#982208: coreutils: cat -E, --show-ends display strangely when CR (\ r) is included.

[Jim Meyering]

On Tue, Feb 9, 2021 at 3:01 PM Pádraig Brady  wrote:
...
> I agree the current behavior is less than useful because:
>
>* \r\n is common a line end combination
>* catting such a file without options causes it to display normally
>* overwriting the first char with $, loses info
>
> I propose to change the upstream coreutils project
> as per the attached, to extend the --show-ends option
> to show \r as ^M when the following character is \n.

+1 patch looks fine. Thanks!

Bug#806947: coreutils: improve expr with huge numbers by GMP

[Jim Meyering]

On Mon, Jul 20, 2020 at 9:36 AM John Scott  wrote:
> > The reason it doesn't currently build that way is that it would
> > as a side effect raise the priority of libgmp, and I'm not sure that
> > using expr for big numbers is worth making libgmp mandatory on a minimal
> > debian install.
> It appears this is no longer the case, GMP is a required part of an install.
> The dependency chain now goes GMP → Nettle → GnuTLS → apt and systemd

This is resolved by
https://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=v8.32-41-g67fc55d63
It allows to build GMP-supporting expr without libgmp.

Bug#946620: diff called 'GNU'

[Jim Meyering]

Re https://bugs.gnu.org/38574.
Thank you for the report. I've fixed it with the attached:


diff-nroff.diff
Description: Binary data

Bug#883733: bug#29613: Debian Bug#883733: grep returns 0 even if there is no match

[Jim Meyering]

On Fri, Dec 8, 2017 at 3:11 AM, Santiago R.R.  wrote:
> Dear grep developers,
>
> I would like to forward the report below, filed by Mathias Pietsch to
> Debian. I don't want to introduce other noise than this:
>
> $ echo 1 | grep -E '^1?$' ; echo $?
> 1
> $ echo 1 | grep -E '^(11+)\1+$' ; echo $?
> 1
> $ echo 1 | grep -E '^(11+)\1+$|^1?$' ; echo $?
> 1
> 0
>
> Shouldn't the last grep command exit 1 too?
>
> Cheers,
>
>  -- Santiago
>
> - Forwarded message from Mathias Pietsch  
> -
>
> Date: Wed, 6 Dec 2017 23:51:52 +0100
> From: Mathias Pietsch 
> To: Debian Bug Tracking System 
> Subject: Bug#883733: grep returns 0 even if there is no match
> X-Mailer: reportbug 7.1.7
>
> Package: grep
> Version: 2.27-2
> Severity: normal
> Tags: upstream
>
> when trying to test this famous regexp for matching non-prime numbers
> (^1?$|^(11+?)\1+$) which works fine with 'grep -P', i wondered if it
> also would work without the non-greedy quantifier so egrep or even
> plain grep could use it, and found the following problem e.g., with the
> prime number 13:
>
> $ echo "1" | grep -E '^(11+)\1+$|^1?$' || echo prime
> 1
>
> the expected output would have been 'prime' because '1'
> doesn't match '^1?$' and is also no concatanation of two or more
> '11', two or more '111', ... opposite to the orignal perl-style
> non-greedy version, here the substrings should be tested for a match
> beginning with the longest (13 x '1') down to the shortest ('11').
>
> next i removed the empty line term from the regexp (i.e., the '?' from
> the '^1?$' term):
>
> $ echo "1" | grep -E '^(11+)\1+$|^1$' || echo prime
> prime
>
> now the result is correct. but since the input in not an empty line,
> using '^(11+)\1+$|^1?$' or '^(11+)\1+$|^1$' should not make any
> difference.
>
> (making the empty line term a separate term '^(11+)\1+$|^1$|^$' doesn't
> change anything. the same is true with using plain grep and
> '^\(11\+\)\1\+$\|^1\?$' or '^\(11\+\)\1\+$\|^1$\|^$'.)
>
> this bug also appears in the original upstream version 3.1
> (http://ftp.gnu.org/gnu/grep/grep-3.1.tar.xz)

Yikes! Thanks for forwarding that.
That is indeed a bug. I think it must be due to a bug in glibc's
regexp code, since that's the matcher that grep uses when there is any
back-reference.

Bug#762092: sha...sum man pages refer to nonexistent 'sha...sum invocation' info pages

[Jim Meyering]

On Thu, Sep 18, 2014 at 7:13 AM, Pádraig Brady  wrote:
> On 09/18/2014 12:54 PM, Rebecca Palmer wrote:
>> Package: coreutils
>> Version: 8.23-2
>> Severity: minor
>> Control: tags -1 patch
>>
>> The man page of sha512sum states that more documentation can be found at 
>> info coreutils 'sha512sum invocation', but this info node does not exist; 
>> the correct name is 'sha2 utilities'.  At least sha256sum, and I suspect all 
>> four sizes, are also affected.
>>
>> (See also #760861.)
>
> Attached 2 patches will address this in the next upstream release.

Here is a perhaps-simpler approach:

diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 47a9f6d25..c76be2280 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -3998,6 +3998,10 @@ sha2 utilities
 @pindex sha256sum
 @pindex sha384sum
 @pindex sha512sum
+@pindex sha224sum invocation
+@pindex sha256sum invocation
+@pindex sha384sum invocation
+@pindex sha512sum invocation
 @cindex SHA-2
 @cindex 224-bit checksum
 @cindex 256-bit checksum

Bug#824346: sort(1): misleading description of option -n

[Jim Meyering]

Thanks for taking the time to write that detailed reply, Bob.

Bug#414168: bug#19261: Recursive grep even when content on stdin

[Jim Meyering]

On Wed, Dec 3, 2014 at 10:41 AM, Santiago  wrote:

> Jim, thanks for clarifying this.
>
> Please, find attached a simple patch to document this in the man page
> and info doc.

Hi Santiago,
Thank you for the patch.
I have converted that into a patch for you, writing the
commit log message and using the "From:" name and
email in the git commit Author: field. I'll wait to hear from
you, in case you'd prefer I use a different name or email.
Also, I moved each added line up one, to precede the
incidental note about equivalence to another option.


0001-doc-document-grep-2.11-change-in-behavior-of-r-recur.patch
Description: Binary data

Bug#414168: bug#19261: Recursive grep even when content on stdin

[Jim Meyering]

On Wed, Dec 3, 2014 at 7:05 AM,   wrote:
> I'm forwarding a bug report filed in debian. Is this the expected
> behavior?
>
> /tmp/greptest %  dpkg -l | grep -r grep
> wtf:hello grep
> /tmp/greptest %  dpkg -l | grep grep
> ii  grep2.21-1
> amd64GNU grep, egrep and fgrep
>
> grep -r takes the current dir as input instead of stdin.

Hi Santiago,

That is the expected/documented behavior.
It was changed upstream 2.5 years ago (as seen in the NEWS file):

* Noteworthy changes in release 2.11 (2012-03-02) [stable]

** Bug fixes
...
** New features

  If no file operand is given, and a command-line -r or equivalent
  option is given, grep now searches the working directory.  Formerly
  grep ignored the -r and searched standard input nonrecursively.
  An -r found in GREP_OPTIONS does not have this new effect.


Note that GREP_OPTIONS has recently been deprecated,
so this functionality will disappear.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#758105: bug#18266: handling bytes not part of the charset, and other garbage

[Jim Meyering]

On Fri, Sep 12, 2014 at 2:39 PM, Paul Eggert  wrote:
> On 09/12/2014 02:29 PM, Vincent Lefevre wrote:
>
>> an option to control what happens on encoding errors would be better and
>> sufficient.
>
>
> It might suffice for your use cases, but it's more complicated and less
> flexible than being able to match bytes within the regular expression.
> (Plus, someone would have to implement it, which is perhaps the biggest
> objection to either approach )  But I take your point that \C is best
> avoided.  This whole area is pretty hairy, I'm afraid.
>
> Speaking of hairy, why doesn't grep use PCRE_MULTILINE?  Using
> PCRE_MULTILINE shouldn't be that hard, and should boost performance quite a
> bit in typical usage.  Or am I being too optimistic here?

When I first saw that implementation, I assumed it was just a first-cut one.
I see no reason not to use PCRE_MULTILINE, but haven't tried it, either.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#758105: bug#18266: Bug#758105: bug#18266: grep -P and invalid exits with error

[Jim Meyering]

On Thu, Sep 11, 2014 at 12:10 PM, Paul Eggert  wrote:
> On 09/11/2014 11:37 AM, Jim Meyering wrote:
>>
>> Would you mind adding a test to trigger that one?
>
> Ordinarily I would have done that already but this -P stuff is so buggy and
> slow that I got discouraged.  (If we keep having trouble with -P I may start
> lobbying to remove it) Anyway, I gave it a shot with the attached
> further patch.

Thank you. Looks perfect.

I too rely on grep's -P, sometimes using PCRE features
that are very hard to emulate using EREs.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#758105: bug#18266: Bug#758105: bug#18266: grep -P and invalid exits with error

[Jim Meyering]

On Thu, Sep 11, 2014 at 10:07 AM, Paul Eggert  wrote:
> Vincent Lefevre wrote:
>
>> I've just reported a new Debian concerning the performance problem.
>
>
> It's not clear from http://bugs.debian.org/761157 that the performance
> problem occurs only with -P, but I assume that's what is meant.
>
> Since this is a performance bug with PCRE, I suggest moving the Debian bug
> report to the Debian libpcre3 package.  Grep cannot go back to the old way,
> which could cause grep to crash, and the bug cannot be fixed in grep because
> libpcre3 does not provide a fast way to search arbitrary data that may
> include encoding errors.  It really is a problem that requires changes to
> libpcre3 to fix; grep cannot fix it.
>
> In the meantime, in order to use 'grep' to search for strings in arbitrary
> data, I suggest omitting the '-P'.  Also, I suggest using the C locale.
>
> As the GNU bug 18266 "grep -P and invalid exits with error" has been fixed,
> I'm closing that bug report.  Please feel free to open a separate GNU bug
> report for the performance issue.
>
> PS.  While composing this email I noticed another bug in grep -P and
> encoding errors, which I fixed by installing the attached patch.

Thanks for fixing yet another bug, Paul.
Would you mind adding a test to trigger that one?


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#751704: libparted questions (was: Bug#751704: partman-base 173: partman overwrites parts of u-boot)

[Jim Meyering]

On Wed, Aug 13, 2014 at 12:07 PM, Karsten Merker  wrote:
> [CCing Otavio Salvador and Jim Meyering; the following is a short summary
>  of the situation; the full history can be read in bug #751704:
>
>  Debian-Installer uses partman for partitioning, which in turn is
>  based on libparted. On sunxi-based systems, upon writing the partition
>  table, partman/libparted overwrites parts of u-boot which are
>  located between the end of the partition table and the beginning of the
>  first partition which results in an unbootable system.
>  An attempt to solve this problem has been to conditionally set
>  PedDisk.needs_clobber to 0 in partman when it detects that it is
>  trying to write to a boot device on sunxi-based systems.]
>
> Colin Watson  wrote:
>> PedDisk.needs_clobber is marked as "office use only" in parted; I
>> interpret that as meaning that it really shouldn't be fiddled with
>> outside parted, even though it's technically exposed.  Could you please
>> look at fixing parted to avoid clobbering the firmware area instead?  I
>> think that would be more correct.
>
> I have no idea what is really meant with the comment
>
>   /* office use only ;-) */
>   int needs_clobber;
>
> in /usr/include/parted/disk.h, so I would like to ask upstream
> for clarification. Otavio, Jim: you are listed as parted
> upstream maintainers on http://www.gnu.org/software/parted/ - could
> you shed some light on this? Is it ok for an application to fiddle
> with the needs_clobber element or is this to be considered
> strictly libparted-internal?
>
>
> @Colin: If the solution is to be completely encapsulated in
> libparted, I see a large problem in how to solve the conflict between
> space after the partition table being very differently used by
> firmware for different SoCs on one side, and libparted trying to make
> sure that there are no remains of other partition table formats in
> the respective area on the other side - at least with the
> prerequisite of keeping both the current defaults (clobbering) as
> well as keeping the libparted API unchanged.  With the current "there
> is one erase size for all platforms" method in ped_disk_clobber() in
> libparted/disk.c, we inevitably end up with conflicting requirements.
> An example: the source for ped_disk_clobber() states:
>
> /* How many sectors to zero out at each end.
>This must be large enough to zero out the magic bytes
>starting at offset 8KiB on a DASD partition table.
>Doing the same from the end of the disk is probably
>overkill, but at least on GPT, we do need to zero out
>the final sector.  */
>
> So for at least one platform (according to Wikipedia DASD seems to be
> some s/390 format), the area at offset 8KiB has to be wiped out while
> for another (armhf/sunxi) it may not be wiped out as the u-boot SPL is
> located there and cannot be relocated because its sector address is
> hardcoded in the SoC.
>
> According to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=751704#25,
> similar problems (but at other offsets) come up for other SoCs.
> According to the examples listed there, for TI SoCs libparted would
> have to stop clobbering the GPT header after writing a DOS MBR, but
> could wipe the DASD area.  For a Freescale i.MX SoC libparted could
> legally clobber the GPT header, but would have to refrain from
> clobbering the areas behind it.  If you extrapolate this to the large
> number of different SoC families, to handle this completely inside
> libparted, the library would have to know what exactly is the target
> system for which it writes a partition table and special-case the
> handling for each of them.  While one might assume that the partition
> table is for an s/390 system when a DASD partition table is
> generated, this does not work as easily for the plethora of different
> architectures and systems that use DOS MBR partition tables.  This
> gets even more complicated by the fact that libparted may run on one
> platform but modify partition tables for another platform, such as
> when operating on disk images for use with emulators or when
> operating on hd-media images for different arm platforms (with
> different firmware/bootloader requirements) on one autobuild host, so
> trying to do runtime detection of the host system would still not cover
> all use cases. In the case of creating images from scratch, the problem
> can be circumvented by (re-)writing the bootloader at the end of the
> process, but when the task is to modify existing images with unknown
> content, this workaround is not available.
>
> As a conclusion, I think that the decision whether to clobbe

Bug#736919: bug#16586: grep: infinite loop in grep -P on some files with invalid UTF-8 sequences

[Jim Meyering]

On Tue, Apr 15, 2014 at 7:48 AM, Paul Eggert  wrote:
> Santiago wrote:
>> it was a debian-pcre-specific bug.
>
> Thanks, closing the bug upstream.

This bug is still present in upstream libpcre version 8.35.
I wrote a patch for it, posted at http://debbugs.gnu.org/17245#26
and Norihiro forwarded it on to the libpcre bug tracker here:
http://bugs.exim.org/show_bug.cgi?id=1468


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#739752: bug#17010: Bug#739752: coreutils: ln segfaults when run with --relative and an empty target

[Jim Meyering]

On Fri, Mar 14, 2014 at 9:27 AM, Jim Meyering  wrote:
> On Fri, Mar 14, 2014 at 4:49 AM, Pádraig Brady  wrote:
> ...
>> Hence since c_f_m() can validly fail even with CAN_MISSING,
>> I agree your patch is correct.
>>
>> Please push.
>
> Done.

Hmm... For the record, I also pushed (accidentally) two additional changes that
had been languishing in the affected repository.  Here they are:
From 42d2377b813507d50f5a67076bb20134ceb2fc10 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Mon, 5 Aug 2013 07:28:34 -0700
Subject: [PATCH 1/2] scripts: autotools-install: update

* scripts/autotools-install: Update version numbers of latest
automake and gettext packages.
---
 scripts/autotools-install | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/autotools-install b/scripts/autotools-install
index 403644f..c6a8e36 100755
--- a/scripts/autotools-install
+++ b/scripts/autotools-install
@@ -24,9 +24,9 @@ tarballs='
   http://pkgconfig.freedesktop.org/releases/pkg-config-0.28.tar.gz
   http://ftp.gnu.org/gnu/m4/m4-1.4.16.tar.gz
   http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz
-  http://ftp.gnu.org/gnu/automake/automake-1.12.6.tar.gz
+  http://ftp.gnu.org/gnu/automake/automake-1.14.tar.gz
   http://ftp.gnu.org/gnu/libtool/libtool-2.4.2.tar.gz
-  http://ftp.gnu.org/gnu/gettext/gettext-0.18.2.tar.gz
+  http://ftp.gnu.org/gnu/gettext/gettext-0.18.3.tar.gz
 '

 usage() {
-- 
1.9.0.167.g384364b


From 4f2118221777dc915f2df4e2ed3b65c68301 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Mon, 18 Nov 2013 10:13:06 -0800
Subject: [PATCH 2/2] maint: avoid "attribute-const"-suggesting warning from
 gcc

* gl/lib/fadvise.c: Use a pragma to turn off this warning option:
-Wsuggest-attribute=const.  Without this change, building with
--enable-gcc-warnings would evoke this error:

lib/fadvise.c:25:1: error: function might be candidate for\
attribute 'const' [-Werror=suggest-attribute=const]
---
 gl/lib/fadvise.c | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/gl/lib/fadvise.c b/gl/lib/fadvise.c
index 3456ce1..562f1eb 100644
--- a/gl/lib/fadvise.c
+++ b/gl/lib/fadvise.c
@@ -14,6 +14,12 @@
You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.  */

+/* Without this pragma, gcc suggests that (given !HAVE_POSIX_FADVISE)
+   the the fdadvise function might be candidate for attribute 'const'.  */
+#if (__GNUC__ == 4 && 6 <= __GNUC_MINOR__) || 4 < __GNUC__
+# pragma GCC diagnostic ignored "-Wsuggest-attribute=const"
+#endif
+
 #include 
 #include "fadvise.h"

-- 
1.9.0.167.g384364b

Bug#739752: bug#17010: Bug#739752: coreutils: ln segfaults when run with --relative and an empty target

[Jim Meyering]

On Fri, Mar 14, 2014 at 4:49 AM, Pádraig Brady  wrote:
...
> Hence since c_f_m() can validly fail even with CAN_MISSING,
> I agree your patch is correct.
>
> Please push.

Done.


--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#739752: bug#17010: Bug#739752: coreutils: ln segfaults when run with --relative and an empty target

[Jim Meyering]

On Thu, Mar 13, 2014 at 7:22 PM, Pádraig Brady  wrote:
> On 03/14/2014 01:42 AM, Jim Meyering wrote:
>> From a6d2db8b6dfe15344aba4aefe9545eb3a4876d45 Mon Sep 17 00:00:00 2001
>> From: Jim Meyering 
>> Date: Thu, 13 Mar 2014 17:05:04 -0700
>> Subject: [PATCH] ln: with -sr, don't segfault for a TARGET of ''
>>
>> Prior to this change, "ln -sr '' F" would segfault, attempting
>> to read path2[1] in relpath.c's path_common_prefix function.
>> This problem arises whenever canonicalize_filename_mode returns
>> NULL.
>> * src/ln.c (convert_abs_rel): Call relpath only when
>> both canonicalize_filename_mode calls return non-NULL.
>> * tests/ln/relative.sh: Add a test to trigger this failure.
>> * THANKS.in: List reporter's name/address.
>> * NEWS (Bug fixes): Mention it.
>> Reported by Erik Bernstein in 739...@bugs.debian.org.
>
> We can amend with the now allocated:
>
>   Fixes http://bugs.gnu.org/17010

Done.

>> diff --git a/NEWS b/NEWS
...
>> +  ln -sr '' F no longer segfaults: now it fails with the expected diagnostic
>
> Probably should add:
>
>  [bug introduced with the --relative feature in coreutils-8.16]

Definitely.  Thanks.

>> diff --git a/src/ln.c b/src/ln.c
>> index aab9cf2..6726699 100644
>> --- a/src/ln.c
>> +++ b/src/ln.c
>> @@ -149,13 +149,17 @@ convert_abs_rel (const char *from, const char *target)
>>char *realdest = canonicalize_filename_mode (targetdir, CAN_MISSING);
>>char *realfrom = canonicalize_filename_mode (from, CAN_MISSING);
>
> Interesting. So canonicalize_filename_mode() can fail in this case,
> even with CAN_MISSING. It's unexpected that c_f_m() sets errno=ENOENT
> when CAN_MISSING is set. I wonder should we change that instead
> in gnulib? With CAN_MISSING I would expect this function to work
> on arbitrary strings, including the empty string.

What would you have c_f_m("", CAN_MISSING) return?
I know of no absolute name corresponding to the dot-relative empty string.

>> diff --git a/tests/ln/relative.sh b/tests/ln/relative.sh
>> +# Expect this to fail with exit status 1.
>> +# Prior to coreutils-8.23, it would segfault.
>> +ln -sr '' F
>> +test $? = 1 || fail=1
>
> Won't the ln succeed on FreeBSD as per:
> http://bugs.gnu.org/13447

You're right.  Good catch.  Adjusted as well.
Thanks for the speedy and thorough review.
Here's a revised patch:
From 0093ac8d57a0f1a16fd09d98f6a524dddb6053e7 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Thu, 13 Mar 2014 17:05:04 -0700
Subject: [PATCH] ln: with -sr, don't segfault for a TARGET of ''

Prior to this change, "ln -sr '' F" would segfault, attempting
to read path2[1] in relpath.c's path_common_prefix function.
This problem arises whenever canonicalize_filename_mode returns
NULL.
* src/ln.c (convert_abs_rel): Call relpath only when
both canonicalize_filename_mode calls return non-NULL.
* tests/ln/relative.sh: Add a test to trigger this failure.
* THANKS.in: List reporter's name/address.
* NEWS (Bug fixes): Mention it.
Reported by Erik Bernstein in 739...@bugs.debian.org.
Fixes http://bugs.gnu.org/17010.
---
 NEWS |  3 +++
 THANKS.in|  1 +
 src/ln.c | 16 ++--
 tests/ln/relative.sh |  5 +
 4 files changed, 19 insertions(+), 6 deletions(-)

diff --git a/NEWS b/NEWS
index 62966b2..35d48e5 100644
--- a/NEWS
+++ b/NEWS
@@ -25,6 +25,9 @@ GNU coreutils NEWS-*- 
outline -*-
   it would display an error, requiring --no-dereference to avoid the issue.
   [bug introduced in coreutils-5.3.0]

+  ln -sr '' F no longer segfaults.  Now works as expected.
+  [bug introduced with the --relative feature in coreutils-8.16]
+
   shuf --repeat no longer dumps core if the input is empty.
   [bug introduced with the --repeat feature in coreutils-8.22]

diff --git a/THANKS.in b/THANKS.in
index 561d18c..2670265 100644
--- a/THANKS.in
+++ b/THANKS.in
@@ -193,6 +193,7 @@ Eric G. Miller  e...@jps.net
 Eric Pementepeme...@northpark.edu
 Eric S. Raymond e...@snark.thyrsus.com
 Erik Bennettbenn...@cvo.oneworld.com
+Erik Bernstein  e...@fscking.org
 Erik Corry  e...@kroete2.freinet.de
 Felix Lee   f...@teleport.com
 Felix Rauch Valenti fra...@cse.unsw.edu.au
diff --git a/src/ln.c b/src/ln.c
index aab9cf2..6726699 100644
--- a/src/ln.c
+++ b/src/ln.c
@@ -149,13 +149,17 @@ convert_abs_rel (const char *from, const char *target)
   char *realdest = canonicalize_filename_mode (targetdir, CAN_MISSING);
   char *realfrom = canoni

Bug#739752: coreutils: ln segfaults when run with --relative and an empty target

[Jim Meyering]

On Sat, Feb 22, 2014 at 1:57 AM, Erik Bernstein  wrote:
> Package: coreutils
> Version: 8.21-1
> Severity: normal
>
> Hi,
>
> when ln is run with --relative --symbolic and and empty string as the
> target, it ungracefully dies with a segmentation fault. The memory
> violation appears to happen in src/relpath.c:38 when the two input paths
> are checked for leading slashes:
>
>   if ((path1[1] == '/') != (path2[1] == '/'))
>
> How to reproduce:
> [1] Open a terminal
> [2] run: ln -sr '' foobar
>
> Result: segmentation fault  ln -sr '' foobar
> Expected result: Some kind of error message
...

Thank you for the bug report!
That also affected the very latest code in git.
Here is a patch:
From a6d2db8b6dfe15344aba4aefe9545eb3a4876d45 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Thu, 13 Mar 2014 17:05:04 -0700
Subject: [PATCH] ln: with -sr, don't segfault for a TARGET of ''

Prior to this change, "ln -sr '' F" would segfault, attempting
to read path2[1] in relpath.c's path_common_prefix function.
This problem arises whenever canonicalize_filename_mode returns
NULL.
* src/ln.c (convert_abs_rel): Call relpath only when
both canonicalize_filename_mode calls return non-NULL.
* tests/ln/relative.sh: Add a test to trigger this failure.
* THANKS.in: List reporter's name/address.
* NEWS (Bug fixes): Mention it.
Reported by Erik Bernstein in 739...@bugs.debian.org.
---
 NEWS |  2 ++
 THANKS.in|  1 +
 src/ln.c | 16 ++--
 tests/ln/relative.sh |  5 +
 4 files changed, 18 insertions(+), 6 deletions(-)

diff --git a/NEWS b/NEWS
index 62966b2..b3ad65c 100644
--- a/NEWS
+++ b/NEWS
@@ -25,6 +25,8 @@ GNU coreutils NEWS-*- 
outline -*-
   it would display an error, requiring --no-dereference to avoid the issue.
   [bug introduced in coreutils-5.3.0]

+  ln -sr '' F no longer segfaults: now it fails with the expected diagnostic
+
   shuf --repeat no longer dumps core if the input is empty.
   [bug introduced with the --repeat feature in coreutils-8.22]

diff --git a/THANKS.in b/THANKS.in
index 561d18c..2670265 100644
--- a/THANKS.in
+++ b/THANKS.in
@@ -193,6 +193,7 @@ Eric G. Miller  e...@jps.net
 Eric Pementepeme...@northpark.edu
 Eric S. Raymond e...@snark.thyrsus.com
 Erik Bennettbenn...@cvo.oneworld.com
+Erik Bernstein  e...@fscking.org
 Erik Corry  e...@kroete2.freinet.de
 Felix Lee   f...@teleport.com
 Felix Rauch Valenti fra...@cse.unsw.edu.au
diff --git a/src/ln.c b/src/ln.c
index aab9cf2..6726699 100644
--- a/src/ln.c
+++ b/src/ln.c
@@ -149,13 +149,17 @@ convert_abs_rel (const char *from, const char *target)
   char *realdest = canonicalize_filename_mode (targetdir, CAN_MISSING);
   char *realfrom = canonicalize_filename_mode (from, CAN_MISSING);

-  /* Write to a PATH_MAX buffer.  */
-  char *relative_from = xmalloc (PATH_MAX);
-
-  if (!relpath (realfrom, realdest, relative_from, PATH_MAX))
+  char *relative_from = NULL;
+  if (realdest && realfrom)
 {
-  free (relative_from);
-  relative_from = NULL;
+  /* Write to a PATH_MAX buffer.  */
+  relative_from = xmalloc (PATH_MAX);
+
+  if (!relpath (realfrom, realdest, relative_from, PATH_MAX))
+{
+  free (relative_from);
+  relative_from = NULL;
+}
 }

   free (targetdir);
diff --git a/tests/ln/relative.sh b/tests/ln/relative.sh
index 7636695..e8c25dc 100755
--- a/tests/ln/relative.sh
+++ b/tests/ln/relative.sh
@@ -45,4 +45,9 @@ mkdir web
 ln -sr latest web/latest
 test $(readlink web/latest) = '../release2' || fail=1

+# Expect this to fail with exit status 1.
+# Prior to coreutils-8.23, it would segfault.
+ln -sr '' F
+test $? = 1 || fail=1
+
 Exit $fail
-- 
1.9.0.167.g384364b

Bug#730472: bug#15758: grep 2.15 calls abort() on larger searches with -P

[Jim Meyering]

On Sat, Dec 21, 2013 at 11:01 AM, Jim Meyering  wrote:
> On Sat, Dec 21, 2013 at 10:46 AM, Jim Meyering  wrote:
>> I expect to push that patch as-is and defer to a separate commit
>> (or maybe even skip altogether) any portability hack that might warn
>> or disable PCRE support when detecting the broken library.
>
> Pushed.  Let's take any discussion of grep's workaround for Debian's
> PCRE problem to a new thread/issue.

Hmm... I was chagrined not to be able to reproduce the output I quoted
above, so dug into it and found the real error (mine), fixed it and
adjusted the test:
From 9311cf9c4f1e6a97c2e01e4a86f8f937c8010a01 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Tue, 31 Dec 2013 08:15:07 -0800
Subject: [PATCH] pcre: use PCRE_NO_UTF8_CHECK properly

In order to obtain the behavior we want, i.e., to disable
error-on-invalid-UTF-in-input, apply this PCRE option in
pcre_exec, not when compiling.
* src/pcresearch.c (Pexecute): Use PCRE_NO_UTF8_CHECK here, ...
(Pcompile): ...rather than here.
* tests/pcre-invalid-utf8-input: Adjust test case to test for this.
---
 src/pcresearch.c  | 12 
 tests/pcre-invalid-utf8-input | 11 ---
 2 files changed, 12 insertions(+), 11 deletions(-)

diff --git a/src/pcresearch.c b/src/pcresearch.c
index 664070d..4abd9c2 100644
--- a/src/pcresearch.c
+++ b/src/pcresearch.c
@@ -63,9 +63,9 @@ Pcompile (char const *pattern, size_t size)
 # if defined HAVE_LANGINFO_CODESET
   if (STREQ (nl_langinfo (CODESET), "UTF-8"))
 {
-  /* Enable PCRE's UTF-8 matching, but disable the check that would
- make an invalid byte seqence *in the input* trigger a failure.   */
-  flags |= PCRE_UTF8 | PCRE_NO_UTF8_CHECK;
+  /* Enable PCRE's UTF-8 matching.  Note also the use of
+ PCRE_NO_UTF8_CHECK when calling pcre_extra, below.   */
+  flags |= PCRE_UTF8;
 }
 # endif

@@ -158,6 +158,10 @@ Pexecute (char const *buf, size_t size, size_t *match_size,
e == PCRE_ERROR_NOMATCH && line_next < buf + size;
start_ofs -= line_next - line_buf)
 {
+  /* Disable the check that would make an invalid byte
+ seqence *in the input* trigger a failure.   */
+  int options = PCRE_NO_UTF8_CHECK;
+
   line_buf = line_next;
   line_end = memchr (line_buf, eolbyte, (buf + size) - line_buf);
   if (line_end == NULL)
@@ -172,7 +176,7 @@ Pexecute (char const *buf, size_t size, size_t *match_size,
 error (EXIT_TROUBLE, 0, _("exceeded PCRE's line length limit"));

   e = pcre_exec (cre, extra, line_buf, line_end - line_buf,
- start_ofs < 0 ? 0 : start_ofs, 0,
+ start_ofs < 0 ? 0 : start_ofs, options,
  sub, sizeof sub / sizeof *sub);
 }

diff --git a/tests/pcre-invalid-utf8-input b/tests/pcre-invalid-utf8-input
index 52a5432..c70951f 100755
--- a/tests/pcre-invalid-utf8-input
+++ b/tests/pcre-invalid-utf8-input
@@ -13,13 +13,10 @@ require_en_utf8_locale_

 fail=0

-printf '\202\n' > in || framework_failure_
-printf 'grep: invalid UTF-8 byte sequence in input\n' \
-   > exp-err || framework_failure_
+printf 'j\202\nj\n' > in || framework_failure_

-LC_ALL=en_US.UTF-8 grep -P anything in > out 2> err
-test $? = 2 || fail=1
-compare /dev/null out || fail=1
-compare exp-err err || fail=1
+LC_ALL=en_US.UTF-8 grep -P j in > out 2>&1 || fail=1
+compare in out || fail=1
+compare /dev/null err || fail=1

 Exit $fail
-- 
1.8.5.rc2.6.gc6f1b92

Bug#730472: bug#15758: grep 2.15 calls abort() on larger searches with -P

[Jim Meyering]

On Sat, Dec 21, 2013 at 10:46 AM, Jim Meyering  wrote:
> I expect to push that patch as-is and defer to a separate commit
> (or maybe even skip altogether) any portability hack that might warn
> or disable PCRE support when detecting the broken library.

Pushed.  Let's take any discussion of grep's workaround for Debian's
PCRE problem to a new thread/issue.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#730472: bug#15758: grep 2.15 calls abort() on larger searches with -P

[Jim Meyering]

On Thu, Dec 19, 2013 at 10:34 AM, Jim Meyering  wrote:
> I have confirmed that grep linked with libpcre.a built from upstream
> sources [commit f9d3a72ea5e86a674a9836b462e1231ecce0d739] (8.34) also
> works the way I expect.

More data points: Fedora 20 and OS/X work both with pcre-8.33, so I
conclude this is a problem specific to some Debian-specific patch.

I expect to push that patch as-is and defer to a separate commit
(or maybe even skip altogether) any portability hack that might warn
or disable PCRE support when detecting the broken library.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#730472: bug#15758: grep 2.15 calls abort() on larger searches with -P

[Jim Meyering]

On Wed, Dec 18, 2013 at 3:09 PM, Santiago  wrote:
> El 18/12/13 a las 09:45, Jim Meyering escribió:
...
>>   printf 'j\x82\nj\n'|LC_ALL=en_US.UTF-8 grep -P j|cat -A; echo $?
>>
>> For me (using pcre-8.33), it works the way I want and both matches:
>>
>>   jM-^B$
>>   j$
>>   0
>>
>> Hmm... I see that with debian unstable's 8.31-2, it does indeed act 
>> differently.
>> I may have to think about excluding pcre support when the version
>> doesn't work the way I want.
>
> I get this:
>
> $ printf 'j\x82\nj\n'|LC_ALL=en_US.UTF-8 src/grep -P j|cat -A; echo $?
> src/grep: invalid UTF-8 byte sequence in input
> 0
>
> I've also tried building debian packages for pcre 8.33 and 8.34, with same
> results. I need to take a look if a debian patch is giving trouble.

I have confirmed that grep linked with libpcre.a built from upstream
sources [commit f9d3a72ea5e86a674a9836b462e1231ecce0d739] (8.34) also
works way I expect.


--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#730472: bug#15758: grep 2.15 calls abort() on larger searches with -P

[Jim Meyering]

On Wed, Dec 18, 2013 at 8:53 AM, Santiago  wrote:
...
> $ src/grep -Pr "DEFINE" /usr/lib/linux-kbuild-3.2/
> src/grep: invalid UTF-8 byte sequence in input
>
> When I'd expected something like:
>
> $ LC_ALL=C src/grep -Pr "DEFINE" /usr/lib/linux-kbuild-3.2/
> /usr/lib/linux-kbuild-3.2/scripts/kernel-doc:   if ($prototype =~ 
> m/DEFINE_SINGLE_EVENT\((.*?),/) {
> /usr/lib/linux-kbuild-3.2/scripts/kernel-doc:   if ($prototype =~ 
> m/DEFINE_EVENT\((.*?),(.*?),/) {
> /usr/lib/linux-kbuild-3.2/scripts/kernel-doc:## if ($prototype =~ 
> m/SYSCALL_DEFINE0\s*\(\s*(a-zA-Z0-9_)*\s*\)/) {
> /usr/lib/linux-kbuild-3.2/scripts/kernel-doc:   if ($prototype =~ 
> m/SYSCALL_DEFINE0/) {
> ...
>
> Maybe, it is a pcre (v. 8.31) issue.

Hi Santiago,
Thanks for testing that.
What do you get when you run the stand-alone example I gave in the
commit log and in the test?

  printf 'j\x82\nj\n'|LC_ALL=en_US.UTF-8 grep -P j|cat -A; echo $?

For me (using pcre-8.33), it works the way I want and both matches:

  jM-^B$
  j$
  0

Hmm... I see that with debian unstable's 8.31-2, it does indeed act differently.
I may have to think about excluding pcre support when the version
doesn't work the way I want.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#730472: bug#15758: grep 2.15 calls abort() on larger searches with -P

[Jim Meyering]

On Fri, Dec 13, 2013 at 10:33 AM, Jim Meyering  wrote:
...
> Thanks for the suggested patches and report.  Your first patch is
> almost right.  The problem is that we cannot remove the PCRE_UTF8 flag.
> If we did that, it would disable UTF-8, reverting an older fix.
> See tests/pcre-utf8 for examples, or run this:
>
>   printf '\342\202\254\n' | LC_ALL=en_US.UTF-8 src/grep -P '^\p{S}'
>
> I've added a commit log, improved a related test and attached
> a slightly different patch, but left you as the "Author".
> I'll wait for an explicit ACK before pushing it.
>
> With that, there is no need to handle PCRE_ERROR_BADUTF8
> because that should not happen.

Patch attached, this time.
Thanks to Eric Blake for the quick off-list prod :-)
From 25b665c0eb04c8fb68034cc7db1ceea08e625b5a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Santiago=20Ruano=20Rinc=C3=B3n?= 
Date: Fri, 13 Dec 2013 07:53:37 -0800
Subject: [PATCH] PCRE: tell grep -P to relax its stance on invalid multibyte
 chars
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Do not exit-2 for invalid UTF-8 characters.  Just prior to this
change, this command would match no lines and fail like this:
  $ printf 'j\x82\nj\n'|LC_ALL=en_US.UTF-8 grep -P j|cat -A; echo $?
  src/grep: invalid UTF-8 byte sequence in input
  2
After this change, the same command matches both lines, and succeeds:
  jM-^B$
  j$
  0
* src/pcresearch.c (Pcompile): Use PCRE_NO_UTF8_CHECK, too, and
add a comment.
* tests/pcre-utf8: Add a test and a comment.
Based on a patch by Santiago Ruano Rincón.
See http://bugs.gnu.org/15758/
---
 src/pcresearch.c | 6 +-
 tests/pcre-utf8  | 6 ++
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/src/pcresearch.c b/src/pcresearch.c
index 9ba1227..43988c6 100644
--- a/src/pcresearch.c
+++ b/src/pcresearch.c
@@ -62,7 +62,11 @@ Pcompile (char const *pattern, size_t size)

 #if defined HAVE_LANGINFO_CODESET
   if (STREQ (nl_langinfo (CODESET), "UTF-8"))
-flags |= PCRE_UTF8;
+{
+  /* Enable PCRE's UTF-8 matching, but disable the check that would
+ make an invalid byte seqence *in the input* trigger a failure.   */
+  flags |= PCRE_UTF8 | PCRE_NO_UTF8_CHECK;
+}
 #endif

   /* FIXME: Remove these restrictions.  */
diff --git a/tests/pcre-utf8 b/tests/pcre-utf8
index b8228d5..a3b9390 100755
--- a/tests/pcre-utf8
+++ b/tests/pcre-utf8
@@ -19,9 +19,15 @@ echo '$' | LC_ALL=en_US.UTF-8 grep -qP '\p{S}' \
 euro='\342\202\254 euro'
 printf "$euro\\n" > in || framework_failure_

+# The euro sign has the unicode "Symbol" property, so this must match:
 LC_ALL=en_US.UTF-8 grep -P '^\p{S}' in > out || fail=1
 compare in out || fail=1

+# This RE must *not* match in the C locale, because the first
+# byte is not a "Symbol".
+LC_ALL=C grep -P '^\p{S}' in > out && fail=1
+compare /dev/null out || fail=1
+
 LC_ALL=en_US.UTF-8 grep -P '^. euro$' in > out2 || fail=1
 compare in out2 || fail=1

-- 
1.8.5.rc2.6.gc6f1b92

Bug#730472: bug#15758: grep 2.15 calls abort() on larger searches with -P

[Jim Meyering]

On Tue, Nov 26, 2013 at 6:30 AM, Santiago  wrote:
> This bug was also reported in Debian ( http://bugs.debian.org/730472 ).
>
> Taking a look on it, I think the most suitable solution for the moment
> is to flag PCRE_NO_UTF8_CHECK instead of PCRE_UTF8, so
> PCRE does not check if inputs are UTF8 valid. Resulting behavior is
> similar to pre-grep-2.15. (See 15758-PCRE-no-check-UTF8.patch)

Thanks for the suggested patches and report.  Your first patch is
almost right.  The problem is that we cannot remove the PCRE_UTF8 flag.
If we did that, it would disable UTF-8, reverting an older fix.
See tests/pcre-utf8 for examples, or run this:

  printf '\342\202\254\n' | LC_ALL=en_US.UTF-8 src/grep -P '^\p{S}'

I've added a commit log, improved a related test and attached
a slightly different patch, but left you as the "Author".
I'll wait for an explicit ACK before pushing it.

With that, there is no need to handle PCRE_ERROR_BADUTF8
because that should not happen.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#684713: support for partitioned linux md devices

[Jim Meyering]

Miquel van Smoorenburg wrote:
> I've also filed this as a debian bugreport,
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=684713
>
> Linux md raid array devices come in two flavours: partionable
> (/dev/md_d0) and non-partitionable (/dev/md0). Or at least,
> that used to be the case, until kernel 2.6.28 where the two have
> been consolidated. Now all md devices can have partitions.
>
> However, there is one minor oversight/bug in the kernel: the
> sysfs "range" key is still set to "1" for md devices. That means
> libparted thinks that it's not possible to partition that device,
> when in fact it is.
>
> The attached patch reckognizes that situation: if running on
> a kernel >= 2.6.28, and the device is a PED_DEVICE_MD, and the
> sysfs 'range' key is set to '1', _device_get_partition_range()
> returns MAX_NUM_PARTS instead.
>
> Mike.
>
> Index: parted-2.3/libparted/arch/linux.c
> ===
> --- parted-2.3.orig/libparted/arch/linux.c2010-05-10 10:57:54.0 
> +
> +++ parted-2.3/libparted/arch/linux.c 2012-08-05 13:24:14.449768577 +
> @@ -2415,6 +2415,11 @@
>  ok = fscanf(fp, "%d", &range) == 1;
>  fclose(fp);
>
> + /* starting at 2.6.28 partitions are OK but "range" doesn't show it */
> + if (dev->type == PED_DEVICE_MD && range == 1 &&
> + _get_linux_version() >= KERNEL_VERSION (2,6,28))
> + ok = 0;
> +
>  /* (range <= 0) is none sense.*/
>  return ok && range > 0 ? range : MAX_NUM_PARTS;
>  }

Thanks for the patch.
FYI, Petr Uzel made the following change in v3.0-61-gca97da9:


http://anonscm.debian.org/gitweb/?p=parted/parted.git;a=commitdiff;h=ca97da905bd21f2a4371f4717f7c46a936af6b2c

commit ca97da905bd21f2a4371f4717f7c46a936af6b2c
Author: Petr Uzel 
Date:   Sat Nov 26 15:45:08 2011 +0100

libparted: use ext_range to find out largest possible partition

Parted uses /sys/block/DEV/range file to find out how many partitions
can the blockdevice hold and uses this number in its algorithm
for informing the kernel about modified partitions. This works
fine for most devices, however, it fails on partitionable MD arrays,
because these have 1 in range file. Using ext_range should be safer
and work for all devices.

* libparted/arch/linux.c (_device_get_partition_range): Use
/sys/block/DEV/ext_range instead of range sysfs file
* NEWS: Mention the change.

Addresses: http://bugzilla.novell.com/567652

diff --git a/NEWS b/NEWS
index 566484c..5d8df35 100644
--- a/NEWS
+++ b/NEWS
@@ -38,6 +38,10 @@ GNU parted NEWS-*- 
outline -*-
   with an HFS or HFS+ signature, but with invalid ->total_blocks and/or
   ->block_size values.

+  parted now uses ext_range device sysfs attribute to determine maximum number
+  of partitions the device can hold.  With this change, parted now correctly
+  informs kernel about new partitions on partitionable MD RAID devices.
+
 ** Changes in behavior

   parted: mkpart command has changed semantics with regard to specifying end
diff --git a/libparted/arch/linux.c b/libparted/arch/linux.c
index ab3d904..1da3343 100644
--- a/libparted/arch/linux.c
+++ b/libparted/arch/linux.c
@@ -2484,7 +2484,7 @@ _loop_get_partition_range(PedDevice const* dev)

 /*
  * The number of partitions that a device can have depends on the kernel.
- * If we don't find this value in /sys/block/DEV/range, we will use our own
+ * If we don't find this value in /sys/block/DEV/ext_range, we will use our own
  * value.
  */
 static unsigned int
@@ -2495,7 +2495,7 @@ _device_get_partition_range(PedDevice const* dev)
 return _loop_get_partition_range(dev);

 int range;
-bool ok = _sysfs_int_entry_from_dev(dev, "range", &range);
+bool ok = _sysfs_int_entry_from_dev(dev, "ext_range", &range);

 return ok && range > 0 ? range : MAX_NUM_PARTS;
 }


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#563254: du bogus warning with bind mounts

[Jim Meyering]

[Re: http://bugs.gnu.org/11844
 du: continue processing after bind-mount induced dir-cycle ]

Ondrej Oprala wrote:
> Sure, that looks great. Thanks for the help.

I added this to the log:

* THANKS.in: Update.
This implements the proposal in http://bugs.gnu.org/11844.
Originally reported in http://bugs.debian.org/563254 by Alan Jenkins
and more recently as http://bugzilla.redhat.com/836557

and pushed:
http://git.sv.gnu.org/cgit/coreutils.git/commit/?id=cf7e1b5b8fb53aef2a9103a32


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#682969: timeout 20 /usr/sbin/ntpd -g -q gives wrong return code

[Jim Meyering]

Pádraig Brady wrote:
> On 07/27/2012 05:56 PM, Jim Meyering wrote:
...
>> When I try to trigger such misbehavior, timeout seems to work fine
>> on a system (Fedora 17) with a working timer_settime function:
>>
>> $ timeout 2 date --set=$(LC_ALL=C date -d 10\ sec +@%s); echo $?
>> 0
>>
>> I.e., when timeout's child terminates, the clock says
>> 10 seconds have elapsed, yet it exits 0, because the
>> duration was less than 2 seconds.
>>
>> I'm using the timeout from coreutils-8.17, but nothing
>> has changed in this area since before 8.13.
>>
>> I even rebuilt my timeout binary, simulating no timer_settime
>> function so that it would take the alarm-using path.  Same result.
>
> Note timeout(1) currently uses timer_create(CLOCK_REALTIME).
> That could jump and cause signals to fire.
> Though I can't reproduce here, even when pushing the
> updated system clock down to the hardware:
>
> # timeout 3 sh -c 'date --set=$(LC_ALL=C date -d 10\ sec +@%s);
> hwclock -w; sleep 1'
> # echo $?
> 0
>
> I was wondering about using CLOCK_MONOTONIC instead,
> though I'd need to test how that functions over a suspend/resume.
> I suspect it only counts up while the system is running.
> Maybe we should default to system running time, rather than
> wall clock time, though then we'd have to look at/document
> the inconsistency on systems without CLOCK_MONOTONIC.

Yes, using CLOCK_MONOTONIC looked tempting to me, too, but finding
no problem with the use of CLOCK_REALTIME, I was leery of switching.


--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#682969: timeout 20 /usr/sbin/ntpd -g -q gives wrong return code

[Jim Meyering]

Goswin von Brederlow wrote:
> Package: coreutils
> Version: 8.13-3.2
> Severity: normal
> File: /usr/bin/timeout
>
> I'm trying to set the time during boot. Unfortunately ntpd hangs forever
> if the timeserver is unavailable. So I added a timeout to it so the
> system still continues to boot without the correct time.
>
> But I would like to log the error. Timeout is supposed to return 124 if the
> time was exceeded. But if ntpd corrects the time by more than the timeout
> then the return value is 124 despite the fact the real time passed was less.
>
> Wouldn't it be possible to detect this case and return the exit code of
> ntpd instead of a false timeout error?

Thanks for the report.
Did you experience an actual problem?

When I try to trigger such misbehavior, timeout seems to work fine
on a system (Fedora 17) with a working timer_settime function:

$ timeout 2 date --set=$(LC_ALL=C date -d 10\ sec +@%s); echo $?
0

I.e., when timeout's child terminates, the clock says
10 seconds have elapsed, yet it exits 0, because the
duration was less than 2 seconds.

I'm using the timeout from coreutils-8.17, but nothing
has changed in this area since before 8.13.

I even rebuilt my timeout binary, simulating no timer_settime
function so that it would take the alarm-using path.  Same result.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#679708: libc-client2007e: uw-imap c-client library version skew, app=2007e library=2007f

[Jim Meyering]

Jonas Smedegaard wrote:
...
> uw-imapd has been dropped from Debian.  I recommend switching to
> Dovecot.
>
> libc-client* and uw-imapd was part of same source package, but if for a
> moment we imagine they weren't, then the solution to this bug would be
> to recompile uw-imapd against up-to-date libc-client. So the bug does
> not belong to libc-client but to uw-mapd - which is no more.

Thanks for the tip.
I've switched to dovecot.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#679708: libc-client2007e: uw-imap c-client library version skew, app=2007e library=2007f

[Jim Meyering]

Package: libc-client2007e
Version: 8:2007f~dfsg-1
Severity: normal

   * What led up to the situation?

I use uw-imapd and updated to today's libc-client2007e,
and now uw-imapd (8:2007e~dfsg) doesn't work at all, in that
I get these in the logs:

Jun 30 21:20:23 *** imapd: IMAP toolkit crash: c-client library version 
skew, app=2007e library=2007f
Jun 30 21:22:27 *** imapd: Fatal error user=??? host= [] mbx=???: 
c-client library version skew, app=2007e library=2007f

Since this is due to version skew between a package and its
dependent library, I suppose the ball might technically be in
uw-imapd's court.


-- System Information:
Debian Release: wheezy/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.2.0-3-amd64 (SMP w/4 CPU cores)
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968) (ignored: LC_ALL set to C)
Shell: /bin/sh linked to /bin/dash

Versions of packages libc-client2007e depends on:
ii  libc6 2.13-34
ii  libcomerr21.42.4-3
ii  libgssapi-krb5-2  1.10.1+dfsg-1
ii  libk5crypto3  1.10.1+dfsg-1
ii  libkrb5-3 1.10.1+dfsg-1
ii  libpam-modules1.1.3-7.1
ii  libpam0g  1.1.3-7.1
ii  libssl1.0.0   1.0.1c-3
ii  mlock 8:2007f~dfsg-1

libc-client2007e recommends no packages.

Versions of packages libc-client2007e suggests:
pn  uw-mailutils  

-- no debconf information



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#582083: patch for grep --color to non-tty output

[Jim Meyering]

Tim Connors wrote:
> On Tue, 24 Apr 2012, Jim Meyering wrote:
>
>> Aníbal Monsalve Salazar wrote:
>> > Debian bug report is posted at:
>> >
>> > http://bugs.debian.org/582083
>> ...
>> >>There's no reason not to obey the user when they ask for "--color",
>> >>regardless of whether the output is to a tty or not.  They wouldn't
>> >>have asked for --color if they didn't want it, and most other gnu
>> >>programs assume --color=yes rather than --color=auto when supplied with
>> >>just --color.  Man and info pages and translations appear to need to
>> >>work since they don't imply what the default is.  Nice easy patch to
>> >>apply!
>> >>
>> >>[1] New version looks like:
>> >>diff -ru grep-2.10//src/main.c /tmp/grep-2.10//src/main.c
>> >>--- grep-2.10//src/main.c   2012-04-24 13:11:57.0 +1000
>> >>+++ /tmp/grep-2.10//src/main.c  2012-04-24 12:56:47.0 +1000
>> >>@@ -2059,7 +2059,7 @@
>> >>   else
>> >> show_help = 1;
>> >> } else
>> >>-  color_option = 2;
>> >>+  color_option = 1;
>> >> if (color_option == 2)
>> >>   {
>> >> char const *t;
>>
>> Thanks for the report of the documentation bug and the patch, but the patch
>> (changing the meaning of --color from --color=auto to --color=always)
>> would break existing usage.
>>
>> Currently, people can use --color in an always-on alias/function
>> or set the GREP_OPTIONS=--color envvar and get colorized output,
>> yet not have those ANSI terminal highlighting bytes interfere
>> with output that is not to a tty.
>>
>> If we were to make your proposed change, they'd find those
>> color codes in unexpected (and undesirable) places.
>
> Easy to fix!  Fix the bug in their login scripts!

If it were only in dot-file alias/function definitions,
it'd be an easier call.  Are there uses of grep --color
in other contexts?  Probably.

>> However, this is definitely a documentation bug, and I'd
>> appreciate a patch for both --help and grep.texi.
>>
>> Jim
>>
>> PS.  True, it is undesirable to have grep's --color(with no value)
>> default to "auto", when in ls it defaults to "always", but changing
>> grep's default now would be too disruptive.  We'd have to warn that
>> the default is going to change for a year or two before making the
>> actual change, and even then, some users would be impacted.

If you know me, you know that the above is just this humble
maintainer's opinion.  If enough people chime in saying that they
think making grep --color align with ls --color behavior is important,
and that they think the risk to existing users is low enough,
I'll be happy to reconsider.

> It doesn't take much to change one's .rc files to say
> GREP_OPTIONS="--color=auto" rather than "--color"!  They should have been
> doing that all along (because they were relying on undocumented behaviour
> :).
>
> I personally have "GREP_OPTIONS=--color=auto".  If the output is a tty,
> that works as expected.  If the output is a pipe, no color as expected;
> all good.  If the output is "less -R", I want --color, so I say
> "echo test | grep --color es", and I don't get color.  That's not what I
> asked for!



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#582083: patch for grep --color to non-tty output

[Jim Meyering]

Aníbal Monsalve Salazar wrote:
> Debian bug report is posted at:
>
> http://bugs.debian.org/582083
...
>>There's no reason not to obey the user when they ask for "--color",
>>regardless of whether the output is to a tty or not.  They wouldn't
>>have asked for --color if they didn't want it, and most other gnu
>>programs assume --color=yes rather than --color=auto when supplied with
>>just --color.  Man and info pages and translations appear to need to
>>work since they don't imply what the default is.  Nice easy patch to
>>apply!
>>
>>[1] New version looks like:
>>diff -ru grep-2.10//src/main.c /tmp/grep-2.10//src/main.c
>>--- grep-2.10//src/main.c   2012-04-24 13:11:57.0 +1000
>>+++ /tmp/grep-2.10//src/main.c  2012-04-24 12:56:47.0 +1000
>>@@ -2059,7 +2059,7 @@
>>   else
>> show_help = 1;
>> } else
>>-  color_option = 2;
>>+  color_option = 1;
>> if (color_option == 2)
>>   {
>> char const *t;

Thanks for the report of the documentation bug and the patch, but the patch
(changing the meaning of --color from --color=auto to --color=always)
would break existing usage.

Currently, people can use --color in an always-on alias/function
or set the GREP_OPTIONS=--color envvar and get colorized output,
yet not have those ANSI terminal highlighting bytes interfere
with output that is not to a tty.

If we were to make your proposed change, they'd find those
color codes in unexpected (and undesirable) places.

However, this is definitely a documentation bug, and I'd
appreciate a patch for both --help and grep.texi.

Jim

PS.  True, it is undesirable to have grep's --color(with no value)
default to "auto", when in ls it defaults to "always", but changing
grep's default now would be too disruptive.  We'd have to warn that
the default is going to change for a year or two before making the
actual change, and even then, some users would be impacted.



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#669555: coreutils: FTBFS: env: setfacl: No such file or directory

[Jim Meyering]

Lucas Nussbaum wrote:
> Source: coreutils
> Version: 8.13-3.1
> Severity: serious
> Tags: wheezy sid
> User: debian...@lists.debian.org
> Usertags: qa-ftbfs-20120419 qa-ftbfs
> Justification: FTBFS on amd64
>
> Hi,
>
> During a rebuild of all packages in sid, your package failed to build on
> amd64.
>
> Relevant part:

Hi Lucas,

Thanks for the report.
The failure is unrelated to env/setfacl.
In general, you need to search for "FAIL:" and (within that block) "fail=1":

This test is ensuring that sort works even when it requires every single
available file descriptor.  If something about sort or one of the programs
it execs happens to keep just one extra file descriptor open, then this
test will fail.

The following upstream fix to the failing test script,
v8.14-38-g91a5bad may solve this problem:

commit 91a5badc7b8b96916147f28b1d094af98efa5aa7
Author: Paul Eggert 
Date:   Sat Nov 12 00:20:01 2011 -0800

* tests/misc/sort-continue: Port to Fedora 15.

Redirect with the shell command, not in a separate 'exec'.
Without this patch, Fedora 15 x86-64 /bin/sh (i.e., Bash 4.2.10)
complained about running out of file descriptors in the shell.


...
>> FAIL: misc/sort-continue (exit: 1)
>> ==
...
>> + tee -a in
>> + echo 31
>> + ulimit -n 6
>> + exec
>> ./misc/sort-continue: redirection error: cannot duplicate fd: Invalid 
>> argument
>> ./misc/sort-continue: line 33: /dev/null: Invalid argument
>> + sort -n -m __test.1 __test.10 __test.11 __test.12 __test.13
> __test.14 __test.15 __test.16 __test.17 __test.18 __test.19 __test.2
> __test.20 __test.21 __test.22 __test.23 __test.24 __test.25 __test.26
> __test.27 __test.28 __test.29 __test.3 __test.30 __test.31 __test.4
> __test.5 __test.6 __test.7 __test.8 __test.9
>> sort: open failed: __test.10: Too many open files
>> + fail=1
>> + echo 'file descriptor exhaustion not handled'
>> file descriptor exhaustion not handled
>> + tee -a in
>> + echo 32
>> + ulimit -n 6
>> + exec
>> ./misc/sort-continue: redirection error: cannot duplicate fd: Invalid 
>> argument
>> ./misc/sort-continue: line 42: 0: Invalid argument
>> + sort -n -m __test.1 __test.10 __test.11 __test.12 __test.13
> __test.14 __test.15 __test.16 __test.17 __test.18 __test.19 __test.2
> __test.20 __test.21 __test.22 __test.23 __test.24 __test.25 __test.26
> __test.27 __test.28 __test.29 __test.3 __test.30 __test.31 __test.4
> __test.5 __test.6 __test.7 __test.8 __test.9 -
>> + compare in out
>> + diff -u in out
>> + Exit 1
>> + set +e
>> + exit 1
>> + exit 1
>> + remove_tmp_
>> + __st=1
>> + cleanup_
>> + :
>> + cd /«PKGBUILDDIR»/tests
>> + chmod -R u+rwx /«PKGBUILDDIR»/tests/gt-sort-continue.HX7W
>> + rm -rf /«PKGBUILDDIR»/tests/gt-sort-continue.HX7W
>> + exit 1



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#669084: grep does not match anything when reading from stdin

[Jim Meyering]

Tino Keitel wrote:
> On Tue, Apr 17, 2012 at 14:23:48 +0200, Tino Keitel wrote:
>
>> Hi,
>>
>> I had problems extracting the patch from the mail, so I had to use
>> copy&paste.  Also, the patch portion in main.c looked totally different
>
> Forgot the patch, it's attached.
>
> Regards,
> Tino
>
> --- grep-2.11_orig/src/main.c 2012-04-17 10:08:47.626634496 +0200
> +++ grep-2.11/src/main.c  2012-04-17 14:17:13.293392207 +0200
> @@ -1251,11 +1251,12 @@
>return 1;
>  }
>
> -  if ((directories == SKIP_DIRECTORIES && S_ISDIR (stats->stat.st_mode))
> -  || (devices == SKIP_DEVICES && (S_ISCHR (stats->stat.st_mode)
> +  if (desc != STDIN_FILENO &&
> +  ((directories == SKIP_DIRECTORIES && S_ISDIR (stats->stat.st_mode))
> +   || (devices == SKIP_DEVICES && (S_ISCHR (stats->stat.st_mode)
>|| S_ISBLK (stats->stat.st_mode)
>|| S_ISSOCK (stats->stat.st_mode)
> -  || S_ISFIFO (stats->stat.st_mode
> +  || S_ISFIFO (stats->stat.st_mode)

Thanks.  Your change is relative to 2.11, while mine is
relative to the latest in the upstream git repository:

  http://git.savannah.gnu.org/cgit/grep.git/



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#669084: grep does not match anything when reading from stdin

[Jim Meyering]

Aníbal Monsalve Salazar wrote:
...
>>>Tino suggests that the possible culprit is the code above.
>>>
>>>The Debian bug report is at http://bugs.debian.org/668585
>
> Typo. The correct address is http://bugs.debian.org/669084

Thanks!
I've adjusted the log:

  * NEWS (Bug fixes): Mention it, and add a few "[fixed in ...] notes.
  Reported by Tino Keitel in http://bugs.debian.org/669084,
  and forwarded to bug-grep by Aníbal Monsalve Salazar.



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#669084: grep does not match anything when reading from stdin

[Jim Meyering]

Aníbal Monsalve Salazar wrote:
...
> Hello Jim,
>
> grep-2.11 introduced a regression related to the "-D skip" parameter
> option.
>
> With grep-2.10 (or less than 2.10):
>
> $ echo foo | grep -D skip foo; echo $?
> foo
> 0
>
> With grep-2.11:
>
> $ echo foo | grep -D skip foo; echo $?
> 1
>
> Timo suggests that the possible culprit is the code above.
>
> The Debian bug report is at http://bugs.debian.org/668585

Hi Aníbal

Thanks for passing that along.
Here's a lightly-tested patch:

>From 415e4e69c8e8c0db288205b30ab2b6a337f62d38 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Tue, 17 Apr 2012 13:37:10 +0200
Subject: [PATCH] grep: --devices=ACTION (-D) no longer affects stdin
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* doc/grep.texi (File and Directory Selection): Clarify this point,
documenting the stdin exemption.
* src/main.c (grepdesc): Ignore skip-related options when reading
from standard input.
* tests/skip-device: New file.  Test for the above.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Bug fixes): Mention it, and add a few "[fixed in ...] notes.
Reported by Sven Joachim in  http://bugs.debian.org/668585,
and forwarded to bug-grep by Aníbal Monsalve Salazar.
---
 NEWS  |7 ++-
 doc/grep.texi |1 +
 src/main.c|9 +
 tests/Makefile.am |1 +
 tests/skip-device |   11 +++
 5 files changed, 24 insertions(+), 5 deletions(-)
 create mode 100755 tests/skip-device

diff --git a/NEWS b/NEWS
index b219b65..a065394 100644
--- a/NEWS
+++ b/NEWS
@@ -4,13 +4,17 @@ GNU grep NEWS-*- outline 
-*-

 ** Bug fixes

+  "echo P|grep --devices=skip P" once again prints P, as it did in 2.10
+  [bug introduced in grep-2.11]
+
   grep no longer segfaults with -r --exclude-dir and no file operand.
   I.e., ":|grep -r --exclude-dir=D PAT" would segfault.
+  [bug introduced in grep-2.11]

   Recursive grep now uses fts for directory traversal, so it can
   handle much-larger directories without reporting things like "File
   name too long", and it can run much faster when dealing with large
-  directory hierarchies.
+  directory hierarchies. [bug present since the beginning]

   grep -E 'a{10}' now reports an overflow error rather than
   silently acting like grep -E 'a\{10}'.
@@ -27,6 +31,7 @@ GNU grep NEWS-*- outline 
-*-
   use -R if you prefer the old behavior of following all symlinks and
   defaulting to reading all devices.

+
 * Noteworthy changes in release 2.11 (2012-03-02) [stable]

 ** Bug fixes
diff --git a/doc/grep.texi b/doc/grep.texi
index 1840e21..000a844 100644
--- a/doc/grep.texi
+++ b/doc/grep.texi
@@ -614,6 +614,7 @@ By default, devices are read if they are on the command 
line or if the
 @option{-R} (@option{--dereference-recursive}) option is used, and are
 skipped if they are encountered recursively and the @option{-r}
 (@option{--recursive}) option is used.
+This option has no effect on a file that is read via standard input.

 @item -d @var{action}
 @itemx --directories=@var{action}
diff --git a/src/main.c b/src/main.c
index 82cae33..c5a8489 100644
--- a/src/main.c
+++ b/src/main.c
@@ -1367,10 +1367,11 @@ grepdesc (int desc, int command_line)
 suppressible_error (filename, errno);
   return status;
 }
-  if ((directories == SKIP_DIRECTORIES && S_ISDIR (st.st_mode))
-  || ((devices == SKIP_DEVICES
-   || (devices == READ_COMMAND_LINE_DEVICES && !command_line))
-  && is_device_mode (st.st_mode)))
+  if (desc != STDIN_FILENO
+  && ((directories == SKIP_DIRECTORIES && S_ISDIR (st.st_mode))
+  || ((devices == SKIP_DEVICES
+   || (devices == READ_COMMAND_LINE_DEVICES && !command_line))
+  && is_device_mode (st.st_mode
 goto closeout;

   /* If there is a regular file on stdout and the current file refers
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 13061fe..d0d622b 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -78,6 +78,7 @@ TESTS =   \
   r-dot\
   repetition-overflow  \
   reversed-range-endpoints \
+  skip-device  \
   sjis-mb  \
   spencer1 \
   spencer1-locale  \
diff --git a/tests/skip-device b/tests/skip-device
new file mode 100755
index 000..efb529f
--- /dev/null
+++ b/tests/skip-device
@@ -0,0 +1,11 @@
+#!/bin/sh
+# grep must ignore --devices=ACTION (-D) when reading stdin
+# For grep-2.11, this test would fail.
+
+. "${srcdir=.}/init.sh"

Bug#662723: ls: -F (--indicator-style=classify) affects on symbolic link dereference

[Jim Meyering]

sergio wrote:
> On 03/06/2012 05:43 AM, Michael Stone wrote:
>> On Tue, Mar 06, 2012 at 03:50:34AM +0400, sergio wrote:
>>> indicator-style mast not affect on dereference.
>
>>> % /bin/ls -F /var/run
>>> /var/run@
>
>> Can you be more specific about what you're looking for?
>
> I mean
> ls -F /var/run should give:
> acpid.pid dhclient.eth0.pid  lock/  rsyslogd.pid sshd/

To get what you want, append a slash to the directory name:

  ls -F /var/run/



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#659593: coreutils: *** glibc detected *** tac: double free or corruption (top): 0x000000000061b030 ***

[Jim Meyering]

Segfault in wrote:

> Package: coreutils
> Version: 8.5-1
> Severity: important
>
> $ tac *>/dev/null
> *** glibc detected *** tac: double free or corruption (top):
> 0x0061b030 ***

Thanks for the report.
That was fixed upstream in coreutils-8.6 (latest is 8.15):

* Noteworthy changes in release 8.6 (2010-10-15) [stable]
...
  tac would perform a double-free when given an input line longer than 
16KiB.
  [bug introduced in coreutils-8.3]



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#654861: coreutils: sort creates temporary files in /var/tmp instead of /tmp thus violating the FHS

[Jim Meyering]

> For some reason, sort generates temporary files in /var/tmp instead of /tmp.
> As /var/tmp is not cleaned regularly or on boot, this means that any left
> over files (eg due to crashes) will be left in /var/tmp forever.
> Also sort cannot use the data on subsequent invocations, which makes the
> persistence of the data meaningless.
>
> This bug is market as important because this violates the FHS. This would
> require a severity of serious, but IMHO the impact of this bug is too
> low to justify such a high severity.

Thanks for the report, but I think this must be due to your environment.
Sort honors the $TMPDIR environment variable, and if it is set to /var/tmp,
then that would explain what you're seeing.

However, if you can run sort like this

env TMPDIR=/tmp sort ...

and show that it still writes to files in /var/tmp, please do let us know.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#649913: Please use posix_fallocate

[Jim Meyering]

Goswin von Brederlow wrote:
> Package: coreutils
> Version: 8.5-1
> Severity: normal
> File: /bin/cp
>
> when copying files the size of the resulting file is known (except in
> race conditions) beforehand and should be communicated to the
> filesystem using posix_fallocate(). This ensures there is enough space
> for the file and more importantly allows the filesystem to place the
> file better, idealy as on continious chunk, on the filesystem.
>
> Some care must be taken to ftruncate() the file to its smaller size if
> the source file has shrunk while copying.
>
> If the file has grown while copying one could posix_fallocate() to the
> new size repeadately till the full file is copied but it probably
> wouldn't hurt to ignore that case and just copy the remainder of the
> grown file without allocating space first.

In upstream discussion, Pádraig Brady just wrote about precisely that:

http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/23023/focus=23409



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#647889: xgraph: "echo 1 2|xgraph" segfaults due to deref of uninitialized pointer

[Jim Meyering]

Package: xgraph
Version: 12.1-14
Severity: important
Tags: upstream patch

Dear Maintainer,

Thank you for maintaining xgraph.

I noticed that xgraph would always segfault for me.
I tracked it down to my setting of MALLOC_PERTURB_ to a nonzero value,
(that makes glibc scribble on freed/malloc'd memory).  That exposed
the latent problem in xgraph.  Debugged it and wrote the patch below.

Also, after cloning from git and running autoreconf, I noticed that git
diff reported many changes in the generated file, aclocal.m4 (because
I use a newer version of autoconf).  I attach a patch to remove aclocal.m4
from version control, too.

>From bd5ae5869e16792c5fa6cb894ed2e50e52278c62 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Mon, 7 Nov 2011 11:01:44 +0100
Subject: [PATCH 2/2] don't dereference a pointer with uninitialized stack
 data

Before this change, xgraph was unusable for me:
(because I set e.g., MALLOC_PERTURB_=39 in my environment)

$ echo 1 2| MALLOC_PERTURB_=39 xgraph
zsh: doneecho 1 2 |
zsh: segmentation fault  xgraph

Given a bogus or unresolvable label or title font name specified with
"-size", like "helvetica-9", this code would determine that there are
no matching fonts, hence never run the body of the for loop, and then
test *font_info, which would be uninitialized stack data.  If that
happened to be nonzero, it would return 1 and the caller would
dereference that font_info pointer.
* params.c (do_font): Do not return uninitialized *font_info as if it
were a valid pointer.
---
 params.c |4 +---
 1 files changed, 1 insertions(+), 3 deletions(-)

diff --git a/params.c b/params.c
index abcde5f..181500e 100644
--- a/params.c
+++ b/params.c
@@ -358,9 +358,7 @@ XFontStruct **font_info;/* Returned font information */
/* Load first one that you can */
for (i = 0; i < font_count; i++)
  if ((*font_info = XLoadQueryFont(param_disp, font_list[i])))
-   break;
-   if (*font_info)
-   return 1;
+   return 1;
}
}
/* Assume normal font name */
--
1.7.8.rc0.35.gee6df




-- System Information:
Debian Release: wheezy/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.37-2-amd64 (SMP w/1 CPU core)
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968) (ignored: LC_ALL set to C)
Shell: /bin/sh linked to /bin/dash

Versions of packages xgraph depends on:
ii  libc6 2.13-21
ii  libice6   2:1.0.7-2
ii  libsm62:1.2.0-2
ii  libx11-6  2:1.4.4-2

xgraph recommends no packages.

Versions of packages xgraph suggests:
pn  ygraph  

-- no debconf information
>From fdb59dd5bf97a46749268d7db51478610baacea4 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Mon, 7 Nov 2011 10:54:47 +0100
Subject: [PATCH 1/2] don't version-control the generated file, aclocal.m4

* aclocal.m4: Remove generated file.
* .gitignore: Tell git to ignore it, too.
---
 .gitignore |1 +
 aclocal.m4 |  979 
 2 files changed, 1 insertions(+), 979 deletions(-)
 delete mode 100644 aclocal.m4

diff --git a/.gitignore b/.gitignore
index fa2acb2..8debbd1 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,6 +1,7 @@
 .deps/
 Makefile
 Makefile.in
+aclocal.m4
 autoconf.h
 autom4te.cache/
 config.log
diff --git a/aclocal.m4 b/aclocal.m4
deleted file mode 100644
index 27326f4..000
--- a/aclocal.m4
+++ /dev/null
@@ -1,979 +0,0 @@
-# generated automatically by aclocal 1.11.1 -*- Autoconf -*-
-
-# Copyright (C) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004,
-# 2005, 2006, 2007, 2008, 2009  Free Software Foundation, Inc.
-# This file is free software; the Free Software Foundation
-# gives unlimited permission to copy and/or distribute it,
-# with or without modifications, as long as this notice is preserved.
-
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY, to the extent permitted by law; without
-# even the implied warranty of MERCHANTABILITY or FITNESS FOR A
-# PARTICULAR PURPOSE.
-
-m4_ifndef([AC_AUTOCONF_VERSION],
-  [m4_copy([m4_PACKAGE_VERSION], [AC_AUTOCONF_VERSION])])dnl
-m4_if(m4_defn([AC_AUTOCONF_VERSION]), [2.67],,
-[m4_warning([this file was generated for autoconf 2.67.
-You have another version of autoconf.  It may work, but is not guaranteed to.
-If you have problems, you may need to regenerate the build system entirely.
-To do so, use the procedure documented by the package, typically `autoreconf'.])])
-
-# Copyright (C) 2002, 2003, 2005, 2006, 2007, 2008  Free Software Foundation, Inc.
-#
-# This file is free software; the Free Software Foundation
-# gives unlimited permission to copy and/or distribute it,
-# with or without modifications, as long as this notice is preserved.
-
-# AM_AUTOMAKE_VERSION(VERSION)
-# 

Bug#514893: cmp: please allow comparing multiple files (fwd)

[Jim Meyering]

Santiago Vila wrote:
> A long time ago, I received this from the Debian bug system.
>
> [ I realize that this is unlikely to be implemented, but I am supposed
> to forward upstream bugs upstream in either case ].

Thanks for forwarding that.
However, I'll bet you can do something similar with od and a program
that can show differences in 3 files.  Here's how you can come close
using only two inputs with sdiff:

$ f() { od -Anone -w1 -v -tc "$@"; }
$ sdiff -w14 <(printf 'a\x06\xff\n'|f) <(printf 'a\x05\xff\n'|f)
   a   a
 006  |  005
 377 377
  \n  \n
[Exit 1]

> -- Forwarded message --
> From: Michael Stransky 
> To: Debian Bug Tracking System 
> Date: Wed, 11 Feb 2009 19:26:52 +0100
> Subject: cmp: please allow comparing multiple files
>
> Package: diff
> Version: 2.8.1-11
> Severity: wishlist
>
>
> It would be great if cmp could handle more then 2 files.
> cmp -c file1 file2 file3 would output for example:
> (byte file1 file2 file3)
> 1 100 101 102
> 2 100 100 101
> 7 100 101 102
> cmp: EOF on file2.
> 9 100 102
> cmp: EOF on file3.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#646022: coreutils: "tail -f" doesn't work with GPFS filesystem

[Jim Meyering]

Martial Bornet (gmail) wrote:

> Package: coreutils
> Version: 8.5-1
> Severity: normal
>
> The "tail" command doesn't know the GPFS filesystem type, so it uses
> inotify() instead of nanosleep() when used with "-f".
>
> The following line should be added to the fs.h file :
>
> # define S_MAGIC_GPFS   0x47504653
>
> and the following line should be added to the fremote() function of the
> tail.c file :
>
> case S_MAGIC_GPFS:

Thanks for the report.
Here's the upstream patch.
We already have the definition of S_MAGIC_GPFS.

>From c07d7486432429eb9e5cadd083d15247b0f4ab0d Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Thu, 20 Oct 2011 19:18:09 +0200
Subject: [PATCH] tail: with -f, use nanosleep, not inotify on a GPFS file
 system

* src/tail.c (fremote): List GPFS as a remote file system type.
* THANKS.in: Update.
* NEWS (Bug fixes): Mention it.
Report and suggested change by Martial Bornet in
http://bugs.debian.org/646022.
---
 NEWS   |4 
 THANKS.in  |1 +
 src/tail.c |1 +
 3 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/NEWS b/NEWS
index 3ed44b2..4d210b5 100644
--- a/NEWS
+++ b/NEWS
@@ -7,6 +7,10 @@ GNU coreutils NEWS-*- 
outline -*-
   tac no longer fails to handle two or more non-seekable inputs
   [bug introduced in coreutils-5.3.0]

+  tail -f no longer tries to use inotify on GPFS file systems
+  [you might say this was introduced in coreutils-7.5, along with inotify
+   support, but the GPFS magic number wasn't in the usual places then.]
+

 * Noteworthy changes in release 8.14 (2011-10-12) [stable]

diff --git a/THANKS.in b/THANKS.in
index 83a7864..b99363b 100644
--- a/THANKS.in
+++ b/THANKS.in
@@ -367,6 +367,7 @@ Mark Melahn mmel...@gmail.com
 Mark Nudelman   mar...@flash.net
 Mark W. Eichin  eic...@cygnus.com
 Markus Demleitner   msdem...@auriga.ari.uni-heidelberg.de
+Martial Bornet  mbornet@gmail.com
 Martin  mar...@dresden.nacamar.de
 Martin Buck martin.b...@ascom.ch
 Martin Gallant  mar...@goodbit.net
diff --git a/src/tail.c b/src/tail.c
index f315776..1641a12 100644
--- a/src/tail.c
+++ b/src/tail.c
@@ -904,6 +904,7 @@ fremote (int fd, const char *name)
 case S_MAGIC_FUSEBLK:
 case S_MAGIC_FUSECTL:
 case S_MAGIC_GFS:
+case S_MAGIC_GPFS:
 case S_MAGIC_KAFS:
 case S_MAGIC_LUSTRE:
 case S_MAGIC_NCP:
--
1.7.7.419.g87009



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#642557: coreutils: "sort -g" hangs when fed large numbers of nans ("not a number").

[Jim Meyering]

Aaron Denney wrote:
> Version: 8.13-2
> "yes -- '-nan' | head -116903 | sort -g > /dev/null" completes in
> under a second.
>
> while
>
> "yes -- '-nan' | head -116904 | sort -g > /dev/null" appears to hang.

Thanks for the report.
Technically, this is not due to a bug in coreutils, but I've patched
sort.c, at least temporarily, to work around the problem:

  http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/23080



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#641166: coreutils: 'man sort': '--random-sort' misleading

[Jim Meyering]

Gwern Branwen wrote:
> On Mon, Sep 12, 2011 at 10:01 AM, Jim Meyering  wrote:
>> Did you see the "real" documentation?
>
> No; when I was younger, I sometimes looked at the info page for
> commands, but invariably they seemed to be useless or copies of the

Please try to reset your misconception, at least for the coreutils.
Though, in general the info documentation for GNU programs is far
superior to the man pages.

> man page, and I wrote them off completely as a strange GNU waste of
> time akin to Guile or other GNU quirks. If length is the problem, how
> about adding '; not perfect shuffle'? 3 words in the places most
> people will look for documentation.

Sorry, but that would be inaccurate, because sometimes (no duplicates),
it does give you a perfect shuffle.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#641166: coreutils: 'man sort': '--random-sort' misleading

[Jim Meyering]

gwern wrote:

> Package: coreutils
> Version: 8.5-1
> Severity: minor
>
> The existing documentation for the option:
>
>-R, --random-sort
>   sort by random hash of keys
>
> This is not wrong, strictly-speaking, but it is misleading: sorting by random 
> hash *sounds* like a perfect shuffle,
> which is what 99% of users want, and sorting by hash is equivalent if and if 
> only there are no duplicate entries.
> If there *are* duplicate entries, then the 'random' sort will put all 
> duplicates in consecutive runs.
>
> I suggest amending the line to read more like
>
>   sort by random hash of keys; equivalent to perfect shuffle on 
> unique keys
>
> or maybe just say
>
>   sort by random hash of keys; not the same as a perfect shuffle
>
> Or at least warn in some fashion that 'random' is not quite what 'random' 
> usually means on lists.
>
> (I do random shuffle with mplayer using 'sort -R', and once, to 'bias' the 
> selection to a particular set of songs, I put that directory in
> 3 or 4 times; I thought I was going crazy when the first such song came up 3 
> times, which I calculated at billions to one against.
> I checked everything until I began to wonder what exactly 'random hash of 
> keys' meant, and then saw how it treated duplicate entries.)

Thanks for the suggestion, but we try to keep the man page pretty terse,
since it's automatically derived from sort --help output.

Did you see the "real" documentation?

  `-R'
  `--random-sort'
  `--sort=random'
   Sort by hashing the input keys and then sorting the hash values.
   Choose the hash function at random, ensuring that it is free of
   collisions so that differing keys have differing hash values.
   This is like a random permutation of the inputs (*note shuf
   invocation::), except that keys with the same value sort together.

   If multiple random sort fields are specified, the same random hash
   function is used for all fields.  To use different random hash
   functions for different fields, you can invoke `sort' more than
   once.

   The choice of hash function is affected by the `--random-source'
   option.

There should be a note like this the end of the man page:

 SEE ALSO
   The  full documentation for sort is maintained as a Texinfo manual.  If
   the info and sort programs are properly installed  at  your  site,  the
   command

  info sort

   should give you access to the complete manual.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#294327: [PATCH] doc: note that cp -l creates _hard_ links

[Jim Meyering]

Benoît Knecht wrote:
> This fact was already noted in the Texinfo manual, but not in the output
> of --help.
>
> * src/cp.c (usage): As above, for --help.
> Reported by Jari Aalto in http://bugs.debian.org/294327.

Thank you.
Applied with the one-line summary edited to start with "doc:"
rather than "Bug#294327: [PATCH] doc:"

BTW, please Cc coreutils@ (not bug-coreutils@) next time.
Messages sent to the latter usually end up automatically creating or
being associated with a bug ID (see http://debbugs.gnu.org/coreutils).



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#115833: [PATCH] date: document that %k and %l are space padded

[Jim Meyering]

Benoît Knecht wrote:
> The help output and man page mention that "by default, date pads numeric
> fields with zeroes," yet the description of %k and %l didn't specify
> that these values were space padded, whereas the description of %e did.
>
> Fixes .
> ---
>
> Hi,
>
> If anyone is still interested in fixing this bug, here's a trivial patch
> that amends the documentation of date accordingly. Otherwise, feel free
> to close this bug as won't fix.

Thanks for the patch.
I've turned that into something I can use upstream.
If you send more patches, please follow this model (note the form of
the commit log and credit the reporter in the log and add mention in
THANKS.in; details in HACKING).

>From 4d6f5ba0477c086fb3258d8bfc638393a16087ed Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Beno=C3=AEt=20Knecht?= 
Date: Thu, 7 Jul 2011 10:55:47 +0200
Subject: [PATCH] date: note %k and %l are space-padded and equivalent to %_H
 and %_I

* src/date.c (usage): As above, for --help.
* doc/coreutils.texi (Time conversion specifiers): Likewise.
Reported by Britton Leo Kerin in http://bugs.debian.org/115833.
---
 THANKS.in  |1 +
 doc/coreutils.texi |4 ++--
 src/date.c |4 ++--
 3 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/THANKS.in b/THANKS.in
index 2979d52..87c5b19 100644
--- a/THANKS.in
+++ b/THANKS.in
@@ -92,6 +92,7 @@ Brian Kimball   b...@footbag.org
 Brian M. Carlsonsand...@crustytoothpaste.ath.cx
 Brian Silverman bsilver...@conceptxdesign.com
 Brian Youmans   3d...@gnu.org
+Britton Leo Kerin   fs...@aurora.uaf.edu
 Bruce Robertson bru...@theodolite.dyndns.org
 Carl Johnsonca...@cjlinux.home.org
 Carl Lowenstein c...@mpl.ucsd.edu
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index c59af2f..11ac7fd 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -13921,10 +13921,10 @@ Time conversion specifiers
 @item %I
 hour (@samp{01}@dots{}@samp{12})
 @item %k
-hour (@samp{ 0}@dots{}@samp{23}).
+hour, space padded (@samp{ 0}@dots{}@samp{23}); equivalent to @samp{%_H}.
 This is a @acronym{GNU} extension.
 @item %l
-hour (@samp{ 1}@dots{}@samp{12}).
+hour, space padded (@samp{ 1}@dots{}@samp{12}); equivalent to @samp{%_I}.
 This is a @acronym{GNU} extension.
 @item %M
 minute (@samp{00}@dots{}@samp{59})
diff --git a/src/date.c b/src/date.c
index 6439d16..755736a 100644
--- a/src/date.c
+++ b/src/date.c
@@ -179,8 +179,8 @@ FORMAT controls the output.  Interpreted sequences are:\n\
   %j   day of year (001..366)\n\
 "), stdout);
   fputs (_("\
-  %k   hour ( 0..23)\n\
-  %l   hour ( 1..12)\n\
+  %k   hour, space padded ( 0..23); same as %_H\n\
+  %l   hour, space padded ( 1..12); same as %_I\n\
   %m   month (01..12)\n\
   %M   minute (00..59)\n\
 "), stdout);
--
1.7.6.430.g34be2



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#630735: coreutils cross-build support

[Jim Meyering]

Steve McIntyre wrote:
> On Thu, Jun 16, 2011 at 02:04:34PM -0400, Michael Stone wrote:
>>On Thu, Jun 16, 2011 at 06:50:21PM +0100, Steve McIntyre wrote:
>>>For now, the attached debdiff simply disables generation of the docs
>>>for cross-builds. The right answer (IMHO) would be to convince
>>>upstream to generate their docs in a more sane way (e.g. having a
>>>common source for help information that could be used both for --help
>>>output *and* the man pages).
>>
>>They don't strictly need to be built at build time, but the
>>alternative does make things a bit more complicated. Basically, any
>>time a patch changes the help you need to rebuild it, but that could
>>be done at packaging time with some kind of option to apply all the
>>patches, build & generate the man pages, save a diff of the new man
>>pages, and add that diff to the package. Patches welcome. :-)
>
> :-)
>
>>I'll actually look closer at it tonight, there may even be a simpler
>>solution than that as I usually try really hard to avoid touching the
>>help output (because it also screws up localization strings) and it
>>might be sufficient to try to just keep the versions from the
>>upstream tar.
>
> Hmmm, OK. Is there no chance of persuading upstream to do things
> better and either just ship man pages (yeah, I know about the GNU info
> thing, ick!) or to split out the help text into separate data files
> that could then be transformed/compiled into both man pages and the
> binaries?

Upstream tarballs already include the generated man pages:

   $ tar tf coreutils-8.12.87-23dd.tar.xz|grep '.1$'|wc -l
   124



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#624387: [PATCH] dfa: fix case folding logic for character ranges

[Jim Meyering]

Paolo Bonzini wrote:
> * src/dfa.c (setbit_case_fold): Remove, replace with...
> (setbit_wc, setbit_c, setbit_case_fold_c): ... these.
> (parse_bracket_exp): Use setbit_case_fold_c when iterating over
> single-byte sequences.  Use setbit_wc for multi-byte character sets,
> and setbit_case_fold_c for single-byte character sets.
> (lex): Use setbit_case_fold_c for single-byte character sets.
> ---
>   > At first I was going to say this:
>   >
>   >   You are using ru_RU.KOI8-R, which is a uni-byte locale, yet your
>   >   inputs (both stdin and the grep regexp) use the two-byte
>   >   representation \xd0\9f for П, instead of the uni-byte \360.
>   >
>   > But it fails even with the single-byte version.
>   > So it is indeed a bug in grep, but at least this time
>   > it affects relatively few locales.
>   >
>   > Here's the fix I expect to use and a test case to exercise it.
>
> The bug affects all single-byte locales except ISO-8859-1 ones.
> It is quite serious---the logic to map wide characters back to
> bytes makes no sense.
>

Thanks again for working on this.
Your mention of case folding implies that this fixes a bug unrelated
to the one I have just fixed.  My patch affected code that was guarded
by "!case_fold".

If your patch does indeed fix a case-folding bug,
I would really like to see a test case.
Grep is at a point in its development that any bug
fix commit really should be accompanied by a test suite
addition, if at all possible.

All that said, your patch looks like a fine improvement.

Ahh.  Perfect.  I see that as I was writing this you have
posted a test case (and a nice optimization!).  Thanks again!

The suggestion below looks like it still applies to your latest.

> diff --git a/src/dfa.c b/src/dfa.c
> index b41cbb6..6602ae8 100644
> --- a/src/dfa.c
> +++ b/src/dfa.c
> @@ -536,51 +536,67 @@ dfasyntax (reg_syntax_t bits, int fold, unsigned char 
> eol)
>eolbyte = eol;
>  }
>
> -/* Like setbit, but if case is folded, set both cases of a letter.
> -   For MB_CUR_MAX > 1, one or both of the two cases may not be set,
> -   so the resulting charset may only be used as an optimization.  */
> -static void
> -setbit_case_fold (
> +/* Set a bit in the charclass for the given wchar_t.  Do nothing if WC
> +   is represented by a multi-byte sequence.  Even for MB_CUR_MAX == 1,
> +   this may happen when folding case in weird Turkish locales where
> +   dotless i/dotted I are not included in the chosen character set.
> +   Return whether a bit was set in the charclass.  */
>  #if MBS_SUPPORT
> -  wint_t b,
> +static bool
> +setbit_wc (wint_t wc, charclass c)
> +{
> +  int b = wctob (wc);
> +  if (b != EOF)
> +{
> +  setbit (b, c);
> +  return true;
> +}
> +  else
> +return false;
> +}

We can avoid the negation, curly braces and the "else".
Not only does this align slightly better with the new description,
but it seems more readable:

  int b = wctob (wc);
  if (b == EOF)
return false;

  setbit (b, c);
  return true;

> +/* Set a bit in the charclass for the given single byte character,
> +   if it is valid in the current character set.  */
> +static void
> +setbit_c (int b, charclass c)
> +{
> +  /* Do nothing if b is invalid in this character set.  */
> +  if (MB_CUR_MAX > 1 && btowc (b) == EOF)
> +return;
> +  setbit (b, c);
> +}



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#624387: [PATCH] dfa: fix case folding logic for character ranges

[Jim Meyering]

Paolo Bonzini wrote:
> * src/dfa.c (setbit_case_fold): Remove, replace with...
> (setbit_wc, setbit_c, setbit_case_fold_c): ... these.
> (parse_bracket_exp): Use setbit_case_fold_c when iterating over
> single-byte sequences.  Use setbit_wc for multi-byte character sets,
> and setbit_case_fold_c for single-byte character sets.
> (lex): Use setbit_case_fold_c for single-byte character sets.
> ---
>   > At first I was going to say this:
>   >
>   >   You are using ru_RU.KOI8-R, which is a uni-byte locale, yet your
>   >   inputs (both stdin and the grep regexp) use the two-byte
>   >   representation \xd0\9f for П, instead of the uni-byte \360.
>   >
>   > But it fails even with the single-byte version.
>   > So it is indeed a bug in grep, but at least this time
>   > it affects relatively few locales.
>   >
>   > Here's the fix I expect to use and a test case to exercise it.
>
> The bug affects all single-byte locales except ISO-8859-1 ones.
> It is quite serious---the logic to map wide characters back to
> bytes makes no sense.
>
> The attached patch fixes it and does not regress high-bit-range,
> while removing the debatable uses of wctob and checks for EOF.  Ok
> to apply together with your testcase?
> ---
>  src/dfa.c |  102 
> ++---
>  1 files changed, 57 insertions(+), 45 deletions(-)

Hi Paolo,

Thanks for following through on this.
At first glance (I'll look carefully today)
this looks like the right approach.

However, I've gone ahead and pushed my patch and test case,
since it does solve the problem at hand, and I have not seen
inputs that make that code misbehave.



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#624387: [bug #33198] Incorrect bracket expression when parsing in ru_RU.KOI8-R (Russian locale)

[Jim Meyering]

Paolo Bonzini wrote:

> On Sat, Jun 4, 2011 at 09:48, Jim Meyering  wrote:
>>> The b2 == EOF part is required for the somewhat similar bug I fixed
>>> a month ago:
>>>
>>>     fix a bug whereby echo c|grep '[c]' would fail for any c in 0x80..0xff
>>>     8da41c930e03a8635cbd8c89e3e591374c232c89
>>>
>>> The corresponding test demonstrates the need:
>>>
>>>     tests: exercise bug with 0x80..0xff in [...]
>>>     d98338ebf842ec9b69631837eee50ebdcd543505
>
> [\xff] is not well defined for a UTF-8 locale at all, actually.
> Perhaps FETCH_WC should return wc = EOF in this case (and c = 255),
> and it could be handled on a case-by-case basis elsewhere.
>
> But if wctob returns EOF, and b > UCHAR_MAX, you have introduced an
> out-of-bounds access in setbit.

You must not have seen the guards I added in the comments just
preceding yours, e.g.,

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=624387#129



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#624387: [bug #33198] Incorrect bracket expression when parsing in ru_RU.KOI8-R (Russian locale)

[Jim Meyering]

Paolo Bonzini wrote:
> On Sat, Jun 4, 2011 at 09:48, Jim Meyering  wrote:
>>> The b2 == EOF part is required for the somewhat similar bug I fixed
>>> a month ago:
>>>
>>>     fix a bug whereby echo c|grep '[c]' would fail for any c in 0x80..0xff
>>>     8da41c930e03a8635cbd8c89e3e591374c232c89
>>>
>>> The corresponding test demonstrates the need:
>>>
>>>     tests: exercise bug with 0x80..0xff in [...]
>>>     d98338ebf842ec9b69631837eee50ebdcd543505
>
> [\xff] is not well defined for a UTF-8 locale at all, actually.
> Perhaps FETCH_WC should return wc = EOF in this case (and c = 255),
> and it could be handled on a case-by-case basis elsewhere.
>
> But if wctob returns EOF, and b > UCHAR_MAX, you have introduced an
> out-of-bounds access in setbit.

Yes, I saw that.
That's why I added the guard I mentioned in previous mail.

I would like a test case that would segfault without the
(b < NOTCHAR) guard below.  If someone can construct one,
I'll be more than happy to add it to the test suite.

Here's the patch I expect to push:

>From 168577596e38981d93ea57d56d325172cfed7dc7 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Thu, 2 Jun 2011 18:03:49 +0200
Subject: [PATCH 1/5] fix the [...] bug also for relatively unusual uni-byte
 encodings
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* src/dfa.c (setbit_case_fold): Also handle uni-byte locales
like the one mentioned in the original report: see 2011-05-07
commit d98338eb.  Re-reported by Santiago Ruano Rincón.
Note that most uni-byte locales are not affected.
* NEWS (Bug fixes): Mention it.
---
 NEWS  |4 
 src/dfa.c |   10 +++---
 2 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/NEWS b/NEWS
index 312c803..67b3fad 100644
--- a/NEWS
+++ b/NEWS
@@ -4,6 +4,10 @@ GNU grep NEWS-*- outline 
-*-

 ** Bug fixes

+  echo c|grep '[c]' would fail for any c in 0x80..0xff, with a uni-byte
+  encoding for which the byte-to-wide-char mapping is nontrivial.  For
+  example, the ISO-88591 locales are not affected, but ru_RU.KOI8-R is.
+
   grep -P no longer aborts when PCRE's backtracking limit is exceeded
   Before, echo aab |grep -P '((a+)*)+$' would abort.  Now,
   it diagnoses the problem and exits with status 2.
diff --git a/src/dfa.c b/src/dfa.c
index b41cbb6..83386aa 100644
--- a/src/dfa.c
+++ b/src/dfa.c
@@ -573,10 +573,14 @@ setbit_case_fold (
   else
 {
 #if MBS_SUPPORT
-  int b2 = wctob ((unsigned char) b);
-  if (b2 == EOF || b2 == b)
+  /* Below, note how when b2 != b and we have a uni-byte locale
+ (MB_CUR_MAX == 1), we set b = b2.  I.e., in a uni-byte locale,
+ we can safely call setbit with a non-EOF value returned by wctob.  */
+  int b2 = wctob (b);
+  if (b2 == EOF || b2 == b || (MB_CUR_MAX == 1 ? (b=b2), 1 : 0))
 #endif
-setbit (b, c);
+if (b < NOTCHAR)
+  setbit (b, c);
 }
 }

--
1.7.6.rc0.254.gf37de



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#624387: [bug #33198] Incorrect bracket expression when parsing in ru_RU.KOI8-R (Russian locale)

[Jim Meyering]

Jim Meyering wrote:

> Paolo Bonzini wrote:
>
>> On 06/02/2011 11:08 PM, Jim Meyering wrote:
>>>   #if MBS_SUPPORT
>>> -  int b2 = wctob ((unsigned char) b);
>>> -  if (b2 == EOF || b2 == b)
>>> +  /* Below, note how when b2 != b and we have a uni-byte locale
>>> + (MB_CUR_MAX == 1), we set b = b2.  I.e., in a uni-byte locale,
>>> + we can safely call setbit with a non-EOF value returned by wctob. 
>>>  */
>>> +  int b2 = wctob (b);
>>> +  if (b2 == EOF || b2 == b || (MB_CUR_MAX == 1 ? (b=b2), 1 : 0))
>>
>> Can you explain again the reason for testing "b2 == EOF"?  It seems
>> wrong, and without it you can just make
>>
>> if (MB_CUR_MAX == 1 || b2 == b)
>>   setbit ((unsigned char) b, c);
>
> Hi Paolo,
>
> Your test would disable DFA-based matching for some bytes in a locale like
> ru_RU.KOI8-R, because a pattern like [\360] leads to "wint_t b" having
> the value 1055 (0x041F), and that is obviously too large to
> be used as the first argument to setbit.  However, converting
> that "B" back to a single-byte value, B2, gives us back \360,
> which is ok to use there.  Hence the "(b=b2)" part of that
> admittedly ugly expression.
>
> The b2 == EOF part is required for the somewhat similar bug I fixed
> a month ago:
>
> fix a bug whereby echo c|grep '[c]' would fail for any c in 0x80..0xff
> 8da41c930e03a8635cbd8c89e3e591374c232c89
>
> The corresponding test demonstrates the need:
>
> tests: exercise bug with 0x80..0xff in [...]
> d98338ebf842ec9b69631837eee50ebdcd543505
>
> Thanks for the feedback.
> If you see a better way, I'm sure you'll let me know.
>
> BTW, seeing your cast, I now think it'd be prudent to
> guard that setbit use:
>
> #if MBS_SUPPORT
>   /* Below, note how when b2 != b and we have a uni-byte locale
>  (MB_CUR_MAX == 1), we set b = b2.  I.e., in a uni-byte locale,
>  we can safely call setbit with a non-EOF value returned by wctob.  */
>   int b2 = wctob (b);
>   if (b2 == EOF || b2 == b || (MB_CUR_MAX == 1 ? (b=b2), 1 : 0))
> #endif
> if (b < 256)
>   setbit (b, c);

Ahem.

s/256/NOTCHAR/



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#624387: [bug #33198] Incorrect bracket expression when parsing in ru_RU.KOI8-R (Russian locale)

[Jim Meyering]

Paolo Bonzini wrote:

> On Fri, Jun 3, 2011 at 19:09, Jim Meyering  wrote:
>> The b2 == EOF part is required for the somewhat similar bug I fixed
>> a month ago:
>>
>>    fix a bug whereby echo c|grep '[c]' would fail for any c in 0x80..0xff
>>    8da41c930e03a8635cbd8c89e3e591374c232c89
>
> Wouldn't that be covered by MB_CUR_MAX == 1, too?

No.
The bug I fixed a month ago also affected locales with MB_CUR_MAX
larger than 1, too, including at least a few key UTF8-based ones.

If you have a better patch in mind, please share.
Or try it and see if it fails any of the existing test cases.



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#624387: [bug #33198] Incorrect bracket expression when parsing in ru_RU.KOI8-R (Russian locale)

[Jim Meyering]

Paolo Bonzini wrote:

> On 06/02/2011 11:08 PM, Jim Meyering wrote:
>>   #if MBS_SUPPORT
>> -  int b2 = wctob ((unsigned char) b);
>> -  if (b2 == EOF || b2 == b)
>> +  /* Below, note how when b2 != b and we have a uni-byte locale
>> + (MB_CUR_MAX == 1), we set b = b2.  I.e., in a uni-byte locale,
>> + we can safely call setbit with a non-EOF value returned by wctob.  
>> */
>> +  int b2 = wctob (b);
>> +  if (b2 == EOF || b2 == b || (MB_CUR_MAX == 1 ? (b=b2), 1 : 0))
>
> Can you explain again the reason for testing "b2 == EOF"?  It seems
> wrong, and without it you can just make
>
> if (MB_CUR_MAX == 1 || b2 == b)
>   setbit ((unsigned char) b, c);

Hi Paolo,

Your test would disable DFA-based matching for some bytes in a locale like
ru_RU.KOI8-R, because a pattern like [\360] leads to "wint_t b" having
the value 1055 (0x041F), and that is obviously too large to
be used as the first argument to setbit.  However, converting
that "B" back to a single-byte value, B2, gives us back \360,
which is ok to use there.  Hence the "(b=b2)" part of that
admittedly ugly expression.

The b2 == EOF part is required for the somewhat similar bug I fixed
a month ago:

fix a bug whereby echo c|grep '[c]' would fail for any c in 0x80..0xff
8da41c930e03a8635cbd8c89e3e591374c232c89

The corresponding test demonstrates the need:

tests: exercise bug with 0x80..0xff in [...]
d98338ebf842ec9b69631837eee50ebdcd543505

Thanks for the feedback.
If you see a better way, I'm sure you'll let me know.

BTW, seeing your cast, I now think it'd be prudent to
guard that setbit use:

#if MBS_SUPPORT
  /* Below, note how when b2 != b and we have a uni-byte locale
 (MB_CUR_MAX == 1), we set b = b2.  I.e., in a uni-byte locale,
 we can safely call setbit with a non-EOF value returned by wctob.  */
  int b2 = wctob (b);
  if (b2 == EOF || b2 == b || (MB_CUR_MAX == 1 ? (b=b2), 1 : 0))
#endif
if (b < 256)
  setbit (b, c);
}



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#624387: [bug #33198] Incorrect bracket expression when parsing in ru_RU.KOI8-R (Russian locale)

[Jim Meyering]

Santiago Ruano Rincón wrote:
> Follow-up Comment #3, bug #33198 (project grep):
> It seems the problem is still unsolved. I've tried both, 2.8 and patching 2.7,
> but I got the same results. Igor Ladygin confirms this.
>
> santiago@nomada:~$ echo Пример| LC_ALL=ru_RU.KOI8-R grep -qE "[Пп]";
> echo $?
> 1

Here's a slightly better patch.
The dfa.c diff is the same, but I've corrected the test name
and added/corrected log comments.


>From cbd5055c976ebc93b657dcdf3783cc91de4f68ed Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Thu, 2 Jun 2011 18:03:49 +0200
Subject: [PATCH 1/2] fix the [...] bug also for relatively unusual uni-byte
 encodings
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* src/dfa.c (setbit_case_fold): Also handle uni-byte locales
like the one mentioned in the original report: see 2011-05-07
commit d98338eb.  Re-reported by Santiago Ruano Rincón.
Note that most uni-byte locales are not affected.
* NEWS (Bug fixes): Mention it.
---
 NEWS  |4 
 src/dfa.c |7 +--
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/NEWS b/NEWS
index 312c803..67b3fad 100644
--- a/NEWS
+++ b/NEWS
@@ -4,6 +4,10 @@ GNU grep NEWS-*- outline 
-*-

 ** Bug fixes

+  echo c|grep '[c]' would fail for any c in 0x80..0xff, with a uni-byte
+  encoding for which the byte-to-wide-char mapping is nontrivial.  For
+  example, the ISO-88591 locales are not affected, but ru_RU.KOI8-R is.
+
   grep -P no longer aborts when PCRE's backtracking limit is exceeded
   Before, echo aab |grep -P '((a+)*)+$' would abort.  Now,
   it diagnoses the problem and exits with status 2.
diff --git a/src/dfa.c b/src/dfa.c
index b41cbb6..0ce6242 100644
--- a/src/dfa.c
+++ b/src/dfa.c
@@ -573,8 +573,11 @@ setbit_case_fold (
   else
 {
 #if MBS_SUPPORT
-  int b2 = wctob ((unsigned char) b);
-  if (b2 == EOF || b2 == b)
+  /* Below, note how when b2 != b and we have a uni-byte locale
+ (MB_CUR_MAX == 1), we set b = b2.  I.e., in a uni-byte locale,
+ we can safely call setbit with a non-EOF value returned by wctob.  */
+  int b2 = wctob (b);
+  if (b2 == EOF || b2 == b || (MB_CUR_MAX == 1 ? (b=b2), 1 : 0))
 #endif
 setbit (b, c);
 }
--
1.7.6.rc0.254.gf37de


>From 713515f036767f4d0c1a162d5263f119bb1d92b4 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Thu, 2 Jun 2011 11:01:35 +0200
Subject: [PATCH 2/2] tests: exercise a uni-byte [...] bug: requires
 ru_RU.KOI8-R

* tests/unibyte-bracket-expr: New file.
* tests/Makefile.am (TESTS): Add it.
* init.cfg (require_ru_RU_koi8_r): New function.
---
 tests/Makefile.am  |1 +
 tests/init.cfg |9 +
 tests/unibyte-bracket-expr |   41 +
 3 files changed, 51 insertions(+), 0 deletions(-)
 create mode 100644 tests/unibyte-bracket-expr

diff --git a/tests/Makefile.am b/tests/Makefile.am
index a01b004..f354e4a 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -63,6 +63,7 @@ TESTS =   \
   inconsistent-range\
   khadafy  \
   max-count-vs-context \
+  unibyte-bracket-expr \
   high-bit-range   \
   options  \
   pcre \
diff --git a/tests/init.cfg b/tests/init.cfg
index 3429f0d..f6ead9c 100644
--- a/tests/init.cfg
+++ b/tests/init.cfg
@@ -69,3 +69,12 @@ require_en_utf8_locale_()
 *) skip_test_ 'en_US.UTF-8 locale not found' ;;
   esac
 }
+
+require_ru_RU_koi8_r()
+{
+  path_prepend_ .
+  case $(get-mb-cur-max ru_RU.KOI8-R) in
+1) ;;
+*) skip_test_ 'ru_RU.KOI8-R locale not found' ;;
+  esac
+}
diff --git a/tests/unibyte-bracket-expr b/tests/unibyte-bracket-expr
new file mode 100644
index 000..a0b51dd
--- /dev/null
+++ b/tests/unibyte-bracket-expr
@@ -0,0 +1,41 @@
+#!/bin/sh
+# Exercise a DFA range bug that arises only with a unibyte encoding
+# for which the wide-char-to-single-byte mapping is nontrivial.
+# E.g., the regexp, [C] would fail to match C in a unibyte locale like
+# ru_RU.KOI8-R for any C whose wide-char representation differed from
+# its single-byte equivalent.
+
+# Copyright (C) 2011 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public

Bug#624387: [bug #33198] Incorrect bracket expression when parsing in ru_RU.KOI8-R (Russian locale)

[Jim Meyering]

Santiago Ruano Rincón wrote:
> Follow-up Comment #3, bug #33198 (project grep):
> It seems the problem is still unsolved. I've tried both, 2.8 and patching 2.7,
> but I got the same results. Igor Ladygin confirms this.
>
> santiago@nomada:~$ echo Пример| LC_ALL=ru_RU.KOI8-R grep -qE "[Пп]";
> echo $?
> 1

Thank you.

At first I was going to say this:

  You are using ru_RU.KOI8-R, which is a uni-byte locale, yet your
  inputs (both stdin and the grep regexp) use the two-byte representation,
  П (\xd0\9f), instead of the uni-byte П (\360).

But it fails even with the single-byte version.
So it is indeed a bug in grep, but at least this time
it affects relatively few locales.

Here's the fix I expect to use and a test case to exercise it.

>From 8e214a2ecc4bac7f8341deb3646b6f1c3819dac3 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Thu, 2 Jun 2011 18:03:49 +0200
Subject: [PATCH 1/2] fix the range bug also for relatively unusual uni-byte
 encodings

* src/dfa.c (setbit_case_fold) Bug fix. FIXME
* NEWS (Bug fixes): Mention it.
---
 NEWS  |4 
 src/dfa.c |7 +--
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/NEWS b/NEWS
index 312c803..67b3fad 100644
--- a/NEWS
+++ b/NEWS
@@ -4,6 +4,10 @@ GNU grep NEWS-*- outline 
-*-

 ** Bug fixes

+  echo c|grep '[c]' would fail for any c in 0x80..0xff, with a uni-byte
+  encoding for which the byte-to-wide-char mapping is nontrivial.  For
+  example, the ISO-88591 locales are not affected, but ru_RU.KOI8-R is.
+
   grep -P no longer aborts when PCRE's backtracking limit is exceeded
   Before, echo aab |grep -P '((a+)*)+$' would abort.  Now,
   it diagnoses the problem and exits with status 2.
diff --git a/src/dfa.c b/src/dfa.c
index b41cbb6..0ce6242 100644
--- a/src/dfa.c
+++ b/src/dfa.c
@@ -573,8 +573,11 @@ setbit_case_fold (
   else
 {
 #if MBS_SUPPORT
-  int b2 = wctob ((unsigned char) b);
-  if (b2 == EOF || b2 == b)
+  /* Below, note how when b2 != b and we have a uni-byte locale
+ (MB_CUR_MAX == 1), we set b = b2.  I.e., in a uni-byte locale,
+ we can safely call setbit with a non-EOF value returned by wctob.  */
+  int b2 = wctob (b);
+  if (b2 == EOF || b2 == b || (MB_CUR_MAX == 1 ? (b=b2), 1 : 0))
 #endif
 setbit (b, c);
 }
--
1.7.6.rc0.254.gf37de


>From c93e621ac20d085abda4cf3c269f5cf902671a84 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Thu, 2 Jun 2011 11:01:35 +0200
Subject: [PATCH 2/2] tests: exercise a non-UTF8 multi-byte range bug:
 requires ru_RU.KOI8-R

* tests/mb-non-utf8-range: New file.
* tests/Makefile.am (TESTS): Add it.
* init.cfg (require_ru_RU_koi8_r): New function.
---
 tests/Makefile.am   |1 +
 tests/init.cfg  |9 +
 tests/mb-non-utf8-range |   41 +
 3 files changed, 51 insertions(+), 0 deletions(-)
 create mode 100644 tests/mb-non-utf8-range

diff --git a/tests/Makefile.am b/tests/Makefile.am
index a01b004..2d0527a 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -63,6 +63,7 @@ TESTS =   \
   inconsistent-range\
   khadafy  \
   max-count-vs-context \
+  mb-non-utf8-range\
   high-bit-range   \
   options  \
   pcre \
diff --git a/tests/init.cfg b/tests/init.cfg
index 3429f0d..f6ead9c 100644
--- a/tests/init.cfg
+++ b/tests/init.cfg
@@ -69,3 +69,12 @@ require_en_utf8_locale_()
 *) skip_test_ 'en_US.UTF-8 locale not found' ;;
   esac
 }
+
+require_ru_RU_koi8_r()
+{
+  path_prepend_ .
+  case $(get-mb-cur-max ru_RU.KOI8-R) in
+1) ;;
+*) skip_test_ 'ru_RU.KOI8-R locale not found' ;;
+  esac
+}
diff --git a/tests/mb-non-utf8-range b/tests/mb-non-utf8-range
new file mode 100644
index 000..a0b51dd
--- /dev/null
+++ b/tests/mb-non-utf8-range
@@ -0,0 +1,41 @@
+#!/bin/sh
+# Exercise a DFA range bug that arises only with a unibyte encoding
+# for which the wide-char-to-single-byte mapping is nontrivial.
+# E.g., the regexp, [C] would fail to match C in a unibyte locale like
+# ru_RU.KOI8-R for any C whose wide-char representation differed from
+# its single-byte equivalent.
+
+# Copyright (C) 2011 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

Bug#624387: [bug #33198] Incorrect bracket expression when parsing in ru_RU.KOI8-R (Russian locale)

[Jim Meyering]

arn...@skeeve.com wrote:
>> diff --git a/src/dfa.c b/src/dfa.c
>> index f2064ed..b41cbb6 100644
>> --- a/src/dfa.c
>> +++ b/src/dfa.c
>> @@ -573,7 +573,8 @@ setbit_case_fold (
>>else
>>  {
>>  #if MBS_SUPPORT
>> -  if (wctob ((unsigned char)b) == b)
>> +  int b2 = wctob ((unsigned char) b);
>> +  if (b2 == EOF || b2 == b)
>>  #endif
>>  setbit (b, c);
>>  }
>
> Any chance this could please be recast as
>
>   int b2 = wctob((unsigned char) b);
>   if (b2 == EOF || b2 == b)
>   ...

Hi Arnold,

You appear to have misread the patch, since that is precisely
what resulted when I made that change.

> The reason is that gawk still supports C90-only compilers (I have users
> wit such compilers) and the declaration after executable code won't fly
> in that case.
>
> This also makes only one call to wctob() instead of two. :-)

I see only one use of wctob, there.
The patch above moves it from the if-condition
into a C89-compatible assignment on the previous line.
There is no decl-after-statement, since the new declaration
of b2 is the first line of the enclosing else-block.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#624387: [bug #33198] Incorrect bracket expression when parsing in ru_RU.KOI8-R (Russian locale)

[Jim Meyering]

Dmitry V. Levin wrote:
> On Mon, May 02, 2011 at 07:20:15PM +0000, Jim Meyering wrote:
...
>> Thanks for the report.
>> however, I cannot reproduce the problem with grep-2.7 built from upstream
>> sources.  Nor can I reproduce it using the grep-2.7 that comes from Fedora
>> 15.
>>
>> To test it, I put these two lines in a file,
>>
>> #!/bin/sh
>> echo Пример| LC_ALL=ru_RU.KOI8-R grep -qE "[Пп]ример"; echo $?
>>
>> and then ran "bash FILE".  It always prints 1.
>> (Of course, I set PATH to select either /bin/grep or ./grep,
>> depending on which I'm testing).
>>
>> On that basis, I've closed this ticket.
>> If you have evidence that there's a real problem with the upstream sources,
>> feel free to reopen it.
>
> I confirm there is a bug, you just have to adjust the test case:
> $ echo Пример |iconv -tutf8 |LANG=ru_RU.utf8 grep -q "$(echo "[Пп]ример" 
> |iconv -tutf8)"; echo $?
> 0
> $ echo Пример |iconv -tkoi8r |LANG=ru_RU.koi8r grep -q "$(echo "[Пп]ример" 
> |iconv -tkoi8r)"; echo $?
> 1
> $ echo Sример |iconv -tkoi8r |LANG=ru_RU.koi8r grep -q "$(echo "[Ss]ример" 
> |iconv -tkoi8r)"; echo $?
> 0

Hi Dmitry,

Thanks for the clarification and for reopening that.
That is indeed a bug.
(actually I realized it right away and posted, but the
web form submission failed, and I noticed that only this morning)



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#622182: FTBFS: test touch/now-owned-by-other fails

[Jim Meyering]

Mathias Brodala wrote:

> Hi.
>
> Jim Meyering, 11.04.2011 09:13:
>> Mathias Brodala wrote:
>>>> FAIL: misc/tail (exit: 1)
>>>> =
>> ...
>>>> f-pipe-1.p...
>>>> tail: test f-pipe-1.p: stderr mismatch, comparing f-pipe-1.p.E
>>>> (actual) and f-pipe-1.p.3 (expected)
>>>> *** f-pipe-1.p.E   Sat Apr  9 22:58:08 2011
>>>> --- f-pipe-1.p.3   Sat Apr  9 22:58:08 2011
>>>> ***
>>>> *** 1 
>>>> - tail: cannot determine location of `standard input'. reverting to
>>>> polling: Function not implemented
>>>> --- 0 
>>
>> Thanks.
>> However, that's nothing serious (certainly not "grave").
>> Even if it were, how often does anyone run tail -f on a pipe?
>
> The usual "dmesg | tail -f" comes into mind. ;-)

tail -f still works on pipes.

The test fails because it's printing a diagnostic
(when none was expected) when reverting from inotify-based
to the classic polling-based method.

> Aside from that I actually meant this report to be about the FTFBS issue
> alone since that pretty much makes the package unusable to me. If I
> can’t compile it, then I cannot make modifications.

Failing a single nit-picky test should not be seen as making
the entire 100+-program package unusable.



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#622182: FTBFS: test touch/now-owned-by-other fails

[Jim Meyering]

Mathias Brodala wrote:
>> FAIL: misc/tail (exit: 1)
>> =
...
>> f-pipe-1.p...
>> tail: test f-pipe-1.p: stderr mismatch, comparing f-pipe-1.p.E
>> (actual) and f-pipe-1.p.3 (expected)
>> *** f-pipe-1.p.E Sat Apr  9 22:58:08 2011
>> --- f-pipe-1.p.3 Sat Apr  9 22:58:08 2011
>> ***
>> *** 1 
>> - tail: cannot determine location of `standard input'. reverting to
>> polling: Function not implemented
>> --- 0 

Thanks.
However, that's nothing serious (certainly not "grave").
Even if it were, how often does anyone run tail -f on a pipe?



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#622182: FTBFS: test touch/now-owned-by-other fails

[Jim Meyering]

Mathias Brodala wrote:
> Package: coreutils
> Version: 8.5-1
> Severity: grave
> Justification: renders package unusable
>
> Building coreutils fails on the test touch/now-owned-by-other. I see the
> same result when building via pbuilder, thus I’d assume it’s nothing
> about my environment.
>
> See the attached build log excerpt for details.
...
> SKIP: touch/now-owned-by-other (exit: 77)
> =
...
> ../touch/now-owned-by-other: skipping test: must be run as root
...
> ==
> 1 of 344 tests failed

Thanks for the report.
However, the test output you included
shows a *skipped* test, not the failing one.



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#432945: close

[Jim Meyering]

close 432945
thanks

> If this bug is fixed then why not close it ?

Done, with the above.
Thanks for the heads up.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#613319: diff --ignore-all-space doesn't ignore newlines after all

[Jim Meyering]

jida...@jidanni.org wrote:
> I forgot to X-debbugs-cc: bug-diffut...@gnu.org . They should please
> look at
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=613319

Contrary to documentation:

   The `-w' and `--ignore-all-space' options are stronger than `-b'.
They ignore difference even if one file has white space where the other
file has none.  "White space" characters include tab, newline, vertical

--ignore-all-space does not make diff ignore newlines:

$ diff -u --ignore-all-space <(seq 3) <(seq 3|fmt)
--- /proc/self/fd/112011-02-19 14:25:31.479058914 +0100
+++ /proc/self/fd/122011-02-19 14:25:31.478058883 +0100
@@ -1,3 +1 @@
-1
-2
-3
+1 2 3

Perhaps because diff is fundamentally line-oriented.
I haven't delved into this.

If you want that functionality, you might prefer wdiff.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#614006: xdiskusage: buffer overrun

[Jim Meyering]

Subject: xdiskusage: stack smash may cause segfault
Package: xdiskusage
Version: 1.48-10
Severity: normal
Tags: upstream patch

*** Please type your report below this line ***

I noticed that xdiskusage-1.48 could segfault given a deep hierarchy,
and tracked it down to a stack-smashing bug.  Three fixed-length buffers
on the stack may be overrun, each by one "long", and the value that is
written beyond the end of one of those buffers is somewhat under control
of the "attacker" (anyone who constructs a hierarchy that xdiskusage is
used to view).  The other two overruns write pointer values.
However, even if you know precisely how xdiskusage is being invoked,
it may be tricky to design a hierarchy that causes more than a segfault.


For an absolute minimal change, this does the job, but is ugly,
inefficient and fragile -- who wants to waste space on hierarchies
512 levels deep?  Plus, depending on the existing 1024-byte maximum
line length limit is fragile.  If that is ever fixed (there is a
FIXME comment) to remove or raise the limit, it would revive the bug
we're trying to fix here.

>From 2140cc66b09f2447ef7e7a89256e04c02bb18b2e Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Wed, 16 Feb 2011 18:16:11 +0100
Subject: [PATCH] MAXDEPTH: increase limit to 512

Otherwise, a long line like this:
  12 a/a/a/a/a/a/a/.../a
with more than 100 components, would cause xdiskusage
to write user-controllable data beyond the end of the "totals" buffer,
and, less so (pointers), beyond the end of the "lastnode" buffer.
---
 xdiskusage.C |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/xdiskusage.C b/xdiskusage.C
index 6a3ee77..f501528 100644
--- a/xdiskusage.C
+++ b/xdiskusage.C
@@ -170,7 +170,7 @@ struct Node {
 int window_w = 600;
 int window_h = 480;
 int ncols = 5;
-#define MAXDEPTH 100
+#define MAXDEPTH 512  // enough for now, considering we chop lines at 1024 
bytes

 class OutputWindow : public Fl_Window {
   void draw();
--
1.7.4.1.16.g759e8


Here's the patch I prefer:

>From c8a820f75e7f8a7012f53bc1c22435cc3f2a7407 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Wed, 16 Feb 2011 18:16:11 +0100
Subject: [PATCH 2/2] avoid buffer overrun for directory of depth 100 or more

Otherwise, a line like this:
  12 a/a/a/a/a/a/a/.../a
with more than 100 components, would cause xdiskusage
to write user-controllable data beyond the end of the "totals" buffer,
and, less so (pointers), beyond the end of the "lastnode" and "parts"
buffers.
---
 xdiskusage.C |6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/xdiskusage.C b/xdiskusage.C
index 6a3ee77..f572623 100644
--- a/xdiskusage.C
+++ b/xdiskusage.C
@@ -463,9 +463,9 @@ OutputWindow* OutputWindow::make(const char* path, Disk* 
disk) {
   root->ordinal = 0;
   ordinal = 0;

-  Node* lastnode[MAXDEPTH];
+  Node* lastnode[MAXDEPTH+1];
   ulong runningtotal;
-  ulong totals[MAXDEPTH];
+  ulong totals[MAXDEPTH+1];
   lastnode[0] = root;
   runningtotal = 0;
   totals[0] = 0;
@@ -519,7 +519,7 @@ OutputWindow* OutputWindow::make(const char* path, Disk* 
disk) {

 // split the path into parts:
 int newdepth = 0;
-const char* parts[MAXDEPTH];
+const char* parts[MAXDEPTH+1];
 if (*p == '/') {
   if (!root->name) root->name = strdup("/");
   p++;
--
1.7.4.1.16.g759e8


If you want a little added insurance, you can add a couple assertions.
The patch below depends on the one just above, but if you adjust the
expressions, they can also apply to the original and serve as a good way
to demonstrate that there is indeed a problem.  Especially since while I
do see a segfault pretty consistently on Fedora 14 (built from sources +
cast-patch), I was unable to evoke a segfault on debian unstable using
xdiskusage-1.48-10 .

Note that without the 1-element-larger "parts" array,
you'd have to adjust this loop not to modify parts[MAXDEPTH]:

   for (newdepth = 0; newdepth < MAXDEPTH && *p;) {
 parts[++newdepth] = p++;


>From af87585dfe2c632d9985484b56e730b5915b5f2c Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Fri, 18 Feb 2011 21:02:19 +0100
Subject: [PATCH] assert

---
 xdiskusage.C |3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/xdiskusage.C b/xdiskusage.C
index f572623..21ed026 100644
--- a/xdiskusage.C
+++ b/xdiskusage.C
@@ -42,6 +42,7 @@ const char* copyright =
 #include 
 #include 
 #include 
+#include 

 #include "panels.H"
 #include 
@@ -533,6 +534,7 @@ OutputWindow* OutputWindow::make(const char* path, Disk* 
disk) {
 // find out how many of the fields match:
 int match = 0;
 for 

Bug#609049: [coreutils] Re: Bug#609049: du: process_file: Assertion `level == prev_level - 1' failed

[Jim Meyering]

Jim Meyering wrote:
...
> Here's the incremental I'm testing with:
>
> diff --git a/tests/du/move-dir-while-traversing 
> b/tests/du/move-dir-while-traversing
> index e42bc93..fe1615c 100755

Thanks again for the quick feedback.
I squashed in the changes above and pushed the result.
I also updated to the latest gnulib in a separate commit,
since it passed the tests I ran.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#609049: [coreutils] Re: Bug#609049: du: process_file: Assertion `level == prev_level - 1' failed

[Jim Meyering]

Pádraig Brady wrote:

> On 09/01/11 22:05, Jim Meyering wrote:
>> diff --git a/tests/du/move-dir-while-traversing
>> b/tests/du/move-dir-while-traversing
>
>> +# We use a python-inotify script, so...
>> +python -m pyinotify -h > /dev/null \
>> +  || skip_ 'python-inotify package not installed'
>
> A small point is that error is a bit fedora specific.
> The package is python-pyinotify on debian/ubuntu for example.

Good point.  I've dropped the "-":

  || skip_ 'python inotify package not installed'

>> +from pyinotify import *
>
> I generally don't include everything from a module,
> to keep the namespace clean.
> I'd do instead:
>
> import pyinotify as pn
> import os,sys
> ...

I prefer that, too.

>> +dir = sys.argv[1]
...
>> +print 'started'
>
> The above print is buffered by default.
> As we're using it for synchronization I'd
>
> sys.stdout.write('started\n')
> sys.stdout.flush()

Good catch!

...
>> +nonempty() { test -s start-msg && return 0; sleep $1; }
>> +retry_delay_ nonempty .1 5
>
> I think the above may iterate only once?
> How about:
>
> nonempty() {
>   test -s start-msg || { sleep $1; return 1; }
> }
> retry_delay_ nonempty .1 5 || framework_failure

Yes, that fixes another bug.

>> +# The above delay is insufficient in ~50% of my trials.
>> +# Sometimes, when under heavy load, a parallel "make check" would
>> +# fail this test when sleeping just 0.1 seconds here.
>> +sleep 1
>
> Hopefully this extra sleep is not required now

Bingo.  It does, so far.
Thanks for the advice and fixes.
Here's the incremental I'm testing with:

diff --git a/tests/du/move-dir-while-traversing 
b/tests/du/move-dir-while-traversing
index e42bc93..fe1615c 100755
--- a/tests/du/move-dir-while-traversing
+++ b/tests/du/move-dir-while-traversing
@@ -21,7 +21,7 @@ print_ver_ du

 # We use a python-inotify script, so...
 python -m pyinotify -h > /dev/null \
-  || skip_ 'python-inotify package not installed'
+  || skip_ 'python inotify package not installed'

 # Move a directory "up" while du is processing its sub-directories.
 # While du is processing a hierarchy .../B/C/D/... this script
@@ -33,12 +33,14 @@ python -m pyinotify -h > /dev/null \

 cat <<'EOF' > inotify-watch-for-dir-access.py
 #!/usr/bin/env python
-from pyinotify import *
+import pyinotify as pn
+import os,sys
+
 dir = sys.argv[1]
 dest_parent = os.path.dirname(os.path.dirname(dir))
 dest = os.path.join(dest_parent, os.path.basename(dir))

-class ProcessDir(ProcessEvent):
+class ProcessDir(pn.ProcessEvent):

 def process_IN_OPEN(self, event):
 os.rename(dir, dest)
@@ -47,10 +49,11 @@ class ProcessDir(ProcessEvent):
 def process_default(self, event):
 pass

-wm = WatchManager()
-notifier = Notifier(wm)
-wm.watch_transient_file(dir, IN_OPEN, ProcessDir)
-print 'started'
+wm = pn.WatchManager()
+notifier = pn.Notifier(wm)
+wm.watch_transient_file(dir, pn.IN_OPEN, ProcessDir)
+sys.stdout.write('started\n')
+sys.stdout.flush()
 notifier.loop()
 EOF
 chmod a+x inotify-watch-for-dir-access.py
@@ -61,14 +64,9 @@ mkdir -p $t/3/a/b/c/$long d2 || framework_failure
 timeout 6 ./inotify-watch-for-dir-access.py $t/3/a/b > start-msg &

 # Wait for the watcher to start...
-nonempty() { test -s start-msg && return 0; sleep $1; }
+nonempty() { test -s start-msg || { sleep $1; return 1; }; }
 retry_delay_ nonempty .1 5

-# The above delay is insufficient in ~50% of my trials.
-# Sometimes, when under heavy load, a parallel "make check" would
-# fail this test when sleeping just 0.1 seconds here.
-sleep 1
-
 # The above watches for an IN_OPEN event on $t/3/a/b,
 # and when it triggers, moves the parent, $t/3/a, up one level
 # so it's directly under $t.



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#609049: [coreutils] Re: Bug#609049: du: process_file: Assertion `level == prev_level - 1' failed

[Jim Meyering]

Jim Meyering wrote:
...
> While I do have a reproducer that will become a test suite addition
> (coming soon), it relies on python (a first) and the python-inotify
> package, so I'll have to be careful to skip the test when those
> prerequisites aren't installed.  Also, there's an inherent race condition,
> so I'll have to find the right compromise between absolute test-robustness
> (too expensive in time and inodes) and reasonableness.

Here's a complete patch, including the test.
I would have preferred to avoid the one-second sleep.
Maybe someone here can see a better way.

>From 9bf47055f64efb56d022feca01ad901e85e21bc1 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Sat, 8 Jan 2011 17:44:55 +0100
Subject: [PATCH] du: don't abort when a subdir is renamed during traversal

* NEWS (Bug fixes): Mention it.
* src/du.c (prev_level): Move declaration "up" to file-scope global.
(du_files): Reset prev_level to 0 upon abnormal fts_read termination.
Reported by Johathan Nieder in http://bugs.debian.org/609049
Also, improve a diagnostic.
* tests/du/move-dir-while-traversing: Test for the above.
* tests/Makefile.am (TESTS): Add it.
---
 NEWS   |8 +++
 src/du.c   |   15 +--
 tests/Makefile.am  |1 +
 tests/du/move-dir-while-traversing |   85 
 4 files changed, 105 insertions(+), 4 deletions(-)
 create mode 100755 tests/du/move-dir-while-traversing

diff --git a/NEWS b/NEWS
index 2a71ca6..5a70243 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,14 @@ GNU coreutils NEWS-*- 
outline -*-

 * Noteworthy changes in release ?.? (-??-??) [?]

+** Bug fixes
+
+  du would abort with a failed assertion when two conditions are met:
+  part of the hierarchy being traversed is moved to a higher level in the
+  directory tree, and there is at least one more command line directory
+  argument following the one containing the moved sub-tree.
+  [bug introduced in coreutils-5.1.0]
+

 * Noteworthy changes in release 8.9 (2011-01-04) [stable]

diff --git a/src/du.c b/src/du.c
index 77deb0c..671cac7 100644
--- a/src/du.c
+++ b/src/du.c
@@ -63,8 +63,11 @@ extern bool fts_debug;
 /* A set of dev/ino pairs.  */
 static struct di_set *di_set;

-/* Define a class for collecting directory information. */
+/* Keep track of the preceding "level" (depth in hierarchy)
+   from one call of process_file to the next.  */
+static size_t prev_level;

+/* Define a class for collecting directory information. */
 struct duinfo
 {
   /* Size of files in directory.  */
@@ -399,7 +402,6 @@ process_file (FTS *fts, FTSENT *ent)
   struct duinfo dui;
   struct duinfo dui_to_print;
   size_t level;
-  static size_t prev_level;
   static size_t n_alloc;
   /* First element of the structure contains:
  The sum of the st_size values of all entries in the single directory
@@ -582,10 +584,15 @@ du_files (char **files, int bit_flags)
 {
   if (errno != 0)
 {
-  /* FIXME: try to give a better message  */
-  error (0, errno, _("fts_read failed"));
+  error (0, errno, _("fts_read failed: %s"),
+ quotearg_colon (fts->fts_path));
   ok = false;
 }
+
+  /* When exiting this loop early, be careful to reset the
+ global, prev_level, used in process_file.  Otherwise, its
+ (level == prev_level - 1) assertion could fail.  */
+  prev_level = 0;
   break;
 }
   FTS_CROSS_CHECK (fts);
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 06d81f0..a5dbd3e 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -370,6 +370,7 @@ TESTS = \
   du/long-from-unreadable  \
   du/long-sloop\
   du/max-depth \
+  du/move-dir-while-traversing \
   du/no-deref  \
   du/no-x  \
   du/one-file-system   \
diff --git a/tests/du/move-dir-while-traversing 
b/tests/du/move-dir-while-traversing
new file mode 100755
index 000..e42bc93
--- /dev/null
+++ b/tests/du/move-dir-while-traversing
@@ -0,0 +1,85 @@
+#!/bin/sh
+# Trigger a failed assertion in coreutils-8.9 and earlier.
+
+# Copyright (C) 2011 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without

Bug#609049: du: process_file: Assertion `level == prev_level - 1' failed

[Jim Meyering]

Jonathan Nieder wrote:
> Package: coreutils
> Version: 8.5-1
>
> Happy new year.  Today I ran
>
>  $ du -sk ~/*
>
> to see where all the space is going.  The traversal almost certainly
> was in ~/src, and being bored, I suspended it to examine that
> directory:
>
>  ^Z
>  [1]+  Stopped du -sk ~/*
>
> In another terminal I ran "cd ~/src && du -sk *".  ~/src/netbsd was
> very big, so I moved it (using chmod +w first) to ~/.cache on the same
> filesystem for future reference.  And now time to resume the toplevel
> search:
>
>  $ fg
>  du -sk ~/*
>  du: fts_read failed: No such file or directory
>  du: du.c:583: process_file: Assertion `level == prev_level - 1' failed.
>  Aborted (core dumped)
>
> Lacking debugging symbols, the core dump does not seem so useful, but
> I still have it.  libc6 is 2.11.2-7, filesystem is ext4, and the
> kernel is 2.6.37-rc7-686.  If there is additional information I could
> provide, just ask.

Thank you for the fine bug report.
Here's a fix:

While I do have a reproducer that will become a test suite addition
(coming soon), it relies on python (a first) and the python-inotify
package, so I'll have to be careful to skip the test when those
prerequisites aren't installed.  Also, there's an inherent race condition,
so I'll have to find the right compromise between absolute test-robustness
(too expensive in time and inodes) and reasonableness.

>From 92bbec0f697575a8bcbbe3fe0cb54b468c1022a4 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Sat, 8 Jan 2011 17:44:55 +0100
Subject: [PATCH] du: don't abort when a subdir is renamed out from under us

* NEWS (Bug fixes): Mention it.
* src/du.c (prev_level): Move declaration "up" to file-scope global.
(du_files): Reset prev_level to 0 upon abnormal fts_read termination.
Also improve a diagnostic.
Reported by Johathan Nieder in http://bugs.debian.org/609049
---
 NEWS |7 +++
 src/du.c |   15 +++
 2 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/NEWS b/NEWS
index 2a71ca6..793a8e6 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,13 @@ GNU coreutils NEWS-*- 
outline -*-

 * Noteworthy changes in release ?.? (-??-??) [?]

+** Bug fixes
+
+  du could abort with a failed assertion when part of the hierarchy being
+  traversed was moved to a higher level in the directory tree, and when
+  there was at least one more command line directory argument following
+  the one containing the moved sub-tree.
+

 * Noteworthy changes in release 8.9 (2011-01-04) [stable]

diff --git a/src/du.c b/src/du.c
index 77deb0c..4dc5456 100644
--- a/src/du.c
+++ b/src/du.c
@@ -63,8 +63,11 @@ extern bool fts_debug;
 /* A set of dev/ino pairs.  */
 static struct di_set *di_set;

-/* Define a class for collecting directory information. */
+/* Keep track of the preceding "level" (depth in hierarchy)
+   from one call of process_file to the next.  */
+static size_t prev_level;

+/* Define a class for collecting directory information. */
 struct duinfo
 {
   /* Size of files in directory.  */
@@ -399,7 +402,6 @@ process_file (FTS *fts, FTSENT *ent)
   struct duinfo dui;
   struct duinfo dui_to_print;
   size_t level;
-  static size_t prev_level;
   static size_t n_alloc;
   /* First element of the structure contains:
  The sum of the st_size values of all entries in the single directory
@@ -582,10 +584,15 @@ du_files (char **files, int bit_flags)
 {
   if (errno != 0)
 {
-  /* FIXME: try to give a better message  */
-  error (0, errno, _("fts_read failed"));
+  error (0, errno, _("%s: fts_read failed"),
+ quotearg_colon (fts->fts_path));
   ok = false;
 }
+
+  /* When exiting this loop early, be careful to reset the
+ global, prev_level, used in process_file.  Otherwise, its
+ (level == prev_level - 1) assertion could fail.  */
+  prev_level = 0;
   break;
 }
   FTS_CROSS_CHECK (fts);
--
1.7.3.5



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#608832: factor compiled without bignum support

[Jim Meyering]

Michael Stone wrote:
> On Mon, Jan 03, 2011 at 01:26:36PM -0600, C de-Avillez wrote:
>>If `factor' is built without using GNU MP, only single-precision
>>arithmetic is available, and so large numbers (typically 2^64 and
>>above) will not be supported.
>
> Pulling in a bignum library into the required list in order to support
> the factor(1) command seems a little obnoxious. If someone wants to
> design a coreutils-factor-bignum package that diverts factor or
> somesuch I guess I'd consider it, but is there really that much demand
> for this feature? (That is, would anyone really use it for anything
> other than novelty value?)

FTR, I noticed that Fedora does build a libgmp-enabled factor.
On F14, the factor binary is 52KB (vs. 32KB on debian unstable
x86_64 without the libgmp-hooked code).

The library itself weighs in at 380KB and has no dependency worth
worrying about:

$ ldd /usr/lib/libgmp.so.3.5.2
linux-vdso.so.1 =>  (0x7fff76dff000)
libc.so.6 => /lib/libc.so.6 (0x7fa5ba897000)
/lib64/ld-linux-x86-64.so.2 (0x7fa5bae74000)

I have no strong opinion, either way.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#436338: df: df rounds sizes incorrectly

[Jim Meyering]

Paul E Condon wrote:
> On 20101223_002101, Bob Proulx wrote:
...
>> > > To see the values that the kernel is returning to df's statfs call
>> > > please run the following command and report the contents of the file.
>> > >
>> > >   $ strace -v -e trace=statfs -o /tmp/df.strace.out df /dev/sde1
...
> Also, /tmp/df.strace.out is empty after running your suggested diagnostic.
> So is the above discussion an instance of sour grapes reasoning?

kernel differences?  The above works fine for me with unstable's
2.6.32-5-amd64 and self-compiled df from coreutils-8.7.x:

This variant might be more useful:

$ strace -v -e file -o /tmp/df.strace.out df /dev/sde1

If it too leaves the output file empty, try this:

$ strace -o /tmp/df.strace.out df /dev/sde1



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#578097: [parted-devel] [Parted-maintainers] Debian Bug #578097: No support for CMS-formatted disks

[Jim Meyering]

Stephen Powell wrote:
> On Tue, 14 Dec 2010 10:46:18 -0500 (EST), Jim Meyering wrote:
>> Stephen Powell wrote:
>>> How is your testing going?
>>
>> Finding time for parted has been a challenge recently.
>>>
>>> It's unclear to me from what you wrote.  Are you asking me
>>> to provide a test case?
>>
>> If you can, that'd be great and would accelerate the process.
>> I would no longer have an excuse not to push your patches.
>> I nearly pushed without a test case, but your changes are
>> large enough that I cannot do that in good conscience.
>>
>> I didn't want to require you to write a test, but in the end
>> my time constraints appear to be having the same effect.
>> Sorry about that.
>
> Hmm.  Well, as I see it, there are two main options.  I can
> write a test script, providing you with expected output,

Thanks for persevering.

> and you can run it too, checking to see that it produces the
> same output for you.  On the plus side, you testing it
> yourself gives you the maximum assurances that it is correct.
> On the minus side, you have to be able to reproduce my
> testing environment.  For example, do you have an s390
> environment in which to test?

Yes, if an s390x will do.  On one, uname -a reports this:
Linux xxx 2.6.32-19.el6.s390x #1 SMP Tue Mar 9 19:03:48 EST 2010 s390x s390x 
s390x GNU/Linux

> Does it have both CKD and FBA DASD?
> Does it run as a guest under z/VM?  Does your
> system have the DIAG driver?  Do you have
> access to a CMS system from which you can FORMAT and/or
> RESERVE the disks in CMS format?  The answer might not be
> "yes" to all those questions, and getting such an environment
> will take time.

I don't know, for any of the above.
If you describe how to determine the answer
in a way that is easy to automate, the goal
would be to run a script that checks for required
pieces and then runs whichever tests it can.

> The other option is for me to test it for you.  On the plus
> side, that requires less time on your part.  On the minus
> side, I will need a copy of the exact source code that you
> are using (which also carries a side benefit in that I will
> get to see if my patches applied to your source as expected.)
> Another minus is that you have to take my word for it, which
> is not as reassuring to you.  Which option do you choose?
> Or do you have a third option in mind?

The best is to put into version control some script
that someone with the right equipment can run via "make check"
(probably with a couple envvars specifying things that cannot
or must not be guessed, like the name of a partition to clobber,
or a device to format).

But if it's really not feasible, I'm sure we'll find a reasonable
middle ground.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#578097: [parted-devel] [Parted-maintainers] Debian Bug #578097: No support for CMS-formatted disks

[Jim Meyering]

Stephen Powell wrote:
> On Thu, 25 Nov 2010 10:49:06 -0500 (EST), Jim Meyering wrote:
>> Thanks for the fix.
>> Until today I had not compiled your changes on an s390x.
>> When I did, there were 3 other minor problems, addressed
>> by the patch below, just after the one you suggested.
>>
>> That got past "make check" as a non-privileged user.
>> However, running "make check" as root exercises several
>> other cases that cannot be tested as a regular user, and
>> at least two of those are failing.  Those failures
>> are not due to your changes, so as soon as I have
>> a test or two to exercise your changes I'll feel
>> comfortable pushing the result.  I expect to use
>> the same mechanism as used in a few other tests
>> whereby you specify a device and its size like this:
>>
>> sudo make check DEVICE_TO_ERASE=/dev/sdd DEVICE_TO_ERASE_SIZE=999MB
>>
>> But it won't be this weekend, so maybe next week...
>
> How is your testing going?  It's unclear to me from what

Finding time for parted has been a challenge recently.

> you wrote.  Are you asking me to provide a test case?

If you can, that'd be great and would accelerate the process.  I would
no longer have an excuse not to push your patches.  I nearly pushed
without a test case, but your changes are large enough that I cannot
do that in good conscience.

I didn't want to require you to write a test, but in the end my time
constraints appear to be having the same effect.  Sorry about that.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#605639: bug#7529: Bug#605639: deal better with different filesystem timestamp resolutions

[Jim Meyering]

Paul Eggert wrote:

> On 12/03/10 02:03, Jim Meyering wrote:
>
>> Would you mind adding a "Bug fixes" entry for this
>> in coreutils' NEWS file?  It'd be nice to commit that
>> along with an update of the gnulib submodule to the latest.
>
> Sure, done, with this notice:
>
>   cp -u no longer does unnecessary copying merely because the source
>   has finer-grained time stamps than the destination.

Thanks!

>> As for a test, it shouldn't be too hard to create a root-only test
>> on linux/gnu systems, since _PC_TIMESTAMP_RESOLUTION is not defined.
>> Create two loop-mounted file systems of types that have the desired
>> difference in time stamp resolution, and run commands like Dan did.
>
> Hmm, well, I can see a lot going wrong with that, such as garbage in the
> mount table if the test is interrupted.  (Also, there's the little problem
> that I lack root access on the hosts that I do builds on these days: does
> that get me off the hook?  :-)

I didn't mean to make it sound like a requirement.
I was merely thinking aloud about how I might do it.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#605639: bug#7529: Bug#605639: deal better with different filesystem timestamp resolutions

[Jim Meyering]

Paul Eggert wrote:
> Good eye!  Thanks for the bug report and example.  I installed
> the following one-byte patch into gnulib; please give it a try.
> It should propagate into coreutils the next time coreutils
> updates from gnulib.
>
> A test case for this would require two file systems, one with
> finer-grained time stamps than the other, where we can create
> files in the latter.  I suspect this goes beyond what coreutils's
> test cases can easily do.

Hi Paul,

Thanks for the speedy patch!
Would you mind adding a "Bug fixes" entry for this
in coreutils' NEWS file?  It'd be nice to commit that
along with an update of the gnulib submodule to the latest.

As for a test, it shouldn't be too hard to create a root-only test
on linux/gnu systems, since _PC_TIMESTAMP_RESOLUTION is not defined.
Create two loop-mounted file systems of types that have the desired
difference in time stamp resolution, and run commands like Dan did.

> Subject: [PATCH] utimecmp: fine-grained src to nearby coarse-grained dest
>
> * lib/utimecmp.c (utimecmp): When UTIMECMP_TRUNCATE_SOURCE is set,
> and the source is on a file system with higher-resolution time
> stamps, than the destination, and _PC_TIMESTAMP_RESOLUTION does
> not work, and the time stamps are close together, the algorithm to
> determine the exact resolution from the read-back mtime was buggy:
> it had a "!=" where it should have had an "==".  This bug has been
> in the code ever since it was introduced to gnulib.
> Problem reported by Dan Jacobson in
> .
...
> diff --git a/lib/utimecmp.c b/lib/utimecmp.c
> index 63a0c9a..8c3ca65 100644
> --- a/lib/utimecmp.c
> +++ b/lib/utimecmp.c
> @@ -325,7 +325,7 @@ utimecmp (char const *dst_name,
>
>  res = SYSCALL_RESOLUTION;
>
> -for (a /= res; a % 10 != 0; a /= 10)
> +for (a /= res; a % 10 == 0; a /= 10)
>{
>  if (res == BILLION)
>{



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#578097: [parted-devel] [Parted-maintainers] Debian Bug #578097: No support for CMS-formatted disks

[Jim Meyering]

Stephen Powell wrote:
...
> In the process of checking out the results, I discovered a bug in my code.
> In include/parted/vtoc.h, at or near line 105, as part of the structure
> declaration for ldl_volume_label, there is a line that says
>
> char ldl_version;   /* Version number, valid for ldl format  
> */
>
> It should say
>
> char ldl_version[1];/* Version number, valid for ldl format  
> */
>
> I was clued into this by a bunch of compile warnings in 
> libparted/labels/dasd.c
> complaining about conversion of integer to pointer without a cast,
> or something like that.

Hi Stephen,

Thanks for the fix.
Until today I had not compiled your changes on an s390x.
When I did, there were 3 other minor problems, addressed
by the patch below, just after the one you suggested.

That got past "make check" as a non-privileged user.
However, running "make check" as root exercises several
other cases that cannot be tested as a regular user, and
at least two of those are failing.  Those failures
are not due to your changes, so as soon as I have
a test or two to exercise your changes I'll feel
comfortable pushing the result.  I expect to use
the same mechanism as used in a few other tests
whereby you specify a device and its size like this:

sudo make check DEVICE_TO_ERASE=/dev/sdd DEVICE_TO_ERASE_SIZE=999MB

But it won't be this weekend, so maybe next week...


>From a0fd447e13ec76fa16a51bfd88b67af705d08c05 Mon Sep 17 00:00:00 2001
From: Stephen Powell 
Date: Thu, 25 Nov 2010 15:52:21 +0100
Subject: [PATCH 1/2] tweak ldl_version member decl

* include/parted/vtoc.h [struct ldl_volume_label] (ldl_version):
Declare as array of length 1, not as a scalar.
---
 include/parted/vtoc.h |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/include/parted/vtoc.h b/include/parted/vtoc.h
index 5ed40b5..d79ce39 100644
--- a/include/parted/vtoc.h
+++ b/include/parted/vtoc.h
@@ -102,7 +102,7 @@ struct __attribute__ ((packed)) ldl_volume_label {
char vollbl[4]; /* Label identifier ("LNX1" in EBCDIC)   */
 char volid[6];  /* Volume identifier */
 char res3[69];  /* Reserved field*/
-char ldl_version;   /* Version number, valid for ldl format  */
+char ldl_version[1];/* Version number, valid for ldl format  */
 u_int64_t formatted_blocks;  /* valid when ldl_version >= "2" (in
 EBCDIC)  */
 };
--
1.7.3.2.765.g642a8


>From 668956027eebd9204e486fdd75e67a981ead5ac5 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Thu, 25 Nov 2010 15:57:00 +0100
Subject: [PATCH 2/2] s390: avoid warnings

* libparted/labels/vtoc.c (vtoc_read_volume_label): Remove decl
of unused var.
* libparted/labels/fdasd.c (fdasd_valid_vtoc_pointer): Return
a value also when successful.
* libparted/labels/dasd.c (dasd_alloc_metadata): Initialize "part"
to NULL solely to avoid a spurious used-uninitialized warning.
---
 libparted/labels/dasd.c  |2 +-
 libparted/labels/fdasd.c |1 +
 libparted/labels/vtoc.c  |1 -
 3 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/libparted/labels/dasd.c b/libparted/labels/dasd.c
index 8114347..bca1662 100644
--- a/libparted/labels/dasd.c
+++ b/libparted/labels/dasd.c
@@ -906,7 +906,7 @@ dasd_alloc_metadata (PedDisk* disk)
PedSector vtoc_end;
LinuxSpecific* arch_specific;
DasdDiskSpecific* disk_specific;
-   PedPartition* part;
+   PedPartition* part = NULL; /* initialize solely to placate gcc */
PedPartition* new_part2;
PedSector trailing_meta_start, trailing_meta_end;
struct fdasd_anchor anchor;
diff --git a/libparted/labels/fdasd.c b/libparted/labels/fdasd.c
index 8da7b1f..6d708f6 100644
--- a/libparted/labels/fdasd.c
+++ b/libparted/labels/fdasd.c
@@ -707,6 +707,7 @@ fdasd_valid_vtoc_pointer(fdasd_anchor_t *anc, unsigned long 
b, int fd)
return 0;

fdasd_error(anc, wrong_disk_format, _("Invalid VTOC."));
+   return 1;
 }

 /*
diff --git a/libparted/labels/vtoc.c b/libparted/labels/vtoc.c
index dab5181..cf2990e 100644
--- a/libparted/labels/vtoc.c
+++ b/libparted/labels/vtoc.c
@@ -290,7 +290,6 @@ vtoc_read_volume_label (int f, unsigned long vlabel_start,
bogus_label_t *bogus_ptr = &mybogus;
vollabel_t *union_ptr = &bogus_ptr->actual_label;
volume_label_t *cdl_ptr = &union_ptr->cdl;
-   cms_volume_label_t *cms_ptr = &union_ptr->cms;

PDEBUG
int rc;
--
1.7.3.2.765.g642a8



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#381091: Word splitting in 'export' arguments (Re: bash: expands $@ within double-quotes inconsistently)

[Jim Meyering]

Eric Blake wrote:
...
> I would love to require altered word splitting behavior for export and
> readonly (the standard doesn't touch local yet).  In fact, it has
> independently come up on the GNU coreutils development lists that the
> standard is long overdue for requiring shells to support 'local' in
> shell functions, and that standardizing local would be another reason
> why word splitting rules would need to be modified.
>
> Consider:
>
> $ mkdir /tmp/t && printf '#!/bin/sh\necho you lose\n' > /tmp/t/cat &&
> chmod a+x /tmp/t/cat
> $ dash -c 'f() { b="1 PATH=/tmp/t"; local z=$b; cat whatever; }; f'
> you lose
> $ bash -c 'f() { b="1 PATH=/tmp/t"; local z=$b; cat whatever; }; f'
> cat: whatever: No such file or directory
>
> Would you like me to take a stab at writing the bug report(s) to add
> support for 'local' in the next revision of the standard, and to require
> the ksh/bash behavior of suppressing word splitting after shell builtins
> that take arguments which can modify the current set of
> shell/environment variables?

That would be most welcome.

To dot the 'i's in my pseudo-exploit, here's part of what I wrote
privately last night:

> On Sun Nov 07 09:56:36 2010, j...@meyering.net wrote:
...
>> The inconsistency with "local" I reported here
>>
>> http://thread.gmane.org/gmane.comp.shells.dash/419
>>
>> may be exploited whenever the suffix of a value like
>> $b's below can be controlled by malicious parties.
>>
>> In a bash or zsh shell script that does this,
>>
>> local v1=$v2
>>
>> $v2 may safely contain any input whatsoever.
>> Quotes are not required around $v2, just as they're never required
>> for a posix-conforming (without "local") assignment of that form.
>>
>> However, with dash, crafted values of $v2 can make dash
>> set any attacker-chosen variable to any attacker-chosen value.
>>
>> Here's a trivial demo:
>>
>> $ mkdir /tmp/t && printf '#!/bin/sh\necho you lose\n' > /tmp/t/cat &&
>> chmod a+x /tmp/t/cat
>> $ dash -c 'f() { b="1 PATH=/tmp/t"; local z=$b; cat whatever; }; f'
>> you lose
>>
>> Luckily, it seems that no evaluation is performed.
>> I.e., no immediate nastiness via the likes of this:
>>
>> $ dash -c 'f() { b="1 t=\`echo\`"; local z=$b; echo "t=<$t>"; }; f'
>> t=<`echo`>
>>
>> It's a good thing that "local" is relatively unportable,
>> and thus is not yet used as much as it might otherwise be...
>>
>> However, even on Fedora and RHEL-based systems, I regularly
>> test shell scripts for portability by using dash (and not always
>> in a sealed-off sandbox VM), so fixing this would make me feel
>> a tiny bit safer.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#578097: [parted-devel] [Parted-maintainers] Debian Bug #578097: No support for CMS-formatted disks

[Jim Meyering]

Stephen Powell wrote:
> As promised earlier today, I now have the following patch files on my
> web site ready for download:
>
> http://www.wowway.com/~zlinuxman/parted/vtoc.h.diff   (apply to 
> include/parted/vtoc.h)
> http://www.wowway.com/~zlinuxman/parted/dasd.c.diff   (apply to 
> libparted/labels/dasd.c)
> http://www.wowway.com/~zlinuxman/parted/fdasd.c.diff  (apply to 
> libparted/labels/fdasd.c)
> http://www.wowway.com/~zlinuxman/parted/vtoc.c.diff   (apply to 
> libparted/labels/vtoc.c)
>
> I have not made any updates to NEWS.  You can edit that as you see
> fit.  These are ordinary "context diffs", with three lines of context.
> I don't understand this "git" stuff.  I've never used it.  Sorry.
> My starting point was the Debian source package for parted, version 2.2-5.
...

Thanks for your patience, Stephen.
I've applied your patches locally, made some tiny changes
and included the result below.
(I moved 2 decls "down", and made indentation more consistent)

Two things remain to be done:

  - add a sentence or two for NEWS; the idea is to be as brief as possible
  while still giving enough of the details so people don't have to
  go look at the commit logs or -- gasp! -- the code.

  - a recipe or two (the more the merrier) for tests that fail
  without your patch, but pass with it.
  For a change of this size and complexity, I would be remiss
  not to include at least one or two patches.  It's ok (expected
  even) that the tests will run only on an s390 and will be skipped
  on other systems.

I've lightly modified words from your email to make the log
message below.  If you'd like something different there, let me know.

I haven't tested these changes at all, but should be able
to do so next week.

Jim

>From 7fd0768feda02e0d51956cddac0eaf6d80fb4ebe Mon Sep 17 00:00:00 2001
From: Stephen Powell 
Date: Sat, 8 May 2010 09:39:24 +0200
Subject: [PATCH] s390: improve/correct DASD support

The long title would be "corrections to partition size and location
calculations for type 1 partitions for s390 dasd".

This could be treated as two separate fixes, one to make corrections
for LDL formatted disks and one to add support for CMS formatted
disks.  I see CMS formatted disks as a variant of LDL formatted
disks, with the additional twist that CMS formatted disks can
be reserved or recomped, which LDL formatted disks cannot be.
This affects the size and location of the partition.

With these patches, parted matches the behavior of the Linux kernel
in recognizing partitions on CMS- and LDL-formatted disks, as documented
in the Linux kernel source code in routine fs/partitions/ibm.c.
Calculation of the metadata has also been changed so that parted will
show no free space on such a disk.  In some cases there are now two
non-contiguous metadata extents: one at the beginning of the disk and
one at the end.

As before, parted only supports CKD DASD using the ECKD driver.
FBA DASD and CKD DASD using the DIAG driver are still not supported.

In my regression testing I have discovered some problems in the area
of recognizing file system options.  However, since I can duplicate
these errors on a version of parted which does not contain my changes,
I have concluded that my changes did not cause this and therefore
this is an unrelated bug.
---
 include/parted/vtoc.h|   56 -
 libparted/labels/dasd.c  |  106 -
 libparted/labels/fdasd.c |   15 ---
 libparted/labels/vtoc.c  |   14 --
 4 files changed, 166 insertions(+), 25 deletions(-)

diff --git a/include/parted/vtoc.h b/include/parted/vtoc.h
index 6b41584..5ed40b5 100644
--- a/include/parted/vtoc.h
+++ b/include/parted/vtoc.h
@@ -49,6 +49,8 @@ typedef struct cchhbcchhb_t;
 typedef struct cchh cchh_t;
 typedef struct labeldatelabeldate_t;
 typedef struct volume_label volume_label_t;
+typedef struct cms_volume_label cms_volume_label_t;
+typedef struct ldl_volume_label ldl_volume_label_t;
 typedef struct extent   extent_t;
 typedef struct dev_constdev_const_t;
 typedef struct format1_labelformat1_label_t;
@@ -81,7 +83,7 @@ struct __attribute__ ((packed)) labeldate {

 struct __attribute__ ((packed)) volume_label {
char volkey[4]; /* volume key = volume label */
-   char vollbl[4]; /* volume label  */
+   char vollbl[4]; /* volume label ("VOL1" in EBCDIC)   */
char volid[6];  /* volume identifier */
u_int8_t security;  /* security byte */
cchhb_t vtoc;   /* VTOC address  */
@@ -93,6 +95,58 @@ struct __attribute__ ((packed)) volume_label {
char res2[4];   /* reserved  */
char lvtoc[14]; /* owner code for LVTOC  */
  

Bug#598438: coreutils: new du corrupted filesystem warning breaks scripts

[Jim Meyering]

Graham Cobb wrote:
> Package: coreutils
> Version: 8.5-1
> Severity: important
>
> Like many systems, my filesystem contains loops due to bind mounts (in my case
> it is for scratchbox, the reporter of bug 563254 has a different reason, I
> have had other reasons in the past as well).
>
> I use a backup script called dt-s3-backup.sh, which has been working for some
> time but has suddenly broken due to the change in du's handling of filesystem
> loops.
>
> Although I exclude the looped files from my backup, the script uses a du
> command which crosses the filesystem loop. Previously, du silently ignored the
> files below the loop -- the script depends on this behaviour and breaks
> completely now that du returns an error.
>
> I know about the filesystem loop.  It in no way represents any filesystem
> corruption and the change in behaviour should not have been made without some
> way to disable it.  I do not object to the new behaviour being the default but
> there needs to be an option to disable the error and silently ignore the
> loop.
>
> A simple test case to reproduce this problem:
>
> mkdir -p /tmp/du-test/loop
> echo hello >/tmp/du-test/a
> mount -o bind /tmp /tmp/du-test/loop
> du -s /tmp
> umount /tmp/du-test/loop
>
> Note: there is no way to avoid this problem using the --exclude option as
> excluding the directory does not stop du reporting the error.  This can be 
> seen
> by replacing the du command above with:
>
> du --exclude /tmp/du-test/loop -s /tmp
>
> It is possible to use "--exclude=/tmp/du-test" but that excludes too much --
> the du command needs to count files such as /tmp/du-test/a.

Thank you for the report and reproducer.
That is a bug in how du's --exclude works in coreutils-8.1 through 8.5.
It was fixed for the next release via this change by Paul Eggert:

du: tune, and fix some -L bugs with dangling or cyclic symlinks
http://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=77428214f13e1

However, this particular fix was not noted in NEWS,
so I'm adding an entry for it:

>From 65b50c6cdd4a140a2974907423a1df1692b6d3ae Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Thu, 30 Sep 2010 14:24:42 +0200
Subject: [PATCH] maint: mention the du-exclude--vs--cycle-dir fix

* NEWS (Bug fixes): Mention the du-exclude--vs--cycle-dir fix.
Reported by Graham Cobb in http://bugs.debian.org/598438,
that bug was fixed by the 2010-07-24 commit, 77428214f,
"du: tune, and fix some -L bugs with dangling or cyclic symlinks"
---
 NEWS |4 
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/NEWS b/NEWS
index 898ee09..11a8b74 100644
--- a/NEWS
+++ b/NEWS
@@ -11,6 +11,10 @@ GNU coreutils NEWS-*- 
outline -*-
   du -H and -L now consistently count pointed-to files instead of
   symbolic links, and correctly diagnose dangling symlinks.

+  du --ignore=D now ignores directory D even when that directory is
+  found to be part of a directory cycle.  Before, du would issue a
+  "NOTIFY YOUR SYSTEM MANAGER" diagnostic and fail.
+
   tac would perform a double-free when given an input line longer than 16KiB.
   [bug introduced in coreutils-8.3]

--
1.7.3.293.gca9a76



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#367011: still reproducible (coreutils: od --width=8 works but od -w 8 fails)

[Jim Meyering]

Timo Juhani Lindfors wrote:
> The bug is still reproducible in both lenny and sid.
>
> (sid)li...@sauna:~$ od -w 8 /etc/debian_version

That is not a valid use of od's -w option.  There may be no space
between the short-named -w and its "optional argument".
Here's the description from "info coreutils od":

  `-w[N]'
  `--width[=N]'
   Dump `n' input bytes per output line.  This must be a multiple of
   the least common multiple of the sizes associated with the
   specified output types.

   If this option is not given at all, the default is 16.  If N is
   omitted, the default is 32.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#594666: /usr/bin/tac: tac aborts

[Jim Meyering]

Salvo Tomaselli wrote:
> Package: coreutils
> Version: 8.5-1
> Severity: normal
> File: /usr/bin/tac
>
> Tac aborts when using it on a particular file.
>
> *** glibc detected *** tac: double free or corruption (top): 
> 0x025c5030 ***
...

Thank you for the report!
That is indeed a bug in the very latest.

For a stand-alone, minimal demonstrator, run this:

valgrind tac <(printf %0$((2**14 + 1))d 0) > /dev/null

It prints this:

 Invalid free() / delete / delete[]
at 0x4A04D72: free (vg_replace_malloc.c:325)
by 0x402294: main (tac.c:669)
  Address 0x4c30040 is 0 bytes inside a block of size 16,388 free'd
at 0x4A05255: realloc (vg_replace_malloc.c:476)
by 0x4117B8: xrealloc (xmalloc.c:57)
by 0x401A68: tac_seekable (tac.c:319)
by 0x402379: main (tac.c:515)

Here is a fix:

>From b3959fc691e606857a3c6e9b316ec34819972245 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Sat, 28 Aug 2010 17:45:29 +0200
Subject: [PATCH] tac: avoid double free

* src/tac.c (main): Reading a line longer than 16KiB would cause
tac to realloc its primary buffer.  Then, just before exit, tac
would mistakenly free the original (now free'd) buffer.
This bug was introduced by commit be6c13e7, "maint: always free a
buffer, to avoid even semblance of a leak".
* NEWS (Bug fixes): Mention it.
* tests/misc/tac (double-free): New test, to exercise this.
Reported by Salvo Tomaselli in <http://bugs.debian.org/594666>.
---
 NEWS   |3 +++
 src/tac.c  |6 --
 tests/misc/tac |6 ++
 3 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/NEWS b/NEWS
index 85f55a2..f29d311 100644
--- a/NEWS
+++ b/NEWS
@@ -11,6 +11,9 @@ GNU coreutils NEWS-*- 
outline -*-
   du -H and -L now consistently count pointed-to files instead of
   symbolic links, and correctly diagnose dangling symlinks.

+  tac would perform a double-free when given an input line longer than 16KiB.
+  [bug introduced in coreutils-8.3]
+
 ** New features

   cp now accepts the --attributes-only option to not copy file data,
diff --git a/src/tac.c b/src/tac.c
index cec9736..859e006 100644
--- a/src/tac.c
+++ b/src/tac.c
@@ -633,7 +633,6 @@ main (int argc, char **argv)
   if (! (read_size < half_buffer_size && half_buffer_size < G_buffer_size))
 xalloc_die ();
   G_buffer = xmalloc (G_buffer_size);
-  void *buf = G_buffer;
   if (sentinel_length)
 {
   strcpy (G_buffer, separator);
@@ -666,6 +665,9 @@ main (int argc, char **argv)
   error (0, errno, "-");
   ok = false;
 }
-  free (buf);
+
+  size_t offset = sentinel_length ? sentinel_length : 1;
+  free (G_buffer - offset);
+
   exit (ok ? EXIT_SUCCESS : EXIT_FAILURE);
 }
diff --git a/tests/misc/tac b/tests/misc/tac
index 7631049..4130c00 100755
--- a/tests/misc/tac
+++ b/tests/misc/tac
@@ -24,6 +24,9 @@ my $prog = 'tac';

 my $bad_dir = 'no/such/dir';

+# This must be longer than 16KiB to trigger the double free in coreutils-8.5.
+my $long_line = 'o' x (16 * 1024 + 1);
+
 my @Tests =
 (
   ['segfault', '-r', {IN=>"a\n"}, {IN=>"b\n"}, {OUT=>"a\nb\n"}],
@@ -67,6 +70,9 @@ my @Tests =
{ERR_SUBST => "s,`$bad_dir': .*,...,"},
{ERR => "$prog: cannot create temporary file in ...\n"},
{EXIT => 1}],
+
+  # coreutils-8.5's tac would double-free its primary buffer.
+  ['double-free', {IN=>$long_line}, {OUT=>$long_line}],
 );

 @Tests = triple_test \...@tests;
--
1.7.2.2.510.g7180a



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#586301: diffutils: doubled slashes in --recursive output (path//path) (fwd)

[Jim Meyering]

Jim Meyering wrote:

> Paul Eggert wrote:
>
>> On 08/14/10 23:46, Jim Meyering wrote:
>>> [PATCH] diff -r: avoid printing excess slashes in concatenated file names
>>
>> Thanks, that looks good to me.  Hmm, at some point we should
>> replace zalloc with xzalloc too, I suppose, and maybe get
>> rid of diffutils' 'concat' function.
>
> Oh, I see that my change has rendered concat unused as well.
> I'll amend it before pushing.  Thanks.

Hm not quite.
There's one use left.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#586301: diffutils: doubled slashes in --recursive output (path//path) (fwd)

[Jim Meyering]

Paul Eggert wrote:

> On 08/14/10 23:46, Jim Meyering wrote:
>> [PATCH] diff -r: avoid printing excess slashes in concatenated file names
>
> Thanks, that looks good to me.  Hmm, at some point we should
> replace zalloc with xzalloc too, I suppose, and maybe get
> rid of diffutils' 'concat' function.

Oh, I see that my change has rendered concat unused as well.
I'll amend it before pushing.  Thanks.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#586301: diffutils: doubled slashes in --recursive output (path//path) (fwd)

[Jim Meyering]

Paul Eggert wrote:
> On 08/14/10 23:46, Jim Meyering wrote:
>> [PATCH] diff -r: avoid printing excess slashes in concatenated file names
>
> Thanks, that looks good to me.  Hmm, at some point we should
> replace zalloc with xzalloc too, I suppose, and maybe get
> rid of diffutils' 'concat' function.

Hi Paul,

Thanks for the quick feedback.
FYI, my patch was relative to a slightly dated repository
and no longer applied to "master".  Also, it included an unnecessary
(but harmless) mode change for an unrelated test.

Better one below (which I'll push).

When I rebased it to work on latest, I found that the
result no longer builds with my compilation options:

cc1: warnings being treated as errors
dir.c: In function 'diff_dirs':
dir.c:278: error: declaration of 'cmp' shadows a parameter [-Wshadow]
dir.c:199: error: shadowed declaration is here [-Wshadow]
make[2]: *** [dir.o] Error 1

Without thinking hard (meaning it might be better to change
the shadowed parameter name, "cmp"), I made this local change
to work around that:

diff --git a/src/dir.c b/src/dir.c
index 1b078ab..df89431 100644
--- a/src/dir.c
+++ b/src/dir.c
@@ -275,10 +275,10 @@ diff_dirs (struct comparison const *cmp,
   *p && compare_names (*p, greater_name) == 0;
   p++)
{
- int cmp = file_name_cmp (*p, greater_name);
- if (0 <= cmp)
+ int c = file_name_cmp (*p, greater_name);
+ if (0 <= c)
{
- if (cmp == 0)
+ if (c == 0)
{
  memmove (lesser + 1, lesser,
   (char *) p - (char *) lesser);


>From 53de393ca335e77f22d3789100734c87868f12b3 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Sat, 14 Aug 2010 17:13:28 -0500
Subject: [PATCH] diff -r: avoid printing excess slashes in concatenated file 
names

* bootstrap.conf (gnulib_modules): Add filenamecat.
* src/diff.c: Include "filenamecat.h".
(compare_files): Use file_name_concat, rather than dir_file_pathname.
* src/util.c (dir_file_pathname): Remove now-unused function.
* src/diff.h: Remove its declaration.
* tests/excess-slash: New script to test for this.
* tests/Makefile.am (TESTS): Add it.
Forwarded by Santiago Vila from ,
reported by Jari Aalto.
---
 bootstrap.conf |1 +
 gnulib |2 +-
 src/diff.c |7 ---
 src/diff.h |1 -
 src/util.c |   12 
 tests/Makefile.am  |3 ++-
 tests/excess-slash |   19 +++
 7 files changed, 27 insertions(+), 18 deletions(-)
 create mode 100644 tests/excess-slash

diff --git a/bootstrap.conf b/bootstrap.conf
index 2f104b3..ea1744c 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -33,6 +33,7 @@ extensions
 fcntl
 fdl
 file-type
+filenamecat
 fnmatch-gnu
 getopt
 gettext
diff --git a/gnulib b/gnulib
index 880f2b6..0c6cf5a 16
--- a/gnulib
+++ b/gnulib
@@ -1 +1 @@
-Subproject commit 880f2b69df57af506439d6aaf1fe185a6f960e43
+Subproject commit 0c6cf5ab43555377b99d94febb2d6f23fc3d2cb0
diff --git a/src/diff.c b/src/diff.c
index cc1b611..807d38f 100644
--- a/src/diff.c
+++ b/src/diff.c
@@ -27,6 +27,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -1067,9 +1068,9 @@ compare_files (struct comparison const *parent,
   else
 {
   cmp.file[0].name = free0
-   = dir_file_pathname (parent->file[0].name, name0);
+   = file_name_concat (parent->file[0].name, name0, NULL);
   cmp.file[1].name = free1
-   = dir_file_pathname (parent->file[1].name, name1);
+   = file_name_concat (parent->file[1].name, name1, NULL);
 }

   /* Stat the files.  */
@@ -1156,7 +1157,7 @@ compare_files (struct comparison const *parent,
   char const *fnm = cmp.file[fnm_arg].name;
   char const *dir = cmp.file[dir_arg].name;
   char const *filename = cmp.file[dir_arg].name = free0
-   = dir_file_pathname (dir, last_component (fnm));
+   = file_name_concat (dir, last_component (fnm), NULL);

   if (STREQ (fnm, "-"))
fatal ("cannot compare `-' to a directory");
diff --git a/src/diff.h b/src/diff.h
index 71b33f4..97f8d96 100644
--- a/src/diff.h
+++ b/src/diff.h
@@ -349,7 +349,6 @@ void print_sdiff_script (struct change *);
 extern char const change_letter[4];
 extern char const pr_program[];
 char *concat (char const *, char const *, char const *);
-char *dir_file_pathname (char const *, char const *);
 bool lines_differ (char const *, char const *);
 lin translate_line_number (struct file_data const *, lin);
 struct change *find_change (struct change *);
diff --git a/src/util.c b/src/util.c
index 3be03e9..dd14e65 100644
--- a/src/util.c
+++ b/src/util.c

Bug#586301: diffutils: doubled slashes in --recursive output (path//path) (fwd)

[Jim Meyering]

Santiago Vila wrote:
> Received this from the Debian BTS.
>
> (I've given the reply-to a suitable value).
>
> -- Forwarded message --
> From: Jari Aalto 
> To: Debian Bug Tracking System 
> Date: Fri, 18 Jun 2010 11:54:30 +0300
> Subject: Bug#586301: diffutils: doubled slashes in --recursive output
> (path//path)
>
> Package: diffutils
> Version: 1:3.0-1
> Severity: minor
>
>
> If given a directory with a trailing slash (as it occurs after TAB
> completion in Shell):
>
> $ diff -rq .  /etc/apache2/
> Files ./apache2.conf and /etc/apache2//apache2.conf differ
> Only in /etc/apache2//conf.d: ganglia-webfrontend
> Only in /etc/apache2//conf.d: javascript-common.conf
> Only in /etc/apache2//conf.d: munin
>
> SUGGESTION
>
> Strip trailing slashes from command line arguments before using them for
> output.

Thanks to both Santiago and Jari.

Here's a quick stand-alone demo:
Before it would do this:

  $ ./diff -r a b/
  Only in b//f: g
  [Exit 1]

Now it works how we'd like:

  $ ./diff -r a b/
  Only in b/f: g
  [Exit 1]


Here's a proposed patch:

>From 280d5a1e6fd399af2fa5c9d0cfb3d88878bc2380 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Sat, 14 Aug 2010 16:23:51 -0500
Subject: [PATCH] diff -r: avoid printing excess slashes in concatenated file 
names

* bootstrap.conf (gnulib_modules): Add filenamecat.
* src/diff.c: Include "filenamecat.h".
(compare_files): Use file_name_concat, rather than dir_file_pathname.
* src/util.c (dir_file_pathname): Remove now-unused function.
* src/diff.h: Remove its declaration.
* tests/excess-slash: New script to test for this.
* tests/Makefile.am (TESTS): Add it.
Forwarded by Santiago Vila from ,
reported by Jari Aalto.
---
 bootstrap.conf  |1 +
 src/diff.c  |7 ---
 src/diff.h  |1 -
 src/util.c  |   12 
 tests/Makefile.am   |1 +
 tests/excess-slash  |   19 +++
 6 files changed, 25 insertions(+), 16 deletions(-)
 create mode 100755 tests/excess-slash
 mode change 100644 => 100755 tests/no-newline-at-eof

diff --git a/bootstrap.conf b/bootstrap.conf
index 71fbd3f..33448fb 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -33,6 +33,7 @@ extensions
 fcntl
 fdl
 file-type
+filenamecat
 fnmatch-gnu
 getopt
 gettext
diff --git a/src/diff.c b/src/diff.c
index cc1b611..807d38f 100644
--- a/src/diff.c
+++ b/src/diff.c
@@ -27,6 +27,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -1067,9 +1068,9 @@ compare_files (struct comparison const *parent,
   else
 {
   cmp.file[0].name = free0
-   = dir_file_pathname (parent->file[0].name, name0);
+   = file_name_concat (parent->file[0].name, name0, NULL);
   cmp.file[1].name = free1
-   = dir_file_pathname (parent->file[1].name, name1);
+   = file_name_concat (parent->file[1].name, name1, NULL);
 }

   /* Stat the files.  */
@@ -1156,7 +1157,7 @@ compare_files (struct comparison const *parent,
   char const *fnm = cmp.file[fnm_arg].name;
   char const *dir = cmp.file[dir_arg].name;
   char const *filename = cmp.file[dir_arg].name = free0
-   = dir_file_pathname (dir, last_component (fnm));
+   = file_name_concat (dir, last_component (fnm), NULL);

   if (STREQ (fnm, "-"))
fatal ("cannot compare `-' to a directory");
diff --git a/src/diff.h b/src/diff.h
index 71b33f4..97f8d96 100644
--- a/src/diff.h
+++ b/src/diff.h
@@ -349,7 +349,6 @@ void print_sdiff_script (struct change *);
 extern char const change_letter[4];
 extern char const pr_program[];
 char *concat (char const *, char const *, char const *);
-char *dir_file_pathname (char const *, char const *);
 bool lines_differ (char const *, char const *);
 lin translate_line_number (struct file_data const *, lin);
 struct change *find_change (struct change *);
diff --git a/src/util.c b/src/util.c
index 3be03e9..dd14e65 100644
--- a/src/util.c
+++ b/src/util.c
@@ -756,18 +756,6 @@ zalloc (size_t size)
   memset (p, 0, size);
   return p;
 }
-
-/* Yield the newly malloc'd pathname
-   of the file in DIR whose filename is FILE.  */
-
-char *
-dir_file_pathname (char const *dir, char const *file)
-{
-  char const *base = last_component (dir);
-  size_t baselen = base_len (base);
-  bool omit_slash = baselen == 0 || base[baselen - 1] == '/';
-  return concat (dir, "/" + omit_slash, file);
-}
 
 void
 debug_script (struct change *sp)
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 6a4858c..5a5f0f4 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -1,6 +1,7 @@
 TESTS = \
   basic \
   binary \
+  excess-slash \
   help-version \
   function-line-vs-leading-space \
   label-vs-func\
diff --git a/tests/excess-slash b/tests/excess-slash
new file mode 100755

Bug#578097: [parted-devel] [Parted-maintainers] Debian Bug #578097: No support for CMS-formatted disks

[Jim Meyering]

Stephen Powell wrote:
> I haven't heard anything for a while.  I just thought I would touch base
> after a couple of months.  Is anything happening?

Hi Stephen.
I'll be able to review your patches in 2-3 weeks.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#579948: [parted-devel] Some debugging info

[Jim Meyering]

Colin Watson wrote:

> On Mon, Jun 14, 2010 at 08:40:12PM +0200, Karel Zak wrote:
>> On Mon, Jun 14, 2010 at 01:06:09PM +0100, Colin Watson wrote:
>> > parted-devel, can anyone comment on this?  It seems to me that either
>> > (a) _get_lax_constraint should be using ped_alignment_any for its start
>> > alignment, rather than aligning to sectors * heads boundaries, or (b)
>> > Sun labels really and truly require cylinder alignment, in which case
>> > requests to use optimal alignment shouldn't be honoured.
>>
>> The begin of the partition has to be defined in cylinders:
>>
>> struct __attribute__ ((packed)) _SunRawPartition {
>> u_int32_t   start_cylinder; /* where the part starts... 
>> */
>> u_int32_t   num_sectors;/* ...and it's length */
>> };
>>
>> IMHO it does not make sense to use non-cylinder alignment here.
>
> In that case, I think that the sun part of
> 723ea23c5df68cbe67d1f518ef484f4c77f516fa should be reverted.  CCing Hans
> since that was his change.
>
>
> From: Colin Watson 
> Date: Tue, 15 Jun 2010 19:49:40 +0100
> Subject: [PATCH] sun: revert "implement disk flag operations"
>
> This reverts the libparted/labels/sun.c part of
> 723ea23c5df68cbe67d1f518ef484f4c77f516fa.  Sun disk labels do not appear
> to be able to handle non-cylinder alignment
> (http://bugs.debian.org/579948).

Thanks!  I've applied that.
However, since it would have made a test fail,
I first applied this change:


>From a582ca642f4817dd02e65a3ecc55e951008969b2 Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Sat, 26 Jun 2010 09:22:59 +0200
Subject: [PATCH] tests: adjust sun-partition-creating test to conform

* tests/t4000-sun-raid-type.sh: Adjust partition size so the
end falls on a cylinder boundary.
---
 tests/t4000-sun-raid-type.sh |4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/t4000-sun-raid-type.sh b/tests/t4000-sun-raid-type.sh
index 5ef8ab4..853809e 100755
--- a/tests/t4000-sun-raid-type.sh
+++ b/tests/t4000-sun-raid-type.sh
@@ -26,7 +26,7 @@ ss=$sector_size_

 N=2000 # number of sectors
 dev=sun-disk-file
-exp="BYT;\n---:${N}s:file:$ss:$ss:sun:;\n1:0s:50s:51s"
+exp="BYT;\n---:${N}s:file:$ss:$ss:sun:;\n1:0s:127s:128s"
 test_expect_success \
 'create an empty file as a test disk' \
 'dd if=/dev/zero of=$dev bs=${ss}c count=$N 2> /dev/null'
@@ -38,7 +38,7 @@ test_expect_success 'check for empty output' 'compare out 
/dev/null'

 test_expect_success \
 'create a single partition' \
-'parted -s $dev unit s mkpart ext2 0s 50s > out 2>&1'
+'parted -s $dev unit s mkpart ext2 0s 127s > out 2>&1'
 test_expect_success 'check for empty output' 'compare out /dev/null'

 test_expect_success \
--
1.7.1.755.geb6f2



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#586581: coreutils: comm - Slight formatting error in manual page

[Jim Meyering]

Jari Aalto wrote:
> Package: coreutils
> Version: 8.5-1
> Severity: minor
>
> The comm(1) manual page reads:
>
>EXAMPLES
>comm -12 file1 file2
>   Print only lines present in both file1 and file2.
>
>>  comm -3
>> file1 file2  Print lines in file1 not in file2, and vice versa.
>
> The last two lines should probably read:
>
>comm -3 file1 file2
>   Print lines in file1 not in file2, and vice versa.

Thank you.
I'll fix it like this:

>From 1419af82433027b03c2d49f7a401beb485bcbe2a Mon Sep 17 00:00:00 2001
From: Jim Meyering 
Date: Mon, 21 Jun 2010 08:55:22 +0200
Subject: [PATCH] doc: fix comm's --help output so we generate a better man page

* src/comm.c (usage): Don't align example comments in --help output,
since the extra space (sequence of two spaces) there would be
interpreted by help2man and induce an unwanted line break
in the resulting man page.  Reported by Jari Aalto.
---
 src/comm.c |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/src/comm.c b/src/comm.c
index 4e4aad5..ff42802 100644
--- a/src/comm.c
+++ b/src/comm.c
@@ -139,7 +139,7 @@ Note, comparisons honor the rules specified by 
`LC_COLLATE'.\n\
 \n\
 Examples:\n\
   %s -12 file1 file2  Print only lines present in both file1 and file2.\n\
-  %s -3  file1 file2  Print lines in file1 not in file2, and vice versa.\n\
+  %s -3 file1 file2  Print lines in file1 not in file2, and vice versa.\n\
 "),
   program_name, program_name);
   emit_ancillary_info ();
--
1.7.1.664.gf1fb2



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#585509: coreutils: missing prototypes when building test-linkat in gnulib testsuite

[Jim Meyering]

Steve Langasek wrote:
> Package: coreutils
> Version: 8.5-1
> Severity: minor
> Tags: patch
> User: ubuntu-de...@lists.ubuntu.com
> Usertags: origin-ubuntu maverick ubuntu-patch
>
> Hi Michael,
>
> The latest coreutils package FTBFS in Ubuntu on armel, powerpc, and sparc
> due to a reproducible failure in the gnulib testsuite (test-linkat).  We
> haven't sorted out why the build failure was limited to these three
> architectures, or why it didn't also affect Debian, but it's resolved by the
> attached patch which corrects a missing header include that was noticed
> while debugging.  With the prototypes in place, the package again passes
> the test suite on all architectures.  So while the package isn't currently
> failing to build in Debian, it might well do so in the future; and fixing
> the missing prototypes is obviously correct, so I would suggest applying the
> patch in Debian (and upstream) as well.
>
> The accompanying changelog entry for this patch is:
>
>   [ John Rigby ]
>   * debian/patches/99_stat_prototype_for_linkat.dpatch: Add missing header
> include for stat() prototype.  LP: #591968.

Thanks.  It was a problem in gnulib.
I've applied the patch:

http://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=85b6cb48e0f3bbf83420d9d4cd2d49049da64a87



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#578097: [parted-devel] [Parted-maintainers] Debian Bug #578097: No support for CMS-formatted disks

[Jim Meyering]

Stephen Powell wrote:
> Well, my paperwork just came through today for assignment of copyright.
> And it's just in time.  Here is a link to my second round of enhancements.
>
>http://www.wowway.com/~zlinuxman/parted/dasd_complete.diff
>
> These changes require my previous changes as a prerequisite.  Let me
> know if you have any trouble with them.  The enhancement is
>
> * Support all valid combinations of DASD format (CDL, LDL, and CMS),
>   DASD type (CKD and FBA), and DASD driver (ECKD, FBA, and DIAG) supported
>   by the Linux kernel.  Only CDL format on CKD DASD using the ECKD
>   driver is supported for read/write operations on partitions
>   (create, delete, move, and resize).  For all other combinations
>   of DASD format, DASD type, and DASD driver that are supported by the
>   Linux kernel, parted recognizes the pre-existing implicit partition;
>   but it does not allow that partition to be deleted, moved, or resized;
>   nor does it allow the creation of another partition on that DASD device.

Hi Stephen,

Now that your paperwork has gone through and I've released parted-2.3,
we can work on getting your changes committed.

Would you please outline a few sets of shell/parted commands
that I can run on a dasd disk to exercise as much of this new
code as possible?

Thanks for all your work.

Jim

P.S. Please excuse the high latency.
I'll be available only sporadically for the next couple months.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#583198: coreutils: tail does not obey --sleep-interval option

[Jim Meyering]

Michael Stone wrote:
> On Wed, May 26, 2010 at 09:06:10AM -0600, Bob Proulx wrote:
>>I don't understand your comment in a bug report.  Being more
>>responsive seems like a good thing to me and good expected behavior.
>>Could you say a few words about why you are filing a bug report on
>>tail being more responsive?
>
> I can imagine that if someone is manually reviewing the output, they
> may wish to have a certain interval to inspect new data before it is
> scrolled by additional data.

In that case, use the deliberately undocumented ---disable-inotify
option that was introduced in coreutils-7.6.
With it, tail will sleep between iterations, as it did before.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#578097: [parted-devel] [Parted-maintainers] Debian Bug #578097: No support for CMS-formatted disks

[Jim Meyering]

Stephen Powell wrote:
> Praise the Lord!  I do believe I've got it now.  Parted now supports all
> DASD types (CKD and FBA), drivers (ECKD, FBA, and DIAG) and disk formats

Nice.
Sounds promising.

> (CDL, LDL, and CMS) that are supported by the Linux kernel on the s390
...
> I intend to put the changed code through some fairly extensive
> final tests next week, and after that I will upload a unified diff
> with all my changes.  It will, however, have my original changes

Considering the number of different formats involved,
I'd guess that your changes include several bug fixes.

If you can find a way to make this into a series of patches,
I would be most grateful.  Teasing one big unified diff into
a series of largely-independent patches can be tedious,
but when done properly, it makes reviewing and maintenance
(e.g., future bisection) a lot easier.

> published via the context diffs as a prerequisite.  Sorry.  I'm
> a novice at this.  In fact, my only qualifications for making these
> changes seems to be that (a) I need the new functionality, and
> (b) I have the necessary hardware with which to test.  Programming
> skills, Linux knowledge, C knowledge, knowledge of "how things are
> done procedurally", etc. are probably all sub-par.  I ask for your
> patience and forbearance as I slowly and painfully learn all that
> stuff.

Also, please include as much as you can in the way of test recipes.
I.e., for each bug, what commands did you run to trigger it,
and what were the symptoms before it was fixed.

Thanks!



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#578097: [parted-devel] [Parted-maintainers] Debian Bug #578097: No support for CMS-formatted disks

[Jim Meyering]

Stephen Powell wrote:
...
> Now, for the specific "bullet points" for NEWS:
>
> o Fixed a bug in which the starting block for the implicit partition
>   of LDL disks was miscalculated if the disk was formatted at a block
>   size other than 4096.  The program now calculates the starting block
>   of the partition correctly, regardless of whether the disk has a
>   block size of 512, 1024, 2048, or 4096.  (These are all the valid
>   block sizes.)
>
> o Enhanced LDL support to use the "formatted blocks" field of an LDL-
>   format disk label (for version 2 and above LDL formats) to determine
>   the size of the partition, rather than relying on disk geometry.
>   This will allow parted to determine the correct partition size if,
>   for example, an image copy backup of a 3390-3 volume is taken at
>   the home site and restored to a 3390-9 volume at a disaster recovery
>   hot site.  The extra space at the end of the 3390-9 volume is not
>   part of the partition, and parted now knows that.
>
> o Added support for CMS-formatted minidisks, both non-reserved and
>   reserved, provided that they are on CKD DASD.  CMS minidisks on
>   FBA DASD are not supported at this time.  Also, CMS minidisks on
>   CKD DASD which use the DIAG driver are not supported either.  The
>   ECKD driver must be used.
>
> The changes to function dasd_alloc_metadata in dasd.c are necessary to
> support both the bug fix and the enhancements.  It is important that
> no metadata/partition overlap occurs, and it is important that all
> sectors on an LDL or CMS disk be covered either by the partition or by
> metadata.

Thanks for the detailed explanation.
I'll add it to NEWS once the paperwork has gone through.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#581065: /bin/rmdir: recursive option for rmdir

[Jim Meyering]

Paul Wise wrote:
> Package: coreutils
> Version: 8.5-1
> Severity: wishlist
> File: /bin/rmdir
> X-Debbugs-CC: bug-coreut...@gnu.org
>
> I would like to be able to do a recursive rmdir. This would save typing
> for the cases where I want to remove a whole empty dir hierarchy.
>
> Currently I do this:
>
> rmdir --parents foo/bar/baz
>
> I'd like to just do this:
>
> rmdir --recursive foo
>
> or this:
>
> rmdir -r foo
>
> Both of these commands should be equivalent to but significantly less
> typing than this command:
>
> find foo -depth -type d -empty -delete

Thanks for the suggestion, but there's no need
to add an option to rmdir, when you can accomplish
the same thing with a simple shell function or alias:

  rmdir_r() { find "$@" -depth -type d -empty -delete; }

Of course, you might want to extend that to give
its own --help and to diagnose a missing argument.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#578097: [parted-devel] [Parted-maintainers] Debian Bug #578097: No support for CMS-formatted disks

[Jim Meyering]

Stephen Powell wrote:
> As promised earlier today, I now have the following patch files on my
> web site ready for download:
>
> http://www.wowway.com/~zlinuxman/parted/vtoc.h.diff   (apply to 
> include/parted/vtoc.h)
> http://www.wowway.com/~zlinuxman/parted/dasd.c.diff   (apply to 
> libparted/labels/dasd.c)
> http://www.wowway.com/~zlinuxman/parted/fdasd.c.diff  (apply to 
> libparted/labels/fdasd.c)
> http://www.wowway.com/~zlinuxman/parted/vtoc.c.diff   (apply to 
> libparted/labels/vtoc.c)
>
> I have not made any updates to NEWS.  You can edit that as you see
> fit.  These are ordinary "context diffs", with three lines of context.
> I don't understand this "git" stuff.  I've never used it.  Sorry.
> My starting point was the Debian source package for parted, version 2.2-5.
>
> To my way of thinking, these patches are logically one fix, which I
> would describe as "corrections to partition size and location
> calculations for type 1 partitions for s390 dasd".  If you want to

I don't often see context diffs, as opposed to unified diffs.
Most people prefer to read the latter.  You can generate them with diff -u
if you're using GNU diff, or with git diff if you ever get around
to using git.

I've confirmed that those patches do apply locally and removed a couple
of trialing spaces so "make syntax-check" still passes.

Once I have confirmation that the FSF has received your paperwork,
I'll review and test.

One way that you can help in the mean time is by writing a slightly
more detailed summary of what changed.  For example
  * what is it that would fail without your patch, but works with it?
  * how does your improvement make things better?
That is the sort of information I'd like to put in NEWS.
Does this count as a bug fix?  It seems so, but there's also
the free-space change that looks more like an improvement than a bug fix.
I haven't looked at details yet...



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org