Bug#669084: grep does not match anything when reading from stdin

[Tino Keitel]

Package: grep
Version: 2.11-2
Severity: important

grep does not match anything when reading from stdin. Reading from a file
works.

$ echo foo  foo
$ grep . foo
foo
$ cat foo | grep .
$ echo foo | grep .
$ 

-- System Information:
Debian Release: wheezy/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 3.3.0 (SMP w/4 CPU cores)
Locale: LANG=C, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages grep depends on:
ii  dpkg  1.16.2
ii  install-info  4.13a.dfsg.1-10
ii  libc6 2.13-27

grep recommends no packages.

Versions of packages grep suggests:
ii  libpcre3  1:8.30-4

-- no debconf information



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#669084: grep does not match anything when reading from stdin

[Aníbal Monsalve Salazar]

On Tue, Apr 17, 2012 at 09:23:58AM +0200, Tino Keitel wrote:
Package: grep
Version: 2.11-2
Severity: important

grep does not match anything when reading from stdin. Reading from a file
works.

$ echo foo  foo
$ grep . foo
foo
$ cat foo | grep .
$ echo foo | grep .
$ 

It works for me, please see below.

dpkg -l grep | grep grep
ii  grep  2.11-2  GNU grep, egrep and fgrep

echo foo  foo

grep . foo
foo

grep . foo
foo

cat foo | grep .
foo

echo foo | grep .
foo

-- System Information:
Debian Release: wheezy/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 3.3.0 (SMP w/4 CPU cores)
Locale: LANG=C, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages grep depends on:
ii  dpkg  1.16.2
ii  install-info  4.13a.dfsg.1-10
ii  libc6 2.13-27

grep recommends no packages.

Versions of packages grep suggests:
ii  libpcre3  1:8.30-4

-- no debconf information



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#669084: grep does not match anything when reading from stdin

[Tino Keitel]

On Tue, Apr 17, 2012 at 18:07:29 +1000, Aníbal Monsalve Salazar wrote:
 On Tue, Apr 17, 2012 at 09:23:58AM +0200, Tino Keitel wrote:
 Package: grep
 Version: 2.11-2
 Severity: important
 
 grep does not match anything when reading from stdin. Reading from a file
 works.
 
 $ echo foo  foo
 $ grep . foo
 foo
 $ cat foo | grep .
 $ echo foo | grep .
 $ 
 
 It works for me, please see below.
 
 dpkg -l grep | grep grep
 ii  grep  2.11-2  GNU grep, egrep and fgrep
 
 echo foo  foo
 
 grep . foo
 foo
 
 grep . foo
 foo
 
 cat foo | grep .
 foo
 
 echo foo | grep .
 foo

I found the reason:

$ echo $GREP_OPTIONS
--exclude=*.svn* -d skip -D skip --color=auto

Without -D skip it works. It looks like device detection is broken
regarding stdin.

Here is some additional info on the breakage:

$ grep .  foo
foo
$ cat foo | grep .
$

Regards,
Tino



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#669084: grep does not match anything when reading from stdin

[Tino Keitel]

On Tue, Apr 17, 2012 at 10:23:28 +0200, Tino Keitel wrote:
 On Tue, Apr 17, 2012 at 18:07:29 +1000, Aníbal Monsalve Salazar wrote:
  On Tue, Apr 17, 2012 at 09:23:58AM +0200, Tino Keitel wrote:
  Package: grep
  Version: 2.11-2
  Severity: important
  
  grep does not match anything when reading from stdin. Reading from a file
  works.
  
  $ echo foo  foo
  $ grep . foo
  foo
  $ cat foo | grep .
  $ echo foo | grep .
  $ 
  
  It works for me, please see below.

The version from testing (2.10-1) works here, too.

Regards,
Tino



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#669084: grep does not match anything when reading from stdin

[Tino Keitel]

On Tue, Apr 17, 2012 at 10:25:03 +0200, Tino Keitel wrote:
 On Tue, Apr 17, 2012 at 10:23:28 +0200, Tino Keitel wrote:
  On Tue, Apr 17, 2012 at 18:07:29 +1000, Aníbal Monsalve Salazar wrote:
   On Tue, Apr 17, 2012 at 09:23:58AM +0200, Tino Keitel wrote:
   Package: grep
   Version: 2.11-2
   Severity: important
   
   grep does not match anything when reading from stdin. Reading from a file
   works.
   
   $ echo foo  foo
   $ grep . foo
   foo
   $ cat foo | grep .
   $ echo foo | grep .
   $ 
   
   It works for me, please see below.
 
 The version from testing (2.10-1) works here, too.

This snipset from the 2.11 main.c is very likely the culprit. The
'S_ISFIFO (stats-stat.st_mode)' causes stdin to be skipped if it is a
pipe.

if ((directories == SKIP_DIRECTORIES  S_ISDIR (stats-stat.st_mode))
|| (devices == SKIP_DEVICES  (S_ISCHR (stats-stat.st_mode)
|| S_ISBLK (stats-stat.st_mode)
|| S_ISSOCK (stats-stat.st_mode)
|| S_ISFIFO (stats-stat.st_mode
  {
if (file)
  close (desc);
return 1;
  }



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#669084: grep does not match anything when reading from stdin

[Aníbal Monsalve Salazar]

On Tue, Apr 17, 2012 at 11:08:16AM +0200, Tino Keitel wrote:
On Tue, Apr 17, 2012 at 10:25:03 +0200, Tino Keitel wrote:
On Tue, Apr 17, 2012 at 10:23:28 +0200, Tino Keitel wrote:
On Tue, Apr 17, 2012 at 18:07:29 +1000, Aníbal Monsalve Salazar wrote:
On Tue, Apr 17, 2012 at 09:23:58AM +0200, Tino Keitel wrote:
Package: grep
Version: 2.11-2
Severity: important

grep does not match anything when reading from stdin. Reading from
a file works.

$ echo foo  foo
$ grep . foo
foo
$ cat foo | grep .
$ echo foo | grep .
$ 

It works for me, please see below.

The version from testing (2.10-1) works here, too.

This snipset from the 2.11 main.c is very likely the culprit. The
'S_ISFIFO (stats-stat.st_mode)' causes stdin to be skipped if it is a
pipe.

if ((directories == SKIP_DIRECTORIES  S_ISDIR (stats-stat.st_mode))
|| (devices == SKIP_DEVICES  (S_ISCHR (stats-stat.st_mode)
|| S_ISBLK (stats-stat.st_mode)
|| S_ISSOCK (stats-stat.st_mode)
|| S_ISFIFO (stats-stat.st_mode
  {
if (file)
  close (desc);
return 1;
  }

Hello Jim,

grep-2.11 introduced a regression related to the -D skip parameter
option.

With grep-2.10 (or less than 2.10):

$ echo foo | grep -D skip foo; echo $?
foo
0

With grep-2.11:

$ echo foo | grep -D skip foo; echo $?
1

Timo suggests that the possible culprit is the code above.

The Debian bug report is at http://bugs.debian.org/668585

Cheers,

Anibal



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#669084: grep does not match anything when reading from stdin

[Aníbal Monsalve Salazar]

On Tue, Apr 17, 2012 at 08:13:49PM +1000, Aníbal Monsalve Salazar wrote:
On Tue, Apr 17, 2012 at 11:08:16AM +0200, Tino Keitel wrote:
On Tue, Apr 17, 2012 at 10:25:03 +0200, Tino Keitel wrote:
On Tue, Apr 17, 2012 at 10:23:28 +0200, Tino Keitel wrote:
On Tue, Apr 17, 2012 at 18:07:29 +1000, Aníbal Monsalve Salazar wrote:
On Tue, Apr 17, 2012 at 09:23:58AM +0200, Tino Keitel wrote:
Package: grep
Version: 2.11-2
Severity: important

grep does not match anything when reading from stdin. Reading from
a file works.

$ echo foo  foo
$ grep . foo
foo
$ cat foo | grep .
$ echo foo | grep .
$ 

It works for me, please see below.

The version from testing (2.10-1) works here, too.

This snipset from the 2.11 main.c is very likely the culprit. The
'S_ISFIFO (stats-stat.st_mode)' causes stdin to be skipped if it is a
pipe.

if ((directories == SKIP_DIRECTORIES  S_ISDIR (stats-stat.st_mode))
|| (devices == SKIP_DEVICES  (S_ISCHR (stats-stat.st_mode)
|| S_ISBLK (stats-stat.st_mode)
|| S_ISSOCK (stats-stat.st_mode)
|| S_ISFIFO (stats-stat.st_mode
  {
if (file)
  close (desc);
return 1;
  }

Hello Jim,

grep-2.11 introduced a regression related to the -D skip parameter
option.

With grep-2.10 (or less than 2.10):

$ echo foo | grep -D skip foo; echo $?
foo
0

With grep-2.11:

$ echo foo | grep -D skip foo; echo $?
1

Timo suggests that the possible culprit is the code above.

The Debian bug report is at http://bugs.debian.org/668585

Typo. The correct address is http://bugs.debian.org/669084

Cheers,

Anibal



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#669084: grep does not match anything when reading from stdin

[Jim Meyering]

Aníbal Monsalve Salazar wrote:
...
 Hello Jim,

 grep-2.11 introduced a regression related to the -D skip parameter
 option.

 With grep-2.10 (or less than 2.10):

 $ echo foo | grep -D skip foo; echo $?
 foo
 0

 With grep-2.11:

 $ echo foo | grep -D skip foo; echo $?
 1

 Timo suggests that the possible culprit is the code above.

 The Debian bug report is at http://bugs.debian.org/668585

Hi Aníbal

Thanks for passing that along.
Here's a lightly-tested patch:

From 415e4e69c8e8c0db288205b30ab2b6a337f62d38 Mon Sep 17 00:00:00 2001
From: Jim Meyering meyer...@redhat.com
Date: Tue, 17 Apr 2012 13:37:10 +0200
Subject: [PATCH] grep: --devices=ACTION (-D) no longer affects stdin
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* doc/grep.texi (File and Directory Selection): Clarify this point,
documenting the stdin exemption.
* src/main.c (grepdesc): Ignore skip-related options when reading
from standard input.
* tests/skip-device: New file.  Test for the above.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Bug fixes): Mention it, and add a few [fixed in ...] notes.
Reported by Sven Joachim in  http://bugs.debian.org/668585,
and forwarded to bug-grep by Aníbal Monsalve Salazar.
---
 NEWS  |7 ++-
 doc/grep.texi |1 +
 src/main.c|9 +
 tests/Makefile.am |1 +
 tests/skip-device |   11 +++
 5 files changed, 24 insertions(+), 5 deletions(-)
 create mode 100755 tests/skip-device

diff --git a/NEWS b/NEWS
index b219b65..a065394 100644
--- a/NEWS
+++ b/NEWS
@@ -4,13 +4,17 @@ GNU grep NEWS-*- outline 
-*-

 ** Bug fixes

+  echo P|grep --devices=skip P once again prints P, as it did in 2.10
+  [bug introduced in grep-2.11]
+
   grep no longer segfaults with -r --exclude-dir and no file operand.
   I.e., :|grep -r --exclude-dir=D PAT would segfault.
+  [bug introduced in grep-2.11]

   Recursive grep now uses fts for directory traversal, so it can
   handle much-larger directories without reporting things like File
   name too long, and it can run much faster when dealing with large
-  directory hierarchies.
+  directory hierarchies. [bug present since the beginning]

   grep -E 'a{10}' now reports an overflow error rather than
   silently acting like grep -E 'a\{10}'.
@@ -27,6 +31,7 @@ GNU grep NEWS-*- outline 
-*-
   use -R if you prefer the old behavior of following all symlinks and
   defaulting to reading all devices.

+
 * Noteworthy changes in release 2.11 (2012-03-02) [stable]

 ** Bug fixes
diff --git a/doc/grep.texi b/doc/grep.texi
index 1840e21..000a844 100644
--- a/doc/grep.texi
+++ b/doc/grep.texi
@@ -614,6 +614,7 @@ By default, devices are read if they are on the command 
line or if the
 @option{-R} (@option{--dereference-recursive}) option is used, and are
 skipped if they are encountered recursively and the @option{-r}
 (@option{--recursive}) option is used.
+This option has no effect on a file that is read via standard input.

 @item -d @var{action}
 @itemx --directories=@var{action}
diff --git a/src/main.c b/src/main.c
index 82cae33..c5a8489 100644
--- a/src/main.c
+++ b/src/main.c
@@ -1367,10 +1367,11 @@ grepdesc (int desc, int command_line)
 suppressible_error (filename, errno);
   return status;
 }
-  if ((directories == SKIP_DIRECTORIES  S_ISDIR (st.st_mode))
-  || ((devices == SKIP_DEVICES
-   || (devices == READ_COMMAND_LINE_DEVICES  !command_line))
-   is_device_mode (st.st_mode)))
+  if (desc != STDIN_FILENO
+   ((directories == SKIP_DIRECTORIES  S_ISDIR (st.st_mode))
+  || ((devices == SKIP_DEVICES
+   || (devices == READ_COMMAND_LINE_DEVICES  !command_line))
+   is_device_mode (st.st_mode
 goto closeout;

   /* If there is a regular file on stdout and the current file refers
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 13061fe..d0d622b 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -78,6 +78,7 @@ TESTS =   \
   r-dot\
   repetition-overflow  \
   reversed-range-endpoints \
+  skip-device  \
   sjis-mb  \
   spencer1 \
   spencer1-locale  \
diff --git a/tests/skip-device b/tests/skip-device
new file mode 100755
index 000..efb529f
--- /dev/null
+++ b/tests/skip-device
@@ -0,0 +1,11 @@
+#!/bin/sh
+# grep must ignore --devices=ACTION (-D) when reading stdin
+# For grep-2.11, this test would fail.
+
+. ${srcdir=.}/init.sh; path_prepend_ ../src
+
+# Test both with no file argument, and with -.
+echo foo | grep -q -D skip foo - || fail=1
+echo foo | grep -q --devices=skip foo || fail=1
+
+Exit $fail
--
1.7.10.208.gb4267



--
To UNSUBSCRIBE, 

Bug#669084: grep does not match anything when reading from stdin

[Aníbal Monsalve Salazar]

On Tue, Apr 17, 2012 at 01:45:43PM +0200, Jim Meyering wrote:
Aníbal Monsalve Salazar wrote:
...
Hello Jim,

grep-2.11 introduced a regression related to the -D skip parameter
option.

With grep-2.10 (or less than 2.10):

$ echo foo | grep -D skip foo; echo $?
foo
0

With grep-2.11:

$ echo foo | grep -D skip foo; echo $?
1

Tino suggests that the possible culprit is the code above.

The Debian bug report is at http://bugs.debian.org/668585

Typo. The correct address is http://bugs.debian.org/669084

Hi Aníbal

Thanks for passing that along.
Here's a lightly-tested patch:

Thanks. I'll build a new Debian package soon.

From 415e4e69c8e8c0db288205b30ab2b6a337f62d38 Mon Sep 17 00:00:00 2001
From: Jim Meyering meyer...@redhat.com
Date: Tue, 17 Apr 2012 13:37:10 +0200
Subject: [PATCH] grep: --devices=ACTION (-D) no longer affects stdin
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* doc/grep.texi (File and Directory Selection): Clarify this point,
documenting the stdin exemption.
* src/main.c (grepdesc): Ignore skip-related options when reading
from standard input.
* tests/skip-device: New file.  Test for the above.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Bug fixes): Mention it, and add a few [fixed in ...] notes.
Reported by Sven Joachim in  http://bugs.debian.org/668585,

Reported by Tino Keitel in  http://bugs.debian.org/669084,

and forwarded to bug-grep by Aníbal Monsalve Salazar.
[...]



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#669084: grep does not match anything when reading from stdin

[Tino Keitel]

On Tue, Apr 17, 2012 at 13:45:43 +0200, Jim Meyering wrote:
 Aníbal Monsalve Salazar wrote:
 ...
  Hello Jim,
 
  grep-2.11 introduced a regression related to the -D skip parameter
  option.
 
  With grep-2.10 (or less than 2.10):
 
  $ echo foo | grep -D skip foo; echo $?
  foo
  0
 
  With grep-2.11:
 
  $ echo foo | grep -D skip foo; echo $?
  1
 
  Timo suggests that the possible culprit is the code above.
 
  The Debian bug report is at http://bugs.debian.org/668585
 
 Hi Aníbal
 
 Thanks for passing that along.
 Here's a lightly-tested patch:

Hi,

I had problems extracting the patch from the mail, so I had to use
copypaste.  Also, the patch portion in main.c looked totally different
than my main.c from 2.11.  I added the desc != STDIN_FILENO and the
brackets manually and it fixed the -D skip behaviour.

Regards,
Tino



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#669084: grep does not match anything when reading from stdin

[Tino Keitel]

On Tue, Apr 17, 2012 at 14:23:48 +0200, Tino Keitel wrote:

 Hi,
 
 I had problems extracting the patch from the mail, so I had to use
 copypaste.  Also, the patch portion in main.c looked totally different

Forgot the patch, it's attached.

Regards,
Tino
--- grep-2.11_orig/src/main.c	2012-04-17 10:08:47.626634496 +0200
+++ grep-2.11/src/main.c	2012-04-17 14:17:13.293392207 +0200
@@ -1251,11 +1251,12 @@
   return 1;
 }
 
-  if ((directories == SKIP_DIRECTORIES  S_ISDIR (stats-stat.st_mode))
-  || (devices == SKIP_DEVICES  (S_ISCHR (stats-stat.st_mode)
+  if (desc != STDIN_FILENO 
+  ((directories == SKIP_DIRECTORIES  S_ISDIR (stats-stat.st_mode))
+   || (devices == SKIP_DEVICES  (S_ISCHR (stats-stat.st_mode)
   || S_ISBLK (stats-stat.st_mode)
   || S_ISSOCK (stats-stat.st_mode)
-  || S_ISFIFO (stats-stat.st_mode
+  || S_ISFIFO (stats-stat.st_mode)
 {
   if (file)
 close (desc);

Bug#669084: grep does not match anything when reading from stdin

[Jim Meyering]

Aníbal Monsalve Salazar wrote:
...
Tino suggests that the possible culprit is the code above.

The Debian bug report is at http://bugs.debian.org/668585

 Typo. The correct address is http://bugs.debian.org/669084

Thanks!
I've adjusted the log:

  * NEWS (Bug fixes): Mention it, and add a few [fixed in ...] notes.
  Reported by Tino Keitel in http://bugs.debian.org/669084,
  and forwarded to bug-grep by Aníbal Monsalve Salazar.



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#669084: grep does not match anything when reading from stdin

[Jim Meyering]

Tino Keitel wrote:
 On Tue, Apr 17, 2012 at 14:23:48 +0200, Tino Keitel wrote:

 Hi,

 I had problems extracting the patch from the mail, so I had to use
 copypaste.  Also, the patch portion in main.c looked totally different

 Forgot the patch, it's attached.

 Regards,
 Tino

 --- grep-2.11_orig/src/main.c 2012-04-17 10:08:47.626634496 +0200
 +++ grep-2.11/src/main.c  2012-04-17 14:17:13.293392207 +0200
 @@ -1251,11 +1251,12 @@
return 1;
  }

 -  if ((directories == SKIP_DIRECTORIES  S_ISDIR (stats-stat.st_mode))
 -  || (devices == SKIP_DEVICES  (S_ISCHR (stats-stat.st_mode)
 +  if (desc != STDIN_FILENO 
 +  ((directories == SKIP_DIRECTORIES  S_ISDIR (stats-stat.st_mode))
 +   || (devices == SKIP_DEVICES  (S_ISCHR (stats-stat.st_mode)
|| S_ISBLK (stats-stat.st_mode)
|| S_ISSOCK (stats-stat.st_mode)
 -  || S_ISFIFO (stats-stat.st_mode
 +  || S_ISFIFO (stats-stat.st_mode)

Thanks.  Your change is relative to 2.11, while mine is
relative to the latest in the upstream git repository:

  http://git.savannah.gnu.org/cgit/grep.git/



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org