Re: installable gnulib library

[Gary V. Vaughan]

Hi Eric,

Thanks for the feedback.

On 27 Sep 2008, at 01:04, Eric Blake wrote:


Gary V. Vaughan  gnu.org> writes:

I have an (undoubtedly caffeine induced) idea... why not enhance
gnulib to provide a shim that sits between the system libraries and
client code that wants to use it without shipping (another copy) of
the particular parts it depends upon?


I am somewhat skeptical of this idea, for sevaral reasons.

Gnulib mutates too quickly.  How would you guarantee that the  
interface the
user installed on their system is new enough to meet your package's  
needs,
particularly when you look through NEWS at how often gnulib  
interfaces change?


Yes, and actually, that bugs me quite a lot.  While gnulib is still  
finding
its feet, that's still acceptable, but at some point (the core  
interfaces of)
gnulib really ought to settle down.  And I've moaned on and off that  
it really
would be very nice (for the sake of being able to rebuild old tarballs  
of
gnulib client releases at least) if gnulib made an occasional formal  
release.


Gnulib includes a LOT of source code hacks (witness all the  
#include_next
header replacements).  A library only solves API needs, but it does  
not solve
source code needs, so packages will still end up shipping lots of  
code from
gnulib.  It's easier to ship all of the source code, than to try and  
pick out

the parts not already covered by an installed library.


That's a good point.

Are any of the modules that do that core functionality?  Is there any  
means
to provide the same functionality without the source code hacks?  If  
not,
maybe it is still a win to have gnulib installed as a library, plus a  
small
amount of the essential glue that packages can choose to add instead  
of the

whole 2MiB of autoconf support code?

Think about gnulib-tool --avoid - how does one exclude entry points  
from an
installed library, if they explicitly want to avoid a particular  
gnulib module
(for example, because of licensing incompatibility).  Source code  
distribution

makes this task a lot easier.


I'm not proposing that we disregard the current system and replace it  
with
an installed library!  Merely that there is an option to install  
gnulib (or

in light of your observations, some core part of gnulib) so that client
packages can choose between full on source code integration plus  
autotools,

or in some cases be able to rely on just the installed gnulib and some
Makefiles.

In that vein, not all gnulib modules play together nicely.  For  
example, just
from today, vasnprintf now behaves differently on mingw depending on  
whether
you are also using sigpipe-die, and this difference is at the source  
code level
(faking SIGPIPE on mingw comes with a lot of baggage).  The choice  
to use
sigpipe-die is a conscious decision of the maintainer, but if gnulib  
were
installed, then you would have to have multiple library versions to  
expose

parameterization to what used to be a compile-time decision.


Bleh.  That's a tough one for sure.  But not enough to discourage me  
from

trying just yet :)


I think http://www.gnu.org/software/gnulib/manual/gnulib.html#Introduction
covers most of these points, as well as mentioning the difficulty in  
trying to
make libiberty an installable library as precedent for the paradigm  
of keeping

gnulib source-only.


Even having reread that, I don't think there's any reason not to try to
find a line between what parts of gnulib can reasonably go into an  
installed
library, and what parts are complex enough that they require configure  
time

assistance to work on the host system.

Cheers,
Gary
--
Email me:  [EMAIL PROTECTED](\(\
Read my blog:  http://blog.azazil.net  ( o.O)
And my other blog: http://www.machaxor.net  (uu )o
...and my book:http://sources.redhat.com/autobook  ("("_)

 

Re: installable gnulib library

[Gary V. Vaughan]

Hi Bruce,

On 27 Sep 2008, at 01:02, Bruce Korb wrote:

On Fri, Sep 26, 2008 at 9:25 AM, Gary V. Vaughan <[EMAIL PROTECTED]> wrote:
I have an (undoubtedly caffeine induced) idea... why not enhance  
gnulib to
provide a shim that sits between the system libraries and client  
code that
wants to use it without shipping (another copy) of the particular  
parts it

depends upon?

If we add a compile-and-install-everything-as-a-library mode to  
gnulib, many
GNU packages could stop distributing the MiBs of autotools'  
generated glue,
and instead go with a much lighter build system that simply depends  
on
gnulib API semantics, and requires that gnulib be installed.  For  
many

modern systems, the installed gnulib might turn out to be vanishingly
small... and for many old and broken systems, having just one copy  
of gnulib
in shared memory ought to provide a nice improvement to speed and  
memory

utilisation.

Please tell me I'm crazy right now.  Or at least before I waste the  
next few

months of my free time figuring out how to do it.


Hey Gary,

Long ago, far away and years ago, that is *precisely* the point I  
was making.

(Remember the autotool bake-off contest?)


I remember it well.  Code Sourcery.


 My thinking then (and now) is that
you just make your project depend upon the pre-installation of this  
common
glue package.  Glue layer not installed?  Then your's won't install  
either.


Yep.

I think some folks won't buy into it because they don't want another  
dependency

for their projects.  (I think that was the main objection to my
proposal for that long-ago "contest".


Really?  I thought the main objection was that writing the glue layer  
would

take a man-decade of effort.

The thing that struck me a few days ago was that gnulib actually  
provides
all the code that such a shim would need, only it has to be copied  
piece-meal
into the packages that want to take advantage of it as it stands.  
Although it

does that very well.

Outside of the autotool and GNU communities, the vast majority of  
people I

encounter strongly dislike the difficulty of using autotools in client
packages, and the megabyte or two of shell scripts and m4 macros that  
every
GNU package seems to carry around with it.  Wouldn't it be great if we  
only

had to run that stuff *once* (when installing gnulib) and every other
command-line package that paid attention could remove all of their  
autotools

glue?


 Can't seem to put together the right Google search to
dredge it up again...)


I'm not sure it's still around in that form any more :)


Anyway, at least _I'd_ like to see it.  :-D


And no one is telling me I'm an idiot yet... looks like a green light  
to me :D


Cheers,
Gary
--
Email me:  [EMAIL PROTECTED](\(\
Read my blog:  http://blog.azazil.net  ( o.O)
And my other blog: http://www.machaxor.net  (uu )o
...and my book:http://sources.redhat.com/autobook  ("("_)

Re: fts: avoid O(n^2) ext3/ext4 performance hit on million+-entry dirs

[Matthew Woehlke]

Bruno Haible wrote:

Jim Meyering wrote:

+   if (nitems > _FTS_INODE_SORT_DIR_ENTRIES_THRESHOLD
+   && !sp->fts_compar
+   && dirent_inode_sort_may_be_useful (sp)) {
+   sp->fts_compar = fts_compare_ino;
+   head = fts_sort (sp, head, nitems);
+   sp->fts_compar = NULL;
+   }


This uses fts_sort, which uses qsort, which is O(n^2) in the worst case.
(Even the implementation in glibc can be O(n^2), if it guesses that mergesort
takes too much memory.)

Couldn't this be avoided by using Paul's mpsort module, and changing fts_sort
to allocate 50% more room, as scratch space for mpsort?


You're talking about inode numbers (i.e. fixed-sized keys), yes? Could 
this be a case for a radix sort?


--
Matthew
Please do not quote my e-mail address unobfuscated in message bodies.
--
"Who wants to sing?" -- Orcs (Warcraft II)

Re: improve "posix-modules --version"

[Bruno Haible]

Hi Eric,

> > ! gnulib-tool (GNU $package $date)$version
> 
> Oops - some over-active copy-n-paste.

Oh, indeed :-)

> And this falls foul of autoconf's advice:
> http://www.gnu.org/software/autoconf/manual/autoconf.html#Shell-Functions
> | Inside a function, IRIX sh sets `$0' to the function name. 

Confirmed with IRIX 6.5 /bin/sh.

> OK to check in this followup?

Thanks for cleaning up all this. I would like if you could mention the newly
added implicit parameter 'progname' in comments for all functions which need
it, directly or indirectly:

# Input:
# - prognamename of this program

It is not possible to maintain a shell script with dozens of functions
without keeping track of which function uses which variable and which has
side effects on which variables.

Bruno

Re: improve "posix-modules --version"

[Eric Blake]

Bruno Haible  clisp.org> writes:

> 
> The --version support in the 'posix-modules' script predates the switch
> from CVS to git. This updates it, using the code we have in gnulib-tool.

> --- 33,224 
>   # outputs to stdout the --version message.
>   func_version ()
>   {
...
> ! gnulib-tool (GNU $package $date)$version

Oops - some over-active copy-n-paste.

> ! func_gnulib_dir ()
> ! {
> !   case "$0" in
> ! /*) self_abspathname="$0" ;;
> ! */*) self_abspathname=`pwd`/"$0" ;;

And this falls foul of autoconf's advice:
http://www.gnu.org/software/autoconf/manual/autoconf.html#Shell-Functions
| Inside a function, IRIX sh sets `$0' to the function name. 

Finally, this patch failed to call the new func_gnulib_dir, with this result in 
the selected 'sh -vx posix-modules' run:

  LC_ALL=C grep -h '^Gnulib module: ' "$gnulib_dir"/doc/posix-headers/* 
2>/dev/null \
...
+ grep -h '^Gnulib module: ' '/doc/posix-headers/*'

leading to an overall result of no output.

OK to check in this followup?
http://repo.or.cz/w/gnulib/ericb.git?a=commitdiff;h=e85ee3d

From: Eric Blake <[EMAIL PROTECTED]>
Date: Fri, 26 Sep 2008 14:47:41 -0600
Subject: [PATCH] Fix previous patch, and tweak references to $0.

* posix-modules: Call func_gnulib_dir before using $gnulib_dir.
(func_version, func_gnulib_dir): Don't call this program
gnulib-tool.
(func_gnulib_dir, func_tmpdir, func_fatal_error): Avoid shell bugs
with using $0 in function.
* gnulib-tool (func_gnulib_dir, func_tmpdir): Likewise.
(func_fatal_error): Reuse the name the user invoked us with.

Signed-off-by: Eric Blake <[EMAIL PROTECTED]>
---
 ChangeLog |   13 -
 gnulib-tool   |   16 
 posix-modules |   28 +---
 3 files changed, 37 insertions(+), 20 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index e4a3d3b..bda6cfb 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,14 @@
+2008-09-26  Eric Blake  <[EMAIL PROTECTED]>
+
+   Fix previous patch, and tweak references to $0.
+   * posix-modules: Call func_gnulib_dir before using $gnulib_dir.
+   (func_version, func_gnulib_dir): Don't call this program
+   gnulib-tool.
+   (func_gnulib_dir, func_tmpdir, func_fatal_error): Avoid shell bugs
+   with using $0 in function.
+   * gnulib-tool (func_gnulib_dir, func_tmpdir): Likewise.
+   (func_fatal_error): Reuse the name the user invoked us with.
+
 2008-09-26  Bruno Haible  <[EMAIL PROTECTED]>
 
* posix-modules (cvsdatestamp, last_checkin_date, version): Remove
diff --git a/gnulib-tool b/gnulib-tool
index 56b185a..de7b289 100755
--- a/gnulib-tool
+++ b/gnulib-tool
@@ -277,9 +277,9 @@ func_exit ()
 # - gnulib_dir   absolute pathname of gnulib repository
 func_gnulib_dir ()
 {
-  case "$0" in
-/*) self_abspathname="$0" ;;
-*/*) self_abspathname=`pwd`/"$0" ;;
+  case "$progname" in
+/*) self_abspathname="$progname" ;;
+*/*) self_abspathname=`pwd`/"$progname" ;;
 *)
   # Look in $PATH.
   # Iterate through the elements of $PATH.
@@ -326,8 +326,8 @@ func_gnulib_dir ()
   for d in $pathx; do
 IFS="$save_IFS"
 test -z "$d" && d=.
-if test -x "$d/$0" && test ! -d "$d/$0"; then
-  self_abspathname="$d/$0"
+if test -x "$d/$progname" && test ! -d "$d/$progname"; then
+  self_abspathname="$d/$progname"
   break
 fi
   done
@@ -375,7 +375,7 @@ func_tmpdir ()
 (umask 077 && mkdir "$tmp")
   } ||
   {
-echo "$0: cannot create a temporary directory in $TMPDIR" >&2
+echo "$progname: cannot create a temporary directory in $TMPDIR" >&2
 func_exit 1
   }
 }
@@ -400,8 +400,8 @@ fi
 # outputs to stderr a fatal error message, and terminates the program.
 func_fatal_error ()
 {
-  echo "gnulib-tool: *** $1" 1>&2
-  echo "gnulib-tool: *** Stop." 1>&2
+  echo "$progname: *** $1" 1>&2
+  echo "$progname: *** Stop." 1>&2
   func_exit 1
 }
 
diff --git a/posix-modules b/posix-modules
index 2fa010b..adb887b 100755
--- a/posix-modules
+++ b/posix-modules
@@ -66,7 +66,7 @@ q
   fi
   year=`"$gnulib_dir"/build-aux/mdate-sh "$self_abspathname" | sed 's,^.* ,,'`
   echo "\
-gnulib-tool (GNU $package $date)$version
+posix-modules (GNU $package $date)$version
 Copyright (C) $year Free Software Foundation, Inc.
 License GPLv3+: GNU GPL version 3 or later 
 This is free software: you are free to change and redistribute it.
@@ -87,13 +87,13 @@ func_exit ()
 # func_gnulib_dir
 # locates the directory where the gnulib repository lives
 # Sets variables
-# - self_abspathname absolute pathname of gnulib-tool
+# - self_abspathname absolute pathname of this program
 # - gnulib_dir   absolute pathname of gnulib repository
 func_gnulib_dir ()
 {
-  case "$0" in
-/*) self_abspathname="$0" ;;
-*/*) self_abspathname=`pwd`/"$0" ;;
+  case "$progname" in
+/*) self_abspathname="$progname" ;;
+*/*) self_abspathname=`pwd`/"$pro

Re: sigpipe module

[Bruno Haible]

Eric Blake wrote:
> vasnprintf now behaves differently on mingw depending on whether 
> you are also using sigpipe-die, and this difference is at the source code 
> level 
> (faking SIGPIPE on mingw comes with a lot of baggage)

The essential idea here is right. Just two details:

  - The feature-enabling module is 'sigpipe'. 'sigpipe-die' is used when the
application wants a behaviour that matches neither the POSIX default
behaviour nor the POSIX SIG_IGN behaviour. Not many programs use that.
For 80% of the programs, the POSIX default behaviour is fine.

  - vfprintf is compiled differently, depending with module 'sigpipe' is
present or not. vasnprintf is not, since it does not deal with streams,
only with strings.

Bruno

Re: installable gnulib library

[Bruno Haible]

Hi Gary,

> why not enhance  
> gnulib to provide a shim that sits between the system libraries and  
> client code that wants to use it without shipping (another copy) of  
> the particular parts it depends upon?

It does not take a lot of steps to do that:
  1) $ gnulib-tool --create-testdir `posix-modules`
 The 'posix-modules' script is in gnulib; it was just out of order for a
 long time.
  2) package up the testdir
  3) build and install it on your platform.
  4) Deal with config.h, i.e. rename it to 'shim-config.h' and adjust
 the generated .h files. Some C macros, like _GNU_SOURCE or __EXTENSIONS__,
 need to be put into CPPFLAGS, however.
  5) Document what to put into CPPFLAGS and what into LDFLAGS.

The problems of this approach are in the presentation / in the relations
with the community:
  - People will say that it's not complete, not understand that it's
work in progress.
  - People will not understand that if they use it for their package,
they may need to contribute at some point.
  - Additional API that are extensions over POSIX would likely be omitted.
But they are also an essential part of gnulib.

and in the points that Eric mentioned:
  - gnulib is doing backward-incompatible changes now and then.
  - gnulib has some "features" modules, like 'sigpipe', which cause additional
code to be injected into many other modules. Not everyone needs these
features.

Bruno

improve "posix-modules --version"

[Bruno Haible]

The --version support in the 'posix-modules' script predates the switch
from CVS to git. This updates it, using the code we have in gnulib-tool.


2008-09-26  Bruno Haible  <[EMAIL PROTECTED]>

* posix-modules (cvsdatestamp, last_checkin_date, version): Remove
variables.
(func_version): Essentially copied from gnulib-tool.
(func_exit, func_gnulib_dir, func_tmpdir, func_fatal_error,
func_readlink): Copied from gnulib-tool.

*** posix-modules.orig  2008-09-26 20:33:25.0 +0200
--- posix-modules   2008-09-26 20:32:51.0 +0200
***
*** 18,26 
  
  progname=$0
  package=gnulib
- cvsdatestamp='$Date: 2007-09-11 00:40:16 $'
- last_checkin_date=`echo "$cvsdatestamp" | sed -e 's,^\$[D]ate: ,,'`
- version=`echo "$last_checkin_date" | sed -e 's/ .*$//' -e 's,/,-,g'`
  
  # func_usage
  # outputs to stdout the --help usage message.
--- 18,23 
***
*** 36,120 
  # outputs to stdout the --version message.
  func_version ()
  {
!   year=`echo "$last_checkin_date" | sed -e 's,/.*$,,'`
echo "\
! $progname (GNU $package) $version
  Copyright (C) $year Free Software Foundation, Inc.
! This is free software; see the source for copying conditions.  There is NO
! warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
  Written by" "Bruno Haible"
  }
  
! case "$0" in
!   /*) self_abspathname="$0" ;;
!   */*) self_abspathname=`pwd`/"$0" ;;
!   *)
! # Look in $PATH.
! # Iterate through the elements of $PATH.
! # We use IFS=: instead of
! #   for d in `echo ":$PATH:" | sed -e 's/:::*/:.:/g' | sed -e 's/:/ /g'`
! # because the latter does not work when some PATH element contains spaces.
! # We use a canonicalized $pathx instead of $PATH, because empty PATH
! # elements are by definition equivalent to '.', however field splitting
! # according to IFS=: loses empty fields in many shells:
! #   - /bin/sh on OSF/1 and Solaris loses all empty fields (at the
! # beginning, at the end, and in the middle),
! #   - /bin/sh on IRIX and /bin/ksh on IRIX and OSF/1 lose empty fields
! # at the beginning and at the end,
! #   - GNU bash, /bin/sh on AIX and HP-UX, and /bin/ksh on AIX, HP-UX,
! # Solaris lose empty fields at the end.
! # The 'case' statement is an optimization, to avoid evaluating the
! # explicit canonicalization command when $PATH contains no empty fields.
! self_abspathname=
! if test "${PATH_SEPARATOR+set}" != set; then
!   func_tmpdir
!   { echo "#! /bin/sh"; echo "exit 0"; } > "$tmp"/conf.sh
!   chmod +x "$tmp"/conf.sh
!   if (PATH="/nonexistent;$tmp"; conf.sh) >/dev/null 2>&1; then
! PATH_SEPARATOR=';'
else
! PATH_SEPARATOR=:
fi
!   rm -rf "$tmp"
! fi
! if test "$PATH_SEPARATOR" = ";"; then
!   # On Windows, programs are searched in "." before $PATH.
!   pathx=".;$PATH"
! else
!   # On Unix, we have to convert empty PATH elements to ".".
!   pathx="$PATH"
!   case :$PATH: in
! *::*)
!   pathx=`echo ":$PATH:" | sed -e 's/:::*/:.:/g' -e 's/^://' -e 
's/:\$//'`
!   ;;
!   esac
! fi
! save_IFS="$IFS"
! IFS="$PATH_SEPARATOR"
! for d in $pathx; do
IFS="$save_IFS"
!   test -z "$d" && d=.
!   if test -x "$d/$0" && test ! -d "$d/$0"; then
! self_abspathname="$d/$0"
! break
fi
! done
! IFS="$save_IFS"
! if test -z "$self_abspathname"; then
!   func_fatal_error "could not locate the posix-modules program - how did 
you invoke it?"
! fi
! ;;
! esac
! while test -h "$self_abspathname"; do
!   # Resolve symbolic link.
!   linkval=`func_readlink "$self_abspathname"`
!   test -n "$linkval" || break
!   case "$linkval" in
! /* ) self_abspathname="$linkval" ;;
! * ) self_abspathname=`echo "$self_abspathname" | sed -e 
's,/[^/]*$,,'`/"$linkval" ;;
esac
! done
! gnulib_dir=`echo "$self_abspathname" | sed -e 's,/[^/]*$,,'`
  
  # Command-line option processing.
  while test $# -gt 0; do
--- 33,224 
  # outputs to stdout the --version message.
  func_version ()
  {
!   func_gnulib_dir
!   if test -d "$gnulib_dir"/.git \
!  && (git --version) >/dev/null 2>/dev/null \
!  && (date --version) >/dev/null 2>/dev/null; then
! # gnulib checked out from git.
! sed_extract_first_date='/^Date/{
! s/^Date:[  ]*//p
! q
! }'
! date=`cd "$gnulib_dir" && git log ChangeLog | sed -n -e 
"$sed_extract_first_date"`
! # Turn "Fri Mar 21 07:16:51 2008 -0600" into "Mar 21 2008 07:16:51 -0600".
! sed_year_before_time='s/^[^ ]* \([^ ]*\) \([0-9]*\) \([0-9:]*\) 
\([0-9]*\) /\1 \2 \4 \3 /'
! date=`echo "$date" | sed -e "$sed_year_before_time"`
! # Use GNU date to compute the time in GMT.
! date=`date -d "$date" -u +"%Y-%m-%d %H:%M:%S"`
! version=' '`cd "$gnulib_dir" && ./build-aux/git-version-gen /dev/null | 
sed -e 's/-dir

improve "gnulib-tool --version"

[Bruno Haible]

Currently, "gnulib-tool --version" works better when called from the
gnulib main directory than from any other directory:

$ ./gnulib-tool --version | head -1
gnulib-tool (GNU gnulib 2008-09-26 18:13:32) 0.0.1070-1d542
$ cd modules
$ ../gnulib-tool --version | head -1
gnulib-tool (GNU gnulib 2008-09-26 18:13:32) UNKNOWN

This fixes it.


2008-09-26  Bruno Haible  <[EMAIL PROTECTED]>

* gnulib-tool (func_version): Change directory to $gnulib_dir before
invoking git-version-gen.

--- gnulib-tool.orig2008-09-26 20:31:09.0 +0200
+++ gnulib-tool 2008-09-26 20:26:48.0 +0200
@@ -216,7 +216,7 @@
 date=`echo "$date" | sed -e "$sed_year_before_time"`
 # Use GNU date to compute the time in GMT.
 date=`date -d "$date" -u +"%Y-%m-%d %H:%M:%S"`
-version=' '`"$gnulib_dir"/build-aux/git-version-gen /dev/null | sed -e 
's/-dirty/-modified/'`
+version=' '`cd "$gnulib_dir" && ./build-aux/git-version-gen /dev/null | 
sed -e 's/-dirty/-modified/'`
   else
 if test -d "$gnulib_dir"/CVS \
&& (cvs --version) >/dev/null 2>/dev/null; then

make posix-modules script work again

[Bruno Haible]

The posix-modules script was broken for a long time. This fixes it.


2008-09-26  Bruno Haible  <[EMAIL PROTECTED]>

* posix-modules: Update to directory names changed on 2008-01-19.
Remove commas in output before splitting into words. No more need to
avoid 'ftruncate' since 2007-02-19.

--- posix-modules.orig  2008-09-26 20:13:07.0 +0200
+++ posix-modules   2008-09-26 20:04:54.0 +0200
@@ -1,6 +1,6 @@
 #!/bin/sh
 #
-# Copyright (C) 2002-2007 Free Software Foundation, Inc.
+# Copyright (C) 2002-2008 Free Software Foundation, Inc.
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -138,17 +138,14 @@
 
 (
   # Get the header modules.
-  LC_ALL=C grep -h '^Gnulib module: ' "$gnulib_dir"/doc/headers/* 2>/dev/null \
+  LC_ALL=C grep -h '^Gnulib module: ' "$gnulib_dir"/doc/posix-headers/* 
2>/dev/null \
 | sed -e 's,^Gnulib module: ,,'
   # Get the function modules.
-  LC_ALL=C grep -h '^Gnulib module: ' "$gnulib_dir"/doc/functions/* 
2>/dev/null \
+  LC_ALL=C grep -h '^Gnulib module: ' "$gnulib_dir"/doc/posix-functions/* 
2>/dev/null \
 | sed -e 's,^Gnulib module: ,,'
-  # Then filter out the words "---", "and", "or" and remove *-gnu modules.
-) | LC_ALL=C sort | LC_ALL=C uniq \
+  # Then filter out the words "---", ",", "and", "or" and remove *-gnu modules.
+) | sed -e 's/,/ /g' | LC_ALL=C sort | LC_ALL=C uniq \
   | { # Then filter out the words "---", "and", "or" and remove *-gnu modules.
   tr ' ' '\012' | sed -e '/^---$/d' -e '/^and$/d' -e '/^or$/d' -e 
'/-gnu$/d'
 } \
-  | LC_ALL=C sort | LC_ALL=C uniq \
-  | { # Except ftruncate, which aborts the configuration on mingw. FiXME
-  sed -e '/^ftruncate$/d'
-}
+  | LC_ALL=C sort | LC_ALL=C uniq

Re: Remove module 'EOVERFLOW'

[Bruno Haible]

Addendum to this patch:

2008-09-26  Bruno Haible  <[EMAIL PROTECTED]>

* doc/posix-headers/errno.texi: Remove mention of module 'EOVERFLOW'.

--- doc/posix-headers/errno.texi.orig   2008-09-26 20:08:01.0 +0200
+++ doc/posix-headers/errno.texi2008-09-26 20:05:45.0 +0200
@@ -3,7 +3,7 @@
 
 POSIX specification: @url{http://www.opengroup.org/susv3xbd/errno.h.html}
 
-Gnulib module: errno, EOVERFLOW
+Gnulib module: errno
 
 Portability problems fixed by Gnulib:
 @itemize

Re: installable gnulib library

[Eric Blake]

Gary V. Vaughan  gnu.org> writes:

> 
> I have an (undoubtedly caffeine induced) idea... why not enhance  
> gnulib to provide a shim that sits between the system libraries and  
> client code that wants to use it without shipping (another copy) of  
> the particular parts it depends upon?

I am somewhat skeptical of this idea, for sevaral reasons.

Gnulib mutates too quickly.  How would you guarantee that the interface the 
user installed on their system is new enough to meet your package's needs, 
particularly when you look through NEWS at how often gnulib interfaces change?

Gnulib includes a LOT of source code hacks (witness all the #include_next 
header replacements).  A library only solves API needs, but it does not solve 
source code needs, so packages will still end up shipping lots of code from 
gnulib.  It's easier to ship all of the source code, than to try and pick out 
the parts not already covered by an installed library.

Think about gnulib-tool --avoid - how does one exclude entry points from an 
installed library, if they explicitly want to avoid a particular gnulib module 
(for example, because of licensing incompatibility).  Source code distribution 
makes this task a lot easier.

In that vein, not all gnulib modules play together nicely.  For example, just 
from today, vasnprintf now behaves differently on mingw depending on whether 
you are also using sigpipe-die, and this difference is at the source code level 
(faking SIGPIPE on mingw comes with a lot of baggage).  The choice to use 
sigpipe-die is a conscious decision of the maintainer, but if gnulib were 
installed, then you would have to have multiple library versions to expose 
parameterization to what used to be a compile-time decision.

I think http://www.gnu.org/software/gnulib/manual/gnulib.html#Introduction 
covers most of these points, as well as mentioning the difficulty in trying to 
make libiberty an installable library as precedent for the paradigm of keeping 
gnulib source-only.

-- 
Eric Blake

Re: installable gnulib library

[Bruce Korb]

On Fri, Sep 26, 2008 at 9:25 AM, Gary V. Vaughan <[EMAIL PROTECTED]> wrote:
> I have an (undoubtedly caffeine induced) idea... why not enhance gnulib to
> provide a shim that sits between the system libraries and client code that
> wants to use it without shipping (another copy) of the particular parts it
> depends upon?
>
> If we add a compile-and-install-everything-as-a-library mode to gnulib, many
> GNU packages could stop distributing the MiBs of autotools' generated glue,
> and instead go with a much lighter build system that simply depends on
> gnulib API semantics, and requires that gnulib be installed.  For many
> modern systems, the installed gnulib might turn out to be vanishingly
> small... and for many old and broken systems, having just one copy of gnulib
> in shared memory ought to provide a nice improvement to speed and memory
> utilisation.
>
> Please tell me I'm crazy right now.  Or at least before I waste the next few
> months of my free time figuring out how to do it.

Hey Gary,

Long ago, far away and years ago, that is *precisely* the point I was making.
(Remember the autotool bake-off contest?)  My thinking then (and now) is that
you just make your project depend upon the pre-installation of this common
glue package.  Glue layer not installed?  Then your's won't install either.
I think some folks won't buy into it because they don't want another dependency
for their projects.  (I think that was the main objection to my
proposal for that
long-ago "contest".  Can't seem to put together the right Google search to
dredge it up again...)

Anyway, at least _I'd_ like to see it.  :-D

Cheers - Bruce

installable gnulib library

[Gary V. Vaughan]

I have an (undoubtedly caffeine induced) idea... why not enhance  
gnulib to provide a shim that sits between the system libraries and  
client code that wants to use it without shipping (another copy) of  
the particular parts it depends upon?


If we add a compile-and-install-everything-as-a-library mode to  
gnulib, many GNU packages could stop distributing the MiBs of  
autotools' generated glue, and instead go with a much lighter build  
system that simply depends on gnulib API semantics, and requires that  
gnulib be installed.  For many modern systems, the installed gnulib  
might turn out to be vanishingly small... and for many old and broken  
systems, having just one copy of gnulib in shared memory ought to  
provide a nice improvement to speed and memory utilisation.


Please tell me I'm crazy right now.  Or at least before I waste the  
next few months of my free time figuring out how to do it.


Cheers,
Gary
--
Email me:  [EMAIL PROTECTED](\(\
Read my blog:  http://blog.azazil.net  ( o.O)
And my other blog: http://www.machaxor.net  (uu )o
...and my book:http://sources.redhat.com/autobook  ("("_)

[PATCH]: Do not allow countable dayshifts in getdate.y

[Ondřej Vašík]

Hello,
this patch for getdate.y module restricts usage of countable dayshifts
e.g. +40 tommorow ago, next yesterday etc. 
Tests to gnulib getdate testsuite were added.
As this usage of dayshifts is quite insane, I guess no documentation for
it is required.

Greetings,
 Ondrej Vasik


From 526c0de4383f29d98b71511d18fed0c0e9272b55 Mon Sep 17 00:00:00 2001
From: =?utf-8?q?Ond=C5=99ej=20Va=C5=A1=C3=ADk?= <[EMAIL PROTECTED]>
Date: Fri, 26 Sep 2008 15:28:49 +0200
Subject: [PATCH] getdate.y: Do not allow countable general dayshifts
 	* lib/getdate.y (relative_time_table) : New type tDAY_SHIFT
 	for exactly specified dayshifts added and used
 	* tests/test-getdate.c : Tests for no longer allowed countable
 	dayshifts (e.g. 4 yesterday ago) added.

---
 ChangeLog|7 +++
 lib/getdate.y|   19 +--
 tests/test-getdate.c |   48 
 3 files changed, 68 insertions(+), 6 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 07f0026..d204816 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,10 @@
+2008-09-27  Ondrej Vasik  <[EMAIL PROTECTED]>
+	getdate.y: Do not allow countable general dayshifts
+	* lib/getdate.y (relative_time_table) : New type tDAY_SHIFT 
+	for exactly specified dayshifts added and used
+	* tests/test-getdate.c : Tests for no longer allowed countable
+	dayshifts (e.g. 4 yesterday ago) added.
+
 2008-09-26  Jim Meyering  <[EMAIL PROTECTED]>
 
 	fts: tweak inode comparison function
diff --git a/lib/getdate.y b/lib/getdate.y
index f9cd86c..877b264 100644
--- a/lib/getdate.y
+++ b/lib/getdate.y
@@ -293,7 +293,7 @@ set_hhmmss (parser_control *pc, long int hour, long int minutes,
 %token tAGO tDST
 
 %token tYEAR_UNIT tMONTH_UNIT tHOUR_UNIT tMINUTE_UNIT tSEC_UNIT
-%token  tDAY_UNIT
+%token  tDAY_UNIT tDAY_SHIFT
 
 %token  tDAY tDAYZONE tLOCAL_ZONE tMERIDIAN
 %token  tMONTH tORDINAL tZONE
@@ -304,7 +304,7 @@ set_hhmmss (parser_control *pc, long int hour, long int minutes,
 %type  o_colon_minutes o_merid
 %type  seconds signed_seconds unsigned_seconds
 
-%type  relunit relunit_snumber
+%type  relunit relunit_snumber dayshift
 
 %%
 
@@ -502,6 +502,8 @@ rel:
   { apply_relative_time (pc, $1, -1); }
   | relunit
   { apply_relative_time (pc, $1, 1); }
+  | dayshift
+  { apply_relative_time (pc, $1, 1); }
   ;
 
 relunit:
@@ -563,6 +565,11 @@ relunit_snumber:
   { $$ = RELATIVE_TIME_0; $$.seconds = $1.value; }
   ;
 
+dayshift:
+tDAY_SHIFT
+  { $$ = RELATIVE_TIME_0; $$.day = $1; }
+  ;
+
 seconds: signed_seconds | unsigned_seconds;
 
 signed_seconds:
@@ -669,10 +676,10 @@ static table const time_units_table[] =
 /* Assorted relative-time words. */
 static table const relative_time_table[] =
 {
-  { "TOMORROW",	tDAY_UNIT,	 1 },
-  { "YESTERDAY",tDAY_UNIT,	-1 },
-  { "TODAY",	tDAY_UNIT,	 0 },
-  { "NOW",	tDAY_UNIT,	 0 },
+  { "TOMORROW",	tDAY_SHIFT,	 1 },
+  { "YESTERDAY",tDAY_SHIFT,	-1 },
+  { "TODAY",	tDAY_SHIFT,	 0 },
+  { "NOW",	tDAY_SHIFT,	 0 },
   { "LAST",	tORDINAL,	-1 },
   { "THIS",	tORDINAL,	 0 },
   { "NEXT",	tORDINAL,	 1 },
diff --git a/tests/test-getdate.c b/tests/test-getdate.c
index 80cf573..5edda4b 100644
--- a/tests/test-getdate.c
+++ b/tests/test-getdate.c
@@ -161,5 +161,53 @@ main (int argc, char **argv)
   p = "UTC+25:00";
   ASSERT (!get_date (&result, p, &now));
 
+	/* Check for several invalid countable dayshifts */
+  now.tv_sec = 4711;
+  now.tv_nsec = 1267;
+  p = "UTC+4:00 +40 yesterday";
+  ASSERT (!get_date (&result, p, &now));
+  p = "UTC+4:00 next yesterday";
+  ASSERT (!get_date (&result, p, &now));
+  p = "UTC+4:00 tomorrow ago";
+  ASSERT (!get_date (&result, p, &now));
+  p = "UTC+4:00 40 now ago";
+  ASSERT (!get_date (&result, p, &now));
+  p = "UTC+4:00 last tomorrow";
+  ASSERT (!get_date (&result, p, &now));
+  p = "UTC+4:00 -4 today";
+  ASSERT (!get_date (&result, p, &now));
+
+  /* And check correct usage of dayshifts */
+  now.tv_sec = 4711;
+  now.tv_nsec = 1267;
+  p = "UTC+400 tomorrow";
+  ASSERT (get_date (&result, p, &now));
+  LOG (p, now, result);
+  p = "UTC+400 +1 day";
+  ASSERT (get_date (&result2, p, &now));
+  LOG (p, now, result2);
+  ASSERT (result.tv_sec == result2.tv_sec
+	  && result.tv_nsec == result2.tv_nsec);
+  now.tv_sec = 4711;
+  now.tv_nsec = 1267;
+  p = "UTC+400 yesterday";
+  ASSERT (get_date (&result, p, &now));
+  LOG (p, now, result);
+  p = "UTC+400 1 day ago";
+  ASSERT (get_date (&result2, p, &now));
+  LOG (p, now, result2);
+  ASSERT (result.tv_sec == result2.tv_sec
+	  && result.tv_nsec == result2.tv_nsec);
+  now.tv_sec = 4711;
+  now.tv_nsec = 1267;
+  p = "UTC+400 now";
+  ASSERT (get_date (&result, p, &now));
+  LOG (p, now, result);
+  p = "UTC+400 +0 minutes"; /* silly, but simple "UTC+400" is different*/
+  ASSERT (get_date (&result2, p, &now));
+  LOG (p, now, result2);
+  ASSERT (result.tv_sec == result2.tv_sec
+	  && result.tv_nsec == result2.tv_nsec);
+
   return 0;
 }
-- 
1.5.6.1.156.ge903b



signature.a

Re: tee logs no output if stdout is closed

[Bruno Haible]

Jim Meyering wrote:
> > * lib/close-stream.c (close_stream): Ignore error EPIPE from fclose.
> >
> I know this condition arises only when ignoring or handling SIGPIPE,
> (which should be rather unusual) but even so, I really dislike the idea
> of ignoring a write error.  Even if the write error would not occur
> with slightly less output data, it's still one less _legitimate_ error
> that can be reported.  If there is an EPIPE error, IMHO, close_stream
> must diagnose it.

This does not convince me, because EPIPE is part of the normal shutdown
protocol of pipes, when the pipe writer chooses to block or ignore SIGPIPE.

The very purpose of having different error codes listed in  is
to be able to react differently on them.

I cannot change 'close-stream', since you own that module. But for 
'fwriteerror',
which I use in GNU gettext - and where I don't want to have spurious, timing-
dependent error messages - I'm applying this:


2008-09-26  Bruno Haible  <[EMAIL PROTECTED]>

* lib/fwriteerror.c (do_fwriteerror): Ignore error EPIPE.

*** lib/fwriteerror.c.orig  2008-09-26 16:07:15.0 +0200
--- lib/fwriteerror.c   2008-09-26 15:38:27.0 +0200
***
*** 1,5 
  /* Detect write error on a stream.
!Copyright (C) 2003-2006 Free Software Foundation, Inc.
 Written by Bruno Haible <[EMAIL PROTECTED]>, 2003.
  
 This program is free software: you can redistribute it and/or modify
--- 1,5 
  /* Detect write error on a stream.
!Copyright (C) 2003-2006, 2008 Free Software Foundation, Inc.
 Written by Bruno Haible <[EMAIL PROTECTED]>, 2003.
  
 This program is free software: you can redistribute it and/or modify
***
*** 38,44 
stdout_closed = true;
  }
  
!   /* Need to
   1. test the error indicator of the stream,
   2. flush the buffers both in userland and in the kernel, through fclose,
  testing for error again.  */
--- 38,56 
stdout_closed = true;
  }
  
!   /* This function returns an error indication if there was a previous failure
!  or if fclose failed, with two exceptions:
!- Ignore an fclose failure if there was no previous error, no data
!remains to be flushed, and fclose failed with EBADF.  That can
!happen when a program like cp is invoked like this `cp a b >&-'
!(i.e., with standard output closed) and doesn't generate any
!output (hence no previous error and nothing to be flushed).
!- Ignore an fclose failure due to EPIPE.  That can happen when a
!program blocks or ignores SIGPIPE, and the output pipe or socket
!has no readers now.  The EPIPE tells us that we should stop writing
!to this output.  That's what we are doing anyway here.
! 
!  Need to
   1. test the error indicator of the stream,
   2. flush the buffers both in userland and in the kernel, through fclose,
  testing for error again.  */
***
*** 71,82 
if (fflush (fp))
goto close_preserving_errno; /* errno is set here */
if (fclose (fp) && errno != EBADF)
!   return -1; /* errno is set here */
  }
else
  {
if (fclose (fp))
!   return -1; /* errno is set here */
  }
  
return 0;
--- 83,94 
if (fflush (fp))
goto close_preserving_errno; /* errno is set here */
if (fclose (fp) && errno != EBADF)
!   goto got_errno; /* errno is set here */
  }
else
  {
if (fclose (fp))
!   goto got_errno; /* errno is set here */
  }
  
return 0;
***
*** 88,95 
  int saved_errno = errno;
  fclose (fp);
  errno = saved_errno;
- return -1;
}
  }
  
  int
--- 100,112 
  int saved_errno = errno;
  fclose (fp);
  errno = saved_errno;
}
+  got_errno:
+   /* There's an error.  Ignore EPIPE.  */
+   if (errno == EPIPE)
+ return 0;
+   else
+ return -1;
  }
  
  int

Re: skipping message for vc-list-files tests

[Bruno Haible]

Jim Meyering wrote:
> Go ahead, but please correct (s/git/cvs/) the diagnostic
> in tests/test-vc-list-files-cvs.sh.

Oops. Reviewing is valuable! - Committed with the fix.

Bruno

Re: mingw and SIGPIPE, new modules 'sigpipe' and 'write'

[Simon Josefsson]

Bruno Haible <[EMAIL PROTECTED]> writes:

> I'm applying this in 7 commits.

It appears to solve the problem for sigpipe-die, see successful mingw
build of it in:

http://autobuild.josefsson.org/gnulib/log-200809261431057164000.txt

I'll get around to look at the other failed modules on mingw in two
weeks when I will have more time, if nobody beats me to it.

Thanks,
/Simon

Re: skipping message for vc-list-files tests

[Jim Meyering]

Bruno Haible <[EMAIL PROTECTED]> wrote:
> Hi Jim,
>
> Eric reminded me that it is good practice to provide a message that explains
> why a test is skipped. (When I did this in GNU gettext, it uncovered a couple
> of real bugs on various systems.)

Hi Bruno,

Go ahead, but please correct (s/git/cvs/) the diagnostic
in tests/test-vc-list-files-cvs.sh.

> 2008-09-26  Bruno Haible  <[EMAIL PROTECTED]>
>
>   * tests/test-vc-list-files-git.sh: Explain reason for skipping test.
>   * tests/test-vc-list-files-cvs.sh: Likewise.
>
> --- tests/test-vc-list-files-cvs.sh.orig  2008-09-26 14:37:52.0 
> +0200
> +++ tests/test-vc-list-files-cvs.sh   2008-09-26 13:46:43.0 +0200
> @@ -45,7 +45,8 @@
>mkdir $tmpdir && cd $tmpdir &&
>  # without cvs, skip the test
>  # The double use of 'exit' is needed for the reference to $? inside the 
> trap.
> -{ ( cvs -Q -d "$repo" init ) > /dev/null 2>&1 || { (exit 77); exit 77; 
> }; } &&
> +{ ( cvs -Q -d "$repo" init ) > /dev/null 2>&1 \
> +  || { echo "Skipping test: git not found in PATH"; (exit 77); exit 77; 
> }; } &&
>  mkdir w && cd w &&
>  mkdir d &&
>  touch d/a b c &&
> --- tests/test-vc-list-files-git.sh.orig  2008-09-26 14:37:52.0 
> +0200
> +++ tests/test-vc-list-files-git.sh   2008-09-26 13:46:43.0 +0200
> @@ -32,7 +32,8 @@
>  mkdir $tmpdir && cd $tmpdir &&
># without git, skip the test
># The double use of 'exit' is needed for the reference to $? inside the 
> trap.
> -  { ( git init -q ) > /dev/null 2>&1 || { (exit 77); exit 77; }; } &&
> +  { ( git init -q ) > /dev/null 2>&1 \
> +|| { echo "Skipping test: git not found in PATH"; (exit 77); exit 77; }; 
> } &&
>mkdir d &&
>touch d/a b c &&
>git add . > /dev/null &&

skipping message for vc-list-files tests

[Bruno Haible]

Hi Jim,

Eric reminded me that it is good practice to provide a message that explains
why a test is skipped. (When I did this in GNU gettext, it uncovered a couple
of real bugs on various systems.)

How about this? OK to commit?


2008-09-26  Bruno Haible  <[EMAIL PROTECTED]>

* tests/test-vc-list-files-git.sh: Explain reason for skipping test.
* tests/test-vc-list-files-cvs.sh: Likewise.

--- tests/test-vc-list-files-cvs.sh.orig2008-09-26 14:37:52.0 
+0200
+++ tests/test-vc-list-files-cvs.sh 2008-09-26 13:46:43.0 +0200
@@ -45,7 +45,8 @@
   mkdir $tmpdir && cd $tmpdir &&
 # without cvs, skip the test
 # The double use of 'exit' is needed for the reference to $? inside the 
trap.
-{ ( cvs -Q -d "$repo" init ) > /dev/null 2>&1 || { (exit 77); exit 77; }; 
} &&
+{ ( cvs -Q -d "$repo" init ) > /dev/null 2>&1 \
+  || { echo "Skipping test: git not found in PATH"; (exit 77); exit 77; }; 
} &&
 mkdir w && cd w &&
 mkdir d &&
 touch d/a b c &&
--- tests/test-vc-list-files-git.sh.orig2008-09-26 14:37:52.0 
+0200
+++ tests/test-vc-list-files-git.sh 2008-09-26 13:46:43.0 +0200
@@ -32,7 +32,8 @@
 mkdir $tmpdir && cd $tmpdir &&
   # without git, skip the test
   # The double use of 'exit' is needed for the reference to $? inside the trap.
-  { ( git init -q ) > /dev/null 2>&1 || { (exit 77); exit 77; }; } &&
+  { ( git init -q ) > /dev/null 2>&1 \
+|| { echo "Skipping test: git not found in PATH"; (exit 77); exit 77; }; } 
&&
   mkdir d &&
   touch d/a b c &&
   git add . > /dev/null &&

Re: fts: avoid O(n^2) ext3/ext4 performance hit on million+-entry dirs

[Jim Meyering]

Jim Meyering <[EMAIL PROTECTED]> wrote:
> Here is a patch that makes it so tools using fts,
> like chmod, chown, chgrp, chcon, du, and find are no
> longer susceptible to an O(n^2) performance penalty when
> processing very large directory-entry counts (as in millions).
> I first noticed the problem on ext3 and ext4 file systems,
> but the patch also improves performance on reiserfs and xfs,
> but not on tmpfs.

I've pushed that with a fix for the comparison function
problem that Ralf spotted:

  
http://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=commitdiff;h=2f2978ede97205c49d3e568ccffa5a04fb53326b

then spotted a typo, and pushed the correction:

  
http://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=commit;h=1d569ca4e7e6147793e6e6510e5a36a4139b2f31

Re: Changing module from LGPL to LGPLv2+

[Eric Blake]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

According to Bruno Haible on 9/26/2008 4:05 AM:
>> fopen
>> signal
>> sigprocmask
> 
> Changed by Eric and me. Fine with me. Eric?

Fine with me.

- --
Don't work too hard, make some time for fun as well!

Eric Blake [EMAIL PROTECTED]
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkjc0QYACgkQ84KuGfSFAYDb6gCgiUZ3oZVpILjzA5B1TZLvpm18
QzQAnRdbKp5Y/rd3Te2BRj+X6RoFVgSX
=Dmz6
-END PGP SIGNATURE-

Re: mingw and SIGPIPE, new modules 'sigpipe' and 'write' (7/7)

[Bruno Haible]

2008-09-26  Bruno Haible  <[EMAIL PROTECTED]>

* modules/sigpipe-die (Depends-on): Add sigpipe.

*** modules/sigpipe-die.orig2008-09-26 12:52:18.0 +0200
--- modules/sigpipe-die 2008-09-26 04:09:29.0 +0200
***
*** 9,14 
--- 9,15 
  error
  gettext-h
  exitfail
+ sigpipe
  sigprocmask
  sigaction
  

Re: mingw and SIGPIPE, new modules 'sigpipe' and 'write' (6/7)

[Bruno Haible]

2008-09-26  Bruno Haible  <[EMAIL PROTECTED]>

* lib/stdio.in.h (fprintf, vfprintf, printf, vprintf, fputc, putc,
putchar, fputs, puts, fwrite): Replace when REPLACE_STDIO_WRITE_FUNCS
and GNULIB_STDIO_H_SIGPIPE are set.
* lib/stdio-write.c: New file.
* m4/stdio_h.m4 (gl_STDIO_H): Set GNULIB_FPRINTF, GNULIB_PRINTF,
GNULIB_VFPRINTF, GNULIB_VPRINTF, GNULIB_FPUTC, GNULIB_PUTC,
GNULIB_PUTCHAR, GNULIB_FPUTS, GNULIB_PUTS, GNULIB_FWRITE,
REPLACE_STDIO_WRITE_FUNCS.
(gl_STDIO_H_DEFAULTS): Initialize GNULIB_FPRINTF, GNULIB_PRINTF,
GNULIB_VFPRINTF, GNULIB_VPRINTF, GNULIB_FPUTC, GNULIB_PUTC,
GNULIB_PUTCHAR, GNULIB_FPUTS, GNULIB_PUTS, GNULIB_FWRITE,
GNULIB_STDIO_H_SIGPIPE, REPLACE_STDIO_WRITE_FUNCS.
* modules/stdio (Files): Add lib/stdio-write.c.
(Makefile.am): Substitute GNULIB_FPRINTF, GNULIB_PRINTF,
GNULIB_VFPRINTF, GNULIB_VPRINTF, GNULIB_FPUTC, GNULIB_PUTC,
GNULIB_PUTCHAR, GNULIB_FPUTS, GNULIB_PUTS, GNULIB_FWRITE,
GNULIB_STDIO_H_SIGPIPE, REPLACE_STDIO_WRITE_FUNCS.
* m4/fprintf-posix.m4 (gl_REPLACE_FPRINTF): Define
REPLACE_FPRINTF_POSIX.
* m4/printf-posix-rpl.m4 (gl_REPLACE_PRINTF): Define
REPLACE_PRINTF_POSIX.
* m4/vfprintf-posix.m4 (gl_REPLACE_VFPRINTF): Define
REPLACE_VFPRINTF_POSIX.
* m4/vprintf-posix.m4 (gl_REPLACE_VPRINTF): Define
REPLACE_VPRINTF_POSIX.
* doc/posix-functions/fprintf.texi: Mention the sigpipe module and the
SIGPIPE issue.
* doc/posix-functions/fputc.texi: Likewise.
* doc/posix-functions/fputs.texi: Likewise.
* doc/posix-functions/fwrite.texi: Likewise.
* doc/posix-functions/printf.texi: Likewise.
* doc/posix-functions/putc.texi: Likewise.
* doc/posix-functions/putchar.texi: Likewise.
* doc/posix-functions/puts.texi: Likewise.
* doc/posix-functions/vfprintf.texi: Likewise.
* doc/posix-functions/vprintf.texi: Likewise.

 lib/stdio-write.c =
/* POSIX compatible FILE stream write function.
   Copyright (C) 2008 Free Software Foundation, Inc.
   Written by Bruno Haible <[EMAIL PROTECTED]>, 2008.

   This program is free software: you can redistribute it and/or modify
   it under the terms of the GNU General Public License as published by
   the Free Software Foundation; either version 3 of the License, or
   (at your option) any later version.

   This program is distributed in the hope that it will be useful,
   but WITHOUT ANY WARRANTY; without even the implied warranty of
   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
   GNU General Public License for more details.

   You should have received a copy of the GNU General Public License
   along with this program.  If not, see .  */

#include 

/* Specification.  */
#include 

/* Replace these functions only if module 'sigpipe' is requested.  */
#if GNULIB_SIGPIPE

/* On native Windows platforms, SIGPIPE does not exist.  When write() is
   called on a pipe with no readers, WriteFile() fails with error
   GetLastError() = ERROR_NO_DATA, and write() in consequence fails with
   error EINVAL.  This write() function is at the basis of the function
   which flushes the buffer of a FILE stream.  */

# if (defined _WIN32 || defined __WIN32__) && ! defined __CYGWIN__

#  include 
#  include 
#  include 

#  define WIN32_LEAN_AND_MEAN  /* avoid including junk */
#  include 

#  define CALL_WITH_SIGPIPE_EMULATION(RETTYPE, EXPRESSION, FAILED) \
  if (ferror (stream))\
return (EXPRESSION);  \
  else\
{ \
  RETTYPE ret;\
  SetLastError (0);   \
  ret = (EXPRESSION); \
  if (FAILED && GetLastError () == ERROR_NO_DATA && ferror (stream))  \
{ \
  int fd = fileno (stream);   \
  if (fd >= 0 && GetFileType (_get_osfhandle (fd)) == FILE_TYPE_PIPE) \
{ \
  /* Try to raise signal SIGPIPE.  */ \
  raise (SIGPIPE);\
  /* If it is currently blocked or ignored, change errno from \
 EINVAL to EPIPE.  */ \
  errno = EPIPE;  

Re: mingw and SIGPIPE, new modules 'sigpipe' and 'write' (5/7)

[Bruno Haible]

2008-09-26  Bruno Haible  <[EMAIL PROTECTED]>

* modules/safe-write (Depends-on): Add write.

*** modules/safe-write.orig 2008-09-26 12:52:18.0 +0200
--- modules/safe-write  2008-09-26 01:25:18.0 +0200
***
*** 8,13 
--- 8,14 
  
  Depends-on:
  safe-read
+ write
  
  configure.ac:
  gl_SAFE_WRITE

Re: mingw and SIGPIPE, new modules 'sigpipe' and 'write' (4/7)

[Bruno Haible]

2008-09-26  Bruno Haible  <[EMAIL PROTECTED]>

* modules/sigpipe-tests: New file.
* tests/test-sigpipe.c: New file.
* tests/test-sigpipe.sh: New file.

=== modules/sigpipe-tests ==
Files:
tests/test-sigpipe.c
tests/test-sigpipe.sh

Depends-on:
write

configure.ac:

Makefile.am:
TESTS += test-sigpipe.sh
TESTS_ENVIRONMENT += EXEEXT='@EXEEXT@' srcdir='$(srcdir)'
check_PROGRAMS += test-sigpipe

=== tests/test-sigpipe.sh ==
#!/bin/sh

tmpfiles=""
trap 'rm -fr $tmpfiles' 1 2 3 15

# Test signal's default behaviour.
tmpfiles="$tmpfiles t-sigpipeA.tmp"
./test-sigpipe${EXEEXT} A 2> t-sigpipeA.tmp | head -1 > /dev/null
if test -s t-sigpipeA.tmp; then
  LC_ALL=C tr -d '\r' < t-sigpipeA.tmp
  rm -fr $tmpfiles; exit 1
fi

# Test signal's ignored behaviour.
tmpfiles="$tmpfiles t-sigpipeB.tmp"
./test-sigpipe${EXEEXT} B 2> t-sigpipeB.tmp | head -1 > /dev/null
if test -s t-sigpipeB.tmp; then
  LC_ALL=C tr -d '\r' < t-sigpipeB.tmp
  rm -fr $tmpfiles; exit 1
fi

# Test signal's behaviour when a handler is installed.
tmpfiles="$tmpfiles t-sigpipeC.tmp"
./test-sigpipe${EXEEXT} B 2> t-sigpipeC.tmp | head -1 > /dev/null
if test -s t-sigpipeC.tmp; then
  LC_ALL=C tr -d '\r' < t-sigpipeC.tmp
  rm -fr $tmpfiles; exit 1
fi

rm -fr $tmpfiles
exit 0
=== tests/test-sigpipe.c ===
/* Test of SIGPIPE handling.
   Copyright (C) 2008 Free Software Foundation, Inc.

   This program is free software; you can redistribute it and/or modify
   it under the terms of the GNU General Public License as published by
   the Free Software Foundation; either version 3, or (at your option)
   any later version.

   This program is distributed in the hope that it will be useful,
   but WITHOUT ANY WARRANTY; without even the implied warranty of
   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
   GNU General Public License for more details.

   You should have received a copy of the GNU General Public License
   along with this program; if not, write to the Free Software Foundation,
   Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.  */

#include 

#include 

/* Check that SIGPIPE is defined.  */
int s = SIGPIPE;

#include 
#include 
#include 
#include 

static void
handler (int sig)
{
  exit (0);
}

int
main (int argc, char **argv)
{
  char mode = argv[1][0];

  switch (mode)
{
case 'A': signal (SIGPIPE, SIG_DFL); break;
case 'B': signal (SIGPIPE, SIG_IGN); break;
case 'C': signal (SIGPIPE, handler); break;
}

  /* Produce infinite output.  Since it is piped into "head -1", the writes
 must ultimately fail.  */
  for (;;)
{
  char c[2] = { 'y', '\n' };
  int ret = write (1, c, sizeof (c));
  if (ret <= 0)
{
  switch (mode)
{
case 'B': /* The write() call should have failed with EPIPE.  */
  if (ret < 0 && errno == EPIPE)
exit (0);
  /*FALLTHROUGH*/
case 'A': /* The process should silently die.  */
case 'C': /* The handler should have been called.  */
  fprintf (stderr, "write() returned %d with error %d.\n", ret, 
errno);
  exit (1);
}
}
}
}

Re: mingw and SIGPIPE, new modules 'sigpipe' and 'write' (3/7)

[Bruno Haible]

2008-09-26  Bruno Haible  <[EMAIL PROTECTED]>

* modules/write: New file.
* lib/unistd.in.h: Include .
(write): New declaration.
* lib/write.c: New file.
* m4/write.m4: New file.
* m4/unistd_h.m4 (gl_UNISTD_H_DEFAULTS): Initialize
GNULIB_UNISTD_H_SIGPIPE, GNULIB_WRITE, REPLACE_WRITE.
* modules/unistd (Makefile.am): Substitute GNULIB_UNISTD_H_SIGPIPE,
GNULIB_WRITE, REPLACE_WRITE.
* doc/posix-functions/write.texi: Mention the write, sigpipe modules
and the SIGPIPE issue.

= modules/write 
Description:
POSIX compatible write() function: write data to a file descriptor

Files:
lib/write.c
m4/write.m4

Depends-on:
unistd

configure.ac:
gl_FUNC_WRITE
gl_UNISTD_MODULE_INDICATOR([write])

Makefile.am:

Include:


License:
LGPLv2+

Maintainer:
Bruno Haible

== lib/write.c =
/* POSIX compatible write() function.
   Copyright (C) 2008 Free Software Foundation, Inc.
   Written by Bruno Haible <[EMAIL PROTECTED]>, 2008.

   This program is free software: you can redistribute it and/or modify
   it under the terms of the GNU General Public License as published by
   the Free Software Foundation; either version 3 of the License, or
   (at your option) any later version.

   This program is distributed in the hope that it will be useful,
   but WITHOUT ANY WARRANTY; without even the implied warranty of
   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
   GNU General Public License for more details.

   You should have received a copy of the GNU General Public License
   along with this program.  If not, see .  */

#include 

/* Specification.  */
#include 

/* Replace this function only if module 'sigpipe' is requested.  */
#if GNULIB_SIGPIPE

/* On native Windows platforms, SIGPIPE does not exist.  When write() is
   called on a pipe with no readers, WriteFile() fails with error
   GetLastError() = ERROR_NO_DATA, and write() in consequence fails with
   error EINVAL.  */

# if (defined _WIN32 || defined __WIN32__) && ! defined __CYGWIN__

#  include 
#  include 
#  include 

#  define WIN32_LEAN_AND_MEAN  /* avoid including junk */
#  include 

ssize_t
rpl_write (int fd, const void *buf, size_t count)
#undef write
{
  ssize_t ret = write (fd, buf, count);

  if (ret < 0)
{
  if (GetLastError () == ERROR_NO_DATA
  && GetFileType (_get_osfhandle (fd)) == FILE_TYPE_PIPE)
{
  /* Try to raise signal SIGPIPE.  */
  raise (SIGPIPE);
  /* If it is currently blocked or ignored, change errno from EINVAL
 to EPIPE.  */
  errno = EPIPE;
}
}
  return ret;
}

# endif
#endif
== m4/write.m4 =
# write.m4 serial 1
dnl Copyright (C) 2008 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.

AC_DEFUN([gl_FUNC_WRITE],
[
  AC_REQUIRE([gl_UNISTD_H_DEFAULTS])
  dnl This ifdef is just an optimization, to avoid performing a configure
  dnl check whose result is not used. It does not make the test of
  dnl GNULIB_UNISTD_H_SIGPIPE or GNULIB_SIGPIPE redundant.
  m4_ifdef([gl_SIGNAL_SIGPIPE], [
gl_SIGNAL_SIGPIPE
if test $gl_cv_header_signal_h_SIGPIPE != yes; then
  REPLACE_WRITE=1
  AC_LIBOBJ([write])
fi
  ])
])

*** lib/unistd.in.h.orig2008-09-26 12:52:18.0 +0200
--- lib/unistd.in.h 2008-09-26 03:14:02.0 +0200
***
*** 35,40 
--- 35,45 
  /* mingw fails to declare _exit in .  */
  #include 
  
+ #if @GNULIB_WRITE@ && @REPLACE_WRITE@ && @GNULIB_UNISTD_H_SIGPIPE@
+ /* Get ssize_t.  */
+ # include 
+ #endif
+ 
  /* The definition of GL_LINK_WARNING is copied here.  */
  
  
***
*** 333,338 
--- 338,353 
  #endif
  
  
+ #if @GNULIB_WRITE@ && @REPLACE_WRITE@ && @GNULIB_UNISTD_H_SIGPIPE@
+ /* Write up to COUNT bytes starting at BUF to file descriptor FD.
+See the POSIX:2001 specification
+.  */
+ # undef write
+ # define write rpl_write
+ extern ssize_t write (int fd, const void *buf, size_t count);
+ #endif
+ 
+ 
  #ifdef __cplusplus
  }
  #endif
*** m4/unistd_h.m4.orig 2008-09-26 12:52:18.0 +0200
--- m4/unistd_h.m4  2008-09-26 02:34:36.0 +0200
***
*** 1,4 
! # unistd_h.m4 serial 11
  dnl Copyright (C) 2006-2008 Free Software Foundation, Inc.
  dnl This file is free software; the Free Software Foundation
  dnl gives unlimited permission to copy and/or distribute it,
--- 1,4 
! # unistd_h.m4 serial 12
  dnl Copyright (C) 2006-2008 Free Software Foundation, Inc.
  

Re: mingw and SIGPIPE, new modules 'sigpipe' and 'write' (2/7)

[Bruno Haible]

2008-09-26  Bruno Haible  <[EMAIL PROTECTED]>

* lib/signal.in.h (SIGPIPE): Define to a replacement value.
(raise): New declaration.
* lib/sigprocmask.c (SIGPIPE_handler): New variable.
(ext_signal): New function.
(rpl_raise): New function.
* m4/signal_h.m4 (gl_SIGNAL_H_DEFAULTS): Initialize
GNULIB_SIGNAL_H_SIGPIPE.
* modules/signal (Makefile.am): Substitute GNULIB_SIGNAL_H_SIGPIPE.
* doc/posix-headers/signal.texi: Mention the SIGPIPE issue.

*** lib/signal.in.h.orig2008-09-26 12:52:18.0 +0200
--- lib/signal.in.h 2008-09-26 02:30:02.0 +0200
***
*** 44,49 
--- 44,60 
  #endif
  
  
+ #if @GNULIB_SIGNAL_H_SIGPIPE@
+ # ifndef SIGPIPE
+ /* Define SIGPIPE to a value that does not overlap with other signals.  */
+ #  define SIGPIPE 13
+ #  define GNULIB_defined_SIGPIPE 1
+ /* To actually use SIGPIPE, you also need the gnulib modules 'sigprocmask',
+'write', 'stdio'.  */
+ # endif
+ #endif
+ 
+ 
  #if [EMAIL PROTECTED]@
  
  /* Maximum signal number + 1.  */
***
*** 92,99 
--- 103,120 
 handler.  */
  extern void (*signal (int sig, void (*func) (int))) (int);
  
+ # if GNULIB_defined_SIGPIPE
+ 
+ /* Raise signal SIG.  */
+ #  undef raise
+ #  define raise rpl_raise
+ extern int raise (int sig);
+ 
+ # endif
+ 
  #endif /* [EMAIL PROTECTED]@ */
  
+ 
  #if [EMAIL PROTECTED]@
  
  # if [EMAIL PROTECTED]@
*** lib/sigprocmask.c.orig  2008-09-26 12:52:18.0 +0200
--- lib/sigprocmask.c   2008-09-26 11:42:37.0 +0200
***
*** 45,50 
--- 45,75 
  
  typedef void (*handler_t) (int);
  
+ /* Handling of gnulib defined signals.  */
+ 
+ #if GNULIB_defined_SIGPIPE
+ static handler_t SIGPIPE_handler = SIG_DFL;
+ #endif
+ 
+ #if GNULIB_defined_SIGPIPE
+ static handler_t
+ ext_signal (int sig, handler_t handler)
+ {
+   switch (sig)
+ {
+ case SIGPIPE:
+   {
+   handler_t old_handler = SIGPIPE_handler;
+   SIGPIPE_handler = handler;
+   return old_handler;
+   }
+ default: /* System defined signal */
+   return signal (sig, handler);
+ }
+ }
+ # define signal ext_signal
+ #endif
+ 
  int
  sigismember (const sigset_t *set, int sig)
  {
***
*** 240,242 
--- 265,293 
return SIG_ERR;
  }
  }
+ 
+ #if GNULIB_defined_SIGPIPE
+ /* Raise the signal SIG.  */
+ int
+ rpl_raise (int sig)
+ # undef raise
+ {
+   switch (sig)
+ {
+ case SIGPIPE:
+   if (blocked_set & (1U << sig))
+   pending_array[sig] = 1;
+   else
+   {
+ handler_t handler = SIGPIPE_handler;
+ if (handler == SIG_DFL)
+   exit (128 + SIGPIPE);
+ else if (handler != SIG_IGN)
+   (*handler) (sig);
+   }
+   return 0;
+ default: /* System defined signal */
+   return raise (sig);
+ }
+ }
+ #endif
*** m4/signal_h.m4.orig 2008-09-26 12:52:18.0 +0200
--- m4/signal_h.m4  2008-09-26 02:29:06.0 +0200
***
*** 1,4 
! # signal_h.m4 serial 5
  dnl Copyright (C) 2007, 2008 Free Software Foundation, Inc.
  dnl This file is free software; the Free Software Foundation
  dnl gives unlimited permission to copy and/or distribute it,
--- 1,4 
! # signal_h.m4 serial 6
  dnl Copyright (C) 2007, 2008 Free Software Foundation, Inc.
  dnl This file is free software; the Free Software Foundation
  dnl gives unlimited permission to copy and/or distribute it,
***
*** 19,24 
--- 19,25 
  
  AC_DEFUN([gl_SIGNAL_H_DEFAULTS],
  [
+   GNULIB_SIGNAL_H_SIGPIPE=0;   AC_SUBST([GNULIB_SIGNAL_H_SIGPIPE])
GNULIB_SIGPROCMASK=0;AC_SUBST([GNULIB_SIGPROCMASK])
GNULIB_SIGACTION=0;  AC_SUBST([GNULIB_SIGACTION])
dnl Assume proper GNU behavior unless another module says otherwise.
*** modules/signal.orig 2008-09-26 12:52:18.0 +0200
--- modules/signal  2008-09-26 02:30:20.0 +0200
***
*** 23,28 
--- 23,29 
  sed -e 's|@''INCLUDE_NEXT''@|$(INCLUDE_NEXT)|g' \
  -e 's|@''PRAGMA_SYSTEM_HEADER''@|@PRAGMA_SYSTEM_HEADER@|g' \
  -e 's|@''NEXT_SIGNAL_H''@|$(NEXT_SIGNAL_H)|g' \
+ -e 's|@''GNULIB_SIGNAL_H_SIGPIPE''@|$(GNULIB_SIGNAL_H_SIGPIPE)|g' 
\
  -e 's|@''GNULIB_SIGPROCMASK''@|$(GNULIB_SIGPROCMASK)|g' \
  -e 's|@''GNULIB_SIGACTION''@|$(GNULIB_SIGACTION)|g' \
  -e 
's|@''HAVE_POSIX_SIGNALBLOCKING''@|$(HAVE_POSIX_SIGNALBLOCKING)|g' \
*** doc/posix-headers/signal.texi.orig  2008-09-26 12:52:18.0 +0200
--- doc/posix-headers/signal.texi   2008-09-26 00:21:07.0 +0200
***
*** 10,27 
  @item
  @code{sigset_t} is only declared in  on some platforms:
  mingw.
- 
  @item
  @code{struct sigaction} and @code{siginfo_t} are missing on some
  platforms:
  mingw.
- 
  @item
  @code{struct sigaction} lacks the @code{sa_sigaction} member on some
  platforms:
  Irix 5.3, In

Re: [PATCH] add new module filevercmp

[Jim Meyering]

Kamil Dudka <[EMAIL PROTECTED]> wrote:
> On Wednesday 24 September 2008 22:52:00 Jim Meyering wrote:

...  if (!strcmp (s1, s2))

>> > +  int rc = verrevcmp (s1, s1_len, s2, s2_len);
>> > +  return (rc == 0)
>> > +/* return 0 if (and only if) strings S1 and S2 are identical */
>> > +? strcmp(s1, s2) : rc;
>>
>> strcmp(s1, s2) is always nonzero here, since
>> it was tested at the beginning of the function.
> That's true, but I don't think here is a problem. Please consider new function
> description:

But calling strcmp a second time with identical
arguments is unnecessary and probably wasteful.
Or perhaps you verified that gcc's optimization is good
enough to identify and eliminate this duplication?

If you leave the latter strcmp, add a space before its opening
parenthesis.  That this spacing issue remains suggests you
haven't checked everything with GNU indent.  Please do that.

I'll look at the rest later today.

Re: mingw and SIGPIPE, new modules 'sigpipe' and 'write' (1/7)

[Bruno Haible]

2008-09-26  Bruno Haible  <[EMAIL PROTECTED]>

* modules/sigpipe: New file.
* m4/sigpipe.m4: New file.

 modules/sigpipe ==
Description:
Emulation of POSIX compatible SIGPIPE behaviour.

Files:
m4/sigpipe.m4

Depends-on:
signal
sigprocmask
stdio
unistd

configure.ac:
gl_SIGNAL_SIGPIPE
dnl Define the C macro GNULIB_SIGPIPE to 1.
gl_MODULE_INDICATOR([sigpipe])
dnl Define the substituted variable GNULIB_SIGNAL_H_SIGPIPE to 1.
AC_REQUIRE([gl_SIGNAL_H_DEFAULTS])
GNULIB_SIGNAL_H_SIGPIPE=1
dnl Define the substituted variable GNULIB_STDIO_H_SIGPIPE to 1.
AC_REQUIRE([gl_STDIO_H_DEFAULTS])
GNULIB_STDIO_H_SIGPIPE=1
dnl Define the substituted variable GNULIB_UNISTD_H_SIGPIPE to 1.
AC_REQUIRE([gl_UNISTD_H_DEFAULTS])
GNULIB_UNISTD_H_SIGPIPE=1

Makefile.am:

Include:


License:
LGPL

Maintainer:
Bruno Haible

= m4/sigpipe.m4 ===
# sigpipe.m4 serial 1
dnl Copyright (C) 2008 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.

dnl Tests whether SIGPIPE is provided by .
dnl Sets gl_cv_header_signal_h_SIGPIPE.
AC_DEFUN([gl_SIGNAL_SIGPIPE],
[
  dnl Use AC_REQUIRE here, so that the default behavior below is expanded
  dnl once only, before all statements that occur in other macros.
  AC_REQUIRE([gl_SIGNAL_SIGPIPE_BODY])
])

AC_DEFUN([gl_SIGNAL_SIGPIPE_BODY],
[
  AC_REQUIRE([AC_PROG_CC])
  AC_CACHE_CHECK([for SIGPIPE], gl_cv_header_signal_h_SIGPIPE, [
AC_EGREP_CPP(booboo,[
#include 
#if !defined SIGPIPE
booboo
#endif
  ],
  [gl_cv_header_signal_h_SIGPIPE=no],
  [gl_cv_header_signal_h_SIGPIPE=yes])
  ])
])
===

mingw and SIGPIPE, new modules 'sigpipe' and 'write'

[Bruno Haible]

Simon Josefsson wrote:
> Right now, the gnulib modules that fail to build under mingw32 are (see
> log): savewd, sigpipe-die, getugroups, idcache, and userspec.

Interesting... 'glob' was also in this list, but was ported to mingw by Yoann.

'sigpipe-die' is new, however. Native Windows does not have SIGPIPE.
What happens when writing a pipe with no readers?

  $ msgcat de.po | head -1
  # German translations for GNU gettext-runtime package
  msgcat.exe: error while writing "standard output" file: Invalid argument

The desired behaviour, as on Unix, is that the msgcat program terminates
without an error message.

In such a situation, the write() call fails with errno = EINVAL,
GetLastError() = ERROR_NO_DATA.

This is quite pervasive. Obviously we don't want to hack every write()
or fwrite() calls in all programs to get this behaviour right. The
following functions all need to be extended in gnulib.
  - : define SIGPIPE
  - signal, sigprocmask, sigaction: support SIGPIPE
  - raise: support SIGPIPE
  - write: check for ERROR_NO_DATA and potentially raise SIGPIPE
  - fwrite, fputs, puts, fputc, putc, putchar, [v]fprintf, [v]printf: likewise

I'm applying this in 7 commits.

Bruno


2008-09-26  Bruno Haible  <[EMAIL PROTECTED]>

* modules/sigpipe-die (Depends-on): Add sigpipe.

* lib/stdio.in.h (fprintf, vfprintf, printf, vprintf, fputc, putc,
putchar, fputs, puts, fwrite): Replace when REPLACE_STDIO_WRITE_FUNCS
and GNULIB_STDIO_H_SIGPIPE are set.
* lib/stdio-write.c: New file.
* m4/stdio_h.m4 (gl_STDIO_H): Set GNULIB_FPRINTF, GNULIB_PRINTF,
GNULIB_VFPRINTF, GNULIB_VPRINTF, GNULIB_FPUTC, GNULIB_PUTC,
GNULIB_PUTCHAR, GNULIB_FPUTS, GNULIB_PUTS, GNULIB_FWRITE,
REPLACE_STDIO_WRITE_FUNCS.
(gl_STDIO_H_DEFAULTS): Initialize GNULIB_FPRINTF, GNULIB_PRINTF,
GNULIB_VFPRINTF, GNULIB_VPRINTF, GNULIB_FPUTC, GNULIB_PUTC,
GNULIB_PUTCHAR, GNULIB_FPUTS, GNULIB_PUTS, GNULIB_FWRITE,
GNULIB_STDIO_H_SIGPIPE, REPLACE_STDIO_WRITE_FUNCS.
* modules/stdio (Files): Add lib/stdio-write.c.
(Makefile.am): Substitute GNULIB_FPRINTF, GNULIB_PRINTF,
GNULIB_VFPRINTF, GNULIB_VPRINTF, GNULIB_FPUTC, GNULIB_PUTC,
GNULIB_PUTCHAR, GNULIB_FPUTS, GNULIB_PUTS, GNULIB_FWRITE,
GNULIB_STDIO_H_SIGPIPE, REPLACE_STDIO_WRITE_FUNCS.
* m4/fprintf-posix.m4 (gl_REPLACE_FPRINTF): Define
REPLACE_FPRINTF_POSIX.
* m4/printf-posix-rpl.m4 (gl_REPLACE_PRINTF): Define
REPLACE_PRINTF_POSIX.
* m4/vfprintf-posix.m4 (gl_REPLACE_VFPRINTF): Define
REPLACE_VFPRINTF_POSIX.
* m4/vprintf-posix.m4 (gl_REPLACE_VPRINTF): Define
REPLACE_VPRINTF_POSIX.
* doc/posix-functions/fprintf.texi: Mention the sigpipe module and the
SIGPIPE issue.
* doc/posix-functions/fputc.texi: Likewise.
* doc/posix-functions/fputs.texi: Likewise.
* doc/posix-functions/fwrite.texi: Likewise.
* doc/posix-functions/printf.texi: Likewise.
* doc/posix-functions/putc.texi: Likewise.
* doc/posix-functions/putchar.texi: Likewise.
* doc/posix-functions/puts.texi: Likewise.
* doc/posix-functions/vfprintf.texi: Likewise.
* doc/posix-functions/vprintf.texi: Likewise.

* modules/safe-write (Depends-on): Add write.

* modules/sigpipe-tests: New file.
* tests/test-sigpipe.c: New file.
* tests/test-sigpipe.sh: New file.

* modules/write: New file.
* lib/unistd.in.h: Include .
(write): New declaration.
* lib/write.c: New file.
* m4/write.m4: New file.
* m4/unistd_h.m4 (gl_UNISTD_H_DEFAULTS): Initialize
GNULIB_UNISTD_H_SIGPIPE, GNULIB_WRITE, REPLACE_WRITE.
* modules/unistd (Makefile.am): Substitute GNULIB_UNISTD_H_SIGPIPE,
GNULIB_WRITE, REPLACE_WRITE.
* doc/posix-functions/write.texi: Mention the write, sigpipe modules
and the SIGPIPE issue.

* lib/signal.in.h (SIGPIPE): Define to a replacement value.
(raise): New declaration.
* lib/sigprocmask.c (SIGPIPE_handler): New variable.
(ext_signal): New function.
(rpl_raise): New function.
* m4/signal_h.m4 (gl_SIGNAL_H_DEFAULTS): Initialize
GNULIB_SIGNAL_H_SIGPIPE.
* modules/signal (Makefile.am): Substitute GNULIB_SIGNAL_H_SIGPIPE.
* doc/posix-headers/signal.texi: Mention the SIGPIPE issue.

* modules/sigpipe: New file.
* m4/sigpipe.m4: New file.

Re: [PATCH] add new module filevercmp

[Kamil Dudka]

Hello Bruno,

thank you for review. I've attempted to cover all your suggestions.


On Thursday 25 September 2008 02:18:50 Bruno Haible wrote:
> Some systems still don't have C99 compilers. Therefore you cannot use
> variable declarations after statements. Either collect the variable
> declarations at the top, or introduce blocks { ... } around the desired
> scopes of the variables.
Nobody wrote it here in the list but it seems that gcc -std=c89 is the way to 
discover such portability problems.


Kamil

Re: [PATCH] add new module filevercmp

[Kamil Dudka]

Hello Jim,

thank you for review. I've made the changes you requested, new patch attached.


On Wednesday 24 September 2008 22:52:00 Jim Meyering wrote:
> > +  int rc = verrevcmp (s1, s1_len, s2, s2_len);
> > +  return (rc == 0)
> > +/* return 0 if (and only if) strings S1 and S2 are identical */
> > +? strcmp(s1, s2) : rc;
>
> strcmp(s1, s2) is always nonzero here, since
> it was tested at the beginning of the function.
That's true, but I don't think here is a problem. Please consider new function 
description:
+   3) If both (PREFIX and  VERSION) are equal, strcmp function is used for
+  comparison. So this function can return 0 if (and only if) strings S1
+  and S2 are identical.

> > +  /* these results differ from strverscmp
> > +  ASSERT (filevercmp ("000", "00") < 0);
> > +  ASSERT (filevercmp ("00", "000") > 0); */
>
> Rather than just commenting out these two, instead, assert
> the correct-for-filevercmp condition and add a comment per line.
>
> > +  ASSERT (filevercmp ("a0", "a") > 0);
> > +  ASSERT (filevercmp ("00", "01") < 0);
> > +  ASSERT (filevercmp ("01", "010") < 0);
> > +  /* these results differ from strverscmp
> > +  ASSERT (filevercmp ("010", "09") < 0);
> > +  ASSERT (filevercmp ("09", "0") < 0); */
>
> Likewise.
I removed both of them completely since there are a bit confusing in the 
context of filevercmp. It ignores leading zeros and if the versions are 
equal, strcmp is used to distinguish theirs (deterministic) order.

Hopefully I didn't forget anything.


Kamil
From 3876edd33ad0295e74bb3888316d637e56603b8d Mon Sep 17 00:00:00 2001
From: Kamil Dudka <[EMAIL PROTECTED]>
Date: Fri, 26 Sep 2008 12:11:10 +0200
Subject: [PATCH] add new module filevercmp

* lib/filevercmp.h: New function filevercmp comparing version strings.
* lib/filevercmp.c: Implementation of filevercmp function.
* modules/filevercmp: Module metadata.
* tests/test-filevercmp.c: Unit test for new module.
* modules/filevercmp-tests: Unit test metadata.
* MODULES.html.sh: Add filevercmp module.
---
 MODULES.html.sh  |1 +
 lib/filevercmp.c |  156 ++
 lib/filevercmp.h |   40 
 modules/filevercmp   |   25 +++
 modules/filevercmp-tests |   11 +++
 tests/test-filevercmp.c  |   96 
 6 files changed, 329 insertions(+), 0 deletions(-)
 create mode 100644 lib/filevercmp.c
 create mode 100644 lib/filevercmp.h
 create mode 100644 modules/filevercmp
 create mode 100644 modules/filevercmp-tests
 create mode 100644 tests/test-filevercmp.c

diff --git a/MODULES.html.sh b/MODULES.html.sh
index afaf8ba..3ab5a90 100755
--- a/MODULES.html.sh
+++ b/MODULES.html.sh
@@ -1856,6 +1856,7 @@ func_all_modules ()
   func_module readtokens
   func_module readtokens0
   func_module strverscmp
+  func_module filevercmp
   func_end_table

   element="Support for systems lacking ISO C 99"
diff --git a/lib/filevercmp.c b/lib/filevercmp.c
new file mode 100644
index 000..e22d763
--- /dev/null
+++ b/lib/filevercmp.c
@@ -0,0 +1,156 @@
+/*
+TODO: copyright?
+
+   This program is free software: you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation, either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see . */
+
+#include 
+#include "filevercmp.h"
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define ISALNUM(c) isalnum ((unsigned char) (c))
+#define ISDIGIT(c) isdigit ((unsigned char) (c))
+#define ISALPHA(c) isalpha ((unsigned char) (c))
+
+/*
+   match file suffix defined as RE (\.[A-Za-z][A-Za-z0-9]*)*$
+
+   Scan string pointed by *str and return pointer to suffix begin or NULL if
+   not found. Pointer *str points to ending zero of scanned string after
+   return. */
+static const char *
+match_suffix (const char **str)
+{
+  const char *match = NULL;
+  bool read_alpha = false;
+  while (**str)
+{
+  if (read_alpha)
+{
+  read_alpha = false;
+  if (!ISALPHA (**str))
+match = NULL;
+}
+  else if ('.'  == **str)
+{
+  read_alpha = true;
+  if (!match)
+match = *str;
+}
+  else if (!ISALNUM (**str))
+match = NULL;
+  (*str)++;
+}
+  return match;
+}
+
+/* verrevcmp helper function */
+static inline int
+order (unsigned char c)
+{
+  if (ISDIGIT (c))
+return 0;
+  else if (ISALPHA (c))
+return c;
+  else if (c == '~')
+return -1;
+  else
+return (int)c + UCHAR_MAX +

Re: Changing module from LGPL to LGPLv2+

[Jim Meyering]

Bruno Haible <[EMAIL PROTECTED]> wrote:
> Yoann Vandoorselaere wrote:
>> would it be possible to change their license from LGPL to LGPLv2+?
...
>> raise
>
> Jim?

Sure.  It's trivial.

Re: Changing module from LGPL to LGPLv2+

[Bruno Haible]

Yoann Vandoorselaere wrote:
> would it be possible to change their license from LGPL to LGPLv2+?

> dup2
> sleep

For these modules, we promised that we would revert to LGPLv2+ upon request,
and there were no changes since then. So, these are OK.

> fopen
> signal
> sigprocmask

Changed by Eric and me. Fine with me. Eric?

> strerror

Changed by Paul and me. Fine with me. Paul?

> intprops

Paul?

> raise

Jim?

> perror

Fine with me.

Bruno

Re: fts: avoid O(n^2) ext3/ext4 performance hit on million+-entry dirs

[Jim Meyering]

Jim Meyering <[EMAIL PROTECTED]> wrote:
> Bruno Haible <[EMAIL PROTECTED]> wrote:
>> Jim Meyering wrote:
>>> +   if (nitems > _FTS_INODE_SORT_DIR_ENTRIES_THRESHOLD
>>> +   && !sp->fts_compar
>>> +   && dirent_inode_sort_may_be_useful (sp)) {
>>> +   sp->fts_compar = fts_compare_ino;
>>> +   head = fts_sort (sp, head, nitems);
>>> +   sp->fts_compar = NULL;
>>> +   }
>>
>> This uses fts_sort, which uses qsort, which is O(n^2) in the worst case.
>> (Even the implementation in glibc can be O(n^2), if it guesses that mergesort
>> takes too much memory.)
>>
>> Couldn't this be avoided by using Paul's mpsort module, and changing fts_sort
>> to allocate 50% more room, as scratch space for mpsort?
>
> Good point!
> That would make a fine improvement, as a separate patch.
>
> The new use of qsort in the code I've just proposed for
> rm would benefit from the same treatment.

On second thought, I'm not so sure.
glibc's qsort function does revert to quicksort, but
only upon failure to malloc a buffer for indirect sorting.
Yet each of fts.c and remove.c is already sorting an array
of pointers, so there's no gain in the indirection.

And even if it does resort to using quicksort, the odds of
non-contrived input (the inode numbers) provoking O(N^2)
performance seem vanishingly small.

However, I note that fts_sort would benefit from a rewrite
to make it use mpsort rather than manually setting up its
array of pointers.

Changing module from LGPL to LGPLv2+

[Yoann Vandoorselaere]

Hi,

I'd like to use the following modules in libprelude, would it be
possible to change their license from LGPL to LGPLv2+?

dup2
fopen
intprops  
perror
raise 
signal
sigprocmask   
sleep 
strerror


Thanks!

-- 
Yoann Vandoorselaere | Responsable R&D / CTO | PreludeIDS Technologies
Tel: +33 (0)9 50 70 21 58  Fax: +33(0)9 57 25 21 58
http://www.prelude-ids.com