Re: [coreutils] Memory usage of parallel sort

[Jim Meyering]

Pádraig Brady wrote:
> On 17/12/10 19:29, Assaf Gordon wrote:
...
> Thanks for looking at this!
>
> I've not looked into the memory details,
> but this is another reason to restrict
> the default number of threads to 8,
> which we talked about previously...
>
> commit 69ef9deef087b0447c05d9a29825d7a714a1
> Author: Pádraig Brady 
> Date:   Sat Dec 18 05:27:46 2010 +
>
> sort: use at most 8 threads by default
>
> * src/sort.c (main): If --parallel isn't specified,
> restrict the number of threads to 8 by default.
> If the --parallel option is specified, then
> allow any number of threads to be set, independent
> of the number of processors on the system.
>
> diff --git a/NEWS b/NEWS
> index 484ed5c..7eda1b2 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -27,6 +27,12 @@ GNU coreutils NEWS-*- 
> outline -*-
>
>sort -m -o f f ... f no longer dumps core when file descriptors are 
> limited.
>
> +** Changes in behavior
> +
> +  sort will not create more than 8 threads by default due to diminishing
> +  performance gains.  Also the --parallel option is no longer restricted
> +  to the number of available processors.

That change looks fine.  Please push it.
I'm glad you are mentioning how --parallel is no longer restricted.

Re: [coreutils] [PATCH] tests: set fail=0 by default

[Paul Eggert]

On 12/17/2010 11:55 PM, Jim Meyering wrote:

> This change probably belongs in gnulib's tests/init.sh, too.

OK, I installed the following (slightly different) version of
the change into gnulib.  Next I'll propagate the latest gnulib version
of init.sh into coreutils.

diff --git a/ChangeLog b/ChangeLog
index 8b02175..1ee355c 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,11 @@
+2010-12-18  Paul Eggert  
+
+   tests: set fail=0 at start
+   * tests/init.sh (setup_): Move fail=0 initialization here ...
+   (mktempd_): ... from here, so that tests can rely on fail being
+   set to 0 initially.  This fixes a problem in coreutils; see:
+   http://lists.gnu.org/archive/html/coreutils/2010-12/msg00083.html
+
 2010-12-18  Bruno Haible  
 
memmem-simple: Stylistic changes.
diff --git a/tests/init.sh b/tests/init.sh
index 14f3b19..4d89a1a 100644
--- a/tests/init.sh
+++ b/tests/init.sh
@@ -324,6 +324,7 @@ setup_()
   fi
 
   initial_cwd_=$PWD
+  fail=0
 
   pfx_=`testdir_prefix_`
   test_dir_=`mktempd_ "$initial_cwd_" "$pfx_-$ME_."` \
@@ -416,8 +417,6 @@ mktempd_()
   *) fail_ "invalid template: $template_ (must have a suffix of at least 4 
X's)";;
   esac
 
-  fail=0
-
   # First, try to use mktemp.
   d=`unset TMPDIR; mktemp -d -t -p "$destdir_" "$template_" 2>/dev/null` \
 || fail=1
-- 
1.7.2

[coreutils] [PATCH] tests: sync init.sh from gnulib

[Paul Eggert]

* tests/init.sh (setup_): Initialize fail=0 before invoking mktempd_.
Ensure that IFS is defined initially.
(mktempd_): Remove fail=0 initialization; no longer needed.
---
 tests/init.sh |   12 +++-
 1 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/tests/init.sh b/tests/init.sh
index 7be5e9e..4d89a1a 100644
--- a/tests/init.sh
+++ b/tests/init.sh
@@ -324,21 +324,25 @@ setup_()
   fi
 
   initial_cwd_=$PWD
+  fail=0
 
   pfx_=`testdir_prefix_`
   test_dir_=`mktempd_ "$initial_cwd_" "$pfx_-$ME_."` \
 || fail_ "failed to create temporary directory in $initial_cwd_"
   cd "$test_dir_"
 
+  # As autoconf-generated configure scripts do, ensure that IFS
+  # is defined initially, so that saving and restoring $IFS works.
+  gl_init_sh_nl_='
+'
+  IFS=" "" $gl_init_sh_nl_"
+
   # This trap statement, along with a trap on 0 below, ensure that the
   # temporary directory, $test_dir_, is removed upon exit as well as
   # upon receipt of any of the listed signals.
   for sig_ in 1 2 3 13 15; do
 eval "trap 'Exit $(expr $sig_ + 128)' $sig_"
   done
-
-  # Set up for the "Exit $fail" at the end of many tests.
-  fail=0
 }
 
 # Create a temporary directory, much like mktemp -d does.
@@ -413,8 +417,6 @@ mktempd_()
   *) fail_ "invalid template: $template_ (must have a suffix of at least 4 
X's)";;
   esac
 
-  fail=0
-
   # First, try to use mktemp.
   d=`unset TMPDIR; mktemp -d -t -p "$destdir_" "$template_" 2>/dev/null` \
 || fail=1
-- 
1.7.2

[coreutils] [PATCH] acl: port to Solaris 8 when copying from tmpfs to ufs

[Paul Eggert]

* lib/copy-acl.c (qcopy_acl): Also allow EINVAL as an ignorable
error number.  Problem observed on Solaris 8 with latest
coreutils, with "mv A B", where A is on a tmpfs file system and B
is on a ufs file system.  This caused coreutils' mv/part-symlink
test to fail.
---
 ChangeLog  |7 +++
 lib/copy-acl.c |2 +-
 2 files changed, 8 insertions(+), 1 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 1ee355c..6656b45 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,12 @@
 2010-12-18  Paul Eggert  
 
+   acl: port to Solaris 8 when copying from tmpfs to ufs
+   * lib/copy-acl.c (qcopy_acl): Also allow EINVAL as an ignorable
+   error number.  Problem observed on Solaris 8 with latest
+   coreutils, with "mv A B", where A is on a tmpfs file system and B
+   is on a ufs file system.  This caused coreutils' mv/part-symlink
+   test to fail.
+
tests: set fail=0 at start
* tests/init.sh (setup_): Move fail=0 initialization here ...
(mktempd_): ... from here, so that tests can rely on fail being
diff --git a/lib/copy-acl.c b/lib/copy-acl.c
index e8a4460..fbae482 100644
--- a/lib/copy-acl.c
+++ b/lib/copy-acl.c
@@ -358,7 +358,7 @@ qcopy_acl (const char *src_name, int source_desc, const 
char *dst_name,
   if (ret < 0 && saved_errno == 0)
 {
   saved_errno = errno;
-  if ((errno == ENOSYS || errno == EOPNOTSUPP)
+  if ((errno == ENOSYS || errno == EOPNOTSUPP || errno == EINVAL)
   && !acl_nontrivial (count, entries))
 saved_errno = 0;
 }
-- 
1.7.2