Re: [bug #17903] cp/mv only read/write 512 byte blocks when filesystem blksize > 4MB

[Tony Ernst]

On Tue, Oct 03, 2006 at 03:42:30PM -0700, Paul Eggert wrote:
> The GNU philosophy is to avoid arbitrary limits, so I installed the
> following patch, in an attempt to do a minimal sort of sanity check
> (catching the HP-UX bug, I hope) while not overly constraining
> st_blksize.

Thank you.



___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils

[bug #17903] cp/mv only read/write 512 byte blocks when filesystem blksize > 4MB

[Paul Eggert]

Update of bug #17903 (project coreutils):

  Status:None => Fixed  
 Open/Closed:Open => Closed 

___

Follow-up Comment #2:

Fixed as described in
.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/



___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils

Re: [bug #17903] cp/mv only read/write 512 byte blocks when filesystem blksize > 4MB

[Paul Eggert]

Tony Ernst <[EMAIL PROTECTED]> writes:

>   && (statbuf).st_blksize != 2147421096) \

I doubt whether the HP-UX bug always results in exactly that number.

Phillip Susi <[EMAIL PROTECTED]> writes:

> In that case I suspect that XFS really should not be
> setting that value to many megabytes.

As I understand it, XFS will let a sysadmin put pretty much whatever
number they like into st_blksize, regardless of the underyling I/O
configuration, I guess under the theory that the sysadmin is always
right.

The GNU philosophy is to avoid arbitrary limits, so I installed the
following patch, in an attempt to do a minimal sort of sanity check
(catching the HP-UX bug, I hope) while not overly constraining
st_blksize.

2006-10-03  Paul Eggert  <[EMAIL PROTECTED]>

* src/system.h (ST_BLKSIZE): Ceiling at SIZE_MAX / 8 + 1, not at 4
MiB, since XFS hosts can legitimately have large values of
st_blksize.  Problem reported by Tony Ernst in
.

--- src/system.h3 Sep 2006 02:53:37 -   1.160
+++ src/system.h3 Oct 2006 22:10:15 -
@@ -206,11 +206,14 @@
 /* Some systems, like Sequents, return st_blksize of 0 on pipes.
Also, when running `rsh hpux11-system cat any-file', cat would
determine that the output stream had an st_blksize of 2147421096.
-   So here we arbitrarily limit the `optimal' block size to 4MB.
-   If anyone knows of a system for which the legitimate value for
-   st_blksize can exceed 4MB, please report it as a bug in this code.  */
+   Conversely st_blksize can be 2 GiB (or maybe even larger) with XFS
+   on 64-bit hosts.  Somewhat arbitrarily, limit the `optimal' block
+   size to SIZE_MAX / 8 + 1.  (Dividing SIZE_MAX by only 4 wouldn't
+   suffice, since "cat" sometimes multiplies the result by 4.)  If
+   anyone knows of a system for which this limit is too small, please
+   report it as a bug in this code.  */
 # define ST_BLKSIZE(statbuf) ((0 < (statbuf).st_blksize \
-  && (statbuf).st_blksize <= (1 << 22)) /* 4MB */ \
+  && (statbuf).st_blksize <= SIZE_MAX / 8 + 1) \
  ? (statbuf).st_blksize : DEV_BSIZE)
 # if defined hpux || defined __hpux__ || defined __hpux
 /* HP-UX counts st_blocks in 1024-byte units.


___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils

Re: [bug #17903] cp/mv only read/write 512 byte blocks when filesystem blksize > 4MB

[Phillip Susi]

Tony Ernst wrote:
I believe the larger block sizes are especially beneficial with RAID. 
I'm adding Geoffrey Wehrman to the CC list, as he understands disk I/O 
much better than I do.


I believe most kernels always performs the actual IO in the same size 
chunks due to the block layer and cache, even if user space passes down 
a large buffer.  The exception to this on Linux would be when you use 
O_DIRECT IO, then using buffer sizes at least as large as the stripe 
width is definitely good for keeping all the disks spinning.


In any case, the raid case falls under the pervue of the kernel buffer 
cache read-ahead mechanism, and is beyond the scope of coreutils.  From 
the perspective of coreutils, using a larger buffer size has the benefit 
of reducing the number of system calls needed, but causes more memory to 
be locked down.  As such you need to have some kind of limit on the size 
of the buffer so you don't try to exhaust system memory.


On the other hand, in a way it is up to the kernel to provide a 
reasonable value for the block size knowing full well that applications 
that use that value use it as a guide to the IO buffer size to use.  In 
that case I suspect that XFS really should not be setting that value to 
many megabytes.




I don't think a cap is really necessary.  This test and arbitrary 
limitation were put in to work around a specific problem on hpux.  But 
it has side-effects that reach beyond hpux and the original problem. 
So the test should either be limited to hpux-only, or fixed to get rid 
of the side-effects by making it more specific to the original problem.







___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils

Re: [bug #17903] cp/mv only read/write 512 byte blocks when filesystem blksize > 4MB

[Tony Ernst]

I believe the larger block sizes are especially beneficial with RAID. 
I'm adding Geoffrey Wehrman to the CC list, as he understands disk I/O 
much better than I do.

I don't think a cap is really necessary.  This test and arbitrary 
limitation were put in to work around a specific problem on hpux.  But 
it has side-effects that reach beyond hpux and the original problem. 
So the test should either be limited to hpux-only, or fixed to get rid 
of the side-effects by making it more specific to the original problem.


On Tue, Oct 03, 2006 at 02:02:58PM -0400, Phillip Susi wrote:
> Why not simply cap the size at 4 MB?  If it is greater than 4 MB just go 
> with 4 MB instead of 512 bytes.  In fact, you might even want to cap it 
> at less than 4 MB, say 1 MB or 512 KB.  I think you will find that any 
> size larger than the 32-128 kb range yields no further performance 
> increase and can even be detrimental due to the increased memory pressure.
> 
> Tony Ernst wrote:
> >Hi Paul,
> >
> >Unfortunately, there isn't really a "largest" legitimate st_blksize 
> >for XFS file systems, or I should say the maximum is whatever fits 
> >into a size_t (64 bits).  It's dependent on the stripe width.  I 
> >talked with an XFS developer who told me that 2GB is not out of the 
> >question today.
> >
> >Now there is also the question of whether or not we really want cp/mv
> >allocating a 2GB buffer, but that's a different issue (and a site with 
> >that kind of disk set-up probably also has plenty of memory).
> >
> >Since the 4MB check was to fix a specific problem on hpux, it seems 
> >like that check should occur inside the "# if defined hpux ..." section.
> >At the very least, since the bogus value returned by hpux is such an 
> >strange number, maybe we could just change:
> > && (statbuf).st_blksize <= (1 << 22)) /* 4MB */ \
> >to:
> > && (statbuf).st_blksize != 2147421096) \
> >
> >I would be very surprised if 2147421096 was ever a valid st_blksize on 
> >any system/filesystem.  It's not a power of 2, or even a multiple 
> >of 128, 512, etc.
> >
> >% factor 2147421096
> >2147421096: 2 2 2 3 3 17 37 47417
> >
> >Thanks,
> >Tony



___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils

Re: [bug #17903] cp/mv only read/write 512 byte blocks when filesystem blksize > 4MB

[Phillip Susi]

Why not simply cap the size at 4 MB?  If it is greater than 4 MB just go 
with 4 MB instead of 512 bytes.  In fact, you might even want to cap it 
at less than 4 MB, say 1 MB or 512 KB.  I think you will find that any 
size larger than the 32-128 kb range yields no further performance 
increase and can even be detrimental due to the increased memory pressure.


Tony Ernst wrote:

Hi Paul,

Unfortunately, there isn't really a "largest" legitimate st_blksize 
for XFS file systems, or I should say the maximum is whatever fits 
into a size_t (64 bits).  It's dependent on the stripe width.  I 
talked with an XFS developer who told me that 2GB is not out of the 
question today.


Now there is also the question of whether or not we really want cp/mv
allocating a 2GB buffer, but that's a different issue (and a site with 
that kind of disk set-up probably also has plenty of memory).


Since the 4MB check was to fix a specific problem on hpux, it seems 
like that check should occur inside the "# if defined hpux ..." section.
At the very least, since the bogus value returned by hpux is such an 
strange number, maybe we could just change:

&& (statbuf).st_blksize <= (1 << 22)) /* 4MB */ \
to:
&& (statbuf).st_blksize != 2147421096) \

I would be very surprised if 2147421096 was ever a valid st_blksize on 
any system/filesystem.  It's not a power of 2, or even a multiple 
of 128, 512, etc.


% factor 2147421096
2147421096: 2 2 2 3 3 17 37 47417

Thanks,
Tony




___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils

Re: [bug #17903] cp/mv only read/write 512 byte blocks when filesystem blksize > 4MB

[Tony Ernst]

On Tue, Oct 03, 2006 at 12:24:14AM -0700, Paul Eggert wrote:
> Tony Ernst <[EMAIL PROTECTED]> writes:
> 
> > XFS filesystems can have legitimate st_blksize values that greatly exceed
> > 4MB.
> 
> Thanks for reporting this.  What is the largest legitimate st_blksize
> value that XFS file systems can have?  We can simply increase the
> value to that limit.

Hi Paul,

Unfortunately, there isn't really a "largest" legitimate st_blksize 
for XFS file systems, or I should say the maximum is whatever fits 
into a size_t (64 bits).  It's dependent on the stripe width.  I 
talked with an XFS developer who told me that 2GB is not out of the 
question today.

Now there is also the question of whether or not we really want cp/mv
allocating a 2GB buffer, but that's a different issue (and a site with 
that kind of disk set-up probably also has plenty of memory).

Since the 4MB check was to fix a specific problem on hpux, it seems 
like that check should occur inside the "# if defined hpux ..." section.
At the very least, since the bogus value returned by hpux is such an 
strange number, maybe we could just change:
&& (statbuf).st_blksize <= (1 << 22)) /* 4MB */ \
to:
&& (statbuf).st_blksize != 2147421096) \

I would be very surprised if 2147421096 was ever a valid st_blksize on 
any system/filesystem.  It's not a power of 2, or even a multiple 
of 128, 512, etc.

% factor 2147421096
2147421096: 2 2 2 3 3 17 37 47417

Thanks,
Tony



___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils

Re: [bug #17903] cp/mv only read/write 512 byte blocks when filesystem blksize > 4MB

[Paul Eggert]

Tony Ernst <[EMAIL PROTECTED]> writes:

> XFS filesystems can have legitimate st_blksize values that greatly exceed
> 4MB.

Thanks for reporting this.  What is the largest legitimate st_blksize
value that XFS file systems can have?  We can simply increase the
value to that limit.


___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils

[bug #17903] cp/mv only read/write 512 byte blocks when filesystem blksize > 4MB

[Tony Ernst]

Follow-up Comment #1, bug #17903 (project coreutils):

I am attaching a patch against CVS that reverses the change (described above)
that caused this problem.

If you don't want to totally remove it, perhaps it could be special cased
somehow to only use the 4MB hack on HP-UX builds?

___

Additional Item Attachment:

File name: coreutils-st_blksize.patch Size:1 KB
Patch to remove artificial 4MB st_blksize limitation.


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/



___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils

[bug #17903] cp/mv only read/write 512 byte blocks when filesystem blksize > 4MB

[Tony Ernst]

URL:
  

 Summary: cp/mv only read/write 512 byte blocks when
filesystem blksize > 4MB
 Project: GNU Core Utilities
Submitted by: tonyernst
Submitted on: Monday 10/02/2006 at 20:29
Category: None
Severity: 3 - Normal
  Item Group: None
  Status: None
 Privacy: Public
 Assigned to: None
 Open/Closed: Open

___

Details:

A user is complaining about serious performance problems after upgrading
their system.  I found that this was due to the following change in the
coreutils source file system.h:

coreutils-4.5.3:

#else /* HAVE_STRUCT_STAT_ST_BLOCKS */
/* Some systems, like Sequents, return st_blksize of 0 on pipes. */
# define ST_BLKSIZE(statbuf) ((statbuf).st_blksize > 0 \
   ? (statbuf).st_blksize : DEV_BSIZE)

coreutils-5.2.1:

#else /* HAVE_STRUCT_STAT_ST_BLOCKS */
/* Some systems, like Sequents, return st_blksize of 0 on pipes.
   Also, when running `rsh hpux11-system cat any-file', cat would
   determine that the output stream had an st_blksize of 2147421096.
   So here we arbitrarily limit the `optimal' block size to 4MB.
   If anyone knows of a system for which the legitimate value for
   st_blksize can exceed 4MB, please report it as a bug in this code.  */
# define ST_BLKSIZE(statbuf) ((0 < (statbuf).st_blksize \
   && (statbuf).st_blksize <= (1 << 22)) /* 4MB
*/ \
  ? (statbuf).st_blksize : DEV_BSIZE)

XFS filesystems can have legitimate st_blksize values that greatly exceed
4MB.

In the case of the user I referred to above, their st_blksize was 6MB.  With
coreutils-4.5.3, they saw 6MB reads/writes.  With coreutils-5.2.1, the
reads/writes were all 512-bytes.







___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/



___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils