Re: better buffer size for copy
What would such network filesystems report as their blocksize? I have a feeling it isn't going to be on the order of a MB. At least for local filesystems, the ideal transfer block size is going to be quite a bit larger than the filesystem block size ( if the filesystem is even block oriented... think reiser4, or cramfs ). In the case of network filesystems, they should be performing readahead in the background between small block copies to keep the pipeline full. As long as the copy program isn't blocked elsewhere for long periods, say in the write to the destination, then the readahead mechanism should keep the pipeline full. Up to a point, using larger block sizes saves some cpu by lowering the number of system calls. After a certain point, the copy program can start to waste enough time in the write that the readahead stops and stalls the pipeline. If you want really fast copies of large files, then you want to send down multiple overlapped aio ( real aio, not the glibc threaded implementation ) O_DIRECT reads and writes, but that gets quite complicated. Simply using blocking O_DIRECT reads into a memory mapped destination file buffer performs nearly as well, provided you use a decent block size. On my system I have found that 128 KB+ buffers are needed to keep the pipeline full because I'm using a 2 disk raid0 with a 64k stripe factor. As a result, blocks smaller than 128 KB only keep one disk going at a time. That's probably getting a bit too complicated though for this conversation. If we are talking about the conventional blocking cached read, followed by a blocking cached write, then I think you will find that using a buffer size of several pages ( say 32 or 64 KB ) will be MUCH more efficient than 1024 bytes ( the typical local filesystem block size ), so using st_blksize for the size of the read/write buffer is not good. I think you may be ascribing meaning to st_blksize that is not there. Robert Latham wrote: In local file systems, i'm sure you are correct. If you are working with a remote file system, however, the optimal size is on the order of megabytes, not kilobytes. For a specific example, consider the PVFS2 file system, where the plateau in "blocksize vs. bandwitdh" is two orders of magnitude larger than 64 KB. PVFS2 is a parallel file system for linux clusters. I am not nearly as familiar with Lustre, GPFS, or GFS, but I suspect those filesystems too would benefit from block sizes larger than 64 KB. Are you taking umbrage at the idea of using st_blksize to direct how large the transfer size should be for I/O? I don't know what other purpose st_blksize should have, nor are there any other fields which are remotely valid for that purpose. Thanks for your feedback. ==rob ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: bug in sort with '.' in strings
> > Hi, > > Sort seems to ignore (incorrectly handle) the character '.' in the > strings. The following output is what I get with: > > $ sort --version > sort (coreutils) 4.5.3 > Written by Mike Haertel and Paul Eggert. This version is quite old. The current stable version is 5.93, see the announcement at http://lists.gnu.org/archive/html/bug-coreutils/2005-11/msg00058.html > > I would expect all of the 'qbits.*' to be either before or after > all of the 'qbits2.*' in the above output. Thanks for the report. However, it is most likely that this is not a bug in sort, but a misunderstanding on your part. Read up on this FAQ: http://www.gnu.org/software/coreutils/faq/coreutils-faq.html#Sort-does-not-sort-in-normal-order_0021 Then inspect the output of the locale command, and try rerunning sort with LC_ALL=C in your environment. -- Eric Blake ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: bug in sort with '.' in strings
On Mon, 21 Nov 2005, Michael J. Dinneen wrote: Sort seems to ignore (incorrectly handle) the character '.' in the strings. Please see this FAQ entry, "Sort does not sort in normal order!": http://www.gnu.org/software/coreutils/faq/coreutils-faq.html#Sort-does-not-sort-in-normal-order_0021 Cheers, Phil ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
bug in sort with '.' in strings
Hi, Sort seems to ignore (incorrectly handle) the character '.' in the strings. The following output is what I get with: $ sort --version sort (coreutils) 4.5.3 Written by Mike Haertel and Paul Eggert. Copyright (C) 2002 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. qbits.18.out qbits.19.out qbits.20.out qbits2.10.out qbits2.11.out qbits2.12.out qbits2.13.out qbits2.14.out qbits2.15.out qbits2.16.out qbits2.17.out qbits2.18.out qbits2.19.out qbits.21.out <--- look here qbits2.20.out qbits2.21.out qbits2.22.out qbits2.23.out qbits2.24.out qbits.22.out <--- and here qbits.23.out I would expect all of the 'qbits.*' to be either before or after all of the 'qbits2.*' in the above output. Michael ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: better buffer size for copy
On Sat, Nov 19, 2005 at 10:49:07AM -0500, Phillip Susi wrote: > I don't see why the filesystem's cluster size should have a thing to do > with the buffer size used to copy files. For optimal performance, the > larger the buffer, the better. Diminishing returns applies of course, > so at some point the increase in buffer size results in little to no > further increase in performance, so that's the size you should use. I > believe that the optimal size is about 64 KB. In local file systems, i'm sure you are correct. If you are working with a remote file system, however, the optimal size is on the order of megabytes, not kilobytes. For a specific example, consider the PVFS2 file system, where the plateau in "blocksize vs. bandwitdh" is two orders of magnitude larger than 64 KB. PVFS2 is a parallel file system for linux clusters. I am not nearly as familiar with Lustre, GPFS, or GFS, but I suspect those filesystems too would benefit from block sizes larger than 64 KB. Are you taking umbrage at the idea of using st_blksize to direct how large the transfer size should be for I/O? I don't know what other purpose st_blksize should have, nor are there any other fields which are remotely valid for that purpose. Thanks for your feedback. ==rob -- Rob Latham Mathematics and Computer Science DivisionA215 0178 EA2D B059 8CDF Argonne National Labs, IL USAB29D F333 664A 4280 315B ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Segmentation fault in su
Ahamed.MLA wrote: > If i login as an ordinary user(admin), and try to switch to ROOT > user, iam getting Segmentating fault, though iam member of wheel > group and given trust in /etc/pam.d/su A segmentation fault indicates access to memory that is out of bounds. Requiring being in the wheel group or not is a local configuration modification to the system. The two would not seem to be related. Since 'su' works in the default configuration I suggest returning the system to a known working state and then making small modifications one at a time. If the problem returns then you know it is the last modification that was made. > I am using Redhat Linux- 7.2 in my servers. That is by today's standards quite old. The code in shellutils has been combined with fileutils and textutils into a single coreutils package. You might consider upgrading. ftp://ftp.gnu.org/gnu/coreutils/coreutils-5.93.tar.gz (7.3MB) ftp://ftp.gnu.org/gnu/coreutils/coreutils-5.93.tar.bz2 (4.7MB) You might want to look into the Fedora Legacy project. http://www.fedoralegacy.org/ > Kindly provide me a solution, as soon as possible. This is a volunteer effort and we are unable to provide solutions on demand. Perhaps contracting with one of the many commercial vendors to provide paid support would be a good model for you? However all of the source code is available making it is possible that you could debug the problem yourself. Bob ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Bug#339136: Changes in stat package output break apt-move
Hello Jim and others, wouldn't it be better to implement the full backslashed sequences inside print_esc() than only `\n'? CU Tom. (Thomas M.Ott) Germany ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Segmentation fault in su
Hi, If i login as an ordinary user(admin), and try to switch to ROOT user, iam getting Segmentating fault, though iam member of wheel group and given trust in /etc/pam.d/su %PAM-1.0 ### /etc/pam.d/su --> auth required /lib/security/pam_wheel.so group=wheel # /etc/group --> wheel:x:10:root,admin I am using Redhat Linux- 7.2 in my servers. Kindly provide me a solution, as soon as possible. Regards Ahamed.MLA Beacon Operations 24/7 Sify Ltd. Chennai ** DISCLAIMER ** Information contained and transmitted by this E-MAIL is proprietary to Sify Limited and is intended for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If this is a forwarded message, the content of this E-MAIL may not have been sent with the authority of the Company. If you are not the intended recipient, an agent of the intended recipient or a person responsible for delivering the information to the named recipient, you are notified that any use, distribution, transmission, printing, copying or dissemination of this information in any way or in any manner is strictly prohibited. If you have received this communication in error, please delete this mail & notify us immediately at [EMAIL PROTECTED] www.sify.com - your homepage on the internet for news, sports, finance, astrology, movies, entertainment, food, languages etc ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
