Re: Bug#339136: Changes in stat package output break apt-move
ThMO [EMAIL PROTECTED] wrote: Hello Jim and others, wouldn't it be better to implement the full backslashed sequences inside print_esc() than only `\n'? Consistency between tools is nice, but do we need support for \t, \v, \a, \b, \c, \f, etc? What about octal escapes? Hexadecimal and unicode, too, like printf(1)? I like the keep-it-simple approach and am leaning toward supporting the backslash-escape sequences that are supported for find's -printf format. In other words, `Yes, depending on what you mean by ``full''.' ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: better buffer size for copy
On Mon, Nov 21, 2005 at 12:45:40AM -0500, Phillip Susi wrote: If we are talking about the conventional blocking cached read, followed by a blocking cached write, then I think you will find that using a buffer size of several pages ( say 32 or 64 KB ) will be MUCH more efficient than 1024 bytes ( the typical local filesystem block size ), so using st_blksize for the size of the read/write buffer is not good. I think you may be ascribing meaning to st_blksize that is not there. I mean no offense cutting out most of your points. You describe great ways to achieve high I/O rates for anyone writing a custom file mover. I shouldn't have mentioned network file systems. It's a distraction from the real point of my patch: cp(1) should consider both the source and the destination st_blksize. All I expect from st_blksize is what the stat(2) manpage suggests: The value st_blocks gives the size of the file in 512-byte blocks. (This may be smaller than st_size/512 e.g. when the file has holes.) The value st_blksize gives the preferred blocksize for efficient file system I/O. (Writing to a file in smaller chunks may cause an inefficient read-modify-rewrite.) All I really want is for cp(1) to do the right thing no matter what the soruce or destination st_blksize value might be. In copying from a 4k blocksize file sytem to a 64k blocksize filesystem, cp(1) will perform well, as it is using a 64k buffer. In copying *from* that 64k blocksize filesystem *to* a 4k blocksize filesytem, cp(1) will not perform as well: it's using a 4k buffer and so reading from the source filesystem in less-than-ideal chunks. Thanks again for taking the time to respond. I hope I have made the intent of my patch more clear. ==rob -- Rob Latham Mathematics and Computer Science DivisionA215 0178 EA2D B059 8CDF Argonne National Labs, IL USAB29D F333 664A 4280 315B ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Bug in md5sum (5.3.0) when checking lists of files?
[EMAIL PROTECTED] writes: This would suggest to me that md5sum -c reads the files in the checklist from the wrong directory when directorynames look similar, perhaps because it is truncating the names or something similar? That would be bad, yes. Can you isolate the problem to a one-line sums.md5 file? Ideally we'd like a test case that would let us reproduce the problem. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: race condition in mkdir -p?
I haven't seen the problem since upgrading to 5.93. Thanks, Noel On 11/11/05, Jim Meyering [EMAIL PROTECTED] wrote: Noel Yap [EMAIL PROTECTED] wrote: Since I've switched to using coreutils-5.92, I've noticed that my builds, which only does mkdir -p now sporadically error out with: mkdir: cannot create directory `blah/blah/blah': file exists Have others seen this as well? This behaviour doesn't occur with coreutils-5.2.1. That's probably fixed in 5.93. Would you please confirm? ftp://ftp.gnu.org/gnu/coreutils/coreutils-5.93.tar.gz ftp://ftp.gnu.org/gnu/coreutils/coreutils-5.93.tar.bz2 ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: better buffer size for copy
Robert Latham wrote: I mean no offense cutting out most of your points. You describe great ways to achieve high I/O rates for anyone writing a custom file mover. I shouldn't have mentioned network file systems. It's a distraction from the real point of my patch: cp(1) should consider both the source and the destination st_blksize. No problem... I kind of went off on a tangent there. All I expect from st_blksize is what the stat(2) manpage suggests: The value st_blocks gives the size of the file in 512-byte blocks. (This may be smaller than st_size/512 e.g. when the file has holes.) The value st_blksize gives the preferred blocksize for efficient file system I/O. (Writing to a file in smaller chunks may cause an inefficient read-modify-rewrite.) All I really want is for cp(1) to do the right thing no matter what the soruce or destination st_blksize value might be. Ok, I see what you are talking about now. Using a copy block size smaller than the filesystem block size can result in a lot of extra IO, thus reducing throughput. Of course, this doesn't really apply in a typical use case because the kernel will cache the writes and combine them when it flushes the IO to disk, however, yes... it is a good idea to use an IO block size that is at least as large as the larger of the source and destination filesystem block sizes. In copying from a 4k blocksize file sytem to a 64k blocksize filesystem, cp(1) will perform well, as it is using a 64k buffer. In copying *from* that 64k blocksize filesystem *to* a 4k blocksize filesytem, cp(1) will not perform as well: it's using a 4k buffer and so reading from the source filesystem in less-than-ideal chunks. Again, this probably won't happen in real practice due to the influence of the filesystem cache, but I do see your point. In practice though, I don't know of any filesystem with a 64k block size. By default ext2/3 use 1k, and reiserfs uses 4k. These are going to be typical values for st_blksize, yet if you use a copy block size of say, 64k, I think you will find the performance to be significantly better than either 1k or 4k. I think that a good case in point is copying to/from a typical ext2/3 filesystem using a 1k block size. Using a buffer smaller than a single 4k page is going to significantly degrade performance. You certainly do not want to go smaller than the block size, but really, you should be going larger. Thanks again for taking the time to respond. I hope I have made the intent of my patch more clear. ==rob You did... and I thank you as well and hope that I have made myself more clear. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Bug#339136: Changes in stat package output break apt-move
Hello Jim and others listening, wouldn't it be better to implement the full backslashed sequences inside print_esc() than only `\n'? Consistency between tools is nice, but do we need support for \t, \v, \a, \b, \c, \f, etc? What about octal escapes? Hexadecimal and unicode, too, like printf(1)? sorry, but unicode has nothing to do with escape sequences, as it denotes a different charset, e.g. an expensive replacement for iso-8859-1. I like the keep-it-simple approach and am leaning toward supporting the backslash-escape sequences that are supported for find's -printf format. `find' v4.1 does support all C backslash-escape sequences with an additional `\c' to be compatible with some `echo' implementations. In other words, `Yes, depending on what you mean by ``full''.' By `full' I mean: · the usual C character escapes · octal escape · hexadecimal escapes IMHO the question will be, why is *only* `\n' an implementation-worthy escape sequence, while many common UNIX tools do support the C escapes already. Surely, there are again many other ways inside s shell-script, to reproduce the old behaviour, OTOH we all know how lazy we get, depending on a long-standing behaviour... ;-) As far as the `stat' command is concerned, and here specifically the --format option: with all the ability to customize a personal output format, one thing can't be done, without the possibility of specifying a simple `\t' escape: when it comes to ensure, that some information need to be formatted in a spe= cific column, since the tabulator eliminates the need for counting the necessary spaces. OTOH one could simulate the `stat' command with the equivalent `find' command, so it's surely some already done work need to be done again... It's surely not an easy decision, but my personal viewpoint is, if it's for= seeable, that further questions might arrive about extending the cut-down escape sequence(s) implemented, it's surely less work, to do it once fully supporting all escapes, but YMMV. CU Tom. (Thomas M.Ott) Germany ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: od command
Vance, Jack D. [EMAIL PROTECTED] writes: Hello - can you please tell me what the numeric value is in the following output? 000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 * 1415102464. The command used was od -Ad -tx1 run on a wiped 10.0GB hard drive with 10,005,037,056 bytes. You're running an old version of 'od', which cannot output offsets greater than 2*32 bytes; the offsets wrap around. That bug was fixed several years ago (in 2000 or 2001 -- I can't recall offhand). I cannot reproduce the bug with coreutils 5.93 od ftp://ftp.gnu.org/pub/gnu/coreutils/coreutils-5.93.tar.gz: 518-penguin $ dd if=/dev/null of=big bs=10MiB seek=1024 0+0 records in 0+0 records out 0 bytes (0 B) copied, 0.010626 seconds, 0.0 kB/s 519-penguin $ ls -l big -rw-r--r-- 1 eggert eggert 10737418240 Nov 22 12:49 big 520-penguin $ od -Ad -tx1 big 000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 * 10737418240 I suggest upgrading to the latest version of 'od'. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
remove.c patch needed to compile CVS cor3eutils with GCC 4.0.2
I didn't install this since I figured remove.c might still be
mutating, and I'm not sure what the assert was supposed to check.
Also, I assumed you wanted to use only declaration-after-statement
C99, not 'for (declaration ...' C99, since only the former is
commonly supported in pre-C99 compilers, but I thought I'd ask to make
sure.
2005-11-22 Paul Eggert [EMAIL PROTECTED]
* src/remove.c (AD_push): Remove assert that doesn't compile.
(rm): Don't assume C99 for (int i = ..., since GCC 4.0.2 rejects
it in default mode.
--- remove.c.~1.137.~ 2005-11-22 13:08:25.0 -0800
+++ remove.c2005-11-22 13:41:39.0 -0800
@@ -574,7 +574,6 @@ AD_push (int fd_cwd, Dirstack_state *ds,
/* Extend the stack. */
obstack_blank (ds-Active_dir, sizeof (struct AD_ent));
- assert (AD_stack_height (ds) == n_lengths + 1);
/* Fill in the new values. */
top = AD_stack_top (ds);
@@ -1361,8 +1360,9 @@ rm (size_t n_files, char const *const *f
enum RM_status status = RM_OK;
Dirstack_state *ds = ds_init ();
bool cwd_restore_failed = false;
+ size_t i;
- for (size_t i = 0; i n_files; i++)
+ for (i = 0; i n_files; i++)
{
if (cwd_restore_failed IS_RELATIVE_FILE_NAME (file[i]))
{
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: remove.c patch needed to compile CVS cor3eutils with GCC 4.0.2
Paul Eggert [EMAIL PROTECTED] wrote: I didn't install this since I figured remove.c might still be mutating, and I'm not sure what the assert was supposed to check. Also, I assumed you wanted to use only declaration-after-statement C99, not 'for (declaration ...' C99, since only the former is commonly supported in pre-C99 compilers, but I thought I'd ask to make sure. 2005-11-22 Paul Eggert [EMAIL PROTECTED] * src/remove.c (AD_push): Remove assert that doesn't compile. (rm): Don't assume C99 for (int i = ..., since GCC 4.0.2 rejects it in default mode. The latter part is fine. I fixed the first part 40 minutes ago by leaving the assertion there and restoring the decl/defn I mistakenly removed. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
FYI: I've rewritten rm (remove.c) to use openat
I've rewritten rm to use the openat et al functions. I.e., no more chdir or fchdir. 2005-11-22 Jim Meyering [EMAIL PROTECTED] * src/remove.c: Rewrite. Now, this module is reentrant on systems that provide openat (Solaris), and on systems like Linux+procfs where our openat emulation code is reentrant. This also fixes a few low-probability leaks and eliminates some code that could, in very unusual circumstances, cause rm() (via a callee) to exit. Currently, only Solaris provides the required functions in the kernel, but lib/openat.c now includes an efficient emulation that works for Linux (assuming PROCFS support). If your Linux kernel lacks procfs support or you're using some other system, the emulation code falls back on using the non-reentrant save_cwd/restore_cwd- based code. This is a pretty big change, and will most probably provoke a test failure or two, due to differing diagnostics on some systems. All of the tests still do pass on a variety of systems (Linux 2.6.x, Solaris 9,10, FreeBSD), but many of them depend on errno values that can easily vary (e.g., EPERM vs EACCES) and on behavior that is not well standardized. I've been using it in at the front of my $PATH for more than a week with no trouble -- but that's not saying much: in my experience, if it passes the test suite on a decent selection of systems, my personal usage patterns won't expose any problems. Here are the diffs: http://savannah.gnu.org/cgi-bin/viewcvs/coreutils/coreutils/src/remove.c.diff?r1=1.134r2=1.135 or, http://tinyurl.com/b7lty $ cvs -n diff -u -r1.134 -r1.135 src/remove.c|diffstat remove.c | 702 +-- 1 file changed, 415 insertions(+), 287 deletions(-) ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
