Hello,
I noticed an error in cp (8.11 upwards till 8.19) when copying many files
from an fragmented ext3fs (to an empty partition).
ERROR:
The copied file has exact length.
The SourceFile has in the middle a 4K-block filled with ZERO, the
DestFile this 4k-block and the previous 4k-block also filled with ZERO.
I can reproduce this:
Conditions:
*I have a strong fragmented 110GB ext3-partition with >300
Files
*I have to copy about 100 files (ca. 15GB) to another
HDD-Partition or ramdisk
*cp must be 8.11 or higher (compiled under SLES11SP2
kernel 3.0) (everything is ok with 6.11. 8.05..8.10)
*Command:
cp819 $FILES $DEST #here the corruption comes
*$FILES is a list of about 100 DatabaseFiles. The
last one is the file that becomes corrupted
I can _NOT_ reproduce this:
*with cp <8.11
*Command:
for F in $FILES; do cp -dp $SRC$F $DEST$F ;done
#no corruption
Reproduce:
*tested with SLES11SP1, SLES11SP2, Ubuntu 12.04
(cp less 8.11 is good, cp 8.11 upwards corrupts the file)
*when I bulk copy the about 100 files, the corruption always
happen with the same file and same position
*if I copy less files or the single file there is no corruption
*if I copy all files there are more corruptions
Additional info:
# ./fragments /mnt/Source/
Summary statistics for /mnt/Source/ including subdirectories:
blocks files fragmented percent fragm./file
---------------------------------------------------
1 1 0 0.00 1.00
<= 12 0 0 0.00 0.00
<= 524 0 0 0.00 0.00
<= 1036 0 0 0.00 0.00
<= 4108 11 11 100.00 54.00
> 4108 307 307 100.00 159.71
---------------------------------------------------
all 319 318 99.69 155.57
> 1 318 318 100.00 156.05
Fragmentation index: 0.428 percent.
frag: 49625 n: 318 n_blocks: 11513587 (n_frag: 318)
#cmp Julius.nsf /mnt/ramdisk/Julius.nsf
Julius.nsf /mnt/ramdisk/Julius.nsf differ: char 18395137, line 345000
18395137=4491*4K + 1
=======================================================================================
#dd if=JuliusGOOD.nsf bs=4096 skip=4491 |hexdump -C |more
00000000 16 00 00 ff 00 10 00 00 00 00 00 00 00 00 02 00
|................|
00000010 00 00 00 00 00 00 00 00 10 0f 00 00 00 00 00 00
|................|
00000020 af 2f 06 00 1d 00 16 0a 00 00 00 00 40 09 88 00
|./..........@...|
....
00000a30 00 00 00 6e 64 65 00 00 00 00 00 00 00 00 00 00
|...nde..........|
00000a40 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
*
00000c00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000c10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
*
00000e00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000e10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
*
00002000 18 00 00 ff 00 10 00 00 00 00 00 00 00 00 02 00
|................|
---------------
#dd if=/mnt/ramdisk/JuliusBAD.nsf bs=4096 skip=4491 |hexdump -C |more
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
*
00002000 18 00 00 ff 00 10 00 00 00 00 00 00 00 00 02 00
|................|
=======================================================================================
# filefrag JuliusGood.nsf -v
Filesystem type is: ef53
Filesystem cylinder groups is approximately 881
File size of JuliusGood.nsf is 175935488 (42953 blocks, blocksize 4096)
ext logical physical expected length flags
0 0 2008262 2 merged
1 2 2008281 2008263 7 merged
2 9 2008303 2008287 1 merged
3 10 2008307 2008303 2 merged
4 12 2008310 2008308 2 merged
5 14 2008319 2008311 1 merged
6 15 2008321 2008319 4 merged
7 19 2031582 2008324 2 merged
8 21 2033663 2031583 41 merged
9 62 2033706 2033703 1 merged
10 63 2033704 2033706 2 merged
11 65 2033707 2033705 22 merged
12 87 2033736 2033728 23 merged
13 110 7961027 2033758 1 merged
14 111 7961587 7961027 1 merged
15 112 7943051 7961587 2 merged
16 114 7932925 7943052 1 merged
17 115 7939414 7932925 1 merged
18 116 2053842 7939414 2 merged
19 119 7961063 2053843 1 merged
20 120 7961059 7961063 2 merged
21 122 2033759 7961060 160 merged
22 282 7943053 2033918 1 merged
23 283 7961035 7943053 1 merged
24 284 7961030 7961035 1 merged
25 285 2033919 7961030 108 merged
26 393 7961054 2034026 1 merged
27 394 7961039 7961054 1 merged
28 395 2034027 7961039 104 merged
29 499 2034143 2034130 3 merged
30 502 2034159 2034145 8 merged
31 510 7961047 2034166 1 merged
32 511 7961033 7961047 1 merged
33 512 2034167 7961033 494 merged
34 1006 2034666 2034660 1 merged
35 1007 2034670 2034666 29 merged
36 1036 2034701 2034698 478 merged
37 1514 2035182 2035178 510 merged
38 2024 2035697 2035691 24 merged
39 2048 2038674 2035720 1 merged
40 2049 2035721 2038674 11 merged
41 2060 2035733 2035731 385 merged
42 2445 2036120 2036117 2 merged
43 2447 2036128 2036121 1 merged
44 2448 2036136 2036128 1 merged
45 2449 2036518 2036136 1 merged
46 2450 2037030 2036518 634 merged
47 3084 2037665 2037663 117 merged
48 3201 2037784 2037781 886 merged
49 4087 7961058 2038669 1 merged
50 4088 7961049 7961058 1 merged
51 4089 2038670 7961049 4 merged
52 4093 2040616 2038673 1 merged
53 4094 2038675 2040616 14 merged
54 4108 2038690 2038688 291 merged
55 4399 7961036 2038980 1 merged
56 4400 7961050 7961036 1 merged
57 4401 2038981 7961050 10 merged
58 4411 2038993 2038990 9 merged
59 4431 2038991 2039001 2 merged
60 4433 7961064 2038992 9 merged
61 4453 2039002 7961072 10 merged
62 4463 2053899 2039011 1 merged
63 4464 7932921 2053899 1 merged
64 4465 7936120 7932921 1 merged
65 4466 2039012 7936120 18 merged
66 4484 7943055 2039029 1 merged
67 4485 7961056 7943055 1 merged
68 4486 7961041 7961056 1 merged
69 4487 7961045 7961041 2 merged
70 4489 7961053 7961046 1 merged
71 4490 7961031 7961053 1 merged
72 4491 7961034 7961031 1 merged
73 4493 7961044 7961034 1 merged
74 4494 2039030 7961044 285 merged
75 4779 2054871 2039314 1 merged
76 4780 2053844 2054871 1 merged
....
Copied corrupted File:
# filefrag /mnt/DEST/JuliusBad.nsf -v
Filesystem type is: ef53
Filesystem cylinder groups is approximately 881
File size of /mnt/DEST/gschouten.nsf is 175935488 (42953 blocks, blocksize
4096)
ext logical physical expected length flags
0 0 3146242 6 merged
1 6 3155036 3146247 6 merged
2 12 3155043 3155041 106 merged
3 119 3155149 917 merged
4 1036 3156068 3156065 1024 merged
5 2060 3157093 3157091 1024 merged
6 3084 3158118 3158116 1024 merged
7 4108 3159143 3159141 312 merged
8 4431 3159455 11 merged
9 4453 3159466 38 merged
10 4493 3159504 639 merged
11 5132 3160144 3160142 417 merged
12 5550 3160561 441 merged
......
###This is the fragmented,good SourceFile:
debugfs 1.41.9 (22-Aug-2009)
debugfs: stat JuliusGood.nsf
Inode: 102 Type: regular Mode: 0644 Flags: 0x0
Generation: 1499014015 Version: 0x00000000
User: 1001 Group: 1001 Size: 175935488
File ACL: 0 Directory ACL: 0
Links: 1 Blockcount: 343576
Fragment: Address: 0 Number: 0 Size: 0
ctime: 0x505e346a -- Sat Sep 22 23:58:02 2012
atime: 0x506041bf -- Mon Sep 24 13:19:27 2012
mtime: 0x505e346a -- Sat Sep 22 23:58:02 2012
Size of extra inode fields: 4
BLOCKS:
(0-1):2008262-2008263, (2-8):2008281-2008287, (9):2008303,
(10-11):2008307-2008308, (IND):2008309, (12-13):2008310-2008311,
(14):2008319, (15-18):2008321-2008324, (19-20):2031582
-2031583, (21-61):2033663-2033703, (62):2033706, (63-64):2033704-2033705,
(65-86):2033707-2033728, (87-109):2033736-2033758, (110):7961027,
(111):7961587, (112-113):7943051-79430
52, (114):7932925, (115):7939414, (116-117):2053842-2053843,
(119):7961063, (120-121):7961059-7961060, (122-281):2033759-2033918,
(282):7943053, (283):7961035, (284):7961030, (28
5-392):2033919-2034026, (393):7961054, (394):7961039,
(395-498):2034027-2034130, (499-501):2034143-2034145,
(502-509):2034159-2034166, (510):7961047, (511):7961033, (512-1005):20
34167-2034660, (1006):2034666, (1007-1035):2034670-2034698,
(DIND):2034699, (IND):2034700, (1036-1513):2034701-2035178,
(1514-2023):2035182-2035691, (2024-2047):2035697-2035720,
(2048):2038674, (2049-2059):2035721-2035731, (IND):2035732,
(2060-2444):2035733-2036117, (2445-2446):2036120-2036121, (2447):2036128,
(2448):2036136, (2449):2036518, (2450-3083):
2037030-2037663, (IND):2037664, (3084-3200):2037665-2037781,
(3201-4086):2037784-2038669, (4087):7961058, (4088):7961049,
(4089-4092):2038670-2038673, (4093):2040616, (4094-4107)
:2038675-2038688, (IND):2038689, (4108-4398):2038690-2038980,
(4399):7961036, (4400):7961050, (4401-4410):2038981-2038990,
(4411-4419):2038993-2039001, (4431-4432):2038991-203899
2, (4433-4441):7961064-7961072, (4453-4462):2039002-2039011,
(4463):2053899, (4464):7932921, (4465):7936120,
(4466-4483):2039012-2039029, (4484):7943055, (4485):7961056, (4486):7
961041, (4487-4488):7961045-7961046, (4489):7961053, (4490):7961031,
(4491):7961034, (4493):7961044, (4494-4778):2039030-2039314,
(4779):2054871, (4780):2053844, (4781-4800):2039
###Here comes the corrupted File:
debugfs 1.41.9 (22-Aug-2009)
debugfs: stat JuliusBad.nsf
Inode: 114 Type: regular Mode: 0644 Flags: 0x0
Generation: 3620008692 Version: 0x00000000
User: 0 Group: 0 Size: 175935488
File ACL: 0 Directory ACL: 0
Links: 1 Blockcount: 343568
Fragment: Address: 0 Number: 0 Size: 0
ctime: 0x507831f2 -- Fri Oct 12 17:06:26 2012
atime: 0x507831f0 -- Fri Oct 12 17:06:24 2012
mtime: 0x507831f2 -- Fri Oct 12 17:06:26 2012
Size of extra inode fields: 4
BLOCKS:
(0-5):3146242-3146247, (6-11):3155036-3155041, (IND):3155042,
(12-117):3155043-3155148, (119-1035):3155149-3156065, (DIND):3156066,
(IND):3156067, (1036-2059):3156068-3157091, (IND):315709
2, (2060-3083):3157093-3158116, (IND):3158117,
(3084-4107):3158118-3159141, (IND):3159142, (4108-4419):3159143-3159454,
(4431-4441):3159455-3159465, (4453-4490):3159466-3159503, (4493-5131
):3159504-3160142,
I am 95% sure that in previous Tests if have also seen a corrupted file
with several 100k filled with ZERO.
Surely it would be most simple to reproduce the error if I could provide
the whole partition as is.
But since the Files are Database files I can not provide it. Are there any
tests that I can do?
Best regards and thanks in advance for your help,
Mike
Mit freundlichen Grüßen
Mike Gerth
Dipl. Inf. Mike Gerth
+49 30 399 76 136
IT-Systemadministrator
AVM Audiovisuelles Marketing und Computersysteme GmbH
Alt-Moabit 95, 10559 Berlin
HRB 23075 AG Charlottenburg
Geschäftsführer: Johannes Nill
