The mkfs.jfs would properly create things right near the 32 TiB border
but I then found out that fsck would crap out even though mkfs.jfs would
properly create the file-system (atleast write more than just the log
and actually write data to the device):
sabayonx86-64 / # mkfs.jfs -s 128 -L data /dev/sdd1
mkfs.jfs version 1.1.14, 06-Apr-2009
Warning! All data on device /dev/sdd1 will be lost!
Continue? (Y/N) y
|
Format completed successfully.
34328444232 kilobytes total disk space.
sabayonx86-64 / # fsck.jfs -v -f /dev/sdd1
fsck.jfs version 1.1.14, 06-Apr-2009
processing started: 6/3/2010 3.4.24
The current device is: /dev/sdd1
Open(...READ/WRITE EXCLUSIVE...) returned rc = 0
Primary superblock is valid.
The type of file system for the device is JFS.
Block size in bytes: 4096
Filesystem size in blocks: 8582111058
**Phase 0 - Replay Journal Log
LOGREDO: Log already redone!
logredo returned rc = 0
**Phase 1 - Check Blocks, Files/Directories, and Directory Entries
**Phase 2 - Count links
**Phase 3 - Duplicate Block Rescan and Directory Connectedness
**Phase 4 - Report Problems
**Phase 5 - Check Connectivity
**Phase 6 - Perform Approved Corrections
**Phase 7 - Rebuild File/Directory Allocation Maps
**Phase 8 - Rebuild Disk Allocation Maps
ujfs_rw_diskblocks: disk_count is 0
Unrecoverable error writing M to /dev/sdd1. CANNOT CONTINUE.
Fatal error (-10091,87) accessing the filesystem (2,221184,0,0).
**** Filesystem was modified. ****
processing terminated: 6/3/2010 3:04:25 with return code: -10091 exit
code: 4.
I ended up having to make the file-system 99.9% of 32 TiB which meant I
cut off 32 GiB off the file-system and then the fsck would correctly
continue:
sabayonx86-64 / # mkfs.jfs -s 128 -L data /dev/sdd1
mkfs.jfs version 1.1.14, 06-Apr-2009
Warning! All data on device /dev/sdd1 will be lost!
Continue? (Y/N) y
|
Format completed successfully.
34325008295 kilobytes total disk space.
sabayonx86-64 / # fsck.jfs -v -f /dev/sdd1
fsck.jfs version 1.1.14, 06-Apr-2009
processing started: 6/3/2010 3.5.20
The current device is: /dev/sdd1
Open(...READ/WRITE EXCLUSIVE...) returned rc = 0
Primary superblock is valid.
The type of file system for the device is JFS.
Block size in bytes: 4096
Filesystem size in blocks: 8581252073
**Phase 0 - Replay Journal Log
LOGREDO: Log already redone!
logredo returned rc = 0
**Phase 1 - Check Blocks, Files/Directories, and Directory Entries
**Phase 2 - Count links
**Phase 3 - Duplicate Block Rescan and Directory Connectedness
**Phase 4 - Report Problems
**Phase 5 - Check Connectivity
**Phase 6 - Perform Approved Corrections
**Phase 7 - Rebuild File/Directory Allocation Maps
**Phase 8 - Rebuild Disk Allocation Maps
Filesystem Summary:
Blocks in use for inodes: 8
Inode count: 64
File count: 0
Directory count: 1
Block count: 8581252073
Free block count: 8579908839
34325008292 kilobytes total disk space.
0 kilobytes in 1 directories.
0 kilobytes in 0 user files.
0 kilobytes in extended attributes
0 kilobytes in access control lists
5372936 kilobytes reserved for system use.
34319635356 kilobytes are available for use.
Filesystem is clean.
All observed inconsistencies have been repaired.
Filesystem has been marked clean.
**** Filesystem was modified. ****
processing terminated: 6/3/2010 3:05:57 with return code: 0 exit code: 0.
So
34325008295 kilobytes
is the largest I could make the file-system and fsck without any errors.
I guess I will try to completely fill the file-system up to make sure
that it doesn't give me any new errors when it is near full but can I
assume it will probably work ok as the mkfs.jfs and fsck is running ok?
Sandon Van Ness wrote:
> I am planning on having a file-system that is over 32TB in the very near
> future and I heard from someone that JFS can't handle file-systems over
> 32 TB correctly (can't expand, fsck, or anything). I believe this is
> indeed true as I just tried formatting a file-system that was 31 TB via
> a sparse file and it took a bit of time to format and ended up with a
> file with about 5 GB of usage:
>
> r...@sabayonx86-64: 05:31 PM :/data# mkfs.jfs -s 1024 /data/sparse_jfs
> mkfs.jfs version 1.1.14, 06-Apr-2009
> Warning! All data on device /data/sparse_jfs will be lost!
>
> Continue? (Y/N) y
> -
>
> Format completed successfully.
>
> 33285996544 kilobytes total disk space.
>
>
> r...@sabayonx86-64: 05:32 PM :~# ls -lsah /data/sparse_jfs
> 4.9G -rw-r--r-- 1 root root 31T 2010-04-20 17:32 /data/sparse_jfs
>
> However, when I made this 2 TB bigger (to 33 TB) the format went much
> quicker and the end result was a sparse file with only 1.1 GB used:
>
> r...@sabayonx86-64: 05:32 PM :/data# mkfs.jfs -s 1024 /data/sparse_jfs
> mkfs.jfs version 1.1.14, 06-Apr-2009
> Warning! All data on device /data/sparse_jfs will be lost!
>
> Continue? (Y/N) y
> -
>
> Format completed successfully.
>
> 35433480192 kilobytes total disk space.
> r...@sabayonx86-64: 05:32 PM :/data# ls -lsah /data/sparse_jfs
> 1.1G -rw-r--r-- 1 root root 33T 2010-04-20 17:33 /data/sparse_jfs
>
>
> So obviously there is some bug here as if anything it should be writing
> more data. Also the used space amount after mounting it is also about
> the size I would expect from the sparse file minus the log size:
>
> Again used size just about doubles with a 64TB file-system but the
> sparse file is still much smaller than with a 31 TB file-system:
>
> r...@sabayonx86-64: 05:37 PM :/data# dd if=/dev/zero of=sparse_jfs bs=1M
> count=0 seek=64M
> 0+0 records in
> 0+0 records out
> 0 bytes (0 B) copied, 1.1877e-05 s, 0.0 kB/s
> r...@sabayonx86-64: 05:37 PM :/data# mkfs.jfs -s 1024 /data/sparse_jfs
> mkfs.jfs version 1.1.14, 06-Apr-2009
> Warning! All data on device /data/sparse_jfs will be lost!
>
> Continue? (Y/N) y
> -
>
> Format completed successfully.
>
> 68719476736 kilobytes total disk space.
> r...@sabayonx86-64: 05:38 PM :/data# mount -o loop -t jfs
> /data/sparse_jfs /mnt/sparse/
> r...@sabayonx86-64: 05:38 PM :/data# df -h /mnt/sparse/
> Filesystem Size Used Avail Use% Mounted on
> /data/sparse_jfs 65T 8.1G 64T 1% /mnt/sparse
>
> However trying to copy anything to it immediately errors out:
>
> r...@sabayonx86-64: 05:39 PM :/mnt/sparse# cp -avf /opt ./
> cp: cannot create directory `./opt': Input/output error
> r...@sabayonx86-64: 05:39 PM :/mnt/sparse#
>
> ERROR: (device loop1): dbAllocNext: Corrupt dmap page
> ERROR: (device loop1): remounting filesystem as read-only
>
> ialloc: diAlloc returned -5!
>
>
> With the 32 TB sparse file it works as expected and the cp runs without
> any problem:
>
> r...@sabayonx86-64: 05:40 PM :/data# dd if=/dev/zero of=sparse_jfs bs=1M
> count=0 seek=31M
> 0+0 records in
> 0+0 records out
> 0 bytes (0 B) copied, 1.2172e-05 s, 0.0 kB/s
> r...@sabayonx86-64: 05:40 PM :/data# mkfs.jfs -s 1024 /data/sparse_jfs
> mkfs.jfs version 1.1.14, 06-Apr-2009
> Warning! All data on device /data/sparse_jfs will be lost!
>
> Continue? (Y/N) y
> -
>
> Format completed successfully.
>
> 33285996544 kilobytes total disk space.
> r...@sabayonx86-64: 05:41 PM :/data# ls -lsah /data/sparse_jfs
> 4.9G -rw-r--r-- 1 root root 31T 2010-04-20 17:41 /data/sparse_jfs
>
> r...@sabayonx86-64: 05:42 PM :/mnt/sparse# df -h /mnt/sparse/
> Filesystem Size Used Avail Use% Mounted on
> /data/sparse_jfs 31T 3.9G 31T 1% /mnt/sparse
> r...@sabayonx86-64: 05:42 PM :/mnt/sparse# cp -avf /opt/ ./ | wc -l
> 17426
> r...@sabayonx86-64: 05:43 PM :/mnt/sparse# df -h /mnt/sparse/
> Filesystem Size Used Avail Use% Mounted on
> /data/sparse_jfs 31T 5.5G 31T 1% /mnt/sparse
> r...@sabayonx86-64: 05:43 PM :/mnt/sparse# ls -lsah /data/sparse_jfs
> 6.0G -rw-r--r-- 1 root root 31T 2010-04-20 17:41 /data/sparse_jfs
> r...@sabayonx86-64: 05:43 PM :/mnt/sparse#
>
> Are there any plans to fix this? Ive got a feeling that its something
> simple as only the log seems to be written when doing the mkfs.
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Jfs-discussion mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/jfs-discussion
>
>
------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit. See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
Jfs-discussion mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/jfs-discussion
