Hello Bernhard,
On 3/29/23 17:17, Bernhard wrote:
The limiting factor is how many inodes a filesystem allows.
This depends on the "inode size" and can be specified when formatting the
filessystem.
32-bit applications can only address 2^32-1 inodes, which is ~ 4 million.
<<<<<
2^32 is ~4 billion.
:-)
To be specific:
Citing from: https://en.m.wikipedia.org/wiki/4,294,967,295
In computing, 4,294,967,295 is the highest unsigned (that is, not negative)
32-bit integer,
which makes it the highest possible number a 32-bit system can store in memory.
Why is ~4 million a limiting factor?
The errors you see:
Can't read directory '/storage/subversion/svn/db/transactions/2337-1sx.txn':
Value too large for defined data type
can have (at least) 2 reasons:
a) The inode number of that directory is bigger than 4,294,967,295 and as such
doesn't fit into glibc's 32-bit ino_t struct,
b) The date of the directory is beyond year ~2038 and doesn't fit into glibc's
32-bit time_t struct.
glibc tests in various functions if a value is bigger than what can be handled
by it's 32-bit variable.
see e.g. sysdeps/unix/sysv/linux/getdents64.c:
ssize_t
__old_getdents64 (int fd, char *buf, size_t nbytes)
{
...
/* Copy out the fixed-size data. */
__ino_t ino = source->d_ino;
__off64_t offset = source->d_off;
unsigned int reclen = source->d_reclen;
unsigned char type = source->d_type;
/* Check for ino_t overflow. */
if (__glibc_unlikely (ino != source->d_ino))
return handle_overflow (fd, previous_offset, p - buf);
Here ino (ino_t) is 32-bit while source->d_ino is a 64-bit variable.
If it doesn't fit your application will receive EOVERFLOW error for the
function you called.
It's done similiar for the time_t type.
Another option is to use xfs filesystem, which tries to work around that
problem....
<<<<<
I use the XFS file system at the 4TB hard drive, which is formatted with the
32Bit OS.
This is the output for the used XFS filesystem with size ~4TB:
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sda1 390701632 9608 390692024 1% /storage
I mixed up inodes and blocks in my last mail.
https://adil.medium.com/ext4-filesystem-data-blocks-super-blocks-inode-structure-1afb95c8e4ab
What I wanted to express is, that the inode number can be
bigger than 4,294,967,295, in which case the application will receive an
overflow error code.
This can happen more easily on bigger drives with many files.
Filesystems are different, I think XFS has some workarounds to cope better
with 32-bit apps than ext3/ext4.
Helge
