On Thu, Dec 22, 2005 at 02:46:51PM -0500, Alain Magloire wrote: > > > > Hello, > > > > > First of all I'd like to announce that our custom mail.local based on > > > GNU Mailutils is operational on our production mail servers. Our mail > > > servers have about 200K users and we deliver ~1-1.5 million messages > > > to mailboxes daily. > > > > Great. > > > > Yes. > > > > I compiled both my mda and GNU Mailutils (snapshot > > > 20051102) with -D_FILE_OFFSET_BITS=64. However this by itself was not > > > enough. I investigated the issue and the problem was the use of fseek(3) > > > instead of feeko(3), so I substituted all occurences of fseek() with > > > feeko() in Mailutils sources (found in > > > ./mailbox/file_stream.c,./lib/getpass.c, > > > ./mimeview/mimetypes-gram.c,./mimeview/mimetypes.y). This was enough > > > to fix the issues observed and have large file support. > > [..] > > > So I ask for these changes to be included upstream. Of course I am not > > > sure if this is enough to cover all possible issues involved. > > > > Of course we are aiming to supporting large files in mailutils. However, > > simply changing fseek to fseeko and ftell to ftello is not enough in > > general. There are more problems involved. Most probably these do not > > affect your installation, however they should be solved before releasing > > the package: > >
Hmm, now that I think of it, I have not changed occurences of ftell() to ftello() as well. But in my setup, I see no errors. > > Glad to see, it worked for you, this give us a good hint that we did not > rely too much on platform dependent behavior, thanks! > > > 1. Using fseeko creates portability problems. It can, however, be solved > > using appropriate autoconf magic. > > > > Agreed, we need to pass the right flags to the compiler, is the OS support > this. Question is not fseeko() and friends part of Posix now? Or whatever > XOpen standard? > To the UNIX systems I could check (GNU/Linux, [Free|Net|Open]BSD, Solaris) fseeko() exists in the standard C library and works the same way. The NetBSD and OpenBSD man pages state that "The fseeko() and ftello() functions conform to X/Open System Interfaces and Headers Issue 5 (``XSH5'')." while the FreeBSD page states "The fseeko() and ftello() functions conform to IEEE Std 1003.1-2001 (``POSIX.1'')." > > > 2. The major problem is the use of off_t in the libmailutils. This data > > type has different sizes depending on whether the library was > > compiled with large file support or not. Thus, if one compiles > > libmailutils with large file support and then links it to the program > > that is compiled without that support, any mailutils functions taking > > off_t as an argument will get incorrect data, which will lead, *in > > the best case*, to a coredump. In the worst case it will trash whatever > > user files involved, as well. > > I learned this the hard way, though I believe the problem is when an off_t is returned from a mailutils function to the program (64-bit value to fit in a 32-bit one). Anyway, both the program and the library should be compiled with the same D_FILE_OFFSET_BITS settings. > > Compiling two variants of libmailutils (with and without large file > > support) is an awkward solution. Therefore I have introduced > > (2005-11-15) a new data type `mu_off_t' which is equivalent to 64-bit > > off_t and have changed all related functions and datatypes > > accordingly. This makes libmailutils safe to use no matter in what > > combination it is linked, although this creates a small overhead in > > the usual 32-bit mode. > > > > It sounds good to me. So basically if I read you right, you proposing to > always use 64 bits i.e. mu_off_t ==> 64 bits. > That means you promote a 32 bit off_t to a 64 bit mu_off_t each time and you use everywhere in mailutils code an mu_off_t, right? This is an elegant cross-platform solution. > > > 3. Most of the libmailutils code formats off_t for printing > > using "%lu" format directive, which does not work for 64-bit data > > types. Using "%llu" is impossible due to portability problems. The > > recent snapshot (mailutils-20051216.tar.gz) solves this problem as > > well. However, these changes are not committed yet, because I do not > > quite like the way I implemented them. I am searching for a more > > elegant solution. > > > > Then the question is: where do we actually need to print/format mu_off_t? > On the top of my head, I do not remember any occurrences, so it may be a > non-issue. And in the event that we need to print/format is it safe to > downcast it to an off_t? The other possibility is simply to always use our > auxiliary functions like asnprintf() that supports the format and to pass > the buffer down to printf() and all for printing. > > I am glad you already address the issues for LFS support. I would really like to have a buildable snapshot when the changes are commited to the repository. Thanks again for everything. Kostas -- Kostas Zorbadelos [EMAIL PROTECTED] contact: kzorba (at) otenet.gr Out there in the darkness, out there in the night out there in the starlight, one soul burns brighter than a thousand suns. _______________________________________________ Bug-mailutils mailing list [email protected] http://lists.gnu.org/mailman/listinfo/bug-mailutils
