Re: O_EXCL and large files
On Wed, 23 Feb 2005, Hrvoje Niksic wrote: > I see. Now I know why I didn't use AC_FUNC_FSEEKO -- Wget doesn't use > fseeko anywhere. The only usage we have is: > > fseek (fp, 0, SEEK_END); > size = ftell (fp); /* replaced with ftello where available */ > > Is there need to use fseeko here? Probably -- the Single Unix Specification, Version 2, which introduced the "o" functions, quotes this error code: [EOVERFLOW] For fseek(), the resulting file offset would be a value which cannot be represented correctly in an object of type long. I don't know if it happens in practice (GNU libc seems to be forgiving, as usual). But we do want to use ftello(), so we need AC_FUNC_FSEEKO anyway as the _LARGEFILE_SOURCE restriction applies to both. This is what autoconf manual says (yes, a reference to ftello() should really be included in the AC_FUNC_FSEEKO description): The LFS introduced the `fseeko' and `ftello' functions to replace their C counterparts `fseek' and `ftell' that do not use `off_t'. Take care to use `AC_FUNC_FSEEKO' to make their prototypes available when using them and large-file support is enabled. And if using ftello(), then we should probably use fseeko() as well, for consistency if nothing else. Maciej
Re: O_EXCL and large files
"Maciej W. Rozycki" <[EMAIL PROTECTED]> writes: >> I wonder what is the difference between AC_FUNC_FSEEKO and >> AC_CHECK_FUNCS(seeko). The manual doesn't seem to explain. > > Well, that's what I have on my local system: > > - Macro: AC_FUNC_FSEEKO > If the `fseeko' function is available, define `HAVE_FSEEKO'. > Define `_LARGEFILE_SOURCE' if necessary to make the prototype > visible on some systems (e.g. glibc 2.2). Otherwise linkage > problems may occur when compiling with `AC_SYS_LARGEFILE' on > largefile-sensitive systems where `off_t' does not default to a > 64bit entity. > > The "_LARGEFILE_SOURCE" bit is the important one -- I have a vague > memory of some program having a problem once in this area. I see. Now I know why I didn't use AC_FUNC_FSEEKO -- Wget doesn't use fseeko anywhere. The only usage we have is: fseek (fp, 0, SEEK_END); size = ftell (fp); /* replaced with ftello where available */ Is there need to use fseeko here?
Re: O_EXCL and large files
On Wed, 23 Feb 2005, Hrvoje Niksic wrote: > > You may also want AC_FUNC_FSEEKO for fseeko(). > > I wonder what is the difference between AC_FUNC_FSEEKO and > AC_CHECK_FUNCS(seeko). The manual doesn't seem to explain. Well, that's what I have on my local system: - Macro: AC_FUNC_FSEEKO If the `fseeko' function is available, define `HAVE_FSEEKO'. Define `_LARGEFILE_SOURCE' if necessary to make the prototype visible on some systems (e.g. glibc 2.2). Otherwise linkage problems may occur when compiling with `AC_SYS_LARGEFILE' on largefile-sensitive systems where `off_t' does not default to a 64bit entity. The "_LARGEFILE_SOURCE" bit is the important one -- I have a vague memory of some program having a problem once in this area. > > I was already a bit surprised having failed to see AC_SYS_LARGEFILE > > being called from your patch, > > Are you sure you looked at my patch? All three versions of the patch > pretty much begin with the "+AC_SYS_LARGEFILE" change to configure.in. Well, I must have been blind. Or I'm simply too busy these days. Anyway, sorry about the confusion and I'm glad you've got this right despite the issues with LFS being too messy on many systems with no good reason. Maciej
Re: O_EXCL and large files
"Maciej W. Rozycki" <[EMAIL PROTECTED]> writes: >> Is it possible to portably use open() and retain large file support? > > Try the AC_SYS_LARGEFILE autoconf macro. That's what I thought I was using. I was just afraid that open() wasn't correctly encompassed by the large file API's, a fear that proved unfounded. > You may also want AC_FUNC_FSEEKO for fseeko(). I wonder what is the difference between AC_FUNC_FSEEKO and AC_CHECK_FUNCS(seeko). The manual doesn't seem to explain. > I was already a bit surprised having failed to see AC_SYS_LARGEFILE > being called from your patch, Are you sure you looked at my patch? All three versions of the patch pretty much begin with the "+AC_SYS_LARGEFILE" change to configure.in.
Re: O_EXCL and large files
On Tue, 22 Feb 2005, Hrvoje Niksic wrote: > However, I suspect that this will not work correctly with large files. > strace shows that, when large files are used, Linux fopen includes > O_LARGEFILE among the flags. Solaris, on the other hand, seems to use > the open64 function. (But will open be automatically mapped to open64 > when _FILE_OFFSET_BITS is 64?) For all I know, other systems may > require something different. > > Is it possible to portably use open() and retain large file support? Try the AC_SYS_LARGEFILE autoconf macro. It has the advantage of being the standard way of adding support for LFS these days and thus is supposed to work for all systems that are in use by a reasonable number of people (and the others can provide fixes directly to autoconf, whether they use wget or not -- you just need to keep your autoconf you're making releases with up to date, which is an expected practice anyway). It also has a side effect of providing the "--disable-largefile" configure option, which may be useful for some people. The macro is reasonably documented in the autoconf manual (as of 2.59, at least). It's used by GNU coreutils among others, which are about the most extensive users of all kinds of file-related calls. You may also want AC_FUNC_FSEEKO for fseeko(). I was already a bit surprised having failed to see AC_SYS_LARGEFILE being called from your patch, but I haven't had time to do any testing yet, so I hesitated making any comments, sorry. Maciej
Re: O_EXCL and large files
[EMAIL PROTECTED] (Steven M. Schweda) writes: >SunOS 5.9 /usr/include/fcntl.h: > > [...] > /* large file compilation environment setup */ > #if !defined(_LP64) && _FILE_OFFSET_BITS == 64 > #ifdef __PRAGMA_REDEFINE_EXTNAME > #pragma redefine_extnameopenopen64 > #pragma redefine_extnamecreat creat64 > [...] > > The idea is to continue to use open(), but to define the right macros to > get the right open(). Good to know. Regarding Linux, it turns out I spoke too soon. Apparently it also has an open64, except it's not visible by trace because the wrapper is in libc. When I actually tried to run the above code snippet, the O_LARGEFILE flag was there all right.
Re: O_EXCL and large files
From: Hrvoje Niksic: > [...] Solaris, on the other hand, seems to use > the open64 function. (But will open be automatically mapped to open64 > when _FILE_OFFSET_BITS is 64?) For all I know, other systems may > require something different. SunOS 5.9 /usr/include/fcntl.h: [...] /* large file compilation environment setup */ #if !defined(_LP64) && _FILE_OFFSET_BITS == 64 #ifdef __PRAGMA_REDEFINE_EXTNAME #pragma redefine_extname openopen64 #pragma redefine_extname creat creat64 [...] The idea is to continue to use open(), but to define the right macros to get the right open(). On VMS (with its RMS I/O layer), open() is not so fundamental, and only functions which explicitly use off_t seem to be affected. And yes, VMS does "require something different". The macro there is "_LARGEFILE". (But that's a problem for config.h and/or the builders.) A quick look at Tru64 UNIX suggests that large-file is all there is. I'd say that if it fails on Linux, then Linux has a problem. (But I'd expect it to be fine if you do it correctly.) Steven M. Schweda (+1) 651-699-9818 382 South Warwick Street[EMAIL PROTECTED] Saint Paul MN 55105-2547
