Re: any update on large file support for linux?

[Lars Holowko]

On Tue, Sep 7, 2010 at 9:34 AM, Andrei Alexandrescu
 wrote:
>> I did not realize that 1024 * 1024 * 1024 * 6 turns negative and then
>> gets converted to a negative long (without even a warning). Overseeing
>> that had killed my own efforts to hack 64-bit support into phobos ;-)
>>
> Hmmm... the compiler could and should warn about integer overflow in
> computed constant. I suggest you file this as an improvement in bugzilla.
>
> Glad to hear large files are working for you.
>
>
> Andrei
>

I filed an enhancement request
http://d.puremagic.com/issues/show_bug.cgi?id=4835 as Andrei had
recommended.

Lars

Re: any update on large file support for linux?

[Andrei Alexandrescu]

On 9/7/10 11:05 CDT, Lars Holowko wrote:

On Tue, Sep 7, 2010 at 1:08 AM, Lars T. Kyllingstad
  wrote:

The SVN version of std.stdio supports large files on Linux and OSX.  The
next release will be a nice one, I think. :)

-Lars



Thanks Lars,

For the hint to the svn versions. Things seem to work there. What
really surprised me is that dmd compiles

  f.seek(1024 * 1024 * 1024 * 6, SEEK_SET);

but fails with

std.exception.errnoexcept...@std/stdio.d(538): Could not seek in file
`test.txt' (Invalid argument)

whereas

  f.seek(1024 * 1024 * 1024 * 6L, SEEK_SET);

works fine.

I did not realize that 1024 * 1024 * 1024 * 6 turns negative and then
gets converted to a negative long (without even a warning). Overseeing
that had killed my own efforts to hack 64-bit support into phobos ;-)

Thanks again,

(another ;-)) Lars


Hmmm... the compiler could and should warn about integer overflow in 
computed constant. I suggest you file this as an improvement in bugzilla.


Glad to hear large files are working for you.


Andrei

Re: any update on large file support for linux?

[Lars Holowko]

On Tue, Sep 7, 2010 at 1:08 AM, Lars T. Kyllingstad
 wrote:
> The SVN version of std.stdio supports large files on Linux and OSX.  The
> next release will be a nice one, I think. :)
>
> -Lars
>

Thanks Lars,

For the hint to the svn versions. Things seem to work there. What
really surprised me is that dmd compiles

 f.seek(1024 * 1024 * 1024 * 6, SEEK_SET);

but fails with

std.exception.errnoexcept...@std/stdio.d(538): Could not seek in file
`test.txt' (Invalid argument)

whereas

 f.seek(1024 * 1024 * 1024 * 6L, SEEK_SET);

works fine.

I did not realize that 1024 * 1024 * 1024 * 6 turns negative and then
gets converted to a negative long (without even a warning). Overseeing
that had killed my own efforts to hack 64-bit support into phobos ;-)

Thanks again,

(another ;-)) Lars

Re: any update on large file support for linux?

[Lars T. Kyllingstad]

The SVN version of std.stdio supports large files on Linux and OSX.  The 
next release will be a nice one, I think. :)

-Lars

any update on large file support for linux?

[Lars Holowko]

Hi everybody,

Are there any status updates on the large file support for dmd on Linux?
I found this bug: http://d.puremagic.com/issues/show_bug.cgi?id=3409
which has not been commented on for quite a while.
I was trying to dig around in druntime and phobos - a lot seems
already in place so I was hoping that I could get it to work with
minor hacking:
-  enable __USE_LARGEFILE64 for 32 bit dmd in
druntime/src/core/sys/posix/config.d (no idea why this is determined
by the processor's native pointer size)
- change std.stdio.File.seek to call fseeko instead of fseek

    void seek(long offset, int origin = SEEK_SET)
    {
        enforce(p && p.handle,
                "Attempting to seek() in an unopened file");
        // @@@ Dubious: why is fseek in std.c.stdio taking an int???
        errnoEnforce(core.sys.posix.stdio.fseeko(
//        errnoEnforce(core.stdc.stdio.fseek(
//                    p.handle, to!int(offset), origin) == 0,
                    p.handle, offset, origin) == 0,
                "Could not seek in file `"~p.name~"'");
    }


But when I run the slightly modified std.stdio sample:

import std.stdio;
import core.sys.posix.sys.types;
void main(string args[])
{
    writefln("Typeof(off_t) = %s", typeid(off_t));
    auto f = File("test.txt", "w"); // open for writing
    f.write("Hello");
    f.seek(1024 * 1024 * 1024 * 6, SEEK_SET);
    if (args.length > 1)
    {
        auto g = f; // now g and f write to the same file
                    // internal reference count is 2
        g.write(", ", args[1]);
        // g exits scope, reference count decreases to 1
    }
    f.writeln("!");
}

I get something like this:

Typeof(off_t) = long
std.exception.errnoexcept...@std/stdio.d(526): Could not seek in file
`test.txt' (Invalid argument)

./io_test() [0x8055833]
./io_test() [0x8054d40]
./io_test() [0x8049859]
./io_test() [0x804f6b6]
./io_test() [0x804f610]
./io_test() [0x804f6fa]
./io_test() [0x804f610]
./io_test() [0x804f5b6]
/lib32/libc.so.6(__libc_start_main+0xe6) [0xf763bbd6]
./io_test() [0x8049741]



nm io_test | grep -e fopen -e fseek
080552f0 T _D3std5stdio5fopenFxAaxAaZPOS4core4stdc5stdio6_iobuf
         U fopen64@@GLIBC_2.1
         U fseeko64@@GLIBC_2.1

The 64 bit fopen and fseek calls seem to be linked, off_t is correctly
aliased to long. Does anyone have an idea what else I might be
missing?
Thanks a lot,

Lars