>I want to be able to read a HUGE file without having such a negative
>impact on the system's buffer cache.
>
>I'm trying:
>
> if hasattr(os, 'O_DIRECT'):
> try:
> flags = fcntl.fcntl(sys.stdin.fileno(), fcntl.F_GETFL)
>
> flags |= os.O_DIRECT
> fcntl.fcntl(sys.stdin.fileno(), fcntl.F_SETFL, flags)
> except:
> sys.stderr.write('Setting O_DIRECT on stdin attempted
> but failed\n')
> else:
> sys.stderr.write('Setting O_DIRECT on stdin succeeded
> :)\n')
>
>...but while this code doesn't error out, I get:
>
>seki-root> reblock -e $[1024*1024*80] $[1024*1024] 300 < /dev/sda1 > /dev/null
>stdin seems seekable, but file length is 0 - no exact percentages
>Estimated filetransfer size is 85899345920 bytes
>Estimated percentages will only be as accurate as your size estimate
>Setting O_DIRECT on stdin succeeded :)
>Traceback (most recent call last):
> File "/Dcs/seki/strombrg/bin/reblock", line 276, in ?
> main()
> File "/Dcs/seki/strombrg/bin/reblock", line 222, in main
> block = os.read(0,blocksize)
>OSError: [Errno 22] Invalid argument
>Mon Nov 07 12:25:53
>
>...but if I comment out the fcntl/O_DIRECT code, then the same thing works
>well.
>
>Any other ideas folks?
Does O_DIRECT perhaps invoke some of the restrictions of "raw"
device files, where the current offset and transfer size must be a
multiple of some block size? (I don't see any mention of that in
FreeBSD's documentation.) What is the value of blocksize at the
time of the traceback above? I suggest keeping it well under 2G.
Gordon L. Burditt
--
http://mail.python.org/mailman/listinfo/python-list
