Andreas,
Thank you for your reply. I am not asking about the Hybrid I/O
feature. A bit more on that later. I am asking about the oti_dio_pages
( Direct I/O ) path through the OSS. With some experimentation last
night I was able to confirm that an OSS uses Direct I/O when the
incoming client request is 8MB or greater. On the system I am running
on ( clients version 2.14 and server version as reported by pminfo is
lustre.sys.version=2 lustre.sys.build=0_ddn218 ) it appears that the
reads and writes use the server cache when the user requests are less
that 8MB and using non-rotational OSTs. The OSS could have been
configured this way. I will check with admin next week.
Back to Hybrid I/O. It concerns me that there is no mechanism is user
space that will allow a client application to specify legacy client-side
buffering on a per-file basis, independent of request size. I envision
plenty of cases where a client application is running dedicated on a
1TB memory client node, which leaves 500 MB for Lustre caching, and the
application is doing large request I/O to a file that is under 500MB.
The client application would now be subject to Hybrid's direct I/O even
though the entire file could be buffer cached on the client. Am I
misunderstanding something here?
Thanks again,
John
On 1/9/2026 7:02 PM, Andreas Dilger wrote:
Hi John,
are you asking about the Hybrid IO feature in 2.16+ Lustre releases,
or something else?
There is a client-side "llite.*.hybrid_io" parameter that can enable/disable
Hybrid IO on a client completely, and the
llite.*.hybrid_io_write_threshold_bytes and
llite.*.hybrid_io_read_threshold_bytes can be used to tune the
IO size threshold where IO changes between buffered and direct.
Applications can of course use open(O_DIRECT) to use DIO instead of
buffered IO.
As for the server-side non-buffered IO path, this is controlled by
the osd-ldiskfs.*.readcache_max_io_mb and .writethrough_max_io_mb
parameters, both default to 8 MiB. Note that flash (non-rotational)
OSTs disable read and write cache entirely by default, since NVMe
devices are typically fast enough to handle incoming IO.
I don't think there is any way for clients to fetch these parameters
directly, since they are more a property of the OST than the client.
Cheers, Andreas
On Jan 9, 2026, at 14:32, John Bauer <[email protected]
<mailto:[email protected]>> wrote:
Hello all,
Is the a way to determine what size a client I/O request must be to trigger the
OSS to use the Large File ( non-buffered ) I/O path? Is this configurable?
Is there a way for a client-side application to trigger this behavior
independent of the I/O size?
---
Andreas Dilger
Principal Lustre Architect
[email protected]
_______________________________________________
lustre-discuss mailing list
[email protected]
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
