I'll assume here you're referring to MPI-IO, which is not really a
"feature" but a way to perform parallel I/O using the message passing
interface (MPI) stack. There can also be different interpretations of
what MPI-IO exactly means: many HPC applications (and benchmarks such as
IOR) use MPI and can write out data in parallel using various I/O APIs.
One of those APIs for IOR is "mpiio", from the web page:
-a S api – API for I/O [POSIX|MPIIO|HDF5|HDFS|S3|S3_EMC|NCMPI|RADOS] In
short, running parallel threads to write files will usually boost your
IOPs to a point, though which API you choose should probably reflect
what your real-life applications use. Cameron
On 3/18/23 2:44 AM, 王烁斌 via lustre-discuss wrote:
Hi all,
This is my Lustre FS.
UUID 1K-blocks Used Available Use% Mounted on
ltfs-MDT0000_UUID 307826072 36904 281574768 1%
/mnt/lfs[MDT:0]
ltfs-MDT0001_UUID 307826072 36452 281575220 1%
/mnt/lfs[MDT:1]
ltfs-MDT0002_UUID 307826072 36600 281575072 1%
/mnt/lfs[MDT:2]
ltfs-MDT0003_UUID 307826072 36300 281575372 1%
/mnt/lfs[MDT:3]
ltfs-OST0000_UUID 15962575136 1027740 15156068868 1%
/mnt/lfs[OST:0]
ltfs-OST0001_UUID 15962575136 1027780 15156067516 1%
/mnt/lfs[OST:1]
ltfs-OST0002_UUID 15962575136 1027772 15156074212 1%
/mnt/lfs[OST:2]
ltfs-OST0003_UUID 15962575136 1027756 15156067860 1%
/mnt/lfs[OST:3]
ltfs-OST0004_UUID 15962575136 1027728 15156058224 1%
/mnt/lfs[OST:4]
ltfs-OST0005_UUID 15962575136 1027772 15156057668 1%
/mnt/lfs[OST:5]
ltfs-OST0006_UUID 15962575136 1027768 15156058568 1%
/mnt/lfs[OST:6]
ltfs-OST0007_UUID 15962575136 1027792 15156056752 1%
/mnt/lfs[OST:7]
filesystem_summary: 127700601088 8222108 121248509668 1% /mnt/lfs
Structure ias flow:
After testing, under the current structure, the write performance of
500,000 "8k" small files is:
NFSclient1——IOPS：28,000; bandwidth——230MB
NFSclient1——IOPS：27,500; bandwidth——220MB
Now I want to improve the performance of small files to a better
level，May I ask if there is a better way。
I have noticed a feature called "MIP-IO" that can improve small file
performance, but I don't know how to deploy this feature. Is there any
way to improve small file performance？
_______________________________________________
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
https://urldefense.us/v3/__http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org__;!!G2kpM7uM-TzIFchu!jpxmiee-6Txz4MkvTdXq-z7hM1I20qZ98shFSOwVie2UEZXxjo7aWF6Im3x0pik$
_______________________________________________
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
