Re: [lustre-discuss] Lustre optimize for spares data files ?
On Sep 8, 2020, at 9:13 PM, Tung-Han Hsieh wrote: > > I would like to ask whether Lustre file system has implemented the > function to optimize for large sparse data files ? > > For example, a 3GB data file but with more than 80% bytes zero, can > Lustre file system optimize the storage not actually taking the whole > 3GB of disk space ? Could you please explain your usage further? Lustre definitely has support for sparse files - if they are written by an application with "seek" or by multiple threads in parallel, then only the blocks that are written will use space on the OST. For ldiskfs the block size is 4KB. For ZFS the OST block size is up to 1MB, if the file size is 1MB or larger. That is why compression on ZFS can help reduce space usage on the OST, because it can effectively compress the 1MB blocks that are nearly full of zeroes, if your sparse writes are smaller than the blocksize. If you are *copying* a sparse file, that depends on the tool that is doing the copy. For example, "cp --sparse=always" will generate a sparse file. We are also working on adding SEEK_HOLE and SEEK_DATA, which will help tools to copy sparse files. Cheers, Andreas > I know that some file systems (e.g., ZFS) has this function. If Lustre > does not have it, is there a roadmap to implement it in the future ? > > Thanks for your reply in advance. > > Best Regards, > > T.H.Hsieh > ___ > lustre-discuss mailing list > lustre-discuss@lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas signature.asc Description: Message signed with OpenPGP ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Lustre optimize for spares data files ?
Dear Tung-Han, Am 09.09.20 um 09:49 schrieb Tung-Han Hsieh: > Dear Robert, > > Thank you very much for your prompt reply. > > I have additional questions. Since we have a large Lustre file > system pool, some are OST with ldiskfs backend, and some are OST > with ZFS backend. Can I just enable compression on the ZFS backend > which is going to be joined into the Lustre file system without > affecting the remaining parts ? Maybe someone else can answer that. Our system runs exclusively on ZFS backends (OST+MDT). I think it is not a problem to mix backends, but I'm not sure and I never tried. > > If my OST with ZFS has run for a while without compression enabled, > could I enable it without hurting the existing data ? Yes. If you set compression=on, then new data will be compressed (with LZ4), old data remains unchanged. If you turn compression off again, then new data will not be compressed anymore. But data already in the system remains compressed. This is the normal ZFS behavior. Moving data with lfs migrate to an OST with compression enabled will result in compressed data. Cheers, Robert > > Thank you very much. > > T.H.Hsieh > > > On Wed, Sep 09, 2020 at 08:32:30AM +0200, Robert Redl wrote: >> Dear Tung-Han, >> >> you can use Lustre with ZFS backend and enabled compression. That has >> the effect you are looking for and works very well. >> >> Cheers, >> Robert >> >> Am 09.09.20 um 05:13 schrieb Tung-Han Hsieh: >>> Dear All, >>> >>> I would like to ask whether Lustre file system has implemented the >>> function to optimize for large sparse data files ? >>> >>> For example, a 3GB data file but with more than 80% bytes zero, can >>> Lustre file system optimize the storage not actually taking the whole >>> 3GB of disk space ? >>> >>> I know that some file systems (e.g., ZFS) has this function. If Lustre >>> does not have it, is there a roadmap to implement it in the future ? >>> >>> Thanks for your reply in advance. >>> >>> Best Regards, >>> >>> T.H.Hsieh >>> ___ >>> lustre-discuss mailing list >>> lustre-discuss@lists.lustre.org >>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org >> ___ >> lustre-discuss mailing list >> lustre-discuss@lists.lustre.org >> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > ___ > lustre-discuss mailing list > lustre-discuss@lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org -- Dr. Robert Redl Scientific Programmer, "Waves to Weather" (SFB/TRR165) Meteorologisches Institut Ludwig-Maximilians-Universität München Theresienstr. 37, 80333 München, Germany Tel.: +49 89 2180 4569 signature.asc Description: OpenPGP digital signature ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
[lustre-discuss] client syslog flood with "client_bulk_callback()) event type 2, status -103"
Hi, all After upgrade lustre client from 2.12.2 to 2.12.5，we found some clients flood with messages like " [Wed Sep 9 15:49:05 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:06 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:06 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:07 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:07 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:08 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:08 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:08 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:09 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:10 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:11 2020] LNetError: 0:0:(o2iblnd_cb.c:3676:kiblnd_qp_event()) 10.10.41.11@o2ib: Async QP event type 3 [Wed Sep 9 15:49:11 2020] LNetError: 0:0:(o2iblnd_cb.c:3676:kiblnd_qp_event()) Skipped 599 previous similar messages [Wed Sep 9 15:49:11 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:11 2020] Lustre: 3845:0:(client.c:2133:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1599637713/real 1599637713] req@9a35c1e83f00 x1676624438686784/t0(0) o3->public1-OST000a-osc-9a3f3b3af000@10.10.41.11@o2ib:6/4 lens 488/440 e 1 to 1 dl 1599637816 ref 2 fl Rpc:eX/2/ rc -11/-1 [Wed Sep 9 15:49:11 2020] Lustre: 3845:0:(client.c:2133:ptlrpc_expire_one_request()) Skipped 599 previous similar messages [Wed Sep 9 15:49:11 2020] Lustre: public1-OST000a-osc-9a3f3b3af000: Connection to public1-OST000a (at 10.10.41.11@o2ib) was lost; in progress operations using this service will wait for recovery to complete [Wed Sep 9 15:49:11 2020] Lustre: Skipped 599 previous similar messages [Wed Sep 9 15:49:12 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:12 2020] Lustre: public1-OST000a-osc-9a3f3b3af000: Connection restored to 10.10.41.11@o2ib (at 10.10.41.11@o2ib) [Wed Sep 9 15:49:12 2020] Lustre: Skipped 599 previous similar messages [Wed Sep 9 15:49:13 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:13 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:14 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:14 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:14 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:14 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:15 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:15 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 [Wed Sep 9 15:49:16 2020] LustreError: 3476:0:(events.c:200:client_bulk_callback()) event type 2, status -103, desc 9a1f3cafd000 " Any suggestions? ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Lustre optimize for spares data files ?
Dear Robert, Thank you very much for your prompt reply. I have additional questions. Since we have a large Lustre file system pool, some are OST with ldiskfs backend, and some are OST with ZFS backend. Can I just enable compression on the ZFS backend which is going to be joined into the Lustre file system without affecting the remaining parts ? If my OST with ZFS has run for a while without compression enabled, could I enable it without hurting the existing data ? Thank you very much. T.H.Hsieh On Wed, Sep 09, 2020 at 08:32:30AM +0200, Robert Redl wrote: > Dear Tung-Han, > > you can use Lustre with ZFS backend and enabled compression. That has > the effect you are looking for and works very well. > > Cheers, > Robert > > Am 09.09.20 um 05:13 schrieb Tung-Han Hsieh: > > Dear All, > > > > I would like to ask whether Lustre file system has implemented the > > function to optimize for large sparse data files ? > > > > For example, a 3GB data file but with more than 80% bytes zero, can > > Lustre file system optimize the storage not actually taking the whole > > 3GB of disk space ? > > > > I know that some file systems (e.g., ZFS) has this function. If Lustre > > does not have it, is there a roadmap to implement it in the future ? > > > > Thanks for your reply in advance. > > > > Best Regards, > > > > T.H.Hsieh > > ___ > > lustre-discuss mailing list > > lustre-discuss@lists.lustre.org > > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > ___ > lustre-discuss mailing list > lustre-discuss@lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
