Frank, For clarity in the understanding the underlying mechanism in GPFS, could you describe what happens in the case say of a particular file that is appended to every 24 hours? ie. as that file gets to 7MB, it then writes to a new sub-block (1/32 of the next 1MB block). I guess that sub block could be 10th in a a block that already has 9 used. Later on, the file grows to need an 11th subblock and so on. So at what point does this growing file at 8MB occupy all 32 sunblocks of 8 full blocks?
Daniel Dr Daniel Kidger IBM Technical Sales Specialist Software Defined Solution Sales + 44-(0)7818 522 266 daniel.kid...@uk.ibm.com > On 6 Nov 2017, at 00:57, Frank Schmuck <fschm...@us.ibm.com> wrote: > > In GPFS blocks within a file are never fragmented. For example, if you have > a file of size 7.3 MB and your file system block size is 1MB, then this file > will be made up of 7 full blocks and one fragment of size 320k (10 > subblocks). Each of the 7 full blocks will be contiguous on a singe diks > (LUN) behind a single NSD server. The fragment that makes up the last part > of the file will also be contiguous on a single disk, just shorter than a > full block. > > Frank Schmuck > IBM Almaden Research Center > > > ----- Original message ----- > From: Aaron Knister <aaron.s.knis...@nasa.gov> > Sent by: gpfsug-discuss-boun...@spectrumscale.org > To: <gpfsug-discuss@spectrumscale.org> > Cc: > Subject: Re: [gpfsug-discuss] file layout API + file fragmentation > Date: Sun, Nov 5, 2017 3:39 PM > > Thanks Marc, that helps. I can't easily use tsdbfs for what I'm working > on since it needs to be run as unprivileged users. > > Perhaps I'm not asking the right question. I'm wondering how the file > layout api behaves if a given "block"-aligned offset in a file is made > up of sub-blocks/fragments that are not all on the same NSD. The > assumption based on how I've seen the API used so far is that all > sub-blocks within a block at a given offset within a file are all on the > same NSD. > > -Aaron > > On 11/5/17 6:01 PM, Marc A Kaplan wrote: > > I googled GPFS_FCNTL_GET_DATABLKDISKIDX > > > > and found this discussion: > > > > Â > > https://www.ibm.com/developerworks/community/forums/html/topic?id=db48b190-4f2f-4e24-a035-25d3e2b06b2d&ps=50 > > > > In general, GPFS files ARE deliberately "fragmented" but we don't say > > that - we say they are "striped" over many disks -- and that is > > generally a good thing for parallel performance. > > > > Also, in GPFS, if the last would-be block of a file is less than a > > block, then it is stored in a "fragment" of a block. Â > > So you see we use "fragment" to mean something different than other file > > systems you may know. > > > > --marc > > > > > > > > From: Â Â Â Â Aaron Knister <aaron.s.knis...@nasa.gov> > > To: Â Â Â Â gpfsug main discussion list > > <gpfsug-discuss@spectrumscale.org> > > Date: Â Â Â Â 11/04/2017 12:22 PM > > Subject: Â Â Â Â [gpfsug-discuss] file layout API + file fragmentation > > Sent by: Â Â Â Â gpfsug-discuss-boun...@spectrumscale.org > > ------------------------------------------------------------------------ > > > > > > > > I've got a question about the file layout API and how it reacts in the > > case of fragmented files. > > > > I'm using the GPFS_FCNTL_GET_DATABLKDISKIDX structure and have some code > > based on tsGetDataBlk.C. I'm basing the block size based off of what's > > returned by filemapOut.blockSize but that only seems to return a value > > > 0 when filemapIn.startOffset is 0. > > > > In a case where a file were to be made up of a significant number of > > non-contiguous fragments (which... would be awful in of itself) how > > would this be reported by the file layout API? Does the interface > > technically just report the disk location information of the first block > > of the $blockSize range and assume that it's contiguous? > > > > Thanks! > > > > -Aaron > > > > -- > > Aaron Knister > > NASA Center for Climate Simulation (Code 606.2) > > Goddard Space Flight Center > > (301) 286-2776 > > _______________________________________________ > > gpfsug-discuss mailing list > > gpfsug-discuss at spectrumscale.org > > https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=wnR7m6d4urZ_8dM4mkHQjMbFD9xJEeesmJyzt1osCnM&s=-dgGO6O5i1EqWj-8MmzjxJ1Iz2I5gT1aRmtyP44Cvdg&e= > > > > > > > > > > > > > > _______________________________________________ > > gpfsug-discuss mailing list > > gpfsug-discuss at spectrumscale.org > > https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=pUdB4fbWLD03ZTAhk9OlpRdIasz628Oa_yG8z8NOjsk&s=kisarJ7IVnyYBx05ZZiGzdwaXnPqNR8UJoywU1OJNRU&e= > > > > -- > Aaron Knister > NASA Center for Climate Simulation (Code 606.2) > Goddard Space Flight Center > (301) 286-2776 > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=pUdB4fbWLD03ZTAhk9OlpRdIasz628Oa_yG8z8NOjsk&s=kisarJ7IVnyYBx05ZZiGzdwaXnPqNR8UJoywU1OJNRU&e= > > > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=WH1GLDCza1Rvd9bzdVYoz2Pdzs7l90XNnhUb40FYCqQ&s=LOkUY79m5Ow2FeKqfCqc31cfXZVmqYlvBuQRPirGOFU&e= > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss