On Mon, Aug 22, 2016 at 11:52:12PM +0300, Alexander Gordeev wrote: > Hi, > > I ran the test over weekend and I think I have some interesting results. > I used 1 new SD card in one device and two fully utilized SD cards, > that have problems with write latency, on two oter devices. > I mounted all the cards with mode=lfs. The new SD card got utilized at 95% > after some time. > Here is the current status after the archive is rotated for some time: > > =====[ partition info(sda1). #0, RW]===== > [SB: 1] [CP: 2] [SIT: 2] [NAT: 68] [SSA: 30] [MAIN: 15112(OverProv:803 > Resv:50)] > > Utilization: 94% (6929763 valid blocks) > - Node: 8113 (Inode: 1255, Other: 6858) > - Data: 6921650 > - Inline_xattr Inode: 0 > - Inline_data Inode: 1 > - Inline_dentry Inode: 0 > - Orphan Inode: 0 > > Main area: 15112 segs, 7556 secs 7556 zones > - COLD data: 5306, 2653, 2653 > - WARM data: 5233, 2616, 2616 > - HOT data: 15100, 7550, 7550 > - Dir dnode: 15097, 7548, 7548 > - File dnode: 4701, 2350, 2350 > - Indir nodes: 15105, 7552, 7552 > > - Valid: 97 > - Dirty: 13798 > - Prefree: 0 > - Free: 1217 (604) > > CP calls: 282 (BG: 0) > GC calls: 0 (BG: 0) > - data segments : 0 (0) > - node segments : 0 (0) > Try to move 0 blocks (BG: 0) > - data blocks : 0 (0) > - node blocks : 0 (0) > > Extent Cache: > - Hit Count: L1-1:3084 L1-2:456 L2:0 > - Hit Ratio: 4% (3540 / 84026) > - Inner Struct Count: tree: 1252(0), node: 0 > > Balancing F2FS Async: > - inmem: 0, wb_bios: 0 > - nodes: 12 in 30 > - dents: 3 in dirs: 2 ( 2) > - datas: 48 in files: 0 > - meta: 9 in 34 > - NATs: 10/ 249 > - SITs: 13/ 15112 > - free_nids: 1797 > > Distribution of User Blocks: [ valid | invalid | free ] > [-----------------------------------------------|--|-] > > IPU: 0 blocks > SSR: 0 blocks in 0 segments > LFS: 912188 blocks in 1781 segments > > BDF: 94, avg. vblocks: 996 > > Memory: 3604 KB > - static: 3270 KB > - cached: 78 KB > - paged : 256 KB > > > The interesting thing here is the very small number of valid and > a huge number of dirty sections. I don't understand this at all.
This is the number of dirty segments, so it needs to consider section and segment at the same time; a dirty section can consist of valid and free segments. How abouting testing 2MB-sized section which is the default option? > Still the archive is working perfectly. Also I see, that there GC is never > called, which is probably an indication of FS working exactly as > we expected. > Also the small number of cold sections does not make problems. > So, well, it works perfect so fat. But I don't understand everything here. > Is this expected? So, I'm in doubt that dirty sections consist of entirely valid or free segments. > > The other two SD cards were tested differently. On one of them I called > ioctl(F2FS_IOC_GARBAGE_COLLECT) for several hours. And indeed the number > of dirty sectoins dropped considerably. It works fine so far. It makes sense that valid segments in dirty sections would be migrated to different free sections. > On the other SD card I called ioctl(F2FS_IOC_DEFRAGMENT) for every > file in the archive. It works fine as well now. But the number of dirty > sections > was still very high at the end of defragmentation. I don't understand this > as well. This is for defragementation to the given file, which would not move blocks in order to decrease the number of dirty sections. I think it's not necessary for this workload. Thanks, > > Thanks! > > 19.08.2016, 14:56, "Alexander Gordeev" <[email protected]>: > > Hi Jaegeuk, > > > > 19.08.2016, 05:41, "Jaegeuk Kim" <[email protected]>: > >> Hello, > >> > >> On Thu, Aug 18, 2016 at 02:04:55PM +0300, Alexander Gordeev wrote: > >> > >> ... > >> > >>> >>>>>>> Here is also /sys/kernel/debug/f2fs/status for reference: > >>> >>>>>>> =====[ partition info(sda). #0 ]===== > >>> >>>>>>> [SB: 1] [CP: 2] [SIT: 4] [NAT: 118] [SSA: 60] [MAIN: > >>> 29646(OverProv:1529 > >>> >>>>>>> Resv:50)] > >>> >>>>>>> > >>> >>>>>>> Utilization: 94% (13597314 valid blocks) > >>> >>>>>>> - Node: 16395 (Inode: 2913, Other: 13482) > >>> >>>>>>> - Data: 13580919 > >>> >>>>>>> > >>> >>>>>>> Main area: 29646 segs, 14823 secs 14823 zones > >>> >>>>>>> - COLD data: 3468, 1734, 1734 > >>> >>>>>>> - WARM data: 12954, 6477, 6477 > >>> >>>>>>> - HOT data: 28105, 14052, 14052 > >>> >>>>>>> - Dir dnode: 29204, 14602, 14602 > >>> >>>>>>> - File dnode: 19960, 9980, 9980 > >>> >>>>>>> - Indir nodes: 29623, 14811, 14811 > >>> >>>>>>> > >>> >>>>>>> - Valid: 13615 > >>> >>>>>>> - Dirty: 13309 > >>> >>>>>>> - Prefree: 0 > >>> >>>>>>> - Free: 2722 (763) > >>> >>>>>>> > >>> >>>>>>> GC calls: 8622 (BG: 4311) > >>> >>>>>>> - data segments : 8560 > >>> >>>>>>> - node segments : 62 > >>> >>>>>>> Try to move 3552161 blocks > >>> >>>>>>> - data blocks : 3540278 > >>> >>>>>>> - node blocks : 11883 > >>> >>>>>>> > >>> >>>>>>> Extent Hit Ratio: 49 / 4171 > >>> >>>>>>> > >>> >>>>>>> Balancing F2FS Async: > >>> >>>>>>> - nodes 6 in 141 > >>> >>>>>>> - dents 0 in dirs: 0 > >>> >>>>>>> - meta 13 in 346 > >>> >>>>>>> - NATs 16983 > 29120 > >>> >>>>>>> - SITs: 17 > >>> >>>>>>> - free_nids: 1861 > >>> >>>>>>> > >>> >>>>>>> Distribution of User Blocks: [ valid | invalid | free ] > >>> >>>>>>> [-----------------------------------------------|-|--] > >>> >>>>>>> > >>> >>>>>>> SSR: 1230719 blocks in 14834 segments > >>> >>>>>>> LFS: 15150190 blocks in 29589 segments > >>> >>>>>>> > >>> >>>>>>> BDF: 89, avg. vblocks: 949 > >>> >>>>>>> > >>> >>>>>>> Memory: 6754 KB = static: 4763 + cached: 1990 > >> > >> ... > >> > >>> >> Per my understanding of f2fs internals, it should write these > >>> "cold" files and > >>> >> usual "hot" files to different sections (that should map > >>> internally to > >>> >> different allocation units). So the sections used by "cold" data > >>> should almost > >>> >> never get "dirty" because most of the time all their blocks > >>> become free at > >>> >> the same time. Of course, the files are not exactly 4MB in size > >>> so the last > >>> >> section of the deleted file will become dirty. If it is moved by > >>> garbage > >>> >> collector and becomes mixed with fresh "cold" data, then indeed > >>> it might cause > >>> >> some problems, I think. What is your opinion? > >>> > > >>> > If your fs is not fragmented, it may be as what you said, > >>> otherwise, SSR will > >>> > still try to reuse invalid block of other temperture segments, then > >>> your cold > >>> > data will be fixed with warm data too. > >>> > > >>> > I guess, what you are facing is the latter one: > >>> > SSR: 1230719 blocks in 14834 segments > >>> > >>> I guess, I need to somehow disable any cleaning or SSR for my archive > >>> and index > >>> files. But keep the cleaning for other data and nodes. > >> > >> Could you test a mount option, "mode=lfs", to disable SSR? > >> (I guess sqlite may suffer from logner latency due to GC though.) > >> > >> Seems like it's caused by SSR starting to make worse before 95% as you > >> described > >> below. > > > > Thanks, I'll run a test with a couple of SD cards over weekend. > > So if I understand it correctly, GC will not cause the problems described > > below, right? > > I.e. it will not mix the new data with old data from dirty sections? > > Longer SQLite latencies should not be a problem because the database is > > written not > > frequently and also it is about 200-250KB in size usually. Maybe forcing > > IPU as > > suggested by Chao would help sqlite, no? > > However looks like setting ipu_policy to 1 has no effect when mode=lfs. > > The IPU counter is still zero on my test system. > > ... > -- > Alexander ------------------------------------------------------------------------------ _______________________________________________ Linux-f2fs-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
