Hi Jaegeuk, Thanks for all the help!
24.08.2016, 00:12, "Jaegeuk Kim" <[email protected]>: > On Mon, Aug 22, 2016 at 11:52:12PM +0300, Alexander Gordeev wrote: >> Hi, >> >> I ran the test over weekend and I think I have some interesting results. >> I used 1 new SD card in one device and two fully utilized SD cards, >> that have problems with write latency, on two oter devices. >> I mounted all the cards with mode=lfs. The new SD card got utilized at 95% >> after some time. >> Here is the current status after the archive is rotated for some time: >> >> =====[ partition info(sda1). #0, RW]===== >> [SB: 1] [CP: 2] [SIT: 2] [NAT: 68] [SSA: 30] [MAIN: 15112(OverProv:803 >> Resv:50)] >> >> Utilization: 94% (6929763 valid blocks) >> - Node: 8113 (Inode: 1255, Other: 6858) >> - Data: 6921650 >> - Inline_xattr Inode: 0 >> - Inline_data Inode: 1 >> - Inline_dentry Inode: 0 >> - Orphan Inode: 0 >> >> Main area: 15112 segs, 7556 secs 7556 zones >> - COLD data: 5306, 2653, 2653 >> - WARM data: 5233, 2616, 2616 >> - HOT data: 15100, 7550, 7550 >> - Dir dnode: 15097, 7548, 7548 >> - File dnode: 4701, 2350, 2350 >> - Indir nodes: 15105, 7552, 7552 >> >> - Valid: 97 >> - Dirty: 13798 >> - Prefree: 0 >> - Free: 1217 (604) >> >> CP calls: 282 (BG: 0) >> GC calls: 0 (BG: 0) >> - data segments : 0 (0) >> - node segments : 0 (0) >> Try to move 0 blocks (BG: 0) >> - data blocks : 0 (0) >> - node blocks : 0 (0) >> >> Extent Cache: >> - Hit Count: L1-1:3084 L1-2:456 L2:0 >> - Hit Ratio: 4% (3540 / 84026) >> - Inner Struct Count: tree: 1252(0), node: 0 >> >> Balancing F2FS Async: >> - inmem: 0, wb_bios: 0 >> - nodes: 12 in 30 >> - dents: 3 in dirs: 2 ( 2) >> - datas: 48 in files: 0 >> - meta: 9 in 34 >> - NATs: 10/ 249 >> - SITs: 13/ 15112 >> - free_nids: 1797 >> >> Distribution of User Blocks: [ valid | invalid | free ] >> [-----------------------------------------------|--|-] >> >> IPU: 0 blocks >> SSR: 0 blocks in 0 segments >> LFS: 912188 blocks in 1781 segments >> >> BDF: 94, avg. vblocks: 996 >> >> Memory: 3604 KB >> - static: 3270 KB >> - cached: 78 KB >> - paged : 256 KB >> >> The interesting thing here is the very small number of valid and >> a huge number of dirty sections. I don't understand this at all. > > This is the number of dirty segments, so it needs to consider section and > segment at the same time; a dirty section can consist of valid and free > segments. > How abouting testing 2MB-sized section which is the default option? I tried what you said. Still the majority of segments are dirty for some reason: =====[ partition info(sda). #0, RW]===== [SB: 1] [CP: 2] [SIT: 6] [NAT: 114] [SSA: 116] [MAIN: 59149(OverProv:3003 Resv:48)] Utilization: 10% (3013093 valid blocks) - Node: 3528 (Inode: 548, Other: 2980) - Data: 3009565 - Inline_xattr Inode: 0 - Inline_data Inode: 0 - Inline_dentry Inode: 0 - Orphan Inode: 0 Main area: 59149 segs, 59149 secs 59149 zones - COLD data: 7183, 7183, 7183 - WARM data: 6654, 6654, 6654 - HOT data: 59134, 59134, 59134 - Dir dnode: 59127, 59127, 59127 - File dnode: 59125, 59125, 59125 - Indir nodes: 59129, 59129, 59129 - Valid: 300 - Dirty: 6438 - Prefree: 0 - Free: 52411 (52411) CP calls: 1023 (BG: 473) GC calls: 470 (BG: 470) - data segments : 466 (466) - node segments : 4 (4) Try to move 152221 blocks (BG: 152221) - data blocks : 151417 (151417) - node blocks : 804 (804) Extent Cache: - Hit Count: L1-1:6262 L1-2:0 L2:0 - Hit Ratio: 2% (6262 / 273606) - Inner Struct Count: tree: 292(0), node: 8 Balancing F2FS Async: - inmem: 0, wb_bios: 0 - nodes: 0 in 0 - dents: 0 in dirs: 0 ( 0) - datas: 0 in files: 0 - meta: 0 in 0 - NATs: 0/ 43 - SITs: 0/ 59149 - free_nids: 3414 Distribution of User Blocks: [ valid | invalid | free ] [-----|--|-------------------------------------------] IPU: 0 blocks SSR: 0 blocks in 0 segments LFS: 3691542 blocks in 7208 segments BDF: 95, avg. vblocks: 444 Memory: 12662 KB - static: 12597 KB - cached: 64 KB - paged : 0 KB But the archive is working perfectly as before. >> Still the archive is working perfectly. Also I see, that there GC is never >> called, which is probably an indication of FS working exactly as >> we expected. >> Also the small number of cold sections does not make problems. >> So, well, it works perfect so fat. But I don't understand everything here. >> Is this expected? > > So, I'm in doubt that dirty sections consist of entirely valid or free > segments. > >> The other two SD cards were tested differently. On one of them I called >> ioctl(F2FS_IOC_GARBAGE_COLLECT) for several hours. And indeed the number >> of dirty sectoins dropped considerably. It works fine so far. > > It makes sense that valid segments in dirty sections would be migrated to > different free sections. > >> On the other SD card I called ioctl(F2FS_IOC_DEFRAGMENT) for every >> file in the archive. It works fine as well now. But the number of dirty >> sections >> was still very high at the end of defragmentation. I don't understand this >> as well. > > This is for defragementation to the given file, which would not move blocks in > order to decrease the number of dirty sections. > I think it's not necessary for this workload. Yes, I'd prefer to not run manual GC or defragmentation. This was just a test. -- Alexander ------------------------------------------------------------------------------ _______________________________________________ Linux-f2fs-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
