Re: [lustre-discuss] quota only in- but not decreasing after upgrading to Lustre 2.5.3
Hi, it might help to disable quota using tune2fs and re-enable it again on the ext2 level on all devices, see LU-3861. (BTW you don't need the e2fsprogs mentioned in the bug, there was an official release last year in September). You have to stop lustre for the tune2fs run and it takes some time, because this triggers a quota check in the background (which does not produce any output on the screen). best regards, Martin On 07/28/2015 09:44 AM, Torsten Harenberg wrote: a further observation: a user deleted a ~100MB file: before. [root@wnfg001 lustre]# lfs quota -u sandhoff /lustre Disk quotas for user sandhoff (uid 11206): Filesystem kbytes quota limit grace files quota limit grace /lustre 811480188 1811480200 2811480200 - 61077 0 0 - after: [root@wnfg001 lustre]# lfs quota -u sandhoff /lustre Disk quotas for user sandhoff (uid 11206): Filesystem kbytes quota limit grace files quota limit grace /lustre 811480188 1811480200 2811480200 - 61076 0 0 - [root@wnfg001 lustre]# so #files decreased by 1, but not the #kbytes. Furthermore, the lfs quota command is pretty slow: [root@wnfg001 lustre]# time lfs quota -u sandhoff /lustre Disk quotas for user sandhoff (uid 11206): Filesystem kbytes quota limit grace files quota limit grace /lustre 811480188 1811480200 2811480200 - 61076 0 0 - real0m2.441s user0m0.001s sys 0m0.004s [root@wnfg001 lustre]# although the system is not overloaded. Couldn't find anything useful in dmesg: [root@lustre2 ~]# dmesg | grep quota VFS: Disk quotas dquot_6.5.2 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. quota=on. Opts: [root@lustre2 ~]# [root@lustre3 ~]# dmesg | grep quota VFS: Disk quotas dquot_6.5.2 LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-7): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-9): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-13): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-12): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-10): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-7): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-9): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-13): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-12): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-10): mounted filesystem with ordered data mode. quota=on. Opts: [root@lustre3 ~]# [root@lustre4 ~]# dmesg | grep quota VFS: Disk quotas dquot_6.5.2 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-8): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-13): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-14): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-8): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-13): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-14): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=on. Opts: [root@lustre4 ~]# Thanks for any hint! Best regards Torsten smime.p7s Description: S/MIME Cryptographic Signature ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] quota only in- but not decreasing after upgrading to Lustre 2.5.3
a further observation: a user deleted a ~100MB file: before. [root@wnfg001 lustre]# lfs quota -u sandhoff /lustre Disk quotas for user sandhoff (uid 11206): Filesystem kbytes quota limit grace files quota limit grace /lustre 811480188 1811480200 2811480200 - 61077 0 0 - after: [root@wnfg001 lustre]# lfs quota -u sandhoff /lustre Disk quotas for user sandhoff (uid 11206): Filesystem kbytes quota limit grace files quota limit grace /lustre 811480188 1811480200 2811480200 - 61076 0 0 - [root@wnfg001 lustre]# so #files decreased by 1, but not the #kbytes. Furthermore, the lfs quota command is pretty slow: [root@wnfg001 lustre]# time lfs quota -u sandhoff /lustre Disk quotas for user sandhoff (uid 11206): Filesystem kbytes quota limit grace files quota limit grace /lustre 811480188 1811480200 2811480200 - 61076 0 0 - real0m2.441s user0m0.001s sys 0m0.004s [root@wnfg001 lustre]# although the system is not overloaded. Couldn't find anything useful in dmesg: [root@lustre2 ~]# dmesg | grep quota VFS: Disk quotas dquot_6.5.2 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. quota=on. Opts: [root@lustre2 ~]# [root@lustre3 ~]# dmesg | grep quota VFS: Disk quotas dquot_6.5.2 LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-7): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-9): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-13): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-12): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-10): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-7): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-9): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-13): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-12): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-10): mounted filesystem with ordered data mode. quota=on. Opts: [root@lustre3 ~]# [root@lustre4 ~]# dmesg | grep quota VFS: Disk quotas dquot_6.5.2 LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-8): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-13): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-14): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-8): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-13): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-14): mounted filesystem with ordered data mode. quota=on. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=on. Opts: [root@lustre4 ~]# Thanks for any hint! Best regards Torsten -- Dr. Torsten Harenberg torsten.harenb...@cern.ch Bergische Universitaet FB C - Physik Tel.: +49 (0)202 439-3521 Gaussstr. 20 Fax : +49 (0)202 439-2811 42097 Wuppertal @CERN: Bat. 1-1-049 Of course it runs NetBSD http://www.netbsd.org ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] quota only in- but not decreasing after upgrading to Lustre 2.5.3
Good morning Malcom, all, Am 28.07.15 um 03:23 schrieb Cowe, Malcolm J: lctl conf_param fsname.quota.ost=ug You can verify status on the servers with: lctl get_param *.*.quota_slave.info think we did that correctly: on the MDS/MGT: [root@lustre2 ~]# lctl get_param *.*.quota_slave.info osd-ldiskfs.lustre-MDT.quota_slave.info= target name:lustre-MDT pool ID:0 type: md quota enabled: ug conn to master: setup space acct: ug user uptodate: glb[1],slv[1],reint[0] group uptodate: glb[1],slv[1],reint[0] [root@lustre2 ~]# on the OSTs: [root@lustre4 ~]# lctl get_param *.*.quota_slave.info | grep quota enable | uniq -c 7 quota enabled: ug [root@lustre4 ~]# [root@lustre3 ~]# lctl get_param *.*.quota_slave.info | grep quota enable | uniq -c 8 quota enabled: ug [root@lustre3 ~]# Best regards Torsten -- Dr. Torsten Harenberg torsten.harenb...@cern.ch Bergische Universitaet FB C - Physik Tel.: +49 (0)202 439-3521 Gaussstr. 20 Fax : +49 (0)202 439-2811 42097 Wuppertal @CERN: Bat. 1-1-049 Of course it runs NetBSD http://www.netbsd.org ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
