Re: [lustre-discuss] Setting infinite grace period with soft quotas
I do see in the lfs setquota usage message and lfs-setquota.1 man page: "The maximum quota grace time is 2^48 - 1 seconds." That's about 9M years, so it should probably be long enough? It might make sense to map "-1" internally to "(1 << 48) - 1" to make this easier. On May 8, 2019, at 17:18, Harr, Cameron wrote: > > I had tested first and couldn't find a way to do so, so I was curious if > there was some undocumented way. I'm proceeding with, "No, there's not a > way." > > On 5/6/19 12:52 PM, Andreas Dilger wrote: >> On Apr 11, 2019, at 11:02, Harr, Cameron wrote: >>> We're exploring an idea where we keep soft quotas enabled so that users >>> will be notified they're nearing their hard quotas (via in-house >>> scripts), but users don't like that the soft quota becomes a hard block >>> after the grace period. I can understand their rationale as well that >>> they should be able to write up to their hard quota always. >>> >>> Is there a way to set the grace period as unlimited (e.g. lfs setquota >>> -t -1 ...)? >> Judging by the lack of response, I don't think anyone has tried this, but >> it also seems like something that could be tested quite easily? >> >> Cheers, Andreas >> -- >> Andreas Dilger >> Principal Lustre Architect >> Whamcloud >> Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Setting infinite grace period with soft quotas
I had tested first and couldn't find a way to do so, so I was curious if there was some undocumented way. I'm proceeding with, "No, there's not a way." On 5/6/19 12:52 PM, Andreas Dilger wrote: > On Apr 11, 2019, at 11:02, Harr, Cameron wrote: >> We're exploring an idea where we keep soft quotas enabled so that users >> will be notified they're nearing their hard quotas (via in-house >> scripts), but users don't like that the soft quota becomes a hard block >> after the grace period. I can understand their rationale as well that >> they should be able to write up to their hard quota always. >> >> Is there a way to set the grace period as unlimited (e.g. lfs setquota >> -t -1 ...)? > Judging by the lack of response, I don't think anyone has tried this, but > it also seems like something that could be tested quite easily? > > Cheers, Andreas > -- > Andreas Dilger > Principal Lustre Architect > Whamcloud > ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
[lustre-discuss] Unable to mount client with 56 MDSes and beyond
We’ve been testing DNE Phase II and tried scaling the number of MDSes(one MDT each for all of our tests) very high, but when we did that, we couldn’t mount the filesystem on a client. After trial and error, we discovered that we were unable to mount the filesystem when there were 56 MDSes. 55 MDSes mounted without issue, and it appears any number below that will mount. This failure at 56 MDSes was replicable across different nodes being used for the MDSes, all of which were tested with working configurations, so it doesn’t seem to be a bad server. Here’s the error info we saw in dmesg on the client: LustreError: 28880:0:(obd_config.c:559:class_setup()) setup lustre-MDT0037-mdc-95923d31b000 failed (-16) LustreError: 28880:0:(obd_config.c:1836:class_config_llog_handler()) MGCx.x.x.x@o2ib: cfg command failed: rc = -16 Lustre:cmd=cf003 0:lustre-MDT0037-mdc 1:lustre-MDT0037_UUID 2:x.x.x.x@o2ib LustreError: 15c-8: MGCx.x.x.x@o2ib: The configuration from log 'lustre-client' failed (-16). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information. LustreError: 28858:0:(obd_config.c:610:class_cleanup()) Device 58 not setup Lustre: Unmounted lustre-client LustreError: 28858:0:(obd_mount.c:1608:lustre_fill_super()) Unable to mount (-16) OS: CentOS 7.6.1810 Kernel: 3.10.0-957.5.1.el7.x86_64 Lustre: 2.12.1 Network card: Qlogic InfiniPath_QLE7340 Other things to note for completeness’ sake: this happened with both ldiskfs and zfs backfstypes, and these tests were using files in memory as the backing devices. Is there something I’m missing as to why more than 56 MDSes won’t mount? Thanks, Scott White Scientist, HPC Los Alamos National Laboratory ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org