Re: [lustre-discuss] Full OST, any way of avoiding it without hanging?

2019-01-07 Thread Harr, Cameron
In my brief attempts to use lfs migrate, I found performance pretty slow (it's serial). I also got some ASSERTs which should be fixed in 2.10 per LU-8807; note that I was using 2.8. On a more trivial level, I found the -v|--verbose option to the command doesn't work. On 01/07/2019 12:26 PM, Mo

Re: [lustre-discuss] Full OST, any way of avoiding it without hanging?

2019-01-07 Thread Mohr Jr, Richard Frank (Rick Mohr)
> On Jan 7, 2019, at 2:09 PM, Jason Williams wrote: > > One last question, How safe is lfs_migrate? The man page on the installation > says it's UNSAFE for possibly in-use files. The lustre manual doesn't have > the same warning and says something about it being a bit more integrated with

Re: [lustre-discuss] Full OST, any way of avoiding it without hanging?

2019-01-07 Thread Jason Williams
liams Cc: lustre-discuss@lists.lustre.org Subject: Re: [lustre-discuss] Full OST, any way of avoiding it without hanging? > On Jan 7, 2019, at 12:53 PM, Jason Williams wrote: > > As I have gone through the testing, I think you may be right. I think I > disabled the OST in a slightly d

Re: [lustre-discuss] Full OST, any way of avoiding it without hanging?

2019-01-07 Thread Mohr Jr, Richard Frank (Rick Mohr)
> On Jan 7, 2019, at 12:53 PM, Jason Williams wrote: > > As I have gone through the testing, I think you may be right. I think I > disabled the OST in a slightly different way and that caused issues. > > Do you happen to know where I could find out a bit more about what the "lctl > set_para

Re: [lustre-discuss] Full OST, any way of avoiding it without hanging?

2019-01-07 Thread Jason Williams
M To: Jason Williams Cc: lustre-discuss@lists.lustre.org Subject: Re: [lustre-discuss] Full OST, any way of avoiding it without hanging? Jason, The results you described sound like the correct behavior when you deactivate an OST on the MDS. When you run “lctl —device deactivate”, you are

Re: [lustre-discuss] Full OST, any way of avoiding it without hanging?

2019-01-07 Thread Mohr Jr, Richard Frank (Rick Mohr)
nt: Monday, January 7, 2019 11:47:09 AM > To: Mohr Jr, Richard Frank (Rick Mohr) > Cc: lustre-discuss@lists.lustre.org > Subject: Re: [lustre-discuss] Full OST, any way of avoiding it without > hanging? > > So I found this: http://wiki.lustre.org/Handling_Full_OSTs which is what I &

Re: [lustre-discuss] Full OST, any way of avoiding it without hanging?

2019-01-07 Thread Jason Williams
u<mailto:jas...@jhu.edu> From: lustre-discuss on behalf of Jason Williams Sent: Monday, January 7, 2019 11:47:09 AM To: Mohr Jr, Richard Frank (Rick Mohr) Cc: lustre-discuss@lists.lustre.org Subject: Re: [lustre-discuss] Full OST, any way of avoiding it without han

Re: [lustre-discuss] Full OST, any way of avoiding it without hanging?

2019-01-07 Thread Jason Williams
Frank (Rick Mohr) Cc: lustre-discuss@lists.lustre.org Subject: Re: [lustre-discuss] Full OST, any way of avoiding it without hanging? Hi Rick, I thought what I had done was disable it on the MDS, but perhaps I was following the wrong instructions. Do you know where the best instructions for wh

Re: [lustre-discuss] Full OST, any way of avoiding it without hanging?

2019-01-06 Thread Jason Williams
From: Mohr Jr, Richard Frank (Rick Mohr) Sent: Sunday, January 6, 2019 4:56 PM To: Jason Williams Cc: lustre-discuss@lists.lustre.org Subject: Re: [lustre-discuss] Full OST, any way of avoiding it without hanging? > On Jan 5, 2019, at 9:49 PM, Jason Williams wrote: > &g

Re: [lustre-discuss] Full OST, any way of avoiding it without hanging?

2019-01-06 Thread Mohr Jr, Richard Frank (Rick Mohr)
> On Jan 5, 2019, at 9:49 PM, Jason Williams wrote: > > I have looked around the internet and found you can disable an OST, but when > I have tried that, any writes (including deletes) to the OST hang the clients > indefinitely. Does anyone know a way to make an OST basically "read-only" >

Re: [lustre-discuss] Full OST, any way of avoiding it without hanging?

2019-01-05 Thread Jeff Johnson
Jason, If there are files located on the full OST that you can delete then delete them without trying to deactivate the OST. There is a process where you can find files solely located on the full OST using lfs then deactivate the full OST, copy the files to new files with some suffix (.new perhaps

[lustre-discuss] Full OST, any way of avoiding it without hanging?

2019-01-05 Thread Jason Williams
Hello, We have a lustre system (version 2.10.4) that has unfortunately fallen victim to a 100% full OST... Every time we clear some space on it, the system fills it right back up again. I have looked around the internet and found you can disable an OST, but when I have tried that, any writes