It sounds like you have a couple of issues that are working against each other then. You’ll probably need to fight one at a time.
My recommendation of clearing up file system space still stands. I don’t have scientific proof, but giving Lustre more space to work with definitely helps. Does your full OST have a lower index than your slow OST? Then you could disable the slow one (and because of the bug everything above it) and let space clear up on the full one. Beyond that you might have to get creative and try something similar to Tommy. Migrate data but manually specify stripe offsets. Shawn On 7/10/15, 11:13 AM, "lustre-discuss on behalf of Kurt Strosahl" <lustre-discuss-boun...@lists.lustre.org on behalf of stros...@jlab.org> wrote: >No, I'm aware of why the ost is getting new writes... it is because I had to >set the qos_threshold_rr to 100 due to >https://jira.hpdd.intel.com/browse/LU-5778 (I have an ost that has to be >ignored due to terrible write performance...) > >w/r, >Kurt > >----- Original Message ----- >From: "Sean Brisbane" <sean.brisb...@physics.ox.ac.uk> >To: "Kurt Strosahl" <stros...@jlab.org> >Cc: "Patrick Farrell" <p...@cray.com>, "lustre-discuss@lists.lustre.org" ><lustre-discuss@lists.lustre.org> >Sent: Friday, July 10, 2015 11:04:27 AM >Subject: RE: [lustre-discuss] lustre 2.5.3 ost not draining > >Dear Kurt, > >Apologies. After leaving it some number of days it did *not* clean itself up, >but I feel that some number of days is long enough to verify that it is a >problem. > >Sounds like you have another issue if the OST is not being marked as full and >writes are not being re-allocated to other OSTS . I also have that second >issue on my system as well and I have only workarounds to offer you for the >problem. > >Thanks, >Sean > >-----Original Message----- >From: Kurt Strosahl [mailto:stros...@jlab.org] >Sent: 10 July 2015 16:01 >To: Sean Brisbane >Cc: Patrick Farrell; lustre-discuss@lists.lustre.org >Subject: Re: [lustre-discuss] lustre 2.5.3 ost not draining > >The problem there is that I cannot afford to leave it "some number of days"... >it is at 97% full, so new writes are going to it faster then it can clean >itself off. > >w/r, >Kurt > >----- Original Message ----- >From: "Sean Brisbane" <sean.brisb...@physics.ox.ac.uk> >To: "Patrick Farrell" <p...@cray.com>, "Kurt Strosahl" <stros...@jlab.org> >Cc: lustre-discuss@lists.lustre.org >Sent: Friday, July 10, 2015 10:44:39 AM >Subject: RE: [lustre-discuss] lustre 2.5.3 ost not draining > >Hi, > >The 'space not freed' issue also happened to me and I left it 'some number of >days' I don't recall how many, it was a while back. > >Cheers, >Sean >_______________________________________________ >lustre-discuss mailing list >lustre-discuss@lists.lustre.org >http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org _______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
