Re: [lustre-discuss] ?= Changelog users failing to clear records in 2.8, can anyone help

2019-01-25 Thread Colin Faber
There was an issue sometime back that we ran into which involved log clear during failover not always flushing the llogs completely and thus leaving orphaned entries in defunct logs. This manifested itself by reporting no available catalog slots error. However in this case, I would highly recommend

Re: [lustre-discuss] ?= Changelog users failing to clear records in 2.8, can anyone help

2019-01-25 Thread Arman Khalatyan
I am no sure if you hit the same bug as in our case: the llog was not cleared several times and filed the whole mdt space, but the upgrade from 2.8.x to 2.9 resolved the log clear problem. Am Fr., 25. Jan. 2019, 07:16 hat Colin Faber geschrieben: > Have you tried manually purging the changelo

Re: [lustre-discuss] ?= Changelog users failing to clear records in 2.8, can anyone help

2019-01-24 Thread Colin Faber
Have you tried manually purging the changelog files and catalog then restarting by re-registering? Also, are you sure that _all_ consumers are requesting to clear the records? On Mon, Jan 7, 2019 at 11:40 AM nan...@luis.uni-hannover.de < nan...@luis.uni-hannover.de> wrote: > > Any advice here wha

Re: [lustre-discuss] Changelog users failing to clear records in 2.8, can anyone help?

2017-06-04 Thread Faccini, Bruno
out to me? :Andy From: Gibbins, Faye [mailto:faye.gibb...@cirrus.com] Sent: Friday, June 02, 2017 2:44 To: Andy Moe mailto:m...@cray.com>>; lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> Subject: RE: [lustre-discuss] Changelog users failing to clear records in 2.8,

Re: [lustre-discuss] Changelog users failing to clear records in 2.8, can anyone help?

2017-06-02 Thread Andy Moe
me? :Andy From: Gibbins, Faye [mailto:faye.gibb...@cirrus.com] Sent: Friday, June 02, 2017 2:44 To: Andy Moe ; lustre-discuss@lists.lustre.org Subject: RE: [lustre-discuss] Changelog users failing to clear records in 2.8, can anyone help? Hi Andy, Yes indeed! There was parallel reading and cleari

Re: [lustre-discuss] Changelog users failing to clear records in 2.8, can anyone help?

2017-06-02 Thread Gibbins, Faye
...@cray.com] Sent: 01 June 2017 19:34 To: lustre-discuss@lists.lustre.org Cc: IT Software Systems - All Subject: RE: [lustre-discuss] Changelog users failing to clear records in 2.8, can anyone help? Faye, Up to the point when you’ve experienced problems clearing records, had at any point Changelogs

Re: [lustre-discuss] Changelog users failing to clear records in 2.8, can anyone help?

2017-06-01 Thread Andy Moe
- All ; lustre-discuss@lists.lustre.org Subject: Re: [lustre-discuss] Changelog users failing to clear records in 2.8, can anyone help? There seems to have been a few instances of this reported here on the list in the last few months, I don't recall the earlier versions of lustre, but we

Re: [lustre-discuss] Changelog users failing to clear records in 2.8, can anyone help?

2017-06-01 Thread Colin Faber
There seems to have been a few instances of this reported here on the list in the last few months, I don't recall the earlier versions of lustre, but we have also seen this in the wild for customer systems, so very likely a bug which results in corruption of llog files. -cf On Thu, Jun 1, 2017 a

Re: [lustre-discuss] Changelog users failing to clear records in 2.8, can anyone help?

2017-06-01 Thread Dilger, Andreas
On Jun 1, 2017, at 10:55, Faccini, Bruno wrote: > > Hello, > According to the error msgs, looks like there is a corrupted plain-LLOG file > for the ChangeLogs of MDT0. And unfortunately, neither e2fsck nor lfsck can > help to recover in this case. Bruno, is this bug fixed in newer Lustre relea

Re: [lustre-discuss] Changelog users failing to clear records in 2.8, can anyone help?

2017-06-01 Thread Faccini, Bruno
Hello, According to the error msgs, looks like there is a corrupted plain-LLOG file for the ChangeLogs of MDT0. And unfortunately, neither e2fsck nor lfsck can help to recover in this case. I think that to clear this situation you need to stop/umount this MDT and re-mount it as ldiskfs to move b

[lustre-discuss] Changelog users failing to clear records in 2.8, can anyone help?

2017-06-01 Thread Gibbins, Faye
Hi, We have 4 file systems on our lustre cluster. All have changelog users registered for robinhood to use. We have discovered that a changelog user for one of the file systems is not catching up to its index. Manual runs of Robinhood fail to read any more records even though according to mdd/