Re: another problem with conversations db

2019-02-19 Thread Michael Menge

Replying to myself,

I still see

IOERROR: conversations_audit on store

and

IOERROR: conversations_audit on load



Quoting Michael Menge :


Quoting Bron Gondwana :



ctl_conversationsdb -b should update the cid. BUT - if you're  
running old mailboxes which have a format which doesn't support  
saving the CID, that would for sure break things!




I did run "reconstruct -V max" during the upgrade. So all mailboxes  
should have been updated but

checking with

mbexamine user/* | grep "  Minor Version: 12"

I found some mailboxes that where not updated correctly. I didn't  
see any errors during the
initial upgrade, but I don't have the logs anymore so i don't know  
what happened.

Any idea what could lead to "reconstruct -V max" failing for some mailboxes?

I did rerun "reconstruct -r -V max user/LoginID" for those users,  
but I had two accounts where
the reconstruct command didn't update the index for the INBOX of  
that account. Rerunning it

again did update it.

The rebuild of the conversation_db worked fine after the Index was updated..
Do this did resolve the endless loop in ctl_conversationsdb. I would suggest
to add a check for the index version in ctl_conversationsdb

I will monitor my log files if the IOERRORS will reappear, or if the  
wrong index version

was the root of that problem too.

@Bron Thanks for the help





M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: another problem with conversations db

2019-02-18 Thread Michael Menge


Quoting Bron Gondwana :



ctl_conversationsdb -b should update the cid. BUT - if you're  
running old mailboxes which have a format which doesn't support  
saving the CID, that would for sure break things!




I did run "reconstruct -V max" during the upgrade. So all mailboxes  
should have been updated but

checking with

mbexamine user/* | grep "  Minor Version: 12"

I found some mailboxes that where not updated correctly. I didn't see  
any errors during the
initial upgrade, but I don't have the logs anymore so i don't know  
what happened.

Any idea what could lead to "reconstruct -V max" failing for some mailboxes?

I did rerun "reconstruct -r -V max user/LoginID" for those users, but  
I had two accounts where
the reconstruct command didn't update the index for the INBOX of that  
account. Rerunning it

again did update it.

The rebuild of the conversation_db worked fine after the Index was updated..
Do this did resolve the endless loop in ctl_conversationsdb. I would suggest
to add a check for the index version in ctl_conversationsdb

I will monitor my log files if the IOERRORS will reappear, or if the  
wrong index version

was the root of that problem too.

@Bron Thanks for the help




M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: another problem with conversations db

2019-02-13 Thread Bron Gondwana
On Wed, Feb 13, 2019, at 00:39, Michael Menge wrote:
> Hi Bron,
> 
> sorry, i had to rearrange some quotes to put them my answers in a more 
> meaningful order.
> 
> 
> Quoting Bron Gondwana :
> 
> >> >> The file was already at 
> >> /srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.
> 
> I was able to fix these problems with reconstruct, and the didn't 
> reappear till now.
> Also there where other accounts which had IOERRORS regarding the 
> conversation db,
> with no cyr_expire archive errors, so i believe that these problems 
> are not related.
> 
> I tried rebuilding the conversation db for the accounts with errors, 
> but some other
> accounts will show up with errors some time later. I counldn't find a 
> some thing in
> common jet.

OK. That's hard to disagnore from remotely :(

> 
> >> >> > Anyway, I don't think that would break anything.
> >> >> >
> >> >> > metapartition-ssd: /srv/cyrus-ssd-be/meta/ssd-part
> >> >> > metapartition_files: header index cache expunge squat annotations
> >> >> > lock dav archivecache
> >> >> >
> >> >> > Ooh, I haven't tested having cache and archivecache on the same
> >> >> > location. That's really interesting. Again, I'd be in favour of
> >> >> > separation here, give them different paths. That might be tricky
> >> >> > with ssd though, the way this is laid out. I assume you have some
> >> >> > kind of symlink farm going on?
> >> >> >
> >> >>
> >> >> I didn't know that there could be a problem with cache and archivecache.
> >> >> At the time we decided on the configuration for cyrus 3.0 I looked at 
> >> >> the
> >> >> imapd.conf man page and for metapartition_files decided that I want all
> >> >> meta files on the ssd storage. There was no indication in the man page
> >> >> that there could be a problem.
> >> >
> >> > Fair. I'd have to test that to see if it works correctly. I would
> >> > hope so, but I haven't tested that configuration. This is the
> >> > downside with having lots of different ways to do things!
> >> >
> >> >> How do I separate location of archivecache from the other
> >> >> metapartition path?
> >> >> And fix the cache and archivecache files?
> >> >
> >> > This I don't know a good answer for. I will test if having the same
> >> > path for cache and archivecache could fail. I THINK that I made the
> >> > code safe for it, but I'm not sure that it's been tested.
> >> >
> >> >> No there is no sysmlink farm. We have mounted different iSCSI volumes to
> >> >> /srv/cyrus-ssd-be, /srv/cyrus-hdd-be and /srv/cyrus-be
> >> >
> >> > Right. That makes sense.
> 
> Did you have time to look into the cache/archivecache situation jet?

Yes, I've looked at the code!

in mailbox_archive():

 /* got a new cache record to write */
 if (differentcache)
 {
 dirtycache = 1;
 copyrecord.cache_offset = 0;
 if (mailbox_append_cache(mailbox, ))
 continue;
 }

And the code for differentcache is:

 char *spoolcache = xstrdup(mailbox_meta_fname(mailbox, META_CACHE));
 char *archivecache = xstrdup(mailbox_meta_fname(mailbox, META_ARCHIVECACHE));
 int differentcache = strcmp(spoolcache, archivecache);

So it looks like the answer is cache/archivecache is fine. It is not your 
problem.

> 
> >> > Right! I do wonder if there are some bugs in 3.0.x which are fixed
> >> > on master around delivery to archive partition. We definitely had
> >> > bugs on master, but I thought they were newly introduced on master
> >> > as well, which is why the fixes weren't backported. But if you're
> >> > having files be in the wrong location, maybe there are bugs on 3.0.x
> >> > as well.
> 
> Are all fixes from master backported to 3.0?

Unfortunately it's hard to tell, because many of the fixes on master are fixes 
to bugs that were only introduced on master, and some bugs on 3.0 we just say 
"the fix is so invasive that it's basically just backporting 90% of master, 
which is pointless for a stable release".

> Is the new Commit "I will try your new commits regarding CID" related to the
> "IOERROR: conversations_audit on load:" and "IOERROR: 
> conversations_audit on store"?

Shouldn't be. It just means we store the G keys regardless of whether the 
record has a CID.

> I will try your new commits in the next days on my test servers to sea 
> if the fix
> the endless loop in ctl_conversationsdb I have seen for some accounts.

I guess one more question - are you running the most recent index version? 
(reconstruct -V max)

> Quoting myself (Re: prblems rebuilding conversations db) Jan 24, 2019
> 
> > The program loops in build_cid_cb (imap/ctl_conversationsdb.c:189)
> >
> > For the problematic mailbox that I tested, for every message
> > record->cid was NULLCONVERSATION, so mailbox_cacherecord,
> > message_update_conversations and mailbox_rewrite_index_record
> > where called, each returned 0.
> >
> > After iterating trough all messages in the mailbox count was > 0, and r==0,
> > so the while condition (!r && count) was true for the next run.
> > At this point record->cid was still 

Re: another problem with conversations db

2019-02-12 Thread Michael Menge

Hi Bron,

sorry, i had to rearrange some quotes to put them my answers in a more  
meaningful order.



Quoting Bron Gondwana :


On Mon, Feb 4, 2019, at 22:00, Michael Menge wrote:


Quoting Bron Gondwana :

> On Mon, Feb 4, 2019, at 20:21, Michael Menge wrote:
>>



>> Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening
>> /srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
>> Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening
>> /srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
>> Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR archive
>> user. 2185 failed to copyfile
>> (/srv/cyrus-be/ssd-part/L/user//2185. =>
>> /srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.): Unknown code
>>  255
>
>
> Ouch. Yeah, that could have been caused by a bug in delivery, and
> would definitely cause conversations DB corruption if the index file
> was updated but the conversations DB wasn't or vice versa.
>
>> The file was already at  
/srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.


I was able to fix these problems with reconstruct, and the didn't  
reappear till now.
Also there where other accounts which had IOERRORS regarding the  
conversation db,
with no cyr_expire archive errors, so i believe that these problems  
are not related.


I tried rebuilding the conversation db for the accounts with errors,  
but some other
accounts will show up with errors some time later. I counldn't find a  
some thing in

common jet.



>> > Anyway, I don't think that would break anything.
>> >
>> > metapartition-ssd: /srv/cyrus-ssd-be/meta/ssd-part
>> > metapartition_files: header index cache expunge squat annotations
>> > lock dav archivecache
>> >
>> > Ooh, I haven't tested having cache and archivecache on the same
>> > location. That's really interesting. Again, I'd be in favour of
>> > separation here, give them different paths. That might be tricky
>> > with ssd though, the way this is laid out. I assume you have some
>> > kind of symlink farm going on?
>> >
>>
>> I didn't know that there could be a problem with cache and archivecache.
>> At the time we decided on the configuration for cyrus 3.0 I looked at the
>> imapd.conf man page and for metapartition_files decided that I want all
>> meta files on the ssd storage. There was no indication in the man page
>> that there could be a problem.
>
> Fair. I'd have to test that to see if it works correctly. I would
> hope so, but I haven't tested that configuration. This is the
> downside with having lots of different ways to do things!
>
>> How do I separate location of archivecache from the other
>> metapartition path?
>> And fix the cache and archivecache files?
>
> This I don't know a good answer for. I will test if having the same
> path for cache and archivecache could fail. I THINK that I made the
> code safe for it, but I'm not sure that it's been tested.
>
>> No there is no sysmlink farm. We have mounted different iSCSI volumes to
>> /srv/cyrus-ssd-be, /srv/cyrus-hdd-be and /srv/cyrus-be
>
> Right. That makes sense.


Did you have time to look into the cache/archivecache situation jet?



> Right! I do wonder if there are some bugs in 3.0.x which are fixed
> on master around delivery to archive partition. We definitely had
> bugs on master, but I thought they were newly introduced on master
> as well, which is why the fixes weren't backported. But if you're
> having files be in the wrong location, maybe there are bugs on 3.0.x
> as well.


Are all fixes from master backported to 3.0?

Is the new Commit "I will try your new commits regarding CID" related to the
"IOERROR: conversations_audit on load:" and "IOERROR:  
conversations_audit on store"?


I will try your new commits in the next days on my test servers to sea  
if the fix

the endless loop in ctl_conversationsdb I have seen for some accounts.

Quoting myself (Re: prblems rebuilding conversations db) Jan 24, 2019


The program loops in build_cid_cb (imap/ctl_conversationsdb.c:189)

For the problematic mailbox that I tested, for every message
record->cid was NULLCONVERSATION, so mailbox_cacherecord,
message_update_conversations and mailbox_rewrite_index_record
where called, each returned 0.

After iterating trough all messages in the mailbox count was > 0, and r==0,
so the while condition (!r && count) was true for the next run.
At this point record->cid was still NULLCONVERSATION for every message,
which I guess should not be the case.


Michael


M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:

Re: another problem with conversations db

2019-02-04 Thread Bron Gondwana
On Mon, Feb 4, 2019, at 22:00, Michael Menge wrote:
> 
> Quoting Bron Gondwana :
> 
> > On Mon, Feb 4, 2019, at 20:21, Michael Menge wrote:
> >> Hi,
> >>
> >> Quoting Bron Gondwana :
> >>
> >> > Hi Michael,
> >> >
> >> > Sorry about the delay in looking at this - I was mad crazy busy
> >> > getting ready to go overseas. At Fosdem now, about to give a talk
> >> > about JMAP!
> >> >
> >> > OK, let's start with the things that give me a little bit of hives...
> >> >
> >> > configdirectory: /srv/cyrus-be
> >> > partition-default: /srv/cyrus-be
> >> > partition-ssd: /srv/cyrus-be/ssd-part
> >> >
> >> > Ouch. There's a couple of things I wouldn't do there - having the
> >> > partition be the same as the config directory, and having a separate
> >> > partition be a subdirectory of a partition. They're both asking for
> >> > trouble. I would probably lay my system out like:
> >> >
> >> > configdirectory: /srv/cyrus-be/conf
> >> > partition-default: /srv/cyrus-be/default-part
> >> > partition-ssd: /srv/cyrus-be/ssd-part
> >> >
> >>
> >> partition-default isn't used any more. To use the metapartition we moved
> >> all accounts form the default partition to the ssd partition which is the
> >> the new defaultpartition ("defaultpartition: ssd")
> >
> > Right - that makes sense.
> >
> >> > And then each tree would only have one type of thing in it.
> >> >
> >> > Anyway, I don't think that would break anything.
> >> >
> >> > metapartition-ssd: /srv/cyrus-ssd-be/meta/ssd-part
> >> > metapartition_files: header index cache expunge squat annotations
> >> > lock dav archivecache
> >> >
> >> > Ooh, I haven't tested having cache and archivecache on the same
> >> > location. That's really interesting. Again, I'd be in favour of
> >> > separation here, give them different paths. That might be tricky
> >> > with ssd though, the way this is laid out. I assume you have some
> >> > kind of symlink farm going on?
> >> >
> >>
> >> I didn't know that there could be a problem with cache and archivecache.
> >> At the time we decided on the configuration for cyrus 3.0 I looked at the
> >> imapd.conf man page and for metapartition_files decided that I want all
> >> meta files on the ssd storage. There was no indication in the man page
> >> that there could be a problem.
> >
> > Fair. I'd have to test that to see if it works correctly. I would 
> > hope so, but I haven't tested that configuration. This is the 
> > downside with having lots of different ways to do things!
> >
> >> How do I separate location of archivecache from the other 
> >> metapartition path?
> >> And fix the cache and archivecache files?
> >
> > This I don't know a good answer for. I will test if having the same 
> > path for cache and archivecache could fail. I THINK that I made the 
> > code safe for it, but I'm not sure that it's been tested.
> >
> >> No there is no sysmlink farm. We have mounted different iSCSI volumes to
> >> /srv/cyrus-ssd-be, /srv/cyrus-hdd-be and /srv/cyrus-be
> >
> > Right. That makes sense.
> >
> >> > Otherwise it all looks OK. Are you getting other IOERRORs in your
> >> > log files which could show things aborting? It really looks like
> >> > your conversations DB is getting out of sync due to other failures.
> >>
> >> I found a few instances of 3 related errors.
> >>
> >> Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening
> >> /srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
> >> Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening
> >> /srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
> >> Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR archive
> >> user. 2185 failed to copyfile
> >> (/srv/cyrus-be/ssd-part/L/user//2185. =>
> >> /srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.): Unknown code
> >>  255
> >
> >
> > Ouch. Yeah, that could have been caused by a bug in delivery, and 
> > would definitely cause conversations DB corruption if the index file 
> > was updated but the conversations DB wasn't or vice versa.
> >
> >> The file was already at 
> >> /srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.
> >
> > Right! I do wonder if there are some bugs in 3.0.x which are fixed 
> > on master around delivery to archive partition. We definitely had 
> > bugs on master, but I thought they were newly introduced on master 
> > as well, which is why the fixes weren't backported. But if you're 
> > having files be in the wrong location, maybe there are bugs on 3.0.x 
> > as well.
> >
> > Do you have the syslog lines at the time that email was delivered?
> 
> I dont' have the log, for that message, but I will search for a
> more recent example.

Great.

> 
> From the mail headers i know that it was not dilivered to the archive 
> partition
> but moved by cyr_expire. The conversation db was not used at that time.

OK - that shouldn't matter then - because the conversations rebuild should have 
found it.

> PS.: the timesamp of the file is not the internal date but 

Re: another problem with conversations db

2019-02-04 Thread Michael Menge


Quoting Bron Gondwana :


On Mon, Feb 4, 2019, at 20:21, Michael Menge wrote:

Hi,

Quoting Bron Gondwana :

> Hi Michael,
>
> Sorry about the delay in looking at this - I was mad crazy busy
> getting ready to go overseas. At Fosdem now, about to give a talk
> about JMAP!
>
> OK, let's start with the things that give me a little bit of hives...
>
> configdirectory: /srv/cyrus-be
> partition-default: /srv/cyrus-be
> partition-ssd: /srv/cyrus-be/ssd-part
>
> Ouch. There's a couple of things I wouldn't do there - having the
> partition be the same as the config directory, and having a separate
> partition be a subdirectory of a partition. They're both asking for
> trouble. I would probably lay my system out like:
>
> configdirectory: /srv/cyrus-be/conf
> partition-default: /srv/cyrus-be/default-part
> partition-ssd: /srv/cyrus-be/ssd-part
>

partition-default isn't used any more. To use the metapartition we moved
all accounts form the default partition to the ssd partition which is the
the new defaultpartition ("defaultpartition: ssd")


Right - that makes sense.


> And then each tree would only have one type of thing in it.
>
> Anyway, I don't think that would break anything.
>
> metapartition-ssd: /srv/cyrus-ssd-be/meta/ssd-part
> metapartition_files: header index cache expunge squat annotations
> lock dav archivecache
>
> Ooh, I haven't tested having cache and archivecache on the same
> location. That's really interesting. Again, I'd be in favour of
> separation here, give them different paths. That might be tricky
> with ssd though, the way this is laid out. I assume you have some
> kind of symlink farm going on?
>

I didn't know that there could be a problem with cache and archivecache.
At the time we decided on the configuration for cyrus 3.0 I looked at the
imapd.conf man page and for metapartition_files decided that I want all
meta files on the ssd storage. There was no indication in the man page
that there could be a problem.


Fair. I'd have to test that to see if it works correctly. I would  
hope so, but I haven't tested that configuration. This is the  
downside with having lots of different ways to do things!


How do I separate location of archivecache from the other  
metapartition path?

And fix the cache and archivecache files?


This I don't know a good answer for. I will test if having the same  
path for cache and archivecache could fail. I THINK that I made the  
code safe for it, but I'm not sure that it's been tested.



No there is no sysmlink farm. We have mounted different iSCSI volumes to
/srv/cyrus-ssd-be, /srv/cyrus-hdd-be and /srv/cyrus-be


Right. That makes sense.


> Otherwise it all looks OK. Are you getting other IOERRORs in your
> log files which could show things aborting? It really looks like
> your conversations DB is getting out of sync due to other failures.

I found a few instances of 3 related errors.

Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening
/srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening
/srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR archive
user. 2185 failed to copyfile
(/srv/cyrus-be/ssd-part/L/user//2185. =>
/srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.): Unknown code
 255



Ouch. Yeah, that could have been caused by a bug in delivery, and  
would definitely cause conversations DB corruption if the index file  
was updated but the conversations DB wasn't or vice versa.



The file was already at /srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.


Right! I do wonder if there are some bugs in 3.0.x which are fixed  
on master around delivery to archive partition. We definitely had  
bugs on master, but I thought they were newly introduced on master  
as well, which is why the fixes weren't backported. But if you're  
having files be in the wrong location, maybe there are bugs on 3.0.x  
as well.


Do you have the syslog lines at the time that email was delivered?


I dont' have the log, for that message, but I will search for a
more recent example.


From the mail headers i know that it was not dilivered to the archive  
partition

but moved by cyr_expire. The conversation db was not used at that time.

PS.: the timesamp of the file is not the internal date but the time
the mail was moved to the archive partition. I was wondering about the reason.

Michael



M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:

Re: another problem with conversations db

2019-02-04 Thread Bron Gondwana
On Mon, Feb 4, 2019, at 20:21, Michael Menge wrote:
> Hi,
> 
> Quoting Bron Gondwana :
> 
> > Hi Michael,
> >
> > Sorry about the delay in looking at this - I was mad crazy busy 
> > getting ready to go overseas. At Fosdem now, about to give a talk 
> > about JMAP!
> >
> > OK, let's start with the things that give me a little bit of hives...
> >
> > configdirectory: /srv/cyrus-be
> > partition-default: /srv/cyrus-be
> > partition-ssd: /srv/cyrus-be/ssd-part
> >
> > Ouch. There's a couple of things I wouldn't do there - having the 
> > partition be the same as the config directory, and having a separate 
> > partition be a subdirectory of a partition. They're both asking for 
> > trouble. I would probably lay my system out like:
> >
> > configdirectory: /srv/cyrus-be/conf
> > partition-default: /srv/cyrus-be/default-part
> > partition-ssd: /srv/cyrus-be/ssd-part
> >
> 
> partition-default isn't used any more. To use the metapartition we moved
> all accounts form the default partition to the ssd partition which is the
> the new defaultpartition ("defaultpartition: ssd")

Right - that makes sense.

> > And then each tree would only have one type of thing in it.
> >
> > Anyway, I don't think that would break anything.
> >
> > metapartition-ssd: /srv/cyrus-ssd-be/meta/ssd-part
> > metapartition_files: header index cache expunge squat annotations 
> > lock dav archivecache
> >
> > Ooh, I haven't tested having cache and archivecache on the same 
> > location. That's really interesting. Again, I'd be in favour of 
> > separation here, give them different paths. That might be tricky 
> > with ssd though, the way this is laid out. I assume you have some 
> > kind of symlink farm going on?
> >
> 
> I didn't know that there could be a problem with cache and archivecache.
> At the time we decided on the configuration for cyrus 3.0 I looked at the
> imapd.conf man page and for metapartition_files decided that I want all
> meta files on the ssd storage. There was no indication in the man page
> that there could be a problem.

Fair. I'd have to test that to see if it works correctly. I would hope so, but 
I haven't tested that configuration. This is the downside with having lots of 
different ways to do things!

> How do I separate location of archivecache from the other metapartition path?
> And fix the cache and archivecache files?

This I don't know a good answer for. I will test if having the same path for 
cache and archivecache could fail. I THINK that I made the code safe for it, 
but I'm not sure that it's been tested.

> No there is no sysmlink farm. We have mounted different iSCSI volumes to
> /srv/cyrus-ssd-be, /srv/cyrus-hdd-be and /srv/cyrus-be

Right. That makes sense.

> > Otherwise it all looks OK. Are you getting other IOERRORs in your 
> > log files which could show things aborting? It really looks like 
> > your conversations DB is getting out of sync due to other failures.
> 
> I found a few instances of 3 related errors.
> 
> Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening 
> /srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
> Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening 
> /srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
> Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR archive 
> user. 2185 failed to copyfile 
> (/srv/cyrus-be/ssd-part/L/user//2185. => 
> /srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.): Unknown code 
>  255


Ouch. Yeah, that could have been caused by a bug in delivery, and would 
definitely cause conversations DB corruption if the index file was updated but 
the conversations DB wasn't or vice versa.

> The file was already at /srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.

Right! I do wonder if there are some bugs in 3.0.x which are fixed on master 
around delivery to archive partition. We definitely had bugs on master, but I 
thought they were newly introduced on master as well, which is why the fixes 
weren't backported. But if you're having files be in the wrong location, maybe 
there are bugs on 3.0.x as well.

Do you have the syslog lines at the time that email was delivered?

Bron.
-- 
 Bron Gondwana
 br...@fastmail.fm


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: another problem with conversations db

2019-02-04 Thread Michael Menge

Hi,

Quoting Bron Gondwana :


Hi Michael,

Sorry about the delay in looking at this - I was mad crazy busy  
getting ready to go overseas. At Fosdem now, about to give a talk  
about JMAP!


OK, let's start with the things that give me a little bit of hives...

configdirectory: /srv/cyrus-be
partition-default: /srv/cyrus-be
partition-ssd: /srv/cyrus-be/ssd-part

Ouch. There's a couple of things I wouldn't do there - having the  
partition be the same as the config directory, and having a separate  
partition be a subdirectory of a partition. They're both asking for  
trouble. I would probably lay my system out like:


configdirectory: /srv/cyrus-be/conf
partition-default: /srv/cyrus-be/default-part
partition-ssd: /srv/cyrus-be/ssd-part



partition-default isn't used any more. To use the metapartition we moved
all accounts form the default partition to the ssd partition which is the
the new defaultpartition ("defaultpartition: ssd")


And then each tree would only have one type of thing in it.

Anyway, I don't think that would break anything.

metapartition-ssd: /srv/cyrus-ssd-be/meta/ssd-part
metapartition_files: header index cache expunge squat annotations  
lock dav archivecache


Ooh, I haven't tested having cache and archivecache on the same  
location. That's really interesting. Again, I'd be in favour of  
separation here, give them different paths. That might be tricky  
with ssd though, the way this is laid out. I assume you have some  
kind of symlink farm going on?




I didn't know that there could be a problem with cache and archivecache.
At the time we decided on the configuration for cyrus 3.0 I looked at the
imapd.conf man page and for metapartition_files decided that I want all
meta files on the ssd storage. There was no indication in the man page
that there could be a problem.

How do I separate location of archivecache from the other metapartition path?
And fix the cache and archivecache files?

No there is no sysmlink farm.  We have mounted different iSCSI volumes to
/srv/cyrus-ssd-be, /srv/cyrus-hdd-be and /srv/cyrus-be


Otherwise it all looks OK. Are you getting other IOERRORs in your  
log files which could show things aborting? It really looks like  
your conversations DB is getting out of sync due to other failures.


I found a few instances of 3 related errors.

Feb  4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening  
/srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
Feb  4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening  
/srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
Feb  4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR archive  
user. 2185 failed to copyfile  
(/srv/cyrus-be/ssd-part/L/user//2185. =>  
/srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.): Unknown code  
 255


The file was already at /srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.

stat 2185.
  File: '2185.'
  Size: 424542  Blocks: 832IO Block: 4096   regular file
Device: 860h/2144d  Inode: 4677438462  Links: 1
Access: (0600/-rw---)  Uid: (   96/   cyrus)   Gid: (   12/mail)
Context: system_u:object_r:unlabeled_t:s0
Access: 2018-11-27 01:12:46.585864239 +0100
Modify: 2018-11-27 01:12:46.585864239 +0100
Change: 2018-11-27 01:12:46.585864239 +0100
 Birth: -




Cheers,

Bron.



On Thu, Jan 31, 2019, at 19:36, Michael Menge wrote:


Quoting Bron Gondwana :

> On Fri, Jan 25, 2019, at 20:00, Michael Menge wrote:
>> Hi Bron
>>
>>
>> Quoting Bron Gondwana :
>>
>> > Sorry I haven't been following along with this earlier. Can you post
>> > your imapd.conf and cyrus.conf as well as let her know if you run
>> > anything which directly messes with files on disk.
>>
>> Thanks for looking into it. I have attached the backend and replica
>> configs. There should be nothing messing with the files on disk
>> directly. Some times we need to restore mails from file based
>> backup (bacula) followed by a reconstruct but this was not the
>> case this time.
>
> Do you always rebuild the conversations db after doing the
> reconstruct? That will be necessary now. We switched to doing all
> our restores using IMAP append a while back so we're never fiddling
> the file system under Cyrus.
>
>> >
>> > Also, what operating system is this on and what Cyrus version?
>> >
>>
>> We are running a cyrus 3.0.8 compiled with the following Options
>> (./configure --enable-murder --enable-http --enable-calalarmd
>> --enable-replication --enable-backup --enable-idled
>> --enable-autocreate CFLAGS="-fPIC -g")
>> in a murder configuration on a RHEL 7.5 System. As filesystem we use xfs.
>
> They should be fine. I'll have a read of the config when I'm at a
> real computer.
>

did you find anything in the config that would explain the problems?

>>
>> > Bron.
>> >
>> > On Fri, Jan 25, 2019, at 04:08, Michael Menge wrote:
>> >> Hi,
>> >>
>> >> I have discovered an other problem with the conversations db:
>> >>
>> >> Thousends of lines with 

Re: another problem with conversations db

2019-02-02 Thread Bron Gondwana
Hi Michael,

Sorry about the delay in looking at this - I was mad crazy busy getting ready 
to go overseas. At Fosdem now, about to give a talk about JMAP!

OK, let's start with the things that give me a little bit of hives...

configdirectory: /srv/cyrus-be
partition-default: /srv/cyrus-be
partition-ssd: /srv/cyrus-be/ssd-part

Ouch. There's a couple of things I wouldn't do there - having the partition be 
the same as the config directory, and having a separate partition be a 
subdirectory of a partition. They're both asking for trouble. I would probably 
lay my system out like:

configdirectory: /srv/cyrus-be/conf
partition-default: /srv/cyrus-be/default-part
partition-ssd: /srv/cyrus-be/ssd-part

And then each tree would only have one type of thing in it.

Anyway, I don't think that would break anything.

metapartition-ssd: /srv/cyrus-ssd-be/meta/ssd-part
metapartition_files: header index cache expunge squat annotations lock dav 
archivecache

Ooh, I haven't tested having cache and archivecache on the same location. 
That's really interesting. Again, I'd be in favour of separation here, give 
them different paths. That might be tricky with ssd though, the way this is 
laid out. I assume you have some kind of symlink farm going on?

Otherwise it all looks OK. Are you getting other IOERRORs in your log files 
which could show things aborting? It really looks like your conversations DB is 
getting out of sync due to other failures.

Cheers,

Bron.



On Thu, Jan 31, 2019, at 19:36, Michael Menge wrote:
> 
> Quoting Bron Gondwana :
> 
> > On Fri, Jan 25, 2019, at 20:00, Michael Menge wrote:
> >> Hi Bron
> >>
> >>
> >> Quoting Bron Gondwana :
> >>
> >> > Sorry I haven't been following along with this earlier. Can you post
> >> > your imapd.conf and cyrus.conf as well as let her know if you run
> >> > anything which directly messes with files on disk.
> >>
> >> Thanks for looking into it. I have attached the backend and replica
> >> configs. There should be nothing messing with the files on disk
> >> directly. Some times we need to restore mails from file based
> >> backup (bacula) followed by a reconstruct but this was not the
> >> case this time.
> >
> > Do you always rebuild the conversations db after doing the 
> > reconstruct? That will be necessary now. We switched to doing all 
> > our restores using IMAP append a while back so we're never fiddling 
> > the file system under Cyrus.
> >
> >> >
> >> > Also, what operating system is this on and what Cyrus version?
> >> >
> >>
> >> We are running a cyrus 3.0.8 compiled with the following Options
> >> (./configure --enable-murder --enable-http --enable-calalarmd
> >> --enable-replication --enable-backup --enable-idled
> >> --enable-autocreate CFLAGS="-fPIC -g")
> >> in a murder configuration on a RHEL 7.5 System. As filesystem we use xfs.
> >
> > They should be fine. I'll have a read of the config when I'm at a 
> > real computer.
> >
> 
> did you find anything in the config that would explain the problems?
> 
> >>
> >> > Bron.
> >> >
> >> > On Fri, Jan 25, 2019, at 04:08, Michael Menge wrote:
> >> >> Hi,
> >> >>
> >> >> I have discovered an other problem with the conversations db:
> >> >>
> >> >> Thousends of lines with "IOERROR: conversations_audit on load:" and
> >> >> "IOERROR: conversations_audit on store:"
> >> >> A look at the source code shows that these errors are logged after
> >> >> "_sanity_check_counts" is called.
> >> >> The log level LOG_ERR and the prefix IOERROR indicate that I have a
> >> >> serious problem. Do I?
> >> >>
> >> >> This problem occurred for accounts where the rebuild of the
> >> >> conversations db was successful.
> >> >>
> >> >> I don't want to rebuild the conversations db every few days.
> >> >>
> >> >> Any help is appreciated.
> >> >>
> >> >> Kind regards
> >> >>
> >> >>
> >> >> Michael Menge
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> 
> >> 
> >> >> M.Menge Tel.: (49) 7071/29-70316
> >> >> Universität Tübingen Fax.: (49) 7071/29-5912
> >> >> Zentrum für Datenverarbeitung mail:
> >> >> michael.me...@zdv.uni-tuebingen.de
> >> >> Wächterstraße 76
> >> >> 72074 Tübingen
> >> >>
> >> >> 
> >> >> Cyrus Home Page: http://www.cyrusimap.org/
> >> >> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> >> >> To Unsubscribe:
> >> >> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> >> >
> >> > --
> >> > Bron Gondwana
> >> > br...@fastmail.fm
> >>
> >>
> >>
> >> 
> >> M.Menge Tel.: (49) 7071/29-70316
> >> Universität Tübingen Fax.: (49) 7071/29-5912
> >> Zentrum für Datenverarbeitung mail:
> >> michael.me...@zdv.uni-tuebingen.de
> >> Wächterstraße 76
> >> 72074 Tübingen
> >>
> >> 
> >> Cyrus Home Page: http://www.cyrusimap.org/
> >> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> >> To Unsubscribe:
> >> 

Re: another problem with conversations db

2019-01-31 Thread Michael Menge


Quoting Bron Gondwana :


On Fri, Jan 25, 2019, at 20:00, Michael Menge wrote:

Hi Bron


Quoting Bron Gondwana :

> Sorry I haven't been following along with this earlier. Can you post
> your imapd.conf and cyrus.conf as well as let her know if you run
> anything which directly messes with files on disk.

Thanks for looking into it. I have attached the backend and replica
configs. There should be nothing messing with the files on disk
directly. Some times we need to restore mails from file based
backup (bacula) followed by a reconstruct but this was not the
case this time.


Do you always rebuild the conversations db after doing the  
reconstruct? That will be necessary now. We switched to doing all  
our restores using IMAP append a while back so we're never fiddling  
the file system under Cyrus.



>
> Also, what operating system is this on and what Cyrus version?
>

We are running a cyrus 3.0.8 compiled with the following Options
(./configure --enable-murder --enable-http --enable-calalarmd
--enable-replication --enable-backup --enable-idled
--enable-autocreate CFLAGS="-fPIC -g")
in a murder configuration on a RHEL 7.5 System. As filesystem we use xfs.


They should be fine. I'll have a read of the config when I'm at a  
real computer.




did you find anything in the config that would explain the problems?



> Bron.
>
> On Fri, Jan 25, 2019, at 04:08, Michael Menge wrote:
>> Hi,
>>
>> I have discovered an other problem with the conversations db:
>>
>> Thousends of lines with "IOERROR: conversations_audit on load:" and
>> "IOERROR: conversations_audit on store:"
>> A look at the source code shows that these errors are logged after
>> "_sanity_check_counts" is called.
>> The log level LOG_ERR and the prefix IOERROR indicate that I have a
>> serious problem. Do I?
>>
>> This problem occurred for accounts where the rebuild of the
>> conversations db was successful.
>>
>> I don't want to rebuild the conversations db every few days.
>>
>> Any help is appreciated.
>>
>> Kind regards
>>
>>
>> Michael Menge
>>
>>
>>
>>
>>
>>  


>> M.Menge Tel.: (49) 7071/29-70316
>> Universität Tübingen Fax.: (49) 7071/29-5912
>> Zentrum für Datenverarbeitung mail:
>> michael.me...@zdv.uni-tuebingen.de
>> Wächterstraße 76
>> 72074 Tübingen
>>
>> 
>> Cyrus Home Page: http://www.cyrusimap.org/
>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>> To Unsubscribe:
>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>
> --
> Bron Gondwana
> br...@fastmail.fm




M.Menge Tel.: (49) 7071/29-70316
Universität Tübingen Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung mail:
michael.me...@zdv.uni-tuebingen.de
Wächterstraße 76
72074 Tübingen


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

*Attachments:*
 * imapd_be.conf
 * cyrus_be.conf
 * cyrus_re.conf
 * imapd_re.conf


--
 Bron Gondwana
 br...@fastmail.fm





M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: another problem with conversations db

2019-01-25 Thread Bron Gondwana
Yes, updating reconstruct is on our roadmap. The biggest problem is that there 
are multiple file types across mailboxes and conversations and the changes 
can't be atomic, so if reconstruct discovers something partially done, it can't 
tell if the conversations were correct.

The longer term grand plan is to reduce the number of db types and make 
conversations updates able to be atomic.

Cheers,

Bron.

On Fri, Jan 25, 2019, at 22:30, Michael Menge wrote:
> Hi,
> 
> 
> Quoting Bron Gondwana :
> 
> > On Fri, Jan 25, 2019, at 20:00, Michael Menge wrote:
> >> Hi Bron
> >>
> >>
> >> Quoting Bron Gondwana :
> >>
> >> > Sorry I haven't been following along with this earlier. Can you post
> >> > your imapd.conf and cyrus.conf as well as let her know if you run
> >> > anything which directly messes with files on disk.
> >>
> >> Thanks for looking into it. I have attached the backend and replica
> >> configs. There should be nothing messing with the files on disk
> >> directly. Some times we need to restore mails from file based
> >> backup (bacula) followed by a reconstruct but this was not the
> >> case this time.
> >
> > Do you always rebuild the conversations db after doing the 
> > reconstruct? That will be necessary now. We switched to doing all 
> > our restores using IMAP append a while back so we're never fiddling 
> > the file system under Cyrus.
> >
> 
> We didn't have to restore any mails from backup since we enabled 
> conversation db
> a few weeks ago. It is good to know that the rebuild is necessary, but 
> shouldn't
> reconstruct also update the conversation db if it re-appends the message?
> At least a hint should be put in the reconstruct man page, like the 
> one for "quota -f"
> 
> 
> >> >
> >> > Also, what operating system is this on and what Cyrus version?
> >> >
> >>
> >> We are running a cyrus 3.0.8 compiled with the following Options
> >> (./configure --enable-murder --enable-http --enable-calalarmd
> >> --enable-replication --enable-backup --enable-idled
> >> --enable-autocreate CFLAGS="-fPIC -g")
> >> in a murder configuration on a RHEL 7.5 System. As filesystem we use xfs.
> >
> > They should be fine. I'll have a read of the config when I'm at a 
> > real computer.
> >
> >>
> >> > Bron.
> >> >
> >> > On Fri, Jan 25, 2019, at 04:08, Michael Menge wrote:
> >> >> Hi,
> >> >>
> >> >> I have discovered an other problem with the conversations db:
> >> >>
> >> >> Thousends of lines with "IOERROR: conversations_audit on load:" and
> >> >> "IOERROR: conversations_audit on store:"
> >> >> A look at the source code shows that these errors are logged after
> >> >> "_sanity_check_counts" is called.
> >> >> The log level LOG_ERR and the prefix IOERROR indicate that I have a
> >> >> serious problem. Do I?
> >> >>
> >> >> This problem occurred for accounts where the rebuild of the
> >> >> conversations db was successful.
> >> >>
> >> >> I don't want to rebuild the conversations db every few days.
> >> >>
> >> >> Any help is appreciated.
> >> >>
> >> >> Kind regards
> >> >>
> >> >>
> >> >> Michael Menge
> 
> 
> 
> 
> M.Menge Tel.: (49) 7071/29-70316
> Universität Tübingen Fax.: (49) 7071/29-5912
> Zentrum für Datenverarbeitung mail: 
> michael.me...@zdv.uni-tuebingen.de
> Wächterstraße 76
> 72074 Tübingen
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

-- 
 Bron Gondwana
 br...@fastmail.fm

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: another problem with conversations db

2019-01-25 Thread Michael Menge

Hi,


Quoting Bron Gondwana :


On Fri, Jan 25, 2019, at 20:00, Michael Menge wrote:

Hi Bron


Quoting Bron Gondwana :

> Sorry I haven't been following along with this earlier. Can you post
> your imapd.conf and cyrus.conf as well as let her know if you run
> anything which directly messes with files on disk.

Thanks for looking into it. I have attached the backend and replica
configs. There should be nothing messing with the files on disk
directly. Some times we need to restore mails from file based
backup (bacula) followed by a reconstruct but this was not the
case this time.


Do you always rebuild the conversations db after doing the  
reconstruct? That will be necessary now. We switched to doing all  
our restores using IMAP append a while back so we're never fiddling  
the file system under Cyrus.




We didn't have to restore any mails from backup since we enabled  
conversation db
a few weeks ago. It is good to know that the rebuild is necessary, but  
shouldn't

reconstruct also update the conversation db if it re-appends the message?
At least a hint should be put in the reconstruct man page, like the  
one for "quota -f"




>
> Also, what operating system is this on and what Cyrus version?
>

We are running a cyrus 3.0.8 compiled with the following Options
(./configure --enable-murder --enable-http --enable-calalarmd
--enable-replication --enable-backup --enable-idled
--enable-autocreate CFLAGS="-fPIC -g")
in a murder configuration on a RHEL 7.5 System. As filesystem we use xfs.


They should be fine. I'll have a read of the config when I'm at a  
real computer.




> Bron.
>
> On Fri, Jan 25, 2019, at 04:08, Michael Menge wrote:
>> Hi,
>>
>> I have discovered an other problem with the conversations db:
>>
>> Thousends of lines with "IOERROR: conversations_audit on load:" and
>> "IOERROR: conversations_audit on store:"
>> A look at the source code shows that these errors are logged after
>> "_sanity_check_counts" is called.
>> The log level LOG_ERR and the prefix IOERROR indicate that I have a
>> serious problem. Do I?
>>
>> This problem occurred for accounts where the rebuild of the
>> conversations db was successful.
>>
>> I don't want to rebuild the conversations db every few days.
>>
>> Any help is appreciated.
>>
>> Kind regards
>>
>>
>> Michael Menge





M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: another problem with conversations db

2019-01-25 Thread Bron Gondwana


On Fri, Jan 25, 2019, at 20:00, Michael Menge wrote:
> Hi Bron
> 
> 
> Quoting Bron Gondwana :
> 
> > Sorry I haven't been following along with this earlier. Can you post 
> > your imapd.conf and cyrus.conf as well as let her know if you run 
> > anything which directly messes with files on disk.
> 
> Thanks for looking into it. I have attached the backend and replica
> configs. There should be nothing messing with the files on disk
> directly. Some times we need to restore mails from file based
> backup (bacula) followed by a reconstruct but this was not the
> case this time.

Do you always rebuild the conversations db after doing the reconstruct? That 
will be necessary now. We switched to doing all our restores using IMAP append 
a while back so we're never fiddling the file system under Cyrus.

> >
> > Also, what operating system is this on and what Cyrus version?
> >
> 
> We are running a cyrus 3.0.8 compiled with the following Options 
> (./configure --enable-murder --enable-http --enable-calalarmd 
> --enable-replication --enable-backup --enable-idled 
> --enable-autocreate CFLAGS="-fPIC -g")
> in a murder configuration on a RHEL 7.5 System. As filesystem we use xfs.

They should be fine. I'll have a read of the config when I'm at a real computer.

> 
> > Bron.
> >
> > On Fri, Jan 25, 2019, at 04:08, Michael Menge wrote:
> >> Hi,
> >>
> >> I have discovered an other problem with the conversations db:
> >>
> >> Thousends of lines with "IOERROR: conversations_audit on load:" and
> >> "IOERROR: conversations_audit on store:"
> >> A look at the source code shows that these errors are logged after
> >> "_sanity_check_counts" is called.
> >> The log level LOG_ERR and the prefix IOERROR indicate that I have a
> >> serious problem. Do I?
> >>
> >> This problem occurred for accounts where the rebuild of the
> >> conversations db was successful.
> >>
> >> I don't want to rebuild the conversations db every few days.
> >>
> >> Any help is appreciated.
> >>
> >> Kind regards
> >>
> >>
> >> Michael Menge
> >>
> >>
> >>
> >>
> >>
> >> 
> >> M.Menge Tel.: (49) 7071/29-70316
> >> Universität Tübingen Fax.: (49) 7071/29-5912
> >> Zentrum für Datenverarbeitung mail:
> >> michael.me...@zdv.uni-tuebingen.de
> >> Wächterstraße 76
> >> 72074 Tübingen
> >>
> >> 
> >> Cyrus Home Page: http://www.cyrusimap.org/
> >> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> >> To Unsubscribe:
> >> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> >
> > --
> > Bron Gondwana
> > br...@fastmail.fm
> 
> 
> 
> 
> M.Menge Tel.: (49) 7071/29-70316
> Universität Tübingen Fax.: (49) 7071/29-5912
> Zentrum für Datenverarbeitung mail: 
> michael.me...@zdv.uni-tuebingen.de
> Wächterstraße 76
> 72074 Tübingen
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 
> *Attachments:*
>  * imapd_be.conf
>  * cyrus_be.conf
>  * cyrus_re.conf
>  * imapd_re.conf

-- 
 Bron Gondwana
 br...@fastmail.fm

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: another problem with conversations db

2019-01-25 Thread Michael Menge

Hi Bron


Quoting Bron Gondwana :

Sorry I haven't been following along with this earlier. Can you post  
your imapd.conf and cyrus.conf as well as let her know if you run  
anything which directly messes with files on disk.


Thanks for looking into it. I have attached the backend and replica
configs. There should be nothing messing with the files on disk
directly. Some times we need to restore mails from file based
backup (bacula) followed by a reconstruct but this was not the
case this time.



Also, what operating system is this on and what Cyrus version?



We are running a cyrus 3.0.8 compiled with the following Options  
(./configure --enable-murder --enable-http --enable-calalarmd  
--enable-replication --enable-backup --enable-idled  
--enable-autocreate CFLAGS="-fPIC -g")

in a murder configuration on a RHEL 7.5 System. As filesystem we use xfs.



Bron.

On Fri, Jan 25, 2019, at 04:08, Michael Menge wrote:

Hi,

I have discovered an other problem with the conversations db:

Thousends of lines with "IOERROR: conversations_audit on load:" and
"IOERROR: conversations_audit on store:"
A look at the source code shows that these errors are logged after
"_sanity_check_counts" is called.
The log level LOG_ERR and the prefix IOERROR indicate that I have a
serious problem. Do I?

This problem occurred for accounts where the rebuild of the
conversations db was successful.

I don't want to rebuild the conversations db every few days.

Any help is appreciated.

Kind regards


 Michael Menge






M.Menge Tel.: (49) 7071/29-70316
Universität Tübingen Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung mail:
michael.me...@zdv.uni-tuebingen.de
Wächterstraße 76
72074 Tübingen


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


--
 Bron Gondwana
 br...@fastmail.fm





M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen
# Configuration for Backend/Failover Instance
# Template MD5SUM: b5b04095d552bb1cbc2b178f7edfd1e2  conf/imapd_master.template
servername: ma01.mail.localhost
configdirectory: /srv/cyrus-be
partition-default: /srv/cyrus-be
partition-ssd: /srv/cyrus-be/ssd-part
metapartition-ssd: /srv/cyrus-ssd-be/meta/ssd-part
metapartition_files: header index cache expunge squat annotations lock dav 
archivecache
archivepartition-ssd: /srv/cyrus-hdd-be/archive/ssd-part
archive_enabled: 1
archive_days: 31
archive_maxsize: 10240

proc_path: /srv/tmpfs/proc-be
mboxname_lockpath: /srv/tmpfs/lock-be
defaultpartition: ssd
admins:  

mupdate_server: mupdate.mail.localhost 
mupdate_port: 3905
# mupdate_authname verwenden nicht mupdate_username
# sonst wird login als root/cyrus versucht
#mupdate_username: 
mupdate_authname: 
mupdate_password: 
proxy_authname: 
proxy_password: 
proxyservers: 

allowusermoves: 1
allowallsubscribe: 1

sync_host: sl01.mail.localhost 
sync_authname: 
sync_password: 
sync_port: 2005
guid_mode: sha1
sync_log: 1
sync_shutdown_file: /srv/cyrus-be/sync/shutdown

sievedir: /srv/cyrus-be/sieve
sieve_extensions: fileinto reject vacation imapflags notify include envelope 
body relational regex subaddress copy
sieve_maxscriptsize: 150

sasl_pwcheck_method: saslauthd
sasl_mech_list: plain login

allowanonymouslogin: no
reject8bit: no
quotawarn: 90
timeout: 45
poptimeout: 10
dracinterval: 0
drachost: localhost
lmtp_over_quota_perm_failure: 1
altnamespace: 1
#flushseenstate: 1
unixhierarchysep: 1
hashimapspool: 1
fulldirhash: 1
duplicatesuppression: 0
expunge_mode: delayed
delete_mode: delayed
deletedprefix: DELETED
anysievefolder: 1
#annotation_db: skiplist
#duplicate_db: skiplist
#mboxlist_db: skiplist
#ptscache_db: skiplist
quota_db: quotalegacy
#seenstate_db: skiplist
subscriptions_db: flat
xlist-sent: Mail.sent
xlist-trash: Mail.trash
xlist-drafts: Mail.drafts
xlist-junk: Mail.v-spam
xlist-spam: Mail.v-spam
specialusealways: 1
syslog_prefix: be 

# https://bugzilla.cyrusimap.org/show_bug.cgi?id=3850
# Vermutlich gefixed
#suppress_capabilities: LIST-EXTENDED


tls_server_cert: /etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_server_key: /etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_client_ca_file: /etc/pki/tls/certs/ca-bundle.crt
tls_ciphers: 
EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA:EECDH:EDH+AESGCM:EDH:+3DES:ECDH+AESGCM:ECDH+AES:ECDH:AES:HIGH:MEDIUM:!RC4:!CAMELLIA:!SEED:!aNULL:!MD5:!eNULL:!LOW:!EXP:!DSS:!PSK:!SRP:!3DES:!IDEA
tls_prefer_server_ciphers: 1

# Änderungen 27.06.2018
reverseacls: 1
search_engine: squat
delete_unsubscribe: 1
statuscache: 1

#tlscache_db deprecated

another problem with conversations db

2019-01-24 Thread Michael Menge

Hi,

I have discovered an other problem with the conversations db:

Thousends of lines with "IOERROR: conversations_audit on load:" and  
"IOERROR: conversations_audit on store:"
A look at the source code shows that these errors are logged after  
"_sanity_check_counts" is called.
The log level LOG_ERR and the prefix IOERROR indicate that I have a  
serious problem. Do I?


This problem occurred for accounts where the rebuild of the  
conversations db was successful.


I don't want to rebuild the  conversations db every few days.

Any help is appreciated.

Kind regards


   Michael Menge






M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus