Re: cyrus master fails with status 71

2016-11-07 Thread ellie timoney via Info-cyrus 
On Tue, Nov 8, 2016, at 05:01 AM, Eric Cunningham wrote:
> Nov  7 10:18:19 imap1 lmtpunix[58768]: unable to setsocketopt(TCP_KEEPCNT): 
> Invalid argument
> Nov  7 10:18:19 imap1 lmtpunix[58768]: unable to setsocketopt(TCP_KEEPIDLE): 
> Invalid argument
> Nov  7 10:18:19 imap1 lmtpunix[58768]: unable to setsocketopt(TCP_KEEPINTVL): 
> Invalid argument
> 
> Why is lmptunix complaining about options passed to imapd?

There's a bug here:

Based on a comment ("/* tcp only */"), master/service.c is trying to
limit the application of the tcp_keepalive config to TCP sockets only. 
But it's doing this by checking for the type being SOCK_STREAM -- which
is not exclusive to TCP sockets.  Notably, AF_UNIX domain sockets may
also have type SOCK_STREAM, and this looks like the default
configuration for "lmtpunix".

So, those errors are because it's trying to set TCP_* socket options on
a socket that is not a TCP socket.  (Notice too that it doesn't error
when trying to set SO_KEEPALIVE, because that's a valid SOCK_STREAM
option.  It only fails to set the TCP_* options.)

Looks like this has been fixed on master already.  I'll see if the fix
backports to 2.5 cleanly, but in the meantime you can ignore these
errors.

Cheers,

ellie

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus master fails with status 71

2016-11-07 Thread Bron Gondwana via Info-cyrus 
On Tue, 8 Nov 2016, at 05:01, Eric Cunningham via Info-cyrus wrote:
> Are such numbers of imapd processes to be expected?

Depends on your usage patterns of course.

> Why is lmptunix complaining about options passed to imapd?

Because the names of the config files are bogus for historical reasons.  
imapd.conf should really be called settings.conf and cyrus.conf should really 
be called master.conf.  It's definitely not a config file for just the imap 
daemon.

Bron.

-- 
  Bron Gondwana
  br...@fastmail.fm

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Xapian search databases

2016-11-07 Thread Bron Gondwana via Info-cyrus 
On Mon, 7 Nov 2016, at 23:23, Adam Tauno Williams via Info-cyrus wrote:
> On Thu, 2016-11-03 at 10:47 +1100, Bron Gondwana via Info-cyrus wrote:
> > Just out of interest - is anyone other than Fastmail currently using
> > the Cyrus Xapian-based search system?
> 
> Not using Xapian.

Cool.  It turns out I didn't have to make it all backwards incompatible just 
yet, because instead I just detect if it's using the old Chert backend and 
force a reindex of all email to upgrade to Glass.

(but can still read from a mix of both, so you can phase the upgrade)

This is going to the rest of FastMail production today after running happily 
for the past 4 days for ~5% of our users.

Bron.

-- 
  Bron Gondwana
  br...@fastmail.fm

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus master fails with status 71

2016-11-07 Thread Eric Cunningham via Info-cyrus 
Hi Ellie, we've been running with your patch since Oct 25 and haven't 
encountered any issues with imapd exiting, thus far.  But, now that 
imapd has had a chance to run uninterrupted for almost 2 weeks, the 
number of imapd processes/connections has steadily climbed every day. 
This morning, it was near 16,000.  This system has a total of 1400 accounts.


To try and control this growth, per 
https://cyrusimap.org/imap/faqs/o-toomanyprocesses.html I've set the 
following:


To cyrus.conf, added "-U 50" option to the SERVICES section for imapd:

  imap  cmd="imapd -U 50" listen="imap" prefork=60
  imaps cmd="imapd -s -U 50"  listen="imaps"prefork=150


To imapd.conf, added the following tcp_keepalive options:

  tcp_keepalive: 1
  tcp_keepalive_cnt: 1
  tcp_keepalive_idle: 30
  tcp_keepalive_intvl: 900

After restarting imapd, the following are now being logged repeatedly:

Nov  7 10:18:19 imap1 lmtpunix[58768]: unable to 
setsocketopt(TCP_KEEPCNT): Invalid argument
Nov  7 10:18:19 imap1 lmtpunix[58768]: unable to 
setsocketopt(TCP_KEEPIDLE): Invalid argument
Nov  7 10:18:19 imap1 lmtpunix[58768]: unable to 
setsocketopt(TCP_KEEPINTVL): Invalid argument



So, a couple of questions for the list:

Are such numbers of imapd processes to be expected?
Why is lmptunix complaining about options passed to imapd?

Thank you.

-Eric


On 10/25/16 8:23 PM, ellie timoney via Info-cyrus wrote:

Hi Eric,

Patch attached.  I'd appreciate if you could advise whether this helps.
Though I guess you won't be able to tell for a couple of weeks.

If it doesn't cause any new problems (I don't expect it to), then it
will be included in 2.5.11 (whenever that comes out).

Cheers,

ellie

On Wed, Oct 26, 2016, at 10:04 AM, ellie timoney via Info-cyrus wrote:

accept failed: Software caused connection abort


Some sleuthing suggests that "Software caused connection abort"
corresponds with "ECONNABORTED".

The man page on my system for accept(2) unhelpfully defines this as:


   ECONNABORTED
  A connection has been aborted.


But some digging around online suggests that this situation occurs when
a client connects, but subsequently disconnects  (RST) before the server
gets around to accept()ing the connection.  When the server does
eventually accept(), the accept() fails with this error.

Which sounds to me like we want to treat ECONNABORTED similarly to
EAGAIN, not as a fatal OS error.  I'll have a patch up for this shortly.

Cheers,

ellie

On Wed, Oct 26, 2016, at 09:27 AM, Eric Cunningham via Info-cyrus wrote:

Having repeatedly experienced the "status 71" issue, I've been
incrementally bumping it's value up.  It's currently set to 32768 (!)
and that value was in place when it most recently failed.


On 10/25/16 4:21 PM, Shawn Bakhtiar via Info-cyrus wrote:

H.. if that’s the case could you be hitting the the maximum number
of accepts??

Check the 11.11.1.2. kern.ipc.soacceptqueue section of the FreeBSD handbook

https://www.freebsd.org/doc/handbook/configtuning-kernel-limits.html

Given the load you described perhaps 128 is just not enough?




On Oct 24, 2016, at 1:22 PM, Eric Cunningham via Info-cyrus
mailto:info-cyrus@lists.andrew.cmu.edu>> wrote:



=
 Eric Cunningham
 Information Services - http://whoi-it.whoi.edu
 Woods Hole Oceanographic Institution - http://www.whoi.edu
 Woods Hole, MA  02543-1541 phone: (508) 289-2224
 fax: (508) 457-2174   e-mail: ecunning...@whoi.edu

=

On 10/24/2016 03:45 PM, Bron Gondwana via Info-cyrus wrote:

On Tue, 25 Oct 2016, at 02:45, Eric Cunningham via Info-cyrus wrote:

Hi list, we're running cyrus imap 2.5.9 built from the FreeBSD 10-2
(release-p7) ports tree.

The cyrus master process is failing periodically (every 1-2 weeks) as
follows:

Oct 22 07:38:48 imap1 master[7767]: process type:SERVICE name:imaps
path:/usr/local/cyrus/bin/imapd age:305.215s pid:32760 exited, status 71
Oct 22 07:38:48 imap1 master[7767]: service imaps/ipv4 pid 32760 in
READY state: terminated abnormally
Oct 22 07:38:48 imap1 master[7767]: too many failures for service
imaps/ipv4, disabling until next SIGHUP

This prevents new connections by clients until cyrus is restarted.  I've
looked around the web but have not seen this issue reported.

A little background:

Our initial thought on this was that we were running out of listen
queues so have upped that incrementally from the default of 32 to a
current setting of 32768 via /usr/local/etc/rc.d/imapd using the -l
option, with increased kern.ipc.soacceptqueue set to 32768, but that
hasn't helped.  Sometimes the "status 71" occurs during periods of light
use during off hours, like on Saturday mornings.

We have ~1400 imap accounts, though the number of impad processes hovers
around 3,000-4,000.  There have been spikes observed as high as 12,000
imapd processes.  In that partic

Re: Xapian search databases

2016-11-07 Thread Adam Tauno Williams via Info-cyrus 
On Thu, 2016-11-03 at 10:47 +1100, Bron Gondwana via Info-cyrus wrote:
> Just out of interest - is anyone other than Fastmail currently using
> the Cyrus Xapian-based search system?

Not using Xapian.

-- 
Adam Tauno Williams  GPG D95ED383
Systems Administrator, Python Developer, LPI / NCLA



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Memory unauthorized access issues in byte2search in Cyrus IMAP

2016-11-07 Thread Egoitz Aurrekoetxea via Info-cyrus 

Good morning,


I have been checking the Cyrus IMAP 2.3.19 and 2.3.18 code because I 
have observed some issues in UID SORT commands in the IMAP protocol. 
When performing a command


like ". UID SORT (SIZE) US-ASCII ALL TEXT avanzada" in a mailbox where 
matches were found caused you to obtain in a debug (or non debug I 
think) log the following entry :


Oct 31 09:17:21 hostname master[78064]: process 78268 exited, signaled 
to death by 11


Lines like this are seen when a process has been signaled by the kernel 
with signal 11. Have been reading this signal is sent to a proccess when 
it performs an unauthorized memory


access attemp (an out of the own variable, pointer... etc, storage 
room). After debugging the code with GDB and doing several checks, have 
seen the issue came from the byte2search()


function when a piece of the string s->substr was trying to be stored in 
b. Concretely the third if in the loop :



for (i = 0, cur = 0; i < s->max_start; i++) {
/* no more active offsets */
if (s->starts[i] == -1)
break;

/* if we've passed one that's not ongoing, copy back */
if (cur < i) {
s->starts[cur] = s->starts[i];
}
/* check that the substring is still maching */
if (b == s->substr[s->offset - s->starts[i]]) {


The issue was caused there because s->starts[i] in this place, was not 
being able to be accesed because it was pointing to to data outside 
s->starts. After searching where this array was being initialized


and it's memory allocated (which was in search_init function), I tried 
to allocate 10 bytes more for that pointer. After doing it, there were 
no more issues. So I tried allocating just one byte more which it seemed


to be enough too (at least for the patterns I have searched for). At 
this moment I understood this pointer (s->starts which was a 
search_state->substr pointer inside the search_state structure) was not 
having


enough room for all the content needed to be stored, or at least accesed 
when calling it. I checked then the code of Cyrus 2.3.18 and 2.3.19 but 
didn't see any kind of differences in the part of the memory


allocation (in search_init()) or usage (in bytesearch) for s->starts. I 
deciced to check Cyrus 2.4 code and I saw it's room was being allocated 
the following way :



s->starts = xmalloc(s->max_start * sizeof(size_t));


instead of that in 2.3 was done :


s->starts = xmalloc(s->max_start * sizeof(int));


So I understood s->starts should be allocated to the size of a size_t 
type defined variable size, instead to the size of an integer variable n 
times. After replacing it, has seen definitively all seemed to be


working. So wouldn't Cyrus 2.3 sources have this allocation in 
search_init done with sizeof(size_t) instead of the sizeof(int)?. I 
think this is important because else, when the first character of a


pattern is repeated more than one time, the pattern has a would say 
patlen of 8-9 bytes and matches exist in the mailbox, that search would 
end up with a proccess died due to a signal 11.



My env is FreeBSD RELENG_9_0 OS with a Cyrus 2.3.18_1 port. Am I wrong, 
shouldn't that allocation be changed?.



Thanks a lot for your time,

Best regards,


--


sarenet
*Egoitz Aurrekoetxea*
Departamento de sistemas
944 209 470
Parque Tecnológico. Edificio 103
48170 Zamudio (Bizkaia)
ego...@sarenet.es 
www.sarenet.es 

Antes de imprimir este correo electrónico piense si es necesario hacerlo.

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Front End not forwarding requests to the backend

2016-11-07 Thread Michael Menge via Info-cyrus 

Hi,

Quoting Alberto Cardenas via Info-cyrus :


Hello,


I'm trying to setup Cyrus Murder for testing and have three virtual  
machines for this purpose (frontend, mupdate and backend). The three  
machines are running Centos 6.5, SASL 2.1.23 and Cyrus 2.3.16




you might want to consider using a more recent cyrus Version. Cyrus 2.3.16
is about 7 years old. 2.5.10 is the latest stable, 3.0 will be coming out
soon. I don't think it has something to do with your problems but, there
have been many bugfixes and improvements since 2.3. And if you set up a
new system it would make things easier in the future.



Looks like communication from frontend to muptade and from backend  
to mupdate is working fine since I can list the mailboxes created in  
the backend using the “cyradm -u cyrus localhost” and “lm” commands,  
which shows the same user’s mailboxes result on the three boxes.


The problem seems to be between the frontend and the backend since  
the frontend is not forwarding the imap requests to the backend. To  
test this, I am using the command “telnet localhost 143” on the  
frontend server. Once that I get connected and authenticated, I  
issue the command “select inbox”, but I receive the message “NO  
Server(s) unavailable to complete operation”. In the  
/usr/log/maillog file there is an error message that says “failed:  
Connection timed out”



I have not configured yet the smtp server, not sure if it is  
required, so I am trying to focus to get the imap part working.




The smtp server is needed to for the mail delivery and sieve redirects
and vacation but not for the imap access, so you can configure it later.



Any guidance would be appreciated.  I have spent several days  
working in this problem without getting any progress at all.


First, you din't set servername in you backend imapd.conf, which could lead
to the problem that the wrong name is used in the mailbox.db on the  
mupdate server.


You can check that name used to register the created mailboxes on the  
mupdate by


mupdate> ctl_mboxlist -d

The 3. column is the name of the backend follwod by the partition name  
seperated by !
Make sure that the name used resolves to the correct ip address and  
the the backend

is listening on the imap port on this ip

The forndend needs to be able to connect to the backend on the imap  
port. You can test this

with telnet, but you can also use imtest which can help you to speak starttls.
Also check your audit logs on the frontend, selinux tends to block  
wanted connections.


also "netstat -tupen | grep :143" can help to check the connection  
between frontend and backend.





Here are my configuration files:
Frontend’s imap.conf
configdirectory: /var/lib/imap
partition-default: /var/spool/imap
admins: cyrus murder
sievedir: /var/lib/imap/sieve
sendmail: /usr/sbin/sendmail
hashimapspool: true
sasl_pwcheck_method: auxprop
sasl_mech_list: PLAIN
#tls_cert_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
#tls_key_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
#tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt
allowplaintext: yes
mupdate_server: cyrus-mupdate.cvacyrus.com
mupdate_port: 3905
mupdate_username: murder
mupdate_authname: murder
mupdate_password: cvacva
cyrus-backend2_password: cvacva
proxy_authname: murder


Frontend’s cyrus.conf
START {
 recover   cmd="ctl_cyrusdb -r"
 idled cmd="idled"
}
SERVICES {
 mupdate   cmd="mupdate" listen=3905 prefork=1
 imap  cmd="proxyd" listen="imap" prefork=5
}
EVENTS {
 checkpointcmd="ctl_cyrusdb -c" period=30
 delprune  cmd="cyr_expire -E 3" at=0400
 tlsprune  cmd="tls_prune" at=0400
}




Backend’s imap.conf
configdirectory: /var/lib/imap
partition-default: /var/spool/imap
admins: cyrus murder
sievedir: /var/lib/imap/sieve
sendmail: /usr/sbin/sendmail
hashimapspool: true
sasl_pwcheck_method: auxprop
sasl_mech_list: PLAIN
altnamespace: yes
allowplaintext: yes
proxyservers: murder
mupdate_server: cyrus-mupdate.cvacyrus.com
mupdate_port: 3905
mupdate_username: murder
mupdate_authname: murder
mupdate_password: cvacva


Backend’s cyrus.conf
START {
 recover   cmd="ctl_cyrusdb -r"
 idled cmd="idled"
 mupdatepush cmd="ctl_mboxlist -m"
}
SERVICES {
 imap  cmd="imapd" listen="imap" prefork=5
}
EVENTS {
 checkpointcmd="ctl_cyrusdb -c" period=30
 delprune  cmd="cyr_expire -E 3" at=0400
 tlsprune  cmd="tls_prune" at=0400
}


Mupdate’s imap.conf
configdirectory: /var/lib/imap
partition-default: /var/spool/imap
admins: cyrus murder
sievedir: /var/lib/imap/sieve
sendmail: /usr/sbin/sendmail
hashimapspool: true
sasl_pwcheck_method: auxprop
sasl_mech_list: PLAIN
allowplaintext: 1


Mupsate’s cyrus.conf
START {
 recover   cmd="ctl_cyrusdb -r"
 idled cmd="idled"
}
SERVICES {
 imap  cmd="imapd" listen="imap" prefork=5
 mupdate cmd="mupdate -m" listen="mupdate" prefork=1
}
EVENTS {
 checkpointcmd="ctl_cyrusdb -c" period=30
 delprune  cmd="cyr_expire -E 3" at=0400
 tlsprune  cmd="