File ownership of setuid processes

2006-12-01 Thread Samuel Moñux

Hi everyone!,

I'm trying to migrate a sympa installation (sympa is a perl written
mailing list manager) from a Linux system into an OpenBSD one, and I'm
getting into a strange problem.

The way a message reaches the sympa is trough a C written, setuid
binary called "queue" which is setuid to sympa's user. The MTA pipes
the message into this binary, which its only mission is to create a
file into a spool direcotry, where the main sympa process can read it.

The problem here is that "queue" in OpenBSD isn't creating files with
sympa ownership, so the main process can't read them(they are owned by
"daemon").

I have done a small test program to be run in both systems, and thats
what I get:

#include 
#include 
#include 

int main (void) {
   int fd;
   char msg1[] = "Cannot open file\n";

   umask(027);
   if (!(fd = open("/tmp/testfile", O_CREAT| O_WRONLY , 0600))) {
   write(2,msg1, sizeof(msg1));
   }
   close(fd);
}

 OpenBSD System
# make test
cc -O2 -pipe-o test test.c
# chown _sympa._sympa test
# chmod u+s test
# ./test
# ls -l
total 32
-rw---  1 _sympa   wheel   2984 Nov 30 16:02 ayud
drwx--  2 _clamav  wheel512 Dec  1 09:30 clamav-1cd2ff8e8aea8f7f
-rwsr-xr-x  1 _sympa   _sympa  6641 Dec  1 09:31 test
-rw-r--r--  1 root wheel255 Dec  1 09:29 test.c
-rw---  1 root wheel  0 Dec  1 09:30 testfile


[EMAIL PROTECTED] tmp]# make test
cc test.c   -o test
[EMAIL PROTECTED] tmp]# chown sympa.sympa test
[EMAIL PROTECTED] tmp]# chmod u+s test
[EMAIL PROTECTED] tmp]# ./test
[EMAIL PROTECTED] tmp]# ls -l
total 16
-rwsr-xr-x1 sympasympa   14092 dic  1 09:33 test
-rw-r--r--1 root root  312 dic  1 09:27 test.c
-rw---1 symparoot0 dic  1 09:34 testfile

It's ok this different behaviour? or may be there is something that
I'm doing wrong?

Thanks in advance.



About pf states

2007-01-16 Thread Samuel Moñux

Hello everyone,

I'm trying to replace a crufty pf.conf which has evolved badly, and I
think doesn't fully uses the stateful capabilities of pf. The problem
is that there must be something I don't really understand about
states.

My plan was to write a pf.conf with almost no "out" clauses. A packet
cames from an interface, if it's "good", keep state and forget about
the entire conversation. So, as a start I have write a pf.conf like
this (three interfaces, lan, inet, dmz):

tcp_inet_services="{ smtp, submission, https, www,  domain }"
tcp_lan_services="{ smtp, submission, https, www, 8080, ssh, \
domain }"
ssh_port="{ 22 }"

udp_inet_services="{ domain, isakmp, ipsec-nat-t }"
udp_lan_services="{ domain }"

#set skip on lo0
#set loginterface $ext_if
#scrub in all fragment reassemble
set state-policy floating


# Redirecciones/NAT
nat on $ext_if from $estaciones_edi to $servidores_edi -> $dmz_if:0


# Por defecto bloquear. Intentamos no usar out, solo in y "keep state"
# clasificando en interfaz de entrada del paquete.
block in log all


# VPN's
pass in quick on $ext_if from 192.168.200.0/24 to any keep state
pass in quick on $int_if from 192.168.0.0/16 to 192.168.200.0/24 keep state

# Trafico entrante INET
pass in quick on $ext_if proto tcp from any to $dmz_range port
$tcp_inet_services flags S/SA keep state
pass in quick on $ext_if inet proto icmp all icmp-type echoreq keep state

#pass in quick on $ext_if proto esp from any to $dmz_range queue ipsec_q
pass in quick on $ext_if proto esp from any to $dmz_range
pass in quick on $ext_if proto udp from any to $dmz_range port
$udp_inet_services keep state

# Trafico entrante LAN
pass in quick on $int_if proto tcp from any to $dmz_range port
$tcp_lan_services flags S/SA modulate state
pass in quick on $dmz_if proto tcp from any to $dmz_range port
$ssh_port flags S/SA modulate state
pass in quick on $int_if proto icmp from any to $dmz_range keep state
#pass in quick on $int_if from { $dmz_if:0 } to $servidores_edi flags
S/SA keep state
pass in quick on $int_if from $estaciones_edi to $servidores_edi flags
S/SA keep state

# Trafico entrante DMZ
pass in quick on $dmz_if proto icmp from any to any keep state
pass in quick on $dmz_if proto tcp from any to any port { www, smtp,
https } keep state

# Trafico hacia fw
pass in quick proto tcp from any to { $int_if:0, $ext_if:0 } port {
ssh }  keep state
pass in quick inet proto icmp all icmp-type echoreq keep state
# Trafico desde fw
pass out quick proto udp from any to any port { domain, ntp } keep state

--

With this config, I can't access "dmz hosts" from lan or internet. The
state gets created:

all tcp $dmz_ip:25 <- 192.168.1.161:19399   CLOSED:SYN_SENT

but the response is blocked:

Jan 16 19:32:59.627083 rule 0/(match) block in on xl2: $dmz_ip.25 >
192.168.1.161.19399: [|tcp] (DF)

So, there's something I don't understand from pf.conf  man page
"
If a packet matches a pass ... keep state rule, the filter creates a
state for this connection and automatically lets pass all subsequent
packets of that connection.
"

If just someone could show me what's wrong here, and why

I'm running 4.0, if that matters.

Thanks in advance, Samuel



Re: About pf states

2007-01-17 Thread Samuel Moñux

2007/1/17, Brian Candler <[EMAIL PROTECTED]>:


Someone please correct me if I'm wrong, but I believe that the 'keep state'
only applies to the opposite packets through the same interface. For
example:

 pkt1++   pkt1'
---> | ext_if  int_if | -->
<--- || <--
 pkt2++   pkt2'

A rule which matches pkt1 inbound on ext_if with "keep state" will also
match pkt2 outbound on ext_if.

But I believe you will need another rule to permit pkt1' outbound via
int_if. If that rule has "keep state" then it will also match pkt2' inbound
via int_if.

If you look at the example at
http://www.openbsd.org/faq/pf/example1.html
you will see that they deal with this by a global "pass out keep state"
rule. Try adding this to your ruleset after your "block in log all"


Thanks Brian. I've added the global pass out this morning and
everything has started to work, but I wasn't sure why this was needed.
Now I think I understand; every state is a [src, dst, direction] tuple
which lets pass [src -> dst, direction ] and [dst -> src,
not(direction)], but not [ src-> dst, not(direction) ] packets. In my
case:

[192.168.1.161 -> dmz_ip, in ] and [dmz_ip -> 192.168.1.161, out ] passed but
[192.168.1.161 -> dmz_ip, out] and [dmz_ip -> 192.168.1.161, in ] did not

is that?

I see that I was also misinterpreting state-policy.

The old pf.conf was full of pass out rules, which I assumed were
needed because of a "block out all" rule. I also realize that a couple
states are needed for every session.

Thanks again



Source Interface for outgoing connections

2007-02-26 Thread Samuel Moñux

Hi everyone,

I'm having some issues with an ipsec connection with vpnc (isakmp is
not an option, since does not support xauth, and I don't control the
other end) from an OpenBSD firewall/router to a Cisco device.

I think problems could be natt related so I would like to eliminate
nat from the equation, but the problem is that the "outside" interface
is a private address. This firewall routes between a DMZ (public /29),
a LAN segment (private /24), and the outside (private /30).


-- LAN --- OpenBSD -- 10.90.0.0/30 --- Outside Router -- INET
 |
 |
 DMZ (public /29)

Right now, I need to NAT on the Outside Router, since internet routed
packets from the OpenBSD box go out with a private address.

What I would like to achieve is that packets destined to internet get
sourced with DMZ's interface, which is internet routable, and without
pf tricks(I don't want NAT, remember).

Bridging is not an option, since the Outside router needs its own IP
for its own purposes.

I don't know it its possible. If it's, please, let me know (pointing a
man page would be OK).

Thanks in advance



Re: Source Interface for outgoing connections

2007-02-27 Thread Samuel Moñux

2007/2/26, Darren Spruell <[EMAIL PROTECTED]>:

On 2/26/07, Samuel Moqux <[EMAIL PROTECTED]> wrote:
> I'm having some issues with an ipsec connection with vpnc (isakmp is
> not an option, since does not support xauth, and I don't control the
> other end) from an OpenBSD firewall/router to a Cisco device.
>
> I think problems could be natt related so I would like to eliminate
> nat from the equation, but the problem is that the "outside" interface
> is a private address. This firewall routes between a DMZ (public /29),
> a LAN segment (private /24), and the outside (private /30).
>
>
> -- LAN --- OpenBSD -- 10.90.0.0/30 --- Outside Router --

INET

>   |
>   |
>   DMZ (public /29)
>
> Right now, I need to NAT on the Outside Router, since internet routed
> packets from the OpenBSD box go out with a private address.
>
> What I would like to achieve is that packets destined to internet get
> sourced with DMZ's interface, which is internet routable, and without
> pf tricks(I don't want NAT, remember).

If you could get vpnc to bind to a specific interface it seems like
that would be possible. Can you see if that's an option?


No it isn't. Looking at the source does not seem hard to do, however.
Maybe I'll make an small patch to address this. It's not an uncommon
situation to sit a public addressable range in a DMZ, and use a
reserved one for routing I think.



The way I see it, NAT may not be an issue; any worthwhile modern IPsec
implementation supports NAT traversal, which vpnc appears to (I see a
reference to '--natt-mode' on their page.) If you can support NAT-T on
the client and server, it may be a non-issue for you.


Yes, I have tried with different natt modes. But connection dies
2h50min after. I'm not sure if the problem is natt related, it's just
to discard options and simplify the configuration.


Haven't used vpnc myself, but just looking at the package install
message there's a couple of considerations:



I had done that. Thanks for your comments, Darren



Sizing an IMAP Server on OpenBSD

2006-07-07 Thread Samuel Moñux

Hi everyone,

I'm planning to deploy a SMTP(Sendmail) and IMAP(Cyrus) server on a
mid-sized organization(~300 remote users, dunno about messages/day),
and since is my first IMAP server (until now we do only POP), I have
some questions about sizing.

First, about hardware requirements. I had tought to use a Dell 1850,
2GB RAM with two controllers: a PERC4e/Si for system + sendmail queue,
and a PERC 4e/DC connected to a PV220s, with 7x300GB (half of
backplane) for imap data (4 or 6 discs in RAID-10 + 1 hot spare) . I
think it should be enough, but it's really? (the hardware it's already
bought, so I really hope so). Any recommendations about stripe size or
raid configuration?, which ami version to use? -stable one? How ami's
performance compares with FreeBSD's amr?

I understand that is advisable to run softupdates on the imap and
/var/spool partitions, and to disable fsck on boot, but what about
increasing buffer cache size? 5% of physical memory seems a bit low
for an I/O intensive app as Cyrus is.

About resource limits of _cyrus user and sysctl values, are there well
known values? Should I increase kern.maxfiles for example? I wouldn't
like to learn it at production time.

Well, this are my questions. May be the hardware is overkill for our
load, but sizing hardware without prior experience it's always a
difficult task, so if  anybody wants to share their experience...

Thanks in advance,

Samuel



Re: Strange 3.9 lock-up

2006-07-16 Thread Samuel Moñux

2006/7/16, Maxim Bourmistrov <[EMAIL PROTECTED]>:

Hi [EMAIL PROTECTED]
As I have mentioned before my 3.9-box locks up in a strange way:
1. it is pingable
2. syn-scan gives out open ports
3. but those ports are not accessable(for ex. I can not drop in into this box 
via ssh or browse port 80)



May be related or not, but I 've suffered similar lockups in mostly
the same hardware. I was migrating 16GB of  mailboxes to Cyrus using
deliver, without limiting the number of lmtpd processes. This put a
lot of stress on the box and it locked-up. The first time I tried the
migration, the machine panicked with a "uvm_mapent_alloc: out of
static map entries" message.

I couldn't get a backtrace because it hadn't a serial console attached
(now it has). But the subsequent ones it didn't crash, only showed the
same behavior that has been described here.  Connections dropped but
responds to ping, no keyboard typping response.

Finally i limited the number of lmtpd processes and I could complete
the migration.

This machine it's in a testing stage by now, and I can reproduce the
problem in an hour or two

Best regards,
Samuel



Re: 3.9 freeze

2006-07-31 Thread Samuel Moñux

2006/7/31, diego <[EMAIL PROTECTED]>:

Pedro, since I set the "option NKMEMPAGES_MAX=65535" on kernel file, the
server doesn't freeze

UVM amap128305 10153K  50705K157284K4071891000 0
16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536

This server has an uptime 12 days, before the change only alive 3 or 4
days


regards,.



Doing that changes I can migrate 16GB of messages in mbox format to
cyrus without limiting the number of lmtpd processes. I get some
"uvm_mapent_alloc: out of static map entries" but the server doesn't
freeze.

Best regards,



STP over an IPSEC bridge?

2006-08-30 Thread Samuel Moñux

Is it possible? brconfig man pages says:

The bridge has support for 802.1D Spanning Tree Protocol (STP), which can
be used to detect and remove loops in a network topology.  Using the stp
or -stp commands to brconfig, STP can be enabled or disabled on each
port.  STP will not work on gif(4) members because they lack a hardware
MAC address.

So I assume that it isn't, but in case anybody knows any workable
solution over OpenBSD I would like to know. I would prefer this
solution instead of having to fight with Cisco gear and L2TP which
seems much more complex (and expensive!).

I need a bridge over the internet in case the long fiber link between
our main and secondary datacenter fails (bandwidth difference is huge,
but it would suffice for our critical apps). STP is necessary for
obvius reasons.

Thanks in advance,
Samuel



Re: STP over an IPSEC bridge?

2006-08-30 Thread Samuel Moñux

2006/8/30, Stuart Henderson <[EMAIL PROTECTED]>:

On 2006/08/30 19:35, Samuel Moqux wrote:
> I need a bridge over the internet in case the long fiber link between
> our main and secondary datacenter fails (bandwidth difference is huge,
> but it would suffice for our critical apps). STP is necessary for
> obvius reasons.

Can't you run a routed connection over gif(4) instead of bridging?
Then you can just use OSPF.



Some apps need to be on the same subnet (SNA, and propietary
redundancy protocols), but others don't. Probably putting those on a
different subnet would configure a saner environment. OpenVPN sounds
interesting, though.

Thanks!!



Re: docs for OpenLDAP and cyrus-imapd on OpenBSD?

2006-10-29 Thread Samuel Moñux

2006/10/29, Paul Pruett <[EMAIL PROTECTED]>:

Looking for docs for OpenLDAP and cyrus-imapd on OpenBSD

I am trying to implement cyrus-imap on OpenBSD sendmail for virtual
hosting with support for  pop3, imap (localhost), and smtp-auth.



I have it working a similar setup. All you need once the openldap side
is runnning (and there are lots of docs about OpenLDAP for users
authentication out there):

In the sendmail side, you just need to map mail adresses to uids
(forget mailAlternateAddress, just needed if you migrated from
qmail-ldap):

FEATURE(`virtusertable',
`ldap -1 -T -v uid -k (|(mail=%0)(mailAlternateAddress=%0))')dnl

configure SASL options in sendmail.mc

define(`confAUTH_MECHANISMS', `CRAM-MD5 DIGEST-MD5 LOGIN PLAIN')dnl
TRUST_AUTH_MECH(`CRAM-MD5 DIGEST-MD5 LOGIN PLAIN')

/usr/local/lib/sasl2/Sendmail.conf

pwcheck_method: saslauthd
mech_list: plain cram-md5 digest-md5 login

saslauthd.conf:

ldap_servers: ldap://ldap.xxx/
ldap_bind_dn: cn=xxx,dc=xxx,dc=sa
ldap_bind_pw: 
ldap_search_base: ou=People,dc=carreras,dc=sa
ldap_filter: (&(uid=%u)(accountStatus=active))
ldap_password_attr: userPassword

Cyrus LDAP authentication:

/etc/imapd.conf

sasl_pwcheck_method: saslauthd
sasl_mech_list: login cram-md5 digest-md5 plain

(*-md5 mechs won't work with hashed userPasswords, but don't harm)

You will need a sendmail compiled with SASL and ldap support, just add
this to /etc/mk.conf:

WANT_LDAP=yes
WANT_SMTPAUTH=yes

And thats all...

Greets



Re: openbsd instead of cisco vpn client

2007-08-28 Thread Samuel Moñux
2007/8/27, Paolo Supino <[EMAIL PROTECTED]>:
> Hi
>
>   I came across the following situation: there's network where several
> employees have access to a client of theirs using Cisco VPN clients.
> To centralize and ease administration I want to put in place an OpenBSD
> box that will create a single VPN.
> The client is so bearucratic that by the time their paperwork  for
> setting up a site to site VPN the need for this VPN will be gone.
> So is it possible to mimick Cisco VPN client connection with OpenBSD
> IPSEC?
>

You can't with base install since it doesn't support xauth(it's in
isakmpd's todo I think), but vpnc works good enough for my needs,
which look similar to yours. I need to reset the connection nightly
because unreliable ike rekeying, but, other than that, It's stable.

http://www.unix-ag.uni-kl.de/~massar/vpnc/

Best regards,
Samuel



Re: How do I configure sendmail?

2007-10-16 Thread Samuel Moñux
2007/10/16, Sunnz <[EMAIL PROTECTED]>:
> Hi, I have read the man pages of afterboot, sendmail, and also looked
> at /usr/share/sendmail/README. I also have tried to google, and are
> now confused then ever.
>

Look at "Providing SMTP AUTH Data when sendmail acts as Client"
section in that file. I think its all what you need.

Best regards.
Samuel



Cyrus IMAP performance problems [Long]

2007-10-16 Thread Samuel Moñux
Hello everyone,

I running into some problems with a Cyrus Imap server. A year ago I
asked about sizing the server and it was defined as "overkill" (Dell
1850, Perc 4e/DC, 4 300GB 10krpm disks in Raid 10) for 300 users, but
now I'm suffering of really high loads and performance problems. I
think it's due I/O contention (more than a thousand t/s in iostat),
but don't know why Cyrus is so intensive in I/O that this hardware
can't handle it. Memory consumption is also very high.

When I started with ~60 users everything was ok, but after surpasing
the one hundred, performance problems arised. I added a imap proxy
which sited between the server and the webmail(Squirrel) and improved
overall performance, but now we have 215 users and the server has
severe problems during peak hours, when load can reach values of 12 or
more. Users perceive degraded response times.

Top usually lota of imap processes waiting for getblk, biowait, lockf,
or select. Vmstat also shows always proceses blocked waiting for I/O.

I had to tweak default kernel values because server crashed when
importing old mailboxes:
maxusers64  # estimated number of users
option NKMEMPAGES_MAX=65535
option BUFCACHEPERCENT=15

Cyrus was compiled from ports, with default options. The server is
running Sendmail and a lightly loaded openldap.

I don't know where to look at or what to tweak. Any ideas will be welcome.

Best regards.

Dmesg and other relevant data (non peak hours).

load averages: 10.36,  6.97,  5.79
17:59:50
259 processes: 258 idle, 1 on processor
CPU states:  3.7% user,  0.0% nice, 20.6% system,  1.9% interrupt, 73.8% idle
Memory: Real: 1135M/1702M act/tot  Free: 312M  Swap: 47M/2196M used/tot

  PID USERNAME PRI NICE  SIZE   RES STATEWAIT TIMECPU COMMAND
 9503 _cyrus 20 2716K 4392K sleepselect   0:08  1.12% imapd
16618 _cyrus-50 3976K 5728K sleepbiowai   0:08  1.03% imapd
14158 _cyrus 20 3544K 5568K sleepselect   0:01  0.68% imapd
23082 _cyrus-50   11M   11M sleepgetblk   0:06  0.54% imapd
 4397 _cyrus 20 1664K 3396K sleepselect   0:00  0.54% imapd
  831 _cyrus-50   14M   16M sleepgetblk   0:01  0.34% imapd
30096 _cyrus 20 6136K 7464K sleepselect   0:03  0.29% imapd
10307 _cyrus-50 5572K 5908K sleepgetblk   0:00  0.29% imapd
28758 _cyrus-50 7664K 9412K sleepbiowai   0:02  0.20% imapd
27091 _cyrus-50   10M   12M sleepgetblk   0:02  0.20% imapd
15191 _cyrus 20 1740K 3664K sleepselect   0:00  0.20% imapd
17387 _cyrus 20 2216K 4056K sleepselect   0:02  0.15% imapd
25614 _cyrus 20 2056K 3952K sleepselect   0:02  0.15% imapd

# iostat 5 5
  ttysd0 sd1 cd0
fd0 cpu
 tin tout  KB/t t/s MB/s   KB/t t/s MB/s   KB/t t/s MB/s   KB/t t/s
MB/s  us ni sy in id
   08 10.77  18 0.19  14.30 242 3.38   0.00   0 0.00   0.00   0
0.00  18  0  5  1 77
   0   54  9.93  34 0.33  15.00 956 14.00   0.00   0 0.00   0.00   0
0.00   3  0 13  0 84
   0   18 12.07  80 0.95  14.33 615 8.61   0.00   0 0.00   0.00   0
0.00   7  0 15  1 77
   0   18  8.68   9 0.08  14.03 575 7.88   0.00   0 0.00   0.00   0
0.00   1  0  6  1 92
   0   18 10.21  30 0.30  13.26 709 9.19   0.00   0 0.00   0.00   0
0.00   1  0  9  2 88

# vmstat 5 5
 procs   memorypagedisks traps cpu
 r b wavmfre   flt  re  pi  po  fr  sr sd0 sd1  int   sys   cs us sy id
 7 2 01206040 321392  1719   0   0  11   0 174  12 430  573 4294967005
 446 18  6 77
 1 7 01200372 329816  7117   0   0   0   0   0  28 1497 1488  8354 1469 11 17 72
 0 2 01198584 331560  4741   0   0   0   0   0  15 1770 1390  5770 1202  3 14 83
 0 2 01189932 341820  2549   0   0   0   0   0  37 778  800  2981  592  2  8 90
 1 5 01196632 334756  5348   0   0   0   0   0  56 1622 1455  6145 1247  3 18 78


/etc/sysctl.conf :

kern.maxproc=1024
kern.maxfiles=8000
net.inet.tcp.sendspace=65535
net.inet.tcp.recvspace=65535

# mount
/dev/sd0a on / type ffs (local)
/dev/sd0i on /home type ffs (local, nodev, softdep)
/dev/sd0d on /tmp type ffs (local, nodev, nosuid)
/dev/sd0f on /usr type ffs (local, nodev)
/dev/sd0e on /var type ffs (local, nodev, nosuid, softdep)
/dev/sd0h on /var/spool type ffs (local, noatime, nodev, nosuid, softdep)
/dev/sd1a on /das0 type ffs (local, noatime, nodev, nosuid, softdep)

# df -h
Filesystem SizeUsed   Avail Capacity  Mounted on
/dev/sd0a  500M   99.9M375M21%/
/dev/sd0i 50.7G   42.4G5.8G88%/home
/dev/sd0d  124M   16.0K117M 0%/tmp
/dev/sd0f  7.9G3.3G4.2G44%/usr
/dev/sd0e  2.0G344M1.5G18%/var
/dev/sd0h  3.9G   55.7M3.7G 1%/var/spool
/dev/sd1a  550G   78.3G444G15%/das0

# cat /etc/cyrus.conf
START {
  # do not delete this entry!
  recover   cmd="ctl_cyrusdb -r"

  # this 

Re: How do I configure sendmail?

2007-10-16 Thread Samuel Moñux
2007/10/16, Sunnz <[EMAIL PROTECTED]>:
> Ohh thanks for the tip.
>
> So does sendmail supports smtp over ssl? When I restart sendmail I got
> something like:

Honestly, don't know. I thought the main problem was authenticating to
the relay, not doing smtp over ssl (you ISP doesn't support standard
STARTTLS in port 25?). Look at comp.mail.sendmail archive.

Best regards,
Samuel



Re: Cyrus IMAP performance problems [Long]

2007-10-16 Thread Samuel Moñux
It seems that Francesco pointed me in the right direction in a private
message. Cyrus in OpenBSD can't use mmap since it assumes that changes
in an mmaped file are inmediately seen by VFS system calls. OpenBSD
hasn't an unified buffer and page cache so this semantic requirement
of mmap isn't met (requires and explicit msync()). Another
implementation based in lseek and xmalloc must be used, which performs
much worse than the mmap based.

I think this explains memory usage, and the high number of IOPS.

Now, I must switch the IMAP server or the operating system.

Thanks to everyone, specially Francesco.

2007/10/16, Samuel Moqux <[EMAIL PROTECTED]>:
> Hello everyone,
>
> I running into some problems with a Cyrus Imap server. A year ago I
> asked about sizing the server and it was defined as "overkill" (Dell
> 1850, Perc 4e/DC, 4 300GB 10krpm disks in Raid 10) for 300 users, but
> now I'm suffering of really high loads and performance problems. I
> think it's due I/O contention (more than a thousand t/s in iostat),
> but don't know why Cyrus is so intensive in I/O that this hardware
> can't handle it. Memory consumption is also very high.
>
> When I started with ~60 users everything was ok, but after surpasing
> the one hundred, performance problems arised. I added a imap proxy
> which sited between the server and the webmail(Squirrel) and improved
> overall performance, but now we have 215 users and the server has
> severe problems during peak hours, when load can reach values of 12 or
> more. Users perceive degraded response times.
>
> Top usually lota of imap processes waiting for getblk, biowait, lockf,
> or select. Vmstat also shows always proceses blocked waiting for I/O.
>
> I had to tweak default kernel values because server crashed when
> importing old mailboxes:
> maxusers64  # estimated number of users
> option NKMEMPAGES_MAX=65535
> option BUFCACHEPERCENT=15
>
> Cyrus was compiled from ports, with default options. The server is
> running Sendmail and a lightly loaded openldap.
>
> I don't know where to look at or what to tweak. Any ideas will be welcome.
>
> Best regards.
>
> Dmesg and other relevant data (non peak hours).
>
> load averages: 10.36,  6.97,  5.79
> 17:59:50
> 259 processes: 258 idle, 1 on processor
> CPU states:  3.7% user,  0.0% nice, 20.6% system,  1.9% interrupt, 73.8%
idle
> Memory: Real: 1135M/1702M act/tot  Free: 312M  Swap: 47M/2196M used/tot
>
>   PID USERNAME PRI NICE  SIZE   RES STATEWAIT TIMECPU COMMAND
>  9503 _cyrus 20 2716K 4392K sleepselect   0:08  1.12% imapd
> 16618 _cyrus-50 3976K 5728K sleepbiowai   0:08  1.03% imapd
> 14158 _cyrus 20 3544K 5568K sleepselect   0:01  0.68% imapd
> 23082 _cyrus-50   11M   11M sleepgetblk   0:06  0.54% imapd
>  4397 _cyrus 20 1664K 3396K sleepselect   0:00  0.54% imapd
>   831 _cyrus-50   14M   16M sleepgetblk   0:01  0.34% imapd
> 30096 _cyrus 20 6136K 7464K sleepselect   0:03  0.29% imapd
> 10307 _cyrus-50 5572K 5908K sleepgetblk   0:00  0.29% imapd
> 28758 _cyrus-50 7664K 9412K sleepbiowai   0:02  0.20% imapd
> 27091 _cyrus-50   10M   12M sleepgetblk   0:02  0.20% imapd
> 15191 _cyrus 20 1740K 3664K sleepselect   0:00  0.20% imapd
> 17387 _cyrus 20 2216K 4056K sleepselect   0:02  0.15% imapd
> 25614 _cyrus 20 2056K 3952K sleepselect   0:02  0.15% imapd
>
> # iostat 5 5
>   ttysd0 sd1 cd0
> fd0 cpu
>  tin tout  KB/t t/s MB/s   KB/t t/s MB/s   KB/t t/s MB/s   KB/t t/s
> MB/s  us ni sy in id
>08 10.77  18 0.19  14.30 242 3.38   0.00   0 0.00   0.00   0
> 0.00  18  0  5  1 77
>0   54  9.93  34 0.33  15.00 956 14.00   0.00   0 0.00   0.00   0
> 0.00   3  0 13  0 84
>0   18 12.07  80 0.95  14.33 615 8.61   0.00   0 0.00   0.00   0
> 0.00   7  0 15  1 77
>0   18  8.68   9 0.08  14.03 575 7.88   0.00   0 0.00   0.00   0
> 0.00   1  0  6  1 92
>0   18 10.21  30 0.30  13.26 709 9.19   0.00   0 0.00   0.00   0
> 0.00   1  0  9  2 88
>
> # vmstat 5 5
>  procs   memorypagedisks traps cpu
>  r b wavmfre   flt  re  pi  po  fr  sr sd0 sd1  int   sys   cs us sy
id
>  7 2 01206040 321392  1719   0   0  11   0 174  12 430  573 4294967005
>  446 18  6 77
>  1 7 01200372 329816  7117   0   0   0   0   0  28 1497 1488  8354 1469 11
17 72
>  0 2 01198584 331560  4741   0   0   0   0   0  15 1770 1390  5770 1202  3
14 83
>  0 2 01189932 341820  2549   0   0   0   0   0  37 778  800  2981  592  2  8
90
>  1 5 01196632 334756  5348   0   0   0   0   0  56 1622 1455  6145 1247  3
18 78
>
>
> /etc/sysctl.conf :
>
> kern.maxproc=1024
> kern.maxfiles=8000
> net.inet.tcp.sendspace=65535
> net.inet.tcp.recvspace=65535
>
> # mount
> /dev/sd0a on / type ffs (local)
> /dev/sd0i on /home type ffs (local, nodev, softdep)
> /