Re: vsz_limit

2020-11-06 Thread Joan Moreau

SOrry, my mistake, the conversion type was wrong.

So restrict_get_process_size is indeed consistent with vsz_limit

Now, for the memory usage of the process, getrusage gives only the /max/ 
of the memory used, not the current


THe only way I found is to fopen("/proc/self/status") and read the 
correct line. Do you have a better way ?


thank you

On 2020-11-06 14:16, Joan Moreau wrote:


ok found it,

However, it returns me some random number. Maybe I am missing something

On 2020-11-06 13:57, Aki Tuomi wrote:
Duh... src/lib/restrict-process-size.h

Should be in the installed include files as well,

/usr/include/dovecot/restrict-process-size.h

Aki

On 06/11/2020 15:56 Joan Moreau  wrote:

Hello
I can't find "src/lib/restrict.h" . Is it in dovecot source ?

On 2020-11-06 13:20, Aki Tuomi wrote: Seems I had forgotten that you 
can use src/lib/restrict.h, in particular, restrict_get_process_size() 
to figure out the limit. You can combine this with getrusage to find 
out current usage.


Aki

On 06/11/2020 13:26 Joan Moreau  wrote:

yes, will do so.
It would be nice however to be able to access the actual dovecot config 
from the plugin side


On 2020-11-04 06:46, Aki Tuomi wrote: You could also add it as setting 
for the fts_xapian plugin parameters?


Aki

On 04/11/2020 08:42 Joan Moreau  wrote:

For machines with low memory, I would like to detect how much ram 
remains available before starting indexing a mail, so I can commit 
everything on disk before the ram is exhausted (and break the process)
I tried to put a "fake" allocation to test if it fails, (so it can fail 
separately, and I can "if ram remaining is above X") but the is really 
not clean


On 2020-11-04 06:28, Aki Tuomi wrote:

On 04/11/2020 05:19 Joan Moreau  wrote:

Hello
I am looking for help around memory management
1 - How to get the current value of "vsz_limit" from inside a plugin 
(namely https://github.com/grosjo/fts-xapian/ ) , especially for 
indexer-worker
2 - Is there a macro or function in dovecot to get the remaining free 
memory from this vsz value ?

Thank you

Hi Joan,

I don't think there is a feasible way to access this setting as of now. 
Is there a reason you need this? We usually recommend setting 
vsz_limit=0 for indexer-worker.


Aki

Re: vsz_limit

2020-11-06 Thread Joan Moreau

ok found it,

However, it returns me some random number. Maybe I am missing something

On 2020-11-06 13:57, Aki Tuomi wrote:


Duh... src/lib/restrict-process-size.h

Should be in the installed include files as well,

/usr/include/dovecot/restrict-process-size.h

Aki

On 06/11/2020 15:56 Joan Moreau  wrote:

Hello
I can't find "src/lib/restrict.h" . Is it in dovecot source ?

On 2020-11-06 13:20, Aki Tuomi wrote: Seems I had forgotten that you 
can use src/lib/restrict.h, in particular, restrict_get_process_size() 
to figure out the limit. You can combine this with getrusage to find 
out current usage.


Aki

On 06/11/2020 13:26 Joan Moreau  wrote:

yes, will do so.
It would be nice however to be able to access the actual dovecot config 
from the plugin side


On 2020-11-04 06:46, Aki Tuomi wrote: You could also add it as setting 
for the fts_xapian plugin parameters?


Aki

On 04/11/2020 08:42 Joan Moreau  wrote:

For machines with low memory, I would like to detect how much ram 
remains available before starting indexing a mail, so I can commit 
everything on disk before the ram is exhausted (and break the process)
I tried to put a "fake" allocation to test if it fails, (so it can fail 
separately, and I can "if ram remaining is above X") but the is really 
not clean


On 2020-11-04 06:28, Aki Tuomi wrote:

On 04/11/2020 05:19 Joan Moreau  wrote:

Hello
I am looking for help around memory management
1 - How to get the current value of "vsz_limit" from inside a plugin 
(namely https://github.com/grosjo/fts-xapian/ ) , especially for 
indexer-worker
2 - Is there a macro or function in dovecot to get the remaining free 
memory from this vsz value ?

Thank you

Hi Joan,

I don't think there is a feasible way to access this setting as of now. 
Is there a reason you need this? We usually recommend setting 
vsz_limit=0 for indexer-worker.


Aki

Re: vsz_limit

2020-11-06 Thread Joan Moreau

Hello

I can't find "src/lib/restrict.h" . Is it in dovecot source ?

On 2020-11-06 13:20, Aki Tuomi wrote:

Seems I had forgotten that you can use src/lib/restrict.h, in 
particular, restrict_get_process_size() to figure out the limit. You 
can combine this with getrusage to find out current usage.


Aki

On 06/11/2020 13:26 Joan Moreau  wrote:

yes, will do so.
It would be nice however to be able to access the actual dovecot config 
from the plugin side


On 2020-11-04 06:46, Aki Tuomi wrote: You could also add it as setting 
for the fts_xapian plugin parameters?


Aki

On 04/11/2020 08:42 Joan Moreau  wrote:

For machines with low memory, I would like to detect how much ram 
remains available before starting indexing a mail, so I can commit 
everything on disk before the ram is exhausted (and break the process)
I tried to put a "fake" allocation to test if it fails, (so it can fail 
separately, and I can "if ram remaining is above X") but the is really 
not clean


On 2020-11-04 06:28, Aki Tuomi wrote:

On 04/11/2020 05:19 Joan Moreau  wrote:

Hello
I am looking for help around memory management
1 - How to get the current value of "vsz_limit" from inside a plugin 
(namely https://github.com/grosjo/fts-xapian/ ) , especially for 
indexer-worker
2 - Is there a macro or function in dovecot to get the remaining free 
memory from this vsz value ?

Thank you

Hi Joan,

I don't think there is a feasible way to access this setting as of now. 
Is there a reason you need this? We usually recommend setting 
vsz_limit=0 for indexer-worker.


Aki

Re: vsz_limit

2020-11-06 Thread Joan Moreau

yes, will do so.

It would be nice however to be able to access the actual dovecot config 
from the plugin side


On 2020-11-04 06:46, Aki Tuomi wrote:


You could also add it as setting for the fts_xapian plugin parameters?

Aki

On 04/11/2020 08:42 Joan Moreau  wrote:

For machines with low memory, I would like to detect how much ram 
remains available before starting indexing a mail, so I can commit 
everything on disk before the ram is exhausted (and break the process)
I tried to put a "fake" allocation to test if it fails, (so it can fail 
separately, and I can "if ram remaining is above X") but the is really 
not clean


On 2020-11-04 06:28, Aki Tuomi wrote:

On 04/11/2020 05:19 Joan Moreau  wrote:

Hello
I am looking for help around memory management
1 - How to get the current value of "vsz_limit" from inside a plugin 
(namely https://github.com/grosjo/fts-xapian/ ) , especially for 
indexer-worker
2 - Is there a macro or function in dovecot to get the remaining free 
memory from this vsz value ?

Thank you

Hi Joan,

I don't think there is a feasible way to access this setting as of now. 
Is there a reason you need this? We usually recommend setting 
vsz_limit=0 for indexer-worker.


Aki

Re: vsz_limit

2020-11-06 Thread Aki Tuomi
Duh... src/lib/restrict-process-size.h

Should be in the installed include files as well,

/usr/include/dovecot/restrict-process-size.h

Aki

> On 06/11/2020 15:56 Joan Moreau  wrote:
> 
> 
> Hello
> I can't find "src/lib/restrict.h" . Is it in dovecot source ?
> 
> 
> On 2020-11-06 13:20, Aki Tuomi wrote:
> > Seems I had forgotten that you can use src/lib/restrict.h, in particular, 
> > restrict_get_process_size() to figure out the limit. You can combine this 
> > with getrusage to find out current usage.
> > 
> > Aki
> > 
> > 
> > > On 06/11/2020 13:26 Joan Moreau  wrote:
> > > 
> > > 
> > > yes, will do so.
> > > It would be nice however to be able to access the actual dovecot config 
> > > from the plugin side
> > > 
> > > 
> > > 
> > > On 2020-11-04 06:46, Aki Tuomi wrote:
> > > > You could also add it as setting for the fts_xapian plugin parameters? 
> > > > 
> > > > Aki
> > > > 
> > > > 
> > > > 
> > > > > On 04/11/2020 08:42 Joan Moreau  wrote:
> > > > > 
> > > > > 
> > > > > For machines with low memory, I would like to detect how much ram 
> > > > > remains available before starting indexing a mail, so I can commit 
> > > > > everything on disk before the ram is exhausted (and break the process)
> > > > > I tried to put a "fake" allocation to test if it fails, (so it can 
> > > > > fail separately, and I can "if ram remaining is above X") but the is 
> > > > > really not clean
> > > > > 
> > > > > 
> > > > > 
> > > > > On 2020-11-04 06:28, Aki Tuomi wrote:
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > > On 04/11/2020 05:19 Joan Moreau  wrote:
> > > > > > > 
> > > > > > > 
> > > > > > > Hello
> > > > > > > I am looking for help around memory management
> > > > > > > 1 - How to get the current value of "vsz_limit" from inside a 
> > > > > > > plugin (namely https://github.com/grosjo/fts-xapian/ ) , 
> > > > > > > especially for indexer-worker
> > > > > > > 2 - Is there a macro or function in dovecot to get the remaining 
> > > > > > > free memory from this vsz value ?
> > > > > > > Thank you
> > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > > Hi Joan,
> > > > > > 
> > > > > > I don't think there is a feasible way to access this setting as of 
> > > > > > now. Is there a reason you need this? We usually recommend setting 
> > > > > > vsz_limit=0 for indexer-worker.
> > > > > > 
> > > > > > Aki
> > > > > >


Re: vsz_limit

2020-11-06 Thread Aki Tuomi
Seems I had forgotten that you can use src/lib/restrict.h, in particular, 
restrict_get_process_size() to figure out the limit. You can combine this with 
getrusage to find out current usage.

Aki

> On 06/11/2020 13:26 Joan Moreau  wrote:
> 
> 
> yes, will do so.
> It would be nice however to be able to access the actual dovecot config from 
> the plugin side
> 
> 
> 
> On 2020-11-04 06:46, Aki Tuomi wrote:
> > You could also add it as setting for the fts_xapian plugin parameters? 
> > 
> > Aki
> > 
> > 
> > > On 04/11/2020 08:42 Joan Moreau  wrote:
> > > 
> > > 
> > > For machines with low memory, I would like to detect how much ram remains 
> > > available before starting indexing a mail, so I can commit everything on 
> > > disk before the ram is exhausted (and break the process)
> > > I tried to put a "fake" allocation to test if it fails, (so it can fail 
> > > separately, and I can "if ram remaining is above X") but the is really 
> > > not clean
> > > 
> > > 
> > > 
> > > On 2020-11-04 06:28, Aki Tuomi wrote:
> > > > 
> > > > 
> > > > 
> > > > > On 04/11/2020 05:19 Joan Moreau  wrote:
> > > > > 
> > > > > 
> > > > > Hello
> > > > > I am looking for help around memory management
> > > > > 1 - How to get the current value of "vsz_limit" from inside a plugin 
> > > > > (namely https://github.com/grosjo/fts-xapian/ ) , especially for 
> > > > > indexer-worker
> > > > > 2 - Is there a macro or function in dovecot to get the remaining free 
> > > > > memory from this vsz value ?
> > > > > Thank you
> > > > > 
> > > > > 
> > > > 
> > > > Hi Joan,
> > > > 
> > > > I don't think there is a feasible way to access this setting as of now. 
> > > > Is there a reason you need this? We usually recommend setting 
> > > > vsz_limit=0 for indexer-worker.
> > > > 
> > > > Aki
> > > >


Re: vsz_limit

2020-11-04 Thread Joan Moreau
For machines with low memory, I would like to detect how much ram 
remains available before starting indexing a mail, so I can commit 
everything on disk before the ram is exhausted (and break the process)


I tried to put a "fake" allocation to test if it fails, (so it can fail 
separately, and I can "if ram remaining is above X") but the is really 
not clean


On 2020-11-04 06:28, Aki Tuomi wrote:


On 04/11/2020 05:19 Joan Moreau  wrote:

Hello
I am looking for help around memory management
1 - How to get the current value of "vsz_limit" from inside a plugin 
(namely https://github.com/grosjo/fts-xapian/ ) , especially for 
indexer-worker
2 - Is there a macro or function in dovecot to get the remaining free 
memory from this vsz value ?

Thank you


Hi Joan,

I don't think there is a feasible way to access this setting as of now. 
Is there a reason you need this? We usually recommend setting 
vsz_limit=0 for indexer-worker.


Aki

Re: vsz_limit

2020-11-03 Thread Aki Tuomi
You could also add it as setting for the fts_xapian plugin parameters? 

Aki

> On 04/11/2020 08:42 Joan Moreau  wrote:
> 
> 
> For machines with low memory, I would like to detect how much ram remains 
> available before starting indexing a mail, so I can commit everything on disk 
> before the ram is exhausted (and break the process)
> I tried to put a "fake" allocation to test if it fails, (so it can fail 
> separately, and I can "if ram remaining is above X") but the is really not 
> clean
> 
> 
> 
> On 2020-11-04 06:28, Aki Tuomi wrote:
> > 
> > 
> > > On 04/11/2020 05:19 Joan Moreau  wrote:
> > > 
> > > 
> > > Hello
> > > I am looking for help around memory management
> > > 1 - How to get the current value of "vsz_limit" from inside a plugin 
> > > (namely https://github.com/grosjo/fts-xapian/ ) , especially for 
> > > indexer-worker
> > > 2 - Is there a macro or function in dovecot to get the remaining free 
> > > memory from this vsz value ?
> > > Thank you
> > > 
> > > 
> > 
> > Hi Joan,
> > 
> > I don't think there is a feasible way to access this setting as of now. Is 
> > there a reason you need this? We usually recommend setting vsz_limit=0 for 
> > indexer-worker.
> > 
> > Aki
> >


Re: vsz_limit

2020-11-03 Thread Aki Tuomi


> On 04/11/2020 05:19 Joan Moreau  wrote:
> 
> 
> Hello
> I am looking for help around memory management
> 1 - How to get the current value of "vsz_limit" from inside a plugin (namely 
> https://github.com/grosjo/fts-xapian/ ) , especially for indexer-worker
> 2 - Is there a macro or function in dovecot to get the remaining free memory 
> from this vsz value ?
> Thank you
> 
>

Hi Joan,

I don't think there is a feasible way to access this setting as of now. Is 
there a reason you need this? We usually recommend setting vsz_limit=0 for 
indexer-worker.

Aki


Re: vsz_limit

2020-11-03 Thread Aki Tuomi


> On 04/11/2020 05:19 Joan Moreau  wrote:
> 
> 
> Hello
> I am looking for help around memory management
> 1 - How to get the current value of "vsz_limit" from inside a plugin (namely 
> https://github.com/grosjo/fts-xapian/ ) , especially for indexer-worker
> 2 - Is there a macro or function in dovecot to get the remaining free memory 
> from this vsz value ?
> Thank you
> 
>


Re: vsz_limit is 18,447 PB

2014-09-24 Thread Dan Langille

On Sep 15, 2014, at 10:20 AM, Dan Langille d...@langille.org wrote:

 
 On Sep 8, 2014, at 6:38 PM, Jiri Bourek bou...@thinline.cz wrote:
 
 
 Background: I’m in the only user on this system, but this server is 
 accessed by my phone, my laptop, my tablet, and perhaps a web interface.
 
 
 
 But there are some interesting values when I look at the output of 
 doveconf.  Specifically, vsz_limit is 18,447 PB… yeah, that’s pretty big.
 
 service imap-login {
 vsz_limit = 18446744073709551615 B
 }
 
 
 According to your log it was process “im ap what ran out of memory. You 
 are showing configuration for imap-login - they are not the same thing, 
 see http://wiki2.dovecot.org/Design/Processes
 
 My errror, sorry.  OK, here’s imap:
 
 service imap {
  chroot =
  client_limit = 1
  drop_priv_before_exec = no
  executable = imap
  extra_groups =
  group =
  idle_kill = 0
  privileged_group =
  process_limit = 1024
  process_min_avail = 0
  protocol = imap
  service_count = 1
  type =
  unix_listener login/imap {
group =
mode = 0666
user =
  }
  user =
  vsz_limit = 18446744073709551615 B
 }
 
 Still the same value…  why not 265MB?
 
 Hm, I can only guess (someone with more knowledge of dovecot source code 
 probably knows better), but it seems like that value is magic number for 
 not set. My doveconf outputs the same thing on services which don't have 
 vsz_limit set explicitly, but when I list /proc/[pid]/limits on such a 
 service, the row called Max address space (which is - AFAIK - limit on 
 vsz) shows 2GB, a value of default_vsz_limit setting.
 
 I'd say don't overthink it, just put explicit vsz_limit = 512M into service 
 imap section (if your machine's RAM allows) and be done with it.
 
 For the record, the error has recurred a few times, so after today’s 
 incident, I added this to dovecot.conf, and restarted dovecot:
 
 service imap {
  vsz_limit = 512MB
 }
 
 FWIW, the server contains 8192 MB of RAM.

For the record, the problem has not recurred.

— 
Dan Langille



signature.asc
Description: Message signed with OpenPGP using GPGMail


Re: vsz_limit is 18,447 PB

2014-09-15 Thread Dan Langille

On Sep 8, 2014, at 6:38 PM, Jiri Bourek bou...@thinline.cz wrote:

 
 Background: I’m in the only user on this system, but this server is 
 accessed by my phone, my laptop, my tablet, and perhaps a web interface.
 
 
 
 But there are some interesting values when I look at the output of 
 doveconf.  Specifically, vsz_limit is 18,447 PB… yeah, that’s pretty big.
 
 service imap-login {
  vsz_limit = 18446744073709551615 B
 }
 
 
 According to your log it was process “im ap what ran out of memory. You 
 are showing configuration for imap-login - they are not the same thing, see 
 http://wiki2.dovecot.org/Design/Processes
 
 My errror, sorry.  OK, here’s imap:
 
 service imap {
   chroot =
   client_limit = 1
   drop_priv_before_exec = no
   executable = imap
   extra_groups =
   group =
   idle_kill = 0
   privileged_group =
   process_limit = 1024
   process_min_avail = 0
   protocol = imap
   service_count = 1
   type =
   unix_listener login/imap {
 group =
 mode = 0666
 user =
   }
   user =
   vsz_limit = 18446744073709551615 B
 }
 
 Still the same value…  why not 265MB?
 
 Hm, I can only guess (someone with more knowledge of dovecot source code 
 probably knows better), but it seems like that value is magic number for not 
 set. My doveconf outputs the same thing on services which don't have 
 vsz_limit set explicitly, but when I list /proc/[pid]/limits on such a 
 service, the row called Max address space (which is - AFAIK - limit on vsz) 
 shows 2GB, a value of default_vsz_limit setting.
 
 I'd say don't overthink it, just put explicit vsz_limit = 512M into service 
 imap section (if your machine's RAM allows) and be done with it.

For the record, the error has recurred a few times, so after today’s incident, 
I added this to dovecot.conf, and restarted dovecot:

service imap {
  vsz_limit = 512MB
}

FWIW, the server contains 8192 MB of RAM.



signature.asc
Description: Message signed with OpenPGP using GPGMail


Re: vsz_limit is 18,447 PB

2014-09-09 Thread Darac Marjal
On Mon, Sep 08, 2014 at 04:41:18PM -0700, Robert L Mathews wrote:
 On 9/8/14 3:38 PM, Jiri Bourek wrote: Hm, I can only guess (someone
 with more knowledge of dovecot source code
  probably knows better), but it seems like that value is magic number for
  not set. My doveconf outputs the same thing on services which don't
  have vsz_limit set explicitly, but when I list /proc/[pid]/limits on
  such a service, the row called Max address space (which is - AFAIK -
  limit on vsz) shows 2GB, a value of default_vsz_limit setting.
 
  I'd say don't overthink it, just put explicit vsz_limit = 512M into
  service imap section (if your machine's RAM allows) and be done with it.
 
 Right. According to my notes from when I upgraded to 2.x,
 18446744073709551615 means it's not set and that the default_vsz_limit
 will apply, which is 256 MB.

That seems sensible to me. 2^64 bytes is 18446744073709551616 (18.45EB),
so the number shown is (2^64 - 1). As a *limit* for a 64-bit program,
that seems about as close to no limit as you're going to get :)

 
 To avoid the problem the original poster had, I used:
 
  default_vsz_limit = 1 G
 
 -- 
 Robert L Mathews, Tiger Technologies, http://www.tigertech.net/


signature.asc
Description: Digital signature


Re: vsz_limit is 18,447 PB

2014-09-09 Thread Joseph Tam

On Tue, 9 Sep 2014, dovecot-requ...@dovecot.org wrote:


 vsz_limit = 18446744073709551615 B
}

Still the same value?  why not 265MB?


I guess this is Dovecot's approximation of infinity.  It's the largest value
that fits into a 64-bit signed integer (2^64-1).

Joseph Tam jtam.h...@gmail.com


Re: vsz_limit is 18,447 PB

2014-09-08 Thread Dan Langille

On Sep 7, 2014, at 4:39 AM, Jiri Bourek bou...@thinline.cz wrote:

 On 6.9.2014 22:38, Dan Langille wrote:
 I encountered this error on FreeBSD 9.3 with dovecot2-2.2.13_3
 
 Short version; vsz_limit is 18,447 PB and it ran out of RAM.  Suggestions 
 for change?
 
 Sep  6 03:39:32 mailjail dovecot: imap(dan): Panic: file imap-fetch.c: line 
 556 (imap_fetch_more): assertion failed: (ctx-client-output_cmd_lock == 
 NULL || ctx-client-output_cmd_lock == cmd)
 Sep  6 03:39:32 mailjail dovecot: imap(dan): Fatal: master: service(imap): 
 child 71153 killed with signal 6 (core not dumped - set service imap { 
 drop_priv_before_exec=yes })
 Sep  6 03:59:41 mailjail dovecot: imap(dan): Fatal: 
 pool_system_realloc(2097152): Out of memory
 Sep  6 03:59:41 mailjail dovecot: imap(dan): Fatal: master: service(imap): 
 child 67732 returned error 83 (Out of memory (service imap { vsz_limit=256 
 MB }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get 
 core dump)
 
 
 Check the message again - it says vsz_limit=256MB

Yes, I saw that…. I can’t explain why I didn’t question the rest of the output.

 
 Background: I’m in the only user on this system, but this server is accessed 
 by my phone, my laptop, my tablet, and perhaps a web interface.
 
 
 
 But there are some interesting values when I look at the output of doveconf. 
  Specifically, vsz_limit is 18,447 PB… yeah, that’s pretty big.
 
 service imap-login {
  vsz_limit = 18446744073709551615 B
 }
 
 
 According to your log it was process “im ap what ran out of memory. You are 
 showing configuration for imap-login - they are not the same thing, see 
 http://wiki2.dovecot.org/Design/Processes

My errror, sorry.  OK, here’s imap:

service imap {
  chroot =
  client_limit = 1
  drop_priv_before_exec = no
  executable = imap
  extra_groups =
  group =
  idle_kill = 0
  privileged_group =
  process_limit = 1024
  process_min_avail = 0
  protocol = imap
  service_count = 1
  type =
  unix_listener login/imap {
group =
mode = 0666
user =
  }
  user =
  vsz_limit = 18446744073709551615 B
}

Still the same value…  why not 265MB?

 Not sure if this behaviour is a bug or something that is to be expected when 
 you run out of VSZ limit in imap process - that needs to be answered by 
 someone more knowledgeable

It seems quite common:

 $ doveconf | grep vsz_limit
default_vsz_limit = 256 M
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
  vsz_limit = 18446744073709551615 B
[dan@mailjail:~] $ 

Thank you.


signature.asc
Description: Message signed with OpenPGP using GPGMail


Re: vsz_limit is 18,447 PB

2014-09-08 Thread Jiri Bourek



Background: I’m in the only user on this system, but this server is accessed by 
my phone, my laptop, my tablet, and perhaps a web interface.





But there are some interesting values when I look at the output of doveconf.  
Specifically, vsz_limit is 18,447 PB… yeah, that’s pretty big.

service imap-login {
  vsz_limit = 18446744073709551615 B
}



According to your log it was process “im ap what ran out of memory. You are 
showing configuration for imap-login - they are not the same thing, see 
http://wiki2.dovecot.org/Design/Processes


My errror, sorry.  OK, here’s imap:

service imap {
   chroot =
   client_limit = 1
   drop_priv_before_exec = no
   executable = imap
   extra_groups =
   group =
   idle_kill = 0
   privileged_group =
   process_limit = 1024
   process_min_avail = 0
   protocol = imap
   service_count = 1
   type =
   unix_listener login/imap {
 group =
 mode = 0666
 user =
   }
   user =
   vsz_limit = 18446744073709551615 B
}

Still the same value…  why not 265MB?


Hm, I can only guess (someone with more knowledge of dovecot source code 
probably knows better), but it seems like that value is magic number for 
not set. My doveconf outputs the same thing on services which don't 
have vsz_limit set explicitly, but when I list /proc/[pid]/limits on 
such a service, the row called Max address space (which is - AFAIK - 
limit on vsz) shows 2GB, a value of default_vsz_limit setting.


I'd say don't overthink it, just put explicit vsz_limit = 512M into 
service imap section (if your machine's RAM allows) and be done with it.


Re: vsz_limit is 18,447 PB

2014-09-08 Thread Robert L Mathews
On 9/8/14 3:38 PM, Jiri Bourek wrote: Hm, I can only guess (someone
with more knowledge of dovecot source code
 probably knows better), but it seems like that value is magic number for
 not set. My doveconf outputs the same thing on services which don't
 have vsz_limit set explicitly, but when I list /proc/[pid]/limits on
 such a service, the row called Max address space (which is - AFAIK -
 limit on vsz) shows 2GB, a value of default_vsz_limit setting.

 I'd say don't overthink it, just put explicit vsz_limit = 512M into
 service imap section (if your machine's RAM allows) and be done with it.

Right. According to my notes from when I upgraded to 2.x,
18446744073709551615 means it's not set and that the default_vsz_limit
will apply, which is 256 MB.

To avoid the problem the original poster had, I used:

 default_vsz_limit = 1 G

-- 
Robert L Mathews, Tiger Technologies, http://www.tigertech.net/


Re: vsz_limit is 18,447 PB

2014-09-07 Thread Jiri Bourek

On 6.9.2014 22:38, Dan Langille wrote:

I encountered this error on FreeBSD 9.3 with dovecot2-2.2.13_3

Short version; vsz_limit is 18,447 PB and it ran out of RAM.  Suggestions for 
change?

Sep  6 03:39:32 mailjail dovecot: imap(dan): Panic: file imap-fetch.c: line 556 
(imap_fetch_more): assertion failed: (ctx-client-output_cmd_lock == NULL || 
ctx-client-output_cmd_lock == cmd)
Sep  6 03:39:32 mailjail dovecot: imap(dan): Fatal: master: service(imap): 
child 71153 killed with signal 6 (core not dumped - set service imap { 
drop_priv_before_exec=yes })
Sep  6 03:59:41 mailjail dovecot: imap(dan): Fatal: 
pool_system_realloc(2097152): Out of memory
Sep  6 03:59:41 mailjail dovecot: imap(dan): Fatal: master: service(imap): 
child 67732 returned error 83 (Out of memory (service imap { vsz_limit=256 MB 
}, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core 
dump)



Check the message again - it says vsz_limit=256MB


Background: I’m in the only user on this system, but this server is accessed by 
my phone, my laptop, my tablet, and perhaps a web interface.





But there are some interesting values when I look at the output of doveconf.  
Specifically, vsz_limit is 18,447 PB… yeah, that’s pretty big.

service imap-login {
  vsz_limit = 18446744073709551615 B
}



According to your log it was process imap what ran out of memory. You 
are showing configuration for imap-login - they are not the same thing, 
see http://wiki2.dovecot.org/Design/Processes


Not sure if this behaviour is a bug or something that is to be expected 
when you run out of VSZ limit in imap process - that needs to be 
answered by someone more knowledgeable