Re: [Mairix-users] Mairix 0.22: Out of memory (at rfc822.c:459, -1168 bytes)

[Edward Burns]

> On Sun, 11 Dec 2016 23:14:21 +1100, Kim Vandry  said:

KV> On 2016-12-11 09:26 , Eric Wong wrote:
>> Edward Burns  wrote:
>>> I'll send you a scrubbed version of the file under separate cover.  This
>>> file gives the "Out of memory" error every time.  If you could use it to
>>> improve Marirx, I'd greatly appreciate it.
>> Sorry, short on time/resources here, maybe others on the list
>> can handle it.
>> 
>> Would you mind if I hosted it publically for others to download?

You can make the scrubbed version publically available.

KV> +1 to making it available for additional diagnostics. I had a brief look 
KV> at the source code the other day to see if anything stood out, but it 
KV> would be easier if it could be reproduced.

KV> Since I touched the source code last and the original author has no more 
KV> time or motivation to take care of mairix, I was hoping to take over the 
KV> project anyway (still no word from Richard on making this official, 
KV> handing over the web site, etc...). Therefore I am interested in fixing 
KV> this issue.

Ed
-- 
| edward.bu...@oracle.com | office: +1 407 458 0017
| 10 business days until planned start of JSF 2.3 Public Review
| 30 business days until DevNexus 2017
| 55 business days until JavaLand 2017

--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
___
Mairix-users mailing list
Mairix-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mairix-users

Re: [Mairix-users] Mairix 0.22: Out of memory (at rfc822.c:459, -1168 bytes)

[Kim Vandry]

On 2016-12-11 09:26 , Eric Wong wrote:
> Edward Burns  wrote:
>> I'll send you a scrubbed version of the file under separate cover.  This
>> file gives the "Out of memory" error every time.  If you could use it to
>> improve Marirx, I'd greatly appreciate it.
> Sorry, short on time/resources here, maybe others on the list
> can handle it.
>
> Would you mind if I hosted it publically for others to download?

+1 to making it available for additional diagnostics. I had a brief look 
at the source code the other day to see if anything stood out, but it 
would be easier if it could be reproduced.

Since I touched the source code last and the original author has no more 
time or motivation to take care of mairix, I was hoping to take over the 
project anyway (still no word from Richard on making this official, 
handing over the web site, etc...). Therefore I am interested in fixing 
this issue.

-kv

--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/xeonphi
___
Mairix-users mailing list
Mairix-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mairix-users

Re: [Mairix-users] Mairix 0.22: Out of memory (at rfc822.c:459, -1168 bytes)

[Eric Wong]

Edward Burns  wrote:
> I'll send you a scrubbed version of the file under separate cover.  This
> file gives the "Out of memory" error every time.  If you could use it to
> improve Marirx, I'd greatly appreciate it.

Sorry, short on time/resources here, maybe others on the list
can handle it.

Would you mind if I hosted it publically for others to download?

--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/xeonphi
___
Mairix-users mailing list
Mairix-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mairix-users

Re: [Mairix-users] Mairix 0.22: Out of memory (at rfc822.c:459, -1168 bytes)

[Edward Burns]

EW> Ed Burns  wrote:
>> > On Mon, 16 May 2011 09:47:41 -0700, Ed Burns  
>> > said:
>> 
EB> Hello Mairix users,
EB> I just built 0.22 from source on "SunOS 5.10 Generic_141445-09 i86pc
EB> i386 i86pc" and when I try to build the database I get 
>> 
EB> Out of memory (at rfc822.c:459, -1168 bytes)

EW> Sorry, I wasn't around for the original message; but that
EW> negative could indicate a signed 32-bit overflow.

> On Wed, 30 Nov 2016 19:02:37 +, Eric Wong  
> said:

EW> I'm new to the mairix source, but the problem could be having an
EW> email body which exceeds 2GB (but wow, do email servers allow
EW> such large messages?).

I don't think the mbox file in question was that large, only 24310
bytes.  There must have been some kind of bogus character in there.

In fact, looking through the file, I see a To: line that looks like
this:

To: 
  f...@bar.com

I'll send you a scrubbed version of the file under separate cover.  This
file gives the "Out of memory" error every time.  If you could use it to
improve Marirx, I'd greatly appreciate it.

Thanks,

Ed

-- 
| edward.bu...@oracle.com | office: +1 407 458 0017

--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/xeonphi
___
Mairix-users mailing list
Mairix-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mairix-users

Re: [Mairix-users] Mairix 0.22: Out of memory (at rfc822.c:459, -1168 bytes)

[Eric Wong]

Ed Burns  wrote:
> > On Mon, 16 May 2011 09:47:41 -0700, Ed Burns  
> > said:
> 
> EB> Hello Mairix users,
> EB> I just built 0.22 from source on "SunOS 5.10 Generic_141445-09 i86pc
> EB> i386 i86pc" and when I try to build the database I get 
> 
> EB> Out of memory (at rfc822.c:459, -1168 bytes)

Sorry, I wasn't around for the original message; but that
negative could indicate a signed 32-bit overflow.

I'm new to the mairix source, but the problem could be having an
email body which exceeds 2GB (but wow, do email servers allow
such large messages?).

Can you try the following patch to print out the values of "input"
and "input_len" before the failure?

diff --git a/rfc822.c b/rfc822.c
index b411f85..3986bf3 100644
--- a/rfc822.c
+++ b/rfc822.c
@@ -456,6 +456,7 @@ static char *unencode_data(struct msg_src *src, char 
*input, int input_len, cons
 
   encoding = decode_encoding_type(enc);
   end_input = input + input_len;
+  fprintf(stderr, "input=%p input_len=%d\n", input, input_len);
 
   /* All mime encodings result in expanded data, so this is guaranteed to
* safely oversize the output array */



> 2. re-index with the -v option, looking closely at the output.  Observe
> what file it was processing when the error is produced.
> 
> 3. redo steps 1 and 2, and verify it happens on the same file.
> 
> 4. delete that file
> 
> 5. redo steps 1 - 4 until all the indexing completes.

I suppose you can narrow it down to a particular email if you're
using an mbox.  Is the email body 2GB or more?   Thanks.

--
___
Mairix-users mailing list
Mairix-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mairix-users

Re: [Mairix-users] Mairix 0.22: Out of memory (at rfc822.c:459, -1168 bytes)

[Ed Burns]

> On Mon, 16 May 2011 09:47:41 -0700, Ed Burns  
> said:

EB> Hello Mairix users,
EB> I just built 0.22 from source on "SunOS 5.10 Generic_141445-09 i86pc
EB> i386 i86pc" and when I try to build the database I get 

EB> Out of memory (at rfc822.c:459, -1168 bytes)

EB> I've never run into this error before and I doubt the machine is
EB> actually out of memory.  In fact, I've tried it out on two separate
EB> hosts each running the same OS, both of these hosts are provided by
EB> my employer's IT department and seem to be reasonably well equipped.

EB> Does anyone have any suggestions about how I might get back to using
EB> Mairix?

I wanted to follow up on this, since I had the unpleasant experience of
running into this problem again, searching the web, and finding this
post with no resolution!

Here's how I resolved this.  It's not the most satisfying resolution but
it worked.

1. Remove the mairix_database and lock file, if necessary.

2. re-index with the -v option, looking closely at the output.  Observe
what file it was processing when the error is produced.

3. redo steps 1 and 2, and verify it happens on the same file.

4. delete that file

5. redo steps 1 - 4 until all the indexing completes.

Thanks,

Ed

-- 
| http://purl.oclc.org/NET/edburns/pgp/

--
___
Mairix-users mailing list
Mairix-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mairix-users