Re: 421 4.3.0 deliver: Trying to unput wrong character

2017-07-24 Thread Ken Murchison 
If the body of the message contains a NUL character, the the CTE MUST be 
"binary" NOT "8bit".


Cyrus WILL transcode a properly formatted MIME message on ingest, if 
necessary.



On 07/24/2017 12:14 PM, Mikhail T. wrote:

On 24.07.2017 10:41, Merlin Hartley wrote:

NUL characters are not permitted in e-mails …
https://helpdesk.pscs.co.uk/690163-Data-stream-contained-NUL-character

As the RFC for IMAP states:
https://tools.ietf.org/html/rfc3501#section-9
(3) The ASCII NUL character, %x00, MUST NOT be used at any time.

But the message did go through sendmail...

Ken Murchison wrote:
What is the Content-Transfer-Encoding of the email? Was the message 
Base64 encoded, or is the PNG included as raw binary data? 
Raw... Here are the full headers (the first line -- the From_ -- is 
removed by sed before feeding the rest into deliver):


From m...@example.com Mon Jul 24 00:53:00 2017
Received: from narawntapu.narawntapu (localhost [127.0.0.1])
by narawntapu.narawntapu (8.15.2/8.15.2) with ESMTPS id
v6O4r05T003160
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384
bits=256 verify=NO)
for ; Mon, 24 Jul 2017 00:53:00
-0400 (EDT)
(envelope-from mi@narawntapu.narawntapu)
Received: (from mi@localhost)
by narawntapu.narawntapu (8.15.2/8.15.2/Submit) id
v6O4r0c9003159;
Mon, 24 Jul 2017 00:53:00 -0400 (EDT)
(envelope-from mi)
Date: Mon, 24 Jul 2017 00:53:00 -0400 (EDT)
Message-Id: <201707240453.v6O4r0c9003159@narawntapu.narawntapu>
From: Cron Daemon 
To: mi@narawntapu.narawntapu
Mime-Version: 1.0
Content-Type: image/png; charset=binary
Content-Transfer-Encoding: 8bit
Subject: Cron  cat foo.png

Should sendmail have re-encoded with something other than 8bit? I 
don't like doing that, because it increases the filesizes to no 
obvious benefit... Thanks!


-mi




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


--
Kenneth Murchison
Cyrus Development Team
FastMail Pty Ltd


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: 421 4.3.0 deliver: Trying to unput wrong character

2017-07-24 Thread Joseph Brennan 
> This is one of those weird inconsistencies between IMAP and SMTP protocols …
> as far as I remember, the NUL character restriction only applies to the IMAP
> protocol … so your MTA (e.g. sendmail) is not breaking the SMTP RFC by
> transferring them.

True, but it's breaking the Internet Message Format RFC, namely RFC
2822. Under "Obsolete Syntax" includes this:

   Finally, certain characters that were formerly allowed in messages
   appear in this section.  The NUL character (ASCII value 0) was once
   allowed, but is no longer for compatibility reasons.

Compatibility with POP and IMAP, presumably.

Sendmail seems to be following still the old "be generous in what you
accept" maxim, and as a result it accepts something that modern
software cannot read-- just in case you have some odd way of reading
mail that can handle it.


-- 
Joseph Brennan
Columbia University


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: 421 4.3.0 deliver: Trying to unput wrong character

2017-07-24 Thread Mikhail T. 

On 24.07.2017 12:34, Merlin Hartley wrote:
In my Exim configuration I have some code which strips the NUL 
characters from externally received messages and prohibits my local 
users from submitting messages which contain them - you could probably 
come-up with a similar thing for sendmail.
This may be useful: 
http://info-cyrus.andrew.cmu.narkive.com/Rro4Nn9M/554-message-contains-nul-characters
The link talks about stripping the offending character -- which would 
break the content in some cases (including mine).


Maybe, it is time deliver is modified to do the right thing -- and 
re-encode if it really has to? Or, perhaps, imapd can do the re-encoding 
on the fly so that the images on the disk can keep these characters the 
way /var/mail/mboxes can?


   -mi


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: 421 4.3.0 deliver: Trying to unput wrong character

2017-07-24 Thread Merlin Hartley 
Hi Mikhail

This is one of those weird inconsistencies between IMAP and SMTP protocols … as 
far as I remember, the NUL character restriction only applies to the IMAP 
protocol … so your MTA (e.g. sendmail) is not breaking the SMTP RFC by 
transferring them.
In my Exim configuration I have some code which strips the NUL characters from 
externally received messages and prohibits my local users from submitting 
messages which contain them - you could probably come-up with a similar thing 
for sendmail.
This may be useful: 
http://info-cyrus.andrew.cmu.narkive.com/Rro4Nn9M/554-message-contains-nul-characters

Merlin
--
Merlin Hartley
Computer Officer
MRC Mitochondrial Biology Unit
Cambridge, CB2 0XY
United Kingdom

> On 24 Jul 2017, at 17:14, Mikhail T.  wrote:
> 
> On 24.07.2017 10:41, Merlin Hartley wrote:
>> NUL characters are not permitted in e-mails … 
>> https://helpdesk.pscs.co.uk/690163-Data-stream-contained-NUL-character 
>> 
>> 
>> As the RFC for IMAP states:
>> https://tools.ietf.org/html/rfc3501#section-9 
>> 
>> (3) The ASCII NUL character, %x00, MUST NOT be used at any time.
> But the message did go through sendmail...
> 
> Ken Murchison wrote:
>> What is the Content-Transfer-Encoding of the email? Was the message Base64 
>> encoded, or is the PNG included as raw binary data?
> Raw... Here are the full headers (the first line -- the From_ -- is removed 
> by sed before feeding the rest into deliver):
> From m...@example.com   Mon Jul 24 00:53:00 2017
> Received: from narawntapu.narawntapu (localhost [127.0.0.1])
> by narawntapu.narawntapu (8.15.2/8.15.2) with ESMTPS id v6O4r05T003160
> (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 
> verify=NO)
> for  ; 
> Mon, 24 Jul 2017 00:53:00 -0400 (EDT)
> (envelope-from mi@narawntapu.narawntapu 
> )
> Received: (from mi@localhost)
> by narawntapu.narawntapu (8.15.2/8.15.2/Submit) id v6O4r0c9003159;
> Mon, 24 Jul 2017 00:53:00 -0400 (EDT)
> (envelope-from mi)
> Date: Mon, 24 Jul 2017 00:53:00 -0400 (EDT)
> Message-Id: <201707240453.v6O4r0c9003159@narawntapu.narawntapu> 
> 
> From: Cron Daemon  
> To: mi@narawntapu.narawntapu 
> Mime-Version: 1.0
> Content-Type: image/png; charset=binary
> Content-Transfer-Encoding: 8bit
> Subject: Cron  cat foo.png
> Should sendmail have re-encoded with something other than 8bit? I don't like 
> doing that, because it increases the filesizes to no obvious benefit... 
> Thanks!
> -mi


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: 421 4.3.0 deliver: Trying to unput wrong character

2017-07-24 Thread Mikhail T. 

On 24.07.2017 10:41, Merlin Hartley wrote:

NUL characters are not permitted in e-mails …
https://helpdesk.pscs.co.uk/690163-Data-stream-contained-NUL-character

As the RFC for IMAP states:
https://tools.ietf.org/html/rfc3501#section-9
(3) The ASCII NUL character, %x00, MUST NOT be used at any time.

But the message did go through sendmail...

Ken Murchison wrote:
What is the Content-Transfer-Encoding of the email? Was the message 
Base64 encoded, or is the PNG included as raw binary data? 
Raw... Here are the full headers (the first line -- the From_ -- is 
removed by sed before feeding the rest into deliver):


From m...@example.com  Mon Jul 24 00:53:00 2017
   Received: from narawntapu.narawntapu (localhost [127.0.0.1])
by narawntapu.narawntapu (8.15.2/8.15.2) with ESMTPS id
   v6O4r05T003160
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384
   bits=256 verify=NO)
for ; Mon, 24 Jul 2017 00:53:00
   -0400 (EDT)
(envelope-from mi@narawntapu.narawntapu)
   Received: (from mi@localhost)
by narawntapu.narawntapu (8.15.2/8.15.2/Submit) id
   v6O4r0c9003159;
Mon, 24 Jul 2017 00:53:00 -0400 (EDT)
(envelope-from mi)
   Date: Mon, 24 Jul 2017 00:53:00 -0400 (EDT)
   Message-Id: <201707240453.v6O4r0c9003159@narawntapu.narawntapu>
   From: Cron Daemon 
   To: mi@narawntapu.narawntapu
   Mime-Version: 1.0
   Content-Type: image/png; charset=binary
   Content-Transfer-Encoding: 8bit
   Subject: Cron  cat foo.png

Should sendmail have re-encoded with something other than 8bit? I don't 
like doing that, because it increases the filesizes to no obvious 
benefit... Thanks!


   -mi


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: 421 4.3.0 deliver: Trying to unput wrong character

2017-07-24 Thread Ken Murchison 
What is the Content-Transfer-Encoding of the email?  Was the message 
Base64 encoded, or is the PNG included as raw binary data?




On 07/24/2017 01:19 AM, Mikhail T. wrote:


My procmail is configured to feed incoming mail into deliver:

:0wr
|sed 1d | /opt/cyrus/bin/deliver mi

Recently I received an e-mail consisting of a PNG-file -- not an 
attachment, the body of the e-mail had content-type image/png. That 
message ended up in my /var/mail/mi mailbox -- because deliver would 
not accept it:


421 4.3.0 deliver: Trying to unput wrong character

Trying to upload the same message using mbox2imap, I get:

mbox2imap: (alert) IMAP protocol error
mbox2imap: (alert) Protocol response: NO
mbox2imap: (alert) Diagnostic message: ['Message contains NUL
characters']

Why is deliver saying this, and what can I do to be able to receive 
such mail through Cyrus the way I was able to receive them through 
regular Unix mail for years? Thank you!


-mi




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


--
Kenneth Murchison
Cyrus Development Team
FastMail Pty Ltd


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

421 4.3.0 deliver: Trying to unput wrong character

2017-07-23 Thread Mikhail T. 

My procmail is configured to feed incoming mail into deliver:

   :0wr
   |sed 1d | /opt/cyrus/bin/deliver mi

Recently I received an e-mail consisting of a PNG-file -- not an 
attachment, the body of the e-mail had content-type image/png. That 
message ended up in my /var/mail/mi mailbox -- because deliver would not 
accept it:


   421 4.3.0 deliver: Trying to unput wrong character

Trying to upload the same message using mbox2imap, I get:

   mbox2imap: (alert) IMAP protocol error
   mbox2imap: (alert) Protocol response: NO
   mbox2imap: (alert) Diagnostic message: ['Message contains NUL
   characters']

Why is deliver saying this, and what can I do to be able to receive such 
mail through Cyrus the way I was able to receive them through regular 
Unix mail for years? Thank you!


   -mi


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus