Bug Tracker item #3536088, was opened at 2012-06-18 11:27 Message generated for change (Comment added) made by drpsycho You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126467&aid=3536088&group_id=250683
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: daemon Group: None Status: Open Resolution: None Priority: 9 Private: No Submitted By: polaris1 (polaris1) Assigned to: Nobody/Anonymous (nobody) Summary: dot stuffing bug Initial Comment: I tried this in both 3.9.0 and in 3.10.1 with similar results. I run dspam in client/daemon mode ( daemon is 3.10.1, clients I tried both 3.9.0 and 3.10.1 ) DSPAM is invoked from C with these arguments: dspam --client --user myuser --deliver=innocent,spam --stdout while the message is being fed in through stdin. I have a copy of the message and the relevant part is here: --_000_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Dear Daniela, According to Michela, I will send the loading schedule for next week to you= . Hello World, how are you quoted printable only allows 76 characters per line and as it happens, that line ends with a dot on the 77th character thus putting it on a new line ( what are the odds, right ? ) situation can be reproduced like this: cat messagefile | dspam --client --user myuser --deliver=innocent,spam --stdout For that message, the output will stop at 'next week to you='. If I modify the message to stuff the dot manually (..) it will output: --_000_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Dear Daniela, According to Michela, I will send the loading schedule for next week to you= .. Hello World, how are you Obviously something is wrong. I see that dspam has dot stuffing mechanisms in client.c but it's obviously not working. Is it because of the quoted printable ? ---------------------------------------------------------------------- Comment By: Edwin Eefting (drpsycho) Date: 2013-03-21 09:00 Message: After hours of digging though the code i found the bug and came up with a workaround: The problem is basically that the deamon removes the dotstuffing, but fogets to add it in the message that is send back. This cause the client to see a single dot and stop reading the message, resulting in a truncated message. However, if the server would re-add the dotstuffing, the client wouldt remove it, resulting in extra dots send to stdout. I couldn't find an easy way of fixing in the right way(tm), since i dont know enough of the dspam architecture, and i since we want to use this in production right away the risk of breaking stuff was too big. (it might even involve a rewrite of some code) So i came with this workaround: https://github.com/psy0rz/stuff/blob/master/patches/dspam-dotstuffing-workaround.patch This fixes the client, but fixing the deamon the right way involves some more steps. I think this should be easy to do for the core developers since the source of the problem is known now. (i've added some FIXME's and comments in the appropriate places) Edwin Eefting ---------------------------------------------------------------------- Comment By: Edwin Eefting (drpsycho) Date: 2013-03-21 05:26 Message: We also ran against this problem. I definitely think this is a bug: According to the smtp rfc, the dot stuffing/destuffing is part of the smtp-conversation (http://tools.ietf.org/html/rfc5321#section-4.5.2) Dspam usually happens AFTER the conversation and should expect whole messages without worrying about any smtp protocol handling. (such as stuffing/destuffing) There is no need for dspam to know about the special meaning of a single dot on an empty line. (in smtp this in the end-of-message indicator) When calling dspam without --client everything is fine and somehow magically works, but when using the --client option the bug is triggered somehow. This inconsistent behaviour only contributes to the fact that something is not right. Tested this with dspam version 3.10.2 Edwin Eefting Edwin Edwin Eefting ---------------------------------------------------------------------- Comment By: Paul van Tilburg (mozillion) Date: 2013-02-25 07:32 Message: I wonder what --client really should have to do with dot stuffing? I mean, --client is just about how it is handle in the backend (either run a sole instance or use the client/daemon model). In my case I use a pipe to forward the (not dot-stuffed message) in combination with --client for scalability reasons, but then suddenly, dot stuffing is not applied and the message gets truncated. Without --client it all works fine. I find this weird and, I hope we can agree on this, at least undocumented behaviour. ---------------------------------------------------------------------- Comment By: polaris1 (polaris1) Date: 2012-06-18 17:18 Message: When running with --stdout, it's obvious that there should not be any dot stuffing happening, so maybe this behavior is ok after all. The question is, why is DSPAM concerned about dots as a message end marker, when the daemon replies. In daemon.c we have: /* Send a terminating '.' if --stdout in 'dspam' mode */ if (ATX->sockfd_output) { if (send_socket(TTX, ".")<=0) goto CLOSE; and in client.c there is while(line != NULL && strcmp(line, ".")) { Why can't that code be removed entirely from daemon.c and the line in client.c turned into: while(line != NULL) { It would take care of the problem it seems. What am I missing ? ---------------------------------------------------------------------- Comment By: polaris1 (polaris1) Date: 2012-06-18 14:54 Message: So yeah, I think we can safely call this a bug now. 1. dspam client properly does dot stuffing 2. dspam daemon properly does dot de-stuffing when analyzing the message 3. dspam daemon does not re-stuff the dot when sending the reply back to the client in client.c we have this line: while(line != NULL && strcmp(line, ".")) { which automatically cuts the message off at the first lone dot, unprocessed by #3 above ---------------------------------------------------------------------- Comment By: polaris1 (polaris1) Date: 2012-06-18 14:40 Message: Yeah, it's definitely within the daemon code where there is no dot stuffing code. Using TCPDUMP here's the output of the outgoing packets towards the daemon: Dear Daniela, According to Michela, I will send the loading schedule for next week to you= .. Hello World, how are you . and Incoming from Daemon to Client: Dear Daniela, According to Michela, I will send the loading schedule for next week to you= . Hello World, how are you . It fails to stuff it properly on the way back... ---------------------------------------------------------------------- Comment By: polaris1 (polaris1) Date: 2012-06-18 14:30 Message: Furthermore, I added this code in client.c in order to make sure the dot stuffing code is working: r = send(TTX.sockfd, buf+t, buflen - t, 0); strncpy(sbuf, buf+t, r); sbuf[r]='\0'; printf ("SENT: %s\n\n\n", sbuf); And it prints properly the following: SENT: "_005_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_"; type="multipart/alternative" MIME-Version: 1.0 --_005_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_ Content-Type: multipart/alternative; boundary="_000_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_" --_000_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Dear Daniela, According to Michela, I will send the loading schedule for next week to you= .. Hello World, how are you So obviously the client function detects the lone dot, stuffs it and sends it over properly. There has to be a bug in the daemon code ? ---------------------------------------------------------------------- Comment By: polaris1 (polaris1) Date: 2012-06-18 12:49 Message: There is no MTA involved here. dspam in --client mode, according to my interpretation of client.c should perform dot stuffing. If I read that incorrectly, can you tell me what all that code I pasted in client.c is supposed to do ? ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2012-06-18 12:43 Message: Yes. DSPAM binary is responsible doing the dot stuffing WHEN sending to a delivery agent OR sending to the DSPAM daemon. But in your case you feed the DSPAM binary with wrong data. So DSPAM stops processing the message after you just send ONE dot. It is the same as if you would open a connection to port 25 on your system and post something like this: < 220 mail.example.com ESMTP Postfix (2.9.3)\r > EHLO mail.example.com\r < 250-mail.example.com\r < 250-PIPELINING\r < 250-SIZE 52428800\r < 250-ETRN\r < 250-STARTTLS\r < 250-AUTH PLAIN LOGIN DIGEST-MD5 CRAM-MD5\r < 250-AUTH=PLAIN LOGIN DIGEST-MD5 CRAM-MD5\r < 250-ENHANCEDSTATUSCODES\r < 250-8BITMIME\r < 250 DSN\r > MAIL FROM:<r...@example.com>\r < 250 2.1.0 Ok\r > RCPT TO:<ste...@bajic.ch>\r < 250 2.1.5 Ok\r > DATA\r < 354 End data with <CR><LF>.<CR><LF>\r > Return-Path: <r...@example.com>\r > Delivered-To: ste...@bajic.ch\r > To: <ste...@bajic.ch>\r > From: <r...@example.com>\r > MIME-Version: 1.0\r > Content-Type: multipart/related;\r > boundary="_004_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_";\r > type="multipart/alternative"\r > Subject: Testing\r > \r > --_004_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_\r > boundary="_000_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_"\r > \r > --_000_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_\r > Content-Type: text/plain; charset="us-ascii"\r > Content-Transfer-Encoding: quoted-printable\r > \r > Dear Daniela,\r > \r > According to Michela, I will send the loading schedule for next week to you=\r > .\r > \r < 250 2.0.0 Ok: queued as 7FDC0A3C170\r > Hello World, how are you\r < 502 5.5.2 Error: command not recognized\r > RSET\r < 250 2.0.0 Ok\r > QUIT\r < 221 2.0.0 Bye\r Assuming you run Postfix, would you say that this is a Postfix bug? I personally would say that this is NOT a bug. Now on the other hand if you have a MTA that is getting the above mail and then sends that message to DSPAM over SMTP then the MTA is responsible to do the dot stuffing, while DSPAM is responsible for doing the dot destuffing. When the message is inside DSPAM (the client) and the client is sending the data to the DSPAM daemon (because of the use of --client) then the DSPAM client is doing dot stuffing while the DSPAM daemon is doing the dot destuffing. ---------------------------------------------------------------------- Comment By: polaris1 (polaris1) Date: 2012-06-18 12:06 Message: Hmm, I saw this in client.c. I thought DSPAM client would take care of that: * DESCRIPTION * connect to a dspam daemon socket and attempt to process a message * this function is called by the dspam agent when --client is specified /* * fill buf with partial msg, replacing \n with \r\n * and do dot stuffing, if needed. */ /* take care of dot stuffing \n */ if (message->data[i + 1] && message->data[i + 1] == '.') { buf[buflen] = '\n'; buflen++; buf[buflen] = '.'; buflen++; buf[buflen] = '.'; buflen++; i += 2; continue; } I haven't really debugged the code, but it seems there is code there that is supposed to do dot stuffing... ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2012-06-18 11:54 Message: I am not really sure if this is a bug or not. Actually the application sending data to DSPAM is responsible for doing the dot stuffing. In your case you use cat to pipe the message to DSPAM. So 'cat' should do the dot stuffing. Correct me if you see that differently. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126467&aid=3536088&group_id=250683 ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar _______________________________________________ Dspam-devel mailing list Dspam-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspam-devel
