A NOTE has been added to this issue. ====================================================================== http://www.dbmail.org/mantis/view.php?id=371 ====================================================================== Reported By: blake Assigned To: ====================================================================== Project: DBMail Issue ID: 371 Category: Database layer Reproducibility: sometimes Severity: major Priority: normal Status: new target: ====================================================================== Date Submitted: 22-Jun-06 22:43 CEST Last Modified: 22-Jun-06 23:10 CEST ====================================================================== Summary: Some messages are truncated during insertion. Description: During insertion into the database, either via dbmail-smtp or IMAP append, messages are being truncated at non ASCII characters. Looking at the trace, the full message is in the insert query, but it doesn't all make it into the databsae. ======================================================================
---------------------------------------------------------------------- blake - 22-Jun-06 22:44 ---------------------------------------------------------------------- I am using MySQL on Gentoo, version 4.1.14-r1. ---------------------------------------------------------------------- blake - 22-Jun-06 23:04 ---------------------------------------------------------------------- I've attached a level 5 log of both the message being inserted by dbmail-smtp, and my MUA appending it to the "Sent" folder. Here is the relavent section of the dbmail-smtp trace: Jun 22 16:08:21 colo dbmail/smtp[28256]: Debug dbmysql.c,db_query: query [INSERT INTO dbmail_physmessage (messagesize, internal_date) VALUES ('0', CURRENT_TIMESTAMP)] Jun 22 16:08:21 colo dbmail/smtp[28256]: Debug dbmysql.c,db_query: query [INSERT INTO dbmail_messages(mailbox_idnr, physmessage_id, unique_id,recent_flag, status) VALUES ('142', '817110', '457daaa08c5f501c5dedd0faa2972375', '1', '5')] Jun 22 16:08:21 colo dbmail/smtp[28256]: Debug dbmysql.c,db_query: query [SELECT physmessage_id FROM dbmail_messages WHERE message_idnr = '1346881'] Jun 22 16:08:21 colo dbmail/smtp[28256]: Debug dbmysql.c,db_query: query [INSERT INTO dbmail_messageblks (is_header, messageblk,blocksize, physmessage_id) VALUES ('1','Return-Path: <[EMAIL PROTECTED]>\nDelivered-To: [EMAIL PROTECTED]: from [192.168.1.89] (ns2.marketmetrix.com [216.27.185.170])\n by travis.medleysoft.com (Postfix) with ESMTP id 6573F1003A451\n for <[EMAIL PROTECTED]>; Thu, 22 Jun 2006 16:08:21 -0400 (EDT)\nMessage-ID: <[EMAIL PROTECTED]>\nDate: Thu, 22 Jun 2006 13:08:17 -0700\nFrom: Blake Mitchell <[EMAIL PROTECTED]>\nUser-Agent: Thunderbird 1.5.0.4 (Windows/20060516)\nTo: Blake Mitchell <[EMAIL PROTECTED]>\nSubject: test message truncate 2\nX-DBMail-PhysMessage-ID: 817110\nMIME-Version: 1.0\nContent-Type: text/plain; charset=ISO-8859-1; format=flowed\nContent-Transfer-Encoding: 8bit\n\n', '689', '817110')] Jun 22 16:08:21 colo dbmail/smtp[28256]: Debug dbmail-message.c,dbmail_message_store: allocating [524288] bytes of memory for readblock Jun 22 16:08:21 colo dbmail/smtp[28256]: Debug pipe.c, store_message_in_blocks: inserting message: 1: Geçerli Degil 2: Geçerli Degil 3: Geçerli Degil 4: Geçerli Degil 5: Geçerli Degil Jun 22 16:08:21 colo dbmail/smtp[28256]: Debug dbmysql.c,db_query: query [SELECT physmessage_id FROM dbmail_messages WHERE message_idnr = '1346881'] Jun 22 16:08:21 colo dbmail/smtp[28256]: Debug dbmysql.c,db_query: query [INSERT INTO dbmail_messageblks (is_header, messageblk,blocksize, physmessage_id) VALUES ('0','1: Geçerli Degil\n2: Geçerli Degil\n3: Geçerli Degil\n4: Geçerli Degil\n5: Geçerli Degil\n', '85', '817110')] Jun 22 16:08:21 colo dbmail/smtp[28256]: Debug dbmysql.c,db_query: query [UPDATE dbmail_messages SET unique_id = '457daaa08c5f501c5dedd0faa2972375', status = '0' WHERE message_idnr = '1346881'] Jun 22 16:08:21 colo dbmail/smtp[28256]: Debug dbmysql.c,db_query: query [SELECT physmessage_id FROM dbmail_messages WHERE message_idnr = '1346881'] Jun 22 16:08:21 colo dbmail/smtp[28256]: Debug dbmysql.c,db_query: query [UPDATE dbmail_physmessage SET messagesize = '774', rfcsize = '795' WHERE id = '817110'] ---------------------------------------------------------------------- blake - 22-Jun-06 23:10 ---------------------------------------------------------------------- if I try to look at the message from above in the databse, you can see that the message is truncated at the first 'ç' character: travis htdocs # mysql -u dbmail -p -e 'select * from dbmail_messageblks where physmessage_id = 817110' dbmail Enter password: +-----------------+----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+-----------+ | messageblk_idnr | physmessage_id | messageblk | blocksize | is_header | +-----------------+----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+-----------+ | 1637260 | 817110 | Return-Path: <[EMAIL PROTECTED]> Delivered-To: [EMAIL PROTECTED] Received: from [192.168.1.89] (ns2.marketmetrix.com [216.27.185.170]) by travis.medleysoft.com (Postfix) with ESMTP id 6573F1003A451 for <[EMAIL PROTECTED]>; Thu, 22 Jun 2006 16:08:21 -0400 (EDT) Message-ID: <[EMAIL PROTECTED]> Date: Thu, 22 Jun 2006 13:08:17 -0700 From: Blake Mitchell <[EMAIL PROTECTED]> User-Agent: Thunderbird 1.5.0.4 (Windows/20060516) To: Blake Mitchell <[EMAIL PROTECTED]> Subject: test message truncate 2 X-DBMail-PhysMessage-ID: 817110 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit | 689 | 1 | | 1637261 | 817110 | 1: Ge | 85 | 0 | +-----------------+----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+-----------+ Issue History Date Modified Username Field Change ====================================================================== 22-Jun-06 22:43 blake New Issue 22-Jun-06 22:44 blake Note Added: 0001270 22-Jun-06 22:53 blake File Added: maillog.txt.gz 22-Jun-06 23:04 blake Note Added: 0001271 22-Jun-06 23:10 blake Note Added: 0001272 ======================================================================