Re: [twsocket] [TMimeDec] Beta version to test
On which machine? (cpu, mainboard, amount of free RAM, operating system...) If it's new/rather new machine, such result is far below expectations - I get 20 sec on old P120 with 32MB RAM and Win95. Pentium M 2,13 GHz - 533 MHz FSB, 1 GB ram, WinXP Pro, so I guess the machine should perform good. Is there a way to make the decoding work faster? I have about the same testresult on the beta and the older version of MimeDec. There's many ways to make it faster. First, do not use TMemoryStream as decoded attachment storage *unless* you set it's size so TMemoryStream won't need to reallocate his memory. Second, you should use larger input buffer (not that 4kbyte one), or change decoding logic so you could use memory- mapped file. Third, do not use OnPartLine to store data - set DestStream and leave OnPartLine unassigned (you'll save one jump into and out of the handler, possibly dirtying less processor's L1/L2 cache). Fourth, use TMemoryStream (or some kind of TBufferedWriteStream) as decoded data storage (but remember about the first note above). Fifth - discard these fancy progressbars (and progressbar updating code) ;) I use TMemoryStream. I don't use any progressbars, but I use the old version MimeDec. And I don't use MimeDec directly, I use it through MimeDecEx from usermade page, It could be there that I should use the improvements you suggests. If you think it's a bad idea to use the MimeDecEx from usermade page, could you provide a working example of how to use the new beta of MimeDec? Regards Bjørnar -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [TMimeDec] Beta version to test
Hello! However, I have a question. Decoding attachments seems to take a long time. Decoding a base64-encoded attachment of 10 MB takes 17 sec. On which machine? (cpu, mainboard, amount of free RAM, operating system...) If it's new/rather new machine, such result is far below expectations - I get 20 sec on old P120 with 32MB RAM and Win95. Is there a way to make the decoding work faster? I have about the same testresult on the beta and the older version of MimeDec. There's many ways to make it faster. First, do not use TMemoryStream as decoded attachment storage *unless* you set it's size so TMemoryStream won't need to reallocate his memory. Second, you should use larger input buffer (not that 4kbyte one), or change decoding logic so you could use memory- mapped file. Third, do not use OnPartLine to store data - set DestStream and leave OnPartLine unassigned (you'll save one jump into and out of the handler, possibly dirtying less processor's L1/L2 cache). Fourth, use TMemoryStream (or some kind of TBufferedWriteStream) as decoded data storage (but remember about the first note above). Fifth - discard these fancy progressbars (and progressbar updating code) ;) -- Piotr Hellrayzer Dalek [EMAIL PROTECTED] -- Sa niesamowite, zobaczysz... ;-) link http://link.interia.pl/f18b9 -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [TMimeDec] Beta version to test
I have tested the beta and had some trouble with it, but I use the component through MimeDecEX from usermade page. Did not do any changes before testing. However, I have a question. Decoding attachments seems to take a long time. Decoding a base64-encoded attachment of 10 MB takes 17 sec. Is there a way to make the decoding work faster? I have about the same testresult on the beta and the older version of MimeDec. Regards Bjørnar -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [TMimeDec] Beta version to test
As far as I can see, no-one else in this mailing has attempted to test the new TMimeDec component, just me. I've found it does not work as well as the previous component, ignoring complete parts of the email. The internals of the component are irrelavant to me and most users, we really don't care about how clever you are in interpreting RFCs, nor do we want to debug your source code to understand how you have changed the component in order to prevent it working properly in existing applications. We just want to decode MIME emails. If it's not backward compatible with existing applications, it's your responsibility to document how applications must be changed, at top of the TMimeDec component source, not in one of your tedious postings in this mailing list. If the beta TMimeDec is released in it's present state, anyone rebuilding an application is likely to find their application useless, as I have. So it should not be released until these problems are resolved. Of course I may be wrong, I make errors. So perhaps everyone else that has tested the beta TMimeDec component can now post their results? Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [TMimeDec] Beta version to test
Ok, you guys need to chill. I agree with both of you, bugs need to be fixed (and documented if they change the usage of the component) and new bugs shouldnt be introduced (missing message parts). Yes, I know my points are common sense. Just chill. Dan - Original Message - From: Angus Robertson - Magenta Systems Ltd [EMAIL PROTECTED] To: twsocket@elists.org Sent: Friday, September 09, 2005 10:17 AM Subject: Re: [twsocket] [TMimeDec] Beta version to test As far as I can see, no-one else in this mailing has attempted to test the new TMimeDec component, just me. I've found it does not work as well as the previous component, ignoring complete parts of the email. The internals of the component are irrelavant to me and most users, we really don't care about how clever you are in interpreting RFCs, nor do we want to debug your source code to understand how you have changed the component in order to prevent it working properly in existing applications. We just want to decode MIME emails. If it's not backward compatible with existing applications, it's your responsibility to document how applications must be changed, at top of the TMimeDec component source, not in one of your tedious postings in this mailing list. If the beta TMimeDec is released in it's present state, anyone rebuilding an application is likely to find their application useless, as I have. So it should not be released until these problems are resolved. Of course I may be wrong, I make errors. So perhaps everyone else that has tested the beta TMimeDec component can now post their results? Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [TMimeDec] Beta version to test
Hello! And since you're too lazy to do a such simple thing, It's unfortunate your messages are so badly composed, full of capital letters which makes them hard to read, So what? My english isn't perfect, that's true. and essentially very irritating. More irritating is trying to convince me that buggy code is ok. And most irritating thing is having to remember that there's a bug in some code and you have to write special code to overcome it. I'll spend my time doing useful things, like enhancing ICS components properly, for the benefit of other. Yeah - properly, you mean, leaving old bugs as-is, just because this would cause component to behave differently? In case of problems, refer to TMimeDec (beta) sources I've really got better things to do than debug other people's source files. You've had to debug YOUR (TMimeDecEx) sources. Much easier to keep using a MineDec that works. It works - just for you. or RFC 2046. And reading an old document is hardly going to help make a Delphi program backward compatible. Got newer one? I don't think so, since there's no newer MIME specs. Just downloaded current rfc-index.txt (from http://www.rfc-editor.org), and... #v+ RFC INDEX - (CREATED ON: 09/07/2005.) This file contains citations for all RFCs in numeric order. [..] 2046 Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types. N. Freed, N. Borenstein. November 1996. (Format: TXT=105854 bytes) (Obsoletes RFC1521, RFC1522, RFC1590) (Updated by RFC2646, RFC3798) (Status: DRAFT STANDARD) [..] 2646 The Text/Plain Format Parameter. R. Gellens, Ed.. August 1999. (Format: TXT=29175 bytes) (Obsoleted by RFC3676) (Updates RFC2046) (Status: PROPOSED STANDARD) [..] 3798 Message Disposition Notification. T. Hansen, Ed., G. Vaudreuil, Ed.. May 2004. (Format: TXT=64049 bytes) (Obsoletes RFC2298) (Updates RFC3461, RFC2046) (Status: DRAFT STANDARD) #v- Certainly, reading RFC 2046 won't make Delphi program backward compatible, but at least may enlighten Delphi programmer. -- Piotr Hellrayzer Dalek [EMAIL PROTECTED] -- Seks, zdrowie, uroda, praca... http://link.interia.pl/f18b3 -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [TMimeDec] Beta version to test
Hello! And what was the cause? In TMimeDecEx you set destination stream after the part was decoded. You lose your data, because you don't save it: Hope this helps... :) Not really, you say there's bug in TMimeDecEx, but you don't say what you changed to get your 'result'. No response to my comments. This means either: 1 - the TMimeDec beta is buggy and should not be released. 2 - the TMimeDec beta is not backward compatible with existing applications, and should not be released. ROTFL You work for Microsoft, or what? The old and nasty bug is found and fixed, but we shouldn't release any patches for it, because our product won't be backward compatible and customers would stop writing to us noting the error. You didn't understood my previous message - I wrote how TMimeDec worked and how it works now - yes, it's not backward compatible - it cannot be backward compatible - but since current TMimeDec version is better (read: easier to use) - moving our asses and fixing ours code so IT WON'T RELY ON BUG ANYMORE should not be that hard. Remember that I've had to do the same with HCM - if you didn't relied on that bug too many times, it shouldn't be big pain anyway (in my case it just took 5 minutes to fix, by the way I've removed some tricky code that fixed that bug on higher level). And you wasted almost a week waiting for piece of information that you could find out on your own. You just had to sit down for 15 minutes, read completely my message, re-read your sources, and make it disable part saving/DestStream setting in OnPartEnd since it is called ON PART END, not ON ANY BOUNDARY DETECTION (like it was until new TMimeDec beta), so when you're past first plain text part, you're past first plain text part, not earlier. And since you're too lazy to do a such simple thing, I decided to not respond. I can be lazy too. If I've made a simple error in TMimeDecEx, please tell me what it is, I've pointed it out already. Twice. and why it's able to correctly decode all content parts except the last, which is always empty, using the same events, yet works fine with the old TMineDec. Everything is in my previous message (message-id: [EMAIL PROTECTED]). Please read it carefully. In case of problems, refer to TMimeDec (beta) sources or RFC 2046. -- Piotr Hellrayzer Dalek [EMAIL PROTECTED] -- Startuj z INTERIA.PL! http://link.interia.pl/f186c -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [TMimeDec] Beta version to test
And since you're too lazy to do a such simple thing, It's unfortunate your messages are so badly composed, full of capital letters which makes them hard to read, and essentially very irritating. I'll spend my time doing useful things, like enhancing ICS components properly, for the benefit of other. In case of problems, refer to TMimeDec (beta) sources I've really got better things to do than debug other people's source files. Much easier to keep using a MineDec that works. or RFC 2046. And reading an old document is hardly going to help make a Delphi program backward compatible. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [TMimeDec] Beta version to test
Hello! Send me one (or all) of these messages (or better post them somewhere and give me the url) and I'll check it out. Not a single MIME message in my mailbox decoded the last attachment correctly, I don't suppose the issue is very hard to reproduce. I'll relay the two examples I posted before so you see the same headers I saw. HCM, which *bases* on my (beta) code decodes it properly! You've sent me a message containing a plain text part and html version of it, a Magenta Systems logo, Magenta Systems main WWW page, a Excel file and a zipfile with adult webcams list. Note that TMimeDec for a long time was broken and proper (RFC-compliant) part decoding required some coding - maybe that's the case. I've not added any extra code, I was using my TMimeDecEx component from the usermade page and reporting all the parts found by events, nothing complex atall. If every other mailer is producing non-compliant MIME that co-incidenally the old MimeDec handling OK, that's what we need to stick with, not changes to lose the message content. Okay, I agree, but having bugged code fix earlier bugs (like I did with the TSmtpCli - it was *not* flushing its buffer, and it was not resetting his buffer position, so although messages actually were concatenated, each of them was sent as a separate one) is not the solution. It's better to have two days of additional work more and then have additional four free days, than having to remember why that must be coded this way and not the logical one, or fighting with those (most of the time invisible) bugs. You have the sample of what hidden (for too long time) bug can do for you ;-) And what was the cause? In TMimeDecEx you set destination stream *after the part was decoded*. You lose your data, because you *don't save it*: In the old code you have: *part begin* Part 0, Content: multipart/mixed, Size: 44, Name: , FileName: , Encoding: *part end* *part begin* Part 1, Content: multipart/alternative, Size: 2, Name: , FileName: , Encoding: *part end* *part begin* Part 2, Content: text/plain, Size: 46, Name: , FileName: , Encoding: quoted-printable *part end* *part begin* Part 3, Content: text/html, Size: 320, Name: , FileName: , Encoding: quoted-printable *part end* *part begin* Part 4, Content: text/plain, Size: 0, Name: , FileName: , Encoding: *part end* *part begin* Part 5, Content: image/gif, Size: 3,640, Name: maglogo.gif, FileName: maglogo.gif, Encoding: base64 *part end* *part begin* Part 6, Content: text/html, Size: 6,825, Name: index.html, FileName: index.html, Encoding: quoted-printable *part end* *part begin* Part 7, Content: application/vnd.ms-excel, Size: 73,728, Name: bands.xls, FileName: bands.xls, Encoding: base64 *part end* *part begin* Part 8, Content: application/x-zip-compressed, Size: 5,327, Name: Defsites.zip, FileName: Defsites.zip, Encoding: base64 *part end* *part begin* Part 9, Content: text/plain, Size: 8, Name: , FileName: , Encoding: *part end* In the new one (note the indent): *part begin* Part 0, Content: multipart/mixed, Size: 44, Name: , FileName: , Encoding: *part begin* Part 1, Content: multipart/alternative, Size: 2, Name: , FileName: , Encoding: *part begin* Part 2, Content: text/plain, Size: 46, Name: , FileName: , Encoding: quoted-printable *part end* *part begin* Part 3, Content: text/html, Size: 320, Name: , FileName: , Encoding: quoted-printable *part end* *part end* *part begin* Part 4, Content: image/gif, Size: 3,640, Name: maglogo.gif, FileName: maglogo.gif, Encoding: base64 *part end* *part begin* Part 5, Content: text/html, Size: 6,825, Name: index.html, FileName: index.html, Encoding: quoted-printable *part end* *part begin* Part 6, Content: application/vnd.ms-excel, Size: 73,728, Name: bands.xls, FileName: bands.xls, Encoding: base64 *part end* *part begin* Part 7, Content: application/x-zip-compressed, Size: 5,327, Name: Defsites.zip, FileName: Defsites.zip, Encoding: base64 *part end* *part end* (actual part sizes may slightly differ in some cases) Old TMimeDec treated starting and ending (with additional -- at the end of line) boundary as one and the same thing, which is (was?) wrong - since parts can be stored in subparts (creating a such tree), not distinguishing between starting and ending MIME boundaries leads to phantom (short-length and empty) parts (in the above example, old TMimeDec's part 4 maps to the part between first text/html part end and multipart/alternative part end). Hope this helps... :) -- Piotr Hellrayzer Dalek Author of ICS-Based Hellcore Mailer - an Outlook Express killer http://www.hcm.prv.pl [EMAIL PROTECTED] -- Startuj z INTERIA.PL! http://link.interia.pl/f186c -- To unsubscribe or change your settings for TWSocket mailing list please goto
Re: [twsocket] [TMimeDec] Beta version to test
And what was the cause? In TMimeDecEx you set destination stream *after the part was decoded*. You lose your data, because you *don't save it*: Hope this helps... :) Not really, you say there's bug in TMimeDecEx, but you don't say what you changed to get your 'result'. The stream is never 'saved', except by TMineDec, it's assigned to DestStream in the PartBegin event, and is found to be empty in the PartEnd event. It's not 'set after the part is decoded, but before. Where do you think it should be 'saved'? TMimeDecEx has worked with TMimeDec for several years, if it now fails that means your changes are not backward compatible and will cause any application using MIME to fail unless changed in some unspecified way. If changes are need, they MUST be a prominent warning at the top of the source file. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
