Bug Tracker item #2981066, was opened at 2010-04-02 15:05 Message generated for change (Comment added) made by ls999 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126467&aid=2981066&group_id=250683
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: daemon Group: v3.9.0 Status: Closed Resolution: None Priority: 5 Private: No Submitted By: LS999 (ls999) Assigned to: Stevan Bajic (sbajic) Summary: email-addr upper|lower case Initial Comment: I'm use now hash_drv for testing dspam & found next problem: 'To' & 'From' mail fields can use uppper & lower case symbols (it not impotant for smtp). As result DB can contain some info for one email ([email protected], [email protected] and more more, more...). Result: - DB contain all versions of [email protected] in all variantes in five days in my case (Oo-disk space is not rubber!) - if mail send to [email protected] i can't retrain DB when send message later from [email protected] to [email protected]. will see: "Apr 2 13:37:37 main dspam[26224]: Unable to open file for reading: /var/local/dspam/data/mydomain.no/aa/aa.sig/4bb554fb268827715621127.sig: No such file or directory Apr 2 13:37:37 main dspam[26224]: Signature retrieval for '4bb554fb268827715621127' failed Apr 2 13:37:37 main dspam[26224]: Unable to find a valid signature. Aborting. Apr 2 13:37:37 main dspam[26224]: process_message returned error -5. dropping message." but real signature is present in the /var/local/dspam/data/MyEmail.NO/Aa/Aa.sig/ directory IMHO it some smtp-server depended & realy possibility limits when use another processing what described in README (i realy can rewrite this data using smtp-server abilites, but it will global for all another processes after dspam. and it not bes way for me as example. it MUST work internally) one-side: all email-addr must processed as is another-side: subsequent analysis must be case-insensitive IMHO daemon CAN (&mb must) process incoming email-addr as is, for collect data, but database processing (exactly - signatures find) must be case-insensitive|config options regulated PS as add: require to future 'opt-in' 'opt-out' domain based checks when compuled with --enable-domain-scale options (global l level is, user level is, where is domain level?) PSS Oooo and more: custom queries to DB in conf. I think not only me already have & use self user+servises databases. Another DB from U & permanent synk on every UID change not show best (therefore i try hash_drv engine for testing now). TY & Best regards. ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-06-08 10:02 Message: all simple: RFC: as real DSPAM MUST work with e-mail as final instance (first part addr case-ins etc) INTERNALLY ONLY (self DB ). But don't must chahge source (here not about mail subject or X-headers). TRUE: DSpam CAN BE NOT final instance - think about it. -if i use "Broken case" then i have case-insesitive-db record (MYMAIL=MyMail in DB). MYMAIL=MyMail->signature. But now i have "broken" sender e-mail on DSpam output: mymail -it wrong; -if i don't use "Broken case" then i have don't changed sender e-mail on DSpam output (nice), but have case sensitive records in DB for sender-signature links (here MYMAIL<>=MyMail in DB). now MYMAIL->signature not same MyMail->signature. 2sbajic: realy TY 4 support. PS another TY 4U, but i don't test DSpam from my firts post. mb all resolved at this time. i'm promise see new code & test it. sorry 4 bad engl. ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2010-05-31 12:45 Message: > 1) mail CAN after DSPAM chained to the anothr service. It service > CAN be case sensetive for incoming data > This is fine. > 2) DSPAM output change incoming data > Only if you tell DSPAM to do so with "Broken case". By default DSPAM DOES NOT change anything regarding recipient. It however does change the subject if you tell DSPAM to do so (tagging of subjects), adds mail headers (X-DSPAM-xxxxx) and modifies content/body (if you tell DSPAM to add signature to body or you tell DSPAM to tag spam/ham messages (preferences tagNonspam/tagSpam)). I don't see this message thread going anywhere. You have your hard time telling me what you really want and I just can not understand what you want. I asked you a bunch of times if you would be satisfied with lowercasing the directories below your DSPAM_HOME (/var/local/dspam/data/) but you fail to tell me if this would help you or not. You still have the possibility to use "dspam_admin change preference XXXX localStore xxxx" or the command line option "--rcpt-to" if you want to unify/lowercase the recipient while still not setting the option "Broken case". I know that this can be a pain for larger installations. Maybe you should ask someone working with you to describe here in better English what you really want? I had the impression that lowercasing the path in your DSPAM_HOME would do the trick but you can not confirm me that and I am not going to code something that is not useful to you or anyone else. Or maybe all you want is that DSPAM internally is lowercasing the username (taken from the FROM mail header) and lowercasing the path inside your DSPAM_HOME while not touching anything inside the original mail header? I just don't know what you want/need. End of story. I am not going to replay any more. In German we have a saying "Der Ton macht die Musik" and yours is definitely not sounding right to me (you call me drunk, childish, etc...) ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-05-31 06:39 Message: 1) mail CAN after DSPAM chained to the anothr service. It service CAN be case sensetive for incoming data 2) DSPAM output change incoming data ---- ALL ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-05-31 06:30 Message: 2sbajic real im think what simple drunk. therefore remove all flood as open U eye (not native) ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2010-05-31 06:04 Message: > & what U show now me? since this self. it not response for > described problem. > What is the problem anyway? The case folding on /var/local/dspam/data/? Is that the issue? Do you want the path on /var/local/dspam/data/ to be all in lowercase? Is it that what you are asking for? ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-05-31 05:59 Message: & what U show now me? since this self. it not response for described problem. ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-05-31 05:55 Message: " Mail must not relay to anther server. It is up to the administrator how to implement that filtering. How have you implemented DSPAM in your setup? As an content filter? Or any other way? What way? " real. I implemented it as contetnt filter for incoming mail on one system. test system. how it resolve common problem about <IN>--<DSPAM>-<in>-<ANOTER_FILTER>? ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2010-05-31 05:54 Message: > "a"="A" on the DSPAM output. & it IS problem. > Then DO NOT USE "Broken case". Problem fixed! Can I ask you something? * [email protected] != [email protected] [_] Yes? [_] No? If Yes then in DSPAM: * [email protected] != [email protected] * /var/local/dspam/data/domain.no/me/me.sig/4bb554fb268827715621127.sig != /var/local/dspam/data/DOMAIN.NO/ME/ME.sig/4bb554fb268827715621127.sig Right? If No then in DSPAM you should set "Broken case" and then: * [email protected] == [email protected] * Storage for [email protected] == /var/local/dspam/data/domain.no/me/me.sig/4bb554fb268827715621127.sig * Storage for [email protected] == /var/local/dspam/data/domain.no/me/me.sig/4bb554fb268827715621127.sig Right? ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2010-05-31 05:48 Message: > are listing? why affter DSAPM mail must be relay to another server? > Mail must not relay to anther server. It is up to the administrator how to implement that filtering. How have you implemented DSPAM in your setup? As an content filter? Or any other way? What way? > whatt > about simple greylist service for example? "a"<>"A" already on DSPAM > output. > How have you implemented DSPAM in your setup? In my setup I use TWO greylisting services. One is SQLgrey while the other is GROSS. And NONE of them is integrated in my Postfix setup that way that if I reinject the mail from DSPAM into Postfix the mail again gets greylisted. Even if I would change the case on the recipient from [email protected] to [email protected]. I simply don't run greylisting after I have processed the mail with my content filter (which is DSPAM). Why should I rerun that greylisting again? What would be the point? ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-05-31 05:47 Message: Edit for last message: "a"="A" on the DSPAM output. & it IS problem. ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-05-31 05:41 Message: " if your first MTA is Postfix and your content filter is DSPAM and your delivery host is Microsoft Exchange then you ask me to: " are listing? why affter DSAPM mail must be relay to another server? whatt about simple greylist service for example? "a"<>"A" already on DSPAM output. ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2010-05-31 05:35 Message: I don't take it as an offense. I just don't understand. In all your stages you want your system to: [email protected] != [email protected] But on the DSPAM storage you want: [email protected] == [email protected] And this is not logical to me. Do you understand what I mean? Look. It is ultra giga easy for me to code that lowercasing stuff on the storage. It is just a bunch of lines in C code. But again: IT IS NOT LOGICAL. If your first MTA is Postfix and your content filter is DSPAM and your delivery host is Microsoft Exchange then you ask me to: * Postfix should threat the recipient [email protected] as a different user then [email protected] * DSPAM should threat [email protected] THE SAME as [email protected] but ONLY INTERNALLY. You don't want DSPAM to change headers nor anything other. Everything should stay as it is but internally it should make that lowercasing stuff BUT NOT SHOW IT TO THE OUTSIDE that it did that. * Microsoft Exchange then again should threat [email protected] DIFFERENT then [email protected] Do you understand that this is wicked? Do you understand that DSPAM would then merge two absolutely different accounts into one account? Do you see the problem I am seeing and the issue I have with it? ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-05-31 05:28 Message: Do not take offence. &Reread self plz.. It child. ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2010-05-31 05:26 Message: > because DSPAM IS NOT FINAL point for detecting spam etc. It ONLY 1step in > chain. Therefore it can't invalidate incoming information. (now i'm not > native fully) > I understand this. But why do you want DSPAM to use ONE FREAKING storage for [email protected] and [email protected] and [email protected] while on every other stage in your decision you make a difference between [email protected], [email protected], [email protected], etc? This is NOT LOGICAL to me. ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2010-05-31 05:24 Message: > ClamAV same filter. why it not change nothing if i want. simple add > X-header. & final processing i can make as I want. > BECAUSE ClamAV does not save any signature or other things on the file system. THAT is different then what DSPAM does. DSPAM is saving a signature/message on the filesystem because it needs that information for retraining. If you don't want DSPAM to save that information on the local disc then just run DSPAM in "NOTRAIN" mode. When doing this then you can compare ClamAV with DSPAM. ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-05-31 05:24 Message: " don't get it. Why should a mail-filter/content-filter act differently on mail headers then a normal mail client or mail server? " because DSPAM IS NOT FINAL point for detecting spam etc. It ONLY 1step in chain. Therefore it can't invalidate incoming information. (now i'm not native fully) ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2010-05-31 05:22 Message: I really have a hard time understanding your English. > but now any another filter flowed DSPAM will receive "a"? but not "A". > what about this? > Then DO NOT USE "Broken case". That easy! You totally confuse me. First you DON'T WANT any case to be converted in lowercase by DSPAM. Perfect. Then don't change anything inside DSPAM and DSPAM will NOT change anything. Then you bark that if the case is not changed to lowercase then you end up in having: /var/local/dspam/data/mydomain.no/aa/aa.sig/ /var/local/dspam/data/MYDOMEIN.NO/AA/AA.sig/ /var/local/dspam/data/MyDomein.No/aa/aa.sig/ etc... So then I told you to use "Broken case" but that does not satisfy you because you still want on the filesystem the name to be converted to lowercase but not on the header. Right? So all you want is a case transformation on the file system BUT NOT in the header. RIGHT? ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-05-31 05:17 Message: ClamAV same filter. why it not change nothing if i want. simple add X-header. & final processing i can make as I want. ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-05-31 05:15 Message: fine: " You mean for the content inspection? Definitively not! When DSPAM is inspecting the content of a message then "A" is not the same as "a". " but now any another filter flowed DSPAM will receive "a"? but not "A". what about this? ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2010-05-31 05:12 Message: > Right - FOR MAIL CLIENT & SERVERS ONLY. BUT NOT FOR > MAIL-FILTER services. > I don't get it. Why should a mail-filter/content-filter act differently on mail headers then a normal mail client or mail server? > sayb me one - what algorithms used dsapm. > In what context? I don't understand this question. > say me second - "a" and "A" for this algorithms is same? > You mean for the content inspection? Definitively not! When DSPAM is inspecting the content of a message then "A" is not the same as "a". ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-05-31 05:11 Message: Simple not sleep. What about RFC? Fine - this project not make final mail receiver - it filter only. And therefore it MUST have ability for don't change ANY INCOMING INFORMATION. i Repeate: " decision on mail processing MUST be dspam independent and data falsification inadmissible here. " ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-05-31 05:02 Message: "Now according to the RFC the domain part can be in any case. Right? So DOMAIN.NO is the same as domain.no and is the same as DoMaIn.No, etc... Right?" Right - FOR MAIL CLIENT & SERVERS ONLY. BUT NOT FOR MAIL-FILTER services. sayb me one - what algorithms used dsapm. & say me second - "a" and "A" for this algorithms is same? ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2010-05-31 04:07 Message: My point is: Message from the internet is arriving for the recipient [email protected]. Now according to the RFC the domain part can be in any case. Right? So DOMAIN.NO is the same as domain.no and is the same as DoMaIn.No, etc... Right? The local part is case sensitive according to the RFC's. Right? So [email protected] is not the same as [email protected]. Right? I then told you to look at "Broken case". And you replied that it changes the header. Right? Okay. Point taken. It does change the header. So a message send to [email protected] will be changed to [email protected] if one uses "Broken case" and then when DSPAM delivers that message to your delivery host then the recipient will be [email protected] but you don't want that. You want it to be [email protected] while in DSPAM you want it to be [email protected] because you want to save the message in the DSPAM storage in lowercase. Right? And here comes my problem. Your MTA should see the message as "[email protected]", while DSPAM should see it as "[email protected]", while yet again your delivery host should see it as "[email protected]". Or to say it in other words: Your MTA should see "[email protected]" and "[email protected]" as TWO different accounts, while DSPAM should use ONE single storage/account for both addresses and your delivery host should then again see TWO different accounts. Do you spot the conflict I am having? From a programming viewpoint it is not difficult to do what you want but I really fail to see the logic behind it. Either one is consistent or not. And what you request has no logic to me. If I look at this scenario: (A) INTERNET --> (B) MTA --> (C) DSPAM --> (D) MTA/DELIVERY Then I fail to see why A + B + D should be case sensitive while C should not be case sensitive. What is the logic behind this? Explain this to me. Please! ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-05-31 02:45 Message: 1Q & 1A must be exchanged. Sorry. ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-05-31 02:12 Message: 2sbajic Sory - very many hard work now... I reread U message about " Anyway... if I understand you right, then all you want is that DSPAM does NOT change any original 'To' headers but use one single store for signature writing/reading. Right? " i think it what I want, but i will test (as I say in prev message i real long time don't track project & as U say "it not my native language"). if I understand U righ then now I can use CLI dsapm_admin to resolve described problems. 1Q: dspam-server has now options for this functionality or I must permanent use dspam_admin? 1A: = aswer to U Q " am now going to close this bug report since DSPAM already offers out of the box the functionality you asked for. " TY for work. ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-05-31 01:35 Message: I think it about item 2. I'm try using another words. If i have thousands users ([email protected], [email protected], ..., [email protected], ..., [email protected]) then i must perform (as you say) for everyone: # dspam_admin change preference [email protected] localStore [email protected] # dspam_admin change preference [email protected] localStore [email protected] It manually or script's work (don't forget that I can't use a web-if). This way is impossible in my case: 1) manually processing is simple delirium for thousands users; 2) script processing is possible as fact, but realtime reaction to changes in mail system (add new e-mail client for example) is impossible (cron) or very mail system depended. Mail system here not ONLY smtp-server, but ALL mail system components & services & it configuration. => many CURRENT-mail-system depended scripts. 2a) this realization type is time critical when any crash of mail system is occured (because it very system depended & stupid support can't real make anything => call me... call me...call me... [email protected] [email protected] - i want sleep sometimes :). Therefore i say again - this is dspam-server problem: 1) if dspam CAN act as simple filter for add marks then it CAN'T & MUST don't change ANY another info in incoming mail on self output (it deforms the information received by following filters) in this mode. The final decision on mail processing MUST be dspam independent and data falsification inadmissible here. 1a) this mode must be configurable by server options (not all need such functioning and i understand it) PS I don't track this project long time (considering speed of development). Mb it all already resolved. While I am compelled to use other decision, but I don't leave dspam ). TY 4 all & Personal 4 U sbajic. ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2010-05-29 12:42 Message: > 2. what about MY1,My2,My...,MyN? manual processing? > I don't understand that. Can you rephrase that question? ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-05-29 05:44 Message: Your are right 100%. Not my native language. 1. i'm don't use web (& can't) 2. what about MY1,My2,My...,MyN? manual processing? ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2010-05-13 22:49 Message: I have real hard time to understand your writing. Probably English is not your native language (it is not my native language too)? Anyway... if I understand you right, then all you want is that DSPAM does NOT change any original 'To' headers but use one single store for signature writing/reading. Right? Well... DSPAM has already that functionality. Assume you have the email [email protected] and want it to be threated the same way as [email protected] then just do this: - DO NOT enable "Broken case" - use dspam_admin to change the store for both emails: # dspam_admin change preference [email protected] localStore [email protected] # dspam_admin change preference [email protected] localStore [email protected] That's it. All mails now arriving to "[email protected]" or "[email protected]" will use "[email protected]" as their store. I am now going to close this bug report since DSPAM already offers out of the box the functionality you asked for. ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-04-06 02:39 Message: I don't think what You are right. It's very situation depended. DSPAM acts as the filter in most cases for add X-headers only. Therefore in this situation: data->[DSPAM]->[filter_1]->[filter_2]->... already "filter_1" (and all following it) don't have original message data (addr modified by DSPAM). Therefore [email protected] and [email protected] IS different addresses in this case (as DSPAM outgoing data). And it is very important for adequate data processing with following filters. IMHO must be one more config option for forward original address as is. Thus if this option OFF, the "broken case" acts as now. When new option is ON, the "broken case" is used for internal data process only, but we have the original "To" (and similar) field(s) on DSPAM output. And i don't see RFC contradiction here, but feature improvement only . Adequate data processing Best regards. ---------------------------------------------------------------------- Comment By: Ion-Mihai "IOnut" Tetcu (itetcu) Date: 2010-04-04 12:40 Message: Per RFC: - the right-hand part (domain) is case insensitive, so we could lowercase it. - the left-hand part case-sensitiveness is local site policy. Personally I don't think we should alter anything here, as it's valid (although not recommended) to have [email protected] and [email protected] different addresses. ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-04-04 05:01 Message: Add. must be [email protected]>[DSPAM [email protected] (can be config option regulated) DSPAM]->[email protected] but not [email protected]>[DSPAM]->[email protected] or [email protected]>[DSPAM]->[email protected] ---------------------------------------------------------------------- Comment By: LS999 (ls999) Date: 2010-04-03 04:54 Message: TY 4 "broken case" option - i missed it - &sorry). But "broken case" option in this case is partial decision of a problem. For example: - DSAPM off. Here i see all letter cases was perfect keeped: Return-Path: <[email protected]> X-Original-To: [email protected] Delivered-To: [email protected] Received: from mcgi64.rambler.ru (mcgi64.rambler.ru [81.19.67.198]) by mx.domain.no (MTA) with ESMTP id C5F6818006E for <[email protected]>; Sat, 3 Apr 2010 02:32:15 +0400 (MSD) Received: from [78.31.73.87] by mcgi64.rambler.ru with HTTP (mailimap); Sat, 03 Apr 2010 02:32:14 +0400 From: <[email protected]> To: <[email protected]> Subject: dspam off - DSPAM on, "broken case" option off. Now before DSPAM we have "[email protected]" recipient adress, but after reinjecting mail from DSPAM back to the mail server (smpt used) we have already "for <[email protected]>": Return-Path: <[email protected]> X-Original-To: [email protected] Delivered-To: [email protected] Received: from localhost (localhost.localdomain [127.0.0.1]) by mx.domain.no (MTA) with SMTP id CFB1A180076 for <[email protected]>; Sat, 3 Apr 2010 02:25:43 +0400 (MSD) Received: from mxa.rambler.ru (mxa.rambler.ru [81.19.66.231]) by [mx.domain.no] (MTA) with ESMTP id 8F0CA18006E for <[email protected]>; Sat, 3 Apr 2010 02:25:41 +0400 (MSD) ... From: <[email protected]> To: <[email protected]> Subject: broken case off - DSPAM on, "broken case" option on. And now we have "[email protected]" in & "[email protected]" out. Predictably: Return-Path: <[email protected]> X-Original-To: [email protected] Delivered-To: [email protected] Received: from localhost (localhost.localdomain [127.0.0.1]) by [email protected] (MTA) with SMTP id 0E095180076 for <[email protected]>; Sat, 3 Apr 2010 02:29:30 +0400 (MSD) Received: from mcgi20.rambler.ru (mcgi20.rambler.ru [81.19.67.53]) by main.eletek.ru (MTA) with ESMTP id BB71318006E for <[email protected]>; Sat, 3 Apr 2010 02:29:27 +0400 (MSD) Received: from [78.31.73.87] by mcgi20.rambler.ru with HTTP (mailimap); Sat, 03 Apr 2010 02:29:27 +0400 From: <[email protected]> To: <[email protected]> Subject: broken case on But in ALL cases with DSPAM i don't have ORIGINAL message headers on output & this restriction for the filters following it. IMHO DSPAM don't need change any original message headers. Best regards. ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2010-04-02 16:32 Message: Have you tried to use "Broken case"? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126467&aid=2981066&group_id=250683 ------------------------------------------------------------------------------ ThinkGeek and WIRED's GeekDad team up for the Ultimate GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the lucky parental unit. See the prize list and enter to win: http://p.sf.net/sfu/thinkgeek-promo _______________________________________________ Dspam-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/dspam-devel
