Body rules hit on Subject
Hi, I recall a conversation in the past about body rules hitting on the Subject of an email, but I don't believe there was ever a solution, or I otherwise missed it. It's now affecting me again, and I hoped someone had some ideas on how to prevent body rules from hitting on the subject too since it's apparently considered part of the message body?
Re: Body rules hit on Subject
On 02/02/2018 12:41 PM, Alex wrote: Hi, I recall a conversation in the past about body rules hitting on the Subject of an email, but I don't believe there was ever a solution, or I otherwise missed it. It's now affecting me again, and I hoped someone had some ideas on how to prevent body rules from hitting on the subject too since it's apparently considered part of the message body? I don't think I have ever run across this problem on my SA instances. Can you send an example of this via pastebin.com? -- David Jones
Re: Body rules hit on Subject
Hi, >> I recall a conversation in the past about body rules hitting on the >> Subject of an email, but I don't believe there was ever a solution, or >> I otherwise missed it. >> >> It's now affecting me again, and I hoped someone had some ideas on how >> to prevent body rules from hitting on the subject too since it's >> apparently considered part of the message body? >> > > I don't think I have ever run across this problem on my SA instances. Can > you send an example of this via pastebin.com? Save this text below which contains no "docusign" in the body in a file and create a rule like: body __BODY_DOCUSIGN /docusign/i >From DUMMY-LINE Fri Feb 2 12:15:01 2018 Return-Path: To: otheru...@example.com Subject: DocuSign Content-Type: text/plain; charset=UTF-8 Message-Id: <20180202171501.e23486800c...@mail01.example.com> Date: Fri, 2 Feb 2018 12:15:01 -0500 (EST) this is a test dbg: rules: ran body rule __BODY_DOCUSIGN ==> got hit: "DocuSign" dbg: rules: ran header rule __DOCUSIGN_SUBJ ==> got hit: "DocuSign"
Re: Body rules hit on Subject
On 02/02/2018 02:19 PM, Alex wrote: Hi, I recall a conversation in the past about body rules hitting on the Subject of an email, but I don't believe there was ever a solution, or I otherwise missed it. It's now affecting me again, and I hoped someone had some ideas on how to prevent body rules from hitting on the subject too since it's apparently considered part of the message body? I don't think I have ever run across this problem on my SA instances. Can you send an example of this via pastebin.com? Save this text below which contains no "docusign" in the body in a file and create a rule like: body __BODY_DOCUSIGN /docusign/i From DUMMY-LINE Fri Feb 2 12:15:01 2018 Return-Path: To: otheru...@example.com Subject: DocuSign Content-Type: text/plain; charset=UTF-8 Message-Id: <20180202171501.e23486800c...@mail01.example.com> Date: Fri, 2 Feb 2018 12:15:01 -0500 (EST) this is a test dbg: rules: ran body rule __BODY_DOCUSIGN ==> got hit: "DocuSign" dbg: rules: ran header rule __DOCUSIGN_SUBJ ==> got hit: "DocuSign" Yep. Same results for me too. Never noticed this before. -- David Jones
Re: Body rules hit on Subject
Alex wrote: Hi, I recall a conversation in the past about body rules hitting on the Subject of an email, but I don't believe there was ever a solution, or I otherwise missed it. It's now affecting me again, and I hoped someone had some ideas on how to prevent body rules from hitting on the subject too since it's apparently considered part of the message body? The only "solution" I've ever come up with is to create a meta rule group to account for the Subject hit: body __FOO /foo/ header __SUBJ_FOO Subject =~ /foo/ meta FOO __FOO && !__SUBJ_FOO I have to admit it's annoyed me on occasion that I can't create a single simple rule that ONLY matches on the message body, but TBH it's never been important enough in context for me to even commit the above horror. -kgd
Re: Body rules hit on Subject
On 2/2/2018 1:41 PM, Alex wrote: Hi, I recall a conversation in the past about body rules hitting on the Subject of an email, but I don't believe there was ever a solution, or I otherwise missed it. It's now affecting me again, and I hoped someone had some ideas on how to prevent body rules from hitting on the subject too since it's apparently considered part of the message body? There is no solution at the moment. The subject is appended to the body of the text for rule parsing. I've added a task I plan to submit for GSOC consideration to add a tflag to disable this behavior. Regards, KAM
Re: Body rules hit on Subject
On Fri, 2 Feb 2018, Alex wrote: I recall a conversation in the past about body rules hitting on the Subject of an email, but I don't believe there was ever a solution, or I otherwise missed it. This behavior is by design. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- ...every time I sit down in front of a Windows machine I feel as if the computer is just a place for the manufacturers to put their advertising. -- fwadling on Y! SCOX --- 4 days until the first Falcon Heavy test launch
Re: Body rules hit on Subject
On Fri, 2018-02-02 at 16:59 -0500, Kevin A. McGrail wrote: > There is no solution at the moment. The subject is appended to the > body of the text for rule parsing. I've added a task I plan to > submit for GSOC consideration to add a tflag to disable this > behavior. > Would it sensible leave the body rule as at present and add two more rules: header NAME Subject =~ /regex/ bodytext NAME /regex/ This would add flexibilty to rule creation. It may also be almost free in execution time terms because the current body rule must already require the subject text and body text to be extracted from the message before concatenating them. Martin
Re: Body rules hit on Subject
On Fri, 2 Feb 2018, Kevin A. McGrail wrote: On 2/2/2018 1:41 PM, Alex wrote: Hi, I recall a conversation in the past about body rules hitting on the Subject of an email, but I don't believe there was ever a solution, or I otherwise missed it. It's now affecting me again, and I hoped someone had some ideas on how to prevent body rules from hitting on the subject too since it's apparently considered part of the message body? There is no solution at the moment. The subject is appended to the body of the text for rule parsing. I've added a task I plan to submit for GSOC consideration to add a tflag to disable this behavior. Globally, or per-rule? -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Users mistake widespread adoption of Microsoft Office for the development of a document format standard. --- 4 days until the first Falcon Heavy test launch
Re: Body rules hit on Subject
On 2 Feb 2018, at 14:27, Kris Deugau wrote: > The only "solution" I've ever come up with is to create a meta rule group to > account for the Subject hit: > > body __FOO /foo/ > header __SUBJ_FOO Subject =~ /foo/ > meta FOO __FOO && !__SUBJ_FOO > > I have to admit it's annoyed me on occasion that I can't create a single > simple rule that ONLY matches on the message body, but TBH it's never been > important enough in context for me to even commit the above horror. It seems the the number of times you want to match ONLY the body and not the body+subject is low enough math this workaround is reasonable. I mean, you could have a new category bodyonly, or something, but I doubt it's necessary. Certainly changing the behavior of body now would be a mistake. -- You start a conversation you can't even finish it You're talkin' a lot, but you're not sayin' anything When I have nothing to say, my lips are sealed Say something once, why say it again?
Re: Body rules hit on Subject
Hi, >> The only "solution" I've ever come up with is to create a meta rule group to >> account for the Subject hit: >> >> body __FOO /foo/ >> header __SUBJ_FOO Subject =~ /foo/ >> meta FOO __FOO && !__SUBJ_FOO >> >> I have to admit it's annoyed me on occasion that I can't create a single >> simple rule that ONLY matches on the message body, but TBH it's never been >> important enough in context for me to even commit the above horror. > > It seems the the number of times you want to match ONLY the body and not the > body+subject is low enough math this workaround is reasonable. > > I mean, you could have a new category bodyonly, or something, but I doubt > it's necessary. > > Certainly changing the behavior of body now would be a mistake. I've also had a problem when trying to write rules that rely on or otherwise measure the length of the body. A more complicated set of rules are needed for that, if it's even possible/reliable.
Re: Body rules hit on Subject
On 2/2/2018 6:09 PM, John Hardin wrote: There is no solution at the moment. The subject is appended to the body of the text for rule parsing. I've added a task I plan to submit for GSOC consideration to add a tflag to disable this behavior. Globally, or per-rule? A tflag affect just one rule. Regards, KAM
Re: Body rules hit on Subject
On 2/3/2018 2:28 PM, Alex wrote: I've also had a problem when trying to write rules that rely on or otherwise measure the length of the body. A more complicated set of rules are needed for that, if it's even possible/reliable. Take a look at these rules for an example: 72_active.cf: body __KAM_BODY_LENGTH_LT_1024 eval:check_body_length('1024') 72_active.cf: body __KAM_BODY_LENGTH_LT_128 eval:check_body_length('128') 72_active.cf: body __KAM_BODY_LENGTH_LT_256 eval:check_body_length('256') 72_active.cf: body __KAM_BODY_LENGTH_LT_512 eval:check_body_length('512') Regards, KAM
Re: Body rules hit on Subject
On 2 Feb 2018, at 16:59 (-0500), Kevin A. McGrail wrote: There is no solution at the moment. The subject is appended to the body of the text for rule parsing. The 2nd sentence is wrong: the subject is *prepended* to the body. Also: the 1st sentence is wrong, there's no *PRETTY* solution. If every rendered 'body' starts with an appended line containing the Subject (with '^Subject: ' stripped off) then one can solve the problem of matching body rules in the Subject header thus: body__DOCUSIGN_BODY_1ST /\A.*\bdocusign\b.*\n/mi body__DOCUSIGN_BODY_NOT1ST /(?!\A).*\bdocusign\b.*\n/mi meta DOCUSIGN_BODY (HAS_SUBJECT && __DOCUSIGN_BODY_NOT1ST) || (__DOCUSIGN_BODY_1ST || __DOCUSIGN_BODY_NOT1ST) -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) Currently Seeking Steady Work: https://linkedin.com/in/billcole
Re: Body rules hit on Subject
On 3 Feb 2018, at 16:37 (-0500), Bill Cole wrote: On 2 Feb 2018, at 16:59 (-0500), Kevin A. McGrail wrote: There is no solution at the moment. The subject is appended to the body of the text for rule parsing. The 2nd sentence is wrong: the subject is *prepended* to the body. Also: the 1st sentence is wrong, there's no *PRETTY* solution. If every rendered 'body' starts with an appended line containing the Subject (with '^Subject: ' stripped off) then one can solve the problem of matching body rules in the Subject header thus: body__DOCUSIGN_BODY_1ST /\A.*\bdocusign\b.*\n/mi body__DOCUSIGN_BODY_NOT1ST /(?!\A).*\bdocusign\b.*\n/mi meta DOCUSIGN_BODY (HAS_SUBJECT && __DOCUSIGN_BODY_NOT1ST) || (__DOCUSIGN_BODY_1ST || __DOCUSIGN_BODY_NOT1ST) make that: meta DOCUSIGN_BODY (HAS_SUBJECT && __DOCUSIGN_BODY_NOT1ST) || (MISSING_SUBJECT && (__DOCUSIGN_BODY_1ST || __DOCUSIGN_BODY_NOT1ST)) -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) Currently Seeking Steady Work: https://linkedin.com/in/billcole
Re: Body rules hit on Subject
On Sat, 3 Feb 2018, Alex wrote: Hi, The only "solution" I've ever come up with is to create a meta rule group to account for the Subject hit: body __FOO /foo/ header __SUBJ_FOO Subject =~ /foo/ meta FOO __FOO && !__SUBJ_FOO I have to admit it's annoyed me on occasion that I can't create a single simple rule that ONLY matches on the message body, but TBH it's never been important enough in context for me to even commit the above horror. It seems the the number of times you want to match ONLY the body and not the body+subject is low enough math this workaround is reasonable. I mean, you could have a new category bodyonly, or something, but I doubt it's necessary. Certainly changing the behavior of body now would be a mistake. I've also had a problem when trying to write rules that rely on or otherwise measure the length of the body. A more complicated set of rules are needed for that, if it's even possible/reliable. Q'n'D: header __SUBJ_LENGTHSubject =~ /./ tflags __SUBJ_LENGTHmultiple body__BODY_LENGTH/./ tflags __BODY_LENGTHmultiple Inefficient as hell, but it should work. Better to use eval:check_body_length() if you can, though. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- After ten years (1998-2008) of draconian gun control in the State of Massachusetts, the results are in: firearms-related assaults up 78%, firearms-related homicides up 67%, assault-related emergency room visits up 331%. Gun Control does not reduce violent crime. --- 3 days until the first Falcon Heavy test launch