I think until we officially migrate to ES 5.x you should write code that would
be compatible with ES 2.x (if you want that code to be generally consumable by
the Metron community).
04.10.2017, 18:04, "Laurens Vets" <laur...@daemon.be>:
> It's working now, so I'm happy :)
>
> On 2017-10-04 14:03, Casey Stella wrote:
>> Ok, so this is subtle. Your rules are wrong and I totally understand
>> why
>> you thought they were right.
>>
>> When we index into ES, we take . and convert them to :, however PRIOR
>> to
>> indexing (when threat triage is running) those fields have .'s not :'s
>> Therefore, your rules should be:
>>
>> userIdentity.sessionContext.attributes.mfaAuthenticated == 'False'
>> and
>> additionalEventData.MFAUsed == 'No'
>>
>> The same general argument goes for your threat triage stellar
>> expressions.
>>
>> Sorry about the confusion, we do that mapping because ES doesn't handle
>> those .'s well. Hey, maybe ES 5 is more sane about that sort of thing
>> and
>> we can avoid doing that transformation.
>>
>> Casey
>>
>> On Wed, Oct 4, 2017 at 4:38 PM, Laurens Vets <laur...@daemon.be> wrote:
>>
>>> No idea whether it's a bug yet, I just need a 2nd set of eyes :)
>>>
>>> This is my event as indexed in ES (Obviously some parts have been
>>> obfuscated):
>>>
>>> {
>>> "_index": "cloudtrail_index_2017.10.04.19",
>>> "_type": "cloudtrail_doc",
>>> "_id": "95617686-bd39-46ff-b5c0-db3aeb5b6bab",
>>> "_score": null,
>>> "_timestamp": 1507143907108,
>>> "_source": {
>>> "eventID": "9e3d5468-2d97-4b9a-9821-5c61fec8c158",
>>> "additionalEventData:MFAUsed": "No",
>>> "adapter:stellaradapter:end:ts": "1507143907145",
>>> "threatinteljoinbolt:joiner:ts": "1507143907153",
>>> "eventVersion": "1.05",
>>> "threat:triage:rules:0:comment": "Checks whether the field is_work
>>> is
>>> true or false.",
>>> "sourceIPAddress": "208.110.73.106",
>>> "eventSource": "signin.amazonaws.com",
>>> "enrichmentsplitterbolt:splitter:begin:ts": "1507143907143",
>>> "enrichmentjoinbolt:joiner:ts": "1507143907147",
>>> "additionalEventData:MobileVersion": "No",
>>> "threat:triage:rules:0:name": "Not WORK",
>>> "source:type": "cloudtrail",
>>> "original_string": "{\"eventVersion\":\"1.05\",\"
>>> userIdentity\":{\"type\":\"IAMUser\",\"principalId\":\"AIDAI
>>> 5ITCMVR3BQV5DUFW\",\"arn\":\"arn:aws:iam::<ACCOUNTID>:user/
>>> <EMAIL>\",\"accountId\":\"<ACCOUNTID>\",\"userName\":\"<
>>> EMAIL>\"},\"eventTime\":\"2017-10-04T18:57:31Z\",\"eventSource\":\"
>>> signin.amazonaws.com\",\"eventName\":\"ConsoleLogin\"
>>> ,\"awsRegion\":\"us-east-1\",\"sourceIPAddress\":\"208.110.7
>>> 3.106\",\"userAgent\":\"Mozilla/5.0 (X11; Ubuntu; Linux x86_64;
>>> rv:56.0)
>>> Gecko/20100101 Firefox/56.0\",\"requestParame
>>> ters\":null,\"responseElements\":{\"ConsoleLogin\":\"
>>> Success\"},\"additionalEventData\":{\"LoginTo\":\"https://
>>> console.aws.amazon.com/console/home?state=hashArgs%23&isauthcode=true\
>>>
>>> <https://console.aws.amazon.com/console/home?state=hashArgs%23&isauthcode=true%5C>
>>> ",\"MobileVersion\":\"No\",\"MFAUsed\":\"No\"},\"eventID\":
>>> \"9e3d5468-2d97-4b9a-9821-5c61fec8c158\",\"eventType\":\
>>> "AwsConsoleSignIn\",\"recipientAccountId\":\"<ACCOUNTID>\"}",
>>> "eventTime": "2017-10-04T18:57:31Z",
>>> "eventName": "ConsoleLogin",
>>> "recipientAccountId": "<ACCOUNTID>",
>>> "userIdentity:principalId": "AIDAI5ITCMVR3BQV5DUFW",
>>> "threatintelsplitterbolt:splitter:end:ts": "1507143907148",
>>> "threat:triage:rules:0:score": 20,
>>> "timestamp": 1507143907108,
>>> "threat:triage:rules:0:reason": "208.110.73.106 is not an WORK
>>> network!",
>>> "awsRegion": "us-east-1",
>>> "is_work": false,
>>> "userIdentity:userName": "<EMAIL>",
>>> "enrichmentsplitterbolt:splitter:end:ts": "1507143907143",
>>> "threat:triage:score": 20,
>>> "is_alert": "true",
>>> "userAgent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:56.0)
>>> Gecko/20100101 Firefox/56.0",
>>> "adapter:stellaradapter:begin:ts": "1507143907145",
>>> "eventType": "AwsConsoleSignIn",
>>> "userIdentity:arn": "arn:aws:iam::<ACCOUNTID>:user/<EMAIL>",
>>> "userIdentity:accountId": "<ACCOUNTID>",
>>> "userIdentity:type": "IAMUser",
>>> "threatintelsplitterbolt:splitter:begin:ts": "1507143907148",
>>> "guid": "95617686-bd39-46ff-b5c0-db3aeb5b6bab",
>>> "additionalEventData:LoginTo": "https://console.aws.amazon.co
>>> m/console/home?state=hashArgs%23&isauthcode=true",
>>> "responseElements:ConsoleLogin": "Success"
>>> },
>>> "fields": {
>>> "adapter:stellaradapter:end:ts": [
>>> 1507143907145
>>> ],
>>> "threatinteljoinbolt:joiner:ts": [
>>> 1507143907153
>>> ],
>>> "enrichmentsplitterbolt:splitter:end:ts": [
>>> 1507143907143
>>> ],
>>> "enrichmentsplitterbolt:splitter:begin:ts": [
>>> 1507143907143
>>> ],
>>> "enrichmentjoinbolt:joiner:ts": [
>>> 1507143907147
>>> ],
>>> "adapter:stellaradapter:begin:ts": [
>>> 1507143907145
>>> ],
>>> "eventTime": [
>>> 1507143451000
>>> ],
>>> "threatintelsplitterbolt:splitter:begin:ts": [
>>> 1507143907148
>>> ],
>>> "threatintelsplitterbolt:splitter:end:ts": [
>>> 1507143907148
>>> ],
>>> "timestamp": [
>>> 1507143907108
>>> ]
>>> },
>>> "sort": [
>>> 1507143451000
>>> ]
>>> }
>>>
>>> This is my sensor configuration:
>>>
>>> {
>>> "enrichment": {
>>> "fieldMap": {
>>> "stellar": {
>>> "config": {
>>> "is_work": "IN_SUBNET(if
>>> IS_IP(sourceIPAddress) then sourceIPAddress else NULL, '1.2.3.4/16', '
>>> 5.6.7.8/23')"
>>> }
>>> }
>>> },
>>> "fieldToTypeMap": {},
>>> "config": {}
>>> },
>>> "threatIntel": {
>>> "fieldMap": {
>>> "stellar": {
>>> "config": [
>>> "is_alert := exists(is_work)
>>> &&
>>> is_work != true && eventName == \"ConsoleLogin\"",
>>> "is_alert := is_alert ||
>>> (eventName == \"ConsoleLogin\" &&
>>> userIdentity:sessionContext:attributes:mfaAuthenticated
>>> == \"False\")",
>>> "is_alert := is_alert ||
>>> (eventName == \"ConsoleLogin\" && additionalEventData:MFAUsed ==
>>> \"No\")"
>>> ]
>>> }
>>> },
>>> "fieldToTypeMap": {},
>>> "config": {},
>>> "triageConfig": {
>>> "riskLevelRules": [
>>> {
>>> "name": "Not WORK",
>>> "comment": "Checks whether the
>>> field is_work is true or false.",
>>> "rule": "is_work == false",
>>> "score": 20,
>>> "reason": "FORMAT('%s is not
>>> an
>>> WORK network!', sourceIPAddress)"
>>> },
>>> {
>>> "name": "MFA",
>>> "comment": "Checks whether MFA
>>> used or not.",
>>> "rule":
>>> "userIdentity:sessionContext:attributes:mfaAuthenticated == 'False'",
>>> "score": 20,
>>> "reason": null
>>> },
>>> {
>>> "name": "MFA2",
>>> "comment": "Checks whether MFA
>>> used or not.",
>>> "rule":
>>> "additionalEventData:MFAUsed == 'No'",
>>> "score": 20,
>>> "reason": null
>>> }
>>> ],
>>> "aggregator": "SUM",
>>> "aggregationConfig": {}
>>> }
>>> },
>>> "configuration": {}
>>> }
>>>
>>> Any idea why the score isn't 40? I would expect riskLevelRule 1 & 2 to
>>> be
>>> SUMmed?
-------------------
Thank you,
James Sirota
PPMC- Apache Metron (Incubating)
jsirota AT apache DOT org
