It looks like you're using the wrong sort of quotes in your "matchers".
- matchers: [ 'realm = middleEarth-moria.middle-earth.com' ] #
this works
- matchers: [ realm = "middleEarth-moria.middle-earth.com" ] #
this works
- matchers: [ realm = 'middleEarth-moria.middle-earth.com' ] #
THIS SILENTLY MATCHES EVERYTHING!
- matchers: [ "realm = 'middleEarth-moria.middle-earth.com'" ]
# SO DOES THIS!
For me, it works correctly, at least in the web tool, if you rewrite to the
following:
route:
routes:
- matchers: [ 'owner = middleEarth' ]
receiver: 'middleEarth-alerts'
routes:
- matchers: [ 'alertname = MordorThreatens' ]
receiver: 'middleEarth-alerts-prod-critical'
routes:
- matchers: [ 'realm =~ "middleEarth-.*.middle-earth.com"' ]
receiver: 'middleEarth-alerts-prod-critical'
continue: true
- matchers: [ 'realm = middleEarth-rohan.middle-earth.com' ]
receiver: 'rohan-alerts'
continue: true
- matchers: [ 'realm = middleEarth-rivendell.middle-earth.com' ]
receiver: 'rivendell-alerts'
continue: true
- matchers: [ 'realm = middleEarth-shire.middle-earth.com' ]
receiver: 'shire-alerts'
continue: true
- matchers: [ 'realm = middleEarth-moria.middle-earth.com' ]
receiver: 'moria-alerts'
continue: true
(Note that I've also quoted the entire matcher expression, to be sure that
it doesn't get broken up by YAML parsing)
I don't know what's going on under the hood, and perhaps alertmanager ought
to give some sort of error if it sees a rule it doesn't understand, rather
than silently passing the test.
On Tuesday, 18 July 2023 at 20:28:32 UTC+1 realElonMusk wrote:
> Hello Brian,
>
> Thank you for your response and for suggesting the use of the tool. After
> using the tool with my labels, I observed that all routes under *alertname
> = MordorThreatens* are highlighted. This result is not what I expected
> based on my understanding of the routing configuration.
>
> My expectation was that only the route with *realm =
> 'middleEarth-shire.middle-earth.com
> <http://middleEarth-shire.middle-earth.com>'* should be highlighted given
> the label configuration I am testing with. Could you help me understand why
> all routes under *alertname = MordorThreatens* are highlighted instead?
>
> Here are the labels I used for testing:
>
> *{owner="middleEarth", alertname="MordorThreatens",
> realm="middleEarth-shire.middle-earth.com
> <http://middleEarth-shire.middle-earth.com>"}*
>
> Any further insights would be greatly appreciated.
>
> On Tuesday, July 18, 2023 at 2:25:59 PM UTC-4 Brian Candler wrote:
>
>> Quite possibly *none* of your sub-matchers are matching, and it's falling
>> back to the default receiver 'middleEarth-alerts-prod-critical' which is at
>> the same level as routes:
>>
>> route:
>> routes:
>> - matchers: [ owner = middleEarth ]
>> receiver: 'middleEarth-alerts'
>> routes:
>> - matchers: [ alertname = MordorThreatens ]
>> * receiver: 'middleEarth-alerts-prod-critical' << this is
>> used if *none* of the routes below match*
>> routes: ...
>>
>> But without seeing your actual alert labels and conditions I can't give
>> any more help.
>>
>> There is an alert route testing tool you can use online, and you can
>> paste in your *real* labels and matchers:
>> https://prometheus.io/webtools/alerting/routing-tree-editor/
>>
>> On Tuesday, 18 July 2023 at 18:42:13 UTC+1 realElonMusk wrote:
>>
>>> Hello,
>>>
>>> I'm experiencing an issue with the *continue: true* option in my
>>> Prometheus alert routing setup. Here's the configuration:
>>>
>>> route:
>>> routes:
>>> - matchers: [ owner = middleEarth ]
>>> receiver: 'middleEarth-alerts'
>>> routes:
>>> - matchers: [ alertname = MordorThreatens ]
>>> receiver: 'middleEarth-alerts-prod-critical'
>>> routes:
>>> - matchers: [ realm =~ 'middleEarth-.*.middle-earth.com' ]
>>> receiver: 'middleEarth-alerts-prod-critical'
>>> continue: true
>>> - matchers: [ realm = 'middleEarth-rohan.middle-earth.com' ]
>>> receiver: 'rohan-alerts'
>>> continue: true
>>> - matchers: [ realm = '
>>> middleEarth-rivendell.middle-earth.com' ]
>>> receiver: 'rivendell-alerts'
>>> continue: true
>>> - matchers: [ realm = 'middleEarth-shire.middle-earth.com' ]
>>> receiver: 'shire-alerts'
>>> continue: true
>>> - matchers: [ realm = 'middleEarth-moria.middle-earth.com' ]
>>> receiver: 'moria-alerts'
>>> continue: true
>>>
>>> In this setup, when an alert with realm='
>>> middleEarth-shire.middle-earth.com' is triggered, it successfully
>>> matches against the first route and is routed to the
>>> 'middleEarth-alerts-prod-critical' receiver as expected. However, the
>>> routing doesn't continue to the next matchers. Specifically, the alert is
>>> never matched against realm='middleEarth-shire.middle-earth.com' to be
>>> sent to the 'shire-alerts' receiver, even though *continue: true* is
>>> set.
>>>
>>> I've verified the labels and they seem to be correct. Why does the alert
>>> routing not continue after the first match? Any insights on how to resolve
>>> this issue would be greatly appreciated.
>>>
>>> Thank you.
>>>
>>>
>>>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/86408c8e-02ac-41ae-89f2-c0134ab48d3en%40googlegroups.com.
