[prometheus-users] How to add custom labels to the default Kubernetes rules?

[radhamani...@gmail.com]

How to add custom labels to the default Kubernetes rules in 
Prometheus-operator? I found we cannot update the existing default rules 
but we need to add new rules in the templates.how to add that? and what 
should be name the alerts?same name so it will override the defaults?

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/f3b8b182-5919-4a0a-8c28-9276b66529e8n%40googlegroups.com.

[prometheus-users] What does count by job rule mean?

[radhamani...@gmail.com]

Targetdown alert rule shows before PromQL query,
(count by(job, namespace, service) (up == 0) / count by(job, namespace, 
service) (up)) > 10 


what does job,service mean in this rule?
Why is there (up) in the ount by(job, namespace, service) (up) 

 
? I understand what is up==0.But I am not sure what is just (up) means in 
the denominator.

If I want to customize this query,how can I do this? is there a document 
reference for it? I am looking for Prometheus operator alert rule.

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/a7858aa1-a383-458b-b74e-587e9802a853n%40googlegroups.com.

[prometheus-users] How to monitor PODS that are in error state

[Kirti Ranjan Parida]

How to monitor PODS that are in error state

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/1ec2f1ba-ae8a-46c4-8224-352e6ea08131n%40googlegroups.com.

[prometheus-users] Issues with prometheus rules once thanos is enabled

[Rodrigo Martinez]

I have noticed that when enabling thanos I now see in prometheus logs of 

Error executing query: found duplicate series for the match group 
{namespace="monitoring", pod="kube-state-metrics-567789848b-9d77w"} on the 
right hand-side of the operation: 
[{__name__="node_namespace_pod:kube_pod_info:", namespace="monitoring", 
node="test01", pod="kube-state-metrics-567789848b-9d77w"}, 
{__name__="node_namespace_pod:kube_pod_info:", namespace="monitoring", 
node="test01", pod="kube-state-metrics-567789848b-9d77w"}];many-to-many 
matching not allowed: matching labels must be unique on one side

Primarily this is coming from kube-state-metrics job and other 
prometheusrule with metrics that have honor_labels set to true 



Just wondering overall how enabling thanos causes these errors now 

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/2da7a037-f8f1-4e84-b238-c7e45ae54498n%40googlegroups.com.

[prometheus-users] kubernetes service discovery quesition

[swapnil surve]

Hi,

When using kubernetes sd config in prometheus, how frequently prometheus 
scrapes kubernetes master to get active targets to scrape? I think it does 
`watch` call to kubernetes master. That means it holds the connection to 
master. Does prometheus periodically close the connections? If there are 
many kubernetes sd configs, doesn't it overload master with many 
connections that remain active for long duration?

Regards,
Swapnil,

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/a66c508d-37c0-4e1c-874c-57aa2e6e34a4n%40googlegroups.com.

[prometheus-users] Is it possible to rename/edit existing label values for targets?

['a z' via Prometheus Users]

We currently add a label to the target scrape job to give a bit of context 
to the service the host provides and route alerts based on service. After 
reviewing some targets, I noticed the service label is incorrect in a few 
cases so have attempted to update the configuration.

 

Following the configuration update, I can see the new metric data now 
includes the update service label (as I intended) however all the old 
metric data still uses the old label value.  While this is not a major 
issue, it has caused some issues with our dashboards when displaying 
historical data as essential its see’s two different scraped targets. 

 

Is there anyway to update/edit all historical metrics which match a certain 
label and value? From what I can tell the Prometheus HTTP API does not seem 
to provide this functionality?  

 

Any input would be appreciated. Thank you

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/3efc4c1a-7101-4ea7-9b56-65540169e5ean%40googlegroups.com.

[prometheus-users] Re: [blackbox exporter] server returned HTTP 400 Bad Request

[Brian Candler]

Firstly, your curl data includes "NO_XX":"317502141012" but your blackbox 
body has "NXK" twice (which is invalid JSON).

Secondly, run curl with -v so you can see the response code.  It's possible 
that the response code is 400 even when you get the answer you expect.

Thirdly, if that still doesn't solve the problem, then run tcpdump to 
capture the traffic, and compare what curl is sending with what 
blackbox_exporter is sending.

tcpdump -i eth0 -nn -s0 -A host 1x.1xx.7x.x and tcp port 48002

(ignore all the binary data from packet headers, and just look at the HTTP 
request from POST onwards).  Note also that you can test blackbox_exporter 
without prometheus:

curl -vg 
'xosb04:9115/probe?target=http%3A%2F%2F1x.1xx.7x.x%3A48002%2Fdukil%2Fget_json%2Fisat%2Fcall_nik&module=http_post_2xx'

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/7f827a4a-047f-441b-b0a8-4115dda46f06o%40googlegroups.com.

[prometheus-users] [blackbox exporter] server returned HTTP 400 Bad Request

[Julian Ade Putra]

Hello,

I use blackbox exporter to monitoring API Endpoint. I able to API using 
json paramter and POST method manually by curl command with below result :

[weblogic@xosb04 mwteam]$ curl -d "@data.json" -X POST -H 
"Content-Type: application/json" 
http://1x.1xx.7x.x:48002/dukil/get_json/isat/call_nik
{"content":[{"NO_XX":317502141012,"NXK":324410130005,"NAME_LP":"DUMMY","AGM":"DUMMY","KAB_NAME":"DUMMY","NO_RW":"00","KEC_NAME":"DUMMY","JEN_PKRJN":"DUMMY","NO_RT":"00","ALT":"DUMMY","TMPT_LHR":"DUMMY","PDDK_AKH":"DUMMY","NM_LGKP_BU":"DUMMY","PROP_NAME":"DUMMY","KEL_NAME":"DUMMY","JNS_KLMIN":"Li
 
Li","TGL_LER":"28-02-2018"}],"lastPage":true,"numberOfElements":1,"sort":null,"totalElements":1,"firstPage":true,"number":0,"size":1}

where is data.json :

{
"NXK":"324410130005",
"NO_XX":"317502141012",
"NM_LGKP_BU":"",
"user_id":"68317101X",
"password":"XX",
"msisdn":"123456789",
"IP_User":"XXX"
}

But, when i setup in blackbox exporter with same json paramater its error - 
server returned HTTP status 400 Bad Request
 
Here is my configuration :

*blackbox.yml :*

modules:

  http_post_2xx:
prober: http
timeout: 5s
http:
  valid_status_codes: [200]
  method: POST
  headers:
Content-Type: application/json
  body: '{"NXK": "324410130005","NXK": "317502141012","NM_LGKP_BU": 
"","user_id": "68317101X","password": "XX","msisdn": 
"123456789","IP_User": "XXX"}'

*prometheus.yml :*

  - job_name: 'blackbox'
metrics_path: /probe
params:
  module: [http_post_2xx]
static_configs:
  - targets:
- http://1x.1xx.7x.x:48002/dukil/get_json/isat/call_nik
relabel_configs:
  - source_labels: [__address__]
target_label: __param_target
  - source_labels: [__param_target]
target_label: instance
  - target_label: __address__
replacement:  xosb04:9115

Please advise if i missed something in my configuration?


Thank You!

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/0efeac8a-425b-4bd0-84f8-17c0160eeb48n%40googlegroups.com.

[prometheus-users] Re: Regexp re-writes on Legend Format -grafana

[goel....@gmail.com]

Thanks a lot Brian, all this really helps very much.


On Monday, 24 August 2020 at 16:35:31 UTC+5:30 b.ca...@pobox.com wrote:

> On Monday, 24 August 2020 10:37:22 UTC+1, goel@gmail.com wrote:
>>
>> Thanks a lot Brian this works, 
>>
>> Can you please suggest me how can i get more info on this type of issue 
>> so that it can be tackled be me in future.
>>
>
> I'm not sure what you mean.
>
> There are lots of good blog posts on prometheus: the ones especially worth 
> reading are those by Brian Brazil at https://www.robustperception.io/blog
> including the older ones which cover fundamentals.  Here are some I have 
> bookmarked:
> https://www.robustperception.io/using-time-series-as-alert-thresholds
>
> https://www.robustperception.io/target-labels-are-for-life-not-just-for-christmas/
> https://www.robustperception.io/controlling-the-instance-label
> https://www.robustperception.io/on-the-naming-of-things/
>
> https://www.robustperception.io/running-into-burning-buildings-because-the-fire-alarm-stopped
> https://www.robustperception.io/how-to-have-labels-for-machine-roles
> https://www.robustperception.io/exposing-the-software-version-to-prometheus
>
> For label_replace, I just read the prometheus documentation on functions:
> https://prometheus.io/docs/prometheus/latest/querying/functions/
>
> It does assume you know about regular expressions.  There is much generic 
> information about regular expressions on the Internet.  The regular 
> expressions used by Prometheus come from the Go regexp library which 
> implements a flavour called RE2:
> https://github.com/google/re2/wiki/Syntax
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/ec6d9bb9-485c-4e9e-9f6b-dcd1f8aa784bn%40googlegroups.com.

[prometheus-users] Re: Regexp re-writes on Legend Format -grafana

[Brian Candler]

On Monday, 24 August 2020 10:37:22 UTC+1, goel@gmail.com wrote:
>
> Thanks a lot Brian this works, 
>
> Can you please suggest me how can i get more info on this type of issue so 
> that it can be tackled be me in future.
>

I'm not sure what you mean.

There are lots of good blog posts on prometheus: the ones especially worth 
reading are those by Brian Brazil at https://www.robustperception.io/blog
including the older ones which cover fundamentals.  Here are some I have 
bookmarked:
https://www.robustperception.io/using-time-series-as-alert-thresholds
https://www.robustperception.io/target-labels-are-for-life-not-just-for-christmas/
https://www.robustperception.io/controlling-the-instance-label
https://www.robustperception.io/on-the-naming-of-things/
https://www.robustperception.io/running-into-burning-buildings-because-the-fire-alarm-stopped
https://www.robustperception.io/how-to-have-labels-for-machine-roles
https://www.robustperception.io/exposing-the-software-version-to-prometheus

For label_replace, I just read the prometheus documentation on functions:
https://prometheus.io/docs/prometheus/latest/querying/functions/

It does assume you know about regular expressions.  There is much generic 
information about regular expressions on the Internet.  The regular 
expressions used by Prometheus come from the Go regexp library which 
implements a flavour called RE2:
https://github.com/google/re2/wiki/Syntax


-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/3342b68c-7260-493d-bf26-22de564fb1bbo%40googlegroups.com.

Re: [prometheus-users] prometheus-snmp-generator error (Debian 10)

[Mario Pranjic]

Hi Ben,

I will gladly help. Just let me know how to approach the matter.
Initially I opened a request on Synology asking if they support Prometheus
node exporter, since it is debian-based Linux on DSM.
I could attempt to install it on my own, but then Synology support is off
and they don't guarantee anything (understandable, but not optimal).

So they mentioned snmp instead. And there we are, now. :)


man. 24. aug. 2020 kl. 11:47 skrev Ben Kochie :

> If we're going to talk to Synology, maybe we can get them to support
> Prometheus directly. I'd sell my QNAP if they did that. ;-)
>
> For example, here's a patch to the MIB file that helps with the types:
>
> https://gitlab.com/superq/mibs/-/commit/40ddc8af074961abfa89d2d94da9fd290e8e3f71
>
> On Mon, Aug 24, 2020 at 11:22 AM Mario Pranjic 
> wrote:
>
>> Hi,
>>
>> A small update: indeed there are errors in provided MIB files by
>> Synology. I ran some of them into
>> http://www.muonics.com/Tools/smicheck.php and reported some examples to
>> Synology.
>> A case is opened on the Synology side and they will send it to their dev
>> team. We will see if/when they come up with something useful.
>>
>> Best regards,
>>
>> Mario.
>>
>> lør. 22. aug. 2020 kl. 20:38 skrev Mario Pranjic > >:
>>
>>> Hi Ben,
>>>
>>> Thank you very much for your time and effort.
>>> Indeed, I am not an expert in snmp (that was always somehow left on TODO
>>> list). Maybe this is a good opportunity to fill in the gaps in knowledge. :)
>>>
>>> The thing is... With snmp.yml which comes as a result of that
>>> snmp_exporter role, I do get some communication with Synology from
>>> Prometheus, but the list is pretty awkward.
>>> By pointing browser to: http://127.0.0.1:9116/snmp?target=192.168.10.187
>>>  i get output as noted in list.txt file.
>>>
>>> I see no info about disks and raid.
>>>
>>> However, if I run snmpget from Prometheus/snmp_exporter node against
>>> Synology, I can read values documented by Synology.
>>>
>>> For example, disk temperature (disk0 and disk1):
>>> root@monnew:/# snmpget -c public -v 2c -O qv 192.168.10.187
>>> 1.3.6.1.4.1.6574.2.1.1.6.0
>>> 29
>>> root@monnew:/# snmpget -c public -v 2c -O qv 192.168.10.187
>>> 1.3.6.1.4.1.6574.2.1.1.6.1
>>> 31
>>>
>>> Or some  info:
>>> root@monnew:/# snmpget -c public -v 2c -O qv 192.168.10.187
>>> 1.3.6.1.4.1.6574.1.5.1.0
>>> "DS216j"
>>>
>>> I tried to use snmp.yml you provided in example or building one based on
>>> your example generator.yml, but in both cases
>>> http://127.0.0.1:9116/snmp?target=192.168.10.187 will return error:
>>> Unkown_module 'if_mib'
>>>
>>> The question why snmp_exporter is returning only a small portion of
>>> values data from Synology is, I presume, related to the snmp.yml file.
>>> How to generate the right one is an enigma. :/
>>>
>>> Attaching snmp.yml provisioned by snmp_exporter role for the reference.
>>>
>>> Best regards,
>>>
>>> Mario.
>>>
>>>
>>>
>>>
>>> lør. 22. aug. 2020 kl. 09:42 skrev Ben Kochie :
>>>
 You will probably need to have a good understanding of SNMP and MIBs in
 order create new generator configs. Unfortunately, the Synology MIBs have a
 large number of mistakes, making it even harder to work with.

 I created a simple example of how to use the generator based on what I
 found in the SYNOLOGY-RAID-MIB.

 https://github.com/SuperQ/tools/tree/master/snmp_exporter/Synology

 I corrected as many of the errors as I could, but Synology didn't
 format the raidStatus as an ENUM.

 On Fri, Aug 21, 2020 at 3:46 PM Mario Pranjic 
 wrote:

> Hi Ben,
>
> Community string was wrong. I fixed that. Now I can tell snmp_exporter
> can communicate properly.
>
> I can get in Prometheus:
> sysUpTime{instance="192.168.10.187",job="snmp"}
> And value is read from the device (incrementing on refresh too).
>
> But as for mibs, I still can't use mibs as described in the Synology
> snmp guide.
>
> https://global.download.synology.com/download/Document/Software/DeveloperGuide/Firmware/DSM/All/enu/Synology_DiskStation_MIB_Guide.pdf
>
> There is still a step missing in the snmp_exporter role in regards to
> mibs and/or proper snmp.yml generation.
>
> Best regards,
>
> Mario.
>
> fre. 21. aug. 2020 kl. 15:28 skrev Ben Kochie :
>
>> The SNMP exporter uses UDP for transport, so telnet to 161 isn't
>> going to help much. I've seen very few devices implement RFC 3430.
>>
>> There are usually two problems people run into.
>> * The community string is wrong (defaults to "public")
>> * The device has a management IP control and won't allow traffic.
>>
>> SNMP implementations generally don't reject incorrect authentication,
>> rather they simply drop the packets. So it can be hard to diagnose.
>>
>> On Fri, Aug 21, 2020 at 2:52 PM Mario Pranjic <
>> mario.pran...@gmail.com> wrote:
>>
>>> Hi Ben,
>>>
>

Re: [prometheus-users] prometheus-snmp-generator error (Debian 10)

[Ben Kochie]

If we're going to talk to Synology, maybe we can get them to support
Prometheus directly. I'd sell my QNAP if they did that. ;-)

For example, here's a patch to the MIB file that helps with the types:
https://gitlab.com/superq/mibs/-/commit/40ddc8af074961abfa89d2d94da9fd290e8e3f71

On Mon, Aug 24, 2020 at 11:22 AM Mario Pranjic 
wrote:

> Hi,
>
> A small update: indeed there are errors in provided MIB files by Synology.
> I ran some of them into  http://www.muonics.com/Tools/smicheck.php and
> reported some examples to Synology.
> A case is opened on the Synology side and they will send it to their dev
> team. We will see if/when they come up with something useful.
>
> Best regards,
>
> Mario.
>
> lør. 22. aug. 2020 kl. 20:38 skrev Mario Pranjic  >:
>
>> Hi Ben,
>>
>> Thank you very much for your time and effort.
>> Indeed, I am not an expert in snmp (that was always somehow left on TODO
>> list). Maybe this is a good opportunity to fill in the gaps in knowledge. :)
>>
>> The thing is... With snmp.yml which comes as a result of that
>> snmp_exporter role, I do get some communication with Synology from
>> Prometheus, but the list is pretty awkward.
>> By pointing browser to: http://127.0.0.1:9116/snmp?target=192.168.10.187
>>  i get output as noted in list.txt file.
>>
>> I see no info about disks and raid.
>>
>> However, if I run snmpget from Prometheus/snmp_exporter node against
>> Synology, I can read values documented by Synology.
>>
>> For example, disk temperature (disk0 and disk1):
>> root@monnew:/# snmpget -c public -v 2c -O qv 192.168.10.187
>> 1.3.6.1.4.1.6574.2.1.1.6.0
>> 29
>> root@monnew:/# snmpget -c public -v 2c -O qv 192.168.10.187
>> 1.3.6.1.4.1.6574.2.1.1.6.1
>> 31
>>
>> Or some  info:
>> root@monnew:/# snmpget -c public -v 2c -O qv 192.168.10.187
>> 1.3.6.1.4.1.6574.1.5.1.0
>> "DS216j"
>>
>> I tried to use snmp.yml you provided in example or building one based on
>> your example generator.yml, but in both cases
>> http://127.0.0.1:9116/snmp?target=192.168.10.187 will return error:
>> Unkown_module 'if_mib'
>>
>> The question why snmp_exporter is returning only a small portion of
>> values data from Synology is, I presume, related to the snmp.yml file.
>> How to generate the right one is an enigma. :/
>>
>> Attaching snmp.yml provisioned by snmp_exporter role for the reference.
>>
>> Best regards,
>>
>> Mario.
>>
>>
>>
>>
>> lør. 22. aug. 2020 kl. 09:42 skrev Ben Kochie :
>>
>>> You will probably need to have a good understanding of SNMP and MIBs in
>>> order create new generator configs. Unfortunately, the Synology MIBs have a
>>> large number of mistakes, making it even harder to work with.
>>>
>>> I created a simple example of how to use the generator based on what I
>>> found in the SYNOLOGY-RAID-MIB.
>>>
>>> https://github.com/SuperQ/tools/tree/master/snmp_exporter/Synology
>>>
>>> I corrected as many of the errors as I could, but Synology didn't format
>>> the raidStatus as an ENUM.
>>>
>>> On Fri, Aug 21, 2020 at 3:46 PM Mario Pranjic 
>>> wrote:
>>>
 Hi Ben,

 Community string was wrong. I fixed that. Now I can tell snmp_exporter
 can communicate properly.

 I can get in Prometheus:
 sysUpTime{instance="192.168.10.187",job="snmp"}
 And value is read from the device (incrementing on refresh too).

 But as for mibs, I still can't use mibs as described in the Synology
 snmp guide.

 https://global.download.synology.com/download/Document/Software/DeveloperGuide/Firmware/DSM/All/enu/Synology_DiskStation_MIB_Guide.pdf

 There is still a step missing in the snmp_exporter role in regards to
 mibs and/or proper snmp.yml generation.

 Best regards,

 Mario.

 fre. 21. aug. 2020 kl. 15:28 skrev Ben Kochie :

> The SNMP exporter uses UDP for transport, so telnet to 161 isn't going
> to help much. I've seen very few devices implement RFC 3430.
>
> There are usually two problems people run into.
> * The community string is wrong (defaults to "public")
> * The device has a management IP control and won't allow traffic.
>
> SNMP implementations generally don't reject incorrect authentication,
> rather they simply drop the packets. So it can be hard to diagnose.
>
> On Fri, Aug 21, 2020 at 2:52 PM Mario Pranjic 
> wrote:
>
>> Hi Ben,
>>
>> I ran another VM and used roles:
>> -  https://github.com/cloudalchemy/ansible-snmp-exporter
>> -  https://github.com/cloudalchemy/ansible-prometheus
>>
>> along with Grafana. Prometheus, node.exporter and Grafana work as
>> intended (as before).
>>
>> However, when I try to enable snmp_exporter:
>>   - job_name: snmp
>> metrics_path: /snmp
>> params:
>>   module:
>>   - if_mib
>> relabel_configs:
>> - source_labels:
>>   - __address__
>>   target_label: __param_target
>> - source_labels:
>>   - __pa

Re: [prometheus-users] mysql 5.1 working config

[deln...@gmail.com]

Thanks. Sadly no Go skills. Unless it's smth trivial. 

On Monday, August 24, 2020 at 10:44:12 AM UTC+2 sup...@gmail.com wrote:

> Ahh, I see, yes, it looks like that was added in msqld_exporter 0.11, so 
> you would have to run 0.10 or older.
>
> It looks like lock_wait_timeout 
> 
>  was 
> introduced in 5.5.
>
> It looks like the way we implemented this is always enabled. I would 
> accept a PR to make it optional.
>
> On Mon, Aug 24, 2020 at 10:13 AM deln...@gmail.com  
> wrote:
>
>> Think I'm already starting exporter with these collectors disabled - 
>> here's the no-collector arguments section:
>> --no-collect.info_schema.innodb_metrics 
>> --no-collect.info_schema.innodb_cmp --no-collect.info_schema.innodb_cmpmem
>>
>> nohup.out
>> time="2020-08-24T01:08:57-07:00" level=info msg="Starting mysqld_exporter 
>> (version=0.12.1, branch=HEAD, 
>> revision=48667bf7c3b438b5e93b259f3d17b70a7c9aff96)" 
>> source="mysqld_exporter.go:257"
>> time="2020-08-24T01:08:57-07:00" level=info msg="Build context 
>> (go=go1.12.7, user=root0b3e56a7bc0a, date=20190729-12:35:58)" 
>> source="mysqld_exporter.go:258"
>> time="2020-08-24T01:08:57-07:00" level=info msg="Enabled scrapers:" 
>> source="mysqld_exporter.go:269"
>> time="2020-08-24T01:08:57-07:00" level=info msg=" 
>> --collect.global_status" source="mysqld_exporter.go:273"
>> time="2020-08-24T01:08:57-07:00" level=info msg=" 
>> --collect.global_variables" source="mysqld_exporter.go:273"
>> time="2020-08-24T01:08:57-07:00" level=info msg=" --collect.slave_status" 
>> source="mysqld_exporter.go:273"
>> time="2020-08-24T01:08:57-07:00" level=info msg=" 
>> --collect.info_schema.query_response_time" source="mysqld_exporter.go:273"
>> time="2020-08-24T01:08:57-07:00" level=info msg="Listening on 
>> 0.0.0.0:9104" source="mysqld_exporter.go:283"
>> time="2020-08-24T01:09:01-07:00" level=error msg="Error pinging mysqld: 
>> Error 1193: Unknown system variable 'lock_wait_timeout'" 
>> source="exporter.go:146"
>> time="2020-08-24T01:09:06-07:00" level=error msg="Error pinging mysqld: 
>> Error 1193: Unknown system variable 'lock_wait_timeout'" 
>> source="exporter.go:146"
>> time="2020-08-24T01:09:11-07:00" level=error msg="Error pinging mysqld: 
>> Error 1193: Unknown system variable 'lock_wait_timeout'" 
>> source="exporter.go:146"
>> time="2020-08-24T01:09:16-07:00" level=error msg="Error pinging mysqld: 
>> Error 1193: Unknown system variable 'lock_wait_timeout'" 
>> source="exporter.go:146"
>> time="2020-08-24T01:09:21-07:00" level=error msg="Error pinging mysqld: 
>> Error 1193: Unknown system variable 'lock_wait_timeout'" 
>> source="exporter.go:146"
>> time="2020-08-24T01:09:26-07:00" level=error msg="Error pinging mysqld: 
>> Error 1193: Unknown system variable 'lock_wait_timeout'" 
>> source="exporter.go:146"
>> time="2020-08-24T01:09:31-07:00" level=error msg="Error pinging mysqld: 
>> Error 1193: Unknown system variable 'lock_wait_timeout'" 
>> source="exporter.go:146"
>> time="2020-08-24T01:09:36-07:00" level=error msg="Error pinging mysqld: 
>> Error 1193: Unknown system variable 'lock_wait_timeout'" 
>> source="exporter.go:146"
>> time="2020-08-24T01:09:41-07:00" level=error msg="Error pinging mysqld: 
>> Error 1193: Unknown system variable 'lock_wait_timeout'" 
>> source="exporter.go:146"
>> time="2020-08-24T01:09:46-07:00" level=error msg="Error pinging mysqld: 
>> Error 1193: Unknown system variable 'lock_wait_timeout'" 
>> source="exporter.go:146"
>> time="2020-08-24T01:09:51-07:00" level=error msg="Error pinging mysqld: 
>> Error 1193: Unknown system variable 'lock_wait_timeout'" 
>> source="exporter.go:146"
>> time="2020-08-24T01:09:56-07:00" level=error msg="Error pinging mysqld: 
>> Error 1193: Unknown system variable 'lock_wait_timeout'" 
>> source="exporter.go:146"
>> time="2020-08-24T01:10:01-07:00" level=error msg="Error pinging mysqld: 
>> Error 1193: Unknown system variable 'lock_wait_timeout'" 
>> source="exporter.go:146"
>> time="2020-08-24T01:10:06-07:00" level=error msg="Error pinging mysqld: 
>> Error 1193: Unknown system variable 'lock_wait_timeout'" 
>> source="exporter.go:146"
>> time="2020-08-24T01:10:11-07:00" level=error msg="Error pinging mysqld: 
>> Error 1193: Unknown system variable 'lock_wait_timeout'" 
>> source="exporter.go:146
>> On Monday, August 24, 2020 at 9:32:27 AM UTC+2 sup...@gmail.com wrote:
>>
>>> Although it's not officially supported, the latest version should still 
>>> work. You will need to disable some collectors that are incompatible with 
>>> 5.1.
>>>
>>> See: https://github.com/prometheus/mysqld_exporter/pull/318
>>>
>>> On Mon, Aug 24, 2020 at 9:28 AM deln...@gmail.com  
>>> wrote:
>>>
 I'm aware that MySQL 5.1 is not supported. Wondering if someone can 
 help with pointing what version of MySQL_exporter is compatible with 5.1 
 version? 

 -- 
 You received this message because you are 

[prometheus-users] Re: Regexp re-writes on Legend Format -grafana

[goel....@gmail.com]

Thanks a lot Brian this works, 

Can you please suggest me how can i get more info on this type of issue so 
that it can be tackled be me in future.

On Monday, 24 August 2020 at 14:34:49 UTC+5:30 b.ca...@pobox.com wrote:

> My mistake, you also have to match the trailing junk after the last dot.  
> Try:
>
> "name", "$1", "name", "(.*)[.].*"
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/2ede258e-48b1-4a6b-9c76-9b2e1362e711n%40googlegroups.com.

Re: [prometheus-users] prometheus-snmp-generator error (Debian 10)

[Mario Pranjic]

Hi,

A small update: indeed there are errors in provided MIB files by Synology.
I ran some of them into  http://www.muonics.com/Tools/smicheck.php and
reported some examples to Synology.
A case is opened on the Synology side and they will send it to their dev
team. We will see if/when they come up with something useful.

Best regards,

Mario.

lør. 22. aug. 2020 kl. 20:38 skrev Mario Pranjic :

> Hi Ben,
>
> Thank you very much for your time and effort.
> Indeed, I am not an expert in snmp (that was always somehow left on TODO
> list). Maybe this is a good opportunity to fill in the gaps in knowledge. :)
>
> The thing is... With snmp.yml which comes as a result of that
> snmp_exporter role, I do get some communication with Synology from
> Prometheus, but the list is pretty awkward.
> By pointing browser to: http://127.0.0.1:9116/snmp?target=192.168.10.187
>  i get output as noted in list.txt file.
>
> I see no info about disks and raid.
>
> However, if I run snmpget from Prometheus/snmp_exporter node against
> Synology, I can read values documented by Synology.
>
> For example, disk temperature (disk0 and disk1):
> root@monnew:/# snmpget -c public -v 2c -O qv 192.168.10.187
> 1.3.6.1.4.1.6574.2.1.1.6.0
> 29
> root@monnew:/# snmpget -c public -v 2c -O qv 192.168.10.187
> 1.3.6.1.4.1.6574.2.1.1.6.1
> 31
>
> Or some  info:
> root@monnew:/# snmpget -c public -v 2c -O qv 192.168.10.187
> 1.3.6.1.4.1.6574.1.5.1.0
> "DS216j"
>
> I tried to use snmp.yml you provided in example or building one based on
> your example generator.yml, but in both cases
> http://127.0.0.1:9116/snmp?target=192.168.10.187 will return error:
> Unkown_module 'if_mib'
>
> The question why snmp_exporter is returning only a small portion of values
> data from Synology is, I presume, related to the snmp.yml file.
> How to generate the right one is an enigma. :/
>
> Attaching snmp.yml provisioned by snmp_exporter role for the reference.
>
> Best regards,
>
> Mario.
>
>
>
>
> lør. 22. aug. 2020 kl. 09:42 skrev Ben Kochie :
>
>> You will probably need to have a good understanding of SNMP and MIBs in
>> order create new generator configs. Unfortunately, the Synology MIBs have a
>> large number of mistakes, making it even harder to work with.
>>
>> I created a simple example of how to use the generator based on what I
>> found in the SYNOLOGY-RAID-MIB.
>>
>> https://github.com/SuperQ/tools/tree/master/snmp_exporter/Synology
>>
>> I corrected as many of the errors as I could, but Synology didn't format
>> the raidStatus as an ENUM.
>>
>> On Fri, Aug 21, 2020 at 3:46 PM Mario Pranjic 
>> wrote:
>>
>>> Hi Ben,
>>>
>>> Community string was wrong. I fixed that. Now I can tell snmp_exporter
>>> can communicate properly.
>>>
>>> I can get in Prometheus: sysUpTime{instance="192.168.10.187",job="snmp"}
>>> And value is read from the device (incrementing on refresh too).
>>>
>>> But as for mibs, I still can't use mibs as described in the Synology
>>> snmp guide.
>>>
>>> https://global.download.synology.com/download/Document/Software/DeveloperGuide/Firmware/DSM/All/enu/Synology_DiskStation_MIB_Guide.pdf
>>>
>>> There is still a step missing in the snmp_exporter role in regards to
>>> mibs and/or proper snmp.yml generation.
>>>
>>> Best regards,
>>>
>>> Mario.
>>>
>>> fre. 21. aug. 2020 kl. 15:28 skrev Ben Kochie :
>>>
 The SNMP exporter uses UDP for transport, so telnet to 161 isn't going
 to help much. I've seen very few devices implement RFC 3430.

 There are usually two problems people run into.
 * The community string is wrong (defaults to "public")
 * The device has a management IP control and won't allow traffic.

 SNMP implementations generally don't reject incorrect authentication,
 rather they simply drop the packets. So it can be hard to diagnose.

 On Fri, Aug 21, 2020 at 2:52 PM Mario Pranjic 
 wrote:

> Hi Ben,
>
> I ran another VM and used roles:
> -  https://github.com/cloudalchemy/ansible-snmp-exporter
> -  https://github.com/cloudalchemy/ansible-prometheus
>
> along with Grafana. Prometheus, node.exporter and Grafana work as
> intended (as before).
>
> However, when I try to enable snmp_exporter:
>   - job_name: snmp
> metrics_path: /snmp
> params:
>   module:
>   - if_mib
> relabel_configs:
> - source_labels:
>   - __address__
>   target_label: __param_target
> - source_labels:
>   - __param_target
>   target_label: instance
> - replacement: 127.0.0.1:9116
>   target_label: __address__
> static_configs:
> - targets:
>   - 192.168.10.187
>
> I get error in syslog:
> Aug 21 14:43:23 monnew snmp_exporter[363]: level=info
> ts=2020-08-21T12:43:23.189Z caller=collector.go:224 module=if_mib
> target=192.168.10.187 msg="Error scraping target" err="scrape canceled
> (possible timeout) getting target 192.16

[prometheus-users] Re: Regexp re-writes on Legend Format -grafana

[Brian Candler]

My mistake, you also have to match the trailing junk after the last dot.  
Try:

"name", "$1", "name", "(.*)[.].*"

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/037412e4-a766-40e0-b4f3-a765c02bdec6o%40googlegroups.com.

Re: [prometheus-users] mysql 5.1 working config

[Ben Kochie]

Ahh, I see, yes, it looks like that was added in msqld_exporter 0.11, so
you would have to run 0.10 or older.

It looks like lock_wait_timeout

was
introduced in 5.5.

It looks like the way we implemented this is always enabled. I would accept
a PR to make it optional.

On Mon, Aug 24, 2020 at 10:13 AM deln...@gmail.com 
wrote:

> Think I'm already starting exporter with these collectors disabled -
> here's the no-collector arguments section:
> --no-collect.info_schema.innodb_metrics
> --no-collect.info_schema.innodb_cmp --no-collect.info_schema.innodb_cmpmem
>
> nohup.out
> time="2020-08-24T01:08:57-07:00" level=info msg="Starting mysqld_exporter
> (version=0.12.1, branch=HEAD,
> revision=48667bf7c3b438b5e93b259f3d17b70a7c9aff96)"
> source="mysqld_exporter.go:257"
> time="2020-08-24T01:08:57-07:00" level=info msg="Build context
> (go=go1.12.7, user=root0b3e56a7bc0a, date=20190729-12:35:58)"
> source="mysqld_exporter.go:258"
> time="2020-08-24T01:08:57-07:00" level=info msg="Enabled scrapers:"
> source="mysqld_exporter.go:269"
> time="2020-08-24T01:08:57-07:00" level=info msg=" --collect.global_status"
> source="mysqld_exporter.go:273"
> time="2020-08-24T01:08:57-07:00" level=info msg="
> --collect.global_variables" source="mysqld_exporter.go:273"
> time="2020-08-24T01:08:57-07:00" level=info msg=" --collect.slave_status"
> source="mysqld_exporter.go:273"
> time="2020-08-24T01:08:57-07:00" level=info msg="
> --collect.info_schema.query_response_time" source="mysqld_exporter.go:273"
> time="2020-08-24T01:08:57-07:00" level=info msg="Listening on 0.0.0.0:9104"
> source="mysqld_exporter.go:283"
> time="2020-08-24T01:09:01-07:00" level=error msg="Error pinging mysqld:
> Error 1193: Unknown system variable 'lock_wait_timeout'"
> source="exporter.go:146"
> time="2020-08-24T01:09:06-07:00" level=error msg="Error pinging mysqld:
> Error 1193: Unknown system variable 'lock_wait_timeout'"
> source="exporter.go:146"
> time="2020-08-24T01:09:11-07:00" level=error msg="Error pinging mysqld:
> Error 1193: Unknown system variable 'lock_wait_timeout'"
> source="exporter.go:146"
> time="2020-08-24T01:09:16-07:00" level=error msg="Error pinging mysqld:
> Error 1193: Unknown system variable 'lock_wait_timeout'"
> source="exporter.go:146"
> time="2020-08-24T01:09:21-07:00" level=error msg="Error pinging mysqld:
> Error 1193: Unknown system variable 'lock_wait_timeout'"
> source="exporter.go:146"
> time="2020-08-24T01:09:26-07:00" level=error msg="Error pinging mysqld:
> Error 1193: Unknown system variable 'lock_wait_timeout'"
> source="exporter.go:146"
> time="2020-08-24T01:09:31-07:00" level=error msg="Error pinging mysqld:
> Error 1193: Unknown system variable 'lock_wait_timeout'"
> source="exporter.go:146"
> time="2020-08-24T01:09:36-07:00" level=error msg="Error pinging mysqld:
> Error 1193: Unknown system variable 'lock_wait_timeout'"
> source="exporter.go:146"
> time="2020-08-24T01:09:41-07:00" level=error msg="Error pinging mysqld:
> Error 1193: Unknown system variable 'lock_wait_timeout'"
> source="exporter.go:146"
> time="2020-08-24T01:09:46-07:00" level=error msg="Error pinging mysqld:
> Error 1193: Unknown system variable 'lock_wait_timeout'"
> source="exporter.go:146"
> time="2020-08-24T01:09:51-07:00" level=error msg="Error pinging mysqld:
> Error 1193: Unknown system variable 'lock_wait_timeout'"
> source="exporter.go:146"
> time="2020-08-24T01:09:56-07:00" level=error msg="Error pinging mysqld:
> Error 1193: Unknown system variable 'lock_wait_timeout'"
> source="exporter.go:146"
> time="2020-08-24T01:10:01-07:00" level=error msg="Error pinging mysqld:
> Error 1193: Unknown system variable 'lock_wait_timeout'"
> source="exporter.go:146"
> time="2020-08-24T01:10:06-07:00" level=error msg="Error pinging mysqld:
> Error 1193: Unknown system variable 'lock_wait_timeout'"
> source="exporter.go:146"
> time="2020-08-24T01:10:11-07:00" level=error msg="Error pinging mysqld:
> Error 1193: Unknown system variable 'lock_wait_timeout'"
> source="exporter.go:146
> On Monday, August 24, 2020 at 9:32:27 AM UTC+2 sup...@gmail.com wrote:
>
>> Although it's not officially supported, the latest version should still
>> work. You will need to disable some collectors that are incompatible with
>> 5.1.
>>
>> See: https://github.com/prometheus/mysqld_exporter/pull/318
>>
>> On Mon, Aug 24, 2020 at 9:28 AM deln...@gmail.com 
>> wrote:
>>
>>> I'm aware that MySQL 5.1 is not supported. Wondering if someone can help
>>> with pointing what version of MySQL_exporter is compatible with 5.1
>>> version?
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Prometheus Users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to prometheus-use...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/prometheus-users/00a0e62

Re: [prometheus-users] mysql 5.1 working config

[deln...@gmail.com]

Think I'm already starting exporter with these collectors disabled - here's 
the no-collector arguments section:
--no-collect.info_schema.innodb_metrics --no-collect.info_schema.innodb_cmp 
--no-collect.info_schema.innodb_cmpmem

nohup.out
time="2020-08-24T01:08:57-07:00" level=info msg="Starting mysqld_exporter 
(version=0.12.1, branch=HEAD, 
revision=48667bf7c3b438b5e93b259f3d17b70a7c9aff96)" 
source="mysqld_exporter.go:257"
time="2020-08-24T01:08:57-07:00" level=info msg="Build context 
(go=go1.12.7, user=root0b3e56a7bc0a, date=20190729-12:35:58)" 
source="mysqld_exporter.go:258"
time="2020-08-24T01:08:57-07:00" level=info msg="Enabled scrapers:" 
source="mysqld_exporter.go:269"
time="2020-08-24T01:08:57-07:00" level=info msg=" --collect.global_status" 
source="mysqld_exporter.go:273"
time="2020-08-24T01:08:57-07:00" level=info msg=" 
--collect.global_variables" source="mysqld_exporter.go:273"
time="2020-08-24T01:08:57-07:00" level=info msg=" --collect.slave_status" 
source="mysqld_exporter.go:273"
time="2020-08-24T01:08:57-07:00" level=info msg=" 
--collect.info_schema.query_response_time" source="mysqld_exporter.go:273"
time="2020-08-24T01:08:57-07:00" level=info msg="Listening on 0.0.0.0:9104" 
source="mysqld_exporter.go:283"
time="2020-08-24T01:09:01-07:00" level=error msg="Error pinging mysqld: 
Error 1193: Unknown system variable 'lock_wait_timeout'" 
source="exporter.go:146"
time="2020-08-24T01:09:06-07:00" level=error msg="Error pinging mysqld: 
Error 1193: Unknown system variable 'lock_wait_timeout'" 
source="exporter.go:146"
time="2020-08-24T01:09:11-07:00" level=error msg="Error pinging mysqld: 
Error 1193: Unknown system variable 'lock_wait_timeout'" 
source="exporter.go:146"
time="2020-08-24T01:09:16-07:00" level=error msg="Error pinging mysqld: 
Error 1193: Unknown system variable 'lock_wait_timeout'" 
source="exporter.go:146"
time="2020-08-24T01:09:21-07:00" level=error msg="Error pinging mysqld: 
Error 1193: Unknown system variable 'lock_wait_timeout'" 
source="exporter.go:146"
time="2020-08-24T01:09:26-07:00" level=error msg="Error pinging mysqld: 
Error 1193: Unknown system variable 'lock_wait_timeout'" 
source="exporter.go:146"
time="2020-08-24T01:09:31-07:00" level=error msg="Error pinging mysqld: 
Error 1193: Unknown system variable 'lock_wait_timeout'" 
source="exporter.go:146"
time="2020-08-24T01:09:36-07:00" level=error msg="Error pinging mysqld: 
Error 1193: Unknown system variable 'lock_wait_timeout'" 
source="exporter.go:146"
time="2020-08-24T01:09:41-07:00" level=error msg="Error pinging mysqld: 
Error 1193: Unknown system variable 'lock_wait_timeout'" 
source="exporter.go:146"
time="2020-08-24T01:09:46-07:00" level=error msg="Error pinging mysqld: 
Error 1193: Unknown system variable 'lock_wait_timeout'" 
source="exporter.go:146"
time="2020-08-24T01:09:51-07:00" level=error msg="Error pinging mysqld: 
Error 1193: Unknown system variable 'lock_wait_timeout'" 
source="exporter.go:146"
time="2020-08-24T01:09:56-07:00" level=error msg="Error pinging mysqld: 
Error 1193: Unknown system variable 'lock_wait_timeout'" 
source="exporter.go:146"
time="2020-08-24T01:10:01-07:00" level=error msg="Error pinging mysqld: 
Error 1193: Unknown system variable 'lock_wait_timeout'" 
source="exporter.go:146"
time="2020-08-24T01:10:06-07:00" level=error msg="Error pinging mysqld: 
Error 1193: Unknown system variable 'lock_wait_timeout'" 
source="exporter.go:146"
time="2020-08-24T01:10:11-07:00" level=error msg="Error pinging mysqld: 
Error 1193: Unknown system variable 'lock_wait_timeout'" 
source="exporter.go:146
On Monday, August 24, 2020 at 9:32:27 AM UTC+2 sup...@gmail.com wrote:

> Although it's not officially supported, the latest version should still 
> work. You will need to disable some collectors that are incompatible with 
> 5.1.
>
> See: https://github.com/prometheus/mysqld_exporter/pull/318
>
> On Mon, Aug 24, 2020 at 9:28 AM deln...@gmail.com  
> wrote:
>
>> I'm aware that MySQL 5.1 is not supported. Wondering if someone can help 
>> with pointing what version of MySQL_exporter is compatible with 5.1 
>> version? 
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Prometheus Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to prometheus-use...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/prometheus-users/00a0e627-563f-4c4f-a23e-fda3454f3427n%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheu

Re: [prometheus-users] mysql 5.1 working config

[Ben Kochie]

Although it's not officially supported, the latest version should still
work. You will need to disable some collectors that are incompatible with
5.1.

See: https://github.com/prometheus/mysqld_exporter/pull/318

On Mon, Aug 24, 2020 at 9:28 AM deln...@gmail.com 
wrote:

> I'm aware that MySQL 5.1 is not supported. Wondering if someone can help
> with pointing what version of MySQL_exporter is compatible with 5.1
> version?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-users/00a0e627-563f-4c4f-a23e-fda3454f3427n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/CABbyFmp3AZJuOp%2B98FEvyV6zLr1WUqX0KvH7KKPG-cy7ycOFXA%40mail.gmail.com.

[prometheus-users] mysql 5.1 working config

[deln...@gmail.com]

I'm aware that MySQL 5.1 is not supported. Wondering if someone can help 
with pointing what version of MySQL_exporter is compatible with 5.1 version?

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/00a0e627-563f-4c4f-a23e-fda3454f3427n%40googlegroups.com.