I did some others tests, on others operating systems (not sure it is
relevant because of Docker but without any clues it is better than
nothing). For anyone to test it, I also changed and reduced my
docker-compose file to:
version: '3.4'
services:
prometheus:
image: prom/prometheus:latest
container_name: "prometheus"
depends_on:
- cadvisor
command:
- '--config.file=/etc/prometheus/prometheus.yml'
- '--storage.tsdb.retention.time=60s'
volumes:
- /tmp/prometheus:/etc/prometheus
ports:
- "9090:9090"
network_mode: "host"
cadvisor:
image: gcr.io/google-containers/cadvisor
container_name: cadvisor
volumes:
- /:/rootfs:ro
- /var/run:/var/run:ro
- /sys:/sys:ro
- /var/lib/docker/:/var/lib/docker:ro
- /dev/disk/:/dev/disk:ro
ports:
- "8080:8080"
network_mode: "host"
And here is my prometheus.yml
scrape_configs:
- job_name: cadvisor
scrape_interval: 5s
static_configs:
- targets:
- localhost:8080
But, the result is still the same with the following curl request:
curl
'http://172.21.0.3:9090/api/v1/query_range?query=go_memstats_alloc_bytes&start=2020-03-26T00:01:00.000Z&end=2020-03-26T17:00:00.000Z&step=15s'
I can still get data that should have been deleted due to out of date of
the timestamp and defined in the storage.tsdb.retention.time parameter.
According to me, it is a bug. But before opening an issue on Prometehus, I
prefer to ask for advices about it.
Thanks,
Best regards.
Le mercredi 25 mars 2020 10:27:22 UTC+1, REMI DRUILHE a écrit :
>
> Anyone to help me on this subject?
>
> Thanks.
>
> Le mardi 24 mars 2020 13:57:05 UTC+1, REMI DRUILHE a écrit :
>>
>> Hello,
>>
>> I am using a Docker version of Prometheus (the latest one) in which I am
>> setting the value for the retention time (option
>> *storage.tsdb.retention.time*) at launch. I trying first with small
>> values like *1m* and *60s* (for 1 minute) or *5m* (for 5 minutes) but I
>> was not able to see any deletion of the data after this period. Thus, I
>> tested again this morning with an higher value: *2h* (for 2 hours). I
>> have seen this information here
>> <https://medium.com/@valyala/prometheus-storage-technical-terms-for-humans-4ab4de6c3d48>
>>
>> (in the retention section) that 2h is the minimum value for Prometheus, not
>> sure if it is true or not. But even with this value, the data stored 4
>> hours ago can still be retrieved using the HTTP API. Here is my query: *curl
>> 'http://172.18.0.3:9090/api/v1/query_range?query=go_memstats_alloc_bytes&start=2020-03-24T00:01:00.000Z&end=2020-03-24T17:00:00.000Z&step=15s
>>
>> <http://172.18.0.3:9090/api/v1/query_range?query=go_memstats_alloc_bytes&start=2020-03-24T00:01:00.000Z&end=2020-03-24T17:00:00.000Z&step=15s>'*
>> .
>>
>> On the Prometheus GUI, the flag for this option is correctly setup.
>>
>> Here is the Docker compose file that is used to launch Prometheus:
>>
>> prometheus:
>> command: '--config.file=/etc/prometheus.yml
>> --storage.tsdb.retention.time=2h'
>> container_name: remi_prometheus
>> depends_on:
>> - cadvisor
>> expose:
>> - '9090'
>> image: prom/prometheus:latest
>> labels:
>> project.run.user: remi
>> networks:
>> project-bridge:
>> aliases:
>> - prometheus
>> ports:
>> - published: 9090
>> target: 9090
>> volumes:
>> - remi-prometheus:/prometheus:rw
>> - /home/remi/Workspace/project/runtime/configuration/prometheus.yml:
>> /etc/prometheus.yml:rw
>>
>> Note that I am using cAdvisor to populate Prometheus in which I get some
>> Golang metrics.
>>
>> Thus here are my questions:
>>
>> - What am I doing wrong?
>> - Was is the minimum value for storage.tsdb.retention.time?
>> - Is there another option that overwrite the value set in
>> storage.tsdb.retention.time that I am not aware of?
>> - Is there a way to test that the option is working if the minimal
>> value is 1 day? I guess it would be to change the system date, but not
>> sure
>> about it.
>>
>>
>> Thanks for the help,
>>
>> Best regards and good luck if you are under confinement because of this
>> damn virus :)
>>
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/c3550107-62af-4120-b2d1-13926b3fff14%40googlegroups.com.
