Adding some more details.. During prometheus instaltion, max-block-duration and storage.tsdb.retention.time is not set. 15d default value is picked for RetentionDuration and max-block-duration is internally calulated as 1d12h(10% of RetentionDuration) . Same can be found in logs as well as prometus UI.
caller=main.go:986 level=debug msg="TSDB options" MinBlockDuration=2h MaxBlockDuration=1d12h MaxBytes=0B NoLockfile=false RetentionDuration=15d WALSegmentSize=0B AllowOverlappingBlocks=false WALCompression=true Even though logs/UI says MaxBlockDuration is 1d12h but compaction is happening only upto 18hr. This is not aligning with "Compaction will create larger blocks containing data spanning up to 10% of the retention time, or 31 days, whichever is smaller." Could you please clarify this. On Sunday, 26 June 2022 at 23:37:11 UTC+5:30 Sriharsha B M wrote: > Hi, > As per prometheus documentation, "Compaction will create larger blocks > containing data spanning up to 10% of the retention time, or 31 days, > whichever is smaller." > > We have set retention time as 4 days. 10% of retention time is 9.6h. If we > align to the 2 hour boundary then it will become 8 hr. But prometheus is > creating bigger blocks only upto 6 hour. Could you please let us know how > it is calculated. > > Following is differnt test which we did with different retention time(10m, > 1d, 2d, 3d, 4d, 7d, 15d). Please note retention size is not set. > > fifteen-day-cpro-server-0 > total 60 > drwxrws---. 2 root 65534 16384 Jun 7 06:07 lost+found > -rw-r--r--. 1 65534 65534 0 Jun 7 06:08 lock > drwxr-sr-x. 3 65534 65534 4096 Jun 7 17:00 01G4ZHR5XPZW1SJ7ZSDKEA8GWG > drwxr-sr-x. 3 65534 65534 4096 Jun 8 09:00 01G518P9M546ZMG7PC6XC435KQ > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8RBDRNCFK0ZX389R1WCG > drwxr-sr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 wal > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X914XWP2ZYXQ5JBCQTWR5 > -rw-r--r--. 1 65534 65534 20001 Jun 8 15:06 queries.active > four-day-cpro-server-0 > total 68 > drwxrws---. 2 root 65534 16384 Jun 7 06:07 lost+found > -rw-r--r--. 1 65534 65534 0 Jun 7 06:09 lock > drwxr-sr-x. 3 65534 65534 4096 Jun 7 17:00 01G4ZHR4TKCW31TX7A988YXEH1 > drwxr-sr-x. 3 65534 65534 4096 Jun 7 21:00 01G4ZZFG1S11VXXJENB17B1QZY > drwxr-sr-x. 3 65534 65534 4096 Jun 8 03:00 01G50M2KBZ4Q5QDKFQT5C5APZ9 > drwxr-sr-x. 3 65534 65534 4096 Jun 8 09:00 01G518NVEEQ58VNCY9YA2Q03SE > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8QDYBJA81XWS9TSB6BDQ > drwxr-sr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 wal > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8ZWY7H9AWDE9SVH2CP5N > -rw-r--r--. 1 65534 65534 20001 Jun 8 15:06 queries.active > one-day-cpro-server-0 > total 96 > drwxrws---. 2 root 65534 16384 Jun 7 06:08 lost+found > -rw-r--r--. 1 65534 65534 0 Jun 7 06:09 lock > drwxr-sr-x. 3 65534 65534 4096 Jun 7 15:00 01G4ZAW0WNFYMP5JK9K12XJ5Q3 > drwxr-sr-x. 3 65534 65534 4096 Jun 7 17:00 01G4ZHQR4N49YYM33NMT2NJCM8 > drwxr-sr-x. 3 65534 65534 4096 Jun 7 19:00 01G4ZRKFCMFPP15V6VBV5DNQQC > drwxr-sr-x. 3 65534 65534 4096 Jun 7 21:00 01G4ZZF6MS3JEF8B8K8CRA1G3X > drwxr-sr-x. 3 65534 65534 4096 Jun 7 23:00 01G506AXWP0R72GCFWRQTNHWHV > drwxr-sr-x. 3 65534 65534 4096 Jun 8 01:00 01G50D6N4NMA9VNVZKWB2W8G4Q > drwxr-sr-x. 3 65534 65534 4096 Jun 8 03:00 01G50M2CDG33SSXC5MH04HXF36 > drwxr-sr-x. 3 65534 65534 4096 Jun 8 05:00 01G50TY3MQV7Z5A5T3Z00FWR4M > drwxr-sr-x. 3 65534 65534 4096 Jun 8 07:00 01G511STWNM6QFD7CJ4NDKE3ZZ > drwxr-sr-x. 3 65534 65534 4096 Jun 8 09:00 01G518NJ4NVWF5VYFBX170B74Z > drwxr-sr-x. 3 65534 65534 4096 Jun 8 11:00 01G51FH8Z4KH8P30EQC776WKMY > drwxr-sr-x. 3 65534 65534 4096 Jun 8 13:00 01G51PD0MNAM4YEYAV5HBXC68M > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8QWNW7K0RG5WPEPA6J75 > drwxr-sr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 wal > -rw-r--r--. 1 65534 65534 20001 Jun 8 15:07 queries.active > seven-day-cpro-server-0 > total 68 > drwxrws---. 2 root 65534 16384 Jun 7 06:07 lost+found > -rw-r--r--. 1 65534 65534 0 Jun 7 06:09 lock > drwxr-sr-x. 3 65534 65534 4096 Jun 7 17:00 01G4ZHR3TQTJ3XRZRZQRQF3YNY > drwxr-sr-x. 3 65534 65534 4096 Jun 7 21:00 01G4ZZFH68NEQJK6T2F3F2XC3Y > drwxr-sr-x. 3 65534 65534 4096 Jun 8 03:00 01G50M2KW3JRK2J7H0TZ0WGYGK > drwxr-sr-x. 3 65534 65534 4096 Jun 8 09:00 01G518NWQ2QX42XAYP46H31D92 > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8R6V0FVSNKQ8KTBXSANS > drwxr-sr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 wal > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X9056BT7YJEZBJZMSAEVV > -rw-r--r--. 1 65534 65534 20001 Jun 8 15:07 queries.active > ten-min-cpro-server-0 > total 48 > drwxrws---. 2 root 65534 16384 Jun 7 06:07 lost+found > -rw-rw-r--. 1 65534 65534 0 Jun 7 06:09 lock > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8QJ2E6E92KDB8CX8FNWY > drwxrwsr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head > drwxrwsr-x. 3 65534 65534 4096 Jun 8 15:00 wal > -rw-rw-r--. 1 65534 65534 20001 Jun 8 15:07 queries.active > three-day-cpro-server-0 > total 68 > drwxrws---. 2 root 65534 16384 Jun 7 06:07 lost+found > -rw-r--r--. 1 65534 65534 0 Jun 7 06:09 lock > drwxr-sr-x. 3 65534 65534 4096 Jun 7 17:00 01G4ZHR5PE5E5W0G8Y3KZHGZ59 > drwxr-sr-x. 3 65534 65534 4096 Jun 7 21:00 01G4ZZFJ9K8HBW0S9AXT07PTS0 > drwxr-sr-x. 3 65534 65534 4096 Jun 8 03:00 01G50M2K3PQHG1XFRKZS9HWCST > drwxr-sr-x. 3 65534 65534 4096 Jun 8 09:00 01G518NVSZ5GKFCPYX4VK8ES5S > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8RBJQ68P709CMNKWHTSH > drwxr-sr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 wal > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X90VJ543W770RJQM7D81W > -rw-r--r--. 1 65534 65534 20001 Jun 8 15:07 queries.active > two-day-cpro-server-0 > total 108 > drwxrws---. 2 root 65534 16384 Jun 7 06:08 lost+found > -rw-r--r--. 1 65534 65534 0 Jun 7 06:09 lock > drwxr-sr-x. 3 65534 65534 4096 Jun 7 09:09 01G4YPT28KE1ERW4K43AXWGYWM > drwxr-sr-x. 3 65534 65534 4096 Jun 7 11:00 01G4YX4HY0JZC3RFAHEQYNNQ31 > drwxr-sr-x. 3 65534 65534 4096 Jun 7 13:00 01G4Z40961X9HKT4WMHSSSXBN4 > drwxr-sr-x. 3 65534 65534 4096 Jun 7 15:00 01G4ZAW0E0YVYQJEHNNC7JZTR4 > drwxr-sr-x. 3 65534 65534 4096 Jun 7 17:00 01G4ZHQQP0W8DAZA3JRXAFPAQQ > drwxr-sr-x. 3 65534 65534 4096 Jun 7 19:00 01G4ZRKEY00KXCMX9X7SG7F5VR > drwxr-sr-x. 3 65534 65534 4096 Jun 7 21:00 01G4ZZF661NZV15S1DFKAFMMXY > drwxr-sr-x. 3 65534 65534 4096 Jun 7 23:00 01G506AXE1H9WXG1E4F6ARF6B2 > drwxr-sr-x. 3 65534 65534 4096 Jun 8 01:00 01G50D6MP0FRJ8BEHMSB0SQEN0 > drwxr-sr-x. 3 65534 65534 4096 Jun 8 03:00 01G50M2BY0N3MB37W5MR1NPWPT > drwxr-sr-x. 3 65534 65534 4096 Jun 8 05:00 01G50TY361R94D1SSD9GGWCNMP > drwxr-sr-x. 3 65534 65534 4096 Jun 8 07:00 01G511STE11YHBY842MG04T2T5 > drwxr-sr-x. 3 65534 65534 4096 Jun 8 09:00 01G518NHP1M4GM2J340CYM97F9 > drwxr-sr-x. 3 65534 65534 4096 Jun 8 11:00 01G51FH8Y0JTPDC3F0Q86JFWST > drwxr-sr-x. 3 65534 65534 4096 Jun 8 13:00 01G51PD060XKRD1GY3G1W149TJ > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8QE1HTTVCDNJVEH4JYZC > drwxr-sr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 wal > -rw-r--r--. 1 65534 65534 20001 Jun 8 15:07 queries.active > > > > > > ReplyForward > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/eb5860cb-2175-461b-b686-108af952438dn%40googlegroups.com.
