If you want a number which represents the number of idle cores, then you
can get it directly without going via percentages:
rate(node_cpu_seconds_total{mode="idle"}[5m]) # gives you the idle
fraction of each core (5 minute average)
sum by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) #
gives you the total number of idle cores, per instance
So for a 64 core system, this should vary between 0 (totally busy) to 64
(totally unused).
Then you need to decide how that relates to memory. Is 1 idle core worth
the same as 1GB of unused RAM? 2GB? Something else? Your choice.
Let's say you want the amount of available memory in 1GB units. Then:
node_memory_MemAvailable_bytes / 1024 / 1024 / 1024
(MemAvailable is probably what you're looking for here, as long as your
kernel supports it <https://github.com/prometheus/node_exporter/issues/1689>.
It's the completely free memory plus buffer cache that could be reclaimed).
For fairness and consistency, maybe you want to average that over 5 minutes
too (or take the min, whatever you prefer).
avg_over_time(node_memory_MemAvailable_bytes[5m]) / 1024 / 1024 / 1024
Then you can add the two together as described before:
sum by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) + on
(instance) (avg_over_time(node_memory_MemAvailable_bytes[5m]) / 1024 / 1024
/ 1024)
I don't know if this is what you want, but maybe it's a starting point and
you can work from here. As before, my advice is to build up the query in
stages, testing each smaller part of the query separately, before combining
the parts.
On Thursday, 29 September 2022 at 09:28:43 UTC+1 Brian Candler wrote:
> I think you first want to describe to yourself, in plain English, how you
> want this "ranking" value to be calculated. Then see if you can apply that
> logic in the form of a PromQL expression.
>
> You haven't said what method you're using to calculate the ranking right
> now. Let's say it's a simple average, where the weights are equal (0.5
> each). This gives:
>
> Instance CPU CPU used Memory Memory used Ranking
> (% used)
> A 32 0.1% 125GB 1.1%
> 0.5*0.1 + 0.5*1.1 = 0.6%
> B 64 0.0% 125GB 3.0%
> 0.5*0.0 + 0.5*3.0 = 1.5%
>
> If you're saying that's not what you want, then you need to be clear in
> your head as to what logic you *do* want to use.
>
> I think that if you want to multiply something by the number cores, you
> may want to work with percentage *idle* rather than percentage *busy*
> (otherwise, 32 cores x 0% used will be the same as 64 cores x 0% used).
> You'll also need to decide the weighting between cores and RAM (i.e. how
> much is 1 idle core worth, compared to 1GB of idle RAM?) But really this
> is up to you to decide.
>
> On Thursday, 29 September 2022 at 08:05:23 UTC+1 chembakay...@gmail.com
> wrote:
>
>> Thanks for your reply.
>>
>> I want to find the least used instances based on CPU and memory usage.
>> But I want to consider the instance which has more cores.
>>
>> For ex:
>>
>> Instance CPU CPU used Memory Memory used
>> A 32 0.1% 125GB 1.1%
>> B 64 0.0% 125GB 3.0%
>>
>> Here while calculating the weighted average of the available percentages
>> of CPU and memory, I am getting instance A as the least used server. ut,
>> instance B has more CPU cores than instance B. I want to give priority to
>> cores first then memory if the above scenario happens. In such case my
>> logic is not working properly.
>>
>> Any help?
>>
>> Thanks & regards,
>> Bharath Kumar.
>> On Thursday, 29 September 2022 at 01:30:07 UTC+5:30 Brian Candler wrote:
>>
>>> Yes, anything is possible. Just go ahead and write the query that does
>>> whatever you want.
>>>
>>> I don't understand what logic you want to calculate this ranking value.
>>> It's up to you to decide on an algorithm, and then implement it in PromQL.
>>> Good luck.
>>>
>>> On Wednesday, 28 September 2022 at 13:52:13 UTC+1 chembakay...@gmail.com
>>> wrote:
>>>
>>>> Thanks for your reply.
>>>>
>>>> But here I want to consider the CPU cores whose value is greater than
>>>> 64 and usage is less and similar to memory whose value is highest and
>>>> usage
>>>> is low.
>>>>
>>>> for example:[image: cpu_latest.PNG]
>>>>
>>>> The above is in the sorting order based on a weighted average of CPU
>>>> and memory percentages.
>>>>
>>>> But the first three instances' CPU cores are 32,24,24 which were less
>>>> than the next three instances. so I want to give priority to 64 cores than
>>>> 24 or 32 cores. So will it be possible to do that using any conditional
>>>> operator or any changes in the query?
>>>>
>>>> Thanks & regards,
>>>> Bharath Kumar.
>>>> On Tuesday, 27 September 2022 at 12:25:41 UTC+5:30 Brian Candler wrote:
>>>>
>>>>> Look at topk and bottomk
>>>>> <https://prometheus.io/docs/prometheus/latest/querying/operators/#aggregation-operators>
>>>>> .
>>>>>
>>>>> *"I thought of combining two queries( Total CPU remaining cores and
>>>>> Total Unused memory) then we will get the value and we will sort it."*
>>>>>
>>>>> That's fine. If you can write a PromQL expression which calculates a
>>>>> combined value for "how much used" which is meaningful to you, then you
>>>>> can
>>>>> rank by that.
>>>>>
>>>>> I think that the *percentage* CPU used and percentage memory used
>>>>> could be meaningfully combined, e.g. by returning the larger of the two,
>>>>> or
>>>>> a weighted average. (Suppose you have one machine with 99% CPU used and
>>>>> 30% RAM, and another with 70% CPU and 70% RAM. Which do you want to
>>>>> appear
>>>>> on the top of the table?)
>>>>>
>>>>> Build up your expression in parts, using the PromQL query browser in
>>>>> the prometheus web interface.
>>>>>
>>>>> On Monday, 26 September 2022 at 16:10:57 UTC+1 chembakay...@gmail.com
>>>>> wrote:
>>>>>
>>>>>> Hello All,
>>>>>>
>>>>>> I have 100 instances where I install node exporter and through
>>>>>> Prometheus, I am pulling those instances metrics. Now I want to find the
>>>>>> least used instances among those based on certain parameters like Total
>>>>>> CPU
>>>>>> cores, CPU used, Total Memory, and Memory Used.
>>>>>>
>>>>>> For example, we will consider some of the values of the instance as
>>>>>> below:
>>>>>>
>>>>>>
>>>>>> [image: cpu.PNG]
>>>>>>
>>>>>> generally, we will high priority on cores and then memory. There are
>>>>>> 64 cores, and 48 cores.. similarly, there are 251 GB, and 125 GB of
>>>>>> memory.
>>>>>>
>>>>>> Considering all these parameters we have to assign ranks in such a
>>>>>> way that these servers are least used.
>>>>>>
>>>>>> I thought of combining two queries( Total CPU remaining cores and
>>>>>> Total Unused memory) then we will get the value and we will sort it. But
>>>>>> I
>>>>>> don't think it is the correct approach.
>>>>>>
>>>>>> Any leads?
>>>>>>
>>>>>> Thanks & regards,
>>>>>> Bharath Kumar.
>>>>>>
>>>>>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/51037fa0-f27e-4855-a0da-06d9e19d664bn%40googlegroups.com.
