Hi Brain,
This is the query that I have used.
sum(scrape_samples_scraped)without(app,app_kubernetes_io_managed_by,clusterName,release,environment,instance,job,k8s_cluster,kubernetes_name,kubernetes_namespace,ou,app_kubernetes_io_component,app_kubernetes_io_name,app_kubernetes_io_version,kustomize_toolkit_fluxcd_io_name,kustomize_toolkit_fluxcd_io_namespace,application,name,role,app_kubernetes_io_instance,app_kubernetes_io_part_of,control_plane,beta_kubernetes_io_arch,beta_kubernetes_io_instance_type,
beta_kubernetes_io_os, failure_domain_beta_kubernetes_io_region,
failure_domain_beta_kubernetes_io_zone,kubernetes_io_arch,
kubernetes_io_hostname, kubernetes_io_os, node_kubernetes_io_instance_type,
nodegroup, topology_kubernetes_io_region,
topology_kubernetes_io_zone,chart,heritage,revised,transit,component,namespace,
pod_name, pod_template_hash, security_istio_io_tlsMode,
service_istio_io_canonical_name,
service_istio_io_canonical_revision,k8s_app,kubernetes_io_cluster_service,kubernetes_io_name,route_reflector)
Which simply excluded every label but still I am getting a result like this
{} 7525871918
It shouldn't return any results right?
Prometheus version: 2.36.2
By increased traffic I meant that, the prometheus servers are getting high
traffic from a specific point of time. Currently prometheus is getting 13
million packets earlier it was like 2 to 3 M packets on an average. And the
prometheus endpoint is not public.
On Thursday, July 27, 2023 at 6:06:10 PM UTC+5:30 Brian Candler wrote:
> scrape_samples_scraped always has the labels which prometheus itself adds
> (i.e. job and instance).
>
> Extraordinary claims require extraordinary evidence. Are you saying that
> the PromQL query *scrape_samples_scraped{job="",instance=""}* returns a
> result? If so, what's the number? What do you mean by "with increased
> size" - increased as compared to what? And what version of prometheus are
> you running?
>
> In any case, what you see with scrape_samples_scraped may be completely
> unrelated to the "high traffic" issue. Is your prometheus server exposed
> to the Internet? Maybe someone is accessing it remotely. Even if not, you
> can use packet capture to work out where the traffic is going to and from.
> A tool like https://www.sniffnet.net/ may be helpful.
>
> On Thursday, 27 July 2023 at 13:14:25 UTC+1 Uvais Ibrahim wrote:
>
>> Hi,
>>
>> Since last night, my Prometheus EC2 servers are getting high traffic
>> unusually. When I was checking in Prometheus I can see this
>> metric scrape_samples_scraped with with increased size but without any
>> labels. What could be the reason?
>>
>>
>> Thanks,
>> Uvais Ibrahim
>>
>>
>>
>>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/c6f46be3-6aca-42b9-9164-7eca6b598dddn%40googlegroups.com.
