>
> what kind of load do you have? Puppet? Facter? Is that ENC? Something
> else?
>
> Can you tell which requests are slow from logs or monitoring?
>
>
Yes, I should have mentioned that - there's very little puppet and ENC work
done by this cluster at this point (more is coming soon though). Host
discovery is by far the largest workload - 7600 discovered systems at this
point. The last spike that I saw the impact to overall flows was when
300-400 systems were trying to register at the same time. Because of the
deadlocks, about 200-300 systems could not register repeatedly and had to
keep retrying for a rather long time.
Rather often these registration attempts would end up creating either
duplicate entries with the same "mac<mac>" but different IDs in a DB or an
"empty" discovery host entry. Both of these would prevent a system
successfully register unless they are removed (I had to write a little
script that runs from the cron to do so). Here are the examples of an
"empty" record and duplicate ones (as they get deleted):
{
"id": 437923,
"name": "mac90e2bae6cc70",
"last_compile": null,
"last_report": null,
"updated_at": "2017-08-22T07:08:54.000Z",
"created_at": "2017-08-22T07:08:54.000Z",
"root_pass": "<removed>",
"architecture_id": null,
"operatingsystem_id": null,
"environment_id": null,
"ptable_id": null,
"medium_id": null,
"build": false,
"comment": null,
"disk": null,
"installed_at": null,
"model_id": null,
"hostgroup_id": null,
"owner_id": null,
"owner_type": null,
"enabled": true,
"puppet_ca_proxy_id": null,
"managed": false,
"use_image": null,
"image_file": null,
"uuid": null,
"compute_resource_id": null,
"puppet_proxy_id": null,
"certname": null,
"image_id": null,
"organization_id": null,
"location_id": null,
"otp": null,
"realm_id": null,
"compute_profile_id": null,
"provision_method": null,
"grub_pass": "",
"global_status": 0,
"lookup_value_matcher": null,
"discovery_rule_id": null,
"salt_proxy_id": null,
"salt_environment_id": null,
"pxe_loader": null
}
Duplicates (usually the later duplicate would be an empty one as well, but
not all the time):
{
"id": 430090,
"name": "mac3417ebe3f8f1",
"last_compile": null,
"last_report": "2017-09-14T19:47:55.000Z",
"updated_at": "2017-09-14T19:47:57.000Z",
"created_at": "2017-03-08T20:24:05.000Z",
"root_pass": "<removed>",
"architecture_id": null,
"operatingsystem_id": null,
"environment_id": null,
"ptable_id": null,
"medium_id": null,
"build": false,
"comment": null,
"disk": null,
"installed_at": null,
"model_id": 3,
"hostgroup_id": null,
"owner_id": 10,
"owner_type": "User",
"enabled": true,
"puppet_ca_proxy_id": null,
"managed": false,
"use_image": null,
"image_file": null,
"uuid": null,
"compute_resource_id": null,
"puppet_proxy_id": null,
"certname": null,
"image_id": null,
"organization_id": null,
"location_id": null,
"otp": null,
"realm_id": null,
"compute_profile_id": null,
"provision_method": null,
"grub_pass": "",
"global_status": 0,
"lookup_value_matcher": null,
"discovery_rule_id": null,
"salt_proxy_id": null,
"salt_environment_id": null,
"pxe_loader": null
}
{
"id": 438146,
"name": "mac3417ebe3f8f1",
"last_compile": null,
"last_report": "2017-09-11T08:58:05.000Z",
"updated_at": "2017-09-11T08:58:07.000Z",
"created_at": "2017-08-24T19:44:23.000Z",
"root_pass": "<removed>",
"architecture_id": null,
"operatingsystem_id": null,
"environment_id": null,
"ptable_id": null,
"medium_id": null,
"build": false,
"comment": null,
"disk": null,
"installed_at": null,
"model_id": null,
"hostgroup_id": null,
"owner_id": null,
"owner_type": null,
"enabled": true,
"puppet_ca_proxy_id": null,
"managed": false,
"use_image": null,
"image_file": null,
"uuid": null,
"compute_resource_id": null,
"puppet_proxy_id": null,
"certname": null,
"image_id": null,
"organization_id": null,
"location_id": null,
"otp": null,
"realm_id": null,
"compute_profile_id": null,
"provision_method": null,
"grub_pass": "",
"global_status": 0,
"lookup_value_matcher": null,
"discovery_rule_id": null,
"salt_proxy_id": null,
"salt_environment_id": null,
"pxe_loader": null
}
I can't tell if any queries are slow - can you remind me how to do that?
Thanks!
--
You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to foreman-users+unsubscr...@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.
