Many thanks to all for your contributions.
My conclusion is that the method I use with many bots would be slow.
It seems the best option is to use nginx maps:
https://forum.nginx.org/read.php?11,255678,270417#msg-270417
https://community.centminmod.com/threads/blocking-bad-or-aggressive-bots.643
Hello,
I have a server with several domains, in the configuration file of each
domain I have a line like this to block bad bots.
If ($ http_user_agent ~ *
(zealbot|MJ12bot|AhrefsBot|sogou|PaperLiBot|uipbot|DotBot|GetIntent|Cliqzbot|YandexBot|Nutch|TurnitinBot|IndeedBot)
Return 403;
}
This works
