Actually this seems to be a thing of the service, not nginx itself. When I stop the service with the old configuration file, manually run /gnu/store/bdhfqs7sx3mal6pzz8z00hw4cpn5dj7x-nginx-1.21.4/sbin/nginx -p /var/run/nginx -c new_configuration_file kill the process, and "herd restart nginx", the herd service uses the old configuration file again.
Andreas