Bug#770706: [PKG-Openstack-devel] Bug#770706: Bug#770706: Bug#770706: Bug#770706: keystone.service does not start, /var/run/keystone not created
On 12/05/2014 03:04 AM, Gaudenz Steinlin wrote: OK I tested keystone on unstable now and this works as it should. It's definitely an improvement over the current unit file which does not work at all. I did not go down the route to do a full openstack installation but I also installed nova and the service units there work too. I did not go all the way to have a working setup, but the problems I have seem to be unrelated to the systemd unit files. One thing I noticed is that all services are disabled by default even though they were configured through debconf and should be ready to use. Is this intentional as an off by default behavior or is this a bug. How is this handled with sysvinit? Gaudenz Hi, I have found out what happened. Indeed, this issue is that we're not calling dh_systemd_enable. Or rather, openstack-pkg-tools is generating the *.service files in the override_dh_installinit target, which happens after the dh_systemd_enable in the standard dh sequence. So I wrote this: # Generate the systemd unit file # Note: because dh_systemd_enable is called by the # dh sequencer *before* dh_installinit, we have # to process it manually. for i in `ls debian/*.init.in` ; do \ pkgos-gen-systemd-unit $$i ; \ MYSERVICE=`echo $$i | sed 's/debian\///'` ; \ MYSERVICE=`echo $$MYSERVICE | sed 's/.init.in/.service/'` ; \ dh_systemd_enable $$MYSERVICE ; \ done in the override_dh_installinit. This works, then, except for keystone where there's no #DEBHELPER# in the postinst (because I wanted to keep the control of when invoke-rc.d is called, ie after the daemon starts, the admin tenant is created, and the keystone endpoint is registered). This means that for Keystone, we need to manually add what dh_systemd_enable does. Then I've been asking myself what would happen in Wheezy, and it's looking like the only thing that will happen is that it's going to add a dependency on init-system-helpers, which IMO is quite fine, because it's already available from the official wheezy-backports. Since the release team already expressed their opinion that fixing systemd issues the way I proposed was fine, I hope to make it happen before Jessie is released, with what's currently in the Git for openstack-pkg-tools (version 20). I would welcome some reviews. Thoughts anyone? Thomas Goirand (zigo) -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#770706: [PKG-Openstack-devel] Bug#770706: Bug#770706: Bug#770706: Bug#770706: keystone.service does not start, /var/run/keystone not created
On 12/06/2014 12:14 AM, Gaudenz Steinlin wrote: The init system (every init system, not just systemd) would then start the nova-compute service with this command: nova-compute --config /etc/openstack/openstack.conf --config /etc/nova/nova.conf --config /etc/nova/nova-compute.conf After a long time thinking about it, I think that's a very cool way to do things, and that indeed, we should implement that rather than the /etc/default thing. Let's work on this after the freeze, indeed! Cheers, Thomas -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#770706: [PKG-Openstack-devel] Bug#770706: Bug#770706: Bug#770706: Bug#770706: keystone.service does not start, /var/run/keystone not created
On 12/05/2014 03:04 AM, Gaudenz Steinlin wrote: OK I tested keystone on unstable now and this works as it should. It's definitely an improvement over the current unit file which does not work at all. Cool, thanks for your time! One thing I noticed is that all services are disabled by default even though they were configured through debconf and should be ready to use. Is this intentional as an off by default behavior or is this a bug. How is this handled with sysvinit? Gaudenz This is a bug. With sysinit, it starts by default. What did you have to do under systemd to make it start? Thomas -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#770706: [PKG-Openstack-devel] Bug#770706: Bug#770706: Bug#770706: Bug#770706: keystone.service does not start, /var/run/keystone not created
Hi,
On 12/06/2014 03:14 AM, Gaudenz Steinlin wrote:
I'm not sure if I understand your concerns.
I think it is the difference between
* having one distribution-managed configuration file per daemon (like
in /usr/share/project/daemon.conf) containing the default log
file, meaning 58 configuration files (see the list at the end)
* versus having it computed in the init-script instead
(LOGFILE=/var/log/${PROJECT_NAME}/${NAME}.log)
I'm more inclined to have it explicitly put anyway because 58 files is
not that much, but as a developper too I can understand the problem that
it is a defactorised form. Maybe the best thing to do would be to allow
some templating in OpenStack configurations themselves.
Also note that using a small shell wrapper could be an idea too, as
systemd doesn't go against using shell scripts to launch service: it
simply enables us to avoid running (maybe 1ms?). The shell wrapper could
be generic and take 2 arguments : the project name and the daemon name.
This would allow us to avoid init-script when sysv init is not in use,
and using only one script means the system doesn't have to load many
files (it will be cached when the first openstack daemon is started).
List of configuration files required to manage the one logfile per
daemon logic, exclude /usr/share prefix:
ceilometer/ceilometer-agent-central.conf
ceilometer/ceilometer-agent-compute.conf
ceilometer/ceilometer-agent-notification.conf
ceilometer/ceilometer-alarm-evaluator.conf
ceilometer/ceilometer-alarm-notifier.conf
ceilometer/ceilometer-api.conf
ceilometer/ceilometer-collector.conf
cinder/cinder-api.conf
cinder/cinder-backup.conf
cinder/cinder-scheduler.conf
cinder/cinder-volume.conf
designate/designate-agent.conf
designate/designate-api.conf
designate/designate-central.conf
designate/designate-sink.conf
fuel-astute/astute.conf
fuel-nailgun/nailgun-api.conf
fuel-nailgun/nailgun-assassin.conf
fuel-nailgun/nailgun-reciever.conf
glance/glance-api.conf
glance/glance-registry.conf
heat/heat-api-cfn.conf
heat/heat-api-cloudwatch.conf
heat/heat-api.conf
heat/heat-engine.conf
ironic/ironic-api.conf
ironic/ironic-conductor.conf
keystone/keystone.conf*
murano-agent/murano-agent.conf
murano/murano-api.conf
murano/murano-engine.conf
neutron/neutron-dhcp-agent.conf
neutron/neutron-l3-agent.conf
neutron/neutron-lbaas-agent.conf
neutron/neutron-metadata-agent.conf
neutron/neutron-metering-agent.conf
neutron/neutron-plugin-linuxbridge-agent.conf
neutron/neutron-plugin-openvswitch-agent.conf
neutron/neutron-server.conf
neutron/neutron-vpn-agent.conf
nova/nova-api.conf
nova/nova-baremetal.conf
nova/nova-cells.conf
nova/nova-cert.conf
nova/nova-compute.conf
nova/nova-conductor.conf
nova/nova-consoleauth.conf
nova/nova-console.conf
nova/nova-consoleproxy.nova-novncproxy.conf
nova/nova-consoleproxy.nova-serialproxy.conf
nova/nova-consoleproxy.nova-spicehtml5proxy.conf
nova/nova-consoleproxy.nova-xenvncproxy.conf
nova/nova-network.conf
nova/nova-scheduler.conf
sahara/sahara.conf
tuskar/tuskar-api.conf
tuskar/tuskar-manager.conf
zuul/zuul-server.conf
Bug#770706: [PKG-Openstack-devel] Bug#770706: Bug#770706: Bug#770706: Bug#770706: keystone.service does not start, /var/run/keystone not created
On 12/05/2014 01:01 AM, Thomas Goirand wrote: Then later we can test that... though the build is for Wheezy, so I'm not sure if that's so relevant. Is systemd working well in Wheezy? I've been using systemd in wheezy for over a year now. It's working well, especially if you use wheezy-backports to get systemd 200 instead of 44 that ships with plain wheezy. So, I think a test with this version is very close to the state we will have in jessie. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#770706: [PKG-Openstack-devel] Bug#770706: Bug#770706: Bug#770706: Bug#770706: keystone.service does not start, /var/run/keystone not created
On 12/05/2014 06:04 AM, Gaudenz Steinlin wrote: One thing I noticed is that all services are disabled by default even though they were configured through debconf and should be ready to use. Is this intentional as an off by default behavior or is this a bug. How is this handled with sysvinit? For what I understood, no, they shouldn't be disabled. I will definitely have to test on a clean install. Hopefully, the week-end is close. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#770706: [PKG-Openstack-devel] Bug#770706: Bug#770706: Bug#770706: Bug#770706: keystone.service does not start, /var/run/keystone not created
Hi Gaudenz,
thank you for your comments. As I tried to explain first, I wanted to
make the smallest possible step that could possibly work because, from
what I understood, once we get in the freeze phase, users can take the
current state as a feature (ie, on the python-django-pyscss package,
Thomas had to selectively choose commits from the upstream's fix-only
branch).
On 12/01/2014 01:48 AM, Gaudenz Steinlin wrote:
I still don't like the idea of using the sysv init script here. I'd
rather simplify the init script to the point where this is no longer
needed
I agree, and I tried to get as close to this as possible. However, the
following feature really doesn't in the systemd's spirit IMHO:
[ x$USE_SYSLOG = xyes ] DAEMON_ARGS=$DAEMON_ARGS --use-syslog
[ x$USE_LOGFILE != xno ] DAEMON_ARGS=$DAEMON_ARGS
--log-file=$LOGFILE
I think a next step will be to get rid of this as it duplicates the
configuration files anyway.
and we can just have the following in our unit file:
EnvironmentFile=/etc/default/openstack /etc/default/${PROJECT_NAME}
And then use the proper variables in the ExecStart setting.
In the same way, we should get rid of these environment files. See the
note here
https://wiki.debian.org/systemd/Packaging#overriding_options_and_.2Fetc.2Fdefault_handling:
Note that it is preferable to configure options using native systemd
mechanisms for new packages. I definitely agree with this statement but
even if I wouldn't, it the Debian project's recommendation.
1. don't auto-create the /var/*/${PROJECT_NAME} folder if not root, as
it will fail anyway.
Most of these should not be created by the init script anyway. Creating
/var/lib/X and /var/log/X there seems like a bug to me. These should be
part of the package. That's how it's done in all other packages I know.
I agree but I think we can't remove this feature in the freeze phase. In
fact, I think that a real systemd approach wouldn't use a specific log
dir anyway, and just let the software use stdout. You then have your log
managed by journald, with all the wonders like consistent timestamping
and filtering by many criterions. Since OpenStack has a microservice
approach, the log would be naturally filtered: I'm not aware of any
requirement for, for instance, an access vs error log like with Apache
HTTPd or the same kind log multiplexing (I can think of ISC bind too).
That's an improvement over the current state. But how do you ensure that
the daemon is started in the foreground?
I tried to get as close as possible to a unit file consisting of this:
[Service]
User=${SYSTEM_USER}
Group=${SYSTEM_GROUP}
WorkingDirectory=/var/lib/${PROJECT_NAME}
ExecStart=${DAEMON} --config-file=/etc/${PROJECT_NAME}/${PROJECT_NAME}.conf
Restart=on-failure
while retaining still running the daemon with the options users may
expect from the sysv init script. Since start-stop-daemon is used with
the --background option and uses a PID file, I know the program runs
in foreground as otherwise stopping would fail, kill a non-existant PID.
So, the do_systemd_start simply does this:
do_systemd_start() {
exec $DAEMON $DAEMON_ARGS
}
which replaces the shell process with the daemon one. In the context of
systemd with a service type of simple, it is strictly equivalent to
start-stop-daemon [...] --background [...] --startas $DAEMON $DAEMON_ARGS
There's another improvement to do: at least keystone supports systemd's
notification system. Which means we could use a Type=notify if we did
adapt the default configuration file to include an onready
configuration:
http://docs.openstack.org/icehouse/config-reference/content/section_keystone.conf.html
|# onready allows you to send a notification when the process|
|# is ready to serve For example, to have it notify using|
|# systemd, one could set shell command: onready = systemd-|
|# notify --ready or a module with notify() method: onready =|
|# keystone.common.systemd.|
There are also the security improvements you suggested (very interesting
link by the way).
The please remove it from the init script alltogether. It's also not
needed when under sysv then.
/var/run is required for the classic sysv system. /var/lib could be
removed from here but it wouldn't be strictly addressing the current
bug, as it is only an issue linked to running the sysv init script with
the service user instead of root.
3. /var/lock/${PROJECT_NAME}, AFAIK, is not needed when using systemd;
What was it used for? Seems strange that there is a difference between
sysv and systemd here.
I think this is only needed by start-stop-daemon to avoid launching
multiple instances, but I may be wrong.
4. /var/log/${PROJECT_NAME} is the only one that is both required and
can be considered volatile.
No this should definitely not be considered volatile. It's ok to delete
the logfiles inside, but no one can expect the daemon to still work if
he just removes this directory completely. For all other
Bug#770706: [PKG-Openstack-devel] Bug#770706: Bug#770706: Bug#770706: Bug#770706: keystone.service does not start, /var/run/keystone not created
On 11/27/2014 10:07 AM, Mikaël Cluseau wrote: The quick-fix of tmpfiles.d is what I did first and will (probably) work. But, if I assemble this On 11/27/2014 11:10 AM, Thomas Goirand wrote: Do we agree that with systemd the pid-file logic becomes useless Unfortunately, no, I don't agree. Not with the (poor) way we've designed things in openstack-pkg-tools, which still uses start-stop-daemon. and this On 11/27/2014 11:14 AM, Thomas Goirand wrote: Best would be if we could just get rid of the sysv-rc wrapper, and just go the systemd way fully. I think we can agree that *if* we implement the systemd fully *then* pid-files become useless. As you say it would be the best way, I would like to (try to) spend a significant time to implement the systemd way this week-end. Just... I don't want to it just for the trash, so I'd like to know if you'd agree with this and if the release team will (probably) accept it, or not. I would be very happy to accept such a patch. However, I am not the release team, and I can't speak for them. Like Gaudenz, I think it'd be nice to work on it anyway, and best would be to ask the release team their opinion *now* (the sooner you ask the better). Cheers, Thomas Goirand (zigo) -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#770706: [PKG-Openstack-devel] Bug#770706: Bug#770706: Bug#770706: Bug#770706: keystone.service does not start, /var/run/keystone not created
On 11/24/2014 04:48 PM, Gaudenz Steinlin wrote: Thomas Goirand z...@debian.org writes: On 11/24/2014 01:52 AM, Gaudenz Steinlin wrote: Why do you set PIDFile at all? IMO pidfiles are unnecessary when the daemons managed by systemd run in the foreground and thus there is no need for a pidfile at all. The point is, the way things are designed currently, we don't use foreground at all right now, we continue to use start-stop-daemon and execute the init.d startup script, even when using systemd. This is a bad design that would need improvement, and I would welcome a better implementation within openstack-pkg-tools. I would very much like to see some patches to use foreground (the way it's done in Fedora?), but it should support all properties and implementation of what's done in the init scripts. I see. Wouldn't it be better to completely rely on the sysvinit compatibility of systemd then and not ship any unit files at all? What's the advantage of having systemd units that only wrap around sysv init scripts? Probably. It may also be what the release team will accept as a fix, rather than going through iteration of a poor design. Best would be if we could just get rid of the sysv-rc wrapper, and just go the systemd way fully. I'm not a systemd user or specialist myself, I just went the way Gustavo showed, and just pushed it a little further. I'm now not very happy about this half-backed result... :( On 11/24/2014 04:54 AM, Mikaël Cluseau wrote: Hi, isn't it a duplicate of https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=767711 ? Yes it is. And that's a general issue with all OpenStack packages in Sid/Jessie right now. We need to fix all of this before the 5th of December deadline. I'll try to find some time to have a look, but can't promise anything. Gaudenz That'd be great, thanks! Cheers, Thomas -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
