Re: unbound and systemd

2015-10-21 Thread Tomas Hozza via Unbound-users 
On 18.10.2015 16:34, Sami Kerola via Unbound-users wrote:
> On 17 October 2015 at 01:55, Paul Wouters  wrote:
> > On Fri, 16 Oct 2015, W.C.A. Wijngaards via Unbound-users wrote:
> >
> >> Your patch seems to want to customize the compile itself, and
> >> always-active, install for systemd.  This is because the Makefile is
> >> also trying to do the package install.  Most package systems can pick
> >> up files and install them, especially system files like init.d/rc.d.
> >> It would be inappropriate (I think) for Unbound's Makefile to install
> >> these files, and even more so, install them when systemd is not
> >> present.  So, I would leave those files in contrib, the Makefile then
> >> does not need any changes (I think).  A packager or package-script can
> >> then pick then up and install them.  Wouldn't that be cleaner and more
> >> portable?
> >
> >
> > But that would cause a "make install" to leave something on the
> > filesystem that is not ready to be enabled as a service or started on
> > boot. So make install does need to make some decision on what init
> > system support files to install.
> >
> > For libreswan, we detect the init system, and we provide an override.
> > Our detection script supports systemd, sysvinit, upstart and openrc. See:
> >
> > https://github.com/libreswan/libreswan/blob/master/packaging/utils/lswan_detect.sh
> >
> > But we might be a little more linux-centric then you are.
> >
> > We allow an override running make with INITSYSTEM=sysvinit which is used
> > to install sysvinit support on RHEL6 (which technically has upstart but
> > no one uses it).
>
> Few updates to my changes.
>
> 1. pkg-config is now used the usual way. If this turns out to be bad
> idea after all there is 'systemd-no-pkg-config' branch in my remote
> git that has the previous style m4 stuff.
>
> 2. If socket activation does not work user will get an error followed
> by normal socket open.
>
> 3. New configuration option 'use-systemd' is added, and it does what
> one would expect.
>
> 4. Appropriate error message is added when systemd is enabled, but not 
> running.
>
> 5. The systemd is not consulted for outside_network sockets.
>
> 6. The service and socket contrib files are still installed. I think
> this is right thing to do, but as Paul said it might be best to have
> discussion and align code accordingly. BTW if the service and socket
> files get to be installed then I think the configuration example ought
> to be made more intelligent to contain use-systemd and do-daemonize
> options with environment specific values.
>
> 
> The following changes since commit 7430760ce6401834cd0414165eb13344c6c6f47e:
>   1.5.6rc1 release tag has been created (2015-10-15 11:44:24 +)
> are available in the git repository at:
>   git://github.com/kerolasa/unbound.git systemd
> for you to fetch changes up to 1b2ddd76c803d1581c6f87cdc4ae6911d1270c1b:
>   systemd: use-systemd config bison and yacc changes (2015-10-18 15:11:45 
> +0100)
> 
> Sami Kerola (13):
>   build-sys: run autoreconf -f version 1.5
>   build-sys: add some autotools output files to .gitignore
>   build-sys: add systemd support to autotools
>   build-sys: rerun autoreconf -f && libtoolize -f
>   build-sys: add ./configure output files to .gitignore
>   build-sys: update .gitignore after build
>   systemd: make systemd activation work for udp sockets
>   systemd: make systemd activation work for tcp sockets
>   systemd: make systemd activation work for unix sockets
>   systemd: add service and socket files
>   systemd: add state notifications
>   systemd: add use-systemd configuration option
>   systemd: use-systemd config bison and yacc changes
>

Hi.

I went slightly through your service file. I wanted to point out
that in Fedora we have systemd service files for years and it works
for us nicely. There were some real world issues and the current
state of unit files is outcome of the feedback we had from users.
Unfortunately when seeing your email I realized we never contributed
them back, like we did for dnssec-trigger.

You can check the unit files in master branch of Fedora git:
http://pkgs.fedoraproject.org/cgit/unbound.git/tree/

Couple of things we have and you miss (from what I looked on github)
are:

- timer unit for unbound-anchor to update the trust anchor accordingly
  to RFC 5011
- one shot unit for generating keys used by unbound-control
- for reloading the daemon we use 'unbound-control reload'

I have one thing I would like to confirm.

When the systemd feature will be enabled, will unbound work also without
socket activation by systemd? IOW directly started by unbound.service.
>From what I checked if the 'use-systemd' is set to 'no' it should work
just fine. The socket activation from systemd will not be used, but the
notifications to

Re: unbound and systemd

2015-10-18 Thread Sami Kerola via Unbound-users 
On 17 October 2015 at 01:55, Paul Wouters  wrote:
> On Fri, 16 Oct 2015, W.C.A. Wijngaards via Unbound-users wrote:
>
>> Your patch seems to want to customize the compile itself, and
>> always-active, install for systemd.  This is because the Makefile is
>> also trying to do the package install.  Most package systems can pick
>> up files and install them, especially system files like init.d/rc.d.
>> It would be inappropriate (I think) for Unbound's Makefile to install
>> these files, and even more so, install them when systemd is not
>> present.  So, I would leave those files in contrib, the Makefile then
>> does not need any changes (I think).  A packager or package-script can
>> then pick then up and install them.  Wouldn't that be cleaner and more
>> portable?
>
>
> But that would cause a "make install" to leave something on the
> filesystem that is not ready to be enabled as a service or started on
> boot. So make install does need to make some decision on what init
> system support files to install.
>
> For libreswan, we detect the init system, and we provide an override.
> Our detection script supports systemd, sysvinit, upstart and openrc. See:
>
> https://github.com/libreswan/libreswan/blob/master/packaging/utils/lswan_detect.sh
>
> But we might be a little more linux-centric then you are.
>
> We allow an override running make with INITSYSTEM=sysvinit which is used
> to install sysvinit support on RHEL6 (which technically has upstart but
> no one uses it).

Few updates to my changes.

1. pkg-config is now used the usual way. If this turns out to be bad
idea after all there is 'systemd-no-pkg-config' branch in my remote
git that has the previous style m4 stuff.

2. If socket activation does not work user will get an error followed
by normal socket open.

3. New configuration option 'use-systemd' is added, and it does what
one would expect.

4. Appropriate error message is added when systemd is enabled, but not running.

5. The systemd is not consulted for outside_network sockets.

6. The service and socket contrib files are still installed. I think
this is right thing to do, but as Paul said it might be best to have
discussion and align code accordingly. BTW if the service and socket
files get to be installed then I think the configuration example ought
to be made more intelligent to contain use-systemd and do-daemonize
options with environment specific values.


The following changes since commit 7430760ce6401834cd0414165eb13344c6c6f47e:
  1.5.6rc1 release tag has been created (2015-10-15 11:44:24 +)
are available in the git repository at:
  git://github.com/kerolasa/unbound.git systemd
for you to fetch changes up to 1b2ddd76c803d1581c6f87cdc4ae6911d1270c1b:
  systemd: use-systemd config bison and yacc changes (2015-10-18 15:11:45 +0100)

Sami Kerola (13):
  build-sys: run autoreconf -f version 1.5
  build-sys: add some autotools output files to .gitignore
  build-sys: add systemd support to autotools
  build-sys: rerun autoreconf -f && libtoolize -f
  build-sys: add ./configure output files to .gitignore
  build-sys: update .gitignore after build
  systemd: make systemd activation work for udp sockets
  systemd: make systemd activation work for tcp sockets
  systemd: make systemd activation work for unix sockets
  systemd: add service and socket files
  systemd: add state notifications
  systemd: add use-systemd configuration option
  systemd: use-systemd config bison and yacc changes

-- 
Sami Kerola
http://www.iki.fi/kerolasa/

Re: unbound and systemd

2015-10-16 Thread W.C.A. Wijngaards via Unbound-users 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Sami,

On 10/14/2015 09:32 PM, Sami Kerola wrote:
> On 14 October 2015 at 08:12, W.C.A. Wijngaards
>  wrote:
> 
> Hi Wouter, et.al.,
> 
>> The patch looks very nice.  I would like to include contrib items
>> that make systemd integration easier.  The code patches are well
>> written.
> 
> Great to hear there's hope to get the stuff to upstream.

Certainly!  I would like to pick up on the extra support for systemd.
 With this I mean the two files in contrib, and some sort of
configure-detection and then ifdeffed and iffed code.

The Makefile changes, please do not do them as part of this patch.
The build system is working today.  The conversion process is likely
to create issues.  (if such a conversion is even possible, or an
improvement).

Your patch seems to want to customize the compile itself, and
always-active, install for systemd.  This is because the Makefile is
also trying to do the package install.  Most package systems can pick
up files and install them, especially system files like init.d/rc.d.
It would be inappropriate (I think) for Unbound's Makefile to install
these files, and even more so, install them when systemd is not
present.  So, I would leave those files in contrib, the Makefile then
does not need any changes (I think).  A packager or package-script can
then pick then up and install them.  Wouldn't that be cleaner and more
portable?

That would leave the systemd changes at: files that are 'rc.d' (they
are now in contrib).  And configure detection for systemd, startup
detection for systemd, and then some code changes?  I think this means
you are already mostly done with the patch as well?

I think it is fine to use pkg-config for the systemd-configure part.
That would make it work best I guess?

Best regards, Wouter

> 
>> I would like to incorporate them (with some changes, like not 
>> installing the systemd integration from the Makefile, but having
>> the packager pick them up out of the contrib directory.).
> 
> I am not entirely sure what you mean with that. I think the
> service and socket files ought to be part of installation when
> systemd support is compiled. Assuming build system is cleaned this
> makes more sense, and is easy to achieve.
> 
>> https://github.com/kerolasa/unbound/compare/systemd?diff=unified=systemd
>>
>>
>> 
The part where it does systemd sd_listen_fds in
>> systemd_get_activated().  Does that mean systemd will open the 
>> listening file descriptors?  And unbound.conf will be ignored
>> for interfaces and port numbers?
> 
> Now when I think about what I did with pre-processor directives
> and socket-activation I must agree you are right. When systemd
> support is complied the old skool daemonization should work as
> fallback. I will change the code to do this, which also means the
> config is not ignored in case of fallback. But when systemd service
> and socket are working the config is meaningless.
> 
>> Is there more stuff you want to do for this integration?
> 
> The stuff I did to avoid pkg-config is not nice. Fixing that would 
> require dependency that not all projects agree. What is your view 
> to add pkg-config dep?
> 
> And then there is the question of build-sys in general. If you
> don't mind me waving sledgehammer I can do that, but give me
> enough time. These changes tend to take quite some hours to write
> and test.
> 
>> How is this packagable, I mean: the code changes, can unbound
>> detect it was started by systemd
> 
> It can be made to know, and inform in log for instance, but it I 
> don't think such is technically mandatory, but perhaps nice to 
> have.
> 
> Sure hater's gona hate systemd, but as far I can tell it is pretty
> transparent library like API to programmers. From that perspective
> support for systemd is the same way packagable as any random third
> party library, that may or may not be present in all systems.
> 
>> and then do the systemd code in lieu of the normal code?
> 
> Let me fix this.
> 
>> On 10/13/2015 06:37 PM, Sami Kerola via Unbound-users wrote:
>>> Hello,
>>> 
>>> First of all sorry that I wrote patches using git, but that's
>>> the version control system I like, so I took liberty to use
>>> unofficial clone as basis for my work.
>>> 
>>> https://github.com/jedisct1/unbound.git
>>> 
>>> That said I have been recently working with patch set that
>>> will glue unbound together with systemd. The patch set is not
>>> tested, but before I go and make all perfectly shiny let me
>>> request if the upstream is even interested of this kind change.
>>> In case not I won't bother making all pitch perfect. See the
>>> changes below in git pull request format.
>>> 
>>> While writing the systemd changes the other thing I notice
>>> was, ehm, a bit crude way of using autotools. Is there any
>>> interest to make configure.ac && Makefile.am stuff to work, and
>>> include requirement to pkg-config? Yep, I do ack adding
>>> pkg-config requirement is a bit

Re: unbound and systemd

2015-10-16 Thread Paul Wouters via Unbound-users 

On Fri, 16 Oct 2015, W.C.A. Wijngaards via Unbound-users wrote:


Your patch seems to want to customize the compile itself, and
always-active, install for systemd.  This is because the Makefile is
also trying to do the package install.  Most package systems can pick
up files and install them, especially system files like init.d/rc.d.
It would be inappropriate (I think) for Unbound's Makefile to install
these files, and even more so, install them when systemd is not
present.  So, I would leave those files in contrib, the Makefile then
does not need any changes (I think).  A packager or package-script can
then pick then up and install them.  Wouldn't that be cleaner and more
portable?


But that would cause a "make install" to leave something on the
filesystem that is not ready to be enabled as a service or started on
boot. So make install does need to make some decision on what init
system support files to install.

For libreswan, we detect the init system, and we provide an override.
Our detection script supports systemd, sysvinit, upstart and openrc. See:

https://github.com/libreswan/libreswan/blob/master/packaging/utils/lswan_detect.sh

But we might be a little more linux-centric then you are.

We allow an override running make with INITSYSTEM=sysvinit which is used
to install sysvinit support on RHEL6 (which technically has upstart but
no one uses it).

Paul

Re: unbound and systemd

2015-10-14 Thread W.C.A. Wijngaards via Unbound-users 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Sami,

The patch looks very nice.  I would like to include contrib items that
make systemd integration easier.  The code patches are well written.

I would like to incorporate them (with some changes, like not
installing the systemd integration from the Makefile, but having the
packager pick them up out of the contrib directory.).

https://github.com/kerolasa/unbound/compare/systemd?diff=unified=systemd

The part where it does systemd sd_listen_fds in
systemd_get_activated().  Does that mean systemd will open the
listening file descriptors?  And unbound.conf will be ignored for
interfaces and port numbers?  Is there more stuff you want to do for
this integration?  How is this packagable, I mean: the code changes,
can unbound detect it was started by systemd and then do the systemd
code in lieu of the normal code?

Best regards, Wouter

On 10/13/2015 06:37 PM, Sami Kerola via Unbound-users wrote:
> Hello,
> 
> First of all sorry that I wrote patches using git, but that's the 
> version control system I like, so I took liberty to use unofficial 
> clone as basis for my work.
> 
> https://github.com/jedisct1/unbound.git
> 
> That said I have been recently working with patch set that will
> glue unbound together with systemd. The patch set is not tested,
> but before I go and make all perfectly shiny let me request if the
> upstream is even interested of this kind change. In case not I
> won't bother making all pitch perfect. See the changes below in git
> pull request format.
> 
> While writing the systemd changes the other thing I notice was,
> ehm, a bit crude way of using autotools. Is there any interest to
> make configure.ac && Makefile.am stuff to work, and include
> requirement to pkg-config? Yep, I do ack adding pkg-config
> requirement is a bit irritating, but it does work fairly well for
> many projects so perhaps it is not that evil.  Assuming there is
> interest to make build tooling nicer, and you wish I would take a
> lead on this, please let me first finish the systemd stuff.
> 
> Cheers, Sami
> 
> p.s. Could the upstream move using git, please, please, please,
> please?
> 
>  
> The following changes since commit
> e3d4f1edfdae567b07cc75a9e3669de5bc8b1a6a: - iana portlist update.
> (2015-10-09 08:08:31 +) are available in the git repository
> at: git://github.com/kerolasa/unbound.git systemd for you to fetch
> changes up to ccda84c59e594f9c918996017ea045256ad64659: systemd:
> add do-daemonize note when running with systemd (2015-10-13 
> 17:15:46 +0100) 
>  
> Sami Kerola (12): build-sys: run autoreconf -f version 1.5 
> build-sys: add some autotools output files to .gitignore build-sys:
> add systemd support to autotools build-sys: rerun autoreconf -f and
> libtoolize -f build-sys: add ./configure output files to
> .gitignore build-sys: update .gitignore after build systemd: make
> systemd activation work for udp sockets systemd: make systemd
> activation work for tcp sockets systemd: make systemd activation
> work for unix sockets systemd: add service and socket files 
> systemd: add state notifications systemd: add do-daemonize note
> when running with systemd
> 
> .gitignore |  204 + Makefile.in|
> 12 +- aclocal.m4 | 2762 +++-- config.h.in
> |6 +- configure  | 2192 ++ configure.ac
> |6 +- contrib/unbound.service.in |   16 + 
> contrib/unbound.socket.in  |   16 + daemon/daemon.c|
> 15 +- doc/example.conf.in|1 + ltmain.sh
> | 9656 +--- 
> services/listen_dnsport.c  |   71 +- systemd.m4 |
> 28 + util/configlexer.c |   19 +- 14 files changed, 3348
> insertions(+), 11656 deletions(-) create mode 100644 .gitignore 
> create mode 100644 contrib/unbound.service.in create mode 100644
> contrib/unbound.socket.in mode change 100644 => 12 ltmain.sh 
> create mode 100644 systemd.m4
> 

-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQIcBAEBAgAGBQJWHgB1AAoJEJ9vHC1+BF+NR8AQAJXBVW4NAHxbmrKpEZun53tY
Cex4/74NwLaAZshraOH0RKwV2hlYWfFh0dILl6DZtykLOjW/W1dkCuVT5/xv9vIX
0iIvoyJiRKTjwFgsKeCRWKHgXyhbgycDyKxcLE35Qd52r7y2aWnC8xGn9eg4JPgH
yPNBKrp3qMtOITOH94Gi3rAOJWpXPVSNnBAxMUZZBJ3ObA5AhII7aIV7m9dDK4+9
lkWKLC9T4hgjAMobXFMCYaGf1oaoYR5AVrFD10DGE8D5DsLCFC0oRSZ8CuAgzBUE
TW+4kfm3b4JhJFggDkhsECmv+juUjbrJaJV4vmOR7zSxg3V9mUQQy2hinec16LOp
RB64XdTKFmO8ZWxB5zloO1KoVMKNZ7wlq4Lyx9UIhw8i96b2RTcqtn8I7onwW2W9
bgJOBb3DZFlBjC0K6Snpe09AdQy+Z/GaVoRC6466/8dHcuETxDa+9XqD8C+P3Azz
CpsKqg96Zl4ESCHTta+ahHqh/18GD799/8OWNpdaeOzcIOGpgokdM3gTaBoKZVDy
hb/G27lCho1qKok+ePvyR6WYSJlPLZjUdZ4DEOHSGA3bjj6Y015HgcUZc9Yb8cqz
4p3Y1vnJhHQTXI85aTJYU4gA/jy3cTWMDmX7Zd3mistBTCAdqxFY9wBgwktRyDiY
snp6L35lTrcQ/dBws7wU
=VEOx
-END PGP SIGNATURE-

Re: unbound and systemd

2015-10-14 Thread Sami Kerola via Unbound-users 
On 14 October 2015 at 08:12, W.C.A. Wijngaards  wrote:

Hi Wouter, et.al.,

> The patch looks very nice.  I would like to include contrib items that
> make systemd integration easier.  The code patches are well written.

Great to hear there's hope to get the stuff to upstream.

> I would like to incorporate them (with some changes, like not
> installing the systemd integration from the Makefile, but having the
> packager pick them up out of the contrib directory.).

I am not entirely sure what you mean with that. I think the service
and socket files ought to be part of installation when systemd
support is compiled. Assuming build system is cleaned this makes
more sense, and is easy to achieve.

> https://github.com/kerolasa/unbound/compare/systemd?diff=unified=systemd
>
> The part where it does systemd sd_listen_fds in
> systemd_get_activated().  Does that mean systemd will open the
> listening file descriptors?  And unbound.conf will be ignored for
> interfaces and port numbers?

Now when I think about what I did with pre-processor directives and
socket-activation I must agree you are right. When systemd support
is complied the old skool daemonization should work as fallback. I
will change the code to do this, which also means the config is not
ignored in case of fallback. But when systemd service and socket
are working the config is meaningless.

> Is there more stuff you want to do for this integration?

The stuff I did to avoid pkg-config is not nice. Fixing that would
require dependency that not all projects agree. What is your view
to add pkg-config dep?

And then there is the question of build-sys in general. If you don't
mind me waving sledgehammer I can do that, but give me enough
time. These changes tend to take quite some hours to write and
test.

> How is this packagable, I mean: the code changes,
> can unbound detect it was started by systemd

It can be made to know, and inform in log for instance, but it I
don't think such is technically mandatory, but perhaps nice to
have.

Sure hater's gona hate systemd, but as far I can tell it is
pretty transparent library like API to programmers. From that
perspective support for systemd is the same way packagable
as any random third party library, that may or may not be
present in all systems.

> and then do the systemd code in lieu of the normal code?

Let me fix this.

> On 10/13/2015 06:37 PM, Sami Kerola via Unbound-users wrote:
>> Hello,
>>
>> First of all sorry that I wrote patches using git, but that's the
>> version control system I like, so I took liberty to use unofficial
>> clone as basis for my work.
>>
>> https://github.com/jedisct1/unbound.git
>>
>> That said I have been recently working with patch set that will
>> glue unbound together with systemd. The patch set is not tested,
>> but before I go and make all perfectly shiny let me request if the
>> upstream is even interested of this kind change. In case not I
>> won't bother making all pitch perfect. See the changes below in git
>> pull request format.
>>
>> While writing the systemd changes the other thing I notice was,
>> ehm, a bit crude way of using autotools. Is there any interest to
>> make configure.ac && Makefile.am stuff to work, and include
>> requirement to pkg-config? Yep, I do ack adding pkg-config
>> requirement is a bit irritating, but it does work fairly well for
>> many projects so perhaps it is not that evil.  Assuming there is
>> interest to make build tooling nicer, and you wish I would take a
>> lead on this, please let me first finish the systemd stuff.
>>
>> Cheers, Sami
>>
>> p.s. Could the upstream move using git, please, please, please,
>> please?
>>
>> 
>> The following changes since commit
>> e3d4f1edfdae567b07cc75a9e3669de5bc8b1a6a: - iana portlist update.
>> (2015-10-09 08:08:31 +) are available in the git repository
>> at: git://github.com/kerolasa/unbound.git systemd for you to fetch
>> changes up to ccda84c59e594f9c918996017ea045256ad64659: systemd:
>> add do-daemonize note when running with systemd (2015-10-13
>> 17:15:46 +0100)
>> 
>> Sami Kerola (12): build-sys: run autoreconf -f version 1.5
>> build-sys: add some autotools output files to .gitignore build-sys:
>> add systemd support to autotools build-sys: rerun autoreconf -f and
>> libtoolize -f build-sys: add ./configure output files to
>> .gitignore build-sys: update .gitignore after build systemd: make
>> systemd activation work for udp sockets systemd: make systemd
>> activation work for tcp sockets systemd: make systemd activation
>> work for unix sockets systemd: add service and socket files
>> systemd: add state notifications systemd: add do-daemonize note
>> when running with systemd
>>
>> .gitignore |  204 + Makefile.in|
>> 12 +- aclocal.m4 | 2762 +++-- config.h.in
>> |6 +-

Re: unbound and systemd

2015-10-14 Thread Robert Edmonds via Unbound-users 
Sami Kerola via Unbound-users wrote:
> The stuff I did to avoid pkg-config is not nice. Fixing that would
> require dependency that not all projects agree. What is your view
> to add pkg-config dep?

Hi,

Using pkg-config is the documented way to detect the correct library to
link against for libsystemd, see:

http://www.freedesktop.org/software/systemd/man/sd-daemon.html

It's trivial to only use pkg-config (via the PKG_CHECK_MODULES macro)
when systemd integration is requested via --enable-systemd, and to not
fail in the absence of the pkg-config binary when systemd support is not
requested.  And systemd only runs on Linux, where pkg-config is normal
and shared libraries are expected to ship .pc files.

The only downside is that the source for the PKG_CHECK_MODULES macro
(usually shipped in /usr/share/aclocal/pkg.m4 in the "pkgconfig" (.rpm)
or "pkg-config" (.deb) package) needs to be available on the machine
where autoreconf is run.  But ordinarily users don't run autoreconf
themselves, so this is not too onerous.

So IMHO just use pkg-config, it's nicer to write one line of autoconf
instead of 50 :-)

-- 
Robert Edmonds
edmo...@debian.org