Re: [systemd-devel] sshd.service fails on boot when primary listener is a bridge (br0) instead of real interface (eth0). What dependency is needed?

[lyndat3]

Hi

On Sun, Apr 12, 2015, at 02:08 PM, Cameron Norman wrote:
> It's probably just a race that you see with the bridge because the
> bridge is up at a later time than the ethernet device.

I guess that's kindof the point here.  

Using network.target and network-online.target doesn't apparently cause a 
unit's exec to wait for the race to resolve, and for the specific bridge(s) to 
be up.

If those targets don't, is there one that does?


LT
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] sshd.service fails on boot when primary listener is a bridge (br0) instead of real interface (eth0). What dependency is needed?

[lyndat3]

> On Sun, Apr 12, 2015, at 12:49 PM, Lennart Poettering wrote:
> > Hmm? why that? why does sshd care?

opensuse has a convenient interface status tool

rpm -q --whatprovides `which ifstatus `
wicked-service-0.6.17-12.1.x86_64

ifstatus br0
br0 up
  link: #6, state up, mtu 1500
  type: bridge
  config:   compat:/etc/sysconfig/network/ifcfg-br0
  leases:   ipv4 static granted
  leases:   ipv6 static granted
  addr: ipv4 172.30.11.2/22 [static]
  addr: ipv6 2001:470:1f05:6a0::2/64 [static]
  route:ipv4 default via 172.30.11.100
  route:ipv6 default via 2001:470:1f05:6a0::100

In the bridge case, with sshd listening on the br0 IP adddess, with this unit

cat /etc/systemd/system/sshd.service 
[Unit]
Description=OpenSSH Daemon
After=syslog.target network.target network-online.target

[Service]
EnvironmentFile=-/etc/sysconfig/ssh
ExecStartPre=/usr/sbin/sshd-gen-keys-start
ExecStart=/usr/sbin/sshd -D $SSHD_OPTS
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=always

[Install]
WantedBy=multi-user.target

SSHD fails to launch on boot, firing the error I shared above.

But if a force a dependency check

cat << EOF > /usr/local/scripts/wait-for-br0.sh
#!/bin/bash
while [[ \$(ifstatus br0 | grep ^br0 | awk '{print \$2}') != 
'up' ]]; do
  sleep 1
done
EOF

cat /etc/systemd/system/sshd.service 
[Unit]
Description=OpenSSH Daemon
After=syslog.target network.target network-online.target

[Service]
EnvironmentFile=-/etc/sysconfig/ssh
ExecStartPre=/usr/sbin/sshd-gen-keys-start
+   ExecStartPre=sh /usr/local/scripts/wait-for-br0.sh
ExecStart=/usr/sbin/sshd -D $SSHD_OPTS
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=always

[Install]
WantedBy=multi-user.target

Then reboot, SSHD boots up correctly .  It's available as you'd expect.

It sure looks like a depdenency issue.

And even though the unit's waiting for 

network.target network-online.target

it doesn't seem to consider br0 being up as prereqs for those targets.

So, if NOT those targets, which one?  How do I convince the sshd unit to 
correctly wait for BRIDGES being up using systemd's ntaive dependency 
management?

Or is the only way to assume that systemd can't detect the bridges and handle 
it manually like I did above?

LT
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] sshd.service fails on boot when primary listener is a bridge (br0) instead of real interface (eth0). What dependency is needed?

[lyndat3]

On Sun, Apr 12, 2015, at 12:49 PM, Lennart Poettering wrote:
> Hmm? why that? why does sshd care?

Iiuc, sshd's LISTENER ip's must exist & be available

I am not having sshd listen on 0.0.0.0.  I am specifying that it listens on 
192.168.1.1

> What precisely fails?

I don't KNOW precisely, other than what I shared in my original post

journalctl -b | grep -i sshd
...
Apr 10 15:30:49 xen01 sshd[1345]: error: Bind to port 22 on 
192.168.1.1 failed: Cannot assign requested address.
...

sshd.service seems like it's trying to launch sshd BEFORE that interface -- 
defined in br0 config -- is available.

I can replicate the exact same error message, "Cannot assign requested 
address", in BOTH cases -- eno1 and br0 -- by simply having sshd listen on a 
non-existent IP, e.g. 10.1.1.1

> I mean, if
> the ip addres 192.168.1.1 is bound to a local interface, then sshd
> really should work, it shouldn't matter what kind of interface that
> is.

It's bound to an EXTERNAL, interface eno1 (or br0) -- attached in this case to 
the LAN.

The only internal interface, 'lo', has 127.0.0.1 & :: bound.

LT
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] sshd.service fails on boot when primary listener is a bridge (br0) instead of real interface (eth0). What dependency is needed?

[lyndat3]

On Sun, Apr 12, 2015, at 12:25 PM, Lennart Poettering wrote:
> > Seems to me that there should be *some* real check for the 'up'
> > state of the bridge, not just "waiting long enough" and hoping.
> 
> Well, I mean, the entire logic of delaying the boot for external
> conditions is already pretty wrong. If you want to fix this, then
> make sure sshd doesn't need the network to be up to start up (for
> example by using socket activation, or by using IP_FREEBIND). 
> 
> But I mean, what you are asking for is basically delaying boot
> completion indefinitely. I doubt that that'd be a good mechanism to
> have...

I don't think so.  Or at least that's not my intention, or my need.

Let me try to resimplify.

sshd.service starts fine on boot when using ethX, and fails when using br0.

Specifically, sshd listens on IP = 192.168.1.1

If interface:IP = eno1:192.168.1.1, then it's all good.

But, if interface:IP = br0(attached to eno1):192.168.1.1, then there's a fail 
of sshd.service on boot.

There's lots of comments online about bridges being 'different' than physical 
ethernet interfaces -- some were yours, IIRC.

No problem that they're different.

I just want sshd.service to start correctly when listening on a bridge 
interface IP.

I'm completely open on how to get there.  THe only thing I can think of so far 
is adding whatever the right dependency is.  If there's a better method, I'm 
all ears.

> Yes, sorry for the typos!

whew! :-)
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] sshd.service fails on boot when primary listener is a bridge (br0) instead of real interface (eth0). What dependency is needed?

[lyndat3]

Hi Lennart

On Sun, Apr 12, 2015, at 11:06 AM, Lennart Poettering wrote:
> This looks like Fedora/Red Hat initscripts networking? 

Close; same style.  But it's opensuse networking.

> If so, please file a bug against the initscripts package. It's up to
> the networking management solution to fill network-line.target with
> something useful, and in your case that appears to be the initscripts
> implementation.

Since I wasn't sure where the problem starts, I'd got that already going here

 Bug 926835 - sshd.service (openssh) fails on boot when primary listener is a 
bridge (br0) instead of real interface (eth0)
  https://bugzilla.opensuse.org/show_bug.cgi?id=926835

So far the advice there is to increase the value of WAIT_FOR_INTERFACES in 
/etc/sysconfig/network/config from 30 sec to > 40 secs.

Seems to me that there should be *some* real check for the 'up' state of the 
bridge, not just "waiting long enough" and hoping.

> systemd just provides the the name network-ionline.target really.


You mentioned both

network-line.target

and

 network-ionline.target 


Just to be sure, are those just typos for 'network-online.target'?

LT
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] sshd.service fails on boot when primary listener is a bridge (br0) instead of real interface (eth0). What dependency is needed?

[lyndat3]

On Fri, Apr 10, 2015, at 05:54 PM, Mantas Mikulėnas wrote:
> By itself, if I remember correctly, network-online.target doesn't wait
> for *anything;
> *it needs the provider-specific services to also be enabled, like
> NetworkManager-wait-online.service (similarly for systemd-networkd and
> ifupdown).

Not completely sure of the details, but it seems to provide the right 
dependency for services like sshd, and firewall(s), unlike network.target 
doesn't always.

"network-online.target

Units that strictly require a configured network connection should pull in 
network-online.target (via a Wants= type dependency) and order themselves after 
it. This target unit is intended to pull in a service that delays further 
execution until the network is sufficiently set up. What precisely this 
requires is left to the implementation of the network managing service.

Note the distinction between this unit and network.target. This unit is an 
active unit (i.e. pulled in by the consumer rather than the provider of this 
functionality) and pulls in a service which possibly adds substantial delays to 
further execution. In contrast, network.target is a passive unit (i.e. pulled 
in by the provider of the functionality, rather than the consumer) that usually 
does not delay execution much. Usually, network.target is part of the boot of 
most systems, while network-online.target is not, except when at least one unit 
requires it. Also see Running Services After the Network is up for more 
information.

All mount units for remote network file systems automatically pull in this 
unit, and order themselves after it. Note that networking daemons that simply 
provide functionality to other hosts generally do not need to pull this in.
"

There's a lot of chatter for example of the need for early init of Shorewall, 
on their list.

So though not clear on the real-intfc-only case, it's clearly not enough of a 
dependency check when the bridge interface comes into play.

Wondering if simply adding an "After=after-local.target" -- i.e. the latest 
unit trigger I'm aware of -- to the sshd deps is the right way to go here.

Seems a blunt instrument, but it does work.


___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] sshd.service fails on boot when primary listener is a bridge (br0) instead of real interface (eth0). What dependency is needed?

[lyndat3]

My /etc/systemd/system/sshd.service has a

  After=network-online.target

dependency.

When my ethernet interface, eno1, is the primary 'net listener,

cat /etc/sysconfig/network/ifcfg-eno1
STARTMODE='auto'  
BOOTPROTO='static'
IPADDR='192.168.1.1/24'

sshd, listening on 192.168.1.1, has no problems on boot.

But, if I switch to using a bridge interface, br0, with eno1 as the real 
interface assigned to it,

cat /etc/sysconfig/network/ifcfg-eno1 
STARTMODE='auto'
BOOTPROTO='none'

cat /etc/sysconfig/network/ifcfg-br0
STARTMODE='auto'
BOOTPROTO='static'
BRIDGE='yes'
BRIDGE_PORTS='eno1'
IPADDR0='192.168.1.1/24'

sshd fails on boot

journalctl -b | grep -i sshd
...
Apr 10 15:30:49 xen01 sshd[1345]: error: Bind to port 22 on 
192.168.1.1 failed: Cannot assign requested address.
...

Apparently, network-online waits for *real* interfaces only.

Once at the shell, br0 is fully up.  sshd can be started, and it functions 
normally.

What dependency do I need to add to my sshd unit to get it to wait for br0 
being fully up?

LT

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel