slopfront
Reddit, read-only. No JavaScript. No cookies. No tracking.
preferences
Saved in links, not cookies.

The /usr/lib/systemd/systemd-networkd-wait-online in Ubuntu 24.04 seems broken compared to the one in 22.04 and prior.

6 points · submitted Mar 2025 by u/Ark42

I'm trying to put together custom USBs that auto-install mostly stock Ubuntu server, but with some custom addons, one of which is that we use ifupdown still, and not netplan, but this worked in 22.04 and 20.04 with similar setups just fine. Everything works except each stage of the boot with cloudinit and then the final working system hangs for 2 minutes on systemd-networkd-wait-online.service.

Now, I've seen a lot of posts from Google that say to just disable/mask this service, but I want it to be in place and working. I have dependencies like keepalived and openvpn that need to wait until the network is up to start. I don't have NetworkManager or any desktop-related things installed either.

Even on a fresh install from Ubuntu 24.04, if I just run /usr/lib/systemd/systemd-networkd-wait-online it will hang for 2 minutes. /usr/lib/systemd/systemd-networkd-wait-online --any will as well. But a command like /usr/lib/systemd/systemd-networkd-wait-online -i enp88s0 will immediately return, so it seems to be able to test specific interfaces just fine.

On Ubuntu 22.04, running this command with no args or with --any will immediately return. Running it with -i any an interface configured with a static IP will also return just as it does on Ubuntu 24.04. I even tried copying the /lib/systemd/libsystemd-shared-249.so and /usr/lib/systemd/systemd-networkd-wait-online files from a working Ubuntu 22.04 machine to Ubuntu 24.04, and those work fine as well, which tells me that it's not a network configuration problem most likely.

So, I'm really quite confused now. Is there something I'm missing? Is there some new systemd-networkd options somewhere that I'm supposed to be setting? Is this just a bug introduced into the systemd-networkd-wait-online binary, or some new intentional behavior?

4 comments · 100% upvoted · other discussions

comments

Read-only. Use the arrows to collapse threads.

TheSpr1te 2 points · Mar 2025

Does networlctl show the interface as configured? If not, is the udev package installed?

Ark42 [OP] 1 points · Mar 2025 
IDX LINK    TYPE     OPERATIONAL SETUP    
  1 lo      loopback carrier     unmanaged
  2 enp88s0 ether    routable    unmanaged
  3 wlo1    wlan     off         unmanaged
  4 tun0    none     routable    unmanaged

Yes, it's routable, but listed as unmanaged because we use ifupdown, and I'm accessing this test machine via ssh. This same exact setup worked in Ubuntu 20.04 and 22.04 live server custom images we made.

udev 255.4-1ubuntu8.5 is installed.

TheSpr1te 2 points · Mar 2025

This is very strange indeed. If -i enp88s0 works then --anyshould work as well, but according to this issue it's not that simple:

--any doesn't work at all for networks that systemd is not managing:

The manual says: Even if several interfaces are in configuring state, systemd-networkd-wait-online exits with success when at least one interface becomes online.

Which is clearly a lie:

(...)

Please at least document that --any doesn't work for non systemd managed interfaces, which seems unreasonable.

Anyways -i works:

Ark42 [OP] 1 points · Mar 2025

Thanks for the link. I too really struggled to get --debug or --verbose info from systemd-networkd-wait-online. I would have never guessed to set some undocumented environment variable SYSTEMD_LOG_LEVEL=debug when running a cli tool like that.

To me, this seems like a major bug and regression in systemd, but now I'm also wondering if this wait-online tool even worked correctly for my use-case in the past or not...

On the same hardware, same installation of Ubuntu 24.04, I have the files libsystemd-shared-249.so + systemd-networkd-wait-online copied from an older Ubuntu 22.04 machine to compare:

systemd 249:

$ LD_LIBRARY_PATH=. ./systemd-networkd-wait-online
Found link 4
Found link 3
Found link 2
Found link 1
enp88s0: link is not managed by networkd (yet?).
tun0: link is not managed by networkd (yet?).
lo: link is ignored
wlo1: link is not managed by networkd (yet?).

$ LD_LIBRARY_PATH=. ./systemd-networkd-wait-online -i enp88s0
Found link 4
Found link 3
Found link 2
Found link 1
enp88s0: link is configured by networkd and online.

systemd 255:

$ /usr/lib/systemd/systemd-networkd-wait-online 
Found link lo(1)
Found link enp88s0(2)
Found link wlo1(3)
Found link tun0(4)
wlo1: link is not managed by networkd.
enp88s0: link is not managed by networkd.
lo: link is ignored
tun0: link is not managed by networkd.
Timeout occurred while waiting for network connectivity.

$ /usr/lib/systemd/systemd-networkd-wait-online -i enp88s0
Found link lo(1)
Found link enp88s0(2)
Found link wlo1(3)
Found link tun0(4)
enp88s0: link is configured by networkd and online.

In both version of systemd, using no args (which I assume is the same as --any) says enp88s0 is "not managed" by networkd, and in both versions using -i says enp88s0 is "configured by networkd and online" but in systemd 255 it will now hang for 2 minutes with no args (the case which matters for the service running by default) and in systemd 249 it either didn't actually block until the network was online, or it worked correctly. The debug logs don't really say what else is going on under the hood.