On 1/15/21 2:50 PM, Tadej Janež wrote:
Hi all!
Hey Tadej
I was recently hit by this and it took me some time to figure out what
was going on.
Thanks for sharing and for investigating.
I provisioned a fresh Fedora 33 cloud instance on DigitalOcean.
After completing the initial upgrade via 'dnf update' and rebooting the
system, DNS resolution stopped working.
I did a little debugging and saw that no DNS servers were configured:
$ resolvectl dns
Global:
Link 2 (eth0):
Link 3 (eth1):
$ resolvectl status
Global
Protocols: LLMNR=resolve -mDNS -DNSOverTLS DNSSEC=no/unsupported
resolv.conf mode: stub
Link 2 (eth0)
Current Scopes: LLMNR/IPv4 LLMNR/IPv6
Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS
DNSSEC=no/unsupported
Link 3 (eth1)
I was confused how could that happen since running 'dnf update' on the
first boot clearly required DNS resolution to work.
It turns out a recent systemd update, namely systemd-246.7-2.fc33
(
https://bodhi.fedoraproject.org/updates/FEDORA-2020-3616681a70),
includes this commit:
https://src.fedoraproject.org/rpms/systemd/c/ed795fb1fc9a2c20ebcac34bdf7e...
which removes the fallback DNS server list (i.e. Cloudflare's and/or
Google's DNS servers) due to concerns over DNS requests being
personally identifying information (PII) in the sense of GDPR?
In combination with NetworkManager's DNS server handling being disabled
via cloud-init:
$ cat /etc/NetworkManager/conf.d/99-cloud-init.conf
# Created by cloud-init on instance boot automatically, do not edit.
#
[main]
dns = none
this results in no DNS servers being configured on the system.
The consequence is that Fedora 33 cloud instances are broken after
updating the systemd package to version 246.7-2.fc33.
There is a bug in cloud-init where it doesn't seem to properly handle
systemd-resolved. It edits /etc/resolv.conf directly (which is already
managed by systemd-resolved) and thus the configuration gets lost
after the first boot.
https://bugzilla.redhat.com/show_bug.cgi?id=1950770
I just tried to reproduce on DigitalOcean and it seems like maybe DO recently
added a workaround for this (i'm not able to reproduce). I see this workaround:
# cat lib/cloud/instances/242253247/boothooks/resolver-fix
#!/bin/sh -x
# DigitalOcean resolver fixup script
[ -f /etc/systemd/resolved.conf ] || exit 0
logger -t DigitalOcean 'writing resolvers to /etc/systemd/resolved.conf'
printf "[Resolve]\nDNS=67.207.67.2 67.207.67.3\n" >
/etc/systemd/resolved.conf
Can you try again and see if a new f33 instance you start is OK?
(I haven't checked but if Fedora 34 cloud images include this version
of systemd, then DNS resolution won't work even on first boot.)
Thanks, I'll look into it.
>
> Please advise on the best steps to fix this issue
>
> Regards,
> Tadej