On Wed, Sep 20, 2017 at 08:50:03AM +1000, Lachlan Musicman via FreeIPA-users wrote: > 2017-09-19T22:30:50Z DEBUG wait_for_open_ports: localhost [8080, 8443] > timeout 300 > 2017-09-19T22:35:51Z ERROR IPA server upgrade failed: Inspect > /var/log/ipaupgrade.log and run command ipa-server-upgrade manually. > 2017-09-19T22:35:51Z DEBUG File "/usr/lib/python2.7/site- > packages/ipapython/admintool.py", line 172, in execute > return_value = self.run() > File "/usr/lib/python2.7/site-packages/ipaserver/install/ipa_ server_upgrade.py", > line 46, in run > server.upgrade() > File "/usr/lib/python2.7/site-packages/ipaserver/install/server/ upgrade.py", > line 1913, in upgrade > upgrade_configuration() > File "/usr/lib/python2.7/site-packages/ipaserver/install/server/ upgrade.py", > line 1652, in upgrade_configuration > ca.start('pki-tomcat') > File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", > line 401, in start > self.service.start(instance_name, capture_output=capture_output, > wait=wait) > File "/usr/lib/python2.7/site-packages/ipaplatform/redhat/service s.py", > line 211, in start > instance_name, capture_output=capture_output, wait=wait) > File "/usr/lib/python2.7/site-packages/ipaplatform/base/services.py", > line 300, in start > self.wait_for_open_ports(self.service_instance(instance_name)) > File "/usr/lib/python2.7/site-packages/ipaplatform/base/services.py", > line 270, in wait_for_open_ports > self.api.env.startup_timeout) > File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 1227, > in wait_for_open_ports > raise socket.timeout("Timeout exceeded") > > 2017-09-19T22:35:51Z DEBUG The ipa-server-upgrade command failed, > exception: timeout: Timeout exceeded > 2017-09-19T22:35:51Z ERROR Timeout exceeded > 2017-09-19T22:35:51Z ERROR The ipa-server-upgrade command failed. See > /var/log/ipaupgrade.log for more information >

Can you please provide log files? Especially /var/log/ipaupgrade.log, to begin with.

On 20 September 2017 at 13:01, Lachlan Musicman <datakid(a)gmail.com&gt; wrote: > https://pagure.io/freeipa/c/bdf9a34dffdf4d7925208e5df9f69e3927b88858 > On 20 September 2017 at 12:30, Fraser Tweedale <ftweedal(a)redhat.com&gt; > wrote: > >> >> Can you please provide log files? Especially >> /var/log/ipaupgrade.log, to begin with. >> > > Fraser, thanks for the reply. I meant to answer my own email with the > solution but I couldn't see it on the list? > > Anyway - the solution was that the /etc/hosts file on the server in > question had a ::1 localhost address. We have the IPv6 disabled > (combination of one of our services not working with IPv6 and our network > not being IPv6 ready) in the OS. > > Once I deleted that line from /etc/hosts, everything went to plan. > > Ok. By the look of this commit (to 4.5): https://pagure.io/freeipa/c/bdf9a34dffdf4d7925208e5df9f69e3927b88858 from this issue https://pagure.io/freeipa/issue/7083 It is (or was) the IPv6 problem. We have an [root@linuxidm ~]# cat /etc/sysctl.d/ipv6.conf # Disable IPv6 net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.ens160.disable_ipv6 = 1 We don't have the 'lo' interface defined in there, but it's never been an issue. The /etc/hosts entry for ::1 must have thrown ipa-server-upgrade.

On 20 September 2017 at 15:54, Alexander Bokovoy <abokovoy(a)redhat.com&gt; wrote: > > Ok. By the look of this commit (to 4.5): >> >> https://pagure.io/freeipa/c/bdf9a34dffdf4d7925208e5df9f69e3927b88858 >> >> from this issue https://pagure.io/freeipa/issue/7083 >> >> It is (or was) the IPv6 problem. >> >> We have an >> >> [root@linuxidm ~]# cat /etc/sysctl.d/ipv6.conf >> # Disable IPv6 >> net.ipv6.conf.all.disable_ipv6 = 1 >> net.ipv6.conf.ens160.disable_ipv6 = 1 >> >> We don't have the 'lo' interface defined in there, but it's never been an >> issue. >> >> The /etc/hosts entry for ::1 must have thrown ipa-server-upgrade. >> > I'm a bit tired to repeat this multiple times but FreeIPA does require > IPv6 stack to be enabled in the kernel. We absolutely do. If you don't > use IPv6 stack, disable it on specific interfaces. However, there is a > practical problem with the way how glibc DNS resolver works: in default > configuration it always prefers IPv6 answers to IPv4 because this is > actually a policy of RFC3484. As result, if you have ::1 in /etc/hosts, > it will be returned first. If you don't have ::1 on any of your > interfaces ('lo' is a typical one), then apps cannot contact ::1 > (localhost) even if those apps that use IPv6 bind to all interfaces. > > FreeIPA uses modern APIs provided by glibc to listen on both IPv6 and > IPv4. It simply means that FreeIPA servers bind to IPv6 addresses (on > all interfaces or on a specific one, if needed) and treat IPv4 as mapped > ones because IPv6 and IPv4 share the same port space on the same > machine. This works transparently thanks to glibc and is a recommended > way to write networking applications. See man ipv6(7) for details. > > Here is how it looks on a real system, for TCP listeners: > > # netstat -nltp > Active Internet connections (only servers) > Proto Recv-Q Send-Q Local Address Foreign Address > State PID/Program name tcp 0 0 127.0.0.1:953 > 0.0.0.0:* LISTEN 13361/named-pkcs11 tcp > 0 0 0.0.0.0:445 0.0.0.0:* LISTEN > 13760/smbd tcp 0 0 0.0.0.0:49152 0.0.0.0:* > LISTEN 13765/smbd tcp 0 0 > 0.0.0.0:135 0.0.0.0:* LISTEN 13763/smbd > tcp 0 0 0.0.0.0:139 0.0.0.0:* > LISTEN 13760/smbd tcp 0 0 0.0.0.0:749 > 0.0.0.0:* LISTEN 13351/kadmind tcp > 0 0 0.0.0.0:464 0.0.0.0:* LISTEN > 13351/kadmind tcp 0 0 192.168.100.233:53 0.0.0.0:* > LISTEN 13361/named-pkcs11 tcp 0 0 > 127.0.0.1:53 0.0.0.0:* LISTEN > 13361/named-pkcs11 tcp 0 0 0.0.0.0:22 0.0.0.0:* > LISTEN 2838/sshd tcp 0 0 > 0.0.0.0:88 0.0.0.0:* LISTEN > 13345/krb5kdc tcp6 0 0 ::1:953 :::* > LISTEN 13361/named-pkcs11 tcp6 0 0 :::8443 > :::* LISTEN 13603/java tcp6 > 0 0 :::443 :::* LISTEN > 13379/httpd tcp6 0 0 :::636 :::* > LISTEN 13296/ns-slapd tcp6 0 0 :::445 > :::* LISTEN 13760/smbd tcp6 > 0 0 :::49152 :::* LISTEN > 13765/smbd tcp6 0 0 :::9090 :::* > LISTEN 1/systemd tcp6 0 0 > 127.0.0.1:8005 :::* LISTEN 13603/java > tcp6 0 0 :::389 :::* > LISTEN 13296/ns-slapd tcp6 0 0 :::135 > :::* LISTEN 13763/smbd tcp6 0 0 > 127.0.0.1:8009 :::* LISTEN 13603/java > tcp6 0 0 :::139 :::* > LISTEN 13760/smbd tcp6 0 0 :::749 > :::* LISTEN 13351/kadmind tcp6 0 0 > :::8080 :::* LISTEN 13603/java > tcp6 0 0 :::80 :::* > LISTEN 13379/httpd tcp6 0 0 :::464 > :::* LISTEN 13351/kadmind tcp6 0 0 > :::53 :::* LISTEN > 13361/named-pkcs11 tcp6 0 0 :::22 :::* > LISTEN 2838/sshd tcp6 0 0 :::88 > :::* LISTEN 13345/krb5kdc > Notice that many ports are only available as tcp6 listeners? Like 636 > (LDAPS), 389 (LDAP), 80 (HTTP), 443 (HTTPS) and so on? This is an effect > of using v6 API that supports v4-mapped-on-v6 addresses. It makes the > code less complex and handles with the same code both IPv6 and IPv4.