Hi,
We experienced the same where we now only see direct memberships.
During the wee hours of Dec 7. We saw a crash in our IPA server, running Centos 7
(we’re using nss-pam-ldapd on our hosts, which are running OEL7)
They’ve gotten indirect/nested memberships without any problems previously.
From our yum logs we can see that the last few days we’ve got the following updated packages:
Nov 22 05:24:29 Installed: kernel.x86_64 3.10.0-1160.80.1.el7
Nov 22 05:25:27 Updated: microcode_ctl.x86_64 2:2.1-73.15.el7_9
Dec 01 05:22:47 Updated: krb5-libs.x86_64 1.15.1-55.el7_9
Dec 01 05:22:47 Updated: libkadm5.x86_64 1.15.1-55.el7_9
Dec 01 05:22:47 Updated: krb5-workstation.x86_64 1.15.1-55.el7_9
Dec 01 05:22:47 Updated: krb5-devel.x86_64 1.15.1-55.el7_9
Dec 01 05:22:48 Updated: krb5-server.x86_64 1.15.1-55.el7_9
Dec 01 05:22:48 Updated: krb5-pkinit.x86_64 1.15.1-55.el7_9
Dec 01 05:22:50 Updated: tzdata.noarch 2022f-1.el7
Dec 01 05:22:50 Updated: hsqldb.noarch 1:1.8.1.3-15.el7_9
Dec 01 05:22:51 Updated: tzdata-java.noarch 2022f-1.el7
Dec 01 05:22:51 Updated: kpartx.x86_64 0.4.9-136.el7_9
We did see the Derectory Service being in a STOPPED state, on `ipactl start`
We get the following:
[root@ipa slapd-REDACTED-REDACTEDSOMEMORE]# ipactl start
IPA version error: data needs to be upgraded (expected version '4.6.8-5.el7.centos.12', current version '4.6.8-5.el7.centos.11')
Automatically running upgrade, for details see /var/log/ipaupgrade.log
Be patient, this may take a few minutes.
[76068899.913648] ns-slapd[6185]: segfault at 10 ip 00007f997c761460 sp 00007f99886cc760 error 4 in libcos-plugin.so[7f997c75e000+a000]
Starting Directory Service
Starting krb5kdc Service
Starting kadmin Service
Starting named Service
Starting httpd Service
Starting ipa-custodia Service
Starting ntpd Service
Starting pki-tomcatd Service
Starting ipa-otpd Service
Starting ipa-dnskeysyncd Service
ipa: INFO: The ipactl command was successful
from the ipaupgrade.log
2022-12-07T03:07:58Z ERROR Introspect error on :1.25883111:/org/fedorahosted/certmonger: dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
2022-12-07T03:07:58Z DEBUG Executing introspect queue due to error
2022-12-07T03:08:23Z ERROR IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
2022-12-07T03:08:23Z DEBUG File "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 178, in execute
return_value = self.run()
File "/usr/lib/python2.7/site-packages/ipaserver/install/ipa_server_upgrade.py", line 54, in run
server.upgrade()
File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py", line 2190, in upgrade
upgrade_configuration()
File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py", line 1930, in upgrade_configuration
http.configure_certmonger_renewal_guard()
File "/usr/lib/python2.7/site-packages/ipaserver/install/httpinstance.py", line 335, in configure_certmonger_renewal_guard
path = iface.find_ca_by_nickname('IPA')
File "/usr/lib64/python2.7/site-packages/dbus/proxies.py", line 70, in __call__
return self._proxy_method(*args, **keywords)
File "/usr/lib64/python2.7/site-packages/dbus/proxies.py", line 145, in __call__
**keywords)
File "/usr/lib64/python2.7/site-packages/dbus/connection.py", line 651, in call_blocking
message, timeout)
2022-12-07T03:08:23Z DEBUG The ipa-server-upgrade command failed, exception: DBusException: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
2022-12-07T03:08:23Z ERROR Unexpected error - see /var/log/ipaupgrade.log for details:
DBusException: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
And
2022-12-07T07:05:05Z DEBUG stderr=certutil: Could not find cert: ipaCert
: PR_FILE_NOT_FOUND_ERROR: File not found
The upgrade log can be provided if needed
Best Regards
Trond Strømme