Hi, I seem to be facing a similar issue with one of my KRAs. My KRA certificates were, for some reason, not automatically renewed when they expired last month. Using `ipa-cert-fix` correctly fixed them on _one_ host. On the other, they seem to be stuck in the renewal state and `ipa-cert-fix` claims there's nothing to do: ``` Request ID '20191031183458': status: MONITORING ca-error: Server at " http://ipa-one.mydomain.org:8080/ca/ee/ca/profileSubmit" replied: Missing credential: sessionID stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-kra',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-kra',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=MYDOMAIN.ORG subject: CN=KRA Audit,O=MYDOMAIN.ORG expires: 2020-06-27 01:54:34 EDT key usage: digitalSignature,nonRepudiation pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-kra" track: yes auto-renew: yes Request ID '20191031183459': status: MONITORING ca-error: Server at " http://ipa-one.mydomain.org:8080/ca/ee/ca/profileSubmit" replied: Missing credential: sessionID stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='transportCert cert-pki-kra',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='transportCert cert-pki-kra',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=MYDOMAIN.ORG subject: CN=KRA Transport Certificate,O=MYDOMAIN.ORG expires: 2020-06-27 01:54:30 EDT key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-clientAuth pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "transportCert cert-pki-kra" track: yes auto-renew: yes Request ID '20191031183500': status: MONITORING ca-error: Server at " http://ipa-one.mydomain.org:8080/ca/ee/ca/profileSubmit" replied: Missing credential: sessionID stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='storageCert cert-pki-kra',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='storageCert cert-pki-kra',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=MYDOMAIN.ORG subject: CN=KRA Storage Certificate,O=MYDOMAIN.ORG expires: 2020-06-27 01:54:32 EDT key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-clientAuth pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "storageCert cert-pki-kra" track: yes auto-renew: yes ``` Here are the sequence of events that seem to have led to this: 1. Install FreeIPA Master many years ago and continue to upgrade it from time to time. 2. Install FreeIPA Replica a few years after and continue to upgrade it from time to time. 3. Allow the certificates to expire on both nodes. 4. Attempt to patch the replica via `yum upgrade` on the second node. 5. Notice after reboot that `pki-tomcatd` is having trouble and discover certificate issues. 5. Issue `ipa-cert-fix`, reboot again, and notice that things are working. Try and create a key in the vault. 6. Attempt to patch the master via `yum upgrade` on the first node. 7. Notice after reboot that everything seems to be ok. Try and create a key in the vault. 8. Notice a few days later that renewal seems to be broken on the first node. At this point `ipa-cert-fix` just shows that everything is fine. If I run it with -v, and then check the "storageCert cert-pki-kra" certificate with `openssl x509 -text -in`, I'm shown: Validity Not Before: Jun 29 00:52:33 2020 GMT Not After : Jun 19 00:52:33 2022 GMT On the second known, `getcert list` shows correct expirations for those certificates: Request ID '20191206005909': status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='storageCert cert-pki-kra',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='storageCert cert-pki-kra',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=MYDOMAIN.ORG subject: CN=KRA Storage Certificate,O=MYDOMAIN.ORG expires: 2022-06-18 20:52:33 EDT key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-clientAuth pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "storageCert cert-pki-kra" track: yes auto-renew: yes It seems like _something_, perhaps `ipa-cert-fix` somehow renewed these certificates but...outside of certmonger? Is this some other version of https://bugzilla.redhat.com/show_bug.cgi?id=1788907? The certificates are not in CA_WORKING though, they're in MONITORING. What can I do to get myself out of this state as it seems like I'm in a "this could explode at any moment" situation? This is on Fedora 30 with IP version: Last metadata expiration check: 0:23:05 ago on Sat 04 Jul 2020 07:59:16 PM EDT. Installed Packages Name : certmonger Version : 0.79.9 Release : 1.fc30 Architecture : x86_64 Size : 3.4 M Source : certmonger-0.79.9-1.fc30.src.rpm Repository : @System From repo : updates .. snip .. Name : freeipa-server Version : 4.8.3 Release : 1.fc30 Architecture : x86_64 Size : 1.3 M Source : freeipa-4.8.3-1.fc30.src.rpm Repository : @System From repo : updates .. snip .. Thanks! Ilya Kogan w: github.com/ikogan e: ikogan(a)mythicnet.org <http://twitter.com/ilkogan> <https://www.linkedin.com/in/ilyakogan/>