Thank you Florence. It was in fact because I did not have renewal master. I actually sent in an update by replying to my initial email about how it was fixed but that email appears to be lost.

I wonder how we got to the situation that we do not have a renewal master. That's probably also the reason why auto renewal did not work...

Regrads,
Qing

On Tue, Jan 2, 2018 at 4:26 AM, Florence Blanc-Renaud <flo@redhat.com> wrote:
On 12/31/2017 12:18 AM, Qing Chang via FreeIPA-users wrote:
Greetings,

we have some certs expired on Dec 27, ipaCert among them, IPA (VERSION: 4.4.0, API_VERSION: 2.213) stopped working.

I have spent many hours to renew the certs to no avail.

I have followed a collection of tips on this list:
  rolled back the clock to before the expiry (Dec 23),
  enabled debug logs for certmonger renewal log (getcert modify-ca -c dogtag-ipa-ca-renew-agent -e '/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit -vv')
  added debug=true to /etc/ipa/default.conf
  ipactl start starts everything successfully
  systemctl start pki-tomcatd@pki-tomcat
  systemctl restart certmonger

Before resubmit, "getcert list" has this, note ca-error: Invalid cookie: '':
-----
getcert list
Number of certificates and requests being tracked: 8.
Request ID '20170201190112':
         status: MONITORING
         ca-error: Invalid cookie: ''
         stuck: no
         key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
         certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
         CA: dogtag-ipa-ca-renew-agent
          issuer: CN=Certificate Authority,O=CAMHRES.CA <http://CAMHRES.CA>
         subject: CN=CA Audit,O=CAMHRES.CA <http://CAMHRES.CA>
         expires: 2017-12-27 14:36:44 UTC
         key usage: digitalSignature,nonRepudiation
         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
         track: yes
         auto-renew: yes
Request ID '20170201190113':
         status: MONITORING
         ca-error: Invalid cookie: ''
         stuck: no
         key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
         certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB'
         CA: dogtag-ipa-ca-renew-agent
          issuer: CN=Certificate Authority,O=CAMHRES.CA <http://CAMHRES.CA>
         subject: CN=OCSP Subsystem,O=CAMHRES.CA <http://CAMHRES.CA>
         expires: 2017-12-27 14:36:43 UTC
         key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
         eku: id-kp-OCSPSigning
         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca"
         track: yes
         auto-renew: yes
Request ID '20170201190114':
         status: MONITORING
         ca-error: Invalid cookie: ''
         stuck: no
         key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set
         certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB'
         CA: dogtag-ipa-ca-renew-agent
          issuer: CN=Certificate Authority,O=CAMHRES.CA <http://CAMHRES.CA>
         subject: CN=CA Subsystem,O=CAMHRES.CA <http://CAMHRES.CA>
         expires: 2017-12-27 14:36:43 UTC
         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
         eku: id-kp-serverAuth,id-kp-clientAuth
         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca"
         track: yes
         auto-renew: yes
Request ID '20170201190115':
         status: MONITORING
         stuck: no
         key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
         certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB'
         CA: dogtag-ipa-ca-renew-agent
          issuer: CN=Certificate Authority,O=CAMHRES.CA <http://CAMHRES.CA>
         subject: CN=Certificate Authority,O=CAMHRES.CA <http://CAMHRES.CA>
         expires: 2036-01-07 14:36:42 UTC
         key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca"
         track: yes
         auto-renew: yes
Request ID '20170201190116':
         status: MONITORING
         ca-error: Invalid cookie: ''
         stuck: no
         key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
         certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB'
         CA: dogtag-ipa-ca-renew-agent
          issuer: CN=Certificate Authority,O=CAMHRES.CA <http://CAMHRES.CA>
         subject: CN=IPA RA,O=CAMHRES.CA <http://CAMHRES.CA>
         expires: 2017-12-27 14:37:02 UTC
         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
         eku: id-kp-serverAuth,id-kp-clientAuth
         pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
         post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
         track: yes
         auto-renew: yes
Request ID '20170201190117':
         status: MONITORING
         stuck: no
         key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set
         certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB'
         CA: dogtag-ipa-renew-agent
          issuer: CN=Certificate Authority,O=CAMHRES.CA <http://CAMHRES.CA>
         subject: CN=rprshipav01.camhres.ca <http://rprshipav01.camhres.ca>,O=CAMHRES.CA <http://CAMHRES.CA>
         expires: 2019-11-19 19:38:26 UTC
         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
         eku: id-kp-serverAuth,id-kp-clientAuth
         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca"
         track: yes
         auto-renew: yes
Request ID '20170201190118':
         status: MONITORING
         stuck: no
         key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-CAMHRES-CA',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-CAMHRES-CA/pwdfile.txt'
         certificate: type=NSSDB,location='/etc/dirsrv/slapd-CAMHRES-CA',nickname='Server-Cert',token='NSS Certificate DB'
         CA: IPA
          issuer: CN=Certificate Authority,O=CAMHRES.CA <http://CAMHRES.CA>
         subject: CN=rprshipav01.camhres.ca <http://rprshipav01.camhres.ca>,O=CAMHRES.CA <http://CAMHRES.CA>
         expires: 2019-12-11 19:38:29 UTC
          principal name: ldap/rprshipav01.camhres.ca@CAMHRES.CA <mailto:rprshipav01.camhres.ca@CAMHRES.CA>
         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
         eku: id-kp-serverAuth,id-kp-clientAuth
         pre-save command:
         post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv CAMHRES-CA
         track: yes
         auto-renew: yes
Request ID '20170201190119':
         status: MONITORING
         stuck: no
         key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
         certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB'
         CA: IPA
          issuer: CN=Certificate Authority,O=CAMHRES.CA <http://CAMHRES.CA>
         subject: CN=rprshipav01.camhres.ca <http://rprshipav01.camhres.ca>,O=CAMHRES.CA <http://CAMHRES.CA>
         expires: 2019-12-11 19:38:38 UTC
          principal name: HTTP/rprshipav01.camhres.ca@CAMHRES.CA <mailto:rprshipav01.camhres.ca@CAMHRES.CA>
         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
         eku: id-kp-serverAuth,id-kp-clientAuth
         pre-save command:
         post-save command: /usr/libexec/ipa/certmonger/restart_httpd
         track: yes
         auto-renew: yes
-----

After resubmitting:
ipa-getcert resubmit -i 20170201190112
ipa-getcert resubmit -i 20170201190113
ipa-getcert resubmit -i 20170201190114
ipa-getcert resubmit -i 20170201190116

getcert list shows this, note status: CA_WORKING:
-----
Number of certificates and requests being tracked: 8.
Request ID '20170201190112':
         status: CA_WORKING
         stuck: no
         key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
         certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
         CA: dogtag-ipa-ca-renew-agent
          issuer: CN=Certificate Authority,O=CAMHRES.CA <http://CAMHRES.CA>
         subject: CN=CA Audit,O=CAMHRES.CA <http://CAMHRES.CA>
         expires: 2017-12-27 14:36:44 UTC
         key usage: digitalSignature,nonRepudiation
         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
         track: yes
         auto-renew: yes
Request ID '20170201190113':
         status: CA_WORKING
         stuck: no
         key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
         certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB'
         CA: dogtag-ipa-ca-renew-agent
          issuer: CN=Certificate Authority,O=CAMHRES.CA <http://CAMHRES.CA>
         subject: CN=OCSP Subsystem,O=CAMHRES.CA <http://CAMHRES.CA>
         expires: 2017-12-27 14:36:43 UTC
         key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
         eku: id-kp-OCSPSigning
         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca"
         track: yes
         auto-renew: yes
Request ID '20170201190114':
         status: CA_WORKING
         stuck: no
         key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set
         certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB'
         CA: dogtag-ipa-ca-renew-agent
          issuer: CN=Certificate Authority,O=CAMHRES.CA <http://CAMHRES.CA>
         subject: CN=CA Subsystem,O=CAMHRES.CA <http://CAMHRES.CA>
         expires: 2017-12-27 14:36:43 UTC
         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
         eku: id-kp-serverAuth,id-kp-clientAuth
         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca"
         track: yes
         auto-renew: yes
Request ID '20170201190115':
         status: MONITORING
         stuck: no
         key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
         certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB'
         CA: dogtag-ipa-ca-renew-agent
          issuer: CN=Certificate Authority,O=CAMHRES.CA <http://CAMHRES.CA>
         subject: CN=Certificate Authority,O=CAMHRES.CA <http://CAMHRES.CA>
         expires: 2036-01-07 14:36:42 UTC
         key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca"
         track: yes
         auto-renew: yes
Request ID '20170201190116':
         status: CA_WORKING
         stuck: no
         key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
         certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB'
         CA: dogtag-ipa-ca-renew-agent
          issuer: CN=Certificate Authority,O=CAMHRES.CA <http://CAMHRES.CA>
         subject: CN=IPA RA,O=CAMHRES.CA <http://CAMHRES.CA>
         expires: 2017-12-27 14:37:02 UTC
         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
         eku: id-kp-serverAuth,id-kp-clientAuth
         pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
         post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
         track: yes
         auto-renew: yes
Request ID '20170201190117':
         status: MONITORING
         stuck: no
         key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set
         certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB'
         CA: dogtag-ipa-renew-agent
          issuer: CN=Certificate Authority,O=CAMHRES.CA <http://CAMHRES.CA>
         subject: CN=rprshipav01.camhres.ca <http://rprshipav01.camhres.ca>,O=CAMHRES.CA <http://CAMHRES.CA>
         expires: 2019-11-19 19:38:26 UTC
         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
         eku: id-kp-serverAuth,id-kp-clientAuth
         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca"
         track: yes
         auto-renew: yes
Request ID '20170201190118':
         status: MONITORING
         stuck: no
         key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-CAMHRES-CA',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-CAMHRES-CA/pwdfile.txt'
         certificate: type=NSSDB,location='/etc/dirsrv/slapd-CAMHRES-CA',nickname='Server-Cert',token='NSS Certificate DB'
         CA: IPA
          issuer: CN=Certificate Authority,O=CAMHRES.CA <http://CAMHRES.CA>
         subject: CN=rprshipav01.camhres.ca <http://rprshipav01.camhres.ca>,O=CAMHRES.CA <http://CAMHRES.CA>
         expires: 2019-12-11 19:38:29 UTC
          principal name: ldap/rprshipav01.camhres.ca@CAMHRES.CA <mailto:rprshipav01.camhres.ca@CAMHRES.CA>
         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
         eku: id-kp-serverAuth,id-kp-clientAuth
         pre-save command:
         post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv CAMHRES-CA
         track: yes
         auto-renew: yes
Request ID '20170201190119':
         status: MONITORING
         stuck: no
         key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
         certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB'
         CA: IPA
          issuer: CN=Certificate Authority,O=CAMHRES.CA <http://CAMHRES.CA>
         subject: CN=rprshipav01.camhres.ca <http://rprshipav01.camhres.ca>,O=CAMHRES.CA <http://CAMHRES.CA>
         expires: 2019-12-11 19:38:38 UTC
          principal name: HTTP/rprshipav01.camhres.ca@CAMHRES.CA <mailto:rprshipav01.camhres.ca@CAMHRES.CA>
         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
         eku: id-kp-serverAuth,id-kp-clientAuth
         pre-save command:
         post-save command: /usr/libexec/ipa/certmonger/restart_httpd
         track: yes
         auto-renew: yes
-----

Nothing happens from now on and /var/log/ipa/renew.log does not log new message after these:
-----
 2017-12-23T05:55:52Z    5538    MainThread      ipa     DEBUG    Initializing principal host/rprshipav01.camhres.ca@CAMHRES.CA <mailto:rprshipav01.camhres.ca@CAMHRES.CA> using keytab /etc/krb5.keytab
2017-12-23T05:55:52Z    5538    MainThread      ipa     DEBUG   using ccache /var/run/certmonger/tmp-1aYw7c/ccache
2017-12-23T05:55:52Z    5538    MainThread      ipa     DEBUG   Attempt 1/1: success
2017-12-23T05:55:52Z    5538    MainThread      ipa     DEBUG   Loading StateFile from '/var/lib/ipa/sysrestore/sysrestore.state'
2017-12-23T05:55:52Z    5538    MainThread      ipa.ipaserver.plugins.ldap2.ldap2       DEBUG   Created connection context.ldap2_80840016
2017-12-23T05:55:52Z    5538    MainThread      ipa.ipapython.ipaldap.SchemaCache       DEBUG   retrieving schema for SchemaCache url=ldapi://%2fvar%2frun%2fslapd-CAMHRES-CA.socket conn=<ldap.ldapobject.SimpleLDAPObject instance at 0x41b2170>
2017-12-23T05:55:52Z    5538    MainThread      ipa.ipaserver.plugins.ldap2.ldap2       DEBUG   Destroyed connection context.ldap2_80840016
 2017-12-23T05:56:02Z    5543    MainThread      ipa     DEBUG    Initializing principal host/rprshipav01.camhres.ca@CAMHRES.CA <mailto:rprshipav01.camhres.ca@CAMHRES.CA> using keytab /etc/krb5.keytab
2017-12-23T05:56:02Z    5543    MainThread      ipa     DEBUG   using ccache /var/run/certmonger/tmp-VDJjQv/ccache
2017-12-23T05:56:02Z    5543    MainThread      ipa     DEBUG   Attempt 1/1: success
2017-12-23T05:56:02Z    5543    MainThread      ipa     DEBUG   Loading StateFile from '/var/lib/ipa/sysrestore/sysrestore.state'
2017-12-23T05:56:03Z    5543    MainThread      ipa.ipaserver.plugins.ldap2.ldap2       DEBUG   Created connection context.ldap2_77880784
2017-12-23T05:56:03Z    5543    MainThread      ipa.ipapython.ipaldap.SchemaCache       DEBUG   retrieving schema for SchemaCache url=ldapi://%2fvar%2frun%2fslapd-CAMHRES-CA.socket conn=<ldap.ldapobject.SimpleLDAPObject instance at 0x4a46e60>
2017-12-23T05:56:03Z    5543    MainThread      ipa.ipaserver.plugins.ldap2.ldap2       DEBUG   Destroyed connection context.ldap2_77880784
 2017-12-23T05:56:12Z    5548    MainThread      ipa     DEBUG    Initializing principal host/rprshipav01.camhres.ca@CAMHRES.CA <mailto:rprshipav01.camhres.ca@CAMHRES.CA> using keytab /etc/krb5.keytab
2017-12-23T05:56:12Z    5548    MainThread      ipa     DEBUG   using ccache /var/run/certmonger/tmp-BQMLXO/ccache
2017-12-23T05:56:12Z    5548    MainThread      ipa     DEBUG   Attempt 1/1: success
2017-12-23T05:56:12Z    5548    MainThread      ipa     DEBUG   Loading StateFile from '/var/lib/ipa/sysrestore/sysrestore.state'
2017-12-23T05:56:12Z    5548    MainThread      ipa.ipaserver.plugins.ldap2.ldap2       DEBUG   Created connection context.ldap2_82537872
2017-12-23T05:56:12Z    5548    MainThread      ipa.ipapython.ipaldap.SchemaCache       DEBUG   retrieving schema for SchemaCache url=ldapi://%2fvar%2frun%2fslapd-CAMHRES-CA.socket conn=<ldap.ldapobject.SimpleLDAPObject instance at 0x4eba710>
2017-12-23T05:56:13Z    5548    MainThread      ipa.ipaserver.plugins.ldap2.ldap2       DEBUG   Destroyed connection context.ldap2_82537872
 2017-12-23T05:56:22Z    5549    MainThread      ipa     DEBUG    Initializing principal host/rprshipav01.camhres.ca@CAMHRES.CA <mailto:rprshipav01.camhres.ca@CAMHRES.CA> using keytab /etc/krb5.keytab

2017-12-23T05:56:22Z    5549    MainThread      ipa     DEBUG   using ccache /var/run/certmonger/tmp-zvyYAy/ccache
2017-12-23T05:56:22Z    5549    MainThread      ipa     DEBUG   Attempt 1/1: success
2017-12-23T05:56:22Z    5549    MainThread      ipa     DEBUG   Loading StateFile from '/var/lib/ipa/sysrestore/sysrestore.state'
2017-12-23T05:56:22Z    5549    MainThread      ipa.ipaserver.plugins.ldap2.ldap2       DEBUG   Created connection context.ldap2_104689040
2017-12-23T05:56:22Z    5549    MainThread      ipa.ipapython.ipaldap.SchemaCache       DEBUG   retrieving schema for SchemaCache url=ldapi://%2fvar%2frun%2fslapd-CAMHRES-CA.socket conn=<ldap.ldapobject.SimpleLDAPObject instance at 0x63dbea8>
2017-12-23T05:56:23Z    5549    MainThread      ipa.ipaserver.plugins.ldap2.ldap2       DEBUG   Destroyed connection context.ldap2_104689040
-----

/var/log/pki/pki-tomcat/ca/ selftests.log does nt log any errores:
-----
0.localhost-startStop-1 - [23/Dec/2017:00:02:48 EST] [20] [1] SelfTestSubsystem: Initializing self test plugins:
0.localhost-startStop-1 - [23/Dec/2017:00:02:48 EST] [20] [1] SelfTestSubsystem:  loading all self test plugin logger parameters
0.localhost-startStop-1 - [23/Dec/2017:00:02:48 EST] [20] [1] SelfTestSubsystem:  loading all self test plugin instances
0.localhost-startStop-1 - [23/Dec/2017:00:02:48 EST] [20] [1] SelfTestSubsystem:  loading all self test plugin instance parameters
0.localhost-startStop-1 - [23/Dec/2017:00:02:48 EST] [20] [1] SelfTestSubsystem:  loading self test plugins in on-demand order
0.localhost-startStop-1 - [23/Dec/2017:00:02:48 EST] [20] [1] SelfTestSubsystem:  loading self test plugins in startup order
0.localhost-startStop-1 - [23/Dec/2017:00:02:48 EST] [20] [1] SelfTestSubsystem: Self test plugins have been successfully loaded!
0.localhost-startStop-1 - [23/Dec/2017:00:02:48 EST] [20] [1] SelfTestSubsystem: Running self test plugins specified to be executed at startup:
0.localhost-startStop-1 - [23/Dec/2017:00:02:48 EST] [20] [1] CAPresence:  CA is present
0.localhost-startStop-1 - [23/Dec/2017:00:02:48 EST] [20] [1] SystemCertsVerification: system certs verification success
0.localhost-startStop-1 - [23/Dec/2017:00:02:48 EST] [20] [1] SelfTestSubsystem: All CRITICAL self test plugins ran SUCCESSFULLY at startup!
0.localhost-startStop-1 - [23/Dec/2017:00:47:25 EST] [20] [1] SelfTestSubsystem: Initializing self test plugins:
0.localhost-startStop-1 - [23/Dec/2017:00:47:25 EST] [20] [1] SelfTestSubsystem:  loading all self test plugin logger parameters
0.localhost-startStop-1 - [23/Dec/2017:00:47:25 EST] [20] [1] SelfTestSubsystem:  loading all self test plugin instances
0.localhost-startStop-1 - [23/Dec/2017:00:47:25 EST] [20] [1] SelfTestSubsystem:  loading all self test plugin instance parameters
0.localhost-startStop-1 - [23/Dec/2017:00:47:25 EST] [20] [1] SelfTestSubsystem:  loading self test plugins in on-demand order
0.localhost-startStop-1 - [23/Dec/2017:00:47:25 EST] [20] [1] SelfTestSubsystem:  loading self test plugins in startup order
0.localhost-startStop-1 - [23/Dec/2017:00:47:25 EST] [20] [1] SelfTestSubsystem: Self test plugins have been successfully loaded!
0.localhost-startStop-1 - [23/Dec/2017:00:47:26 EST] [20] [1] SelfTestSubsystem: Running self test plugins specified to be executed at startup:
0.localhost-startStop-1 - [23/Dec/2017:00:47:26 EST] [20] [1] CAPresence:  CA is present
0.localhost-startStop-1 - [23/Dec/2017:00:47:26 EST] [20] [1] SystemCertsVerification: system certs verification success
0.localhost-startStop-1 - [23/Dec/2017:00:47:26 EST] [20] [1] SelfTestSubsystem: All CRITICAL self test plugins ran SUCCESSFULLY at startup!
0.localhost-startStop-1 - [23/Dec/2017:00:48:20 EST] [20] [1] SelfTestSubsystem: Initializing self test plugins:
0.localhost-startStop-1 - [23/Dec/2017:00:48:20 EST] [20] [1] SelfTestSubsystem:  loading all self test plugin logger parameters
0.localhost-startStop-1 - [23/Dec/2017:00:48:20 EST] [20] [1] SelfTestSubsystem:  loading all self test plugin instances
0.localhost-startStop-1 - [23/Dec/2017:00:48:20 EST] [20] [1] SelfTestSubsystem:  loading all self test plugin instance parameters
0.localhost-startStop-1 - [23/Dec/2017:00:48:20 EST] [20] [1] SelfTestSubsystem:  loading self test plugins in on-demand order
0.localhost-startStop-1 - [23/Dec/2017:00:48:20 EST] [20] [1] SelfTestSubsystem:  loading self test plugins in startup order
0.localhost-startStop-1 - [23/Dec/2017:00:48:20 EST] [20] [1] SelfTestSubsystem: Self test plugins have been successfully loaded!
0.localhost-startStop-1 - [23/Dec/2017:00:48:20 EST] [20] [1] SelfTestSubsystem: Running self test plugins specified to be executed at startup:
0.localhost-startStop-1 - [23/Dec/2017:00:48:20 EST] [20] [1] CAPresence:  CA is present
0.localhost-startStop-1 - [23/Dec/2017:00:48:20 EST] [20] [1] SystemCertsVerification: system certs verification success
0.localhost-startStop-1 - [23/Dec/2017:00:48:20 EST] [20] [1] SelfTestSubsystem: All CRITICAL self test plugins ran SUCCESSFULLY at startup!
-----

Can someone shed some light on this? I may have missed some logs but can provide them if required.

Many thanks,
Qing





_______________________________________________
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org


Hi,

first of all, can you check if the machine where you are trying to renew the certificates is the renewal master? It can be found using the following command:
$ ipa config-show| grep "IPA CA renewal master"
  IPA CA renewal master: master.ipadomain.com

The procedure that you followed will only work if it is run on the renewal master.

If you have multiple masters, you need to find which one is the renewal master and start repairing this node first.
If you have a single master but it is not the renewal master (for instance because the renewal master was decommissioned), you can make this node the renewal master with the instructions detailed here:
How to promote CA to renewal and CRL master [1]
or there (depending on your version):
6.5.2.1. Changing the Current CA Renewal Master [2]

Once your node is the renewal master, the procedure with going back in time should allow you to renew the ipaCert.
HTH,
Flo

[1] https://www.freeipa.org/page/Howto/Promote_CA_to_Renewal_and_CRL_Master
[2] https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/linux_domain_identity_authentication_and_policy_guide/server-roles#promote-ca-renewal