after patching our centos 7 hosts to the latest version today, one of the two replicas is having trouble.
2019-11-20T18:18:29Z DEBUG stderr=
2019-11-20T18:18:31Z INFO Certmonger certificate renewal configuration already up-to-date
2019-11-20T18:18:31Z INFO [Enable PKIX certificate path discovery and validation]
2019-11-20T18:18:31Z DEBUG Loading StateFile from '/var/lib/ipa/sysupgrade/sysupgrade.state'
2019-11-20T18:18:31Z INFO PKIX already enabled
2019-11-20T18:18:31Z INFO [Authorizing RA Agent to modify profiles]
2019-11-20T18:18:31Z INFO [Authorizing RA Agent to manage lightweight CAs]
2019-11-20T18:18:31Z INFO [Ensuring Lightweight CAs container exists in Dogtag database]
2019-11-20T18:18:31Z DEBUG Created connection context.ldap2_139740162547472
2019-11-20T18:18:31Z DEBUG flushing ldapi://%2fvar%2frun%2fslapd-L-DOMAIN-IT.socket from SchemaCache
2019-11-20T18:18:31Z DEBUG retrieving schema for SchemaCache url=ldapi://%2fvar%2frun%2fslapd-L-DOMAIN-IT.socket conn=<ldap.ldapobject.SimpleLDAPObject instance at 0x7f17cc24b638>
2019-11-20T18:18:31Z DEBUG Destroyed connection context.ldap2_139740162547472
2019-11-20T18:18:31Z INFO [Adding default OCSP URI configuration]
2019-11-20T18:18:31Z INFO [Ensuring CA is using LDAPProfileSubsystem]
2019-11-20T18:18:31Z INFO [Migrating certificate profiles to LDAP]
2019-11-20T18:18:31Z DEBUG Created connection context.ldap2_139740160021648
2019-11-20T18:18:31Z DEBUG flushing ldapi://%2fvar%2frun%2fslapd-L-DOMAIN-IT.socket from SchemaCache
2019-11-20T18:18:31Z DEBUG retrieving schema for SchemaCache url=ldapi://%2fvar%2frun%2fslapd-L-DOMAIN-IT.socket conn=<ldap.ldapobject.SimpleLDAPObject instance at 0x7f17cc289b00>
2019-11-20T18:18:31Z DEBUG Destroyed connection context.ldap2_139740160021648
2019-11-20T18:18:31Z DEBUG request GET
https://kdc2.l.domain.it:8443/ca/rest/account/login2019-11-20T18:18:31Z DEBUG request body ''
2019-11-20T18:18:31Z DEBUG response status 401
2019-11-20T18:18:31Z DEBUG response headers Server: Apache-Coyote/1.1
Cache-Control: private
Expires: Thu, 01 Jan 1970 01:00:00 CET
WWW-Authenticate: Basic realm="Certificate Authority"
Content-Type: text/html;charset=utf-8
Content-Language: en
Content-Length: 951
Date: Wed, 20 Nov 2019 18:18:31 GMT
2019-11-20T18:18:31Z DEBUG response body '<html><head><title>Apache Tomcat/7.0.76 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 401 - </h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u></u></p><p><b>description</b> <u>This request requires HTTP authentication.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/7.0.76</h3></body></html>'
2019-11-20T18:18:31Z ERROR IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
2019-11-20T18:18:31Z DEBUG File "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 178, in execute
return_value = self.run()
File "/usr/lib/python2.7/site-packages/ipaserver/install/ipa_server_upgrade.py", line 54, in run
server.upgrade()
File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py", line 2146, in upgrade
upgrade_configuration()
File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py", line 2018, in upgrade_configuration
ca_enable_ldap_profile_subsystem(ca)
File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py", line 406, in ca_enable_ldap_profile_subsystem
cainstance.migrate_profiles_to_ldap()
File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 2027, in migrate_profiles_to_ldap
_create_dogtag_profile(profile_id, profile_data, overwrite=False)
File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 2033, in _create_dogtag_profile
with api.Backend.ra_certprofile as profile_api:
File "/usr/lib/python2.7/site-packages/ipaserver/plugins/dogtag.py", line 1315, in __enter__
raise errors.RemoteRetrieveError(reason=_('Failed to authenticate to CA REST API'))
2019-11-20T18:18:31Z DEBUG The ipa-server-upgrade command failed, exception: RemoteRetrieveError: Failed to authenticate to CA REST API
2019-11-20T18:18:31Z ERROR Unexpected error - see /var/log/ipaupgrade.log for details:
RemoteRetrieveError: Failed to authenticate to CA REST API