Replica not renewing IPA certificates
by Roderick Johnstone
Hi
This is freeipa (ipa-server-4.6.5-11.el7_7.3.x86_64) on RHEL7 with
freeipa's own internal CA.
One of my ipa server replicas (host3) has not renewed its IPA system
certificates and is now showing
ca-error: Invalid cookie: u''
in the 'getcert list' output for certificates:
"auditSigningCert cert-pki-ca", "ocspSigningCert cert-pki-ca",
"subsystemCert cert-pki-ca", and the
certificate in the file /var/lib/ipa/ra-agent.pem
As far as I can see, the sequence of events has been as follows:
host3 noticed the certificates needed renewing at 30 Jan 2020 05:37 and
certmonger initiated a renewal.
The state of those certificates went from MONITORING to CA_WORKING but
the certificates were not renewed.
The CA renewal master (host1) noticed its same set of certificates (plus
"Server-Cert cert-pki-ca") needed renewing at 30 Jan 2020 07:28 and
renewed them successfully.
Another replica (host2) noticed that its certificates needed renewing at
30 Jan 2020 07:32 and renewed them successfully.
At 30 Jan 13:37 on host3 the certificates needing to be renewed went
from CA_WORKING back to MONITORING, but 'getcert list' now shows them with:
ca-error: Invalid cookie: u''
and they still haven't renewed.
I haven't seen certmonger attempt to try the renewal again on host3
(nothing from certmonger in /var/log/messages since 30 Jan 13:37).
While I could try a getcert resubmit on host3 to force it to try again,
I'd like to know if what I am seeing is the expected behaviour when a
replica tried to renew certificates before the renewal master.
How long should I have to wait till certmonger on host3 tries again? - I
couldn't find any reference to how often certmonger tries the renewal.
Rob Crittenden's freeipa-healthcheck script is now showing the following
for host3:
ERROR: ipahealthcheck.ipa.certs.IPARAAgent: RA agent description does
not match 2;16;CN=Certificate Authority,O=EXAMPLE.COM;CN=IPA
RA,O=EXAMPLE.COM in LDAP and 2;7;CN=Certificate
Authority,O=EXAMPLE.COM;CN=IPA RA,O=EXAMPLE.COM expected
ERROR: ipahealthcheck.ipa.certs.IPACertRevocation.20180926040924:
Request for certificate failed, Certificate operation cannot be
completed: EXCEPTION (Invalid Credential.)
ERROR: ipahealthcheck.ipa.certs.IPACertRevocation.20180926040920:
Request for certificate failed, Certificate operation cannot be
completed: EXCEPTION (Invalid Credential.)
ERROR: ipahealthcheck.ipa.certs.IPACertRevocation.20180926040921:
Request for certificate failed, Certificate operation cannot be
completed: EXCEPTION (Invalid Credential.)
ERROR: ipahealthcheck.ipa.certs.IPACertRevocation.20180926040922:
Request for certificate failed, Certificate operation cannot be
completed: EXCEPTION (Invalid Credential.)
ERROR: ipahealthcheck.ipa.certs.IPACertRevocation.20180926040923:
Request for certificate failed, Certificate operation cannot be
completed: EXCEPTION (Invalid Credential.)
ERROR: ipahealthcheck.ipa.certs.IPACertRevocation.20180926040925:
Request for certificate failed, Certificate operation cannot be
completed: EXCEPTION (Invalid Credential.)
ERROR: ipahealthcheck.ipa.certs.IPACertRevocation.20180926040927:
Request for certificate failed, Certificate operation cannot be
completed: EXCEPTION (Invalid Credential.)
ERROR: ipahealthcheck.ipa.certs.IPACertRevocation.20180926040926:
Request for certificate failed, Certificate operation cannot be
completed: EXCEPTION (Invalid Credential.)
ERROR: ipahealthcheck.ipa.certs.IPACertRevocation.20180831064406:
Request for certificate failed, Certificate operation cannot be
completed: EXCEPTION (Invalid Credential.)
ERROR: ipahealthcheck.dogtag.ca.DogtagCertsConnectivityCheck: Request
for certificate failed, Certificate operation cannot be completed:
EXCEPTION (Invalid Credential.)
Each of host1, host2 and host3 are showing serial number 16 in ldap using:
ldapsearch -D "cn=directory manager" -W -b uid=ipara,ou=people,o=ipaca
description
At this stage I'm not sure whether this will resolve itself when
certmonger tries to renew certificates again or whether I need to be
more proactive.
I'm happy to supply more logs as necessary.
Thanks
Roderick
3 years, 6 months
Replication failure during replica setup
by William Muriithi
Evening,
I am attempting to setup a new replica this afternoon and it failed with an
error message that I haven't been able to decipher. Really haven't been
able to get past it as I can't figure out what really tripped the setup?
Have someone seen this in their logs and how did you go about fixing it?
The complete logs are on
https://pastebin.pl/view/85208dbb
2020-09-28T20:12:34Z DEBUG Successfully updated nsDS5ReplicaId.
2020-09-28T20:12:34Z DEBUG Add or update replica config
cn=replica,cn=dc\=external\,dc\=example\,dc\=com,cn=mapping tree,cn=config
2020-09-28T20:12:34Z DEBUG Added replica config
cn=replica,cn=dc\=external\,dc\=example\,dc\=com,cn=mapping tree,cn=config
2020-09-28T20:12:34Z DEBUG Add or update replica config
cn=replica,cn=dc\=external\,dc\=example\,dc\=com,cn=mapping tree,cn=config
2020-09-28T20:12:34Z DEBUG No update to
cn=replica,cn=dc\=external\,dc\=example\,dc\=com,cn=mapping tree,cn=config
necessary
2020-09-28T20:12:34Z DEBUG Waiting up to 300 seconds for replication
(ldapi://%2Fvar%2Frun%2Fslapd-EXTERNAL-EXAMPLE-COM.socket) cn=
meToneptune.external.example.com,cn=replica,cn=dc\=external\,dc\=example\,dc\=com,cn=mapping
tree,cn=config (objectclass=*)
2020-09-28T20:12:34Z DEBUG Entry found [LDAPEntry(ipapython.dn.DN('cn=
meToneptune.external.example.com,cn=replica,cn=dc\=external\,dc\=example\,dc\=com,cn=mapping
tree,cn=config'), {'objectClass': [b'nsds5replicationagreement', b'top'],
'cn': [b'meToneptune.external.example.com'], 'nsDS5ReplicaHost': [b'
neptune.external.example.com'], 'nsDS5ReplicaPort': [b'389'],
'nsds5replicaTimeout': [b'120'], 'nsDS5ReplicaRoot':
[b'dc=external,dc=example,dc=com'], 'description': [b'me to
neptune.external.example.com'], 'nsDS5ReplicatedAttributeList':
[b'(objectclass=*) $ EXCLUDE memberof idnssoaserial entryusn
krblastsuccessfulauth krblastfailedauth krbloginfailedcount'],
'nsDS5ReplicaTransportInfo': [b'LDAP'], 'nsDS5ReplicaBindMethod':
[b'SASL/GSSAPI'], 'nsds5ReplicaStripAttrs': [b'modifiersName
modifyTimestamp internalModifiersName internalModifyTimestamp'],
'nsDS5ReplicatedAttributeListTotal': [b'(objectclass=*) $ EXCLUDE entryusn
krblastsuccessfulauth krblastfailedauth krbloginfailedcount'],
'nsds5replicareapactive': [b'0'], 'nsds5replicaLastUpdateStart':
[b'19700101000000Z'], 'nsds5replicaLastUpdateEnd': [b'19700101000000Z'],
'nsds5replicaChangesSentSinceStartup': [b''],
'nsds5replicaLastUpdateStatus': [b'Error (0) No replication sessions
started since server startup'], 'nsds5replicaLastUpdateStatusJSON':
[b'{"state": "green", "ldap_rc": "0", "ldap_rc_text": "success", "repl_rc":
"0", "repl_rc_text": "replica acquired", "date": "2020-09-28T20:12:34Z",
"message": "Error (0) No replication sessions started since server
startup"}'], 'nsds5replicaUpdateInProgress': [b'FALSE'],
'nsds5replicaLastInitStart': [b'19700101000000Z'],
'nsds5replicaLastInitEnd': [b'19700101000000Z']})]
2020-09-28T20:12:50Z DEBUG Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/ipaserver/install/service.py",
line 603, in start_creation
run_step(full_msg, method)
File "/usr/lib/python3.6/site-packages/ipaserver/install/service.py",
line 589, in run_step
method()
File "/usr/lib/python3.6/site-packages/ipaserver/install/dsinstance.py",
line 427, in __setup_replica
cacert=self.ca_file
File "/usr/lib/python3.6/site-packages/ipaserver/install/replication.py",
line 1861, in setup_promote_replication
raise RuntimeError("Failed to start replication")
RuntimeError: Failed to start replication
Regards,
William
3 years, 6 months
User account lock
by Pierre Labanowski
Hello,
Regularly, i have user accounts that are locked by the policy password.
following error message: "Client's credentials have been revoked"
It's possible to unlock the account when this happens. However, this
often happens when a third-party application misconfigured that made too
many wrong requests.
the account is locked sometimes 1 hour after it is unlocked ...
it is difficult to find the source of these connection errors. I don't
know which log file i can check to identify the source of the errors,
which could make it easier to understand the source of the problem.
Do you have a pointer on how to monitor and control in the log files the
various errors that lead to the locking of a user account ? thx
Regards,
Pierre
3 years, 6 months
IPA CA request ID reuse issue
by Boris Sukhinin
I try to add a new replica to a cluster of 3 freeipa servers. ipa-replica-install --setup-ca fails with an error:
[5/28]: configuring certificate server instance
ipaserver.install.dogtaginstance: CRITICAL Failed to configure CA instance: Command '/usr/sbin/pkispawn -s CA -f /tmp/tmp_3KfOZ' returned non-zero exit status 1
ipaserver.install.dogtaginstance: CRITICAL See the installation logs and the following files/directories for more information:
ipaserver.install.dogtaginstance: CRITICAL /var/log/pki/pki-tomcat
[error] RuntimeError: CA configuration failed.
/var/log/pki/pki-tomcat/ca/debug gives more info:
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: === Processing ocsp_signing cert ===
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: === Processing sslserver cert ===
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: SystemConfigService.processKeyPair(sslserver)
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: SystemConfigService: san_server_cert not found
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: SystemConfigService: loading existing key pair from NSS database
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: ConfigurationUtils: loadKeyPair(Server-Cert cert-pki-ca, )
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: SystemConfigService: storing key pair into CS.cfg
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: ConfigurationUtils: storeKeyPair(sslserver)
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: SystemConfigService.processCert(sslserver)
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: SystemConfigService: checking sslserver cert in NSS database
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: configCert: caType is local
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: configCert: caType is remote (revised)
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: ConfigurationUtils: updateConfig() for certTag sslserver
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: updateConfig() done
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: configCert: remote CA
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: confgCert: tag: sslserver
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: CertRequestPanel: got public key
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: CertRequestPanel: got private key
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: ConfigurationUtils: For this Cloned CA, always use its Master CA to generate the 'sslserver' certificate to avoid any changes which may have been made to the X500Name directory string encoding order.
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: ConfigurationUtils: injectSAN: false
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: configRemoteCert: tag: sslserver : setting profileId to: caInternalAuthServerCert
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: configRemoteCert: tag: sslserver calculated profileId: caInternalAuthServerCert
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: CertUtil: content: {xmlOutput=[true], cert_request_type=[pkcs10], profileId=[caInternalAuthServerCert], cert_request=[...cut...], requestor_name=[CA-srvXXX.local.domain-8443], sessionID=[6184760106499759096]}
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: ConfigurationUtils: POST https://srvYYY.local.domain:443/ca/ee/ca/profileSubmit
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: CertUtil: status: 1
[15/Sep/2020:15:42:06][http-bio-8443-exec-3]: CertUtil: error: Request 19990005 - Server Internal Error
java.io.IOException: Request 19990005 - Server Internal Error
at com.netscape.cms.servlet.csadmin.CertUtil.createRemoteCert(CertUtil.java:103)
at com.netscape.cms.servlet.csadmin.ConfigurationUtils.configRemoteCert(ConfigurationUtils.java:2737)
at com.netscape.cms.servlet.csadmin.ConfigurationUtils.configCert(ConfigurationUtils.java:2593)
at org.dogtagpki.server.rest.SystemConfigService.processCert(SystemConfigService.java:484)
at org.dogtagpki.server.rest.SystemConfigService.processCerts(SystemConfigService.java:303)
at org.dogtagpki.server.rest.SystemConfigService.configure(SystemConfigService.java:166)
at org.dogtagpki.server.rest.SystemConfigService.configure(SystemConfigService.java:101)
<...long stacktrace...>
LDAP access logs on existing server show that certificate request ID was reused. ADD queries for request as well as for certificate have failed, but nevertheless the request was overwritten with a third query:
[15/Sep/2020:15:42:06.259655845 +0300] conn=2608 op=1804 ADD dn="cn=19990005,ou=ca,ou=requests,o=ipaca"
[15/Sep/2020:15:42:06.260525381 +0300] conn=2608 op=1804 RESULT err=68 tag=105 nentries=0 etime=0.0001251547
[15/Sep/2020:15:42:06.295774388 +0300] conn=2608 op=1805 ADD dn="cn=536805381,ou=certificateRepository,ou=ca,o=ipaca"
[15/Sep/2020:15:42:06.296210847 +0300] conn=2608 op=1805 RESULT err=68 tag=105 nentries=0 etime=0.0000755866
[15/Sep/2020:15:42:06.301651990 +0300] conn=2608 op=1806 MOD dn="cn=19990005,ou=ca,ou=requests,o=ipaca"
[15/Sep/2020:15:42:06.315983780 +0300] conn=2608 op=1806 RESULT err=0 tag=103 nentries=0 etime=0.0014827998 csn=5f60b69f000000110000
I can confirm that each server has its own request ID range configured in /etc/pki/pki-tomcat/ca/CS.cfg:
1. dbs.beginRequestNumber=19970001
dbs.endRequestNumber=19980000
2. dbs.beginRequestNumber=19980001
dbs.endRequestNumber=19990000
3. dbs.beginRequestNumber=19990001
dbs.endRequestNumber=20000000
So my questions are:
1. How does PKI track request IDs and prevent their reuse?
2. Does overwriting certificate request in LDAP affect certificate renewal or have any other negative consequences?
3. What's the best way to fix this problem? LDAP backups are available and restoring o=ipaca is an option. Domain database rollback is not an option though.
3 years, 6 months
Anyone integrate FreeIPA 4.8.x with Okta LDAPS service recently?
by Chris Dagdigian
Hi folks,
Has anyone configured the LDAP service of Okta to push users into
FreeIPA recently? Looking for tips/tricks more recent than this page
https://www.freeipa.org/page/HowTo/Integrate_With_Okta which I think
dates back to 2014.
I can get the Okta agent running on the FreeIPA host and talking to Okta
but user provisioning fails with a DN parsing related error that makes
me think that something is now different about (a) telling Okta what
LDAP type/scheme is used on the other end or (b) setting up the
attribute mapping.
This is my Okta ldap agent error when a user is pushed into FreeIPA -- I
100% understand this is an Okta config and Okta agent config thing but
am just wondering if anyone has been down this road recently. If not
I'll try to write up my notes if I can get it working.
This is my error as of now. The RDN value is mapped to Okta 'uid'
attribute which always resolves to an email address like DN. I'm going
to blow everything away and restart fresh as I changed too many things
while debugging the current config:
[ 2020-09-25 21:39:14.859 ] [ Thread-15 ] [ INFO ] [LdapRestClient:478]
- GET
https://XXXX.okta.com/api/1/internal/app/agent/ldap_sun_one/0oa5o6gyetYbG...
[ 2020-09-25 21:39:14.859 ] [ pool-2-thread-3 ] [ ERROR ]
[UnboundIDLdapClient:531] - Error during ModifyRequest. ResultCode=34
(invalid DN syntax) exception=
com.unboundid.ldap.sdk.LDAPException: Unable to parse string
'dag(a)XXX.net' as a DN because it does not have an equal sign after RDN
attribute 'dag(a)XXX.net'.
at com.unboundid.ldap.sdk.DN.<init>(DN.java:434)
at com.unboundid.ldap.sdk.DN.<init>(DN.java:300)
at com.unboundid.ldap.sdk.DN.getParentString(DN.java:1055)
at
com.okta.ldap_agent.client.unboundid.UnboundIDLdapClient.moveEntry(UnboundIDLdapClient.java:902)
at
com.okta.ldap_agent.client.unboundid.UnboundIDLdapClient.modifyEntry(UnboundIDLdapClient.java:483)
at
com.okta.ldap_agent.connectors.ldap.LdapConnectorExecutorImpl.modifyEntry(LdapConnectorExecutorImpl.java:67)
at
com.okta.ldap_agent.adapters.LdapDirectoryAdapter.modifyEntry(LdapDirectoryAdapter.java:175)
at
com.okta.ldap_agent.handlers.WriteObjectActionHandler.performAction(WriteObjectActionHandler.java:43)
at
com.okta.ldap_agent.LdapAgent.lambda$dispatchAction$0(LdapAgent.java:253)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[ 2020-09-25 21:39:14.860 ] [ pool-2-thread-3 ] [ ERROR ]
[WriteObjectActionHandler:65] - Interchange error: 34, Unable to parse
string 'dag(a)XXX.net' as a DN because it does not have an equal sign
after RDN attribute 'dag(a)XXX.net'.
3 years, 6 months
mod_nss fails apache start missing existing certificate.
by Naor Weissmann
Hi guys.
I understand it is not a pure FreeIPA question but it is supporting middleware and im out of ideas.
We have an old ipa-server-selinux-3.0.0 on Centos6.
after restart i cant start http service. error log in debug mode points me to nss.
"[error] Certificate not found: 'Server-Cert'"
certutil on the database returns:
Server-Cert CPu,Cu,u
and checks for validity. selinux is in permissive mode.
if i remove password file i get a password error so mod_nss is going in right direction.
i dont know what else to look for. maybe you have any ideas? thank you in advance.
3 years, 6 months
BadRequest when using freeipa-python
by Ronald Wimmer
Anyone using freeipa-python here? When I try to use
client.host_mod('myserver.mydomain.at', userclass='SomeUserClass')
the user class is set correctly on the host above but I do get an Exception:
File "./modifyHosts.py", line 34, in <module>
client.host_mod('myserver.mydomain.at', userclass='SomeUserClass')
File
"/usr/local/lib/python3.6/site-packages/python_freeipa/client_meta.py",
line 11192, in host_mod
return self._request(method, _args, _params)
File
"/usr/local/lib/python3.6/site-packages/python_freeipa/client.py", line
335, in _request
parse_error(error)
File
"/usr/local/lib/python3.6/site-packages/python_freeipa/exceptions.py",
line 117, in parse_error
raise exception_class(message, code)
python_freeipa.exceptions.BadRequest: no modifications to be performed
What am I doing wrong?
3 years, 6 months
hbactest returns different result in GUI and CLI
by Ben Aveling
We have a user who can login.
hbactest says they shouldn't be allowed to - when it's run from the command line.
When run from the GUI, hbactest says that the user should be allowed to login.
Looking at the rules, there doesn't seem to be any reason why the user shouldn't be allowed to login, so it seems that the GUI is right and the CLI is wrong.
Just wondering if anyone has seen this before, or has any thoughts on whether we should worry about this or not?
Regards, Ben
3 years, 6 months