Broken ipa replica
by Giulio Casella
Hi everyone,
I'm stuck with a broken replica. I had a setup with two ipa server in
replica (ipa-server-4.6.4 on CentOS 7.6), let's say "idc01" and "idc02".
Due to heavy load idc01 crashed many times, and was not working anymore.
So I tried to redo the replica again. At first I tried to
"ipa-replica-manage re-initialize", with no success.
Now I'm trying to redo from scratch the replica setup: on idc02 I
removed the segments (ipa topologysegment-del, for both ca and domain
suffix), on idc01 I removed everything (ipa-server-install --uninstall),
then I joined domain (ipa-client-install), and everything is working so far.
When doing "ipa-replica-install" on idc01 I get:
[...]
[28/41]: setting up initial replication
Starting replication, please wait until this has completed.
Update in progress, 22 seconds elapsed
[ldap://idc02.my.dom.ain:389] reports: Update failed! Status: [Error
(-11) connection error: Unknown connection error (-11) - Total update
aborted]
And on idc02 (the working server), in
/var/log/dirsrv/slapd-MY-DOM-AIN/errors I find lines stating:
[20/Mar/2019:09:28:06.545187923 +0100] - INFO - NSMMReplicationPlugin -
repl5_tot_run - Beginning total update of replica
"agmt="cn=meToidc01.my.dom.ain" (idc01:389)".
[20/Mar/2019:09:28:26.528046160 +0100] - ERR - NSMMReplicationPlugin -
perform_operation - agmt="cn=meToidc01.my.dom.ain" (idc01:389): Failed
to send extended operation: LDAP error -1 (Can't contact LDAP server)
[20/Mar/2019:09:28:26.530763939 +0100] - ERR - NSMMReplicationPlugin -
repl5_tot_log_operation_failure - agmt="cn=meToidc01.my.dom.ain"
(idc01:389): Received error -1 (Can't contact LDAP server): for total
update operation
[20/Mar/2019:09:28:26.532678072 +0100] - ERR - NSMMReplicationPlugin -
release_replica - agmt="cn=meToidc01.my.dom.ain" (idc01:389): Unable to
send endReplication extended operation (Can't contact LDAP server)
[20/Mar/2019:09:28:26.534307539 +0100] - ERR - NSMMReplicationPlugin -
repl5_tot_run - Total update failed for replica
"agmt="cn=meToidc01.my.dom.ain" (idc01:389)", error (-11)
[20/Mar/2019:09:28:26.561763168 +0100] - INFO - NSMMReplicationPlugin -
bind_and_check_pwp - agmt="cn=meToidc01.my.dom.ain" (idc01:389):
Replication bind with GSSAPI auth resumed
[20/Mar/2019:09:28:26.582389258 +0100] - WARN - NSMMReplicationPlugin -
repl5_inc_run - agmt="cn=meToidc01.my.dom.ain" (idc01:389): The remote
replica has a different database generation ID than the local database.
You may have to reinitialize the remote replica, or the local replica.
It seems that idc02 remembers something about the old replica.
Any hint?
Thank you in advance,
Giulio
1 year, 11 months
IPA CA allow CSR SAN names in external domains
by Steve Dainard
Hello
I have a RHEL7 IPA server installed as a subordinate CA. I'd like to be
able to add SAN's for a different dns domain than exists in the IPA realm.
The dns for 'otherdomain.com' is handled by active directory which my IPA
server has a cross-forest trust with.
ie:
host: client1.ipadomain.com
certificate: CN = client1.ipadomain.com, SAN = client1.ipadomain.com,
servicename.otherdomain.com
When I try to submit this CSR with 'ipa-getcert request' the IPA server
denies with: "The service principal for subject alt name
servicename.otherdomain.com in certificate request does not exist"
It seems that the default CAACL enforces a profile named
'caIPAserviceCert', but I'm having some trouble determining what can be
modified (or cloned and changed in a new profile) that would allow the CA
to sign a CSR that contains *.ipadomain.com and *.otherdomain.com in the
SAN.
This is the only section in the profile that contains SAN:
policyset.serverCertSet.12.constraint.class_id=noConstraintImpl
policyset.serverCertSet.12.constraint.name=No Constraint
policyset.serverCertSet.12.default.class_id=commonNameToSANDefaultImpl
policyset.serverCertSet.12.default.name=Copy Common Name to Subject
Alternative Name
Thanks,
Steve
2 years, 1 month
ipa-dnskeysyncd DEBUG messages
by Kees Bakker
Hi,
On the two CentOS 8 Stream masters (upgraded a few days ago) we now get quite
a few DEBUG messages. I haven't seen these before.
There is also a WARN - content-sync-plugin.
Is this something to be worried about?
Jul 13 14:06:56 linge.example.com ipa-dnskeysyncd[283246]: ipaserver.dnssec.syncrepl: DEBUG Detected modify of entry: idnsname=example.com.,cn=dns,dc=example,dc=com 1e89eb86-e201-11e8-8820-f96efc0c60a4
Jul 13 14:06:56 linge.example.com ipa-dnskeysyncd[283246]: ipaserver.dnssec.syncrepl: DEBUG New cookie is: linge.example.com:389#krbprincipalname=ipa-dnskeysyncd/linge.example.com@example.com,cn=services,cn=accounts,dc=example,dc=com:cn=dns,dc=example,dc=com:(|(objectClass=idnsZone)(objectClass=idnsSecKey)(objectClass=ipk11PublicKey))#206227
Jul 13 14:06:56 linge.example.com named-pkcs11[283005]: zone example.com/IN: sending notifies (serial 1626178016)
Jul 13 14:06:56 linge.example.com named-pkcs11[283005]: client @0x7f54e416c880 172.16.16.31#45677: received notify for zone 'example.com'
Jul 13 14:06:56 linge.example.com ipa-dnskeysyncd[283246]: ipaserver.dnssec.syncrepl: DEBUG Detected modify of entry: idnsname=example.com.,cn=dns,dc=example,dc=com 1e89eb86-e201-11e8-8820-f96efc0c60a4
Jul 13 14:06:56 linge.example.com ipa-dnskeysyncd[283246]: ipaserver.dnssec.syncrepl: DEBUG New cookie is: linge.example.com:389#krbprincipalname=ipa-dnskeysyncd/linge.example.com@example.com,cn=services,cn=accounts,dc=example,dc=com:cn=dns,dc=example,dc=com:(|(objectClass=idnsZone)(objectClass=idnsSecKey)(objectClass=ipk11PublicKey))#206230
Jul 13 14:06:56 linge.example.com ns-slapd[282944]: [13/Jul/2021:14:06:56.745067868 +0200] - WARN - content-sync-plugin - sync_update_persist_betxn_pre_op - DB retried operation targets "changenumber=206231,cn=changelog" (op=0x7fd372024000 idx_pl=1) => op not changed in PL
Jul 13 14:06:56 linge.example.com ipa-dnskeysyncd[283246]: ipaserver.dnssec.syncrepl: DEBUG Detected modify of entry: idnsname=example.com.,cn=dns,dc=example,dc=com 1e89eb86-e201-11e8-8820-f96efc0c60a4
Jul 13 14:06:56 linge.example.com ipa-dnskeysyncd[283246]: ipaserver.dnssec.syncrepl: DEBUG New cookie is: linge.example.com:389#krbprincipalname=ipa-dnskeysyncd/linge.example.com@example.com,cn=services,cn=accounts,dc=example,dc=com:cn=dns,dc=example,dc=com:(|(objectClass=idnsZone)(objectClass=idnsSecKey)(objectClass=ipk11PublicKey))#206232
Jul 13 14:06:56 linge.example.com named-pkcs11[283005]: client @0x7f54e416c880 172.16.16.75#48866: received notify for zone '30.16.172.in-addr.arpa'
Jul 13 14:06:57 linge.example.com ipa-dnskeysyncd[283246]: ipaserver.dnssec.syncrepl: DEBUG Detected modify of entry: idnsname=example.com.,cn=dns,dc=example,dc=com 1e89eb86-e201-11e8-8820-f96efc0c60a4
Jul 13 14:06:57 linge.example.com ipa-dnskeysyncd[283246]: ipaserver.dnssec.syncrepl: DEBUG New cookie is: linge.example.com:389#krbprincipalname=ipa-dnskeysyncd/linge.example.com@example.com,cn=services,cn=accounts,dc=example,dc=com:cn=dns,dc=example,dc=com:(|(objectClass=idnsZone)(objectClass=idnsSecKey)(objectClass=ipk11PublicKey))#206235
Jul 13 14:06:57 linge.example.com ipa-dnskeysyncd[283246]: ipaserver.dnssec.syncrepl: DEBUG Detected modify of entry: idnsname=30.16.172.in-addr.arpa.,cn=dns,dc=example,dc=com d79d0401-e29b-11e8-8820-f96efc0c60a4
Jul 13 14:06:57 linge.example.com ipa-dnskeysyncd[283246]: ipaserver.dnssec.syncrepl: DEBUG New cookie is: linge.example.com:389#krbprincipalname=ipa-dnskeysyncd/linge.example.com@example.com,cn=services,cn=accounts,dc=example,dc=com:cn=dns,dc=example,dc=com:(|(objectClass=idnsZone)(objectClass=idnsSecKey)(objectClass=ipk11PublicKey))#206236
Jul 13 14:06:57 linge.example.com ipa-dnskeysyncd[283246]: ipaserver.dnssec.syncrepl: DEBUG Detected modify of entry: idnsname=30.16.172.in-addr.arpa.,cn=dns,dc=example,dc=com d79d0401-e29b-11e8-8820-f96efc0c60a4
Jul 13 14:06:57 linge.example.com ipa-dnskeysyncd[283246]: ipaserver.dnssec.syncrepl: DEBUG New cookie is: linge.example.com:389#krbprincipalname=ipa-dnskeysyncd/linge.example.com@example.com,cn=services,cn=accounts,dc=example,dc=com:cn=dns,dc=example,dc=com:(|(objectClass=idnsZone)(objectClass=idnsSecKey)(objectClass=ipk11PublicKey))#206237
--
Kees
2 years, 1 month
freeipa with sudo and 2FA (OTP)
by John Ratliff
I'm trying to setup freeipa with OTP. I created a TOTP under my user in
freeipa and updated my user to use 2FA (password + OTP).
When I try to do sudo, it only asks for my password and it fails every
time (presumably because it isn't getting the OTP first).
I didn't see anything useful in the sss_sudo logs, even after adding
debug_level = 6 in the config.
What can I do to further troubleshoot this?
Thanks.
2 years, 2 months
IPA broken after dnf update on CentOS 8
by Vinícius Ferrão
Hello, I’ve a single IPA machine that provides authentication for itself. It does not even have any client or host.
After def -y update and reboot, IPA fails to load an it’s in broken state.
[root@headnode ~]# systemctl status ipa
● ipa.service - Identity, Policy, Audit
Loaded: loaded (/usr/lib/systemd/system/ipa.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Wed 2021-01-06 16:14:48 -03; 45min ago
Process: 1278 ExecStart=/usr/sbin/ipactl start (code=exited, status=1/FAILURE)
Main PID: 1278 (code=exited, status=1/FAILURE)
Jan 06 16:14:48 headnode.cluster.tmc.if.ufrj.br ipactl[1278]: CRL tree already moved
Jan 06 16:14:48 headnode.cluster.tmc.if.ufrj.br ipactl[1278]: IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command i>
Jan 06 16:14:48 headnode.cluster.tmc.if.ufrj.br ipactl[1278]: Unexpected error - see /var/log/ipaupgrade.log for details:
Jan 06 16:14:48 headnode.cluster.tmc.if.ufrj.br ipactl[1278]: CalledProcessError: CalledProcessError(Command ['/bin/systemctl', 'start', '>
Jan 06 16:14:48 headnode.cluster.tmc.if.ufrj.br ipactl[1278]: The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log for more >
Jan 06 16:14:48 headnode.cluster.tmc.if.ufrj.br ipactl[1278]: See the upgrade log for more details and/or run /usr/sbin/ipa-server-upgrade>
Jan 06 16:14:48 headnode.cluster.tmc.if.ufrj.br ipactl[1278]: Aborting ipactl
Jan 06 16:14:48 headnode.cluster.tmc.if.ufrj.br systemd[1]: ipa.service: Main process exited, code=exited, status=1/FAILURE
Jan 06 16:14:48 headnode.cluster.tmc.if.ufrj.br systemd[1]: ipa.service: Failed with result 'exit-code'.
Jan 06 16:14:48 headnode.cluster.tmc.if.ufrj.br systemd[1]: Failed to start Identity, Policy, Audit.
If asks for look on /var/log/ipaupgrade.log; but this log is just overwhelming. You must know what you should be looking for for actually find something.
The relevant thing that I’ve found by myself is:
2021-01-06T19:09:51Z DEBUG The ipa-server-upgrade command failed, exception: CalledProcessError: CalledProcessError(Command ['/bin/systemctl', 'start', 'pki-tomcatd(a)pki-tomcat.service<mailto:pki-tomcatd@pki-tomcat.service>'] returned non-zero exit status 1: 'Job for pki-tomcatd(a)pki-tomcat.service<mailto:pki-tomcatd@pki-tomcat.service> failed because a timeout was exceeded.\nSee "systemctl status pki-tomcatd(a)pki-tomcat.service<mailto:pki-tomcatd@pki-tomcat.service>" and "journalctl -xe" for details.\n’)
Is that Java regression again that happened a month or two ago?
Thank you all.
2 years, 3 months
Unable to communicate with CMS (403)
by lejeczek
Hi guys.
I get:
-> $ ipa host-del c8kubernode1.private.lot
ipa: ERROR: Certificate operation cannot be completed:
Unable to communicate with CMS (403)
-> $ ipa cert-show 1
ipa: ERROR: Certificate operation cannot be completed:
Request failed with status 403: Non-2xx response from CA
REST API: 403. (403)
I searched mailing list and what I found about certs being
out or in sync I checked, I verified but it's still possible
I missed something there.
I also see this: https://access.redhat.com/solutions/3624671
- which I thought was a bit dated issue thus I want to ask:
Should that be in ipa-server-4.9.6-4 ? because my
'/etc/httpd/conf.d/ipa-pki-proxy.conf' indeed lacks
"^/ca/rest/account/login...
many thanks, L
2 years, 4 months
RA Agent certificate authorisation fails – how to debug?
by Tomasz Torcz
Hi,
I've encountered some authentication problems with my FreeIPA
installation, which I've traced to RA Agent certification auth problems.
I've done typical steps to verify certs in LDAP and hit a wall.
Please suggest further steps.
My setup is 2 masters on Fedora 34:
freeipa-server-4.9.6-2.fc34.x86_64
pki-base-10.10.6-1.fc34.noarch
Steps I've done:
First I noticed problems when enabling ACME:
$ ipa-acme-manage enable
Failed to authenticate to CA REST API
The ipa-acme-manage command failed.
This is caused by authentication failure (401 return code), which
I confirmed using curl:
$ curl -X POST https://kaitain.pipebreaker.pl:8443/acme/enable --cert /var/lib/ipa/ra-agent.pem --key /var/lib/ipa/ra-agent.key
<!doctype html><html lang="en"><head><title>HTTP Status 401 – Unauthorized</title><style type="text/css">body
{font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2
{font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line
{height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP
Status 401 – Unauthorized</h1><hr class="line" /><p><b>Type</b> Status Report</p><p><b>Description</b> The request has not been applied because
it lacks valid authentication credentials for the target resource.</p><hr class="line" /><h3>Apache Tomcat/9.0.52</h3></body></html>
Errors from catalina.out:
02-Oct-2021 16:29:39.618 INFO [https-jsse-nio-8443-exec-6] com.netscape.cms.tomcat.ExternalAuthenticationValve.invoke ExternalAuthenticationValve: authType: null
02-Oct-2021 16:29:39.618 INFO [https-jsse-nio-8443-exec-6] com.netscape.cms.tomcat.ExternalAuthenticationValve.invoke ExternalAuthenticationValve: principal: null
02-Oct-2021 16:29:39.620 INFO [https-jsse-nio-8443-exec-6] com.netscape.cms.tomcat.AbstractPKIAuthenticator.doAuthenticate PKIAuthenticator: Authenticate with client certificate authentication
02-Oct-2021 16:29:39.620 INFO [https-jsse-nio-8443-exec-6] com.netscape.cms.tomcat.ProxyRealm.authenticate Authenticating certificate chain:
02-Oct-2021 16:29:39.621 INFO [https-jsse-nio-8443-exec-6] com.netscape.cms.tomcat.ProxyRealm.authenticate - CN=IPA RA,O=PIPEBREAKER.PL
02-Oct-2021 16:29:39.621 INFO [https-jsse-nio-8443-exec-6] com.netscape.cms.tomcat.ProxyRealm.authenticate - CN=Certificate Authority,O=PIPEBREAKER.PL
02-Oct-2021 16:29:39.624 INFO [https-jsse-nio-8443-exec-6] com.netscape.cms.tomcat.AbstractPKIAuthenticator.doAuthenticate PKIAuthenticator: Result: false
I've made sure that /var/lib/ipa/ra-agent.pem is the same as in LDAP.
$ openssl x509 -in /var/lib/ipa/ra-agent.pem -noout -text | grep Serial
Serial Number: 105 (0x69)
$ ldapsearch -o ldif_wrap=no -D "cn=directory manager" -W -b o=ipaca "(uid=ipara)"
Enter LDAP Password:
# extended LDIF
#
# LDAPv3
# base <o=ipaca> with scope subtree
# filter: (uid=ipara)
# requesting: ALL
#
# ipara, people, ipaca
dn: uid=ipara,ou=people,o=ipaca
description: 2;105;CN=Certificate Authority,O=PIPEBREAKER.PL;CN=IPA RA,O=PIPEBREAKER.PL
userCertificate;binary:: <SNIPPED>
cn: ipara
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
objectClass: cmsuser
usertype: agentType
sn: ipara
uid: ipara
userstate: 1
# search result
search: 2
result: 0 Success
# numResponses: 2
# numEntries: 1
Then SNIPPED portion is the same data as in /var/lib/ipa/ra-agent.pem.
This is the same certificate; serial number matches, too.
Certificate is NOT expired:
$ openssl x509 -in /var/lib/ipa/ra-agent.pem -noout -dates
notBefore=Jun 16 04:34:42 2021 GMT
notAfter=Jun 6 04:34:42 2023 GMT
What should I do next to resolve this authentication issue?
--
Tomasz Torcz Morality must always be based on practicality.
tomek(a)pipebreaker.pl — Baron Vladimir Harkonnen
2 years, 5 months
replica fails - Dogtag CA is not installed
by lejeczek
Hi guys.
I'm trying to add replica but process bellies up early with:
-> $ ipa-replica-install --setup-dns --setup-kra --no-forwarders
Lookup failed: Preferred host c8kubermaster1.private.lot
does not provide DNS.
Reverse DNS resolution of address 10.3.1.222
(c8kubermaster2.private.lot) failed. Clients may not
function properly. Please check your DNS setup. (Note that
this check queries IPA DNS directly and ignores /etc/hosts.)
Continue? [no]: yes
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.
Dogtag CA is not installed. Please install the CA first
The ipa-replica-install command failed. See
/var/log/ipareplica-install.log for more information
..
First errors in log I spot:
...
2021-10-27T23:27:06Z DEBUG Starting external process
2021-10-27T23:27:06Z DEBUG args=['pki-server',
'subsystem-show', 'kra']
2021-10-27T23:27:06Z DEBUG Process finished, return code=1
2021-10-27T23:27:06Z DEBUG stdout=
What is the culprit here?
many thanks, L
2 years, 5 months
pam_sss account Access denied (permission denied)
by Mark Johnson
I've been struggling with this all day and I'm getting nowhere. We're wanting to migrate from a 389-DS authenticated network to FreeIPA. We have a few Linux servers scattered around the world that authenticate against our current 389 directory and we're wanting to do this with minimal changes to these servers. The thought process is to perform LDAP auth against FreeIPA and filter access permissions by way of an LDAP access filter based on group membership as we are currently doing with 389, so we just need to make config changes to sssd to point to the new servers (and install the required certificate to do so).
FreeIPA servers are already setup and replicating. Set up a couple of test groups and a handful of test user accounts. I can successfully authenticate these users, but I get a permission denied seemingly at the access filter stage.
Oct 27 04:15:09 autugd6998 sshd[9984]: pam_sss(sshd:auth): authentication success; logname= uid=0 euid=0 tty=ssh ruser= rhost=10.66.67.69 user=markj
Oct 27 04:15:09 autugd6998 sshd[9984]: pam_sss(sshd:account): Access denied for user markj: 6 (Permission denied)
Same result for a console login. To test this, I changed the access_provider to 'permit' and I can successfully log in to the server. So, it's as if I'm having issues with my access filter, but everything I've tried is giving me the same result. I've used these same filters in ldapsearch tests and they seem to work fine. For instance, I've created a group called "serveradmins" and placed a couple of users in that group. My sssd.conf ldap_access_filter looks like this:
access_provider = ldap
ldap_access_filter = memberOf=cn=serveradmins,cn=groups,cn=accounts,dc=ipa,dc=domain,dc=com
But that just isn't working. However, if I issue the following, I can see the group members:
$ ldapsearch -x -W -LLL -H ldap://ussv4p6004.ipa.domain.com -b cn=serveradmins,cn=groups,cn=accounts,dc=ipa,dc=domain,dc=com -D "uid=markj,cn=users,cn=accounts,dc=ipa,dc=domain,dc=com"
Enter LDAP Password:
dn: cn=serveradmins,cn=groups,cn=accounts,dc=ipa,dc=domain,dc=com
member: uid=mark,cn=users,cn=accounts,dc=ipa,dc=domain,dc=com
member: uid=markj,cn=users,cn=accounts,dc=ipa,dc=domain,dc=com
cn: serveradmins
objectClass: top
objectClass: groupofnames
objectClass: nestedgroup
objectClass: ipausergroup
objectClass: ipaobject
ipaUniqueID: 2e489422-36c2-11ec-a8a8-52540031af07
I've tried different groups including the default 'ipausers' group which everyone is a member of but I'm getting nowhere.
For the record, here's a snippet from the server audit.log when I fail to login. Not sure if that "PAM:accounting grantors=?" bit where the USER_ACCT fails is indicative of the problem or not but if so, I'm not sure what that means and how to resolve it. However, the same server works on the old 389 directory using LDAP auth - just have no idea what I'm missing.
type=USER_AUTH msg=audit(1635321247.300:1039): pid=10122 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=pubkey acct="markj" exe="/usr/sbin/sshd" hostname=? addr=10.66.67.69 terminal=ssh res=failed'
type=USER_AUTH msg=audit(1635321252.664:1040): pid=10122 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=PAM:authentication grantors=pam_succeed_if,pam_succeed_if,pam_sss acct="markj" exe="/usr/sbin/sshd" hostname=10.66.67.69 addr=10.66.67.69 terminal=ssh res=success'
type=USER_ACCT msg=audit(1635321252.855:1041): pid=10122 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=PAM:accounting grantors=? acct="markj" exe="/usr/sbin/sshd" hostname=10.66.67.69 addr=10.66.67.69 terminal=ssh res=failed'
type=USER_AUTH msg=audit(1635321252.856:1042): pid=10122 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=password acct="markj" exe="/usr/sbin/sshd" hostname=? addr=10.66.67.69 terminal=ssh res=failed'
2 years, 6 months
Group not in compat subtree
by Cyrus
Good morning,
I've seen some of the groups present in the compat subgroup, but not
all of them. Can anybody provide a hint of what's needed for a group
to be present in that subtree?.
All of them are internal FreeIPA groups.
Regards,
CI.-
2 years, 6 months
