Hi Rob.
Actually nothing that relies on Kerberos Keytabs is working.
Kerberos is working. The kinit was successful.
I can properly issue kinit’s and login, but I can’t use ‘ipa’
commands
for instance. named-pkcs11 is only starting up because I’ve changed the
authentication method on /etc/named.conf:
/* WARNING: This part of the config file is IPA-managed.
* Modifications may break IPA setup or upgrades.
*/
dyndb "ipa" "/usr/lib64/bind/ldap.so" {
uri "ldapi://%2fvar%2frun%2fslapd-CLUSTER-CETENE-GOV-BR.socket";
base "cn=dns, dc=cluster,dc=cetene,dc=gov,dc=br";
server_id "neumann2.cluster.cetene.gov.br
<
http://neumann2.cluster.cetene.gov.br>";
#auth_method "sasl";
#sasl_mech "GSSAPI";
#sasl_user "DNS/neumann2.cluster.cetene.gov.br
<
http://neumann2.cluster.cetene.gov.br>";
/* Desespero */
auth_method "simple";
bind_dn "uid=admin,cn=users,cn=accounts,dc=cluster,dc=cetene,dc=gov,dc=br";
password “REDACTED";
};
/* End of IPA-managed part. */
I’ve done the test that you’ve asked, and was a no go:
[root@neumann2 ~]# kinit -kt /etc/ipa/dnssec/ipa-dnskeysyncd.keytab
ipa-dnskeysyncd/neumann2.cluster.cetene.gov.br
<
http://neumann2.cluster.cetene.gov.br>
[root@neumann2 ~]# klist
Ticket cache: KEYRING:persistent:0:krb_ccache_dngnA1P
Default principal:
ipa-dnskeysyncd/neumann2.cluster.cetene.gov.br(a)CLUSTER.CETENE.GOV.BR
<mailto:ipa-dnskeysyncd/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
Valid starting Expires Service principal
02/12/2021 22:42:03 02/13/2021 22:42:03
krbtgt/CLUSTER.CETENE.GOV.BR(a)CLUSTER.CETENE.GOV.BR
<mailto:krbtgt/CLUSTER.CETENE.GOV.BR@CLUSTER.CETENE.GOV.BR>
[root@neumann2 ~]# ipa user-show admin
ipa: ERROR: Insufficient access: Invalid credentials
[root@neumann2 ~]# ipa -v user-show admin
ipa: INFO: trying
https://neumann2.cluster.cetene.gov.br/ipa/session/json
ipa: INFO: [try 1]: Forwarding 'schema' to json server
'https://neumann2.cluster.cetene.gov.br/ipa/session/json'
ipa: INFO: trying
https://neumann2.cluster.cetene.gov.br/ipa/session/json
ipa: INFO: [try 2]: Forwarding 'schema' to json server
'https://neumann2.cluster.cetene.gov.br/ipa/session/json'
ipa: INFO: trying
https://neumann2.cluster.cetene.gov.br/ipa/session/json
ipa: INFO: [try 3]: Forwarding 'schema' to json server
'https://neumann2.cluster.cetene.gov.br/ipa/session/json'
ipa: INFO: trying
https://neumann2.cluster.cetene.gov.br/ipa/session/json
ipa: INFO: [try 4]: Forwarding 'schema' to json server
'https://neumann2.cluster.cetene.gov.br/ipa/session/json'
ipa: INFO: trying
https://neumann2.cluster.cetene.gov.br/ipa/session/json
ipa: INFO: [try 5]: Forwarding 'schema' to json server
'https://neumann2.cluster.cetene.gov.br/ipa/session/json'
ipa: INFO: trying
https://neumann2.cluster.cetene.gov.br/ipa/session/json
ipa: ERROR: cannot connect to
'https://neumann2.cluster.cetene.gov.br/ipa/session/json': Exceeded
number of tries to forward a request.
I never seen this on FreeIPA.
Subsequent queries of IPA commands just returns the same error:
[root@neumann2 ~]# ipa user-show admin
ipa: ERROR: cannot connect to
'https://neumann2.cluster.cetene.gov.br/ipa/session/json': Exceeded
number of tries to forward a request.
Did you get a HTTP service ticket? (klist)
Check the Apache error log for more details.
rob
Thank you.
> On 12 Feb 2021, at 18:11, Rob Crittenden <rcritten(a)redhat.com
> <mailto:rcritten@redhat.com>> wrote:
>
> Just to confirm, the system is working with the exception of
> ipa-dnskeysyncd.service?
>
> Does this work?
>
> # kinit -kt /etc/ipa/dnssec/ipa-dnskeysyncd.keytab
> ipa-dnskeysyncd/neumann2.cluster.cetene.gov.br
> <
http://neumann2.cluster.cetene.gov.br/>
> # ipa user-show admin
>
> This will get a ticket and then use that ticket.
>
> rob
>
> Vinícius Ferrão via FreeIPA-users wrote:
>> Hello,
>>
>> I still not sure of what is happening but, I got some interesting error
>> message on ipa-healthcheck:
>>
>> [root@neumann2 keytabs]# ipa-healthcheck --failures-only
>> --output-type human
>> CRITICAL: ipahealthcheck.ipa.dna.IPADNARangeCheck: Insufficient access:
>> Invalid credentials
>> ERROR: ipahealthcheck.system.filesystemspace.FileSystemSpaceCheck:
>> /var/lib/ipa/backup/: free space percentage under threshold: 16% < 20%
>> ERROR: ipahealthcheck.system.filesystemspace.FileSystemSpaceCheck: /tmp:
>> free space percentage under threshold: 16% < 20%
>> ERROR: ipahealthcheck.system.filesystemspace.FileSystemSpaceCheck:
>> /var/lib/dirsrv/: free space percentage under threshold: 16% < 20%
>> ERROR: ipahealthcheck.system.filesystemspace.FileSystemSpaceCheck:
>> /var/log/: free space percentage under threshold: 16% < 20%
>> ERROR: ipahealthcheck.system.filesystemspace.FileSystemSpaceCheck:
>> /var/tmp/: free space percentage under threshold: 16% < 20%
>> ERROR: ipahealthcheck.system.filesystemspace.FileSystemSpaceCheck:
>> /var/log/audit/: free space percentage under threshold: 16% < 20%
>>
>> I tried to search for the critical message but nothing comes up. There’s
>> a lot of GSSAPI errors on all logs.
>>
>> I tried to regenerate all keytabs of the system but it was a no go
>> either:
>> # gssproxy
>> ipa-getkeytab -D "cn=directory manager" -w 86dNCxFFCpNMLEf6kr -s
>> neumann2.cluster.cetene.gov.br
>>
<
http://neumann2.cluster.cetene.gov.br/> <http://neumann2.cluster.ce...
>> <
http://neumann2.cluster.cetene.gov.br/>>
>> -p 'HTTP/neumann2.cluster.cetene.gov.br
>> <
http://neumann2.cluster.cetene.gov.br/>
>> <
http://neumann2.cluster.cetene.gov.br
>> <
http://neumann2.cluster.cetene.gov.br/>>' -r -k
>> /var/lib/ipa/gssproxy/http.keytab
>>
>> # Dogtag
>> ipa-getkeytab -D "cn=directory manager" -w 86dNCxFFCpNMLEf6kr -s
>> neumann2.cluster.cetene.gov.br
>>
<
http://neumann2.cluster.cetene.gov.br/> <http://neumann2.cluster.ce...
>> <
http://neumann2.cluster.cetene.gov.br/>>
>> -p 'dogtag/neumann2.cluster.cetene.gov.br
>> <
http://neumann2.cluster.cetene.gov.br/>
>> <
http://neumann2.cluster.cetene.gov.br
>> <
http://neumann2.cluster.cetene.gov.br/>>' -r -k
>> /etc/pki/pki-tomcat/dogtag.keytab
>>
>> # DNSKeySync
>> ipa-getkeytab -D "cn=directory manager" -w 86dNCxFFCpNMLEf6kr -s
>> neumann2.cluster.cetene.gov.br
>>
<
http://neumann2.cluster.cetene.gov.br/> <http://neumann2.cluster.ce...
>> <
http://neumann2.cluster.cetene.gov.br/>>
>> -p 'ipa-dnskeysyncd/neumann2.cluster.cetene.gov.br
>> <
http://neumann2.cluster.cetene.gov.br/>
>> <
http://neumann2.cluster.cetene.gov.br
>> <
http://neumann2.cluster.cetene.gov.br/>>' -r -k
>> /etc/ipa/dnssec/ipa-dnskeysyncd.keytab
>>
>> # Host Keytab
>> ipa-getkeytab -D "cn=directory manager" -w 86dNCxFFCpNMLEf6kr -s
>> neumann2.cluster.cetene.gov.br
>>
<
http://neumann2.cluster.cetene.gov.br/> <http://neumann2.cluster.ce...
>> <
http://neumann2.cluster.cetene.gov.br/>>
>> -p 'host/neumann2.cluster.cetene.gov.br
>> <
http://neumann2.cluster.cetene.gov.br/>
>> <
http://neumann2.cluster.cetene.gov.br
>> <
http://neumann2.cluster.cetene.gov.br/>>' -r -k /etc/krb5.keytab
>>
>> # named
>> ipa-getkeytab -D "cn=directory manager" -w 86dNCxFFCpNMLEf6kr -s
>> neumann2.cluster.cetene.gov.br
>>
<
http://neumann2.cluster.cetene.gov.br/> <http://neumann2.cluster.ce...
>> <
http://neumann2.cluster.cetene.gov.br/>>
>> -p 'DNS/neumann2.cluster.cetene.gov.br
>> <
http://neumann2.cluster.cetene.gov.br/>
>> <
http://neumann2.cluster.cetene.gov.br
>> <
http://neumann2.cluster.cetene.gov.br/>>' -r -k /etc/named.keytab
>>
>> # 389ds
>> ipa-getkeytab -D "cn=directory manager" -w 86dNCxFFCpNMLEf6kr -s
>> neumann2.cluster.cetene.gov.br
>>
<
http://neumann2.cluster.cetene.gov.br/> <http://neumann2.cluster.ce...
>> <
http://neumann2.cluster.cetene.gov.br/>>
>> -p 'ldap/neumann2.cluster.cetene.gov.br
>> <
http://neumann2.cluster.cetene.gov.br/>
>> <
http://neumann2.cluster.cetene.gov.br
>> <
http://neumann2.cluster.cetene.gov.br/>>' -r -k
/etc/dirsrv/ds.keytab
>>
>> Some error messages:
>>
>> [10/Feb/2021:23:05:57.501853962 -0300] conn=92 op=1 RESULT err=49 tag=97
>> nentries=0 etime=0.001927716 - SASL(-1): generic failure: GSSAPI Error:
>> Unspecified GSS failure. Minor code may provide more information
>> (Cannot create replay cache file /var/tmp/ldap_389: Operation not
>> permitted)
>>
>> ==> /var/log/messages <==
>> Feb 10 23:05:14 neumann2 systemd: ipa-dnskeysyncd.service holdoff time
>> over, scheduling restart.
>> Feb 10 23:05:14 neumann2 systemd: Stopped IPA key daemon.
>> Feb 10 23:05:14 neumann2 systemd: Started IPA key daemon.
>> Feb 10 23:05:16 neumann2 ipa-dnskeysyncd: ipa-dnskeysyncd: INFO LDAP
>> bind...
>> Feb 10 23:05:16 neumann2 ipa-dnskeysyncd: ipa-dnskeysyncd: ERROR
>> Login to LDAP server failed: {'desc': 'Invalid credentials'}
>> Feb 10 23:05:16 neumann2 ipa-dnskeysyncd: Traceback (most recent call
>> last):
>> Feb 10 23:05:16 neumann2 ipa-dnskeysyncd: File
>> "/usr/libexec/ipa/ipa-dnskeysyncd", line 96, in <module>
>> Feb 10 23:05:16 neumann2 ipa-dnskeysyncd:
>> ldap_connection.sasl_interactive_bind_s("", ipaldap.SASL_GSSAPI)
>> Feb 10 23:05:16 neumann2 ipa-dnskeysyncd: File
>> "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 850, in
>> sasl_interactive_bind_s
>> Feb 10 23:05:16 neumann2 ipa-dnskeysyncd: res =
>> self._apply_method_s(SimpleLDAPObject.sasl_interactive_bind_s,*args,**kwargs)
>> Feb 10 23:05:16 neumann2 ipa-dnskeysyncd: File
>> "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 818, in
>> _apply_method_s
>> Feb 10 23:05:16 neumann2 ipa-dnskeysyncd: return
>> func(self,*args,**kwargs)
>> Feb 10 23:05:16 neumann2 ipa-dnskeysyncd: File
>> "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 229, in
>> sasl_interactive_bind_s
>> Feb 10 23:05:16 neumann2 ipa-dnskeysyncd: return
>>
self._ldap_call(self._l.sasl_interactive_bind_s,who,auth,RequestControlTuples(serverctrls),RequestControlTuples(clientctrls),sasl_flags)
>> Feb 10 23:05:16 neumann2 ipa-dnskeysyncd: File
>> "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 99, in
>> _ldap_call
>> Feb 10 23:05:16 neumann2 ipa-dnskeysyncd: result = func(*args,**kwargs)
>> Feb 10 23:05:16 neumann2 ipa-dnskeysyncd: INVALID_CREDENTIALS: {'desc':
>> 'Invalid credentials'}
>> Feb 10 23:05:16 neumann2 systemd: ipa-dnskeysyncd.service: main process
>> exited, code=exited, status=1/FAILURE
>> Feb 10 23:05:16 neumann2 systemd: Unit ipa-dnskeysyncd.service entered
>> failed state.
>> Feb 10 23:05:16 neumann2 systemd: ipa-dnskeysyncd.service failed.
>>
>> Thanks,
>>
>>> On 10 Feb 2021, at 02:01, Vinícius Ferrão via FreeIPA-users
>>> <freeipa-users(a)lists.fedorahosted.org
>>> <mailto:freeipa-users@lists.fedorahosted.org>
>>> <mailto:freeipa-users@lists.fedorahosted.org>> wrote:
>>>
>>> Hello,
>>>
>>> FreeIPA on CentOS 7.8 just stopped working and I’m unable to fix it by
>>> myself. After reading a lot of threads here on the list, it appears
>>> that I’ve the same issue as this
>>>
topic: https://www.mail-archive.com/freeipa-users@lists.fedorahosted.org/...
>>>
>>> Since Kerberos is apparently not working as expected, I cannot use
>>> FreeIPA and none of the services are working correctly. Following the
>>> debug guide I was able to at least start named with single
>>> authentication to further debug. (Workaround 1
>>>
of https://docs.pagure.org/bind-dyndb-ldap/BIND9/NamedCannotStart.html)
>>>
>>> And now I’m stuck on item 5 of the same manual.
>>>
>>> [root@neumann2 ~]# KRB5_TRACE=/dev/stderr ldapsearch -H
>>> 'ldapi://%2fvar%2frun%2fslapd-CLUSTER-CETENE-GOV-BR.socket' -Y
GSSAPI
>>> -b 'cn=dns,dc=cluster,dc=cetene,dc=gov,dc=br
>>> <ldapi://%2fvar%2frun%2fslapd-CLUSTER-CETENE-GOV-BR.socket' -Y GSSAPI
>>> -b 'cn=dns,dc=cluster,dc=cetene,dc=gov,dc=br>'
>>> SASL/GSSAPI authentication started
>>> [6588] 1612932571.244080: ccselect module realm chose cache
>>> KEYRING:persistent:0:krb_ccache_UuVdVRC with client principal
>>> DNS/neumann2.cluster.cetene.gov.br(a)CLUSTER.CETENE.GOV.BR
>>> <mailto:DNS/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
>>> <mailto:DNS/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR> for
>>> server principal
>>> ldap/neumann2.cluster.cetene.gov.br(a)CLUSTER.CETENE.GOV.BR
>>> <mailto:ldap/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
>>> <mailto:ldap/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
>>> [6588] 1612932571.244081: Getting credentials
>>> DNS/neumann2.cluster.cetene.gov.br(a)CLUSTER.CETENE.GOV.BR
>>> <mailto:DNS/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
>>> <mailto:DNS/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
->
>>> ldap/neumann2.cluster.cetene.gov.br(a)CLUSTER.CETENE.GOV.BR
>>> <mailto:ldap/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
>>> <mailto:ldap/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
>>> using ccache KEYRING:persistent:0:krb_ccache_UuVdVRC
>>> [6588] 1612932571.244082: Retrieving
>>> DNS/neumann2.cluster.cetene.gov.br(a)CLUSTER.CETENE.GOV.BR
>>> <mailto:DNS/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
>>> <mailto:DNS/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
->
>>> ldap/neumann2.cluster.cetene.gov.br(a)CLUSTER.CETENE.GOV.BR
>>> <mailto:ldap/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
>>> <mailto:ldap/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
>>> from KEYRING:persistent:0:krb_ccache_UuVdVRC with result: 0/Success
>>> [6588] 1612932571.244084: Creating authenticator for
>>> DNS/neumann2.cluster.cetene.gov.br(a)CLUSTER.CETENE.GOV.BR
>>> <mailto:DNS/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
>>> <mailto:DNS/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
->
>>> ldap/neumann2.cluster.cetene.gov.br(a)CLUSTER.CETENE.GOV.BR
>>> <mailto:ldap/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
>>> <mailto:ldap/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>,
>>> seqnum 1040975659, subkey aes256-cts/48E9, session key aes256-cts/DF1E
>>> ldap_sasl_interactive_bind_s: Invalid credentials (49)
>>>
>>> [root@neumann2 ~]# ipa privilege-show 'DNS Servers' --all --raw
>>> ipa: ERROR: Insufficient access: Invalid credentials
>>>
>>> [root@neumann2 ~]# klist
>>> Ticket cache: KEYRING:persistent:0:krb_ccache_UuVdVRC
>>> Default principal:
>>> DNS/neumann2.cluster.cetene.gov.br(a)CLUSTER.CETENE.GOV.BR
>>> <mailto:DNS/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
>>> <mailto:DNS/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
>>>
>>> Valid starting Expires Service principal
>>> 02/10/2021 01:52:43 02/11/2021 01:49:04
>>> HTTP/neumann2.cluster.cetene.gov.br(a)CLUSTER.CETENE.GOV.BR
>>> <mailto:HTTP/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
>>> <mailto:HTTP/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
>>> 02/10/2021 01:49:16 02/11/2021 01:49:04
>>> ldap/neumann2.cluster.cetene.gov.br(a)CLUSTER.CETENE.GOV.BR
>>> <mailto:ldap/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
>>> <mailto:ldap/neumann2.cluster.cetene.gov.br@CLUSTER.CETENE.GOV.BR>
>>> 02/10/2021 01:49:04 02/11/2021 01:49:04
>>> krbtgt/CLUSTER.CETENE.GOV.BR(a)CLUSTER.CETENE.GOV.BR
>>> <mailto:krbtgt/CLUSTER.CETENE.GOV.BR@CLUSTER.CETENE.GOV.BR>
>>> <mailto:krbtgt/CLUSTER.CETENE.GOV.BR@CLUSTER.CETENE.GOV.BR>
>>>
>>> Any ideia on how to fix this?
>>>
>>> Thanks,
>>> Vinícius.
>>>
>>> PS: Before the workaround named-pkcs11 fails to start with the
>>> following error:
>>>
>>> Feb 10 01:40:46 neumann2 named-pkcs11[4532]: set up managed keys zone
>>> for view _default, file '/var/named/dynamic/managed-keys.bind'
>>> Feb 10 01:40:46 neumann2 named-pkcs11[4532]: loading DynDB instance
>>> 'ipa' driver '/usr/lib64/bind/ldap.so'
>>> Feb 10 01:40:46 neumann2 named-pkcs11[4532]: bind-dyndb-ldap version
>>> 11.1 compiled at 02:16:24 Apr 1 2020, compiler 4.8.5 20150623 (Red
>>> Hat 4.8.5-39)
>>> Feb 10 01:40:46 neumann2 named-pkcs11[4532]: LDAP error: Invalid
>>> credentials: bind to LDAP server failed
>>> Feb 10 01:40:46 neumann2 named-pkcs11[4532]: couldn't establish
>>> connection in LDAP connection pool: permission denied
>>> Feb 10 01:40:46 neumann2 named-pkcs11[4532]: dynamic database 'ipa'
>>> configuration failed: permission denied
>>> Feb 10 01:40:46 neumann2 named-pkcs11[4532]: loading configuration:
>>> permission denied
>>> Feb 10 01:40:46 neumann2 named-pkcs11[4532]: exiting (due to fatal
>>> error)
>>> Feb 10 01:40:46 neumann2 systemd: named-pkcs11.service: control
>>> process exited, code=exited status=1
>>> Feb 10 01:40:46 neumann2 systemd: Failed to start Berkeley Internet
>>> Name Domain (DNS) with native PKCS#11.
>>>
>>> _______________________________________________
>>> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>>> <mailto:freeipa-users@lists.fedorahosted.org>
>>> <mailto:freeipa-users@lists.fedorahosted.org>
>>> To unsubscribe send an email to
>>> freeipa-users-leave(a)lists.fedorahosted.org
>>> <mailto:freeipa-users-leave@lists.fedorahosted.org>
>>> <mailto:freeipa-users-leave@lists.fedorahosted.org>
>>> Fedora Code of Conduct:
>>>
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>>> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
>>> List Archives:
>>>
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
>>
>>
>> _______________________________________________
>> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>> <mailto:freeipa-users@lists.fedorahosted.org>
>> To unsubscribe send an email to
>> freeipa-users-leave(a)lists.fedorahosted.org
>> <mailto:freeipa-users-leave@lists.fedorahosted.org>
>> Fedora Code of Conduct:
>>
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
>> List Archives:
>>
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
>> Do not reply to spam on the list, report it:
>>
https://pagure.io/fedora-infrastructure