I am running into a strange issue with a few user accounts where logging into the web interface gives them the error message "Login failed due to an unknown reason”. It also prevents them from SSH’ing into IPA bound systems using passwords. Pubkeys work fine (as long as it is manually added to the local accounts) and any services I have bound to it (Gitlab, Mattermost, Owncloud, etc) seem to work fine. I ’think’ this is kerberos related since the only services that are using it is SSH and probably the IPA web interface. Here is the apache error log for it:
[Thu Jan 13 09:15:38.688228 2022] [wsgi:error] [pid 579266:tid 139812542121728] [remote xx.xxx.xx.xxx:52162] ipa: INFO: 401 Unauthorized: Major (851968): Unspecified GSS failure. Minor code may provide more information, Minor (2598844948): TGT has been revoked
I ’think’ the message "TGT has been revoked” is due to the 401 error, since the user is not showing as being authorized to login. However, this user is enabled and I have tried a number of things to try to fix it:
1. Disable/Re-enable account 2. Reset passwords 3. Kinit username (seems to get a ticket, but logins still do not work) 4. Run the account migration task (using the web gui) 5. Restart the IPA server and services 6. Re-initialize the IPA server from another master
Also, I can confirm that the passwords are correct since a failed password error message shows up differently and other services are using it correctly. Going down the Kerberos path, here is the krb5kdc log file:
Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): AS_REQ (7 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25), DEPRECATED:arcfour-hmac(23)}) yy.yy.yy.yy: NEEDED_PREAUTH: WELLKNOWN/ANONYMOUS@EXAMPLE.COM for krbtgt/EXAMPLE.COM@EXAMPLE.COM, Additional pre-authentication required Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): closing down fd 12 Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): AS_REQ (7 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25), DEPRECATED:arcfour-hmac(23)}) yy.yy.yy.yy: ISSUE: authtime 1642094138, etypes {rep=aes256-cts-hmac-sha1-96(18), tkt=aes256-cts-hmac-sha1-96(18), ses=aes256-cts-hmac-sha1-96(18)}, WELLKNOWN/ANONYMOUS@EXAMPLE.COM for krbtgt/EXAMPLE.COM@EXAMPLE.COM Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): closing down fd 12 Jan 13 09:15:38 ipa.example.com krb5kdc[579225](info): AS_REQ (7 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25), DEPRECATED:arcfour-hmac(23)}) yy.yy.yy.yy: NEEDED_PREAUTH: testuser@EXAMPLE.COM for krbtgt/EXAMPLE.COM@EXAMPLE.COM, Additional pre-authentication required Jan 13 09:15:38 ipa.example.com krb5kdc[579225](info): closing down fd 12 Jan 13 09:15:38 ipa.example.com krb5kdc[579225](info): AS_REQ (7 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25), DEPRECATED:arcfour-hmac(23)}) yy.yy.yy.yy: ISSUE: authtime 1642094138, etypes {rep=aes256-cts-hmac-sha1-96(18), tkt=aes256-cts-hmac-sha1-96(18), ses=aes256-cts-hmac-sha1-96(18)}, testuser@EXAMPLE.COM for krbtgt/EXAMPLE.COM@EXAMPLE.COM Jan 13 09:15:38 ipa.example.com krb5kdc[579225](info): closing down fd 12 Jan 13 09:15:38 ipa.example.com krb5kdc[579226](Error): PAC issue: PAC record claims domain SID different to local domain SID or any trusted domain SID: local [S-1-5-21-997841278-3584560916-1456654135], PAC [S-1-5-21-2108153867-2082035330-3701898995] Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): TGS_REQ : handle_authdata (-1765328364) Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): TGS_REQ (7 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25), DEPRECATED:arcfour-hmac(23)}) yy.yy.yy.yy: HANDLE_AUTHDATA: authtime 1642094138, etypes {rep=UNSUPPORTED:(0)} testuser@EXAMPLE.COM for HTTP/ipa.example.com@EXAMPLE.COM, TGT has been revoked Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): closing down fd 12 Jan 13 09:15:38 ipa.example.com krb5kdc[579226](Error): PAC issue: PAC record claims domain SID different to local domain SID or any trusted domain SID: local [S-1-5-21-997841278-3584560916-1456654135], PAC [S-1-5-21-2108153867-2082035330-3701898995] Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): TGS_REQ : handle_authdata (-1765328364) Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): TGS_REQ (7 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25), DEPRECATED:arcfour-hmac(23)}) yy.yy.yy.yy: HANDLE_AUTHDATA: authtime 1642094138, etypes {rep=UNSUPPORTED:(0)} testuser@EXAMPLE.COM for HTTP/ipa.example.com@EXAMPLE.COM, TGT has been revoked
I only see two errors that might be related:
"PAC record claims domain SID different to local domain SID or any trusted domain SID” "DEPRECATED:arcfour-hmac(23)”
However, those might just be red herrings or something else that is unrelated.
So far, there are only a small number of accounts that have this problem, but more seem to be popping up on a daily basis. The only fix I have found is the nuclear option, where I completely remove the account and then add it back in with the same UID/GID, group memberships and policies. After that it seems to work fine. However, I would rather not want to do this to all accounts since that would be a logistical nightmare.
Are there any suggestions for either troubleshooting or fixing this problem with a lighter approach? Is it possible to reset or regenerate the users kerberos authentication?
Thanks,
Dan West Systems Administrator Galois Inc. http://galois.com
On to, 13 tammi 2022, Dan West via FreeIPA-users wrote:
I am running into a strange issue with a few user accounts where logging into the web interface gives them the error message "Login failed due to an unknown reason”. It also prevents them from SSH’ing into IPA bound systems using passwords. Pubkeys work fine (as long as it is manually added to the local accounts) and any services I have bound to it (Gitlab, Mattermost, Owncloud, etc) seem to work fine. I ’think’ this is kerberos related since the only services that are using it is SSH and probably the IPA web interface. Here is the apache error log for it:
[Thu Jan 13 09:15:38.688228 2022] [wsgi:error] [pid 579266:tid 139812542121728] [remote xx.xxx.xx.xxx:52162] ipa: INFO: 401 Unauthorized: Major (851968): Unspecified GSS failure. Minor code may provide more information, Minor (2598844948): TGT has been revoked
I ’think’ the message "TGT has been revoked” is due to the 401 error, since the user is not showing as being authorized to login. However, this user is enabled and I have tried a number of things to try to fix it:
- Disable/Re-enable account
- Reset passwords
- Kinit username (seems to get a ticket, but logins still do not work)
- Run the account migration task (using the web gui)
- Restart the IPA server and services
- Re-initialize the IPA server from another master
Also, I can confirm that the passwords are correct since a failed password error message shows up differently and other services are using it correctly. Going down the Kerberos path, here is the krb5kdc log file:
Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): AS_REQ (7 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25), DEPRECATED:arcfour-hmac(23)}) yy.yy.yy.yy: NEEDED_PREAUTH: WELLKNOWN/ANONYMOUS@EXAMPLE.COM for krbtgt/EXAMPLE.COM@EXAMPLE.COM, Additional pre-authentication required Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): closing down fd 12 Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): AS_REQ (7 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25), DEPRECATED:arcfour-hmac(23)}) yy.yy.yy.yy: ISSUE: authtime 1642094138, etypes {rep=aes256-cts-hmac-sha1-96(18), tkt=aes256-cts-hmac-sha1-96(18), ses=aes256-cts-hmac-sha1-96(18)}, WELLKNOWN/ANONYMOUS@EXAMPLE.COM for krbtgt/EXAMPLE.COM@EXAMPLE.COM Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): closing down fd 12 Jan 13 09:15:38 ipa.example.com krb5kdc[579225](info): AS_REQ (7 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25), DEPRECATED:arcfour-hmac(23)}) yy.yy.yy.yy: NEEDED_PREAUTH: testuser@EXAMPLE.COM for krbtgt/EXAMPLE.COM@EXAMPLE.COM, Additional pre-authentication required Jan 13 09:15:38 ipa.example.com krb5kdc[579225](info): closing down fd 12 Jan 13 09:15:38 ipa.example.com krb5kdc[579225](info): AS_REQ (7 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25), DEPRECATED:arcfour-hmac(23)}) yy.yy.yy.yy: ISSUE: authtime 1642094138, etypes {rep=aes256-cts-hmac-sha1-96(18), tkt=aes256-cts-hmac-sha1-96(18), ses=aes256-cts-hmac-sha1-96(18)}, testuser@EXAMPLE.COM for krbtgt/EXAMPLE.COM@EXAMPLE.COM Jan 13 09:15:38 ipa.example.com krb5kdc[579225](info): closing down fd 12 Jan 13 09:15:38 ipa.example.com krb5kdc[579226](Error): PAC issue: PAC record claims domain SID different to local domain SID or any trusted domain SID: local [S-1-5-21-997841278-3584560916-1456654135], PAC [S-1-5-21-2108153867-2082035330-3701898995] Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): TGS_REQ : handle_authdata (-1765328364) Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): TGS_REQ (7 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25), DEPRECATED:arcfour-hmac(23)}) yy.yy.yy.yy: HANDLE_AUTHDATA: authtime 1642094138, etypes {rep=UNSUPPORTED:(0)} testuser@EXAMPLE.COM for HTTP/ipa.example.com@EXAMPLE.COM, TGT has been revoked Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): closing down fd 12 Jan 13 09:15:38 ipa.example.com krb5kdc[579226](Error): PAC issue: PAC record claims domain SID different to local domain SID or any trusted domain SID: local [S-1-5-21-997841278-3584560916-1456654135], PAC [S-1-5-21-2108153867-2082035330-3701898995] Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): TGS_REQ : handle_authdata (-1765328364) Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): TGS_REQ (7 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25), DEPRECATED:arcfour-hmac(23)}) yy.yy.yy.yy: HANDLE_AUTHDATA: authtime 1642094138, etypes {rep=UNSUPPORTED:(0)} testuser@EXAMPLE.COM for HTTP/ipa.example.com@EXAMPLE.COM, TGT has been revoked
I only see two errors that might be related:
"PAC record claims domain SID different to local domain SID or any trusted domain SID” "DEPRECATED:arcfour-hmac(23)”
However, those might just be red herrings or something else that is unrelated.
No, that's exactly your problem. Your domain has SID S-1-5-21-997841278-3584560916-1456654135 but user account has domain SID S-1-5-21-2108153867-2082035330-3701898995. This is against a policy: all accounts in the same domain should have the SID from this domain.
Perhaps these users were migrated from earlier IPA installation (test deployment?) with 'ipa migrate-ds'?
A fix is either to re-create a user from scratch or replace ipaNTSecurityIdentifier value in the user's LDAP entry with the right one.
A correct way to replace that would be a bit complicated -- you need to remove both
objectclass: ipaNTUserAttrs ipaNTSecurityIdentifier: <value>
at the same time because this object class has MUST on ipaNTSecurityIdentifier.
If you have groups with SIDs from wrong domains, do the same for them with these attributes:
objectclass: ipaNTGroupAttrs ipaNTSecurityIdentifier: <value>
After removal of the values from all 'offending' entries, run
kinit admin ipa config-mod --add-sids --enable-sid
This will force re-issue of SIDs to accounts where they are missing. It might take quite a lot of time and 389-ds on that IPA master will be restarted in a process.
So far, there are only a small number of accounts that have this problem, but more seem to be popping up on a daily basis. The only fix I have found is the nuclear option, where I completely remove the account and then add it back in with the same UID/GID, group memberships and policies. After that it seems to work fine. However, I would rather not want to do this to all accounts since that would be a logistical nightmare.
Are there any suggestions for either troubleshooting or fixing this problem with a lighter approach? Is it possible to reset or regenerate the users kerberos authentication?
Thanks,
Dan West Systems Administrator Galois Inc. http://galois.com
Yes, that was the issue. I had migrated from an older FreeIPA instance to a newer one using "ipa migrate-ds" this past summer. I’m not sure why it was just now causing problems, though. Looking at the “ipaNTSecurityIdentifier” for all the accounts gave me a pretty good idea as to which users were affected by this, so I can surgically correct them. Here is my quick one-off fix for it:
ldapmodify -Y GSSAPI <<EOF dn: uid=testuser,cn=users,cn=accounts,dc=example,dc=com changetype: modify delete: ipaNTHash - delete: objectclass objectclass: ipaNTUserAttrs - delete: ipaNTSecurityIdentifier
dn: cn=testuser,cn=groups,cn=accounts,dc=example,dc=com changetype: modify delete: objectclass objectclass: ipaNTGroupAttrs - delete: ipaNTSecurityIdentifier EOF
Thanks for the assist.
Dan West Systems Administrator Galois Inc. http://galois.com
On Jan 13, 2022, at 10:05 AM, Alexander Bokovoy abokovoy@redhat.com wrote:
On to, 13 tammi 2022, Dan West via FreeIPA-users wrote:
I am running into a strange issue with a few user accounts where logging into the web interface gives them the error message "Login failed due to an unknown reason”. It also prevents them from SSH’ing into IPA bound systems using passwords. Pubkeys work fine (as long as it is manually added to the local accounts) and any services I have bound to it (Gitlab, Mattermost, Owncloud, etc) seem to work fine. I ’think’ this is kerberos related since the only services that are using it is SSH and probably the IPA web interface. Here is the apache error log for it:
[Thu Jan 13 09:15:38.688228 2022] [wsgi:error] [pid 579266:tid 139812542121728] [remote xx.xxx.xx.xxx:52162] ipa: INFO: 401 Unauthorized: Major (851968): Unspecified GSS failure. Minor code may provide more information, Minor (2598844948): TGT has been revoked
I ’think’ the message "TGT has been revoked” is due to the 401 error, since the user is not showing as being authorized to login. However, this user is enabled and I have tried a number of things to try to fix it:
- Disable/Re-enable account
- Reset passwords
- Kinit username (seems to get a ticket, but logins still do not work)
- Run the account migration task (using the web gui)
- Restart the IPA server and services
- Re-initialize the IPA server from another master
Also, I can confirm that the passwords are correct since a failed password error message shows up differently and other services are using it correctly. Going down the Kerberos path, here is the krb5kdc log file:
Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): AS_REQ (7 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25), DEPRECATED:arcfour-hmac(23)}) yy.yy.yy.yy: NEEDED_PREAUTH: WELLKNOWN/ANONYMOUS@EXAMPLE.COM for krbtgt/EXAMPLE.COM@EXAMPLE.COM, Additional pre-authentication required Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): closing down fd 12 Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): AS_REQ (7 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25), DEPRECATED:arcfour-hmac(23)}) yy.yy.yy.yy: ISSUE: authtime 1642094138, etypes {rep=aes256-cts-hmac-sha1-96(18), tkt=aes256-cts-hmac-sha1-96(18), ses=aes256-cts-hmac-sha1-96(18)}, WELLKNOWN/ANONYMOUS@EXAMPLE.COM for krbtgt/EXAMPLE.COM@EXAMPLE.COM Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): closing down fd 12 Jan 13 09:15:38 ipa.example.com krb5kdc[579225](info): AS_REQ (7 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25), DEPRECATED:arcfour-hmac(23)}) yy.yy.yy.yy: NEEDED_PREAUTH: testuser@EXAMPLE.COM for krbtgt/EXAMPLE.COM@EXAMPLE.COM, Additional pre-authentication required Jan 13 09:15:38 ipa.example.com krb5kdc[579225](info): closing down fd 12 Jan 13 09:15:38 ipa.example.com krb5kdc[579225](info): AS_REQ (7 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25), DEPRECATED:arcfour-hmac(23)}) yy.yy.yy.yy: ISSUE: authtime 1642094138, etypes {rep=aes256-cts-hmac-sha1-96(18), tkt=aes256-cts-hmac-sha1-96(18), ses=aes256-cts-hmac-sha1-96(18)}, testuser@EXAMPLE.COM for krbtgt/EXAMPLE.COM@EXAMPLE.COM Jan 13 09:15:38 ipa.example.com krb5kdc[579225](info): closing down fd 12 Jan 13 09:15:38 ipa.example.com krb5kdc[579226](Error): PAC issue: PAC record claims domain SID different to local domain SID or any trusted domain SID: local [S-1-5-21-997841278-3584560916-1456654135], PAC [S-1-5-21-2108153867-2082035330-3701898995] Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): TGS_REQ : handle_authdata (-1765328364) Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): TGS_REQ (7 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25), DEPRECATED:arcfour-hmac(23)}) yy.yy.yy.yy: HANDLE_AUTHDATA: authtime 1642094138, etypes {rep=UNSUPPORTED:(0)} testuser@EXAMPLE.COM for HTTP/ipa.example.com@EXAMPLE.COM, TGT has been revoked Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): closing down fd 12 Jan 13 09:15:38 ipa.example.com krb5kdc[579226](Error): PAC issue: PAC record claims domain SID different to local domain SID or any trusted domain SID: local [S-1-5-21-997841278-3584560916-1456654135], PAC [S-1-5-21-2108153867-2082035330-3701898995] Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): TGS_REQ : handle_authdata (-1765328364) Jan 13 09:15:38 ipa.example.com krb5kdc[579226](info): TGS_REQ (7 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17), aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25), DEPRECATED:arcfour-hmac(23)}) yy.yy.yy.yy: HANDLE_AUTHDATA: authtime 1642094138, etypes {rep=UNSUPPORTED:(0)} testuser@EXAMPLE.COM for HTTP/ipa.example.com@EXAMPLE.COM, TGT has been revoked
I only see two errors that might be related:
"PAC record claims domain SID different to local domain SID or any trusted domain SID” "DEPRECATED:arcfour-hmac(23)”
However, those might just be red herrings or something else that is unrelated.
No, that's exactly your problem. Your domain has SID S-1-5-21-997841278-3584560916-1456654135 but user account has domain SID S-1-5-21-2108153867-2082035330-3701898995. This is against a policy: all accounts in the same domain should have the SID from this domain.
Perhaps these users were migrated from earlier IPA installation (test deployment?) with 'ipa migrate-ds'?
A fix is either to re-create a user from scratch or replace ipaNTSecurityIdentifier value in the user's LDAP entry with the right one.
A correct way to replace that would be a bit complicated -- you need to remove both
objectclass: ipaNTUserAttrs ipaNTSecurityIdentifier: <value>
at the same time because this object class has MUST on ipaNTSecurityIdentifier.
If you have groups with SIDs from wrong domains, do the same for them with these attributes:
objectclass: ipaNTGroupAttrs ipaNTSecurityIdentifier: <value>
After removal of the values from all 'offending' entries, run
kinit admin ipa config-mod --add-sids --enable-sid
This will force re-issue of SIDs to accounts where they are missing. It might take quite a lot of time and 389-ds on that IPA master will be restarted in a process.
So far, there are only a small number of accounts that have this problem, but more seem to be popping up on a daily basis. The only fix I have found is the nuclear option, where I completely remove the account and then add it back in with the same UID/GID, group memberships and policies. After that it seems to work fine. However, I would rather not want to do this to all accounts since that would be a logistical nightmare.
Are there any suggestions for either troubleshooting or fixing this problem with a lighter approach? Is it possible to reset or regenerate the users kerberos authentication?
Thanks,
Dan West Systems Administrator Galois Inc. http://galois.com
-- / Alexander Bokovoy Sr. Principal Software Engineer Security / Identity Management Engineering Red Hat Limited, Finland
Hello Dan, I also have such problem after migration with the "ipa migrate-ds" from the old Freeipa.
Can you explain more how do you fix it? I tried your solution with ldapmodify, but it's not working for me. After removing the attribute and the objectclass and starting kinit admin ipa config-mod --add-sids --enable-sid nothing happened.
As I understand my problem isn't related to SID probably. All my users obtained correct (as I think) SIDs after the migration. But I have such different logs for the admin user (was not migrated) and for the test.1 user which was migrated from the old FreeIPA 4.6.8 on CentOS7 the the new Ubuntu 22.04 docker instance of the FreeIPA 4.10.2 with the same realm, but on the different domain.
The migration was made with the command:
ipa migrate-ds --bind-dn="cn=Directory Manager" --user-container=cn=users,cn=accounts --group-container=cn=groups,cn=accounts --group-objectclass=posixgroup --user-ignore-attribute={krbPrincipalName,krbextradata,krblastfailedauth,krblastpwdchange,krblastsuccessfulauth,krbloginfailedcount,krbpasswordexpiration,krbticketflags,krbpwdpolicyreference,mepManagedEntry,ipaNTSecurityIdentifier,pwmlastpwdupdate,pwmeventlog} --user-ignore-objectclass={mepOriginEntry,pwmuser,ipaNTUserAttrs} --group-ignore-attribute=ipaNTSecurityIdentifier --group-ignore-objectclass=ipaNTGroupAttrs --exclude-users={pwm.proxy,pwm.test} --exclude-groups={pwm.proxy,pwm.test} --group-overwrite-gid --with-compat ldaps://old.somedomain.net
[root@ldap-2 /]# KRB5_TRACE=/dev/stderr kinit admin 2>&1 [7363] 1709655365.522471: Getting initial credentials for admin@SOMEDOMAIN.NET [7363] 1709655365.522473: Sending unauthenticated request [7363] 1709655365.522474: Sending request (169 bytes) to SOMEDOMAIN.NET [7363] 1709655365.522475: Initiating TCP connection to stream 172.18.0.3:88 [7363] 1709655365.522476: Sending TCP request to stream 172.18.0.3:88 [7363] 1709655365.522477: Received answer (526 bytes) from stream 172.18.0.3:88 [7363] 1709655365.522478: Terminating TCP connection to stream 172.18.0.3:88 [7363] 1709655365.522479: Response was from primary KDC [7363] 1709655365.522480: Received error from KDC: -1765328359/Additional pre-authentication required [7363] 1709655365.522483: Preauthenticating using KDC method data [7363] 1709655365.522484: Processing preauth types: PA-PK-AS-REQ (16), PA-FX-FAST (136), PA-ETYPE-INFO2 (19), PA-PKINIT-KX (147), PA-SPAKE (151), PA-ENC-TIMESTAMP (2), PA_AS_FRESHNESS (150), PA-FX-COOKIE (133) [7363] 1709655365.522485: Selected etype info: etype aes256-sha2, salt "1u\ ]=_tjHbc>-/e", params "" [7363] 1709655365.522486: Received cookie: MIT1\x00\x00\x00\x01\x1b\xb8\x99\xd8b\x8b\xe8\xc0\xe1\xca\x82\x0c\x9c"\x06\x7f3\x83o]\xbb\x172\xb5A\x053\ni\xd1\x88\x1e&>\xaaS\xd9\x15|\x84\xdb\xe9\xb1azEs\x99\xfb\x91\xaa\xb5\x08\x9c+\xb1\xb6\x02\xba\x85\x08 \xa1RV\x7f\xd3\xa3\x0b\x99\x9e\xda\xbap?U\xde\xd3\x9c\x0d\xe9T\x98\xbc+\xc4\xe8|\x7f=\xfa\x1f\xde\xae\x93\x12\x81m\xc2\xf5cFs\xf7\x12\x157\xb8c\xd1\x11\x9c\x8d\xa8\xf2\x9b\xd5\x94X\xb2%\x08\x91\x11a?L\x03d\xbc5\x9f4GmV\xa96fe [7363] 1709655365.522487: PKINIT client has no configured identity; giving up [7363] 1709655365.522488: Preauth module pkinit (147) (info) returned: 0/Success [7363] 1709655365.522489: PKINIT client received freshness token from KDC [7363] 1709655365.522490: Preauth module pkinit (150) (info) returned: 0/Success [7363] 1709655365.522491: PKINIT client has no configured identity; giving up [7363] 1709655365.522492: Preauth module pkinit (16) (real) returned: 22/Invalid argument [7363] 1709655365.522493: SPAKE challenge received with group 1, pubkey 22D477D5D4218DC8C5FFF38EC21FE6E08D9A6488F3F96D69A3D6D15C929D2EC2 Password for admin@SOMEDOMAIN.NET: [7363] 1709655418.745247: SPAKE key generated with pubkey 344A6368A2BE4535EB68237F9996F92FF4418A19661AFA4B5B84CE5780DF909A [7363] 1709655418.745248: SPAKE algorithm result: F630A33BAA4143B978F659D6A401A53174E43A82E6F70140BA99CAC959A2C29F [7363] 1709655418.745249: SPAKE final transcript hash: 9CF0C027377C1287D946DB78876076A46B97D95E962AA30A05634184107222F9 [7363] 1709655418.745250: Sending SPAKE response [7363] 1709655418.745251: Preauth module spake (151) (real) returned: 0/Success [7363] 1709655418.745252: Produced preauth for next request: PA-FX-COOKIE (133), PA-SPAKE (151) [7363] 1709655418.745253: Sending request (452 bytes) to MEA-DEV.NET [7363] 1709655418.745254: Initiating TCP connection to stream 172.18.0.3:88 [7363] 1709655418.745255: Sending TCP request to stream 172.18.0.3:88 [7363] 1709655418.745256: Received answer (1761 bytes) from stream 172.18.0.3:88 [7363] 1709655418.745257: Terminating TCP connection to stream 172.18.0.3:88 [7363] 1709655418.745258: Response was from primary KDC [7363] 1709655418.745259: Processing preauth types: PA-ETYPE-INFO2 (19) [7363] 1709655418.745260: Selected etype info: etype aes256-sha2, salt "1u\ ]=_tjHbc>-/e", params "" [7363] 1709655418.745261: Produced preauth for next request: (empty) [7363] 1709655418.745262: AS key determined by preauth: aes256-sha2/B7BD [7363] 1709655418.745263: Decrypted AS reply; session key is: aes256-sha2/5E1A [7363] 1709655418.745264: FAST negotiation: available [7363] 1709655418.745265: Resolving unique ccache of type MEMORY [7363] 1709655418.745266: Initializing MEMORY:yGYZJ2v with default princ admin@SOMEDOMAIN.NET [7363] 1709655418.745267: Storing config in MEMORY:yGYZJ2v for krbtgt/SOMEDOMAIN.NET@SOMEDOMAIN.NET: fast_avail: yes [7363] 1709655418.745268: Storing admin@SOMEDOMAIN.NET -> krb5_ccache_conf_data/fast_avail/krbtgt/SOMEDOMAIN.NET@SOMEDOMAIN.NET@X-CACHECONF: in MEMORY:yGYZJ2v [7363] 1709655418.745269: Storing config in MEMORY:yGYZJ2v for krbtgt/SOMEDOMAIN.NET@SOMEDOMAIN.NET: pa_type: 151 [7363] 1709655418.745270: Storing admin@SOMEDOMAIN.NET -> krb5_ccache_conf_data/pa_type/krbtgt/SOMEDOMAIN.NET@SOMEDOMAIN.NET@X-CACHECONF: in MEMORY:yGYZJ2v [7363] 1709655418.745271: Storing admin@SOMEDOMAIN.NET -> krbtgt/SOMEDOMAIN.NET@SOMEDOMAIN.NET in MEMORY:yGYZJ2v [7363] 1709655418.745272: Moving ccache MEMORY:yGYZJ2v to FILE:/tmp/krb5cc_0 [7363] 1709655418.745273: Destroying ccache MEMORY:yGYZJ2v
and for the test.1 (migrated user)
KRB5_TRACE=/dev/stderr kinit test.1 2>&1 [7364] 1709655454.364392: Getting initial credentials for test.1@SOMEDOMAIN.NET [7364] 1709655454.364394: Sending unauthenticated request [7364] 1709655454.364395: Sending request (170 bytes) to SOMEDOMAIN.NET [7364] 1709655454.364396: Initiating TCP connection to stream 172.18.0.3:88 [7364] 1709655454.364397: Sending TCP request to stream 172.18.0.3:88 [7364] 1709655454.364398: Received answer (250 bytes) from stream 172.18.0.3:88 [7364] 1709655454.364399: Terminating TCP connection to stream 172.18.0.3:88 [7364] 1709655454.364400: Response was from primary KDC [7364] 1709655454.364401: Received error from KDC: -1765328359/Additional pre-authentication required [7364] 1709655454.364404: Preauthenticating using KDC method data [7364] 1709655454.364405: Processing preauth types: PA-PK-AS-REQ (16), PA-FX-FAST (136), PA-PKINIT-KX (147), PA_AS_FRESHNESS (150), PA-FX-COOKIE (133) [7364] 1709655454.364406: Received cookie: MIT [7364] 1709655454.364407: PKINIT client has no configured identity; giving up [7364] 1709655454.364408: Preauth module pkinit (147) (info) returned: 0/Success [7364] 1709655454.364409: PKINIT client received freshness token from KDC [7364] 1709655454.364410: Preauth module pkinit (150) (info) returned: 0/Success [7364] 1709655454.364411: PKINIT client has no configured identity; giving up [7364] 1709655454.364412: Preauth module pkinit (16) (real) returned: 22/Invalid argument kinit: Pre-authentication failed: Invalid argument while getting initial credentials
On Аўт, 05 сак 2024, ITreers UA via FreeIPA-users wrote:
As I understand my problem isn't related to SID probably. All my users obtained correct (as I think) SIDs after the migration. But I have such different logs for the admin user (was not migrated) and for the test.1 user which was migrated from the old FreeIPA 4.6.8 on CentOS7 the the new Ubuntu 22.04 docker instance of the FreeIPA 4.10.2 with the same realm, but on the different domain.
The migration was made with the command:
ipa migrate-ds --bind-dn="cn=Directory Manager" --user-container=cn=users,cn=accounts --group-container=cn=groups,cn=accounts --group-objectclass=posixgroup --user-ignore-attribute={krbPrincipalName,krbextradata,krblastfailedauth,krblastpwdchange,krblastsuccessfulauth,krbloginfailedcount,krbpasswordexpiration,krbticketflags,krbpwdpolicyreference,mepManagedEntry,ipaNTSecurityIdentifier,pwmlastpwdupdate,pwmeventlog} --user-ignore-objectclass={mepOriginEntry,pwmuser,ipaNTUserAttrs} --group-ignore-attribute=ipaNTSecurityIdentifier --group-ignore-objectclass=ipaNTGroupAttrs --exclude-users={pwm.proxy,pwm.test} --exclude-groups={pwm.proxy,pwm.test} --group-overwrite-gid --with-compat ldaps://old.somedomain.net
[root@ldap-2 /]# KRB5_TRACE=/dev/stderr kinit admin 2>&1 [7363] 1709655365.522471: Getting initial credentials for admin@SOMEDOMAIN.NET [7363] 1709655365.522473: Sending unauthenticated request [7363] 1709655365.522474: Sending request (169 bytes) to SOMEDOMAIN.NET [7363] 1709655365.522475: Initiating TCP connection to stream 172.18.0.3:88 [7363] 1709655365.522476: Sending TCP request to stream 172.18.0.3:88 [7363] 1709655365.522477: Received answer (526 bytes) from stream 172.18.0.3:88 [7363] 1709655365.522478: Terminating TCP connection to stream 172.18.0.3:88 [7363] 1709655365.522479: Response was from primary KDC [7363] 1709655365.522480: Received error from KDC: -1765328359/Additional pre-authentication required [7363] 1709655365.522483: Preauthenticating using KDC method data [7363] 1709655365.522484: Processing preauth types: PA-PK-AS-REQ (16), PA-FX-FAST (136), PA-ETYPE-INFO2 (19), PA-PKINIT-KX (147), PA-SPAKE (151), PA-ENC-TIMESTAMP (2), PA_AS_FRESHNESS (150), PA-FX-COOKIE (133) [7363] 1709655365.522485: Selected etype info: etype aes256-sha2, salt "1u\ ]=_tjHbc>-/e", params "" [7363] 1709655365.522486: Received cookie: MIT1\x00\x00\x00\x01\x1b\xb8\x99\xd8b\x8b\xe8\xc0\xe1\xca\x82\x0c\x9c"\x06\x7f3\x83o]\xbb\x172\xb5A\x053\ni\xd1\x88\x1e&>\xaaS\xd9\x15|\x84\xdb\xe9\xb1azEs\x99\xfb\x91\xaa\xb5\x08\x9c+\xb1\xb6\x02\xba\x85\x08 \xa1RV\x7f\xd3\xa3\x0b\x99\x9e\xda\xbap?U\xde\xd3\x9c\x0d\xe9T\x98\xbc+\xc4\xe8|\x7f=\xfa\x1f\xde\xae\x93\x12\x81m\xc2\xf5cFs\xf7\x12\x157\xb8c\xd1\x11\x9c\x8d\xa8\xf2\x9b\xd5\x94X\xb2%\x08\x91\x11a?L\x03d\xbc5\x9f4GmV\xa96fe [7363] 1709655365.522487: PKINIT client has no configured identity; giving up [7363] 1709655365.522488: Preauth module pkinit (147) (info) returned: 0/Success [7363] 1709655365.522489: PKINIT client received freshness token from KDC [7363] 1709655365.522490: Preauth module pkinit (150) (info) returned: 0/Success [7363] 1709655365.522491: PKINIT client has no configured identity; giving up [7363] 1709655365.522492: Preauth module pkinit (16) (real) returned: 22/Invalid argument [7363] 1709655365.522493: SPAKE challenge received with group 1, pubkey 22D477D5D4218DC8C5FFF38EC21FE6E08D9A6488F3F96D69A3D6D15C929D2EC2 Password for admin@SOMEDOMAIN.NET: [7363] 1709655418.745247: SPAKE key generated with pubkey 344A6368A2BE4535EB68237F9996F92FF4418A19661AFA4B5B84CE5780DF909A [7363] 1709655418.745248: SPAKE algorithm result: F630A33BAA4143B978F659D6A401A53174E43A82E6F70140BA99CAC959A2C29F [7363] 1709655418.745249: SPAKE final transcript hash: 9CF0C027377C1287D946DB78876076A46B97D95E962AA30A05634184107222F9 [7363] 1709655418.745250: Sending SPAKE response [7363] 1709655418.745251: Preauth module spake (151) (real) returned: 0/Success [7363] 1709655418.745252: Produced preauth for next request: PA-FX-COOKIE (133), PA-SPAKE (151) [7363] 1709655418.745253: Sending request (452 bytes) to MEA-DEV.NET [7363] 1709655418.745254: Initiating TCP connection to stream 172.18.0.3:88 [7363] 1709655418.745255: Sending TCP request to stream 172.18.0.3:88 [7363] 1709655418.745256: Received answer (1761 bytes) from stream 172.18.0.3:88 [7363] 1709655418.745257: Terminating TCP connection to stream 172.18.0.3:88 [7363] 1709655418.745258: Response was from primary KDC [7363] 1709655418.745259: Processing preauth types: PA-ETYPE-INFO2 (19) [7363] 1709655418.745260: Selected etype info: etype aes256-sha2, salt "1u\ ]=_tjHbc>-/e", params "" [7363] 1709655418.745261: Produced preauth for next request: (empty) [7363] 1709655418.745262: AS key determined by preauth: aes256-sha2/B7BD [7363] 1709655418.745263: Decrypted AS reply; session key is: aes256-sha2/5E1A [7363] 1709655418.745264: FAST negotiation: available [7363] 1709655418.745265: Resolving unique ccache of type MEMORY [7363] 1709655418.745266: Initializing MEMORY:yGYZJ2v with default princ admin@SOMEDOMAIN.NET [7363] 1709655418.745267: Storing config in MEMORY:yGYZJ2v for krbtgt/SOMEDOMAIN.NET@SOMEDOMAIN.NET: fast_avail: yes [7363] 1709655418.745268: Storing admin@SOMEDOMAIN.NET -> krb5_ccache_conf_data/fast_avail/krbtgt/SOMEDOMAIN.NET@SOMEDOMAIN.NET@X-CACHECONF: in MEMORY:yGYZJ2v [7363] 1709655418.745269: Storing config in MEMORY:yGYZJ2v for krbtgt/SOMEDOMAIN.NET@SOMEDOMAIN.NET: pa_type: 151 [7363] 1709655418.745270: Storing admin@SOMEDOMAIN.NET -> krb5_ccache_conf_data/pa_type/krbtgt/SOMEDOMAIN.NET@SOMEDOMAIN.NET@X-CACHECONF: in MEMORY:yGYZJ2v [7363] 1709655418.745271: Storing admin@SOMEDOMAIN.NET -> krbtgt/SOMEDOMAIN.NET@SOMEDOMAIN.NET in MEMORY:yGYZJ2v [7363] 1709655418.745272: Moving ccache MEMORY:yGYZJ2v to FILE:/tmp/krb5cc_0 [7363] 1709655418.745273: Destroying ccache MEMORY:yGYZJ2v
and for the test.1 (migrated user)
KRB5_TRACE=/dev/stderr kinit test.1 2>&1 [7364] 1709655454.364392: Getting initial credentials for test.1@SOMEDOMAIN.NET [7364] 1709655454.364394: Sending unauthenticated request [7364] 1709655454.364395: Sending request (170 bytes) to SOMEDOMAIN.NET [7364] 1709655454.364396: Initiating TCP connection to stream 172.18.0.3:88 [7364] 1709655454.364397: Sending TCP request to stream 172.18.0.3:88 [7364] 1709655454.364398: Received answer (250 bytes) from stream 172.18.0.3:88 [7364] 1709655454.364399: Terminating TCP connection to stream 172.18.0.3:88 [7364] 1709655454.364400: Response was from primary KDC [7364] 1709655454.364401: Received error from KDC: -1765328359/Additional pre-authentication required [7364] 1709655454.364404: Preauthenticating using KDC method data [7364] 1709655454.364405: Processing preauth types: PA-PK-AS-REQ (16), PA-FX-FAST (136), PA-PKINIT-KX (147), PA_AS_FRESHNESS (150), PA-FX-COOKIE (133) [7364] 1709655454.364406: Received cookie: MIT [7364] 1709655454.364407: PKINIT client has no configured identity; giving up [7364] 1709655454.364408: Preauth module pkinit (147) (info) returned: 0/Success [7364] 1709655454.364409: PKINIT client received freshness token from KDC [7364] 1709655454.364410: Preauth module pkinit (150) (info) returned: 0/Success [7364] 1709655454.364411: PKINIT client has no configured identity; giving up [7364] 1709655454.364412: Preauth module pkinit (16) (real) returned: 22/Invalid argument kinit: Pre-authentication failed: Invalid argument while getting initial credentials
Look at the preauth types. The second user has no password, hence no PA-SPAKE or PA-ENC-TIMESTAMP preauthentication methods.
Once migrated via 'ipa migrate-ds', users will lack Kerberos keys. You need to follow migration instructions and enable migration mode, then login for this user through SSSD or a web page for migration.
Thank you for the reply.
As I understood from your reply it's not possible to migrate passwords without "migration" procedure after the ipa migrate-ds? During my test migrations from earlier (start of the last month) I have managed to migrate and login with old passwords after the ipa migrate-ds. I used docker image "#rocky-9" and until image was updated with the new OS version or some security updates I don't know I have 2 or 3 successful attempt of the migration of users with the passwords. I was able to login using kinit and web. How it possible?
ITreers UA via FreeIPA-users wrote:
Thank you for the reply.
As I understood from your reply it's not possible to migrate passwords without "migration" procedure after the ipa migrate-ds? During my test migrations from earlier (start of the last month) I have managed to migrate and login with old passwords after the ipa migrate-ds. I used docker image "#rocky-9" and until image was updated with the new OS version or some security updates I don't know I have 2 or 3 successful attempt of the migration of users with the passwords. I was able to login using kinit and web. How it possible?
I think you are overusing the word migrate. After migrate-ds the users only have an LDAP password at best. In order to generate Kerberos keys they need to authenticate to LDAP while IPA is still in migration mode (ipa config-mod --enable-migration).
Logging into an IPA-enrolled client will do this key generation automatically if IPA is still in migration mode. Or, as Alexander said, there is a web site for this as well.
If you turn off the IPA migration then you will need to reset users's passwords so that keys can be generated.
rob
Is it any way to simulate such login from every user to generate Kerberos keys?
I also tried to migrate user statuses with the simple custom script like below and got strange output. #!/bin/bash
echo -e "${ADM_PW}" | kinit admin DIS_USERS=$(tail -n +2 ldap_dis.txt)
for USR in ${DIS_USERS}; do ipa user-disable ${USR} done
cat ldap_dis.txt SOME_NOT_REQ_STRING test.1 test.2 test.3
ipa user-disable test.1 (started inside the FreeIPA docker container) working as expected, but output from the same command in the script look like below
Password for admin@SOMEDOMAIN.COM: ----------------------------------------- "isabled user account "test.1 ----------------------------------------- ----------------------------------- "isabled user account "test.2 ----------------------------------- ----------------------------------- "isabled user account "test.3 -----------------------------------
freeipa-users@lists.fedorahosted.org