Hello,
We are currently experiencing strange behavior on FreeIPA system related to PKINIT OpenSSL error when trying to log in through FreeIPA web gui. This started happening as we upgraded our second replica with "dnf ugprade". Freeipa packages in themself haven't been updated.
Our setup is basically as follows.
ipa.tre-1.web1.fi ipa.tku-2.web1.fi <-- the one not working.
GUI throws an error "Login failed due to an unknown reason" httpd error log has the following line after error:
[Fri Feb 25 19:32:50.776457 2022] [wsgi:error] [pid 17977:tid 18319] [remote 10.20.11.2:49472] ipapython.ipautil.CalledProcessError: CalledProcessError(Command ['/usr/bin/kinit', '-n', '-c', '/run/ipa/ccaches/armor_17977', '-X', 'X509_anchors=FILE:/var/kerberos/krb5kdc/kdc.crt', '-X', 'X509_anchors=FILE:/var/lib/ipa-client/pki/kdc-ca-bundle.pem'] returned non-zero exit status 1: 'kinit: Cannot read password while getting initial credentials\n')
Now if I try to run " KRB5_TRACE=/dev/stdout /usr/bin/kinit -n -c /var/run/ipa/ccaches/armor_15581 -X X509_anchors=FILE:/var/kerberos/krb5kdc/kdc.crt -X X509_anchors=FILE:/var/lib/ipa-client/pki/kdc-ca-bundle.pem"
I get the following output
[19260] 1645812760.557398: Getting initial credentials for WELLKNOWN/ANONYMOUS@IPA.WEB1.FI [19260] 1645812760.557400: Sending unauthenticated request [19260] 1645812760.557401: Sending request (186 bytes) to IPA.WEB1.FI [19260] 1645812760.557402: Initiating TCP connection to stream 10.20.13.5:88 [19260] 1645812760.557403: Sending TCP request to stream 10.20.13.5:88 [19260] 1645812760.557404: Received answer (538 bytes) from stream 10.20.13.5:88 [19260] 1645812760.557405: Terminating TCP connection to stream 10.20.13.5:88 [19260] 1645812760.557406: Response was from primary KDC [19260] 1645812760.557407: Received error from KDC: -1765328359/Additional pre-authentication required [19260] 1645812760.557410: Preauthenticating using KDC method data [19260] 1645812760.557411: Processing preauth types: PA-PK-AS-REQ (16), PA-FX-FAST (136), PA-ETYPE-INFO2 (19), PA-PKINIT-KX (147), PA-SPAKE (151), PA-ENC-TIMESTAMP (2), PA_AS_FRESHNESS (150), PA-FX-COOKIE (133) [19260] 1645812760.557412: Selected etype info: etype aes256-cts, salt "IPA.WEB1.FIWELLKNOWNANONYMOUS", params "" [19260] 1645812760.557413: Received cookie: MIT1\x00\x00\x00\x01\x8f\xcb\x99\x9c~\xed!^Qj\xa3\x0a\x82~\xe94\x04\x0ck[j=\x08\xd2\x97j'K2\x8f\xa0\xf6\xc3\x89Z@\x8b]\xc3K\xc2h\xfa\xaek\x11\x91y\xc9\xf0\xadG\x13\x9a\xb2\xb6\x1c\x12\xbfr\x0a'Z\xfe\x12\x81\x1a>2\x8c\x1a\xf2\x96\xdc]&qH\x08\x1f\x0d\xc0a{\xe8\xff\xbbF\x9c\x86`\xd6G\xc4*5\xccL\xc1m\xc0\xa7b\x8b]od\xfa*\xd4.bmB\x9d\x92\xb7\xf9($\xa4D\xea\xcd\xc6\xe3p\xac$\xf4 [19260] 1645812760.557414: Preauth module pkinit (147) (info) returned: 0/Success [19260] 1645812760.557415: PKINIT client received freshness token from KDC [19260] 1645812760.557416: Preauth module pkinit (150) (info) returned: 0/Success [19260] 1645812760.557417: PKINIT loading CA certs and CRLs from FILE [19260] 1645812760.557418: PKINIT loading CA certs and CRLs from FILE [19260] 1645812760.557419: PKINIT loading CA certs and CRLs from FILE [19260] 1645812760.557420: PKINIT client computed kdc-req-body checksum 9/B79768B0DAD630709ABFE35C1E2B6FDAB714913D [19260] 1645812760.557422: PKINIT client making DH request [19260] 1645812760.557423: Preauth module pkinit (16) (real) returned: 0/Success [19260] 1645812760.557424: Produced preauth for next request: PA-FX-COOKIE (133), PA-PK-AS-REQ (16) [19260] 1645812760.557425: Sending request (1674 bytes) to IPA.WEB1.FI [19260] 1645812760.557426: Initiating TCP connection to stream 10.20.13.5:88 [19260] 1645812760.557427: Sending TCP request to stream 10.20.13.5:88 [19260] 1645812760.557428: Received answer (2619 bytes) from stream 10.20.13.5:88 [19260] 1645812760.557429: Terminating TCP connection to stream 10.20.13.5:88 [19260] 1645812760.557430: Response was from primary KDC [19260] 1645812760.557431: Processing preauth types: PA-PK-AS-REP (17), PA-PKINIT-KX (147) [19260] 1645812760.557432: Preauth module pkinit (147) (info) returned: 0/Success [19260] 1645812760.557433: PKINIT OpenSSL error: Failed to verify CMS message [19260] 1645812760.557434: PKINIT OpenSSL error: error:1700006B:CMS routines::content type not enveloped data [19260] 1645812760.557435: PKINIT OpenSSL error: error:03000098:digital envelope routines::invalid digest [19260] 1645812760.557436: PKINIT client could not verify DH reply [19260] 1645812760.557437: Preauth module pkinit (17) (real) returned: -1765328320/Failed to verify CMS message: content type not enveloped data [19260] 1645812760.557438: Produced preauth for next request: (empty) [19260] 1645812760.557439: Getting AS key, salt "IPA.WEB1.FIWELLKNOWNANONYMOUS", params "" Password for WELLKNOWN/ANONYMOUS@IPA.WEB1.FI: [19260] 1645812776.928337: AS key obtained from gak_fct: aes256-cts/3840 kinit: Password incorrect while getting initial credentials
But this only happens on the "dc2" one. If I would run this on the "dc1" it would work just fine.
I have tried running ipa-pkinit-manage disable ipa-pkinit-manage enable
to regen the cert but it didn't help. Any suggestions / pointers at why the OpenSSL error on the tku-2 is showing up.
Ville Teronen via FreeIPA-users wrote:
Hello,
We are currently experiencing strange behavior on FreeIPA system related to PKINIT OpenSSL error when trying to log in through FreeIPA web gui. This started happening as we upgraded our second replica with "dnf ugprade". Freeipa packages in themself haven't been updated.
Our setup is basically as follows.
ipa.tre-1.web1.fi ipa.tku-2.web1.fi <-- the one not working.
GUI throws an error "Login failed due to an unknown reason" httpd error log has the following line after error:
[Fri Feb 25 19:32:50.776457 2022] [wsgi:error] [pid 17977:tid 18319] [remote 10.20.11.2:49472] ipapython.ipautil.CalledProcessError: CalledProcessError(Command ['/usr/bin/kinit', '-n', '-c', '/run/ipa/ccaches/armor_17977', '-X', 'X509_anchors=FILE:/var/kerberos/krb5kdc/kdc.crt', '-X', 'X509_anchors=FILE:/var/lib/ipa-client/pki/kdc-ca-bundle.pem'] returned non-zero exit status 1: 'kinit: Cannot read password while getting initial credentials\n')
Now if I try to run " KRB5_TRACE=/dev/stdout /usr/bin/kinit -n -c /var/run/ipa/ccaches/armor_15581 -X X509_anchors=FILE:/var/kerberos/krb5kdc/kdc.crt -X X509_anchors=FILE:/var/lib/ipa-client/pki/kdc-ca-bundle.pem"
I get the following output
[19260] 1645812760.557398: Getting initial credentials for WELLKNOWN/ANONYMOUS@IPA.WEB1.FI [19260] 1645812760.557400: Sending unauthenticated request [19260] 1645812760.557401: Sending request (186 bytes) to IPA.WEB1.FI [19260] 1645812760.557402: Initiating TCP connection to stream 10.20.13.5:88 [19260] 1645812760.557403: Sending TCP request to stream 10.20.13.5:88 [19260] 1645812760.557404: Received answer (538 bytes) from stream 10.20.13.5:88 [19260] 1645812760.557405: Terminating TCP connection to stream 10.20.13.5:88 [19260] 1645812760.557406: Response was from primary KDC [19260] 1645812760.557407: Received error from KDC: -1765328359/Additional pre-authentication required [19260] 1645812760.557410: Preauthenticating using KDC method data [19260] 1645812760.557411: Processing preauth types: PA-PK-AS-REQ (16), PA-FX-FAST (136), PA-ETYPE-INFO2 (19), PA-PKINIT-KX (147), PA-SPAKE (151), PA-ENC-TIMESTAMP (2), PA_AS_FRESHNESS (150), PA-FX-COOKIE (133) [19260] 1645812760.557412: Selected etype info: etype aes256-cts, salt "IPA.WEB1.FIWELLKNOWNANONYMOUS", params "" [19260] 1645812760.557413: Received cookie: MIT1\x00\x00\x00\x01\x8f\xcb\x99\x9c~\xed!^Qj\xa3\x0a\x82~\xe94\x04\x0ck[j=\x08\xd2\x97j'K2\x8f\xa0\xf6\xc3\x89Z@\x8b]\xc3K\xc2h\xfa\xaek\x11\x91y\xc9\xf0\xadG\x13\x9a\xb2\xb6\x1c\x12\xbfr\x0a'Z\xfe\x12\x81\x1a>2\x8c\x1a\xf2\x96\xdc]&qH\x08\x1f\x0d\xc0a{\xe8\xff\xbbF\x9c\x86`\xd6G\xc4*5\xccL\xc1m\xc0\xa7b\x8b]od\xfa*\xd4.bmB\x9d\x92\xb7\xf9($\xa4D\xea\xcd\xc6\xe3p\xac$\xf4 [19260] 1645812760.557414: Preauth module pkinit (147) (info) returned: 0/Success [19260] 1645812760.557415: PKINIT client received freshness token from KDC [19260] 1645812760.557416: Preauth module pkinit (150) (info) returned: 0/Success [19260] 1645812760.557417: PKINIT loading CA certs and CRLs from FILE [19260] 1645812760.557418: PKINIT loading CA certs and CRLs from FILE [19260] 1645812760.557419: PKINIT loading CA certs and CRLs from FILE [19260] 1645812760.557420: PKINIT client computed kdc-req-body checksum 9/B79768B0DAD630709ABFE35C1E2B6FDAB714913D [19260] 1645812760.557422: PKINIT client making DH request [19260] 1645812760.557423: Preauth module pkinit (16) (real) returned: 0/Success [19260] 1645812760.557424: Produced preauth for next request: PA-FX-COOKIE (133), PA-PK-AS-REQ (16) [19260] 1645812760.557425: Sending request (1674 bytes) to IPA.WEB1.FI [19260] 1645812760.557426: Initiating TCP connection to stream 10.20.13.5:88 [19260] 1645812760.557427: Sending TCP request to stream 10.20.13.5:88 [19260] 1645812760.557428: Received answer (2619 bytes) from stream 10.20.13.5:88 [19260] 1645812760.557429: Terminating TCP connection to stream 10.20.13.5:88 [19260] 1645812760.557430: Response was from primary KDC [19260] 1645812760.557431: Processing preauth types: PA-PK-AS-REP (17), PA-PKINIT-KX (147) [19260] 1645812760.557432: Preauth module pkinit (147) (info) returned: 0/Success [19260] 1645812760.557433: PKINIT OpenSSL error: Failed to verify CMS message [19260] 1645812760.557434: PKINIT OpenSSL error: error:1700006B:CMS routines::content type not enveloped data [19260] 1645812760.557435: PKINIT OpenSSL error: error:03000098:digital envelope routines::invalid digest [19260] 1645812760.557436: PKINIT client could not verify DH reply [19260] 1645812760.557437: Preauth module pkinit (17) (real) returned: -1765328320/Failed to verify CMS message: content type not enveloped data [19260] 1645812760.557438: Produced preauth for next request: (empty) [19260] 1645812760.557439: Getting AS key, salt "IPA.WEB1.FIWELLKNOWNANONYMOUS", params "" Password for WELLKNOWN/ANONYMOUS@IPA.WEB1.FI: [19260] 1645812776.928337: AS key obtained from gak_fct: aes256-cts/3840 kinit: Password incorrect while getting initial credentials
But this only happens on the "dc2" one. If I would run this on the "dc1" it would work just fine.
I have tried running ipa-pkinit-manage disable ipa-pkinit-manage enable
to regen the cert but it didn't help. Any suggestions / pointers at why the OpenSSL error on the tku-2 is showing up.
Can you narrow down what packages were updated?
rob
So I actually ended up creating a fresh replica with the latest packages but it would end up 1:1 same state so I am guessing this has something to do with openssl packages?
Here is a list of packages that dnf would upgrade on the tre-1 ipa-server that is currently working and which I would prefer not to upgrade atleast before the tku-2 problem is solved.
389-ds-base.x86_64 2.0.14-1.el9 389-ds-base-libs.x86_64 2.0.14-1.el9 NetworkManager.x86_64 1:1.36.0-0.9.el9 NetworkManager-libnm.x86_64 1:1.36.0-0.9.el9 NetworkManager-team.x86_64 1:1.36.0-0.9.el9 NetworkManager-tui.x86_64 1:1.36.0-0.9.el9 audit.x86_64 3.0.7-101.el9 audit-libs.x86_64 3.0.7-101.el9 bash.x86_64 5.1.8-4.el9 cairo.x86_64 1.17.4-7.el9 cairo-gobject.x86_64 1.17.4-7.el9 centos-gpg-keys.noarch 9.0-9.el9 centos-stream-release.noarch 9.0-9.el9 centos-stream-repos.noarch 9.0-9.el9 crypto-policies.noarch 20220203-1.gitf03e75e.el9 crypto-policies-scripts.noarch 20220203-1.gitf03e75e.el9 cryptsetup-libs.x86_64 2.4.3-4.el9 cups-libs.x86_64 1:2.3.3op2-11.el9 cyrus-sasl-gssapi.x86_64 2.1.27-19.el9 cyrus-sasl-lib.x86_64 2.1.27-19.el9 cyrus-sasl-md5.x86_64 2.1.27-19.el9 cyrus-sasl-plain.x86_64 2.1.27-19.el9 device-mapper.x86_64 9:1.02.183-4.el9 device-mapper-event.x86_64 9:1.02.183-4.el9 device-mapper-event-libs.x86_64 9:1.02.183-4.el9 device-mapper-libs.x86_64 9:1.02.183-4.el9 dnf.noarch 4.10.0-4.el9 dnf-data.noarch 4.10.0-4.el9 dnf-plugins-core.noarch 4.0.24-3.el9 dracut.x86_64 055-30.git20220216.el9 dracut-config-rescue.x86_64 055-30.git20220216.el9 dracut-network.x86_64 055-30.git20220216.el9 dracut-squash.x86_64 055-30.git20220216.el9 e2fsprogs.x86_64 1.46.5-2.el9 e2fsprogs-libs.x86_64 1.46.5-2.el9 expat.x86_64 2.2.10-9.el9 gawk.x86_64 5.1.0-6.el9 gawk-all-langpacks.x86_64 5.1.0-6.el9 glib2.x86_64 2.68.4-5.el9 glibc.x86_64 2.34-25.el9 glibc-common.x86_64 2.34-25.el9 glibc-gconv-extra.x86_64 2.34-25.el9 glibc-langpack-en.x86_64 2.34-25.el9 gnutls.x86_64 3.7.3-6.el9 gpgme.x86_64 1.15.1-6.el9 grub2-common.noarch 1:2.06-23.el9 grub2-pc.x86_64 1:2.06-23.el9 grub2-pc-modules.noarch 1:2.06-23.el9 grub2-tools.x86_64 1:2.06-23.el9 grub2-tools-minimal.x86_64 1:2.06-23.el9 grubby.x86_64 8.40-55.el9 gtk-update-icon-cache.x86_64 3.24.31-2.el9 gtk3.x86_64 3.24.31-2.el9 hwdata.noarch 0.348-9.3.el9 initscripts-service.noarch 10.11.2-1.el9 ipa-client.x86_64 4.9.8-5.el9 ipa-client-common.noarch 4.9.8-5.el9 ipa-common.noarch 4.9.8-5.el9 ipa-selinux.noarch 4.9.8-5.el9 ipa-server.x86_64 4.9.8-5.el9 ipa-server-common.noarch 4.9.8-5.el9 ipa-server-dns.noarch 4.9.8-5.el9 iptables-libs.x86_64 1.8.7-28.el9 iptables-nft.x86_64 1.8.7-28.el9 iwl100-firmware.noarch 39.31.5.1-124.el9 iwl1000-firmware.noarch 1:39.31.5.1-124.el9 iwl105-firmware.noarch 18.168.6.1-124.el9 iwl135-firmware.noarch 18.168.6.1-124.el9 iwl2000-firmware.noarch 18.168.6.1-124.el9 iwl2030-firmware.noarch 18.168.6.1-124.el9 iwl3160-firmware.noarch 1:25.30.13.0-124.el9 iwl5000-firmware.noarch 8.83.5.1_1-124.el9 iwl5150-firmware.noarch 8.24.2.2-124.el9 iwl6000g2a-firmware.noarch 18.168.6.1-124.el9 iwl6050-firmware.noarch 41.28.5.1-124.el9 iwl7260-firmware.noarch 1:25.30.13.0-124.el9 jss.x86_64 5.0.3-1.el9 kernel.x86_64 5.14.0-66.el9 kernel-core.x86_64 5.14.0-66.el9 kernel-modules.x86_64 5.14.0-66.el9 kernel-tools.x86_64 5.14.0-66.el9 kernel-tools-libs.x86_64 5.14.0-66.el9 kexec-tools.x86_64 2.0.23-8.el9 kpartx.x86_64 0.8.7-7.el9 libarchive.x86_64 3.5.3-1.el9 libcap.x86_64 2.48-8.el9 libcap-ng.x86_64 0.8.2-7.el9 libcap-ng-python3.x86_64 0.8.2-7.el9 libcom_err.x86_64 1.46.5-2.el9 libdnf.x86_64 0.65.0-3.el9 libgcc.x86_64 11.2.1-9.4.el9 libgcrypt.x86_64 1.10.0-2.el9 libgomp.x86_64 11.2.1-9.4.el9 libipa_hbac.x86_64 2.6.2-2.el9 libnfsidmap.x86_64 1:2.5.4-9.el9 libnsl2.x86_64 2.0.0-1.el9 libsemanage.x86_64 3.3-2.el9 libss.x86_64 1.46.5-2.el9 libsss_autofs.x86_64 2.6.2-2.el9 libsss_certmap.x86_64 2.6.2-2.el9 libsss_idmap.x86_64 2.6.2-2.el9 libsss_nss_idmap.x86_64 2.6.2-2.el9 libsss_sudo.x86_64 2.6.2-2.el9 libstdc++.x86_64 11.2.1-9.4.el9 libsysfs.x86_64 2.1.1-10.el9 libtool-ltdl.x86_64 2.4.6-45.el9 libusbx.x86_64 1.0.25-2.el9 libwbclient.x86_64 4.15.5-104.el9 libxslt.x86_64 1.1.34-9.el9 libzstd.x86_64 1.5.1-2.el9 linux-firmware.noarch 20211216-124.el9 linux-firmware-whence.noarch 20211216-124.el9 lua-posix.x86_64 35.0-6.el9 lvm2.x86_64 9:2.03.14-4.el9 lvm2-libs.x86_64 9:2.03.14-4.el9 microcode_ctl.noarch 4:20220207-1.el9 nfs-utils.x86_64 1:2.5.4-9.el9 nspr.x86_64 4.32.0-9.el9 nss.x86_64 3.71.0-7.el9 nss-softokn.x86_64 3.71.0-7.el9 nss-softokn-freebl.x86_64 3.71.0-7.el9 nss-sysinit.x86_64 3.71.0-7.el9 nss-tools.x86_64 3.71.0-7.el9 nss-util.x86_64 3.71.0-7.el9 oddjob.x86_64 0.34.7-5.el9 oddjob-mkhomedir.x86_64 0.34.7-5.el9 opencryptoki.x86_64 3.17.0-4.el9 opencryptoki-icsftok.x86_64 3.17.0-4.el9 opencryptoki-libs.x86_64 3.17.0-4.el9 openssh.x86_64 8.7p1-8.el9 openssh-clients.x86_64 8.7p1-8.el9 openssh-server.x86_64 8.7p1-8.el9 openssl.x86_64 1:3.0.1-12.el9 openssl-libs.x86_64 1:3.0.1-12.el9 openssl-perl.x86_64 1:3.0.1-12.el9 p11-kit.x86_64 0.24.1-2.el9 p11-kit-trust.x86_64 0.24.1-2.el9 perl-Net-SSLeay.x86_64 1.92-1.el9 policycoreutils.x86_64 3.3-5.el9 policycoreutils-python-utils.noarch 3.3-5.el9 protobuf-c.x86_64 1.3.3-12.el9 publicsuffix-list.noarch 20210518-3.el9 publicsuffix-list-dafsa.noarch 20210518-3.el9 python-unversioned-command.noarch 3.9.10-2.el9 python3.x86_64 3.9.10-2.el9 python3-audit.x86_64 3.0.7-101.el9 python3-cffi.x86_64 1.14.5-5.el9 python3-chardet.noarch 4.0.0-5.el9 python3-cryptography.x86_64 3.4.7-8.el9 python3-dnf.noarch 4.10.0-4.el9 python3-dnf-plugins-core.noarch 4.0.24-3.el9 python3-gpg.x86_64 1.15.1-6.el9 python3-hawkey.x86_64 0.65.0-3.el9 python3-idna.noarch 2.10-7.el9 python3-ipaclient.noarch 4.9.8-5.el9 python3-ipalib.noarch 4.9.8-5.el9 python3-ipaserver.noarch 4.9.8-5.el9 python3-lib389.noarch 2.0.14-1.el9 python3-libdnf.x86_64 0.65.0-3.el9 python3-libipa_hbac.x86_64 2.6.2-2.el9 python3-libs.x86_64 3.9.10-2.el9 python3-libsemanage.x86_64 3.3-2.el9 python3-lxml.x86_64 4.6.5-2.el9 python3-mod_wsgi.x86_64 4.7.1-9.el9 python3-pip-wheel.noarch 21.2.3-6.el9 python3-ply.noarch 3.11-14.el9 python3-policycoreutils.noarch 3.3-5.el9 python3-psutil.x86_64 5.8.0-12.el9 python3-pycparser.noarch 2.20-6.el9 python3-pysocks.noarch 1.7.1-12.el9 python3-pyyaml.x86_64 5.4.1-6.el9 python3-requests.noarch 2.25.1-6.el9 python3-rpm.x86_64 4.16.1.3-11.el9 python3-setuptools.noarch 53.0.0-10.el9 python3-setuptools-wheel.noarch 53.0.0-10.el9 python3-six.noarch 1.15.0-9.el9 python3-sss.x86_64 2.6.2-2.el9 python3-sss-murmur.x86_64 2.6.2-2.el9 python3-sssdconfig.noarch 2.6.2-2.el9 python3-unbound.x86_64 1.13.1-12.el9 python3-urllib3.noarch 1.26.5-3.el9 qemu-guest-agent.x86_64 17:6.2.0-9.el9 rpm.x86_64 4.16.1.3-11.el9 rpm-build-libs.x86_64 4.16.1.3-11.el9 rpm-libs.x86_64 4.16.1.3-11.el9 rpm-plugin-audit.x86_64 4.16.1.3-11.el9 rpm-plugin-selinux.x86_64 4.16.1.3-11.el9 rpm-plugin-systemd-inhibit.x86_64 4.16.1.3-11.el9 rpm-sign-libs.x86_64 4.16.1.3-11.el9 samba-client-libs.x86_64 4.15.5-104.el9 samba-common.noarch 4.15.5-104.el9 samba-common-libs.x86_64 4.15.5-104.el9 selinux-policy.noarch 34.1.26-1.el9 selinux-policy-targeted.noarch 34.1.26-1.el9 sg3_utils.x86_64 1.47-8.el9 sg3_utils-libs.x86_64 1.47-8.el9 slapi-nis.x86_64 0.56.7-4.el9 sssd-client.x86_64 2.6.2-2.el9 sssd-common.x86_64 2.6.2-2.el9 sssd-common-pac.x86_64 2.6.2-2.el9 sssd-dbus.x86_64 2.6.2-2.el9 sssd-ipa.x86_64 2.6.2-2.el9 sssd-kcm.x86_64 2.6.2-2.el9 sssd-krb5-common.x86_64 2.6.2-2.el9 sssd-nfs-idmap.x86_64 2.6.2-2.el9 sssd-tools.x86_64 2.6.2-2.el9 systemd.x86_64 250-3.el9 systemd-libs.x86_64 250-3.el9 systemd-pam.x86_64 250-3.el9 systemd-rpm-macros.noarch 250-3.el9 systemd-udev.x86_64 250-3.el9 tpm2-tss.x86_64 3.0.3-7.el9 unbound-libs.x86_64 1.13.1-12.el9 vim-minimal.x86_64 2:8.2.2637-14.el9 xfsprogs.x86_64 5.14.2-1.el9 yum.noarch 4.10.0-4.el9
Ok, after trying the obvious myself I think I have figure out what was causing the issue.
"dnf upgrade" upgraded openssl version from 3.0.1-1.el9 to 3.0.1-12.el9 and the problem started after that.
After I downgraded the openssl back to 3.0.1-1.el9 and restarted Kerberos (systemctl restart krb5kdc.service) the problem went away.
So it would seem the openssl is the root cause. Should I create a bug report or something like that? (not really familiar with the process regarding this)
On la, 26 helmi 2022, Ville Teronen via FreeIPA-users wrote:
Ok, after trying the obvious myself I think I have figure out what was causing the issue.
"dnf upgrade" upgraded openssl version from 3.0.1-1.el9 to 3.0.1-12.el9 and the problem started after that.
After I downgraded the openssl back to 3.0.1-1.el9 and restarted Kerberos (systemctl restart krb5kdc.service) the problem went away.
So it would seem the openssl is the root cause. Should I create a bug report or something like that? (not really familiar with the process regarding this
I assume you are using CentOS 9 Stream, right? RHEL 9 is going to disable SHA-1 globally and openssl 3.0.1-12.el9 is one of the starting points. More so in FIPS mode (I hope you have no need for that, definitely not a good idea in CentOS 9 Stream right now). The output below is what I have on Fedora 35, for the reference.
Can you give me output of
[root@dc ~]# update-crypto-policies --show DEFAULT
[root@dc ~]# cat /etc/krb5.conf.d/crypto-policies [libdefaults] permitted_enctypes = aes256-cts-hmac-sha1-96 aes256-cts-hmac-sha384-192 camellia256-cts-cmac aes128-cts-hmac-sha256-128 aes128-cts-hmac-sha1-96 camellia128-cts-cmac
and
[root@dc ~]# getcert list -f /var/kerberos/krb5kdc/kdc.crt Number of certificates and requests being tracked: 9. Request ID '20220128174302': status: MONITORING stuck: no key pair storage: type=FILE,location='/var/kerberos/krb5kdc/kdc.key' certificate: type=FILE,location='/var/kerberos/krb5kdc/kdc.crt' CA: IPA issuer: CN=Certificate Authority,O=IPA.TEST subject: CN=dc.ipa.test,O=IPA.TEST issued: 2022-01-28 17:43:02 UTC expires: 2024-01-29 17:43:02 UTC dns: dc.ipa.test principal name: krbtgt/IPA.TEST@IPA.TEST key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-pkinit-KPKdc profile: KDCs_PKINIT_Certs pre-save command: post-save command: /usr/libexec/ipa/certmonger/renew_kdc_cert track: yes auto-renew: yes
It should look like the above if you are using IPA CA. Specifically, check that issuer, subject, principal name and EKUs are correct, but the rest is important as well.
If you are not using IPA CA, then there might be other issues too, but I need to see the output.
For SHA-1 issues, if I'm right, you may want to fix that with
# update-crypto-policies --set DEFAULT:SHA1
This would restore use of SHA-1 HMAC for Kerberos.
Hello,
Here's the output for these commands
[root@ipa ~]# update-crypto-policies --show DEFAULT [root@ipa ~]# cat /etc/krb5.conf.d/crypto-policies [libdefaults] permitted_enctypes = aes256-cts-hmac-sha1-96 aes256-cts-hmac-sha384-192 aes128-cts-hmac-sha256-128 aes128-cts-hmac-sha1-96 [root@ipa ~]# getcert list -f /var/kerberos/krb5kdc/kdc.crt Number of certificates and requests being tracked: 9. Request ID '20220225155300': status: MONITORING stuck: no key pair storage: type=FILE,location='/var/kerberos/krb5kdc/kdc.key' certificate: type=FILE,location='/var/kerberos/krb5kdc/kdc.crt' CA: IPA issuer: CN=Certificate Authority,O=IPA.WEB1.FI subject: CN=ipa.tku-2.web1.fi,O=IPA.WEB1.FI issued: 2022-02-25 17:53:01 EET expires: 2024-02-26 17:53:01 EET dns: ipa.tku-2.web1.fi principal name: krbtgt/IPA.WEB1.FI@IPA.WEB1.FI key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-pkinit-KPKdc profile: KDCs_PKINIT_Certs pre-save command: post-save command: /usr/libexec/ipa/certmonger/renew_kdc_cert track: yes auto-renew: yes
This is run on a updated openssl version. it's 1:1 with the earlier "working openssl" if I compare the output to our tre-1 ipa server (ofc. CN is pointing elsewhere there, but ciphers etc)
Hi Ville,
it seems the same problem I faced some months ago with one of three nodes. I searched for the issues and find a possible trouble with server certificate after an upgrade (the field SubjectAltName is not created; on the other )
try to:
1) check if the attribute is filled: (CLI) `openssl x509 -in /var/kerberos/krb5kdc/kdc.crt -noout -text` (the value can even be <unsupported> so it was on the other two nodes where I managed to login (web GUI) 2) getcert list -f /var/kerberos/krb5kdc/kdc.crt (*note the request Id* <request id>) 3) getcert resubmit -i <request id> 4) getcert list -i <request id> 5) ipactl restart
cheers Stefano
On 2/26/22 10:23, Ville Teronen via FreeIPA-users wrote:
So I actually ended up creating a fresh replica with the latest packages but it would end up 1:1 same state so I am guessing this has something to do with openssl packages?
Here is a list of packages that dnf would upgrade on the tre-1 ipa-server that is currently working and which I would prefer not to upgrade atleast before the tku-2 problem is solved.
389-ds-base.x86_64 2.0.14-1.el9 389-ds-base-libs.x86_64 2.0.14-1.el9 NetworkManager.x86_64 1:1.36.0-0.9.el9 NetworkManager-libnm.x86_64 1:1.36.0-0.9.el9 NetworkManager-team.x86_64 1:1.36.0-0.9.el9 NetworkManager-tui.x86_64 1:1.36.0-0.9.el9 audit.x86_64 3.0.7-101.el9 audit-libs.x86_64 3.0.7-101.el9 bash.x86_64 5.1.8-4.el9 cairo.x86_64 1.17.4-7.el9 cairo-gobject.x86_64 1.17.4-7.el9 centos-gpg-keys.noarch 9.0-9.el9 centos-stream-release.noarch 9.0-9.el9 centos-stream-repos.noarch 9.0-9.el9 crypto-policies.noarch 20220203-1.gitf03e75e.el9 crypto-policies-scripts.noarch 20220203-1.gitf03e75e.el9 cryptsetup-libs.x86_64 2.4.3-4.el9 cups-libs.x86_64 1:2.3.3op2-11.el9 cyrus-sasl-gssapi.x86_64 2.1.27-19.el9 cyrus-sasl-lib.x86_64 2.1.27-19.el9 cyrus-sasl-md5.x86_64 2.1.27-19.el9 cyrus-sasl-plain.x86_64 2.1.27-19.el9 device-mapper.x86_64 9:1.02.183-4.el9 device-mapper-event.x86_64 9:1.02.183-4.el9 device-mapper-event-libs.x86_64 9:1.02.183-4.el9 device-mapper-libs.x86_64 9:1.02.183-4.el9 dnf.noarch 4.10.0-4.el9 dnf-data.noarch 4.10.0-4.el9 dnf-plugins-core.noarch 4.0.24-3.el9 dracut.x86_64 055-30.git20220216.el9 dracut-config-rescue.x86_64 055-30.git20220216.el9 dracut-network.x86_64 055-30.git20220216.el9 dracut-squash.x86_64 055-30.git20220216.el9 e2fsprogs.x86_64 1.46.5-2.el9 e2fsprogs-libs.x86_64 1.46.5-2.el9 expat.x86_64 2.2.10-9.el9 gawk.x86_64 5.1.0-6.el9 gawk-all-langpacks.x86_64 5.1.0-6.el9 glib2.x86_64 2.68.4-5.el9 glibc.x86_64 2.34-25.el9 glibc-common.x86_64 2.34-25.el9 glibc-gconv-extra.x86_64 2.34-25.el9 glibc-langpack-en.x86_64 2.34-25.el9 gnutls.x86_64 3.7.3-6.el9 gpgme.x86_64 1.15.1-6.el9 grub2-common.noarch 1:2.06-23.el9 grub2-pc.x86_64 1:2.06-23.el9 grub2-pc-modules.noarch 1:2.06-23.el9 grub2-tools.x86_64 1:2.06-23.el9 grub2-tools-minimal.x86_64 1:2.06-23.el9 grubby.x86_64 8.40-55.el9 gtk-update-icon-cache.x86_64 3.24.31-2.el9 gtk3.x86_64 3.24.31-2.el9 hwdata.noarch 0.348-9.3.el9 initscripts-service.noarch 10.11.2-1.el9 ipa-client.x86_64 4.9.8-5.el9 ipa-client-common.noarch 4.9.8-5.el9 ipa-common.noarch 4.9.8-5.el9 ipa-selinux.noarch 4.9.8-5.el9 ipa-server.x86_64 4.9.8-5.el9 ipa-server-common.noarch 4.9.8-5.el9 ipa-server-dns.noarch 4.9.8-5.el9 iptables-libs.x86_64 1.8.7-28.el9 iptables-nft.x86_64 1.8.7-28.el9 iwl100-firmware.noarch 39.31.5.1-124.el9 iwl1000-firmware.noarch 1:39.31.5.1-124.el9 iwl105-firmware.noarch 18.168.6.1-124.el9 iwl135-firmware.noarch 18.168.6.1-124.el9 iwl2000-firmware.noarch 18.168.6.1-124.el9 iwl2030-firmware.noarch 18.168.6.1-124.el9 iwl3160-firmware.noarch 1:25.30.13.0-124.el9 iwl5000-firmware.noarch 8.83.5.1_1-124.el9 iwl5150-firmware.noarch 8.24.2.2-124.el9 iwl6000g2a-firmware.noarch 18.168.6.1-124.el9 iwl6050-firmware.noarch 41.28.5.1-124.el9 iwl7260-firmware.noarch 1:25.30.13.0-124.el9 jss.x86_64 5.0.3-1.el9 kernel.x86_64 5.14.0-66.el9 kernel-core.x86_64 5.14.0-66.el9 kernel-modules.x86_64 5.14.0-66.el9 kernel-tools.x86_64 5.14.0-66.el9 kernel-tools-libs.x86_64 5.14.0-66.el9 kexec-tools.x86_64 2.0.23-8.el9 kpartx.x86_64 0.8.7-7.el9 libarchive.x86_64 3.5.3-1.el9 libcap.x86_64 2.48-8.el9 libcap-ng.x86_64 0.8.2-7.el9 libcap-ng-python3.x86_64 0.8.2-7.el9 libcom_err.x86_64 1.46.5-2.el9 libdnf.x86_64 0.65.0-3.el9 libgcc.x86_64 11.2.1-9.4.el9 libgcrypt.x86_64 1.10.0-2.el9 libgomp.x86_64 11.2.1-9.4.el9 libipa_hbac.x86_64 2.6.2-2.el9 libnfsidmap.x86_64 1:2.5.4-9.el9 libnsl2.x86_64 2.0.0-1.el9 libsemanage.x86_64 3.3-2.el9 libss.x86_64 1.46.5-2.el9 libsss_autofs.x86_64 2.6.2-2.el9 libsss_certmap.x86_64 2.6.2-2.el9 libsss_idmap.x86_64 2.6.2-2.el9 libsss_nss_idmap.x86_64 2.6.2-2.el9 libsss_sudo.x86_64 2.6.2-2.el9 libstdc++.x86_64 11.2.1-9.4.el9 libsysfs.x86_64 2.1.1-10.el9 libtool-ltdl.x86_64 2.4.6-45.el9 libusbx.x86_64 1.0.25-2.el9 libwbclient.x86_64 4.15.5-104.el9 libxslt.x86_64 1.1.34-9.el9 libzstd.x86_64 1.5.1-2.el9 linux-firmware.noarch 20211216-124.el9 linux-firmware-whence.noarch 20211216-124.el9 lua-posix.x86_64 35.0-6.el9 lvm2.x86_64 9:2.03.14-4.el9 lvm2-libs.x86_64 9:2.03.14-4.el9 microcode_ctl.noarch 4:20220207-1.el9 nfs-utils.x86_64 1:2.5.4-9.el9 nspr.x86_64 4.32.0-9.el9 nss.x86_64 3.71.0-7.el9 nss-softokn.x86_64 3.71.0-7.el9 nss-softokn-freebl.x86_64 3.71.0-7.el9 nss-sysinit.x86_64 3.71.0-7.el9 nss-tools.x86_64 3.71.0-7.el9 nss-util.x86_64 3.71.0-7.el9 oddjob.x86_64 0.34.7-5.el9 oddjob-mkhomedir.x86_64 0.34.7-5.el9 opencryptoki.x86_64 3.17.0-4.el9 opencryptoki-icsftok.x86_64 3.17.0-4.el9 opencryptoki-libs.x86_64 3.17.0-4.el9 openssh.x86_64 8.7p1-8.el9 openssh-clients.x86_64 8.7p1-8.el9 openssh-server.x86_64 8.7p1-8.el9 openssl.x86_64 1:3.0.1-12.el9 openssl-libs.x86_64 1:3.0.1-12.el9 openssl-perl.x86_64 1:3.0.1-12.el9 p11-kit.x86_64 0.24.1-2.el9 p11-kit-trust.x86_64 0.24.1-2.el9 perl-Net-SSLeay.x86_64 1.92-1.el9 policycoreutils.x86_64 3.3-5.el9 policycoreutils-python-utils.noarch 3.3-5.el9 protobuf-c.x86_64 1.3.3-12.el9 publicsuffix-list.noarch 20210518-3.el9 publicsuffix-list-dafsa.noarch 20210518-3.el9 python-unversioned-command.noarch 3.9.10-2.el9 python3.x86_64 3.9.10-2.el9 python3-audit.x86_64 3.0.7-101.el9 python3-cffi.x86_64 1.14.5-5.el9 python3-chardet.noarch 4.0.0-5.el9 python3-cryptography.x86_64 3.4.7-8.el9 python3-dnf.noarch 4.10.0-4.el9 python3-dnf-plugins-core.noarch 4.0.24-3.el9 python3-gpg.x86_64 1.15.1-6.el9 python3-hawkey.x86_64 0.65.0-3.el9 python3-idna.noarch 2.10-7.el9 python3-ipaclient.noarch 4.9.8-5.el9 python3-ipalib.noarch 4.9.8-5.el9 python3-ipaserver.noarch 4.9.8-5.el9 python3-lib389.noarch 2.0.14-1.el9 python3-libdnf.x86_64 0.65.0-3.el9 python3-libipa_hbac.x86_64 2.6.2-2.el9 python3-libs.x86_64 3.9.10-2.el9 python3-libsemanage.x86_64 3.3-2.el9 python3-lxml.x86_64 4.6.5-2.el9 python3-mod_wsgi.x86_64 4.7.1-9.el9 python3-pip-wheel.noarch 21.2.3-6.el9 python3-ply.noarch 3.11-14.el9 python3-policycoreutils.noarch 3.3-5.el9 python3-psutil.x86_64 5.8.0-12.el9 python3-pycparser.noarch 2.20-6.el9 python3-pysocks.noarch 1.7.1-12.el9 python3-pyyaml.x86_64 5.4.1-6.el9 python3-requests.noarch 2.25.1-6.el9 python3-rpm.x86_64 4.16.1.3-11.el9 python3-setuptools.noarch 53.0.0-10.el9 python3-setuptools-wheel.noarch 53.0.0-10.el9 python3-six.noarch 1.15.0-9.el9 python3-sss.x86_64 2.6.2-2.el9 python3-sss-murmur.x86_64 2.6.2-2.el9 python3-sssdconfig.noarch 2.6.2-2.el9 python3-unbound.x86_64 1.13.1-12.el9 python3-urllib3.noarch 1.26.5-3.el9 qemu-guest-agent.x86_64 17:6.2.0-9.el9 rpm.x86_64 4.16.1.3-11.el9 rpm-build-libs.x86_64 4.16.1.3-11.el9 rpm-libs.x86_64 4.16.1.3-11.el9 rpm-plugin-audit.x86_64 4.16.1.3-11.el9 rpm-plugin-selinux.x86_64 4.16.1.3-11.el9 rpm-plugin-systemd-inhibit.x86_64 4.16.1.3-11.el9 rpm-sign-libs.x86_64 4.16.1.3-11.el9 samba-client-libs.x86_64 4.15.5-104.el9 samba-common.noarch 4.15.5-104.el9 samba-common-libs.x86_64 4.15.5-104.el9 selinux-policy.noarch 34.1.26-1.el9 selinux-policy-targeted.noarch 34.1.26-1.el9 sg3_utils.x86_64 1.47-8.el9 sg3_utils-libs.x86_64 1.47-8.el9 slapi-nis.x86_64 0.56.7-4.el9 sssd-client.x86_64 2.6.2-2.el9 sssd-common.x86_64 2.6.2-2.el9 sssd-common-pac.x86_64 2.6.2-2.el9 sssd-dbus.x86_64 2.6.2-2.el9 sssd-ipa.x86_64 2.6.2-2.el9 sssd-kcm.x86_64 2.6.2-2.el9 sssd-krb5-common.x86_64 2.6.2-2.el9 sssd-nfs-idmap.x86_64 2.6.2-2.el9 sssd-tools.x86_64 2.6.2-2.el9 systemd.x86_64 250-3.el9 systemd-libs.x86_64 250-3.el9 systemd-pam.x86_64 250-3.el9 systemd-rpm-macros.noarch 250-3.el9 systemd-udev.x86_64 250-3.el9 tpm2-tss.x86_64 3.0.3-7.el9 unbound-libs.x86_64 1.13.1-12.el9 vim-minimal.x86_64 2:8.2.2637-14.el9 xfsprogs.x86_64 5.14.2-1.el9 yum.noarch 4.10.0-4.el9 _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Hi,
SubjectAltName was included in the cert so this seems unrelated. I still tested it but it didn't have effect on the outcome.
From what I can gather if I upgrade the openssl package to 3.0.1-12.el9 it stops working. The latest version that I can use is 3.0.1-5.el9
Hello Ville,
Could you please check which hashing algorithm was used to generate the signature of your KDC certificate?
To do so, please run this command:
openssl x509 -in /var/kerberos/krb5kdc/kdc.crt -noout -text | grep -Fi 'Signature Algorithm'
-- Julien Rische Software Engineer Red Hat
On ma, 28 helmi 2022, Julien Rische via FreeIPA-users wrote:
Hello Ville,
Could you please check which hashing algorithm was used to generate the signature of your KDC certificate?
To do so, please run this command:
openssl x509 -in /var/kerberos/krb5kdc/kdc.crt -noout -text | grep -Fi 'Signature Algorithm'
Folks from OpenStack side did open a bug https://bugzilla.redhat.com/show_bug.cgi?id=2060798 for the same issue.
I've done a bit of investigation to where this issue might be, please see in the bug.
These steps worked for me after trying pretty much everything.
My FreeIPA cert has expired, I have initially used ipa-cert-fix, then ipa-certupdate. It worked for like 2-3 days then stopped working.
After doing the getcert resubmit -i “certid” and restarting IPA, everything started working again.
OS: RockyLinux 8.10 OpenSSL version: openssl-1.1.1k-12.el8_9.x86_64
Tried to downgrade OpenSSL (previously it was openssl-1.1.1k-16.el8_9.x86_64) but that didn’t work
Bogdan Stoica via FreeIPA-users wrote:
These steps worked for me after trying pretty much everything.
My FreeIPA cert has expired, I have initially used ipa-cert-fix, then ipa-certupdate. It worked for like 2-3 days then stopped working.
After doing the getcert resubmit -i “certid” and restarting IPA, everything started working again.
OS: RockyLinux 8.10 OpenSSL version: openssl-1.1.1k-12.el8_9.x86_64
Tried to downgrade OpenSSL (previously it was openssl-1.1.1k-16.el8_9.x86_64) but that didn’t work
I'm glad you got it working.
As a shortcut you can also resubmit using path options, like getcert resubmit -f /var/kerberos/krb5kdc/kdc.crt
I like including the -vw flags. v will give status as the cert processes through the reneal and w will wait until the request is done, either pass or fail.
It seems unusual that after ipa-certfix the cert only worked for a couple of days but I'm not sure what we could do to troubleshoot that now. I guess I'm glad it's working.
You might want to double-check the Not after date of the certificate to be sure it isn't going to expire again soon. Just in case.
rob
freeipa-users@lists.fedorahosted.org
-
Alexander Bokovoy -
antonelli＠cnaf -
Bogdan Stoica -
Julien Rische -
Rob Crittenden -
Ville Teronen