Last week I ran the command:
> [root@utility ~]# ipa-certupdate
>
> cannot connect to 'https://utility.idm.nac-issa.org/ipa/json': [SSL:
> CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)
> The ipa-certupdate command failed.

We tested the root CA cert and server cert.  Both were found to be good.  Is it possible I'd need to chain the two together? 
Which certificate is the error message complaining about?  What directory?

Update-ca-trust had no affect.  Is this because of the above error message?

From: Jeremy Tourville <jeremy_tourville@hotmail.com>
Sent: Friday, September 10, 2021 3:41 PM
To: FreeIPA users list <freeipa-users@lists.fedorahosted.org>
Cc: Florence Renaud <flo@redhat.com>; Rob Crittenden <rcritten@redhat.com>
Subject: Re: [Freeipa-users] Re: Why is ipa-ods-exporter broken after running ipa-dns-install? (Was - Unable to start directory server after updates)
 
ACK, the typo was why certutil failed initially.  It works now.

>>>So update-ca-trust had no affect or was this run beforehand?

Update-ca-trust had no affect.  it was run after doing the ipa-cacert-mange renew .

>>>You didn't happen to touch /etc/httpd/conf.d/ssl.conf did you?
No, I left it alone

>>>This renews the CA certificate. The CA is good for 20 years, you didn't
need to do this.
 ACK

>>>We now have another CA certificate for IPA in the mix because of the
renewal.

OK, I'll stand by so I don't really mess it up.




From: Rob Crittenden <rcritten@redhat.com>
Sent: Friday, September 10, 2021 3:26 PM
To: FreeIPA users list <freeipa-users@lists.fedorahosted.org>
Cc: Florence Renaud <flo@redhat.com>; Jeremy Tourville <jeremy_tourville@hotmail.com>
Subject: Re: [Freeipa-users] Re: Why is ipa-ods-exporter broken after running ipa-dns-install? (Was - Unable to start directory server after updates)
 
Jeremy Tourville via FreeIPA-users wrote:
> I was doing some reading and troubleshooting
>
> https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/linux_domain_identity_authentication_and_policy_guide/cert-renewal#manual-cert-renewal
>
> which basically says:
> #1 ipa-cacert-manage renew
> #2 ipa-certupdate
> #3 certutil -L -d /etc/pki/pki-tomcat/alias (to test the certs)
>
> See my output.  Step #1 and #3 work now but #2 still fails
>
>
> [root@utility certs]# ipa-certupdate
>
> cannot connect to 'https://utility.idm.nac-issa.org/ipa/json': [SSL:
> CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)
> The ipa-certupdate command failed.

So update-ca-trust had no affect or was this run beforehand?

> [root@utility certs]# certutil -L -d /etc/pki/pik-tomcat/alias
>
> certutil: function failed: SEC_ERROR_BAD_DATABASE: security library: bad
> database.

It failed because of a typo, pik -> pki.

> [root@utility certs]# ipa-cacert-manage renew
>
> Renewing CA certificate, please wait
> CA certificate successfully renewed
> The ipa-cacert-manage command was successful

This renews the CA certificate. The CA is good for 20 years, you didn't
need to do this.

> [root@utility certs]# ipa-certupdate
>
> cannot connect to 'https://utility.idm.nac-issa.org/ipa/json': [SSL:
> CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)
> The ipa-certupdate command failed.

We now have another CA certificate for IPA in the mix because of the
renewal.

>
> [root@utility certs]# certutil -L -d /etc/pki/pki-tomcat/alias
>
> Certificate Nickname                                         Trust
> Attributes
>                                                            
>  SSL,S/MIME,JAR/XPI
>
> ocspSigningCert cert-pki-ca                                  u,u,u
> subsystemCert cert-pki-ca                                    u,u,u
> auditSigningCert cert-pki-ca                                 u,u,Pu
> Server-Cert cert-pki-ca                                      u,u,u
> caSigningCert cert-pki-ca                                    CTu,Cu,Cu
> IDM.NAC-ISSA.ORG IPA CA                                      CTu,Cu,Cu
> [root@utility certs]# reboot

It isn't a problem with the CA. The system doesn't trust the CA for some
reason, though the openssl command verified that it is ok.

> [root@utility certs]# reboot
>
> [root@utility ~]# ipa-certupdate
>
> cannot connect to 'https://utility.idm.nac-issa.org/ipa/json': [SSL:
> CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)
> The ipa-certupdate command failed.

You didn't happen to touch /etc/httpd/conf.d/ssl.conf did you?

rob

>
> [root@utility ~]# ipactl status
>
> Directory Service: RUNNING
> krb5kdc Service: RUNNING
> kadmin Service: RUNNING
> named Service: RUNNING
> httpd Service: RUNNING
> ipa-custodia Service: RUNNING
> pki-tomcatd Service: RUNNING
> smb Service: RUNNING
> winbind Service: RUNNING
> ipa-otpd Service: RUNNING
> ipa-ods-exporter Service: STOPPED
> ods-enforcerd Service: RUNNING
> ipa-dnskeysyncd Service: RUNNING
> ipa: INFO: The ipactl command was successful
>
> ------------------------------------------------------------------------
> *From:* Rob Crittenden <rcritten@redhat.com>
> *Sent:* Friday, September 10, 2021 9:49 AM
> *To:* Jeremy Tourville <jeremy_tourville@hotmail.com>; FreeIPA users
> list <freeipa-users@lists.fedorahosted.org>
> *Cc:* Florence Renaud <flo@redhat.com>
> *Subject:* Re: [Freeipa-users] Re: Why is ipa-ods-exporter broken after
> running ipa-dns-install? (Was - Unable to start directory server after
> updates)
>  
> Jeremy Tourville wrote:
>> [root@utility certs]# curl https://utility.idm.nac-issa.org/
>> curl: (60) SSL certificate problem: self signed certificate in
>> certificate chain
>> More details here: https://curl.haxx.se/docs/sslcerts.html
>>
>> curl failed to verify the legitimacy of the server and therefore could not
>> establish a secure connection to it. To learn more about this situation and
>> how to fix it, please visit the web page mentioned above.
>>
>> [root@utility certs]# update-ca-trust
>>
>> [root@utility certs]# ausearch -m AVC -ts recent
>> <no matches>
>>
>> [root@utility certs]# ipa-healthcheck
>> -bash: ipa-healthcheck: command not found
>
> I should have mentioned, try the curl after running update-ca-trust.
>
> ipa-healthcheck is not installed by default, you'd need to install the
> {free}ipa-healthcheck package.
>
> rob
>
>>
>>
>>
>> ------------------------------------------------------------------------
>> *From:* Rob Crittenden <rcritten@redhat.com>
>> *Sent:* Friday, September 10, 2021 9:33 AM
>> *To:* Jeremy Tourville <jeremy_tourville@hotmail.com>; FreeIPA users
>> list <freeipa-users@lists.fedorahosted.org>
>> *Cc:* Florence Renaud <flo@redhat.com>
>> *Subject:* Re: [Freeipa-users] Re: Why is ipa-ods-exporter broken after
>> running ipa-dns-install? (Was - Unable to start directory server after
>> updates)
>>  
>> Jeremy Tourville wrote:
>>> [root@utility certs]# ipa-certupdate
>>> cannot connect to 'https://utility.idm.nac-issa.org/ipa/json': [SSL:
>>> CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)
>>> The ipa-certupdate command failed.
>>>
>>> Sort of a bad catch 22 I guess?
>>
>> Yeah, I was afraid of that.
>>
>> Let's walk through it. Try a simple command for another data point. I'm
>> not sure what we'd do with this but it will exercise the system-wide
>> trust as well:
>>
>> $ curl https://`hostname`/
>>
>> Rebuilding the CA trust db may help
>>
>> # update-ca-trust
>>
>> I suppose also look for AVCs in case something is way out-of-whack:
>>
>> # ausearch -m AVC -ts recent
>>
>> ipa-healthcheck may be something to try as well but you're likely to get
>> a crapton of false positives since it can't talk to the web interface.
>>
>> rob
>>
>>>
>>> ------------------------------------------------------------------------
>>> *From:* Rob Crittenden <rcritten@redhat.com>
>>> *Sent:* Friday, September 10, 2021 9:09 AM
>>> *To:* Jeremy Tourville <jeremy_tourville@hotmail.com>; FreeIPA users
>>> list <freeipa-users@lists.fedorahosted.org>
>>> *Cc:* Florence Renaud <flo@redhat.com>
>>> *Subject:* Re: [Freeipa-users] Re: Why is ipa-ods-exporter broken after
>>> running ipa-dns-install? (Was - Unable to start directory server after
>>> updates)
>>>  
>>> Jeremy Tourville wrote:
>>>> Now I understand how to test the cert(s) after re-reading your comments
>>>> Rob and Flo 🙂
>>>>
>>>> [root@utility certs]# openssl verify -verbose -show_chain -CAfile
>>>> /etc/ipa/ca.crt /var/lib/ipa/certs/httpd.crt
>>>> /var/lib/ipa/certs/httpd.crt: OK
>>>> Chain:
>>>> depth=0: O = IDM.NAC-ISSA.ORG, CN = utility.idm.nac-issa.org (untrusted)
>>>> depth=1: O = IDM.NAC-ISSA.ORG, CN = Certificate Authority
>>>
>>> I'd try running ipa-certupdate. I have the feeling some of the
>>> system-wide certificates are out-of-sync.
>>>
>>> rob
>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>> *From:* Jeremy Tourville <jeremy_tourville@hotmail.com>
>>>> *Sent:* Thursday, September 9, 2021 5:45 PM
>>>> *To:* FreeIPA users list <freeipa-users@lists.fedorahosted.org>
>>>> *Cc:* Florence Renaud <flo@redhat.com>; Rob Crittenden <rcritten@redhat.com>
>>>> *Subject:* Re: [Freeipa-users] Re: Why is ipa-ods-exporter broken after
>>>> running ipa-dns-install? (Was - Unable to start directory server after
>>>> updates)
>>>>  
>>>> Oh wait!!! Which set of certs do I need to test against for my
>>>> certificate chain?
>>>> I realized I didn't include the proper path when testing.  It should be
>>>> something like-
>>>>
>>>>  # openssl verify -verbose -show_chain -CAfile <path to root or
>>>> intermediate cert> /etc/ipa/ca.crt
>>>> # openssl verify -verbose -show_chain -CAfile <path to root or
>>>> intermediate cert> /var/lib/ipa/certs/httpd.crt
>>>>
>>>> This would give you output (presuming you are using the correct set of
>>>> certs)
>>>>  /etc/ipa/ca.crt: OK
>>>> /var/lib/ipa/certs/httpd.crt: OK
>>>>
>>>> Which path contains the intermediate or root CA certs I need to test
>>>> against?
>>>>
>>>> [root@utility ~]# ls -la | find / -name *.crt
>>>> /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
>>>> /etc/pki/ca-trust/source/ca-bundle.legacy.crt
>>>> /etc/pki/tls/certs/ca-bundle.crt
>>>> /etc/pki/tls/certs/ca-bundle.trust.crt
>>>> /etc/pki/tls/certs/localhost.crt
>>>> /etc/pki/pki-tomcat/alias/ca.crt
>>>> /etc/ipa/ca.crt
>>>> /etc/dirsrv/ssca/ca.crt
>>>> /etc/dirsrv/slapd-IDM-NAC-ISSA-ORG/Server-Cert.crt
>>>> /etc/dirsrv/slapd-IDM-NAC-ISSA-ORG/ca.crt
>>>> /var/lib/ipa/certs/httpd.crt
>>>> /var/kerberos/krb5kdc/kdc.crt
>>>> /usr/share/pki/ca-trust-legacy/ca-bundle.legacy.default.crt
>>>> /usr/share/pki/ca-trust-legacy/ca-bundle.legacy.disable.crt
>>>> /usr/share/ipa/html/ca.crt
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>> *From:* Jeremy Tourville <jeremy_tourville@hotmail.com>
>>>> *Sent:* Thursday, September 9, 2021 3:13 PM
>>>> *To:* FreeIPA users list <freeipa-users@lists.fedorahosted.org>
>>>> *Cc:* Florence Renaud <flo@redhat.com>; Rob Crittenden <rcritten@redhat.com>
>>>> *Subject:* Re: [Freeipa-users] Re: Why is ipa-ods-exporter broken after
>>>> running ipa-dns-install? (Was - Unable to start directory server after
>>>> updates)
>>>>  
>>>>>>>It isn't complaining that the certificate isn't valid, it's complaining
>>>> that it isn't trusted.
>>>>  Thanksfor pointing out my mistake.  I'm wearing some egg on my face.  I
>>>> was thinking about it wrong at the time of my reply.
>>>>
>>>> I attempted to verify trust-
>>>> [root@utility ipa]# openssl verify -verbose -show_chain -CAfile
>>>> /etc/ipa/ca.crt
>>>> ^C
>>>> [root@utility ipa]# openssl verify -verbose -show_chain -CAfile
>>>> /var/lib/ipa/certs/httpd.crt
>>>> ^C
>>>>
>>>> As you can see, no output, so yeah, they are not trusted.
>>>>
>>>>>>Where did httpd.crt come from/what issuer?
>>>> I recall not using a 3rd party CA.  The certs were just self-signed when
>>>> the ipa server was initially built.  I never did replace the certs as it
>>>> wasn't required for our situation.
>>>>
>>>> Next steps I guess would be to generate some new certs?  Thoughts?
>>>>
>>>> ------------------------------------------------------------------------
>>>> *From:* Rob Crittenden <rcritten@redhat.com>
>>>> *Sent:* Thursday, September 9, 2021 12:53 PM
>>>> *To:* FreeIPA users list <freeipa-users@lists.fedorahosted.org>
>>>> *Cc:* Florence Renaud <flo@redhat.com>; Jeremy Tourville
>>>> <jeremy_tourville@hotmail.com>
>>>> *Subject:* Re: [Freeipa-users] Re: Why is ipa-ods-exporter broken after
>>>> running ipa-dns-install? (Was - Unable to start directory server after
>>>> updates)
>>>>  
>>>> Jeremy Tourville via FreeIPA-users wrote:
>>>>> /var/lib/ipa/certs/httpd.crt
>>>>> looks valid and has a 3 year validity date starting from Nov 23, 2020
>>>>>
>>>>> /etc/ipa/ca.crt
>>>>> looks valid and has a 20 year validity date starting from Nov 23, 2020
>>>>
>>>> It isn't complaining that the certificate isn't valid, it's complaining
>>>> that it isn't trusted. You also need to look at the signer and ensure
>>>> that the system trusts it globally. Where did httpd.crt come from/what
>>>> issuer?
>>>>
>>>> You might try running:
>>>>
>>>> openssl verify -verbose -show_chain -CAfile /etc/ipa/ca.crt
>>>> /var/lib/ipa/certs/httpd.crt
>>>>
>>>> See the default.conf(5) man page for a description of default.conf,
>>>> server.conf, etc. In this case server is a context so the configuration
>>>> only applies there.
>>>>
>>>> rob
>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------
>>>>> *From:* Florence Renaud <flo@redhat.com>
>>>>> *Sent:* Tuesday, September 7, 2021 11:38 AM
>>>>> *To:* Jeremy Tourville <jeremy_tourville@hotmail.com>
>>>>> *Cc:* FreeIPA users list <freeipa-users@lists.fedorahosted.org>
>>>>> *Subject:* Re: [Freeipa-users] Re: Why is ipa-ods-exporter broken after
>>>>> running ipa-dns-install? (Was - Unable to start directory server after
>>>>> updates)
>>>>>  
>>>>> Hi Jeremy,
>>>>>
>>>>> to enable debugging you can simply create /etc/ipa/server.conf if the
>>>>> file does not exist:
>>>>> # cat /etc/ipa/server.conf
>>>>> [global]
>>>>> debug=True
>>>>> # systemctl restart httpd
>>>>>
>>>>> The HTTPd certificate is stored in /var/lib/ipa/certs/httpd.crt, you can
>>>>> examine its content with
>>>>> # openssl x509 -noout -text -in /var/lib/ipa/certs/httpd.crt
>>>>> If the IPA deployment includes an embedded CA, the CA that issued the
>>>>> httpd cert is stored in /etc/ipa/ca.crt and can also be checked with
>>>>> openssl command.
>>>>>
>>>>> flo
>>>>>
>>>>> On Tue, Sep 7, 2021 at 6:09 PM Jeremy Tourville
>>>>> <jeremy_tourville@hotmail.com <mailto:jeremy_tourville@hotmail.com>> wrote:
>>>>>
>>>>>     I think I see the issue but I am unsure what to do to fix it.  See
>>>>>     below.
>>>>>
>>>>>     To answer your question, yes I did accept the security exception.
>>>>>
>>>>>     Also, I don't see a server.conf file at /etc/ipa so that I may
>>>>>     enable debugging.  What can you suggest for this issue?
>>>>>
>>>>>
>>>>>     [root@utility ~]# ipactl status
>>>>>     Directory Service: RUNNING
>>>>>     krb5kdc Service: RUNNING
>>>>>     kadmin Service: RUNNING
>>>>>     named Service: RUNNING
>>>>>     httpd Service: RUNNING
>>>>>     ipa-custodia Service: RUNNING
>>>>>     pki-tomcatd Service: RUNNING
>>>>>     smb Service: RUNNING
>>>>>     winbind Service: RUNNING
>>>>>     ipa-otpd Service: RUNNING
>>>>>     ipa-ods-exporter Service: STOPPED
>>>>>     ods-enforcerd Service: RUNNING
>>>>>     ipa-dnskeysyncd Service: RUNNING
>>>>>     ipa: INFO: The ipactl command was successful
>>>>>
>>>>>     [root@utility ~]# kinit admin
>>>>>     Password for admin@IDM.NAC-ISSA.ORG <mailto:admin@IDM.NAC-ISSA.ORG>:
>>>>>
>>>>>     [root@utility ~]# klist
>>>>>     Ticket cache: KCM:0:43616
>>>>>     Default principal: admin@IDM.NAC-ISSA.ORG
>>>>>     <mailto:admin@IDM.NAC-ISSA.ORG>
>>>>>
>>>>>     Valid starting       Expires              Service principal
>>>>>     09/07/2021 10:59:23  09/08/2021 10:09:04
>>>>>      krbtgt/IDM.NAC-ISSA.ORG@IDM.NAC-ISSA.ORG
>>>>>     <mailto:IDM.NAC-ISSA.ORG@IDM.NAC-ISSA.ORG>
>>>>>
>>>>>     [root@utility ~]# ipa config-show
>>>>>     ipa: ERROR: cannot connect to
>>>>>     'https://utility.idm.nac-issa.org/ipa/json': [SSL:
>>>>>     CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)
>>>>>
>>>>>
>>>>>     ------------------------------------------------------------------------
>>>>>     *From:* Florence Renaud <flo@redhat.com <mailto:flo@redhat.com>>
>>>>>     *Sent:* Tuesday, September 7, 2021 10:47 AM
>>>>>     *To:* FreeIPA users list <freeipa-users@lists.fedorahosted.org
>>>>>     <mailto:freeipa-users@lists.fedorahosted.org>>
>>>>>     *Cc:* Jeremy Tourville <jeremy_tourville@hotmail.com
>>>>>     <mailto:jeremy_tourville@hotmail.com>>
>>>>>     *Subject:* Re: [Freeipa-users] Re: Why is ipa-ods-exporter broken
>>>>>     after running ipa-dns-install? (Was - Unable to start directory
>>>>>     server after updates)
>>>>>      
>>>>>     Hi Jeremy,
>>>>>     Did you accept the security exception displayed by the browser (I'm
>>>>>     trying to eliminate obvious issues)?
>>>>>     If nothing is displayed, can you check if ipa command-line is
>>>>>     working as expected (for instance do "kinit admin; ipa config-show")?
>>>>>     You may want to enable debug logs (add debug=True to the [global]
>>>>>     section of /etc/ipa/server.conf and restart httpd service), retry
>>>>>     WebUI authentication and check the generated logs in
>>>>>     /var/log/http/error_log
>>>>>
>>>>>     flo
>>>>>
>>>>>     On Tue, Sep 7, 2021 at 2:01 PM Jeremy Tourville via FreeIPA-users
>>>>>     <freeipa-users@lists.fedorahosted.org
>>>>>     <mailto:freeipa-users@lists.fedorahosted.org>> wrote:
>>>>>
>>>>>         OK,
>>>>>         Why don't I see anything on the initial login page?
>>>>>         All I see is the URL and the fact that the certificate is not
>>>>>         trusted.  The certificate is not expired yet.  Not until Nov 2021.
>>>>>         The login in page is mostly solid white with no login or
>>>>>         password field.
>>>>>         _______________________________________________
>>>>>         FreeIPA-users mailing list --
>>>>>         freeipa-users@lists.fedorahosted.org
>>>>>         <mailto:freeipa-users@lists.fedorahosted.org>
>>>>>         To unsubscribe send an email to
>>>>>         freeipa-users-leave@lists.fedorahosted.org
>>>>>         <mailto:freeipa-users-leave@lists.fedorahosted.org>
>>>>>         Fedora Code of Conduct:
>>>>>         https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>>>>>         List Guidelines:
>>>>>         https://fedoraproject.org/wiki/Mailing_list_guidelines
>>>>>         List Archives:
>>>>>         https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org
>>>>>         Do not reply to spam on the list, report it:
>>>>>         https://pagure.io/fedora-infrastructure
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
>>>>> To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
>>>>> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>>>>> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>>>>> List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org
>>>>> Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
>>>>>
>>>>
>>>
>>
>
>
> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
> To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org
> Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
>