I am having an issue attempting to install IPA Server.  The server component install processes correctly, but when it comes to set up the client components it fails:

2020-04-28T22:41:42Z DEBUG failed to find session_cookie in persistent storage for principal 'host/ipa.mydomain.com@MYDOMAIN.COM'
2020-04-28T22:41:42Z INFO trying https://
ipa.mydomain.com/ipa/json
2020-04-28T22:41:42Z DEBUG Created connection context.rpcclient_1954644240
2020-04-28T22:41:42Z INFO [try 1]: Forwarding 'schema' to json server 'https://
ipa.mydomain.com/ipa/json'
2020-04-28T22:41:42Z DEBUG New HTTP connection (
ipa.mydomain.com)
2020-04-28T22:41:53Z DEBUG HTTP connection destroyed (
ipa.mydomain.com)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ipalib/rpc.py", line 732, in single_request
    response.msg)
ProtocolError: <ProtocolError for 
ipa.mydomain.com/ipa/json: 500 Internal Server Error>
2020-04-28T22:41:53Z DEBUG Destroyed connection context.rpcclient_1954644240
2020-04-28T22:41:53Z DEBUG   File "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 178, in execute
    return_value = self.run()
  File "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", line 319, in run
    return cfgr.run()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 360, in run
    return self.execute()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 386, in execute
    for rval in self._executor():
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 431, in __runner
    exc_handler(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 460, in _handle_execute_exception
    self._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 450, in _handle_exception
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 421, in __runner
    step()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 418, in <lambda>
    step = lambda: next(self.__gen)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from
    value = gen.send(prev_value)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 655, in _configure
    next(executor)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 431, in __runner
    exc_handler(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 460, in _handle_execute_exception
    self._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 518, in _handle_exception
    self.__parent._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 450, in _handle_exception
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 515, in _handle_exception
    super(ComponentBase, self)._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 450, in _handle_exception
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 421, in __runner
    step()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 418, in <lambda>
    step = lambda: next(self.__gen)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from
    value = gen.send(prev_value)
  File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", line 65, in _install
    for unused in self._installer(self.parent):
  File "/usr/lib/python2.7/site-packages/ipaclient/install/client.py", line 3671, in main
    install(self)
  File "/usr/lib/python2.7/site-packages/ipaclient/install/client.py", line 2392, in install
    _install(options)
  File "/usr/lib/python2.7/site-packages/ipaclient/install/client.py", line 2734, in _install
    api.finalize()
  File "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 739, in finalize
    self.__do_if_not_done('load_plugins')
  File "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 431, in __do_if_not_done
    getattr(self, name)()
  File "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 619, in load_plugins
    for package in self.packages:
  File "/usr/lib/python2.7/site-packages/ipalib/__init__.py", line 949, in packages
    ipaclient.remote_plugins.get_package(self),
  File "/usr/lib/python2.7/site-packages/ipaclient/remote_plugins/__init__.py", line 134, in get_package
    plugins = schema.get_package(server_info, client)
  File "/usr/lib/python2.7/site-packages/ipaclient/remote_plugins/schema.py", line 553, in get_package
    schema = Schema(client)
  File "/usr/lib/python2.7/site-packages/ipaclient/remote_plugins/schema.py", line 401, in __init__
    fingerprint, ttl = self._fetch(client, ignore_cache=read_failed)
  File "/usr/lib/python2.7/site-packages/ipaclient/remote_plugins/schema.py", line 426, in _fetch
    schema = client.forward(u'schema', **kwargs)['result']
  File "/usr/lib/python2.7/site-packages/ipalib/rpc.py", line 1190, in forward
    raise NetworkError(uri=server, error=e.errmsg)

2020-04-28T22:41:53Z DEBUG The ipa-client-install command failed, exception: NetworkError: cannot connect to 'https://
ipa.mydomain.com/ipa/json': Internal Server Error

The relevant services appear to be running

  certmonger.service         loaded active running   Certificate monitoring and
  dirsrv@
MYDOMAIN-COM.service loaded active running   389 Directory S
  gssproxy.service           loaded active running   GSSAPI Proxy Daemon
  httpd.service              loaded active running   The Apache HTTP Server
  ipa-custodia.service       loaded active running   IPA Custodia Service
  ipa-dnskeysyncd.service    loaded active running   IPA key daemon
  ipa.service                loaded active exited    Identity, Policy, Audit
  kadmin.service             loaded active running   Kerberos 5 Password-changin
  krb5kdc.service            loaded active running   Kerberos 5 KDC
  named-pkcs11.service       loaded active running   Berkeley Internet Name Doma
  ntpd.service               loaded active running   Network Time Service
  oddjobd.service            loaded active running   privileged operations for u
  pki-tomcatd@pki-tomcat.service loaded active running   PKI Tomcat Server pki-t


I can use kinit to obtain a ticket for admin, but any ipa command that I attempt to run gives an error along the following lines

ipa: DEBUG: failed to find session_cookie in persistent storage for principal 'admin@MYDOMAIN.COM'
ipa: INFO: trying https://
ipa.mydomain.com/ipa/json
ipa: DEBUG: Created connection context.rpcclient_1964217648
ipa: INFO: [try 1]: Forwarding 'schema' to json server 'https://
ipa.mydomain.com/ipa/json'
ipa: DEBUG: New HTTP connection (
ipa.mydomain.com)
ipa: DEBUG: HTTP connection destroyed (
ipa.mydomain.com)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ipalib/rpc.py", line 732, in single_request
    response.msg)
ProtocolError: <ProtocolError for 
ipa.mydomain.com/ipa/json: 500 Internal Server Error>
ipa: DEBUG: Destroyed connection context.rpcclient_1964217648
ipa: ERROR: cannot connect to 'https://
ipa.mydomain.com/ipa/json': Internal Server Error

In the httpd error log, I see the same error for every ipa command issued

[Wed Apr 29 14:51:19.119357 2020] [:error] [pid 8505] ipa: ERROR: 500 Internal Server Error: KerberosWSGIExecutioner.__call__: KRB5CCNAME not defined in HTTP request environment
[Wed Apr 29 14:51:19.120223 2020] [:error] [pid 8505] [remote 192.168.0.2:16498] mod_wsgi (pid=8505): Exception occurred processing WSGI script '/usr/share/ipa/wsgi.py'.
[Wed Apr 29 14:51:19.120335 2020] [:error] [pid 8505] [remote 192.168.0.2:16498] RuntimeError: response has not been started


The same error is present at the time the install failed.

The Kerberos ticket is valid as ldapsearch works using it

[root@ipa1 ~]# ldapsearch -h ipa.mydomain.com -b ou=people,o=ipaca -Y GSSAPI -s sub "(uid=admin)" dn uid
SASL/GSSAPI authentication started
SASL username: admin@
MYDOMAIN.COM
SASL SSF: 256
SASL data security layer installed.
# extended LDIF
#
# LDAPv3
# base <ou=people,o=ipaca> with scope subtree
# filter: (uid=admin)
# requesting: dn uid
#

# admin, people, ipaca
dn: uid=admin,ou=people,o=ipaca
uid: admin

# search result
search: 4
result: 0 Success

# numResponses: 2
# numEntries: 1


but doesn't without it

[root@ipa1 ~]# ldapsearch -h ipa.mydomain.com -b ou=people,o=ipaca -s sub "(uid=admin)" dn uid
SASL/EXTERNAL authentication started
ldap_sasl_interactive_bind_s: Unknown authentication method (-6)
additional info: SASL(-4): no mechanism available: 


Does anyone have any ideas? I'm tearing my hair out here!