For the past couple months, I've been struggling to get replicas up and running. Have
tried using containers and VMs, ended up rebuilding my FreeIPA install from the ground up
to eliminate corruption as an issue. The failures are consistent, regardless of install
options and appear to be related to replication itself. Initial replication works, but
replication after that fails. Attached are the errors encountered during the
ipa-replica-install command, along with the relevant log entries.
The primary server is currently on a Fedora 35 VM running the following RPMs.
freeipa-client-common-4.9.8-1.fc35.noarch
freeipa-server-common-4.9.8-1.fc35.noarch
freeipa-common-4.9.8-1.fc35.noarch
freeipa-client-4.9.8-1.fc35.x86_64
freeipa-healthcheck-core-0.9-3.fc35.noarch
freeipa-server-4.9.8-1.fc35.x86_64
freeipa-server-dns-4.9.8-1.fc35.noarch
freeipa-server-trust-ad-4.9.8-1.fc35.x86_64
freeipa-selinux-4.9.8-1.fc35.noarch
freeipa-healthcheck-0.9-3.fc35.noarch
Here are the replica installs for the container and VM along with the relevant
ipareplica-install.log entries.
Container first, here's the output from ipa-replica-install command.
[9/21]: configuring httpd
Nothing to do for configure_httpd_wsgi_conf
[10/21]: setting up httpd keytab
[error] NotFound: wait_for_entry timeout on ldap://primary.example.com:389 for
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.
wait_for_entry timeout on ldap://primary.example.com:389 for
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more
information
/var/log/ipareplica-install.log entries
2021-12-28T18:46:57Z DEBUG stderr=Keytab successfully retrieved and stored in:
/var/lib/ipa/gssproxy/http.keytab
2021-12-28T18:46:57Z DEBUG Waiting up to 300 seconds for replication
(ldap://primary.example.com:389)
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=ac
counts,dc=example,dc=com (objectclass=*)
2021-12-28T18:47:06Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:47:16Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:47:26Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:47:36Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:47:46Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:47:56Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:48:06Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:48:16Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:48:26Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:48:36Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:48:46Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:48:56Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:49:06Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:49:16Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:49:26Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:49:36Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:49:46Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:49:56Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:50:06Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:50:16Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:50:26Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:50:36Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:50:46Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:50:56Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:51:06Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:51:16Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:51:26Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:51:36Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:51:46Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:51:56Z DEBUG Still waiting for replication of
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:51:57Z DEBUG Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/ipaserver/install/service.py", line
635, in start_creation
run_step(full_msg, method)
File "/usr/lib/python3.10/site-packages/ipaserver/install/service.py", line
621, in run_step
method()
File "/usr/lib/python3.10/site-packages/ipaserver/install/httpinstance.py",
line 634, in request_service_keytab
replication.wait_for_entry(
File "/usr/lib/python3.10/site-packages/ipaserver/install/replication.py",
line 208, in wait_for_entry
raise errors.NotFound(
ipalib.errors.NotFound: wait_for_entry timeout on ldap://primary.example.com:389 for
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=roadfel
dt,dc=com
2021-12-28T18:51:57Z DEBUG [error] NotFound: wait_for_entry timeout on
ldap://primary.example.com:389 for
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services
,cn=accounts,dc=example,dc=com
2021-12-28T18:51:57Z DEBUG File
"/usr/lib/python3.10/site-packages/ipapython/admintool.py", line 180, in
execute
return_value = self.run()
File "/usr/lib/python3.10/site-packages/ipapython/install/cli.py", line 342,
in run
return cfgr.run()
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 360,
in run
return self.execute()
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 386,
in execute
for rval in self._executor():
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 431,
in __runner
exc_handler(exc_info)
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 460,
in _handle_execute_exception
self._handle_exception(exc_info)
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 450,
in _handle_exception
six.reraise(*exc_info)
File "/usr/lib/python3.10/site-packages/six.py", line 719, in reraise
raise value
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 421,
in __runner
step()
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 418,
in <lambda>
step = lambda: next(self.__gen)
File "/usr/lib/python3.10/site-packages/ipapython/install/util.py", line 81,
in run_generator_with_yield_from
six.reraise(*exc_info)
File "/usr/lib/python3.10/site-packages/six.py", line 719, in reraise
raise value
File "/usr/lib/python3.10/site-packages/ipapython/install/util.py", line 59,
in run_generator_with_yield_from
value = gen.send(prev_value)
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 655,
in _configure
next(executor)
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 431,
in __runner
exc_handler(exc_info)
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 460,
in _handle_execute_exception
self._handle_exception(exc_info)
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 518,
in _handle_exception
self.__parent._handle_exception(exc_info)
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 450,
in _handle_exception
six.reraise(*exc_info)
File "/usr/lib/python3.10/site-packages/six.py", line 719, in reraise
raise value
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 515,
in _handle_exception
super(ComponentBase, self)._handle_exception(exc_info)
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 450,
in _handle_exception
six.reraise(*exc_info)
File "/usr/lib/python3.10/site-packages/six.py", line 719, in reraise
raise value
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 421,
in __runner
step()
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 418,
in <lambda>
step = lambda: next(self.__gen)
File "/usr/lib/python3.10/site-packages/ipapython/install/util.py", line 81,
in run_generator_with_yield_from
six.reraise(*exc_info)
File "/usr/lib/python3.10/site-packages/six.py", line 719, in reraise
raise value
File "/usr/lib/python3.10/site-packages/ipapython/install/util.py", line 59,
in run_generator_with_yield_from
value = gen.send(prev_value)
File "/usr/lib/python3.10/site-packages/ipapython/install/common.py", line 65,
in _install
for unused in self._installer(self.parent):
File "/usr/lib/python3.10/site-packages/ipaserver/install/server/__init__.py",
line 603, in main
replica_install(self)
File
"/usr/lib/python3.10/site-packages/ipaserver/install/server/replicainstall.py",
line 401, in decorated
func(installer)
File
"/usr/lib/python3.10/site-packages/ipaserver/install/server/replicainstall.py",
line 1315, in install
install_http(
File
"/usr/lib/python3.10/site-packages/ipaserver/install/server/replicainstall.py",
line 163, in install_http
http.create_instance(
File "/usr/lib/python3.10/site-packages/ipaserver/install/httpinstance.py",
line 151, in create_instance
self.start_creation()
File "/usr/lib/python3.10/site-packages/ipaserver/install/service.py", line
635, in start_creation
run_step(full_msg, method)
File "/usr/lib/python3.10/site-packages/ipaserver/install/service.py", line
621, in run_step
method()
File "/usr/lib/python3.10/site-packages/ipaserver/install/httpinstance.py",
line 634, in request_service_keytab
replication.wait_for_entry(
File "/usr/lib/python3.10/site-packages/ipaserver/install/replication.py",
line 208, in wait_for_entry
raise errors.NotFound(
2021-12-28T18:51:57Z DEBUG The ipa-replica-install command failed, exception: NotFound:
wait_for_entry timeout on ldap://primary.example.com:389 for
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:51:57Z ERROR wait_for_entry timeout on ldap://primary.example.com:389 for
krbprincipalname=HTTP/replica1.example.com(a)EXAMPLE.COM,cn=services,cn=accounts,dc=example,dc=com
2021-12-28T18:51:57Z ERROR The ipa-replica-install command failed. See
/var/log/ipareplica-install.log for more information
VM install output
Done configuring ipa-otpd.
Custodia uses 'primary.example.com' as master peer.
Configuring ipa-custodia
[1/4]: Generating ipa-custodia config file
[2/4]: Generating ipa-custodia keys
[3/4]: starting ipa-custodia
[4/4]: configuring ipa-custodia to start on boot
Done configuring ipa-custodia.
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.
Incorrect number of results (0) searching for public key for
host/primary.example.com(a)EXAMPLE.COM
/var/log/ipareplica-install.log entries
2021-12-29T00:40:10Z DEBUG Done configuring ipa-custodia.
2021-12-29T00:40:10Z DEBUG service duration: ipa-custodia 2.37 sec
2021-12-29T00:40:10Z DEBUG Loading StateFile from
'/var/lib/ipa/sysupgrade/sysupgrade.state'
2021-12-29T00:40:10Z DEBUG Saving StateFile to
'/var/lib/ipa/sysupgrade/sysupgrade.state'
2021-12-29T00:40:10Z DEBUG Waiting up to 300 seconds to see our keys appear on host
ldap://primary.example.com
2021-12-29T00:40:10Z DEBUG File
"/usr/lib/python3.10/site-packages/ipapython/admintool.py", line 180, in
execute
return_value = self.run()
File "/usr/lib/python3.10/site-packages/ipapython/install/cli.py", line 342,
in run
return cfgr.run()
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 360,
in run
return self.execute()
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 386,
in execute
for rval in self._executor():
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 431,
in __runner
exc_handler(exc_info)
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 460,
in _handle_execute_exception
self._handle_exception(exc_info)
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 450,
in _handle_exception
six.reraise(*exc_info)
File "/usr/lib/python3.10/site-packages/six.py", line 719, in reraise
raise value
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 421,
in __runner
step()
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 418,
in <lambda>
step = lambda: next(self.__gen)
File "/usr/lib/python3.10/site-packages/ipapython/install/util.py", line 81,
in run_generator_with_yield_from
six.reraise(*exc_info)
File "/usr/lib/python3.10/site-packages/six.py", line 719, in reraise
raise value
File "/usr/lib/python3.10/site-packages/ipapython/install/util.py", line 59,
in run_generator_with_yield_from
value = gen.send(prev_value)
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 655,
in _configure
next(executor)
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 431,
in __runner
exc_handler(exc_info)
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 460,
in _handle_execute_exception
self._handle_exception(exc_info)
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 518,
in _handle_exception
self.__parent._handle_exception(exc_info)
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 450,
in _handle_exception
six.reraise(*exc_info)
File "/usr/lib/python3.10/site-packages/six.py", line 719, in reraise
raise value
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 515,
in _handle_exception
super(ComponentBase, self)._handle_exception(exc_info)
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 450,
in _handle_exception
six.reraise(*exc_info)
File "/usr/lib/python3.10/site-packages/six.py", line 719, in reraise
raise value
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 421,
in __runner
step()
File "/usr/lib/python3.10/site-packages/ipapython/install/core.py", line 418,
in <lambda>
step = lambda: next(self.__gen)
File "/usr/lib/python3.10/site-packages/ipapython/install/util.py", line 81,
in run_generator_with_yield_from
six.reraise(*exc_info)
File "/usr/lib/python3.10/site-packages/six.py", line 719, in reraise
raise value
File "/usr/lib/python3.10/site-packages/ipapython/install/util.py", line 59,
in run_generator_with_yield_from
value = gen.send(prev_value)
File "/usr/lib/python3.10/site-packages/ipapython/install/common.py", line 65,
in _install
for unused in self._installer(self.parent):
File "/usr/lib/python3.10/site-packages/ipaserver/install/server/__init__.py",
line 603, in main
replica_install(self)
File
"/usr/lib/python3.10/site-packages/ipaserver/install/server/replicainstall.py",
line 401, in decorated
func(installer)
File
"/usr/lib/python3.10/site-packages/ipaserver/install/server/replicainstall.py",
line 1345, in install
ca.install(False, config, options, custodia=custodia)
File "/usr/lib/python3.10/site-packages/ipaserver/install/ca.py", line 270, in
install
install_step_0(standalone, replica_config, options, custodia=custodia)
File "/usr/lib/python3.10/site-packages/ipaserver/install/ca.py", line 306, in
install_step_0
custodia.get_ca_keys(
File
"/usr/lib/python3.10/site-packages/ipaserver/install/custodiainstance.py", line
296, in get_ca_keys
self._get_keys(cacerts_file, cacerts_pwd, data)
File
"/usr/lib/python3.10/site-packages/ipaserver/install/custodiainstance.py", line
252, in _get_keys
cli = self._get_custodia_client()
File
"/usr/lib/python3.10/site-packages/ipaserver/install/custodiainstance.py", line
241, in _get_custodia_client
return CustodiaClient(
File "/usr/lib/python3.10/site-packages/ipaserver/secrets/client.py", line 70,
in __init__
self._server_keys(), self._client_keys()
File "/usr/lib/python3.10/site-packages/ipaserver/secrets/client.py", line 80,
in _server_keys
sk = JWK(**json_decode(self.ikk.find_key(principal, KEY_USAGE_SIG)))
File "/usr/lib/python3.10/site-packages/ipaserver/secrets/kem.py", line 224,
in find_key
return conn.get_key(usage, kid)
File "/usr/lib/python3.10/site-packages/ipaserver/secrets/kem.py", line 78, in
get_key
raise ValueError("Incorrect number of results (%d) searching for "
2021-12-29T00:40:10Z DEBUG The ipa-replica-install command failed, exception: ValueError:
Incorrect number of results (0) searching for public key for
host/primary.example.com(a)EXAMPLE.COM
2021-12-29T00:40:10Z ERROR Incorrect number of results (0) searching for public key for
host/primary.example.com(a)EXAMPLE.COM
2021-12-29T00:40:10Z ERROR The ipa-replica-install command failed. See
/var/log/ipareplica-install.log for more information