6:31 a.m.
Dear flo,
Thanks for the helpful links.
To check whether replication is possible between the three freeipa
servers, via the web interface on each, I have successfully created three
new users:
+ On server 1 create a new user 1 and check they appear on servers 2 & 3 - yes
+ On server 2 create a new user 2 and check they appear on servers 1 & 3 - yes
+ On server 3 create a new user 3 and check they appear on servers 1 & 2 - yes
Then for each user remove them from a different server to where they were
created. This worked as expected.
However I do appear to have one user whose information failed to
correctly replicate some time ago, causing an inconsistency that I am
uncertain on the best way to fix.
On server 1 via web attempt to view this specific user:
Operations Error
Some operations failed.
XXX: user not found
which should be displayed as a preserved user entry but lacks any details
shown other than the "user login".
On server 2 this user appears as an active user, with details, status
disabled.
On server 3 this user appears as a preserved user, with full details, and
a note I added to the Class field a few days ago to see whether it would
then just replicate but failed.
In logs, searching for the user in question on server 1
./httpd/error_log:[Sun Sep 20 10:34:14.256333 2020] [wsgi:error] [pid 29637] [remote
IP_ADDRESS:56930] ipa: INFO: [jsonserver_session] sm@OUR_DOMAIN:
user_find(u'USER_XXX', sizelimit=0, version=u'2.215', pkey_only=True):
SUCCESS
./httpd/error_log:[Sun Sep 20 10:34:28.721951 2020] [wsgi:error] [pid 29637] [remote
IP_ADDRESS:56932] ipa: INFO: [jsonserver_session] sm@OUR_DOMAIN:
user_find(u'USER_XXX', preserved=True, sizelimit=0, version=u'2.215',
pkey_only=True): SUCCESS
./httpd/error_log:[Sun Sep 20 10:34:28.738772 2020] [wsgi:error] [pid 29636] [remote
IP_ADDRESS:56932] ipa: INFO: sm@OUR_DOMAIN: batch: user_show(u'USER_XXX',
no_members=True): NotFound
./httpd/error_log:[Sun Sep 20 10:34:28.738952 2020] [wsgi:error] [pid 29636] [remote
IP_ADDRESS:56932] ipa: INFO: [jsonserver_session] sm@OUR_DOMAIN:
batch(({u'params': ((u'USER_XXX',), {u'no_members': True}),
u'method': u'user_show'},), version=u'2.215'): SUCCESS
Which just confirms what was seen via the web interface.
Next go looking in /var/log/dirsrv/slapd-OUR-DOMAIN/errors and find lots
of these messages:
[20/Sep/2020:11:18:12.449932996 +0100] - ERR - NSMMReplicationPlugin - changelog program -
repl_plugin_name_cl - agmt="cn=caToserver2" (server2:389): CSN
5bb473ef000000070000 not found, we aren't as up to date, or we purged
[20/Sep/2020:11:18:12.452051936 +0100] - ERR - NSMMReplicationPlugin - send_updates -
agmt="cn=caToserver2" (server2:389): Data required to update replica has been
purged from the changelog. If the error persists the replica must be reinitialized.
[20/Sep/2020:11:18:15.479226915 +0100] - ERR - agmt="cn=caToserver2"
(server2:389) - clcache_load_buffer - Can't locate CSN 5bb473ef000000070000 in the
changelog (DB rc=-30988). If replication stops, the consumer may need to be
reinitialized.
[20/Sep/2020:11:18:15.481677959 +0100] - ERR - NSMMReplicationPlugin - changelog program -
repl_plugin_name_cl - agmt="cn=caToserver2" (server2:389): CSN
5bb473ef000000070000 not found, we aren't as up to date, or we purged
[20/Sep/2020:11:18:15.483458741 +0100] - ERR - NSMMReplicationPlugin - send_updates -
agmt="cn=caToserver2" (server2:389): Data required to update replica has been
purged from the changelog. If the error persists the replica must be reinitialized.
But is that the only CSN found to be missing, seems to be based on this
search:
grep -v 5bb473ef000000070000 ./dirsrv/slapd-OUR-DOMAIN/errors | grep -v "If the error
persists"
389-Directory/1.3.6.6 B2017.145.2031
server1.OUR-DOMAIN:636 (/etc/dirsrv/slapd-OUR-DOMAIN)
[18/Sep/2020:05:57:26.038958649 +0100] - ERR - NSMMReplicationPlugin - release_replica -
agmt="cn=meToserver3" (server3:389): Unable to parse the response to the
endReplication extended operation.
[18/Sep/2020:17:45:55.236862951 +0100] - ERR - NSMMReplicationPlugin - release_replica -
agmt="cn=caToserver2" (server2:389): Unable to parse the response to the
endReplication extended operation.
Turning next to server 2 and the dirsrv logs there, lots of messages of the form:
[17/Sep/2020:19:33:24.075845464 +0100] - ERR - DSRetroclPlugin - delete_changerecord:
could not delete change record 6773158 (rc: 32)
[17/Sep/2020:19:33:24.076296207 +0100] - ERR - DSRetroclPlugin - delete_changerecord:
could not delete change record 6773159 (rc: 32)
[17/Sep/2020:19:33:24.077311010 +0100] - ERR - DSRetroclPlugin - delete_changerecord:
could not delete change record 6773160 (rc: 32)
[17/Sep/2020:19:33:24.077830624 +0100] - ERR - DSRetroclPlugin - delete_changerecord:
could not delete change record 6773161 (rc: 32)
[19/Sep/2020:19:59:28.091654453 +0100] - ERR - NSMMReplicationPlugin - release_replica -
agmt="cn=meToserver1" (server1:389): Unable to parse the response to the
endReplication extended operation.
Finally for server 3, dirsrv logs appear to be more about getting started
after certificate problems the other day, rather than issues seen on the
other servers, but this server does appear to display the user fully as
would be desired everywhere.
Our old version appears to lack the 389 command dsconf so not been able to
use that to help.
Summary:
+ Replication appears to work for e.g. new users
+ I guess one entry has failed to be replicated correctly causing the
inconsistent state on the other two servers
+ This should ideally be corrected to avoid further issues???
+ Uncertain about the best way to do this, suggestions very welcome as
first time fixing such problems and I don't want to make it any worse,
after all replications seems to be working for everything else.
Thanks
Best wishes
Stuart
4:49 a.m.
New subject: Renewing a failed to auto-renewal certificate
On 9/20/20 1:31 PM, Stuart McRobert via FreeIPA-users wrote:
Dear flo,
Thanks for the helpful links.
To check whether replication is possible between the three freeipa
servers, via the web interface on each, I have successfully created
three new users:
+ On server 1 create a new user 1 and check they appear on servers 2 & 3
- yes
+ On server 2 create a new user 2 and check they appear on servers 1 & 3
- yes
+ On server 3 create a new user 3 and check they appear on servers 1 & 2
- yes
Then for each user remove them from a different server to where they
were created. This worked as expected.
However I do appear to have one user whose information failed to
correctly replicate some time ago, causing an inconsistency that I am
uncertain on the best way to fix.
On server 1 via web attempt to view this specific user:
Operations Error
Some operations failed.
XXX: user not found
which should be displayed as a preserved user entry but lacks any
details shown other than the "user login".
On server 2 this user appears as an active user, with details, status
disabled.
On server 3 this user appears as a preserved user, with full details,
and a note I added to the Class field a few days ago to see whether it
would then just replicate but failed.
In logs, searching for the user in question on server 1
./httpd/error_log:[Sun Sep 20 10:34:14.256333 2020] [wsgi:error] [pid
29637] [remote IP_ADDRESS:56930] ipa: INFO: [jsonserver_session]
sm@OUR_DOMAIN: user_find(u'USER_XXX', sizelimit=0, version=u'2.215',
pkey_only=True): SUCCESS
./httpd/error_log:[Sun Sep 20 10:34:28.721951 2020] [wsgi:error] [pid
29637] [remote IP_ADDRESS:56932] ipa: INFO: [jsonserver_session]
sm@OUR_DOMAIN: user_find(u'USER_XXX', preserved=True, sizelimit=0,
version=u'2.215', pkey_only=True): SUCCESS
./httpd/error_log:[Sun Sep 20 10:34:28.738772 2020] [wsgi:error] [pid
29636] [remote IP_ADDRESS:56932] ipa: INFO: sm@OUR_DOMAIN: batch:
user_show(u'USER_XXX', no_members=True): NotFound
./httpd/error_log:[Sun Sep 20 10:34:28.738952 2020] [wsgi:error] [pid
29636] [remote IP_ADDRESS:56932] ipa: INFO: [jsonserver_session]
sm@OUR_DOMAIN: batch(({u'params': ((u'USER_XXX',),
{u'no_members':
True}), u'method': u'user_show'},), version=u'2.215'): SUCCESS
Which just confirms what was seen via the web interface.
Next go looking in /var/log/dirsrv/slapd-OUR-DOMAIN/errors and find lots
of these messages:
[20/Sep/2020:11:18:12.449932996 +0100] - ERR - NSMMReplicationPlugin -
changelog program - repl_plugin_name_cl - agmt="cn=caToserver2"
(server2:389): CSN 5bb473ef000000070000 not found, we aren't as up to
date, or we purged
[20/Sep/2020:11:18:12.452051936 +0100] - ERR - NSMMReplicationPlugin -
send_updates - agmt="cn=caToserver2" (server2:389): Data required to
update replica has been purged from the changelog. If the error persists
the replica must be reinitialized.
[20/Sep/2020:11:18:15.479226915 +0100] - ERR - agmt="cn=caToserver2"
(server2:389) - clcache_load_buffer - Can't locate CSN
5bb473ef000000070000 in the changelog (DB rc=-30988). If replication
stops, the consumer may need to be reinitialized.
[20/Sep/2020:11:18:15.481677959 +0100] - ERR - NSMMReplicationPlugin -
changelog program - repl_plugin_name_cl - agmt="cn=caToserver2"
(server2:389): CSN 5bb473ef000000070000 not found, we aren't as up to
date, or we purged
[20/Sep/2020:11:18:15.483458741 +0100] - ERR - NSMMReplicationPlugin -
send_updates - agmt="cn=caToserver2" (server2:389): Data required to
update replica has been purged from the changelog. If the error persists
the replica must be reinitialized.
But is that the only CSN found to be missing, seems to be based on this
search:
grep -v 5bb473ef000000070000 ./dirsrv/slapd-OUR-DOMAIN/errors | grep -v
"If the error persists"
389-Directory/1.3.6.6 B2017.145.2031
server1.OUR-DOMAIN:636 (/etc/dirsrv/slapd-OUR-DOMAIN)
[18/Sep/2020:05:57:26.038958649 +0100] - ERR - NSMMReplicationPlugin -
release_replica - agmt="cn=meToserver3" (server3:389): Unable to parse
the response to the endReplication extended operation.
[18/Sep/2020:17:45:55.236862951 +0100] - ERR - NSMMReplicationPlugin -
release_replica - agmt="cn=caToserver2" (server2:389): Unable to parse
the response to the endReplication extended operation.
Turning next to server 2 and the dirsrv logs there, lots of messages of
the form:
[17/Sep/2020:19:33:24.075845464 +0100] - ERR - DSRetroclPlugin -
delete_changerecord: could not delete change record 6773158 (rc: 32)
[17/Sep/2020:19:33:24.076296207 +0100] - ERR - DSRetroclPlugin -
delete_changerecord: could not delete change record 6773159 (rc: 32)
[17/Sep/2020:19:33:24.077311010 +0100] - ERR - DSRetroclPlugin -
delete_changerecord: could not delete change record 6773160 (rc: 32)
[17/Sep/2020:19:33:24.077830624 +0100] - ERR - DSRetroclPlugin -
delete_changerecord: could not delete change record 6773161 (rc: 32)
[19/Sep/2020:19:59:28.091654453 +0100] - ERR - NSMMReplicationPlugin -
release_replica - agmt="cn=meToserver1" (server1:389): Unable to parse
the response to the endReplication extended operation.
Finally for server 3, dirsrv logs appear to be more about getting
started after certificate problems the other day, rather than issues
seen on the other servers, but this server does appear to display the
user fully as would be desired everywhere.
Our old version appears to lack the 389 command dsconf so not been able
to use that to help.
Summary:
+ Replication appears to work for e.g. new users
+ I guess one entry has failed to be replicated correctly causing the
inconsistent state on the other two servers
+ This should ideally be corrected to avoid further issues???
+ Uncertain about the best way to do this, suggestions very welcome as
first time fixing such problems and I don't want to make it any worse,
after all replications seems to be working for everything else.
Hi,
if you can identify one of your servers as the true source for all your
data, it is possible to re-initialize the other ones with the content
from this good server.
Please see the details provided in this other thread:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
HTH,
flo
Thanks
Best wishes
Stuart
_______________________________________________
FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
1296
days inactive
1308
days old
freeipa-users@lists.fedorahosted.org
1 comments
2 participants
participants (2)
-
Florence Blanc-Renaud -
Stuart McRobert