Hi Theirry.
To tell the truth my configuration was already set to on, on both VMs:
*[root@srv01 ~]# ldapsearch -D "cn=Directory Manager" -h localhost -b
"cn=config" -w $PASS | grep
nsslapd-ignore-time-skewnsslapd-ignore-time-skew: on*
*[root@srv02 ~]# ldapsearch -D "cn=Directory Manager" -h localhost -b
"cn=config" -w $PASS | grep
nsslapd-ignore-time-skewnsslapd-ignore-time-skew: on*
Anyway, I tried to set up it to *off* and then to *on* again, but now I
have a new issue into logs of the 2nd server :(
Srv01 logs are similar as before:
*[23/Apr/2020:09:28:45.958922636 +0200] - WARN - csngen_new_csn - Too much
time skew (-22777212 secs). Current seqnum=5ca0*
Srv02 logs now are like these:
*[23/Apr/2020:09:32:14.919328803 +0200] - ERR -
(srv01:389) - clcache_load_buffer - Can't locate CSN 5e7cfe03000400030000
in the changelog (DB rc=-30988). If replication stops, the consumer may
need to be reinitialized.[23/Apr/2020:09:32:14.920873489 +0200] - ERR -
NSMMReplicationPlugin - changelog program - repl_plugin_name_cl -
(srv01:389): CSN 5e7cfe03000400030000 not found, we aren't as up to date,
or we purged[23/Apr/2020:09:32:14.922161821 +0200] - ERR -
NSMMReplicationPlugin - send_updates -
(srv01:389): Data required to update
replica has been purged from the changelog. If the error persists the
replica must be reinitialized.*
I have just tried to force a replica on both sides, without success:
*[root@srv01 ~]# ipa-replica-manage force-sync --from
status yetNo
status yetNo status yet[root@srv02 ~]# ipa-replica-manage force-sync --from
status yetNo
status yetNo status yet*
What could I do now?
Thanks, bye
Il giorno mer 22 apr 2020 alle ore 17:08 thierry bordaz via FreeIPA-users <
freeipa-users(a)lists.fedorahosted.org> ha scritto:
Hi Morgan,
Sure. The most immediate and safest action is to do
dn: cn=config
changetype: modify
replace: nsslapd-ignore-time-skew
nsslapd-ignore-time-skew: on
On all servers in the topology (no need to restart). Then monitor if
replication is catching up.
Okay NTP issues is likely the RC of your time skew but there is not easy
way to prove it if any.
best regards
theirry
On 4/22/20 3:16 PM, Morgan Marodin via FreeIPA-users wrote:
Hi.
I don't have access to RedHat portal :(
There are similar articles in a public forum?
Anyway ... could I stop ipa-server, change the value of
*nsslapd-ignore-time-skew* into
*/etc/dirsrv/slapd-IPA-MYDOMAIN-COM/dse.ldif* and start again the server?
Or is more complicated to change the configuration?
VMs are local, but the cluster where the 1st server is running is affected
by NTP problems ...
For this reason I want to remove the First Master and install another
replica in the new cluster.
Thanks, bye.
Morgan
Il giorno mer 22 apr 2020 alle ore 11:33 thierry bordaz via FreeIPA-users <
freeipa-users(a)lists.fedorahosted.org> ha scritto:
> Hi,
>
> CSN generator time skew is a pending issue still under investigation.
>
> At the moment the way your csn generator is messed up looks not fatal.
> You can allow replication to continue with the setting of
> nsslapd-ignore-time-skew on all servers. (
>
https://access.redhat.com/solutions/1162703)
>
> If it does not allow replication to continue there is a recovery
> procedure but I would recommend to first try ignore-time-skew (
>
https://access.redhat.com/solutions/3543811)
>
> NTP tuning or specific VMs are suspected to contribute to time skew. What
> type of VMs are you using (local or cloud (AWS)) ?
>
> best regards
> thierry
>
> On 4/21/20 5:42 PM, Morgan Marodin via FreeIPA-users wrote:
>
> Hi.
>
> Into my environment I have two IPA server, replicating each other.
> They are both 7.6 OS systems, ipa-server RPM version is
> 4.6.4-10.0.1.el7_6.2.x86_64.
>
> The first server installed was srv01 (many years ago), then I installed
> the replica into srv02 (like a year later the 1st node).
> When I had a single server I did also a trust with my corporate Active
> Directory.
> VMs are running in 2 different hypervisor clusters.
>
> Now the replication doesn't works. Into log files I have this error:
>
>
> *[16/Apr/2020:12:25:36.856632697 +0200] - ERR - csngen_adjust_time -
> Adjustment limit exceeded; value - 23221226, limit - 86400
> [16/Apr/2020:12:25:36.857909222 +0200] - ERR - NSMMReplicationPlugin -
> repl5_inc_run -
agmt="cn=meTosrv01.ipa.mydomain.com
> <
http://meTosrv01.ipa.mydomain.com>" (srv01:389): Fatal error - too much
> time skew between replicas! [16/Apr/2020:12:25:36.862233147 +0200] - ERR -
> NSMMReplicationPlugin - repl5_inc_run -
agmt="cn=meTosrv01.ipa.mydomain.com
> <
http://meTosrv01.ipa.mydomain.com>" (srv01:389): Incremental update
failed
> and requires administrator action*
>
> I tried to force the replica, but the limit exceeded problem doesn't
> allow the sync.
> I know that the problem is that CSN generator has become grossly skewed.
> Using the external script readNsState.py I found that there was as offset
> time for about a month, so ... I waited for a month and then the issue
> disappeared.
> But now the offset is about 9 months ... I can't wait so much time :)
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *[root@srv01 scripts]# ./readNsState.py
> /etc/dirsrv/slapd-IPA-MYDOMAIN-COM/dse.ldif nsState is
> BAAAAAAAAACCN/xfAAAAAHbiBAAAAAAABCgAAAAAAAANdQAAAAAAAA== Little Endian For
> replica cn=replica,cn=dc\3Dipa\2Cdc\3Dmydomain\2Cdc\3Dcom,cn=mapping
> tree,cn=con fmtstr=[H6x3QH6x] size=40 len of nsstate is 40 CSN
> generator state: Replica ID : 4 Sampled Time : 1610364802
> Gen as csn : 5ffc37822996500040000 Time as str : Mon Jan 11
> 12:33:22 2021 Local Offset : 320118 Remote Offset : 10244 Seq.
> num : 29965 System time : Tue Apr 21 15:03:45 2020 Diff in
> sec. : -22890577 Day:sec diff : -265:5423 nsState is
> YAAAAAAAAAADLZheAAAAAAAAAAAAAAAAXSgAAAAAAAATAAAAAAAAAA== Little Endian For
> replica cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
> fmtstr=[H6x3QH6x] size=40 len of nsstate is 40 CSN generator state:
> Replica ID : 96 Sampled Time : 1587031299 Gen as csn :
> 5e982d03001900960000 Time as str : Thu Apr 16 12:01:39 2020 Local
> Offset : 0 Remote Offset : 10333 Seq. num : 19 System
> time : Tue Apr 21 15:03:45 2020 Diff in sec. : 442926 Day:sec
> diff : 5:10926 [root@srv02 scripts]# ./readNsState.py
> /etc/dirsrv/slapd-IPA-MYDOMAIN-COM/dse.ldif nsState is
> AwAAAAAAAABU7p5eAAAAAAAAAAAAAAAAsVNiAQAAAAAAAAAAAAAAAA== Little Endian For
> replica cn=replica,cn=dc\3Dipa\2Cdc\3Dmydomain\2Cdc\3Dcom,cn=mapping
> tree,cn=con fmtstr=[H6x3QH6x] size=40 len of nsstate is 40 CSN
> generator state: Replica ID : 3 Sampled Time : 1587474004
> Gen as csn : 5e9eee54000000030000 Time as str : Tue Apr 21
> 15:00:04 2020 Local Offset : 0 Remote Offset : 23221169 Seq.
> num : 0 System time : Tue Apr 21 15:02:38 2020 Diff in sec.
> : 154 Day:sec diff : 0:154 nsState is
> YQAAAAAAAAAuLZheAAAAAEUBAAAAAAAA7SYAAAAAAAASAAAAAAAAAA== Little Endian For
> replica cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
> fmtstr=[H6x3QH6x] size=40 len of nsstate is 40 CSN generator state:
> Replica ID : 97 Sampled Time : 1587031342 Gen as csn :
> 5e982d2e001800970000 Time as str : Thu Apr 16 12:02:22 2020 Local
> Offset : 325 Remote Offset : 9965 Seq. num : 18 System
> time : Tue Apr 21 15:02:38 2020 Diff in sec. : 442816 Day:sec
> diff : 5:10816*
>
> As you can see in the 1st node the Time as str is Jan 11 of 2021.
> With timedatectl command I see that both VMs use the same Time zone and
> the clock is correct.
>
> I found this old article to fix my issue:
> *https://www.redhat.com/archives/freeipa-users/2014-February/msg00007.html
> <
https://www.redhat.com/archives/freeipa-users/2014-February/msg00007.html...
>
> But ... I had the same issue in the past, always in the 1st server. So,
> in my mind I don't want to try to use that fix.
> I have a new hypervisor cluster, so I would prefer to reinstall the 1st
> server, using these steps:
>
> 1) check if all roles (also the CA) is installed in srv02
> You can find here some data about the VMs:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *[root@srv01 ~]# ipa server-show
srv01.ipa.mydomain.com
> <
http://srv01.ipa.mydomain.com> Server name:
srv01.ipa.mydomain.com
> <
http://srv01.ipa.mydomain.com> Managed suffixes: domain, ca Min domain
> level: 0 Max domain level: 1 Enabled server roles: CA server, IPA
> master, DNS server, NTP server, AD trust controller [root@srv02 ~]# ipa
> server-show
srv02.ipa.mydomain.com <
http://srv02.ipa.mydomain.com> Server
> name:
srv02.ipa.mydomain.com <
http://srv02.ipa.mydomain.com> Managed
> suffixes: domain, ca Min domain level: 0 Max domain level: 1 Enabled
> server roles: CA server, IPA master, DNS server, NTP server [root@srv01 ~]#
> ipa config-show Maximum username length: 32 Home directory base: /home
> Default shell: /bin/bash Default users group: ipausers Default e-mail
> domain:
ipa.mydomain.com <
http://ipa.mydomain.com> Search time limit: 2
> Search size limit: 100 User search fields:
> uid,givenname,sn,telephonenumber,ou,title Group search fields:
> cn,description Enable migration mode: FALSE Certificate Subject base:
>
O=IPA.MYDOMAIN.COM <
http://IPA.MYDOMAIN.COM> Password Expiration
> Notification (days): 4 Password plugin features: AllowNThash SELinux
> user map order:
> guest_u:s0$xguest_u:s0$user_u:s0$staff_u:s0-s0:c0.c1023$unconfined_u:s0-s0:c0.c1023
> Default SELinux user: unconfined_u:s0-s0:c0.c1023 Default PAC types:
> MS-PAC, nfs:NONE IPA masters:
srv01.ipa.mydomain.com
> <
http://srv01.ipa.mydomain.com>,
srv02.ipa.mydomain.com
> <
http://srv02.ipa.mydomain.com> IPA CA servers:
srv01.ipa.mydomain.com
> <
http://srv01.ipa.mydomain.com>,
srv02.ipa.mydomain.com
> <
http://srv02.ipa.mydomain.com> IPA NTP servers:
srv01.ipa.mydomain.com
> <
http://srv01.ipa.mydomain.com>,
srv02.ipa.mydomain.com
> <
http://srv02.ipa.mydomain.com> IPA CA renewal master:
>
srv01.ipa.mydomain.com <
http://srv01.ipa.mydomain.com> [root@srv02 ~]# ipa
> config-show Maximum username length: 32 Home directory base: /home
> Default shell: /bin/bash Default users group: ipausers Default e-mail
> domain:
ipa.mydomain.com <
http://ipa.mydomain.com> Search time limit: 2
> Search size limit: 100 User search fields:
> uid,givenname,sn,telephonenumber,ou,title Group search fields:
> cn,description Enable migration mode: FALSE Certificate Subject base:
>
O=IPA.MYDOMAIN.COM <
http://IPA.MYDOMAIN.COM> Password Expiration
> Notification (days): 4 Password plugin features: AllowNThash SELinux
> user map order:
> guest_u:s0$xguest_u:s0$user_u:s0$staff_u:s0-s0:c0.c1023$unconfined_u:s0-s0:c0.c1023
> Default SELinux user: unconfined_u:s0-s0:c0.c1023 Default PAC types:
> MS-PAC, nfs:NONE IPA masters:
srv01.ipa.mydomain.com
> <
http://srv01.ipa.mydomain.com>,
srv02.ipa.mydomain.com
> <
http://srv02.ipa.mydomain.com> IPA CA servers:
srv01.ipa.mydomain.com
> <
http://srv01.ipa.mydomain.com>,
srv02.ipa.mydomain.com
> <
http://srv02.ipa.mydomain.com> IPA NTP servers:
srv01.ipa.mydomain.com
> <
http://srv01.ipa.mydomain.com>,
srv02.ipa.mydomain.com
> <
http://srv02.ipa.mydomain.com> IPA CA renewal master:
>
srv01.ipa.mydomain.com <
http://srv01.ipa.mydomain.com> [root@srv01 ~]#
> ipactl status Directory Service: RUNNING krb5kdc Service: RUNNING kadmin
> Service: RUNNING named Service: RUNNING httpd Service: RUNNING ipa-custodia
> Service: RUNNING ntpd Service: RUNNING pki-tomcatd Service: STOPPED smb
> Service: RUNNING winbind Service: RUNNING ipa-otpd Service: RUNNING
> ipa-dnskeysyncd Service: RUNNING ipa: INFO: The ipactl command was
> successful [root@srv02 ~]# ipactl status Directory Service: RUNNING krb5kdc
> Service: RUNNING kadmin Service: RUNNING named Service: RUNNING httpd
> Service: RUNNING ipa-custodia Service: RUNNING ntpd Service: RUNNING
> pki-tomcatd Service: STOPPED ipa-otpd Service: RUNNING ipa-dnskeysyncd
> Service: RUNNING ipa: INFO: The ipactl command was successful [root@srv01
> ~]# certutil -L -d /etc/pki/pki-tomcat/alias Certificate Nickname
> Trust Attributes
> SSL,S/MIME,JAR/XPI Server-Cert cert-pki-ca
> u,u,u subsystemCert cert-pki-ca
> u,u,u caSigningCert cert-pki-ca
> CTu,Cu,Cu ocspSigningCert cert-pki-ca
> u,u,u auditSigningCert cert-pki-ca
> u,u,Pu [root@srv02 ~]# certutil -L -d /etc/pki/pki-tomcat/alias
> Certificate Nickname Trust
> Attributes
> SSL,S/MIME,JAR/XPI Server-Cert cert-pki-ca
> u,u,u subsystemCert cert-pki-ca
> u,u,u caSigningCert cert-pki-ca CTu,u,u
> ocspSigningCert cert-pki-ca u,u,u
> auditSigningCert cert-pki-ca u,u,Pu*
>
>
> It seems that AD trust controller role, IPA CA renewal master, smb and
> windbind are only in the 1st server.
> And also caSigningCert cert-pki-ca entry is different (CTu,Cu,Cu vs
> CTu,u,u).
>
> I can see only in the 1st server these DNS records:
>
>
>
>
>
> *_kerberos._tcp.Default-First-Site-Name._sites.dc._msdcs SRV 0 100 88
> srv01 _kerberos._tcp.dc._msdcs SRV 0 100 88 srv01
> _kerberos._udp.Default-First-Site-Name._sites.dc._msdcs SRV 0 100 88 srv01
> _kerberos._udp.dc._msdcs SRV 0 100 88 srv01
> _ldap._tcp.Default-First-Site-Name._sites.dc._msdcs 0 100 389 srv01
> _ldap._tcp.dc._msdcs 0 100 389 srv01*
>
> Srv01 is the first master, I know, but is the server VM that has clock
> problems, in both situations.
> So I want to keep srv02 and install a new one.
>
> What do I have to do to let the 2nd VM be a single server?
> Could I use these URLs?
>
> *https://www.freeipa.org/page/Backup_and_Restore#One_Server_Loss_-_First_Master
>
<
https://www.freeipa.org/page/Backup_and_Restore#One_Server_Loss_-_First_M...
>
https://www.freeipa.org/page/V4/Server_Roles#Upgrade
> <
https://www.freeipa.org/page/V4/Server_Roles#Upgrade>*
>
>
> 2) uninstall ipa-server from the 1st server (srv01) and then powering off
> it, assuming that all data into the 2nd one are ok (srv02)
>
> 3) update freeipa and all other RPM packages into the VM srv02
>
> 4) install a new fresh VM, always with 7 release, and create a new replica
> Could I use the same old hostname (srv01) and IP address for this new VM?
> Or is better to use the same IP but a new name, like srv03?
>
>
> Do you think this is the right way to solve my issue?
> Or do you have any better idea?
>
> Please let me know, thanks.
> Bye, Morgan
>
> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
>
>
> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> To unsubscribe send an email to
> freeipa-users-leave(a)lists.fedorahosted.org
> Fedora Code of Conduct:
>
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
>
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
>
_______________________________________________
FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
_______________________________________________
FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...