I need some help with this.  I am working with FreeIPA runnning on CentOS 7.4 verssion 4.5.0-22.  I have 2 servers in my AWS VPC and 2 servers at my local office.  

For some reason I am not seeing replication happen (over ldaps?) from 1 server in my local office to the two servers up there.

AWS servers:

[centos@freeipa03 ~]$ sudo ipa-replica-manage list -v freeipa01.stl1.gatewayblend.net
freeipa03.east.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (0) Replica acquired successfully: Incremental update succeeded
  last update ended: 2018-03-21 02:25:31+00:00
freeipa04.east.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (0) Replica acquired successfully: Incremental update succeeded
  last update ended: 2018-03-21 02:25:31+00:00
freeipa03.stl1.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (0) Replica acquired successfully: Incremental update succeeded
  last update ended: 2018-03-21 02:30:31+00:00
[centos@freeipa03 ~]$ sudo ipa-replica-manage list -v freeipa03.stl1.gatewayblend.net
freeipa03.east.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (-1) Problem connecting to replica - LDAP error: Can't contact LDAP server (connection error)
  last update ended: 1970-01-01 00:00:00+00:00
freeipa04.east.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (-1) Problem connecting to replica - LDAP error: Can't contact LDAP server (connection error)
  last update ended: 1970-01-01 00:00:00+00:00
freeipa01.stl1.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (-1) Problem connecting to replica - LDAP error: Can't contact LDAP server (connection error)
  last update ended: 1970-01-01 00:00:00+00:00
[centos@freeipa03 ~]$

[root@freeipa04 log]# ipa-replica-manage list -v freeipa03.stl1.gatewayblend.net
freeipa03.east.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (-1) Problem connecting to replica - LDAP error: Can't contact LDAP server (connection error)
  last update ended: 1970-01-01 00:00:00+00:00
freeipa04.east.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (-1) Problem connecting to replica - LDAP error: Can't contact LDAP server (connection error)
  last update ended: 1970-01-01 00:00:00+00:00
freeipa01.stl1.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (-1) Problem connecting to replica - LDAP error: Can't contact LDAP server (connection error)
  last update ended: 1970-01-01 00:00:00+00:00
[root@freeipa04 log]# ipa-replica-manage list -v freeipa01.stl1.gatewayblend.net
freeipa03.east.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (0) Replica acquired successfully: Incremental update succeeded
  last update ended: 2018-03-21 02:25:31+00:00
freeipa04.east.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (0) Replica acquired successfully: Incremental update succeeded
  last update ended: 2018-03-21 02:25:31+00:00
freeipa03.stl1.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (0) Replica acquired successfully: Incremental update succeeded
  last update ended: 2018-03-21 02:30:31+00:00
[root@freeipa04 log]#

Local office:
server 1

[gatewayblend@freeipa01 ~]$ sudo ipa-replica-manage list -v freeipa04.east.gatewayblend.net
freeipa01.stl1.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (0) Replica acquired successfully: Incremental update succeeded
  last update ended: 2018-03-21 13:24:41+00:00
freeipa03.stl1.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (0) Replica acquired successfully: Incremental update succeeded
  last update ended: 2018-03-21 13:24:32+00:00
freeipa03.east.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (-1) Problem connecting to replica - LDAP error: Can't contact LDAP server (connection error)
  last update ended: 1970-01-01 00:00:00+00:00
[gatewayblend@freeipa01 ~]$ sudo ipa-replica-manage list -v freeipa03.east.gatewayblend.net
freeipa01.stl1.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (0) Replica acquired successfully: Incremental update succeeded
  last update ended: 2018-03-21 13:30:53+00:00
freeipa03.stl1.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (0) Replica acquired successfully: Incremental update succeeded
  last update ended: 2018-03-21 13:30:53+00:00
freeipa04.east.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (-1) Problem connecting to replica - LDAP error: Can't contact LDAP server (connection error)
  last update ended: 1970-01-01 00:00:00+00:00
[gatewayblend@freeipa01 ~]$

[gatewayblend@freeipa03 ~]$ sudo ipa-replica-manage list -v freeipa04.east.gatewayblend.net
freeipa01.stl1.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (0) Replica acquired successfully: Incremental update succeeded
  last update ended: 2018-03-21 02:08:00+00:00
freeipa03.stl1.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (0) Replica acquired successfully: Incremental update succeeded
  last update ended: 2018-03-21 02:07:54+00:00
freeipa03.east.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (-1) Problem connecting to replica - LDAP error: Can't contact LDAP server (connection error)
  last update ended: 1970-01-01 00:00:00+00:00
[gatewayblend@freeipa03 ~]$ sudo vim /etc/resolv.conf
[gatewayblend@freeipa03 ~]$ sudo ipa-replica-manage list -v freeipa03.east.gatewayblend.net
freeipa01.stl1.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (0) Replica acquired successfully: Incremental update succeeded
  last update ended: 2018-03-21 02:40:35+00:00
freeipa03.stl1.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (0) Replica acquired successfully: Incremental update succeeded
  last update ended: 2018-03-21 02:40:35+00:00
freeipa04.east.gatewayblend.net: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (-1) Problem connecting to replica - LDAP error: Can't contact LDAP server (connection error)
  last update ended: 1970-01-01 00:00:00+00:00
[gatewayblend@freeipa03 ~]$

The topologysegment shows we have 2-way connectivity all the way around:
[root@freeipa04 log]# ipa topologysegment-find --all
Suffix name: domain
------------------
6 segments matched
------------------
  dn: cn=freeipa01.stl1.gatewayblend.net-to-freeipa03.stl1.gatewayblend.net,cn=domain,cn=topology,cn=ipa,cn=etc,dc=gatewayblend,dc=net
  Segment name: freeipa01.stl1.gatewayblend.net-to-freeipa03.stl1.gatewayblend.net
  Left node: freeipa01.stl1.gatewayblend.net
  Right node: freeipa03.stl1.gatewayblend.net
  Connectivity: both
  iparepltoposegmentstatus: autogen
  objectclass: iparepltoposegment, top

  dn: cn=freeipa01.stl1.gatewayblend.net-to-freeipa04.east.gatewayblend.net,cn=domain,cn=topology,cn=ipa,cn=etc,dc=gatewayblend,dc=net
  Segment name: freeipa01.stl1.gatewayblend.net-to-freeipa04.east.gatewayblend.net
  Left node: freeipa01.stl1.gatewayblend.net
  Right node: freeipa04.east.gatewayblend.net
  Connectivity: both
  objectclass: iparepltoposegment, top

  dn: cn=freeipa03.east.gatewayblend.net-to-freeipa01.stl1.gatewayblend.net,cn=domain,cn=topology,cn=ipa,cn=etc,dc=gatewayblend,dc=net
  Segment name: freeipa03.east.gatewayblend.net-to-freeipa01.stl1.gatewayblend.net
  Left node: freeipa03.east.gatewayblend.net
  Right node: freeipa01.stl1.gatewayblend.net
  Connectivity: both
  objectclass: iparepltoposegment, top

  dn: cn=freeipa03.east.gatewayblend.net-to-freeipa04.east.gatewayblend.net,cn=domain,cn=topology,cn=ipa,cn=etc,dc=gatewayblend,dc=net
  Segment name: freeipa03.east.gatewayblend.net-to-freeipa04.east.gatewayblend.net
  Left node: freeipa03.east.gatewayblend.net
  Right node: freeipa04.east.gatewayblend.net
  Connectivity: both
  iparepltoposegmentstatus: autogen
  objectclass: iparepltoposegment, top

  dn: cn=freeipa03.stl1.gatewayblend.net-to-freeipa03.east.gatewayblend.net,cn=domain,cn=topology,cn=ipa,cn=etc,dc=gatewayblend,dc=net
  Segment name: freeipa03.stl1.gatewayblend.net-to-freeipa03.east.gatewayblend.net
  Left node: freeipa03.stl1.gatewayblend.net
  Right node: freeipa03.east.gatewayblend.net
  Connectivity: both
  objectclass: iparepltoposegment, top

  dn: cn=freeipa03.stl1.gatewayblend.net-to-freeipa04.east.gatewayblend.net,cn=domain,cn=topology,cn=ipa,cn=etc,dc=gatewayblend,dc=net
  Segment name: freeipa03.stl1.gatewayblend.net-to-freeipa04.east.gatewayblend.net
  Left node: freeipa03.stl1.gatewayblend.net
  Right node: freeipa04.east.gatewayblend.net
  Connectivity: both
  objectclass: iparepltoposegment, top
----------------------------
Number of entries returned 6
----------------------------
[root@freeipa04 log]#

When I add a user everything gets sync'ed.  When I add a DNS entry its gets sync'ed all the way around.  

Is the error i'm getting a false positive?  It seems like it is.

This is the error I'm getting in /var/log/messages.  However I think this pertains to DNSSEC and can be ignored, correct?

Mar 21 13:35:25 freeipa01 systemd: ipa-dnskeysyncd.service: main process exited, code=exited, status=1/FAILURE
Mar 21 13:35:25 freeipa01 systemd: Unit ipa-dnskeysyncd.service entered failed state.
Mar 21 13:35:25 freeipa01 systemd: ipa-dnskeysyncd.service failed.
Mar 21 13:36:25 freeipa01 systemd: ipa-dnskeysyncd.service holdoff time over, scheduling restart.
Mar 21 13:36:25 freeipa01 systemd: Started IPA key daemon.
Mar 21 13:36:25 freeipa01 systemd: Starting IPA key daemon...
Mar 21 13:36:28 freeipa01 ipa-dnskeysyncd: ipa         : INFO     LDAP bind...
Mar 21 13:36:28 freeipa01 ipa-dnskeysyncd: ipa         : INFO     Commencing sync process
Mar 21 13:36:29 freeipa01 ipa-dnskeysyncd: ipa.ipaserver.dnssec.keysyncer.KeySyncer: INFO     Initial LDAP dump is done, sychronizing with ODS and BIND
Mar 21 13:36:32 freeipa01 ipa-dnskeysyncd: Traceback (most recent call last):
Mar 21 13:36:32 freeipa01 ipa-dnskeysyncd: File "/usr/libexec/ipa/ipa-dnskeysyncd", line 114, in <module>
Mar 21 13:36:32 freeipa01 ipa-dnskeysyncd: while ldap_connection.syncrepl_poll(all=1, msgid=ldap_search):
Mar 21 13:36:32 freeipa01 ipa-dnskeysyncd: File "/usr/lib64/python2.7/site-packages/ldap/syncrepl.py", line 405, in syncrepl_poll
Mar 21 13:36:32 freeipa01 ipa-dnskeysyncd: self.syncrepl_refreshdone()
Mar 21 13:36:32 freeipa01 ipa-dnskeysyncd: File "/usr/lib/python2.7/site-packages/ipaserver/dnssec/keysyncer.py", line 115, in syncrepl_refreshdone
Mar 21 13:36:32 freeipa01 ipa-dnskeysyncd: self.hsm_replica_sync()
Mar 21 13:36:32 freeipa01 ipa-dnskeysyncd: File "/usr/lib/python2.7/site-packages/ipaserver/dnssec/keysyncer.py", line 181, in hsm_replica_sync
Mar 21 13:36:32 freeipa01 ipa-dnskeysyncd: ipautil.run([paths.IPA_DNSKEYSYNCD_REPLICA])
Mar 21 13:36:32 freeipa01 ipa-dnskeysyncd: File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 512, in run
Mar 21 13:36:32 freeipa01 ipa-dnskeysyncd: raise CalledProcessError(p.returncode, arg_string, str(output))
Mar 21 13:36:32 freeipa01 ipa-dnskeysyncd: subprocess.CalledProcessError: Command '/usr/libexec/ipa/ipa-dnskeysync-replica' returned non-zero exit status 1
Mar 21 13:36:33 freeipa01 systemd: ipa-dnskeysyncd.service: main process exited, code=exited, status=1/FAILURE
Mar 21 13:36:33 freeipa01 systemd: Unit ipa-dnskeysyncd.service entered failed state.
Mar 21 13:36:33 freeipa01 systemd: ipa-dnskeysyncd.service failed.
Mar 21 13:37:33 freeipa01 systemd: ipa-dnskeysyncd.service holdoff time over, scheduling restart.
Mar 21 13:37:33 freeipa01 systemd: Started IPA key daemon.
Mar 21 13:37:33 freeipa01 systemd: Starting IPA key daemon...
Mar 21 13:37:36 freeipa01 ipa-dnskeysyncd: ipa         : INFO     LDAP bind...
Mar 21 13:37:36 freeipa01 ipa-dnskeysyncd: ipa         : INFO     Commencing sync process
Mar 21 13:37:36 freeipa01 ipa-dnskeysyncd: ipa.ipaserver.dnssec.keysyncer.KeySyncer: INFO     Initial LDAP dump is done, sychronizing with ODS and BIND
Mar 21 13:37:40 freeipa01 ipa-dnskeysyncd: Traceback (most recent call last):
Mar 21 13:37:40 freeipa01 ipa-dnskeysyncd: File "/usr/libexec/ipa/ipa-dnskeysyncd", line 114, in <module>
Mar 21 13:37:40 freeipa01 ipa-dnskeysyncd: while ldap_connection.syncrepl_poll(all=1, msgid=ldap_search):
Mar 21 13:37:40 freeipa01 ipa-dnskeysyncd: File "/usr/lib64/python2.7/site-packages/ldap/syncrepl.py", line 405, in syncrepl_poll
Mar 21 13:37:40 freeipa01 ipa-dnskeysyncd: self.syncrepl_refreshdone()
Mar 21 13:37:40 freeipa01 ipa-dnskeysyncd: File "/usr/lib/python2.7/site-packages/ipaserver/dnssec/keysyncer.py", line 115, in syncrepl_refreshdone
Mar 21 13:37:40 freeipa01 ipa-dnskeysyncd: self.hsm_replica_sync()
Mar 21 13:37:40 freeipa01 ipa-dnskeysyncd: File "/usr/lib/python2.7/site-packages/ipaserver/dnssec/keysyncer.py", line 181, in hsm_replica_sync
Mar 21 13:37:40 freeipa01 ipa-dnskeysyncd: ipautil.run([paths.IPA_DNSKEYSYNCD_REPLICA])
Mar 21 13:37:40 freeipa01 ipa-dnskeysyncd: File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 512, in run
Mar 21 13:37:40 freeipa01 ipa-dnskeysyncd: raise CalledProcessError(p.returncode, arg_string, str(output))
Mar 21 13:37:40 freeipa01 ipa-dnskeysyncd: subprocess.CalledProcessError: Command '/usr/libexec/ipa/ipa-dnskeysync-replica' returned non-zero exit status 1
Mar 21 13:37:40 freeipa01 systemd: ipa-dnskeysyncd.service: main process exited, code=exited, status=1/FAILURE
Mar 21 13:37:40 freeipa01 systemd: Unit ipa-dnskeysyncd.service entered failed state.
Mar 21 13:37:40 freeipa01 systemd: ipa-dnskeysyncd.service failed.
[gatewayblend@freeipa01 ~]$

I'm not sure what the issue is.

Any help is appreciated.

Thank you,
Andrew Meyer