1:13 a.m.
Hi guys,
This is a new install, software used is:
ipa-server.x86_64 4.8.4-7.module+el8.2.0+6046+aaa49f96
389-ds-base.x86_64 1.4.2.4-8.module+el8.2.0+5959+cfcaedbd
I followed the install instructions in the documentation, and
everything went fine. I haven't added any users or groups yet.
I have a master and a replica. The master dies, but the replica seems
totally happy.
I restarted the master yesterday at 13:25. However, after a short time
running, the log files in /var/log/pki/pki-tomcat/ca/debug.. starts
filling up with java SocketExceptions like these:
2020-07-08 13:57:49 [profileChangeMonitor] SEVERE: Profile change
monitor: Caught exception: netscape.ldap.LDAPException: Server or
network error (81)
netscape.ldap.LDAPException: Server or network error (81)
at netscape.ldap.LDAPConnThread.networkError(Unknown Source)
at netscape.ldap.LDAPConnThread.run(Unknown Source)
at java.lang.Thread.run(Thread.java:748)
2020-07-08 13:57:49 [AuthorityMonitor] WARNING: AuthorityMonitor:
Failed to execute LDAP search for lightweight CAs:
netscape.ldap.LDAPException: Server or network error (81)
netscape.ldap.LDAPException: Server or network error (81)
at netscape.ldap.LDAPConnThread.networkError(Unknown Source)
at netscape.ldap.LDAPConnThread.run(Unknown Source)
at java.lang.Thread.run(Thread.java:748)
2020-07-08 13:57:49 [profileChangeMonitor] SEVERE: Unable to create
socket: java.net.SocketException: Network is unreachable (connect
failed)
java.net.SocketException: Network is unreachable (connect failed)
...
So it's pretty obvious that the LDAP server is not working properly.
In /var/log/dirsrv/slapd-YAK2-NET/errors I see:
389-Directory/1.4.2.4 B2020.121.2358
fipa001.yak2.net:636 (/etc/dirsrv/slapd-YAK2-NET)
[08/Jul/2020:13:25:36.982866396 +0200] - INFO - slapd_extract_cert - CA
CERT NAME: YAK2.NET IPA CA
[08/Jul/2020:13:25:37.002136210 +0200] - WARN - Security Initialization
- SSL alert: Sending pin request to SVRCore. You may need to run
systemd-tty-ask-password-agent to provide the password.
[08/Jul/2020:13:25:37.030257707 +0200] - INFO - slapd_extract_cert -
SERVER CERT NAME: Server-Cert
[08/Jul/2020:13:25:37.049921505 +0200] - INFO - Security Initialization
- SSL info: Enabling default cipher set.
[08/Jul/2020:13:25:37.074564197 +0200] - INFO - Security Initialization
- SSL info: Configured NSS Ciphers
[08/Jul/2020:13:25:37.091262372 +0200] - INFO - Security Initialization
- SSL info: TLS_AES_128_GCM_SHA256: enabled
...
All ciphers enabled
...
[08/Jul/2020:13:25:37.601009917 +0200] - INFO - Security Initialization
- slapd_ssl_init2 - Configured SSL version range: min: TLS1.0, ma
x: TLS1.2
[08/Jul/2020:13:25:37.616336615 +0200] - INFO - Security Initialization
- slapd_ssl_init2 - NSS adjusted SSL version range: min: TLS1.2,
max: TLS1.2
...
[08/Jul/2020:13:25:38.547791102 +0200] - NOTICE - bdb_start - Detected
Disorderly Shutdown last time Directory Server was running, recovering
database.
[08/Jul/2020:13:25:38.792474100 +0200] - ERR - attrcrypt_unwrap_key -
Failed to unwrap key for cipher AES
[08/Jul/2020:13:25:38.817182778 +0200] - ERR - attrcrypt_cipher_init -
Symmetric key failed to unwrap with the private key; Cert might have
been renewed since the key is wrapped. To recover the encrypted
contents, keep the wrapped symmetric key value.
[08/Jul/2020:13:25:38.852524974 +0200] - ERR - attrcrypt_unwrap_key -
Failed to unwrap key for cipher 3DES
[08/Jul/2020:13:25:38.871423186 +0200] - ERR - attrcrypt_cipher_init -
Symmetric key failed to unwrap with the private key; Cert might have
been renewed since the key is wrapped. To recover the encrypted
contents, keep the wrapped symmetric key value.
[08/Jul/2020:13:25:38.880537833 +0200] - ERR - attrcrypt_init - All
prepared ciphers are not available. Please disable attribute
encryption.
[08/Jul/2020:13:25:38.891681971 +0200] - ERR - attrcrypt_unwrap_key -
Failed to unwrap key for cipher AES
[08/Jul/2020:13:25:38.916694998 +0200] - ERR - attrcrypt_cipher_init -
Symmetric key failed to unwrap with the private key; Cert might have
been renewed since the key is wrapped. To recover the encrypted
contents, keep the wrapped symmetric key value.
[08/Jul/2020:13:25:38.925787580 +0200] - ERR - attrcrypt_unwrap_key -
Failed to unwrap key for cipher 3DES
[08/Jul/2020:13:25:38.934688993 +0200] - ERR - attrcrypt_cipher_init -
Symmetric key failed to unwrap with the private key; Cert might have
been renewed since the key is wrapped. To recover the encrypted
contents, keep the wrapped symmetric key value.
[08/Jul/2020:13:25:38.943696327 +0200] - ERR - attrcrypt_init - All
prepared ciphers are not available. Please disable attribute
encryption.
[08/Jul/2020:13:25:38.963177992 +0200] - ERR - schema-compat-plugin -
scheduled schema-compat-plugin tree scan in about 5 seconds after the
server startup!
[08/Jul/2020:13:25:39.005290186 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=dns,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.016002719 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=dns,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.025021142 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=keys,cn=sec,cn=dns,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.034249703 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=dns,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.043141171 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=dns,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.052203339 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=groups,cn=compat,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.061377434 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=computers,cn=compat,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.070224153 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=ng,cn=compat,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.079482188 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target ou=sudoers,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.088423317 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=users,cn=compat,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.097485949 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=vaults,cn=kra,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.106344275 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=vaults,cn=kra,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.115510556 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=vaults,cn=kra,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.124588060 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=vaults,cn=kra,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.133557279 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=vaults,cn=kra,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.142529788 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=vaults,cn=kra,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.151543892 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=vaults,cn=kra,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.160697059 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=vaults,cn=kra,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.169730169 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=vaults,cn=kra,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.178596620 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=vaults,cn=kra,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.187737945 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=vaults,cn=kra,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.200014978 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=ad,cn=etc,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.215529397 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=casigningcert cert-pki-
ca,cn=ca_renewal,cn=ipa,cn=etc,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.223802735 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=casigningcert cert-pki-
ca,cn=ca_renewal,cn=ipa,cn=etc,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.223802735 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=casigningcert cert-pki-
ca,cn=ca_renewal,cn=ipa,cn=etc,dc=yak2,dc=net does not exist
[08/Jul/2020:13:25:39.295259710 +0200] - WARN - NSACLPlugin - acl_parse
- The ACL target cn=automember rebuild membership,cn=tasks,cn=config
does not exist
[08/Jul/2020:13:25:39.307599195 +0200] - ERR - cos-plugin -
cos_dn_defs_cb - Skipping CoS Definition cn=Password
Policy,cn=accounts,dc=yak2,dc=net--no CoS Templates found, which should
be added before the CoS Definition.
...
[08/Jul/2020:13:25:39.414188379 +0200] - WARN - NSMMReplicationPlugin -
replica_check_for_data_reload - Disorderly shutdown for replica
dc=yak2,dc=net. Check if DB RUV needs to be updated
[08/Jul/2020:13:25:39.422657691 +0200] - ERR - set_krb5_creds - Could
not get initial credentials for principal [
ldap/fipa001.yak2.net(a)YAK2.NET] in keytab [FILE:/etc/dirsrv/ds.keytab]:
-1765328228 (Cannot contact any KDC for requested realm)
[08/Jul/2020:13:25:39.431578492 +0200] - WARN - NSMMReplicationPlugin -
replica_check_for_data_reload - Disorderly shutdown for replica
o=ipaca. Check if DB RUV needs to be updated
[08/Jul/2020:13:25:39.441463670 +0200] - ERR - NSMMReplicationPlugin -
bind_and_check_pwp - agmt="cn=caTofipa002.yak2.net" (fipa002:389) -
Replication bind with GSSAPI auth failed: LDAP error -1 (Can't contact
LDAP server) ()
[08/Jul/2020:13:25:39.458768924 +0200] - ERR - set_krb5_creds - Could
not get initial credentials for principal [
ldap/fipa001.yak2.net(a)YAK2.NET] in keytab [FILE:/etc/dirsrv/ds.keytab]:
-1765328228 (Cannot contact any KDC for requested realm)
[08/Jul/2020:13:25:39.477747370 +0200] - ERR - NSMMReplicationPlugin -
bind_and_check_pwp - agmt="cn=meTofipa002.yak2.net" (fipa002:389) -
Replication bind with GSSAPI auth failed: LDAP error -1 (Can't contact
LDAP server) ()
[08/Jul/2020:13:25:39.494528440 +0200] - INFO - slapd_daemon - slapd
started. Listening on All Interfaces port 389 for LDAP requests
[08/Jul/2020:13:25:39.503748199 +0200] - INFO - slapd_daemon -
Listening on All Interfaces port 636 for LDAPS requests
[08/Jul/2020:13:25:39.512913530 +0200] - INFO - slapd_daemon -
Listening on /var/run/slapd-YAK2-NET.socket for LDAPI requests
[08/Jul/2020:13:25:39.521834655 +0200] - ERR - schema-compat-plugin -
schema-compat-plugin tree scan will start in about 5 seconds!
[08/Jul/2020:13:25:42.619282707 +0200] - ERR - set_krb5_creds - Could
not get initial credentials for principal [
ldap/fipa001.yak2.net(a)YAK2.NET] in keytab [FILE:/etc/dirsrv/ds.keytab]:
-1765328228 (Cannot contact any KDC for requested realm)
[08/Jul/2020:13:25:42.660769148 +0200] - ERR - set_krb5_creds - Could
not get initial credentials for principal [
ldap/fipa001.yak2.net(a)YAK2.NET] in keytab [FILE:/etc/dirsrv/ds.keytab]:
-1765328228 (Cannot contact any KDC for requested realm)
[08/Jul/2020:13:25:44.517541271 +0200] - ERR - schema-compat-plugin -
warning: no entries set up under ou=sudoers,dc=yak2,dc=net
[08/Jul/2020:13:25:44.570564330 +0200] - ERR - schema-compat-plugin -
warning: no entries set up under cn=ng, cn=compat,dc=yak2,dc=net
[08/Jul/2020:13:25:44.614807034 +0200] - ERR - schema-compat-plugin -
warning: no entries set up under cn=computers, cn=compat,dc=yak2,dc=net
[08/Jul/2020:13:25:44.635448724 +0200] - ERR - schema-compat-plugin -
Finished plugin initialization.
[08/Jul/2020:13:25:48.807433248 +0200] - INFO - NSMMReplicationPlugin -
bind_and_check_pwp - agmt="cn=meTofipa002.yak2.net" (fipa002:389):
Replication bind with GSSAPI auth resumed
[08/Jul/2020:13:25:48.853993542 +0200] - INFO - NSMMReplicationPlugin -
bind_and_check_pwp - agmt="cn=caTofipa002.yak2.net" (fipa002:389):
Replication bind with GSSAPI auth resumed
[08/Jul/2020:15:00:49.240767214 +0200] - ERR - NSMMReplicationPlugin -
bind_and_check_pwp - agmt="cn=caTofipa002.yak2.net" (fipa002:389) -
Replication bind with GSSAPI auth failed: LDAP error -1 (Can't contact
LDAP server) ()
[09/Jul/2020:07:42:10.753040947 +0200] - INFO - NSMMReplicationPlugin -
bind_and_check_pwp - agmt="cn=caTofipa002.yak2.net" (fipa002:389):
Replication bind with GSSAPI auth resumed
The log files grow to huge sizes very quickly and fills up the /var
filesystem which stops the server.
It's not clear to me why this is happening, any advice and tips to get
it fixed are appreciated.
/tony
--
Tony Albers - Systems Architect - IT Development
Royal Danish Library, Victor Albecks Vej 1, 8000 Aarhus C, Denmark
Tel: +45 2566 2383 - CVR/SE: 2898 8842 - EAN: 5798000792142