On 8/9/17 3:05 AM, thierry bordaz wrote:
> Hi Ian,
>
> Thanks for having gather those data.
>
> #
> # So pkidbuser entries have a same (old) userCertificate likely
> generated during install
> # But only freeipa-sea has a new one created on freeipa-sea
> around Jun 8th 2017 05:54:16
> # This recent certificate is identified by 5938e688000104290000
> #
> [root@freeipa-sea ianh]# ldapsearch -LLL -D 'cn=directory
> manager' -W -b "uid=pkidbuser,ou=people,o=ipaca" nscpentrywsi
> dn: uid=pkidbuser,ou=people,o=ipaca
> ...
> nscpentrywsi: userCertificate::
> MIIDczCCAlugAwIBAgIBBDANBgkqhkiG9w0BAQsFADA0MR <snip>
> nscpentrywsi: userCertificate;vucsn-5938e688000104290000::
> MIIDbjCCAlagAwIBAgI <snip>
>
> [root@seattlenfs ianh]# ldapsearch -LLL -D 'cn=directory manager'
> -W -b uid=pkidbuser,ou=people,o=ipaca nscpentrywsi
> dn: uid=pkidbuser,ou=people,o=ipaca
> nscpentrywsi: userCertificate::
> MIIDczCCAlugAwIBAgIBBDANBgkqhkiG9w0BAQsFADA0MR <snip>
>
>
>
> #
> # why 5938e688000104290000 value was not propagated to seattlenfs ?
> # The most recent update (from freeipa-sea) that was replicated
> to seattlenfs
> # is 1 year old (57be8043000704290000 - Aug 25th 2016 05:21:07).
> # In addition seattlenfs received direct update (last one was in
> Jan 1017) that were not
> # replicated to freeipa-sea
> #
> # The two servers have diverged because they can not replicate to
> eachother because
> # they were not correctly initialize.
> # They have different "replicageneration" (57c291d9000004290000
> vs 55c8f3ae000000600000)
> #
> # It is looking like freeipa-sea was created one or two years ago
> and used to initialized seattlenfs.
> # But later freeipa-sea was recreated.
>
That's about right.
>
> #
> [root@freeipa-sea ianh]# ldapsearch -LLL -D 'cn=directory
> manager' -W -b "o=ipaca"
>
"(&(objectclass=nstombstone)(nsUniqueId=ffffffff-ffffffff-ffffffff-ffffffff))"
> Enter LDAP Password:
> dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
> nsds50ruv: {replicageneration} 57c291d9000004290000
> nsds50ruv: {replica 1065 ldap://freeipa-sea.bpt.rocks:389}
> 57f840bf000004290000 598a1c40000104290000
> nsds50ruv: {replica 1290 ldap://seattlenfs.bpt.rocks:389}
> nsruvReplicaLastModified: {replica 1065
> ldap://freeipa-sea.bpt.rocks:389} 598a1c16
> nsruvReplicaLastModified: {replica 1290
> ldap://seattlenfs.bpt.rocks:389} 00000000
>
> [root@seattlenfs ianh]# ldapsearch -LLL -D 'cn=directory manager'
> -W -b "o=ipaca"
>
"(&(objectclass=nstombstone)(nsUniqueId=ffffffff-ffffffff-ffffffff-ffffffff))"
> dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
> nsds50ruv: {replicageneration} 55c8f3ae000000600000
> nsds50ruv: {replica 1065 ldap://freeipa-sea.bpt.rocks:389}
> 57b103d4000004290000 57be8043000704290000
> nsds50ruv: {replica 1290 ldap://seattlenfs.bpt.rocks:389}
> 57be804c0000050a0000 587236150000050a0000
> nsruvReplicaLastModified: {replica 1290
> ldap://seattlenfs.bpt.rocks:389} 00000000
> nsruvReplicaLastModified: {replica 1065
> ldap://freeipa-sea.bpt.rocks:389} 00000000
>
>
>
> In conclusion:
> From replication pov, the two instances can not communicate.
> One solution would be to identify which instance is the good one,
> you want to keep.
> and reinit the second one from that reference.
>
What exactly does reinit mean? I have run
ipa-replica-manage re-initialize --from freeipa-sea.bpt.rocks
This triggers the reinit of 'dc=ipadomain,dc=com' suffix but not for
o=ipaca suffix. To reinit o=ipaca you may use
ipa-cs-replica-manager re-initialize --from freeipa-sea.bpt.rocks.
Note that it will overwrite all data of ipaca on seattlenfs with the
data from freeipa-sea. This is why it is important to know which server
is the valid one.
On freeipa-sea and seattlenfs you may check the ipaca replica agreements
status before/after reinit. I would guess they are currently reporting
failures between these two replicas.
ldapsearch -LLL -D 'cn=directory manager' -W -b
"cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config"
"objectClass=nsds5replicationagreement" nsds5replicaLastUpdateStatus
several times over the years when replication has stopped.
Replication actually is working, as far as user and machine accounts
and attributes are concerned anyway, and has been for a while.
There is a zombie server freeipa-dal.bpt.rocks that I can't get rid
of... it shows up in the GUI on the Topology page but generates a
Server not found error. I don't know if that's related.
Zombie servers have been discussed a lot on this mailing list. You may
get rid of them with list-ruv/clean-ruv subcommand. Replication usually
manage to work fine even if some zombie entries exist but it is a good
practice to clean them.
regards
thierry
>
> On 08/08/2017 10:33 PM, Ian Harding via FreeIPA-users wrote:
>>
>> On 8/7/17 1:44 AM, thierry bordaz wrote:
>>
>>>
>>>
>>> On 08/07/2017 09:22 AM, Florence Blanc-Renaud via FreeIPA-users wrote:
>>>> On 08/04/2017 11:02 PM, Ian Harding via FreeIPA-users wrote:
>>>>> On 8/4/17 2:16 AM, Florence Blanc-Renaud wrote:
>>>>>
>>>>>> On 08/03/2017 11:13 PM, Ian Harding via FreeIPA-users wrote:
>>>>>>> On 08/03/2017 12:28 AM, Florence Blanc-Renaud wrote:
>>>>>>>> On 08/02/2017 11:51 PM, Ian Harding via FreeIPA-users
wrote:
>>>>>>>>> On 08/02/2017 12:11 AM, Florence Blanc-Renaud wrote:
>>>>>>>>>> On 08/02/2017 01:43 AM, Ian Harding wrote:
>>>>>>>>>>> On 08/01/2017 12:03 PM, Rob Crittenden
wrote:
>>>>>>>>>>>> Ian Harding wrote:
>>>>>>>>>>>>> On 08/01/2017 07:39 AM, Florence
Blanc-Renaud wrote:
>>>>>>>>>>>>>> On 08/01/2017 03:11 PM, Ian
Harding wrote:
>>>>>>>>>>>>>>> On 08/01/2017 01:48 AM,
Florence Blanc-Renaud wrote:
>>>>>>>>>>>>>>>> On 08/01/2017 01:32 AM,
Ian Harding via FreeIPA-users
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 07/31/2017 11:34
AM, Rob Crittenden wrote:
>>>>>>>>>>>>>>>>>> Ian Harding via
FreeIPA-users wrote:
>>>>>>>>>>>>>>>>>>> I had an
unexpected restart of an IPA server that
>>>>>>>>>>>>>>>>>>> had
apparently had
>>>>>>>>>>>>>>>>>>> updates run
but had not been restarted. ipactl says
>>>>>>>>>>>>>>>>>>> pki-tomcatd
>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>> not start.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Strangely,
the actual service appears to be running:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> dogtag is an
application within tomcat so tomcat can
>>>>>>>>>>>>>>>>>> run without
>>>>>>>>>>>>>>>>>> dogtag
>>>>>>>>>>>>>>>>>> running.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> We need to see
more of the dogtag debug log to see
>>>>>>>>>>>>>>>>>> what is going
on.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> It looks like an
authentication problem...
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
[28/Jul/2017:10:08:47][localhost-startStop-1]: SSL
>>>>>>>>>>>>>>>>> handshake happened
>>>>>>>>>>>>>>>>> Could not connect to
LDAP server host
>>>>>>>>>>>>>>>>> seattlenfs.bpt.rocks
port 636
>>>>>>>>>>>>>>>>> Error
netscape.ldap.LDAPException: Authentication
>>>>>>>>>>>>>>>>> failed (49)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> dogtag stores its
internal data in the LDAP server and
>>>>>>>>>>>>>>>> needs to
>>>>>>>>>>>>>>>> establish a secure LDAP
connection. You can check how
>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>> connection is configured
in
>>>>>>>>>>>>>>>>
/etc/pki/pki-tomcat/ca/CS.cfg, look for
>>>>>>>>>>>>>>>> the lines:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
internaldb.ldapauth.authtype=SslClientAuth
>>>>>>>>>>>>>>>>
internaldb.ldapauth.bindDN=cn=Directory Manager
>>>>>>>>>>>>>>>>
internaldb.ldapauth.bindPWPrompt=internaldb
>>>>>>>>>>>>>>>>
internaldb.ldapauth.clientCertNickname=subsystemCert
>>>>>>>>>>>>>>>> cert-pki-ca
>>>>>>>>>>>>>>>>
internaldb.ldapconn.host=vm-...
>>>>>>>>>>>>>>>>
internaldb.ldapconn.port=636
>>>>>>>>>>>>>>>>
internaldb.ldapconn.secureConn
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> authtype can be
SslClientAuth (authentication with a ssl
>>>>>>>>>>>>>>>> certificate) or BasicAuth
(authentication with a bind
>>>>>>>>>>>>>>>> DN and
>>>>>>>>>>>>>>>> password stored in
>>>>>>>>>>>>>>>>
/var/lib/pki/pki-tomcat/conf/password.conf).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> You can use this
information to manually check the
>>>>>>>>>>>>>>>> credentials. For
>>>>>>>>>>>>>>>> instance with
sslclientauth:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> export
LDAPTLS_CACERTDIR=/etc/pki/pki-tomcat/alias
>>>>>>>>>>>>>>>> export
LDAPTLS_CERT='subsystemCert cert-pki-ca'
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ldapsearch -H
ldaps://`hostname`:636 -b "" -s base -Y
>>>>>>>>>>>>>>>> EXTERNAL
>>>>>>>>>>>>>>>> (provide the password
from
>>>>>>>>>>>>>>>>
/etc/pki/pki-tomcat/alias/pwdfile.txt)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I found this:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
internaldb.ldapauth.authtype=SslClientAuth
>>>>>>>>>>>>>>>
internaldb.ldapauth.bindDN=uid=pkidbuser,ou=people,o=ipaca
>>>>>>>>>>>>>>>
internaldb.ldapauth.bindPWPrompt=internaldb
>>>>>>>>>>>>>>>
internaldb.ldapauth.clientCertNickname=subsystemCert
>>>>>>>>>>>>>>> cert-pki-ca
>>>>>>>>>>>>>>>
internaldb.ldapconn.cloneReplicationPort=389
>>>>>>>>>>>>>>> ...
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> and when I try the ldapsearch
I am presented with a
>>>>>>>>>>>>>>> prompt to provide
>>>>>>>>>>>>>>> a pin/password
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Please enter pin, password,
or pass phrase for security
>>>>>>>>>>>>>>> token 'ldap(0)':
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> but there is no password
file...
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> you are right, in 4.4. there is
no pwdfile.txt and the
>>>>>>>>>>>>>> password can be
>>>>>>>>>>>>>> found in
/var/lib/pki/pki-tomcat/conf/password.conf
>>>>>>>>>>>>>> (with the tag
>>>>>>>>>>>>>> internal=...)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Can you check if the password
with the tag internal=...
>>>>>>>>>>>>>> allows to read
>>>>>>>>>>>>>> the keys from the NSS db?
>>>>>>>>>>>>>> certutil -K -d
/etc/pki/pki-tomcat/alias
>>>>>>>>>>>>>> (provide password)
>>>>>>>>>>>>>
>>>>>>>>>>>>> That works...
>>>>>>>>>>>>>
>>>>>>>>>>>>> # certutil -K -d
/etc/pki/pki-tomcat/alias
>>>>>>>>>>>>> certutil: Checking token "NSS
Certificate DB" in slot
>>>>>>>>>>>>> "NSS User Private
>>>>>>>>>>>>> Key and Certificate Services"
>>>>>>>>>>>>> Enter Password or Pin for "NSS
Certificate DB":
>>>>>>>>>>>>> < 0> rsa
0f327e760a7eecdcf6973f5dc57ca5367c592d64 (orphan)
>>>>>>>>>>>>> < 1> rsa
b12580c7c696cfcd8aefc9405a7a870b24b7b96a NSS
>>>>>>>>>>>>> Certificate
>>>>>>>>>>>>> DB:auditSigningCert cert-pki-ca
>>>>>>>>>>>>> < 2> rsa
881b7254c40fa40bc50681bcc8d37bb3eb49937e
>>>>>>>>>>>>> caSigningCert
>>>>>>>>>>>>> cert-pki-ca
>>>>>>>>>>>>> < 3> rsa
fa9a255a1d15585ac28064c0f4986e416bc48403 NSS
>>>>>>>>>>>>> Certificate
>>>>>>>>>>>>> DB:ocspSigningCert cert-pki-ca
>>>>>>>>>>>>> < 4> rsa
3fb609d0f7d72c2d325d6a2dc16577a7f7e5a01f
>>>>>>>>>>>>> Server-Cert
>>>>>>>>>>>>> cert-pki-ca
>>>>>>>>>>>>> < 5> rsa
1e9479a9556af9339bb5e4552ccbd381d3c38856 NSS
>>>>>>>>>>>>> Certificate
>>>>>>>>>>>>> DB:subsystemCert cert-pki-ca
>>>>>>>>>>>>>
>>>>>>>>>>>>> But this doesn't (with the same
password from password.conf)
>>>>>>>>>>>>>
>>>>>>>>>>>>> # ldapsearch -H
ldaps://`hostname`:636 -b "" -s base -Y
>>>>>>>>>>>>> EXTERNAL
>>>>>>>>>>>>> Please enter pin, password, or pass
phrase for security
>>>>>>>>>>>>> token 'ldap(0)':
>>>>>>>>>>>>> SASL/EXTERNAL authentication started
>>>>>>>>>>>>> ldap_sasl_interactive_bind_s: Invalid
credentials (49)
>>>>>>>>>>>>>
>>>>>>>>>>>>> That password is getting me somewhere
though, since if I
>>>>>>>>>>>>> put in a
>>>>>>>>>>>>> nonsense or incorrect password it
just prompts over and
>>>>>>>>>>>>> over.
>>>>>>>>>>>>
>>>>>>>>>>>> Let's step back a second. You
upgraded from what to what?
>>>>>>>>>>>
>>>>>>>>>>> There wasn't much of a change... I just
assumed someone ran
>>>>>>>>>>> yum upgrade and didn't restart, then the
power outage... it
>>>>>>>>>>> looks like not much of a version change
though.
>>>>>>>>>>>
>>>>>>>>>>> # grep ipa /var/log/yum.log
>>>>>>>>>>> Jan 08 04:45:32 Installed:
>>>>>>>>>>> ipa-common-4.4.0-14.el7.centos.1.1.noarch
>>>>>>>>>>> Jan 08 04:45:32 Installed:
>>>>>>>>>>>
ipa-client-common-4.4.0-14.el7.centos.1.1.noarch
>>>>>>>>>>> Jan 08 04:46:06 Updated:
libipa_hbac-1.14.0-43.el7_3.4.x86_64
>>>>>>>>>>> Jan 08 04:46:07 Updated:
>>>>>>>>>>> python-libipa_hbac-1.14.0-43.el7_3.4.x86_64
>>>>>>>>>>> Jan 08 04:46:08 Installed:
>>>>>>>>>>> python-ipaddress-1.0.16-2.el7.noarch
>>>>>>>>>>> Jan 08 04:47:04 Installed:
>>>>>>>>>>>
python2-ipalib-4.4.0-14.el7.centos.1.1.noarch
>>>>>>>>>>> Jan 08 04:47:05 Installed:
>>>>>>>>>>>
python2-ipaclient-4.4.0-14.el7.centos.1.1.noarch
>>>>>>>>>>> Jan 08 04:47:05 Updated:
>>>>>>>>>>>
ipa-admintools-4.4.0-14.el7.centos.1.1.noarch
>>>>>>>>>>> Jan 08 04:47:05 Installed:
>>>>>>>>>>>
ipa-server-common-4.4.0-14.el7.centos.1.1.noarch
>>>>>>>>>>> Jan 08 04:47:06 Installed:
>>>>>>>>>>>
python2-ipaserver-4.4.0-14.el7.centos.1.1.noarch
>>>>>>>>>>> Jan 08 04:47:07 Updated:
sssd-ipa-1.14.0-43.el7_3.4.x86_64
>>>>>>>>>>> Jan 08 04:47:17 Updated:
>>>>>>>>>>> ipa-client-4.4.0-14.el7.centos.1.1.x86_64
>>>>>>>>>>> Jan 08 04:47:23 Updated:
>>>>>>>>>>> ipa-server-4.4.0-14.el7.centos.1.1.x86_64
>>>>>>>>>>> Jan 08 04:47:27 Updated:
>>>>>>>>>>>
ipa-server-dns-4.4.0-14.el7.centos.1.1.noarch
>>>>>>>>>>> Jan 08 04:47:36 Installed:
>>>>>>>>>>>
ipa-python-compat-4.4.0-14.el7.centos.1.1.noarch
>>>>>>>>>>> Jan 08 04:50:07 Erased:
>>>>>>>>>>> ipa-python-4.2.0-15.0.1.el7.centos.19.x86_64
>>>>>>>>>>> Jul 28 09:55:41 Updated:
>>>>>>>>>>> ipa-common-4.4.0-14.el7.centos.7.noarch
>>>>>>>>>>> Jul 28 09:55:42 Updated:
>>>>>>>>>>>
ipa-client-common-4.4.0-14.el7.centos.7.noarch
>>>>>>>>>>> Jul 28 09:55:43 Updated:
>>>>>>>>>>>
ipa-server-common-4.4.0-14.el7.centos.7.noarch
>>>>>>>>>>> Jul 28 09:55:43 Updated:
>>>>>>>>>>> python2-ipalib-4.4.0-14.el7.centos.7.noarch
>>>>>>>>>>> Jul 28 09:55:44 Updated:
>>>>>>>>>>>
python2-ipaclient-4.4.0-14.el7.centos.7.noarch
>>>>>>>>>>> Jul 28 09:55:45 Updated:
>>>>>>>>>>>
python2-ipaserver-4.4.0-14.el7.centos.7.noarch
>>>>>>>>>>> Jul 28 09:55:45 Updated:
>>>>>>>>>>> ipa-admintools-4.4.0-14.el7.centos.7.noarch
>>>>>>>>>>> Jul 28 09:55:46 Updated:
>>>>>>>>>>> ipa-client-4.4.0-14.el7.centos.7.x86_64
>>>>>>>>>>> Jul 28 09:55:48 Updated:
>>>>>>>>>>> ipa-server-4.4.0-14.el7.centos.7.x86_64
>>>>>>>>>>> Jul 28 09:55:48 Updated:
>>>>>>>>>>> ipa-server-dns-4.4.0-14.el7.centos.7.noarch
>>>>>>>>>>> Jul 28 09:55:48 Updated:
>>>>>>>>>>>
ipa-python-compat-4.4.0-14.el7.centos.7.noarch
>>>>>>>>>>>
>>>>>>>>>>> # grep pki /var/log/yum.log
>>>>>>>>>>> Jan 08 04:46:05 Updated:
pki-base-10.3.3-14.el7_3.noarch
>>>>>>>>>>> Jan 08 04:46:09 Updated:
krb5-pkinit-1.14.1-27.el7_3.x86_64
>>>>>>>>>>> Jan 08 04:47:19 Installed:
>>>>>>>>>>> pki-base-java-10.3.3-14.el7_3.noarch
>>>>>>>>>>> Jan 08 04:47:19 Updated:
pki-tools-10.3.3-14.el7_3.x86_64
>>>>>>>>>>> Jan 08 04:47:21 Updated:
pki-server-10.3.3-14.el7_3.noarch
>>>>>>>>>>> Jan 08 04:47:22 Updated:
pki-kra-10.3.3-14.el7_3.noarch
>>>>>>>>>>> Jan 08 04:47:22 Updated:
pki-ca-10.3.3-14.el7_3.noarch
>>>>>>>>>>> Jul 28 10:13:16 Updated:
pki-base-10.3.3-19.el7_3.noarch
>>>>>>>>>>> Jul 28 10:13:17 Updated:
pki-base-java-10.3.3-19.el7_3.noarch
>>>>>>>>>>> Jul 28 10:13:17 Updated:
pki-tools-10.3.3-19.el7_3.x86_64
>>>>>>>>>>> Jul 28 10:13:21 Updated:
pki-server-10.3.3-19.el7_3.noarch
>>>>>>>>>>> Jul 28 10:13:21 Updated:
pki-kra-10.3.3-19.el7_3.noarch
>>>>>>>>>>> Jul 28 10:13:21 Updated:
pki-ca-10.3.3-19.el7_3.noarch
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Are you running in SELinux enforcing
mode?
>>>>>>>>>>>
>>>>>>>>>>> # getenforce
>>>>>>>>>>> Enforcing
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> If so can you run:
>>>>>>>>>>>>
>>>>>>>>>>>> # ausearch -m AVC
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> # ausearch -m AVC
>>>>>>>>>>> The NOLOG option to log_format is deprecated.
Please use
>>>>>>>>>>> the write_logs option.
>>>>>>>>>>> The NOLOG option is overriding the write_logs
current setting.
>>>>>>>>>>> ----
>>>>>>>>>>> time->Mon Mar 14 10:39:01 2016
>>>>>>>>>>> type=SYSCALL msg=audit(1457977141.767:17537):
arch=c000003e
>>>>>>>>>>> syscall=2 success=no exit=-13 a0=7fbcab038d10
a1=800 a2=1
>>>>>>>>>>> a3=7fbca60f42e0 items=0 ppid=1406 pid=12965
auid=4294967295
>>>>>>>>>>> uid=0 gid=0 euid=581400001 suid=0
fsuid=581400001
>>>>>>>>>>> egid=581400001 sgid=0 fsgid=581400001
tty=(none)
>>>>>>>>>>> ses=4294967295 comm="sshd"
exe="/usr/sbin/sshd"
>>>>>>>>>>> subj=system_u:system_r:sshd_t:s0-s0:c0.c1023
key=(null)
>>>>>>>>>>> type=AVC msg=audit(1457977141.767:17537):
avc: denied {
>>>>>>>>>>> read } for pid=12965 comm="sshd"
name="authorized_keys"
>>>>>>>>>>> dev="dm-2" ino=116393517
>>>>>>>>>>>
scontext=system_u:system_r:sshd_t:s0-s0:c0.c1023
>>>>>>>>>>> tcontext=system_u:object_r:unlabeled_t:s0
tclass=file
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> I suspect that the subsystem cert was
renewed and
>>>>>>>>>>>> everything wasn't
>>>>>>>>>>>> updated properly. If I'm right this
is unrelated to the
>>>>>>>>>>>> upgrade it just
>>>>>>>>>>>> shone a spotlight on it.
>>>>>>>>>>>>
>>>>>>>>>>>> Can you run these commands:
>>>>>>>>>>>>
>>>>>>>>>>>> $ ldapsearch -LLL -D 'cn=directory
manager' -W -b
>>>>>>>>>>>> uid=pkidbuser,ou=people,o=ipaca
userCertificate description
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> This returns
>>>>>>>>>>>
>>>>>>>>>>> $ ldapsearch -LLL -D 'cn=directory
manager' -W -b
>>>>>>>>>>> uid=pkidbuser,ou=people,o=ipaca
userCertificate description
>>>>>>>>>>> Enter LDAP Password:
>>>>>>>>>>> dn: uid=pkidbuser,ou=people,o=ipaca
>>>>>>>>>>> userCertificate::
>>>>>>>>>>>
MIIDczCCAlugAwIBAgIBBDANBgkqhkiG9w0BAQsFADA0MRIwEAYDVQQKDAlC
>>>>>>>>>>> ... stuff ...
>>>>>>>>>>>
ZNo4eX7jNT8Aa23gDl6xuLVa1SxlyE8XEo8dfAPXMJoFc=
>>>>>>>>>>> description: 2;4;CN=Certificate
Authority,O=BPT.ROCKS;CN=CA
>>>>>>>>>>> Subsystem,O=BPT.ROCKS
>>>>>>>>>>>
>>>>>>>>>>>> and
>>>>>>>>>>>>
>>>>>>>>>>>> # certutil -L -d
/etc/pki/pki-tomcat/alias -n 'subsystemCert
>>>>>>>>>>>> cert-pki-ca' | grep Serial
>>>>>>>>>>>> # certutil -L -d
/etc/pki/pki-tomcat/alias -n
>>>>>>>>>>>> 'subsystemCert cert-pki-ca' -a
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> These both return
>>>>>>>>>>>
>>>>>>>>>>> # certutil -K -d /etc/pki/pki-tomcat/alias -n
>>>>>>>>>>> 'subsystemCert cert-pki-ca' | grep
Serial
>>>>>>>>>>> Enter Password or Pin for "NSS
Certificate DB":
>>>>>>>>>>> certutil: problem listing keys:
SEC_ERROR_UNRECOGNIZED_OID:
>>>>>>>>>>> Unrecognized Object Identifier.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> # certutil -K -d /etc/pki/pki-tomcat/alias -n
>>>>>>>>>>> 'subsystemCert cert-pki-ca' -a
>>>>>>>>>>> certutil: Checking token "NSS
Certificate DB" in slot "NSS
>>>>>>>>>>> User Private Key and Certificate
Services"
>>>>>>>>>>> Enter Password or Pin for "NSS
Certificate DB":
>>>>>>>>>>> certutil: problem listing keys:
SEC_ERROR_UNRECOGNIZED_OID:
>>>>>>>>>>> Unrecognized Object Identifier.
>>>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> you made a typo and requested the keys (-K)
instead of the
>>>>>>>>>> certs (-L). Please retry with -L and you should
be able to
>>>>>>>>>> see the certificate serial.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [root@seattlenfs ianh]# ldapsearch -LLL -D
'cn=directory
>>>>>>>>> manager' -W -b uid=pkidbuser,ou=people,o=ipaca
>>>>>>>>> userCertificate description
>>>>>>>>> Enter LDAP Password:
>>>>>>>>> dn: uid=pkidbuser,ou=people,o=ipaca
>>>>>>>>> userCertificate::
>>>>>>>>>
MIIDczCCAlugAwIBAgIBBDANBgkqhkiG9w0BAQsFADA0MRIwEAYDVQQKDAlC
>>>>>>>>> ...
>>>>>>>>> ZNo4eX7jNT8Aa23gDl6xuLVa1SxlyE8XEo8dfAPXMJoFc=
>>>>>>>>> description: 2;4;CN=Certificate
Authority,O=BPT.ROCKS;CN=CA
>>>>>>>>> Subsystem,O=BPT.ROCKS
>>>>>>>>>
>>>>>>>>> So the serial is 4?
>>>>>>>>>
>>>>>>>>> [root@seattlenfs ianh]# certutil -L -d
>>>>>>>>> /etc/pki/pki-tomcat/alias -n 'subsystemCert
cert-pki-ca' |
>>>>>>>>> grep Serial
>>>>>>>>> Serial Number: 1341718541 (0x4ff9000d)
>>>>>>>>>
>>>>>>>>> Which is NOT 4...
>>>>>>>>>
>>>>>>>>> [root@seattlenfs ianh]# certutil -L -d
>>>>>>>>> /etc/pki/pki-tomcat/alias -n 'subsystemCert
cert-pki-ca' -a
>>>>>>>>> -----BEGIN CERTIFICATE-----
>>>>>>>>>
MIIDbjCCAlagAwIBAgIET/kADTANBgkqhkiG9w0BAQsFADA0MRIwEAYDVQQKDAlC
>>>>>>>>> ...
>>>>>>>>> sjeiFbUFtoMnfmHLIYpe5QAR
>>>>>>>>> -----END CERTIFICATE-----
>>>>>>>>>
>>>>>>>>> And this cert is NOT the same.
>>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> when certmonger renews a certificate, it runs pre-save
and
>>>>>>>> post-save commands (which you can see in the output of
getcert
>>>>>>>> list). In your case, the post-save command for
subsystemCert
>>>>>>>> cert-pki-ca probably failed.
>>>>>>>>
>>>>>>>> This command (on the renewal master) is responsible for
>>>>>>>> copying the new cert from the NSS db to the LDAP server.
You
>>>>>>>> need first to check which IPA server is the renewal
master (as
>>>>>>>> the serial numbers are in a completely different range, I
>>>>>>>> assume that you have multiple IPA servers configured with
a CA
>>>>>>>> instance):
>>>>>>>>
>>>>>>>> $ export BASEDN=`grep basedn /etc/ipa/default.conf | cut
-d=
>>>>>>>> -f2-`
>>>>>>>> $ kinit admin
>>>>>>>> $ ldapsearch -LLL -Q -h `hostname` -p 389 -Y GSSAPI -b
>>>>>>>> "cn=masters,cn=ipa,cn=etc,$BASEDN"
>>>>>>>> '(&(cn=CA)(ipaConfigString=caRenewalMaster))'
dn
>>>>>>>> dn:
>>>>>>>>
cn=CA,cn=vm-ipaserver.ipadomain.com,cn=masters,cn=ipa,cn=etc,dc=ipadomain,dc=com
>>>>>>>>
>>>>>>>>
>>>>>>>> In this example the renewal master is vm-ipaserver.
>>>>>>>>
>>>>>>>
>>>>>>> I get this
>>>>>>>
>>>>>>> $ ldapsearch -LLL -Q -h `hostname` -p 389 -Y GSSAPI -b
>>>>>>> "cn=masters,cn=ipa,cn=etc,$BASEDN"
>>>>>>> '(&(cn=CA)(ipaConfigString=caRenewalMaster))' dn
>>>>>>> dn:
>>>>>>>
cn=CA,cn=freeipa-sea.bpt.rocks,cn=masters,cn=ipa,cn=etc,dc=bpt,dc=rocks
>>>>>>>
>>>>>>>
>>>>>>> Which is the other FreeIPA server.
>>>>>>>
>>>>>> Hi Ian,
>>>>>>
>>>>>> it's getting difficult to follow this thread, so could we
step
>>>>>> back and summarize?
>>>>> Yes, it is. Sorry.
>>>>>>
>>>>>> - On server freeipa-sea (=renewal master), the CA component is
>>>>>> installed and 'subsystemCert cert-pki-ca' has Serial
number 4 in
>>>>>> the NSSDB /etc/pki/pki-tomcat/alias
>>>>> No?
>>>>>
>>>>> [root@freeipa-sea ianh]# certutil -L -d /etc/pki/pki-tomcat/alias
>>>>> -n 'subsystemCert cert-pki-ca' | grep Serial
>>>>> Serial Number: 1341718541 (0x4ff9000d)
>>>>>> - On server seattlenfs (where PKI fails to restart), the CA
>>>>>> component is installed and 'subsystemCert cert-pki-ca'
has
>>>>>> Serial number 1341718541 in the NSSDB /etc/pki/pki-tomcat/alias.
>>>>> Yes
>>>>>
>>>>> [root@seattlenfs ianh]# certutil -L -d /etc/pki/pki-tomcat/alias
>>>>> -n 'subsystemCert cert-pki-ca' | grep Serial
>>>>> Serial Number: 1341718541 (0x4ff9000d)
>>>>>
>>>>>>
>>>>>> - In the LDAP servers, the cert is stored in the entry
>>>>>> uid=pkidbuser,ou=people,o=ipaca with Serial number 4 (i.e. it is
>>>>>> the cert from freeipa-sea).
>>>>> This is where the difference is. There are two certificates on
>>>>> the master and only one on the other. I wonder if this is a
>>>>> result of the slow motion replication trainwreck I have going on
>>>>> due to a zombie replica in the ldap database that no longer
>>>>> exists in the real world and I can't seem to get rid of.
>>>>>
>>>> Hi,
>>>>
>>>> the remaining issue is a replication problem. There is a
>>>> "Troubleshooting replication" section [1] in IdM Admin guide
that
>>>> may help you.
>>>>
>>>> Regarding your zombie replica, which commands did you use to try
>>>> to remove it, and what was the output?
>>>>
>>>> Flo
>>>>
>>>> [1]
>>>>
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/...
>>>>
>>>
>>> Hi,
>>>
>>> To debug replication latency or problem we would need several info
>>> but starting with the results of the following commands
>>>
>>> [root@freeipa-sea ianh]# ldapsearch -LLL -D 'cn=directory manager'
>>> -W -b "uid=pkidbuser,ou=people,o=ipaca" nscpentrywsi
>> [root@freeipa-sea ianh]# ldapsearch -LLL -D 'cn=directory manager'
>> -W -b "uid=pkidbuser,ou=people,o=ipaca" nscpentrywsi
>> Enter LDAP Password:
>> dn: uid=pkidbuser,ou=people,o=ipaca
>> nscpentrywsi: dn: uid=pkidbuser,ou=people,o=ipaca
>> nscpentrywsi:
>> entryusn;adcsn-5938e688000104290005;vucsn-5938e688000104290005:
>> 75274897
>> nscpentrywsi:
>> modifyTimestamp;adcsn-5938e688000104290004;vucsn-5938e6880001042
>> 90004: 20170608055334Z
>> nscpentrywsi:
>> modifiersName;adcsn-5938e688000104290003;vucsn-5938e688000104290
>> 003: cn=Directory Manager
>> nscpentrywsi: nsUniqueId: 48809b40-2bb911e5-96a0b8b8-1fdd8074
>> nscpentrywsi: cn: pkidbuser
>> nscpentrywsi: createTimestamp: 20150716125146Z
>> nscpentrywsi: creatorsName: cn=directory manager
>> nscpentrywsi: description;vucsn-5938e688000104290001:
>> 2;1341718541;CN=Certific
>> ate Authority,O=BPT.ROCKS;CN=CA Subsystem,O=BPT.ROCKS
>> nscpentrywsi: description;vdcsn-5938e688000104290002;deleted:
>> 2;4;CN=Certifica
>> te Authority,O=BPT.ROCKS;CN=CA Subsystem,O=BPT.ROCKS
>> nscpentrywsi: mail:
>> nscpentrywsi: objectClass: top
>> nscpentrywsi: objectClass: person
>> nscpentrywsi: objectClass: organizationalPerson
>> nscpentrywsi: objectClass: inetOrgPerson
>> nscpentrywsi: objectClass: cmsuser
>> nscpentrywsi: seeAlso: CN=CA Subsystem,O=BPT.ROCKS
>> nscpentrywsi: sn: pkidbuser
>> nscpentrywsi: uid: pkidbuser
>> nscpentrywsi: userCertificate::
>> MIIDczCCAlugAwIBAgIBBDANBgkqhkiG9w0BAQsFADA0MR
>> <snip>
>> cBd4nOV4m9A+qRZNo4eX7jNT8Aa23gDl6xuLVa1SxlyE8XEo8dfAPXMJoFc=
>> nscpentrywsi: userCertificate;vucsn-5938e688000104290000::
>> MIIDbjCCAlagAwIBAgI
>> <snip>
>> AR
>> nscpentrywsi: userPassword:: <snip>
>> nscpentrywsi: userstate: 1
>> nscpentrywsi: usertype: agentType
>> nscpentrywsi: parentid: 2
>> nscpentrywsi: entryid: 51
>>> and
>>> [root@seattlenfs ianh]# ldapsearch -LLL -D 'cn=directory manager'
>>> -W -b uid=pkidbuser,ou=people,o=ipaca nscpentrywsi
>>
>> [root@seattlenfs ianh]# ldapsearch -LLL -D 'cn=directory manager' -W
>> -b uid=pkidbuser,ou=people,o=ipaca nscpentrywsi
>> Enter LDAP Password:
>> dn: uid=pkidbuser,ou=people,o=ipaca
>> nscpentrywsi: dn: uid=pkidbuser,ou=people,o=ipaca
>> nscpentrywsi: seeAlso: CN=CA Subsystem,O=BPT.ROCKS
>> nscpentrywsi: userCertificate::
>> MIIDczCCAlugAwIBAgIBBDANBgkqhkiG9w0BAQsFADA0MR
>> <snip>
>> cBd4nOV4m9A+qRZNo4eX7jNT8Aa23gDl6xuLVa1SxlyE8XEo8dfAPXMJoFc=
>> nscpentrywsi: description: 2;4;CN=Certificate
>> Authority,O=BPT.ROCKS;CN=CA Subs
>> ystem,O=BPT.ROCKS
>> nscpentrywsi: entryid: 51
>> nscpentrywsi: parentid: 2
>> nscpentrywsi: modifyTimestamp: 20150716125146Z
>> nscpentrywsi: createTimestamp: 20150716125146Z
>> nscpentrywsi: modifiersName: cn=directory manager
>> nscpentrywsi: creatorsName: cn=directory manager
>> nscpentrywsi: userPassword:: <snip>
>> nscpentrywsi: userstate: 1
>> nscpentrywsi: usertype: agentType
>> nscpentrywsi: mail:
>> nscpentrywsi: cn: pkidbuser
>> nscpentrywsi: sn: pkidbuser
>> nscpentrywsi: uid: pkidbuser
>> nscpentrywsi: objectClass: top
>> nscpentrywsi: objectClass: person
>> nscpentrywsi: objectClass: organizationalPerson
>> nscpentrywsi: objectClass: inetOrgPerson
>> nscpentrywsi: objectClass: cmsuser
>> nscpentrywsi: nsUniqueId: 48809b40-2bb911e5-96a0b8b8-1fdd8074
>> nscpentrywsi: entryusn: 86
>>>
>>> From the output you may only copy/past the ones related to
>>> userCertificate
>>>
>>>
>>> Also to dump the current replication status, send the result of
>>> [root@freeipa-sea ianh]# ldapsearch -LLL -D 'cn=directory manager'
>>> -W -b
>>>
"o=ipaca"//"(&(objectclass=nstombstone)(nsUniqueId=ffffffff-ffffffff-ffffffff-ffffffff))"
>> [root@freeipa-sea ianh]# ldapsearch -LLL -D 'cn=directory manager'
>> -W -b "o=ipaca"
>>
"(&(objectclass=nstombstone)(nsUniqueId=ffffffff-ffffffff-ffffffff-ffffffff))"
>> Enter LDAP Password:
>> dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
>> cn: replica
>> nsDS5Flags: 1
>> nsDS5ReplicaBindDN: cn=Replication Manager
>> cloneAgreement1-freeipa-sea.bpt.roc
>> ks-pki-tomcat,ou=csusers,cn=config
>> nsDS5ReplicaBindDN: cn=Replication Manager
>> masterAgreement1-seattlenfs.bpt.roc
>> ks-pki-tomcat,ou=csusers,cn=config
>> nsDS5ReplicaId: 1065
>> nsDS5ReplicaName: b21a1f1e-627911e6-93e6ef4b-69dcc2d1
>> nsDS5ReplicaRoot: o=ipaca
>> nsDS5ReplicaType: 3
>> nsState:: KQQAAAAAAAAhHIpZAAAAAAEAAAAAAAAAKgAAAAAAAAABAAAAAAAAAA==
>> nsds5replicabinddngroup: cn=replication
>> managers,cn=sysaccounts,cn=etc,dc=bpt,
>> dc=rocks
>> nsds5replicabinddngroupcheckinterval: 60
>> objectClass: top
>> objectClass: nsDS5Replica
>> objectClass: extensibleobject
>> nsds50ruv: {replicageneration} 57c291d9000004290000
>> nsds50ruv: {replica 1065 ldap://freeipa-sea.bpt.rocks:389}
>> 57f840bf00000429000
>> 0 598a1c40000104290000
>> nsds50ruv: {replica 1290 ldap://seattlenfs.bpt.rocks:389}
>> nsds5agmtmaxcsn:
>> o=ipaca;cloneAgreement1-freeipa-sea.bpt.rocks-pki-tomcat;seat
>> tlenfs.bpt.rocks;389;unavailable
>> nsds5agmtmaxcsn:
>> o=ipaca;masterAgreement1-seattlenfs.bpt.rocks-pki-tomcat;seat
>> tlenfs.bpt.rocks;389;unavailable
>> nsruvReplicaLastModified: {replica 1065
>> ldap://freeipa-sea.bpt.rocks:389} 598a
>> 1c16
>> nsruvReplicaLastModified: {replica 1290
>> ldap://seattlenfs.bpt.rocks:389} 00000
>> 000
>> nsds5ReplicaChangeCount: 885
>> nsds5replicareapactive: 0
>>> and
>>> [root@seattlenfs ianh]# ldapsearch -LLL -D 'cn=directory manager'
>>> -W -b
>>>
"o=ipaca"//"(&(objectclass=nstombstone)(nsUniqueId=ffffffff-ffffffff-ffffffff-ffffffff))"
>>
>>
>> [root@seattlenfs ianh]# ldapsearch -LLL -D 'cn=directory manager' -W
>> -b "o=ipaca"
>>
"(&(objectclass=nstombstone)(nsUniqueId=ffffffff-ffffffff-ffffffff-ffffffff))"
>> Enter LDAP Password:
>> dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
>> cn: replica
>> nsDS5Flags: 1
>> nsDS5ReplicaBindDN: cn=Replication Manager
>> cloneAgreement1-seattlenfs.bpt.rock
>> s-pki-tomcat,ou=csusers,cn=config
>> nsDS5ReplicaId: 1290
>> nsDS5ReplicaName: 8706ab1e-6a8311e6-a9bfdbbf-74dc7323
>> nsDS5ReplicaRoot: o=ipaca
>> nsDS5ReplicaType: 3
>> nsState:: CgUAAAAAAAAKFYpZAAAAAAAAAAAAAAAAKgAAAAAAAAABAAAAAAAAAA==
>> nsds5replicabinddngroup: cn=replication
>> managers,cn=sysaccounts,cn=etc,dc=bpt,
>> dc=rocks
>> nsds5replicabinddngroupcheckinterval: 60
>> objectClass: top
>> objectClass: nsDS5Replica
>> objectClass: extensibleobject
>> nsds50ruv: {replicageneration} 55c8f3ae000000600000
>> nsds50ruv: {replica 1290 ldap://seattlenfs.bpt.rocks:389}
>> 57be804c0000050a0000
>> 587236150000050a0000
>> nsds50ruv: {replica 1065 ldap://freeipa-sea.bpt.rocks:389}
>> 57b103d400000429000
>> 0 57be8043000704290000
>> nsds5agmtmaxcsn:
>> o=ipaca;cloneAgreement1-seattlenfs.bpt.rocks-pki-tomcat;freei
>> pa-sea.bpt.rocks;389;1065;587236150000050a0000
>> nsruvReplicaLastModified: {replica 1290
>> ldap://seattlenfs.bpt.rocks:389} 00000
>> 000
>> nsruvReplicaLastModified: {replica 1065
>> ldap://freeipa-sea.bpt.rocks:389} 0000
>> 0000
>> nsds5ReplicaChangeCount: 3
>> nsds5replicareapactive: 0
>>>
>>> Regards
>>> thierry
>>>>> [root@freeipa-sea ianh]# ldapsearch -LLL -D 'cn=directory
>>>>> manager' -W -b uid=pkidbuser,ou=people,o=ipaca userCertificate
>>>>> description
>>>>> Enter LDAP Password:
>>>>> dn: uid=pkidbuser,ou=people,o=ipaca
>>>>> userCertificate::
>>>>> MIIDczCCAlugAwIBAgIBBDANBgkqhkiG9w0BAQsFADA0MRIwEAYDVQQKDAlC
>>>>> <truncated>
>>>>> ZNo4eX7jNT8Aa23gDl6xuLVa1SxlyE8XEo8dfAPXMJoFc=
>>>>> userCertificate::
>>>>> MIIDbjCCAlagAwIBAgIET/kADTANBgkqhkiG9w0BAQsFADA0MRIwEAYDVQQK
>>>>> <truncated> fuQQZmP1XrKkrfsjeiFbUFtoMnfmHLIYpe5QAR
>>>>> description: 2;1341718541;CN=Certificate
>>>>> Authority,O=BPT.ROCKS;CN=CA Subsystem
>>>>> ,O=BPT.ROCKS
>>>>>
>>>>>
>>>>> [root@seattlenfs ianh]# ldapsearch -LLL -D 'cn=directory
manager'
>>>>> -W -b uid=pkidbuser,ou=people,o=ipaca userCertificate description
>>>>> Enter LDAP Password:
>>>>> dn: uid=pkidbuser,ou=people,o=ipaca
>>>>> userCertificate::
>>>>> MIIDczCCAlugAwIBAgIBBDANBgkqhkiG9w0BAQsFADA0MRIwEAYDVQQKDAlC
>>>>> <truncated>
>>>>> ZNo4eX7jNT8Aa23gDl6xuLVa1SxlyE8XEo8dfAPXMJoFc=
>>>>> description: 2;4;CN=Certificate Authority,O=BPT.ROCKS;CN=CA
>>>>> Subsystem,O=BPT.RO
>>>>> CKS
>>>>>
>>>>>> If you check the certificates dates, I guess that the most
>>>>>> recent one is on server seattlenfs (because of serial numbers).
>>>>>> Can you confirm this (because our goal is to have the most
>>>>>> recent cert stored in all the NSSDBs and in LDAP)?
>>>>>>
>>>>>> Thanks,
>>>>>> Flo
>>>>>>
>>>>>>
>>>>>>>> If your renewal master is the machine on which the NSS DB
>>>>>>>> /etc/pki/pki-tomcat/alias contains the newer cert, then
you
>>>>>>>> can manually run the scripts:
>>>>>>>
>>>>>>> If I'm interpreting this correctly, then it is... so I
ran this
>>>>>>> on the broken machine (seattlenfs)
>>>>>>>
>>>>>>>> $ sudo /usr/libexec/ipa/certmonger/stop_pkicad
>>>>>>>> $ sudo /usr/libexec/ipa/certmonger/renew_ca_cert
>>>>>>>> "subsystemCert cert-pki-ca"
>>>>>>>>
>>>>>>>
>>>>>>> I ran this, it took a while, then I re-checked and the serial
>>>>>>> still seems to be 4
>>>>>>>
>>>>>>> $ ldapsearch -LLL -D 'cn=directory manager' -W -b
>>>>>>> uid=pkidbuser,ou=people,o=ipaca userCertificate description
>>>>>>> Enter LDAP Password:
>>>>>>> dn: uid=pkidbuser,ou=people,o=ipaca
>>>>>>> userCertificate::
>>>>>>> MIIDczCCAlugAwIBAgIBBDANBgkqhkiG9w0BAQsFADA0MRIwEAYDVQQKDAlC
>>>>>>> ...
>>>>>>> ZNo4eX7jNT8Aa23gDl6xuLVa1SxlyE8XEo8dfAPXMJoFc=
>>>>>>> description: 2;4;CN=Certificate Authority,O=BPT.ROCKS;CN=CA
>>>>>>> Subsystem,O=BPT.ROCKS
>>>>>>>
>>>>>>> Here's what was in /var/log/pki/pki-tomcat/ca/debug
>>>>>>>
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>> ============================================
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: ===== DEBUG
>>>>>>> SUBSYSTEM INITIALIZED =======
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>> ============================================
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
>>>>>>> restart at autoShutdown? false
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
>>>>>>> autoShutdown crumb file path?
>>>>>>> /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
about
>>>>>>> to look for cert for auto-shutdown support:auditSigningCert
>>>>>>> cert-pki-ca
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
found
>>>>>>> cert:auditSigningCert cert-pki-ca
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
done
>>>>>>> init id=debug
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
>>>>>>> initialized debug
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
>>>>>>> initSubsystem id=log
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
ready
>>>>>>> to init id=log
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: Creating
>>>>>>>
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/signedAudit/ca_audit)
>>>>>>>
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: Creating
>>>>>>> RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/system)
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: Creating
>>>>>>> RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/transactions)
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
>>>>>>> restart at autoShutdown? false
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
>>>>>>> autoShutdown crumb file path?
>>>>>>> /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
about
>>>>>>> to look for cert for auto-shutdown support:auditSigningCert
>>>>>>> cert-pki-ca
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
found
>>>>>>> cert:auditSigningCert cert-pki-ca
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
done
>>>>>>> init id=log
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
>>>>>>> initialized log
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
>>>>>>> initSubsystem id=jss
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
ready
>>>>>>> to init id=jss
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
>>>>>>> restart at autoShutdown? false
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
>>>>>>> autoShutdown crumb file path?
>>>>>>> /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
about
>>>>>>> to look for cert for auto-shutdown support:auditSigningCert
>>>>>>> cert-pki-ca
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
found
>>>>>>> cert:auditSigningCert cert-pki-ca
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
done
>>>>>>> init id=jss
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
>>>>>>> initialized jss
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
>>>>>>> initSubsystem id=dbs
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: CMSEngine:
ready
>>>>>>> to init id=dbs
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: DBSubsystem:
>>>>>>> init() mEnableSerialMgmt=true
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: Creating
>>>>>>> LdapBoundConnFactor(DBSubsystem)
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>> LdapBoundConnFactory: init
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>> LdapBoundConnFactory:doCloning true
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: LdapAuthInfo:
>>>>>>> init()
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: LdapAuthInfo:
>>>>>>> init begins
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: LdapAuthInfo:
>>>>>>> init ends
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: init: before
>>>>>>> makeConnection errorIfDown is true
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
makeConnection:
>>>>>>> errorIfDown true
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: TCP
Keep-Alive:
>>>>>>> true
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>> SSLClientCertificateSelectionCB: Setting desired cert
nickname
>>>>>>> to: subsystemCert cert-pki-ca
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>> LdapJssSSLSocket: set client auth cert nickname subsystemCert
>>>>>>> cert-pki-ca
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>> SSLClientCertificatSelectionCB: Entering!
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: Candidate
cert:
>>>>>>> ocspSigningCert cert-pki-ca
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: Candidate
cert:
>>>>>>> subsystemCert cert-pki-ca
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>> SSLClientCertificateSelectionCB: desired cert found in list:
>>>>>>> subsystemCert cert-pki-ca
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>> SSLClientCertificateSelectionCB: returning: subsystemCert
>>>>>>> cert-pki-ca
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]: SSL handshake
>>>>>>> happened
>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>> CMSEngine.shutdown()
>>>>>>>
>>>>>>>
>>>>>>> I really appreciate you sticking with me through this. I owe
you.
>>>>>>>
>>>>>>> - Ian
>>>>>>>
>>>>>>>> After that, check that the LDAP entry has been updated.
If it
>>>>>>>> is the case, you should be able to restart pki.
>>>>>>>>
>>>>>>>> Flo
>>>>>>>>
>>>>>>>>> Thank you again for your time!
>>>>>>>>>
>>>>>>>>> Ian
>>>>>>>>>
>>>>>>>>>> Flo.
>>>>>>>>>>>
>>>>>>>>>>> which seems weird.
>>>>>>>>>>>
>>>>>>>>>>>> The description attribute in the
ldapsearch output is of
>>>>>>>>>>>> the form 2;#;DN;DN
>>>>>>>>>>>>
>>>>>>>>>>>> The # should match the serial number in
the first certutil
>>>>>>>>>>>> output
>>>>>>>>>>>>
>>>>>>>>>>>> The ASCII blob (minus the BEGIN/END
headers) should match
>>>>>>>>>>>> one of the
>>>>>>>>>>>> userCertificate attributes.
>>>>>>>>>>>>
>>>>>>>>>>>> rob
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>>>>> To unsubscribe send an email to
>>>>> freeipa-users-leave(a)lists.fedorahosted.org
>>>> _______________________________________________
>>>> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>>>> To unsubscribe send an email to
>>>> freeipa-users-leave(a)lists.fedorahosted.org
>>>
>>
>>
>>
>> _______________________________________________
>> FreeIPA-users mailing list --freeipa-users(a)lists.fedorahosted.org
>> To unsubscribe send an email tofreeipa-users-leave(a)lists.fedorahosted.org
>
_______________________________________________
FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org