Hi Lutz,
Good catch! Not sure where the -J came from. Sorry for the confusion.
Thanks,
Dave
David Ritenour
Senior Directory Engineer
MartinFederal Consulting, LLC
513 Madison Street SE
Huntsville, AL 35801
From: Lutz Berger lutz.berger@multigrid.de Sent: Sunday, March 20, 2022 12:15 PM To: General discussion list for the 389 Directory server project. 389-users@lists.fedoraproject.org; David Ritenour d.ritenour@martinfed.com Subject: Re: [389-users] Re: docker, 389ds/dirsrv:2.0, vendorVersion 2.1.0, ssca, certificate chain, orphan key, cert9.db, key4.db
** WARNING: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
Hi David,
forget my statement about the "-J" option, I noticed in your examples that you actually meant "-F" :-)
Best regards, Lutz Dipl.-Inform. Univ. Lutz Berger Kettwiger Straße 43 45468 Mülheim an der Ruhr Telefon: +49 208 88351714 Telefax: +49 208 88351707 Mobil: +49 173 3289783 E-Mail: lutz.berger@multigrid.demailto:lutz.berger@multigrid.de USt-IdNr.: DE280466318
[Multigrid-Logo]http://multigrid.de/ On 18.03.22 22:50, Lutz Berger wrote: Hi David,
thanks for your input. There are a couple of versions of "certutil" out there. None of mine support the "-J" option. I did even look into the source code.
The nature of my question is not about curing the symptoms by removing the orphaned private key, it's more about removing the cause, why it gets stored into the databases.
To be more specific here:
When I look into my docker container with "docker exec -i -t ur1 /bin/bash", I see:
89653e311778:/ # cd /data 89653e311778:/data # ls -l total 16 ... drwxrwxrwx. 2 root root 156 Mar 16 14:42 ssca drwxr-xr-x. 3 root root 52 Mar 16 12:27 tls 89653e311778:/data #
So, the ssca subdirectory is populated with certificate/key database related stuff, e.g. noise file, pin file, pwdfile and self-signed certificate and private key.
But: The ssca is generated after all certificates are given in the tls directory ( see time stamps of the subdirectories ).
If you look into the errors log, you'll notice that there is a mechanism "slapd_extract_cert" that scans for certificates and keys:
[16/Mar/2022:14:56:10.258591345 +0000] - INFO - slapd_extract_cert - CA CERT NAME: /data/tls/ca/XXXXXXServerCA2015.crt [16/Mar/2022:14:56:10.259368225 +0000] - INFO - slapd_extract_cert - CA CERT NAME: Self-Signed-CA
At the end of the day everything gets stored into config subdirectory..
I thought, that the "Self-Signed-CA" cert and key should not be stored into databases at all - because unnecessary when the tls directory is given, because it contains cert and private key.
So "slapd_extract_cert" should not extract cert/key from "ssca" if a valid cert/key is given in "tls".
Hopefully my explanation is clear enough.
As William pointed out: Everything works fine... it's more a side effect for the certificate connoisseur ;-)
Feedback is welcome!
Thanks and best regards, Lutz
Dipl.-Inform. Univ. Lutz Berger Kettwiger Straße 43 45468 Mülheim an der Ruhr E-Mail: lutz.berger@multigrid.demailto:lutz.berger@multigrid.de USt-IdNr.: DE280466318
[Multigrid-Logo]http://multigrid.de/ On 18.03.22 14:04, David Ritenour wrote:
Hello Lutz and William,
I have used the certutil -J option to remove orphaned private keys from the NSS key database (in this case key3.db). I always make a tarball backup of the three NSS database files (cert8.db, key3.db, secmod.db) before performing the operation just in case something goes awry.
Example:
#List private keys in NSS keystore
certutil -K -d /etc/dirsrv/slapd-mydir/alias/
certutil: Checking token "NSS FIPS 140-2 Certificate DB" in slot "NSS FIPS 140-2 User Private Key Services"
Enter Password or Pin for "NSS FIPS 140-2 Certificate DB":
< 0> rsa ea4730830deb0dd40a7b2f6d4578a28132ae0251 NSS FIPS 140-2 Certificate DB:Server-Cert
< 1> rsa 0c7130b409a03ae2752b0cba6a6dc834d7d5dfb0 (orphan)
#Remove the orphaned key using its keyid
certutil -F -k 0c7130b409a03ae2752b0cba6a6dc834d7d5dfb0 -d /etc/dirsrv/slapd-mydir/alias/
Enter Password or Pin for "NSS FIPS 140-2 Certificate DB":
##List private keys again
certutil -K -d /etc/dirsrv/slapd-mydir/alias/
certutil: Checking token "NSS FIPS 140-2 Certificate DB" in slot "NSS FIPS 140-2 User Private Key Services"
Enter Password or Pin for "NSS FIPS 140-2 Certificate DB":
< 0> rsa ea4730830deb0dd40a7b2f6d4578a28132ae0251 NSS FIPS 140-2 Certificate DB:Server-Cert
Thanks,
Dave
David Ritenour
Senior Directory Engineer
MartinFederal Consulting, LLC
513 Madison Street SE
Huntsville, AL 35801
-----Original Message-----
From: William Brown william.brown@suse.commailto:william.brown@suse.com
Sent: Thursday, March 17, 2022 7:02 PM
To: Lutz Berger lutz.berger@multigrid.demailto:lutz.berger@multigrid.de
Cc: 389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org
Subject: [389-users] Re: docker, 389ds/dirsrv:2.0, vendorVersion 2.1.0, ssca, certificate chain, orphan key, cert9.db, key4.db
** WARNING: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
On 17 Mar 2022, at 20:50, Lutz Berger lutz.berger@multigrid.demailto:lutz.berger@multigrid.de wrote:
On 16.03.22 23:39, William Brown wrote:
An orphan key doesn't look nice, but I am more worried about the
unnecessary stuff in the databases
So the orphan key is the "original" server-cert key that was orphaned since you loaded your own key. It's honestly harmless. Everything else appears to have imported correctly which is excellent!
OK, agreed. It is harmless, but also not needed. Usually, I choose to use only one private key in my key3.db or key4.db.
My assumption was, that if I provide certificates in the tls
subdirectory, the ssca directory is not even used at all, since the key and cert that are effectively used are stored in the config directory and its databases.
and the failing openssl certificate validation.
We'll need to see the output of 'openssl -_client -connect url1.XXXXXX.de:3636 -showcerts' to see what is or isn't self signed in the chain. It could just simply be that your ROOTCA/ServerCA aren't trusted by your openssl install of the host.
Due to NDA I can't provide more details. But the problem is not
related to self-signed-certs as indicated by openssl's error messages, it's really that I didn't properly specify rootCA/ServerCA.
It works now with:
cat XXXXXXROOTCA2015.crt > ./chain.crt cat XXXXXXServerCA2015.crt >>
./chain.crt openssl s_client -connect ur1.sedevsso.XXXXXX.de:3636
-CAfile <path>/ca/chain.crt :
...
...
SSL handshake has read 4776 bytes and written 427 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256 Server
public key is 2048 bit Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES128-GCM-SHA256
Session-ID: 003BB677D15FC7C0490D3E795F193AB103D1E579D2F554086B63853DA9916525
Session-ID-ctx:
Master-Key: D050F13AD8343F825E4602F57BFFDB7BFBF38438E9ED497C9626F973C7772EC0D52C92B68E4BE087AF49C1DE4C2FB06A
Key-Arg : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
Start Time: 1647338825
Timeout : 300 (sec)
Verify return code: 0 (ok)
---
You should only need ROOTCA for -CA, since the chain will be presented by 389-ds itself as you have the chain on the server. Otherwise yep, sounds like you just need to ensure clients have the ca cert setup correctly.
Happy to help!
This is to confirm, that
openssl s_client -connect ur1.XXXXXX.XXXXXX.de:3636 -CAfile /root/389ds/tls/ca/XXXXXROOTCA2015.crt
works. So, yes, only the root cert of a chain (and not the whole chain) is needed for server-cert validation done by openssl.
You are the man, William!
Thank you so much.
You are most welcome :)
Still, I suggest to remove the ssca stuff, if a customer provides his own cert chain.
Even if everything works properly, I think it's unnecessary to store the ssca cert
and key in the databases. From a troubleshooting perspective it's a bit misleading
in my opinion. Or is there a benefit of keeping it that I do not see?
It's not that there is a benefit as much as "NSS makes it a complete pain" to remove keys. That's why I never really bothered to actually do it in these cases when we are loading in the data from PEM's :(
--
Sincerely,
William Brown
Senior Software Engineer,
Identity and Access Management
SUSE Labs, Australia
_______________________________________________
389-users mailing list -- 389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-leave@lists.fedoraproject.orgmailto:389-users-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject....
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
This email and any files transmitted with it are confidential and are intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient or the person responsible for delivering the email to the intended recipient, be advised that you have received this email and any such files in error and that any use, dissemination, forwarding, printing or copying of this email and/or any such files is strictly prohibited. If you have received this email in error please immediately notify hr@martinfed.commailto:hr@martinfed.com - (855) 212-1810 , and destroy the original message and any such files.
_______________________________________________
389-users mailing list -- 389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-leave@lists.fedoraproject.orgmailto:389-users-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject....
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
_______________________________________________
389-users mailing list -- 389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-leave@lists.fedoraproject.orgmailto:389-users-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject....
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
This email and any files transmitted with it are confidential and are intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient or the person responsible for delivering the email to the intended recipient, be advised that you have received this email and any such files in error and that any use, dissemination, forwarding, printing or copying of this email and/or any such files is strictly prohibited. If you have received this email in error please immediately notify hr@martinfed.com - (855) 212-1810 , and destroy the original message and any such files.
So "slapd_extract_cert" should not extract cert/key from "ssca" if a valid cert/key is given in "tls".
It's a bit more difficult than that - I'm the author of a big chunk of the lib389 python, dscreate/instance setup, ssca, and the cli tools in question.
The main reason for this is how the bootstrapping process works - when we setup an instance, in order to configure it with TLS and other security properties we need a valid NSS database present. NSS is how we access all our internal key materials, and NSS can't use PEM files directly.
So during the setup, we have to create the "self signed" keys, because without them existing, we can NOT install the other elements of the encryption and TLS setup.
The TLS extract and setup phase is *seperate* to this setup, and has to be completed after, because the NSS DB path is the same directory as the dse.ldif config. As well, we need all those paths setup with the correct permissions defined by the uid/gid given to the dscreate calls (setup instance under the hood). Because of this, the tls extract runs *after*, so that we aren't re-inventing all those moving parts.
The TLS extract automation is also specific to the container, it's not done as a normal part of dscreate. We could move it in there if people wanted to supply their own certs on dscreate, and that would let us move some of these steps around in the container too.
But there is a risk - you could also imagine if the person passed us *invalid* keys. We'd be part way into the setup process, and we'd have to terminate because of some property of the certs being invalid. It's far cleaner to limit the surface area here to allow setup to proceed and succeed, and then have the TLS import as a seperate step subsequent that can fail with it's own error (but does NOT prevent the instance from being correctly installed). This ends up saving a lot of time and hassle.
So right now, the reason for this is pretty much about ensuring a consistent and deterministic setup state. A lot of thought and care went into the instance creation code to make sure it was extremely robust.
So while it may seem "messy" that there is a left-over and orphaned private key in the NSS db, it's really a minor problem in the scheme of things compared to creating a really robust installer and setup system.
Hopefully my explanation is clear enough.
As William pointed out: Everything works fine... it's more a side effect for the certificate connoisseur ;-)
Feedback is welcome!
Thanks and best regards, Lutz
-- Sincerely,
William Brown
Senior Software Engineer, Identity and Access Management SUSE Labs, Australia
On 22 Mar 2022, at 08:54, Lutz Berger lutz.berger@multigrid.de wrote:
Hi William,
thanks for the brilliant explanation!
I think the correct English phrase is: The devil is in the details.
Absolutely :)
It makes a lot of sense to me to focus on the robustness of the installation, rather than optimizing minor things.
For certain. I wanted the users of 389 to have a good experience and reliability is a huge part of that. So I'm glad that you appreciate it :)
Anyway, it was a good exercise for me to look into "certutil" again. I definitely like "certutil ... -d sql:." ;-)
I thought certutil defaulted to sql mode now but maybe I'm wrong ... I have a guide on some of the commands too:
https://fy.blackhats.net.au/blog/html/pages/nss_and_openssl_command_referenc...
It's a bit old and needs a rework, but it has a lot of helpful tips.
Anyway, happy to have helped out, and feel free to mail the list if you have other questions :)
Take care, Lutz
Dipl.-Inform. Univ. Lutz Berger Kettwiger Straße 43 45468 Mülheim an der Ruhr E-Mail: lutz.berger@multigrid.de USt-IdNr.: DE280466318
On 21.03.22 01:02, William Brown wrote:
So "slapd_extract_cert" should not extract cert/key from "ssca" if a valid cert/key is given in "tls".
It's a bit more difficult than that - I'm the author of a big chunk of the lib389 python, dscreate/instance setup, ssca, and the cli tools in question.
The main reason for this is how the bootstrapping process works - when we setup an instance, in order to configure it with TLS and other security properties we need a valid NSS database present. NSS is how we access all our internal key materials, and NSS can't use PEM files directly.
So during the setup, we have to create the "self signed" keys, because without them existing, we can NOT install the other elements of the encryption and TLS setup.
The TLS extract and setup phase is *seperate* to this setup, and has to be completed after, because the NSS DB path is the same directory as the dse.ldif config. As well, we need all those paths setup with the correct permissions defined by the uid/gid given to the dscreate calls (setup instance under the hood). Because of this, the tls extract runs *after*, so that we aren't re-inventing all those moving parts.
The TLS extract automation is also specific to the container, it's not done as a normal part of dscreate. We could move it in there if people wanted to supply their own certs on dscreate, and that would let us move some of these steps around in the container too.
But there is a risk - you could also imagine if the person passed us *invalid* keys. We'd be part way into the setup process, and we'd have to terminate because of some property of the certs being invalid. It's far cleaner to limit the surface area here to allow setup to proceed and succeed, and then have the TLS import as a seperate step subsequent that can fail with it's own error (but does NOT prevent the instance from being correctly installed). This ends up saving a lot of time and hassle.
So right now, the reason for this is pretty much about ensuring a consistent and deterministic setup state. A lot of thought and care went into the instance creation code to make sure it was extremely robust.
So while it may seem "messy" that there is a left-over and orphaned private key in the NSS db, it's really a minor problem in the scheme of things compared to creating a really robust installer and setup system.
Hopefully my explanation is clear enough.
As William pointed out: Everything works fine... it's more a side effect for the certificate connoisseur ;-)
Feedback is welcome!
Thanks and best regards, Lutz
-- Sincerely,
William Brown
Senior Software Engineer, Identity and Access Management SUSE Labs, Australia _______________________________________________ 389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject....
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
-- Sincerely,
William Brown
Senior Software Engineer, Identity and Access Management SUSE Labs, Australia
389-users@lists.fedoraproject.org