Hi Harry,
On ma, 20 joulu 2021, Harry G. Coin wrote:
On 12/16/21 11:10 PM, Alexander Bokovoy wrote:
>On to, 16 joulu 2021, Harry G. Coin via FreeIPA-users wrote:
>>Alexander and others who care about dnssec:
>>
>>Given the ongoing problems with opendnssec/libp11 and the many
>>freeipa routines and resources dedicated to working around it, has
>>bind9's native dnssec implementation improved to the point we can
>>greatly reduce the freeipa package count by just using bind's
>>dnssec provisions? What's the roadmap on that?
>
>We did a research some time ago and a result was that native DNSSEC
>implementation in BIND required some tooling that would have been an
>equivalent of what is being done with SoftHSM/OpenDNSSEC:
>
>The BIND / DNSSec guide is rather naive when it comes to storing
>private keys:
>https://bind9.readthedocs.io/en/latest/dnssec-guide.html#key-storage
>
>Quoting the guide: "Ideally, private keys should be stored offline, in
>secure devices such as a smart card. Operationally, however, this
>creates certain challenges, since the private key is needed to create
>RRSIG resource records, and it is a hassle to bring the private key out
>of storage every time the zone file changes or signatures expire. A
>common approach to strike the balance between security and practicality
>is to have two sets of keys (...) For example, a KSK private key stored
>on a USB flash drive that is kept in a fireproof safe, only brought
>online once a year to sign a new pair of ZSKs, combined with a ZSK
>private key stored on the network file system and available for routine
>use, may be a good balance between operational flexibility and
>security."
>
>Effectively, this is a repeatance of what OpenDNSSEC provides if you'd
>look into making HSM-backed keys.
>
>>My dnssec database was corrupted by a recent freeipa upgrade that
>>reverted to upstream code I patched locally some time ago. The
>>upstream doesn't really comprehend/value/test multi-concurrent
>>thread machine speed key/context/pin updates as deployed by
>>opendnssec / softhsm2. Even after I fixed the database by rolling
>>keys and identifying corrupt files, etc, etc , the 'new and
>>improved' upstream code corrupted it again within minutes. So
>>I've disabled dnssec entirely as money to fund those minutes has
>>run out.
>
>Probably a good way going forward would be to have an isolated example
>with OpenDNSSEC/SoftHSMv2 alone that shows this set of problems and work
>with that to identify the issue. Right now, it seems, there is no a
>single place with end to end description of the problem.
>
Thinking about it for a few days, given ICANN's call for full dnssec
deployment, and the fact that dns-over-tls only provides improved
results between end-user and server (nothing regarding confidence
about server data integrity): I think it's fair to ask the freeipa
community whether now is the time to re-assess the reasons why, after
so many years, freeipa's caution 'dnssec support is experimental'
remains (
https://www.freeipa.org/page/Howto/DNSSEC).
Alexander correctly suggests that creating an isolated example that
highlights an issue be done. Notice that becomes much easier to ask
for than produce when the problem is a race condition among threads
that manifests itself in a library that's, what, 4 or 5 packages deep
from the launching daemons, each package maintained by different
groups with different dev priorities abilities regarding
multithreading. Remember 'softhsm' is an add-on to a 'smart card'
technology most use not at machine speeds but as humans insert and
remove physical security devices. And significantly -- almost always
just one at a time per host. Multi-processing, multi-threading and
re-entrancy (such as during bulk reconsideration of multiple zsk's,
ksks and sqlite db contents while other threads are assessing
master-slave sync issues) are not top of softhsm/libp11 dev minds and
quite understandably. Just a quick look at libp11's eng_back.c will
show even a casual reader externally accessible routines could find
essential variables in an inconsistent state from a multi-threading
perspective ( e.g. ctx->pin ctx->pin_length).
Moreover, since freeipa last reassessed the matter in the big picture,
bind's dnssec shortcomings have been improved. From a maintainability
perspective, there's obviously something to be said for the same
results being delivered by fewer dependencies and fewer points of
failure in sub packages. After all, in the present case, to cite just
one example, we have freeipa's dnssec routines (call it 'top level')
running code quite literally to 'fix' cert/key file permissions so
that softhsm, opendnssec, libp11, openssl-pkcs11, and, bind, can
interoperate with dyndb-ldap and, last, freeipa. While that's
obviously necessary to deliver function, I don't think anybody would
call it 'their first design choice'.
I respectfully ask freeipa folks to re-assess whether bind's
implementation has improved to the point the, ah, 'unfortunate'
documentation Alexander points at might be less important than the
tenuous reliability we experience now owing to reliance on so many
levels of differently maintained packages.
Whether this particular issues persuades the freeipa community or does
not: At least to me, most significantly, it has been many years that
freeipa has written 'dnssec is experimental'. After so long, it's
fair to ask what is lacking that dnssec can't be considered
'production ready' and take steps to cause that to occur.
"FreeIPA folks" are community members. Some of them work for Red Hat,
some aren't. If you want to focus on a certain area, you are free to do
so. Even if Red Hat right now is not considering DNSSEC support as
priority (hence keeping it in experimental state) and not adding more
resources to solve that problem, community members can step up and
contribute.
Traditionally there were not so many contributions in these core areas
from outside, for various reasons. Our community has many more
administrators than developers. However, our current BIND support is
kept in a good shape also thanks to community contributors (Stas did a
great part of that, thank you! A lot of Ubuntu users contributed with
bug reports too). I certainly hope we can get DNSSEC support to a better
shape but that needs more involvement from everyone who can help, not
just core team.
Myself, I am deep in making FreeIPA to work with upcoming MIT Kerberos
1.20 which changes its KDB interface in a serious way to step up few
very important security properties of Kerberos use in real life
deployments -- this is a result of several years of improving
interoperability with Active Directory but not only that. Latest
Microsoft's security release for Windows Server which fixed at least
four CVEs reported by Samba Team members made a number of changes that
we hope will allow open source implementations to get much better
semantics around authentication and authorization. And this needs our
work 'now' (my team was busy with it for last half a year, at least). I
am simply not able to look into DNSSEC in a serious way before this and
other prioritised tasks are done.
So if you feel you can contribute to DNSSEC support improvements, please
do so. We typically work by creating a design document by iterating
through it in reviews, identifying the affected components and then step
by step implementing the changes. This is a lengthy process but it
doesn't really matter who'd initiate and contribute to it.
Next few weeks are special for Red Hatters, though. Traditionally, Red
Hat encourages employees to spend time with their families around
Christmas and new year celebrations across the world. Most of Red Hat
contributors to FreeIPA and related open source projects are either
already on their vacation or will be going for vacation soon. I myself
will start that later this week and be back on the second week of
January. I might spent some of my family time looking at Fedora and
FreeIPA but this year was so pressing that a proper isolation of an
online presence might be a good way to recharge.
--
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland