9:37 a.m.
>> >> I still think allowing entryuuid to diverge from
nsuniqueid is the right call. It opens more scenarioes up to us for migration.
> >
> > Just fyi, when replicating from 389DS to OpenLDAP, we directly map nsUniqueId to
entryUuid. They're
> > both UUIDs and both serve the same purpose on their respective servers, so why
is there
> > any reason to allow proliferation of other IDs?
Ahh I believe that you may be referring to the sundsee syncrepl consumer mode from your
2.5 or git master perhaps? Sadly, this is not the target of my work - if it was, life
would be much easier, and this would not be a problem. In this case, we have a customer
who wants to create read-only openldap consumers which are able to get data from 389-ds,
but they are on version 2.4.41 of openldap which does not appear to perform this mapping
from my reading of the code (but i've had some mistakes reading code lately, so
perhaps I'm wrong).
Yes, the consumer code is only in 2.5, not 2.4. But the mapping is trivial; the nsUniqueId
format
has the same number of digits as the entryUuid format and only differs by having one less
hyphen.
So it's quite simple to write an overlay or plugin to convert from one to the other.
e.g.
https://git.openldap.org/openldap/openldap/-/blob/master/servers/slapd/sy...
Their goal is to move from openldap to 389-ds due to support reasons.
So this means entryuuid which has been a stable ID for a long time in their fleet is
something we need to support - and many external applications do read entryuuid as a
primary key to identify objects in the directory, despite the fact it should be an
internal replication-only element IMO (vmware and nextcloud come to mind immediately).
nsuniqueid (annoyingly) is not the same format as entryuuid, so we can't just ask them
to change to reading nsunique (even if the bytes were the same). Every day we get to live
with the decisions of the past, and we have the joy of working through and around them.
So I think as much as your suggestion is probably correct technically, it doesn't
apply in this situation.
I am considering that the "solution" in this case though, is that when we see
an entryuuid on an import/add, we derive nsuniqueid from that, so that entryuuid /
nsunique are equivalents in the respective directories, and we can then re-synthesise the
entryuuid from nsunique on the syncrepl. We still need to store entryuuid however, because
client applications will be expecting it to remain consistent and present during an
openldap to 389 migration.
Why wouldn't you just generate entryUuid from nsUniqueId on the fly, as a virtual
attribute, when requested in
a search/compare? And yes, derive nsUniqueId from entryUuid on an Add.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
6:07 p.m.
New subject: 389-devel] Advice on a problem (syncrepl + entryuuid)
On 25 Jun 2020, at 00:37, Howard Chu <hyc(a)symas.com> wrote:
>>>>> I still think allowing entryuuid to diverge from nsuniqueid is the
right call. It opens more scenarioes up to us for migration.
>>>
>>> Just fyi, when replicating from 389DS to OpenLDAP, we directly map nsUniqueId
to entryUuid. They're
>>> both UUIDs and both serve the same purpose on their respective servers, so
why is there
>>> any reason to allow proliferation of other IDs?
>
> Ahh I believe that you may be referring to the sundsee syncrepl consumer mode from
your 2.5 or git master perhaps? Sadly, this is not the target of my work - if it was, life
would be much easier, and this would not be a problem. In this case, we have a customer
who wants to create read-only openldap consumers which are able to get data from 389-ds,
but they are on version 2.4.41 of openldap which does not appear to perform this mapping
from my reading of the code (but i've had some mistakes reading code lately, so
perhaps I'm wrong).
Yes, the consumer code is only in 2.5, not 2.4. But the mapping is trivial; the
nsUniqueId format
has the same number of digits as the entryUuid format and only differs by having one less
hyphen.
So it's quite simple to write an overlay or plugin to convert from one to the other.
e.g.
https://git.openldap.org/openldap/openldap/-/blob/master/servers/slapd/sy...
While it's probably trivial and I agree, customers are "afraid" of change in
ancient versions, which means I have to work in the constraints of "can not change
openldap" :(. If they were able to upgrade to 2.5 I think a collaboration would be
the better approach. An even better approach is "don't mix and match 389-ds and
openldap", because they really are subtly different enough, that there will be some
problem along the way we haven't predicted yet.
Apparently even changing version numbers, rather than patching continually makes people
afraid (even if the version update is literally just an upstream managed patch backport
...). I think that there has been a lot of social and cultural damage done based on past
software practices and side effects which causes this situation, but that's a
discussion/rant/soliloquy for another time ...
> Their goal is to move from openldap to 389-ds due to support reasons. So this means
entryuuid which has been a stable ID for a long time in their fleet is something we need
to support - and many external applications do read entryuuid as a primary key to identify
objects in the directory, despite the fact it should be an internal replication-only
element IMO (vmware and nextcloud come to mind immediately). nsuniqueid (annoyingly) is
not the same format as entryuuid, so we can't just ask them to change to reading
nsunique (even if the bytes were the same). Every day we get to live with the decisions of
the past, and we have the joy of working through and around them.
>
> So I think as much as your suggestion is probably correct technically, it doesn't
apply in this situation.
>
> I am considering that the "solution" in this case though, is that when we
see an entryuuid on an import/add, we derive nsuniqueid from that, so that entryuuid /
nsunique are equivalents in the respective directories, and we can then re-synthesise the
entryuuid from nsunique on the syncrepl. We still need to store entryuuid however, because
client applications will be expecting it to remain consistent and present during an
openldap to 389 migration.
Why wouldn't you just generate entryUuid from nsUniqueId on the fly, as a virtual
attribute, when requested in
a search/compare?
That's an approach I considered. Virtual attributes are a trade off (like any other
decision, and I'm sure you already understand ...) between Mem/IO, CPU, and my time.
Extracting the attribute on the write path, and storing it in the entry will reduce your
CPU load during the search path at a cost of more IO to load the entry, and memory to
store it. Using a virtual attribute would require a filter transformer (which I think we
only just gained support for ...), an attribute transform on read, and depending on *that*
cost, potentially caching - all of which is CPU on the search path. We'd also need to
intercept any entryUUID write on the write path too. Virtual will also probably take me
longer to implement and test too.
Without having implemented both and testing my "intuition" and experience with
the quirks of 389 lead me to think that not using virtual attributes is a better option
here, both in less development time, but also less complexity and less edge cases that
could lead to surprises.
And yes, derive nsUniqueId from entryUuid on an Add.
Yes, that's what I'm thinking right now. This discussion and time away from the
problem has certainly helped me to consider that it's the best way to advance and get
a consistent outcome.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
_______________________________________________
389-devel mailing list -- 389-devel(a)lists.fedoraproject.org
To unsubscribe send an email to 389-devel-leave(a)lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproje...
—
Sincerely,
William Brown
Senior Software Engineer, 389 Directory Server
SUSE Labs
1395
days inactive
1395
days old
389-devel@lists.fedoraproject.org
1 comments
2 participants
participants (2)
-
Howard Chu -
William Brown