Hi, We have been experiencing an intermittent problem with our AD sync, where updates to a group in 389 have resulted in the group being emptied of users. This has been occurring at various times but not consistently, so was very difficult to track. Previously, the group would be emptied in the AD, which would then replicate back to 389, resulting in an empty group in both Directories. Since installing a fresh CentOS 6.3 server and the latest stable 389 (at the time, 1.2.10.12-1) the behaviour has only changed slightly, in that now the 389 group gets emptied and the AD group remains intact. When this happens, initiating a full re-sync will not fix the issue. We have since discovered that this behaviour is, in fact, consistent and repeatable if the group contains more than 1500 members. Below that threshold, adding or subtracting users from the 389 group replicates perfectly. As soon as you exceed that limit, the group gets emptied. Turning on replication logging revealed the following..... (domain and server names have been made anonymous) [27/Feb/2013:12:20:33 +1300] - Calling dirsync search request plugin [27/Feb/2013:12:20:33 +1300] - Sending dirsync search request [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - received entry from dirsync: CN=students,OU=Groups,OU=Active,OU=People,DC=domain,DC=com [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - agmt="cn=DC01" (DC01:636): map_entry_dn_inbound: looking for local entry matching AD entry [CN=students,OU=Groups,OU=Active,OU=People,DC=domain,DC=com] [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - agmt="cn=DC01" (DC01:636): map_entry_dn_inbound: looking for local entry by guid [919561f60fe49f409afcdf80a63eb089] [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - agmt="cn=DC01" (DC01:636): map_entry_dn_inbound: found local entry [CN=students,ou=Groups,ou=Active,ou=People,dc=domain,dc=com] [27/Feb/2013:12:20:33 +1300] - Calling windows entry search request plugin [27/Feb/2013:12:20:33 +1300] - windows_search_entry: received 2 messages, 1 entries, 0 references [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - agmt="cn=DC01" (DC01:636): map_entry_dn_inbound: looking for local entry matching AD entry [CN=students,OU=Groups,OU=Active,OU=People,DC=domain,DC=com] [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - windows_generate_update_mods: CN=students,ou=Groups,ou=Active,ou=People,dc=domain,dc=com, description : values are equal [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - windows_generate_update_mods: CN=students,ou=Groups,ou=Active,ou=People,dc=domain,dc=com, ntUserDomainId : values are equal [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - windows_generate_update_mods: deleting uniquemember attribute from local entry [27/Feb/2013:12:20:33 +1300] - smod - windows sync [27/Feb/2013:12:20:33 +1300] - smod 0 - delete: uniquemember The particularly interesting line is this [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - windows_generate_update_mods: deleting uniquemember attribute from local entry This only appears to happen when the group contains more than 1500 entries. Surely there must be someone else out there syncing groups with more than 1500 members between 389 and AD? It wasn't until I used Apache Directory Studio to compare entries between the 389 server and the AD that I noticed the attribute name was represented differently when the group contained over 1500 entries. This is a result of the range retrieval limit in AD. When you hit the range limit, the attribute name changes from "member" to "member;range=0-1499", which causes the mismatch that leads to the uniquemember attribute being deleted. In order to prevent this from happening, we have had to increase the MaxValRange setting in our Active Directory as per http://support.microsoft.com/kb/2009267 and http://support.microsoft.com/kb/315071 The value defaults to 1500 for Windows Server 2003 or 5000 for Windows Server 2008. Personally I consider this a bug in the AD sync plugin, as it fails to correctly handle range retrieval. At the very least, the documentation for Windows Sync should contain information about this limit.

David. -- 389 users mailing list 389-users(a)lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users

On 02/26/2013 10:17 PM, Nathan Kinder wrote: > On 02/26/2013 08:42 PM, David Baird wrote: >> Hi, >> >> We have been experiencing an intermittent problem with our AD sync, >> where updates to a group in 389 have resulted in the group being >> emptied of users. >> >> This has been occurring at various times but not consistently, so >> was very difficult to track. Previously, the group would be emptied >> in the AD, which would then replicate back to 389, resulting in an >> empty group in both Directories. >> >> Since installing a fresh CentOS 6.3 server and the latest stable 389 >> (at the time, 1.2.10.12-1) the behaviour has only changed slightly, >> in that now the 389 group gets emptied and the AD group remains >> intact. When this happens, initiating a full re-sync will not fix >> the issue. >> >> We have since discovered that this behaviour is, in fact, consistent >> and repeatable if the group contains more than 1500 members. Below >> that threshold, adding or subtracting users from the 389 group >> replicates perfectly. As soon as you exceed that limit, the group >> gets emptied. >> >> Turning on replication logging revealed the following..... >> (domain and server names have been made anonymous) >> >> [27/Feb/2013:12:20:33 +1300] - Calling dirsync search request plugin >> [27/Feb/2013:12:20:33 +1300] - Sending dirsync search request >> [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - received entry >> from dirsync: >> CN=students,OU=Groups,OU=Active,OU=People,DC=domain,DC=com >> [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - agmt="cn=DC01" >> (DC01:636): map_entry_dn_inbound: looking for local entry matching >> AD entry [CN=students,OU=Groups,OU=Active,OU=People,DC=domain,DC=com] >> [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - agmt="cn=DC01" >> (DC01:636): map_entry_dn_inbound: looking for local entry by guid >> [919561f60fe49f409afcdf80a63eb089] >> [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - agmt="cn=DC01" >> (DC01:636): map_entry_dn_inbound: found local entry >> [CN=students,ou=Groups,ou=Active,ou=People,dc=domain,dc=com] >> [27/Feb/2013:12:20:33 +1300] - Calling windows entry search request >> plugin >> [27/Feb/2013:12:20:33 +1300] - windows_search_entry: received 2 >> messages, 1 entries, 0 references >> [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - agmt="cn=DC01" >> (DC01:636): map_entry_dn_inbound: looking for local entry matching >> AD entry [CN=students,OU=Groups,OU=Active,OU=People,DC=domain,DC=com] >> [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - >> windows_generate_update_mods: >> CN=students,ou=Groups,ou=Active,ou=People,dc=domain,dc=com, >> description : values are equal >> [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - >> windows_generate_update_mods: >> CN=students,ou=Groups,ou=Active,ou=People,dc=domain,dc=com, >> ntUserDomainId : values are equal >> [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - >> windows_generate_update_mods: deleting uniquemember attribute from >> local entry >> [27/Feb/2013:12:20:33 +1300] - smod - windows sync >> [27/Feb/2013:12:20:33 +1300] - smod 0 - delete: uniquemember >> >> >> The particularly interesting line is this >> [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - >> windows_generate_update_mods: deleting uniquemember attribute from >> local entry >> >> This only appears to happen when the group contains more than 1500 >> entries. >> >> Surely there must be someone else out there syncing groups with more >> than 1500 members between 389 and AD? >> >> It wasn't until I used Apache Directory Studio to compare entries >> between the 389 server and the AD that I noticed the attribute name >> was represented differently when the group contained over 1500 entries. >> >> This is a result of the range retrieval limit in AD. When you hit >> the range limit, the attribute name changes from "member" to >> "member;range=0-1499", which causes the mismatch that leads to the >> uniquemember attribute being deleted. >> >> In order to prevent this from happening, we have had to increase the >> MaxValRange setting in our Active Directory as per >> http://support.microsoft.com/kb/2009267 and >> http://support.microsoft.com/kb/315071 >> >> The value defaults to 1500 for Windows Server 2003 or 5000 for >> Windows Server 2008. >> >> Personally I consider this a bug in the AD sync plugin, as it fails >> to correctly handle range retrieval. At the very least, the >> documentation for Windows Sync should contain information about this >> limit. > This does sound like a bug. Please open a ticket in our Trac > instance here: > > https://fedorahosted.org/389/ > > The Windows Sync plug-in needs to be modified to understand how to > use ranged searches. https://fedorahosted.org/389/ticket/472

> > > Thanks, > -NGK >> >> David. >> -- >> 389 users mailing list >> 389-users(a)lists.fedoraproject.org >> https://admin.fedoraproject.org/mailman/listinfo/389-users > > -- > 389 users mailing list > 389-users(a)lists.fedoraproject.org > https://admin.fedoraproject.org/mailman/listinfo/389-users

On 02/27/2013 08:48 AM, Nathan Kinder wrote: > On 02/27/2013 06:57 AM, Rich Megginson wrote: >> On 02/26/2013 10:17 PM, Nathan Kinder wrote: >>> On 02/26/2013 08:42 PM, David Baird wrote: >>>> Hi, >>>> >>>> We have been experiencing an intermittent problem with our AD >>>> sync, where updates to a group in 389 have resulted in the group >>>> being emptied of users. >>>> >>>> This has been occurring at various times but not consistently, so >>>> was very difficult to track. Previously, the group would be >>>> emptied in the AD, which would then replicate back to 389, >>>> resulting in an empty group in both Directories. >>>> >>>> Since installing a fresh CentOS 6.3 server and the latest stable >>>> 389 (at the time, 1.2.10.12-1) the behaviour has only changed >>>> slightly, in that now the 389 group gets emptied and the AD group >>>> remains intact. When this happens, initiating a full re-sync will >>>> not fix the issue. >>>> >>>> We have since discovered that this behaviour is, in fact, >>>> consistent and repeatable if the group contains more than 1500 >>>> members. Below that threshold, adding or subtracting users from >>>> the 389 group replicates perfectly. As soon as you exceed that >>>> limit, the group gets emptied. >>>> >>>> Turning on replication logging revealed the following..... >>>> (domain and server names have been made anonymous) >>>> >>>> [27/Feb/2013:12:20:33 +1300] - Calling dirsync search request plugin >>>> [27/Feb/2013:12:20:33 +1300] - Sending dirsync search request >>>> [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - received >>>> entry from dirsync: >>>> CN=students,OU=Groups,OU=Active,OU=People,DC=domain,DC=com >>>> [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - >>>> agmt="cn=DC01" (DC01:636): map_entry_dn_inbound: looking for local >>>> entry matching AD entry >>>> [CN=students,OU=Groups,OU=Active,OU=People,DC=domain,DC=com] >>>> [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - >>>> agmt="cn=DC01" (DC01:636): map_entry_dn_inbound: looking for local >>>> entry by guid [919561f60fe49f409afcdf80a63eb089] >>>> [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - >>>> agmt="cn=DC01" (DC01:636): map_entry_dn_inbound: found local entry >>>> [CN=students,ou=Groups,ou=Active,ou=People,dc=domain,dc=com] >>>> [27/Feb/2013:12:20:33 +1300] - Calling windows entry search >>>> request plugin >>>> [27/Feb/2013:12:20:33 +1300] - windows_search_entry: received 2 >>>> messages, 1 entries, 0 references >>>> [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - >>>> agmt="cn=DC01" (DC01:636): map_entry_dn_inbound: looking for local >>>> entry matching AD entry >>>> [CN=students,OU=Groups,OU=Active,OU=People,DC=domain,DC=com] >>>> [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - >>>> windows_generate_update_mods: >>>> CN=students,ou=Groups,ou=Active,ou=People,dc=domain,dc=com, >>>> description : values are equal >>>> [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - >>>> windows_generate_update_mods: >>>> CN=students,ou=Groups,ou=Active,ou=People,dc=domain,dc=com, >>>> ntUserDomainId : values are equal >>>> [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - >>>> windows_generate_update_mods: deleting uniquemember attribute from >>>> local entry >>>> [27/Feb/2013:12:20:33 +1300] - smod - windows sync >>>> [27/Feb/2013:12:20:33 +1300] - smod 0 - delete: uniquemember >>>> >>>> >>>> The particularly interesting line is this >>>> [27/Feb/2013:12:20:33 +1300] NSMMReplicationPlugin - >>>> windows_generate_update_mods: deleting uniquemember attribute from >>>> local entry >>>> >>>> This only appears to happen when the group contains more than 1500 >>>> entries. >>>> >>>> Surely there must be someone else out there syncing groups with >>>> more than 1500 members between 389 and AD? >>>> >>>> It wasn't until I used Apache Directory Studio to compare entries >>>> between the 389 server and the AD that I noticed the attribute >>>> name was represented differently when the group contained over >>>> 1500 entries. >>>> >>>> This is a result of the range retrieval limit in AD. When you hit >>>> the range limit, the attribute name changes from "member" to >>>> "member;range=0-1499", which causes the mismatch that leads to the >>>> uniquemember attribute being deleted. >>>> >>>> In order to prevent this from happening, we have had to increase >>>> the MaxValRange setting in our Active Directory as per >>>> http://support.microsoft.com/kb/2009267 and >>>> http://support.microsoft.com/kb/315071 >>>> >>>> The value defaults to 1500 for Windows Server 2003 or 5000 for >>>> Windows Server 2008. >>>> >>>> Personally I consider this a bug in the AD sync plugin, as it >>>> fails to correctly handle range retrieval. At the very least, the >>>> documentation for Windows Sync should contain information about >>>> this limit. >>> This does sound like a bug. Please open a ticket in our Trac >>> instance here: >>> >>> https://fedorahosted.org/389/ >>> >>> The Windows Sync plug-in needs to be modified to understand how to >>> use ranged searches. >> https://fedorahosted.org/389/ticket/472 > This ticket is for simple paged results, which is different. Paged > results is used for returning a large number of entries. Range > retrieval is used for a large number of values for a multi-valued > attributes. Even when using paged results, AD will trim a > multi-valued attribute whose values pass the range retrieval limit. > > Do we even need to use simple paged results for AD sync? We are > simply searching for one entry at a time as we process through the > Dirsync results, so I don't think that we have a case where we expect > to receive a large number of entries as the result of a search > operation. We do need to deal with receiving a large number of > values for a multi-valued attribute (like member). Is there a case > I'm not thinking of that would require simple paged results? If not, > I propose we change ticket 472 to be used to add range retrieval > support and re-triage it. For the initial sync search request, we can retrieve a large number of entries, so we need to at least support paged searches in that case.

> > -NGK >>> >>> >>> Thanks, >>> -NGK >>>> >>>> David. >>>> -- >>>> 389 users mailing list >>>> 389-users(a)lists.fedoraproject.org >>>> https://admin.fedoraproject.org/mailman/listinfo/389-users >>> >>> -- >>> 389 users mailing list >>> 389-users(a)lists.fedoraproject.org >>> https://admin.fedoraproject.org/mailman/listinfo/389-users >> >