On 04/19/2012 11:38 AM, Russell Beall wrote:
That is by far the largest example we have.  We use groups with the uniquemember attribute linking to account entries and more than a few of the groups have tens of thousands of values for uniquemember.  We create more of these groups regularly and it will be a problem for it to take many hours to construct such a group versus seconds/minutes with Sun DS.  Our metadirectory process does not use ldapadd to create the group pre-populated, the group is created and then ldapmodify is run to add members.  Three times a year at the change of semester, many thousands of group membership changes are processed and we already have a problem with it taking multiple days to process the entire set...

OK.  If you've ruled out the possibility that some plugin is interfering with the processing, then it must be something we will have to fix in the core server.  Please file a ticket at https://fedorahosted.org/389


We also have large quantities of eduPersonEntitlement on account records, but those sets are not nearly as numerously populated.  I can delete and re-add the eduPersonEntitlement attribute across 110,000 records in about 40 minutes (20 minutes each way -- with 389).

Russ.

On Apr 19, 2012, at 10:18 AM, Rich Megginson wrote:

On 04/19/2012 10:50 AM, Russell Beall wrote:
Thanks for the tips.  I scanned the dse.ldif for those plugins and I found definitions for them all, but they all have nsslapd-pluginEnabled: off.

There is something special about the uniquemember attribute that requires additional processing different from other attributes...  Ldapmodify of other attributes runs pretty quick.

Is uniquemember the only attribute using large numbers of multiple values in ldapmodify operations?


Regards,
Russ.

On Apr 19, 2012, at 2:20 AM, Andrey Ivanov wrote:

Hi Russel,


Le 18 avril 2012 23:06, Russell Beall <beall@usc.edu> a écrit :
On Apr 18, 2012, at 11:15 AM, Rich Megginson wrote:
Yeah, this particular operation has not been optimized.  I believe SunDS added explicit optimizations for this particular case.


It is becoming painfully apparent as I write more detailed tests.  389 takes time to add or delete uniquemember values proportionate to the number of values being operated on and is using about twice as much time to delete as it does to add.  Sun DS appears to have perhaps an almost O(1) algorithm in play on both adding and deleting values.

Is there perhaps some kind of referential integrity setting that is being used and forcing some kind of lookup of each value, one that we could perhaps turn off?  We wouldn't need such a check because our metadirectory process handles the integrity/consistency checking already.

There is memberOf plugin that maintains the memberOf attribute for groups. I don't know whether  it is activated by default or not. You could try to disable it. There is also referential integrity plugin, attribute uniqueness plugin, maybe USN plugin or custom indexes that could consume a lot of CPU. Make sure you've disabled them if you don't need them.

@+
--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users



--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users




--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users