I've forgotten Linked Attributes plugin, you could also disable it.
Don't you have some exotic type of index activated for uniqueMember (like
substring)? The default value is only the equality index in dse.ldif
dn: cn=uniquemember,cn=default indexes,cn=config,cn=ldbm
database,cn=plugins,c
n=config
objectClass: top
objectClass: nsIndex
cn: uniquemember
nsSystemIndex: false
nsIndexType: eq
In any case, batch write loads are quite particular. You could try to play
with nsslapd-db-checkpoint-interval and nsslapd-db-durable-transaction
config attributes while you run your batch uniqueMember modifications. You
could also try disabling completely or limiting logging intensity (
http://directory.fedoraproject.org/wiki/Named_Pipe_Log_Script) .
In my VM tests (RHEL5, 1 vCPU Xeon E5640 @ 2.67GHz, 1Gb mem, 389DS
v1.2.10.6) with our production environment (~20k users, groups of ~6000
members created as in your case with perl scripts using ldapmodify running
on the same VM) a group of 6000 uniqueMembers is created in 3 minutes 10
sec (190s) from scratch. Using "dstat" i see that the main problem is disk
writes (transaction logs of db4):
----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read writ| recv send| in out | int csw
91 0 0 9 0 0| 0 40M| 120B 978B| 0 0 | 316 325
90 3 0 7 0 0| 0 40M| 60B 310B| 0 0 | 184 294
92 2 0 5 0 1| 0 40M| 60B 310B| 0 0 | 160 297
93 1 0 5 0 1| 0 40M| 60B 310B| 0 0 | 159 312
93 3 1 1 0 2| 0 24M| 60B 310B| 0 0 | 206 372
76 4 2 18 0 0| 0 40M| 60B 310B| 0 0 | 165 265
94 0 0 6 0 0| 0 40M| 60B 310B| 0 0 | 221 275
90 1 0 7 1 1| 0 41M| 60B 310B| 0 0 | 479 313
86 2 0 11 1 0| 0 40M| 60B 310B| 0 0 | 403 306
93 0 0 6 0 1| 0 20M| 120B 364B| 0 0 | 489 298
90 1 0 9 0 0| 0 40M| 60B 310B| 0 0 | 389 296
88 1 0 11 0 0| 0 40M| 60B 310B| 0 0 | 358 319
76 0 0 23 1 0| 0 41M| 60B 310B| 0 0 | 403 303
@+
Le 19 avril 2012 18:50, Russell Beall <beall(a)usc.edu> a écrit :
Thanks for the tips. I scanned the dse.ldif for those plugins and I
found
definitions for them all, but they all have nsslapd-pluginEnabled: off.
There is something special about the uniquemember attribute that requires
additional processing different from other attributes... Ldapmodify of
other attributes runs pretty quick.
Regards,
Russ.
On Apr 19, 2012, at 2:20 AM, Andrey Ivanov wrote:
Hi Russel,
Le 18 avril 2012 23:06, Russell Beall <beall(a)usc.edu> a écrit :
> On Apr 18, 2012, at 11:15 AM, Rich Megginson wrote:
>
> Yeah, this particular operation has not been optimized. I believe SunDS
> added explicit optimizations for this particular case.
>
>
>
> It is becoming painfully apparent as I write more detailed tests. 389
> takes time to add or delete uniquemember values proportionate to the number
> of values being operated on and is using about twice as much time to delete
> as it does to add. Sun DS appears to have perhaps an almost O(1) algorithm
> in play on both adding and deleting values.
>
> Is there perhaps some kind of referential integrity setting that is being
> used and forcing some kind of lookup of each value, one that we could
> perhaps turn off? We wouldn't need such a check because our metadirectory
> process handles the integrity/consistency checking already.
>
There is memberOf plugin that maintains the memberOf attribute for groups.
I don't know whether it is activated by default or not. You could try to
disable it. There is also referential integrity plugin, attribute
uniqueness plugin, maybe USN plugin or custom indexes that could consume a
lot of CPU. Make sure you've disabled them if you don't need them.
@+
--
389 users mailing list
389-users(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
--
389 users mailing list
389-users(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users