389-users March 2016

389-users@lists.fedoraproject.org

28 participants
26 discussions

locking performance and scalability (eye candy gnuplots inside!)
by liblfds admin 09 Mar '16

09 Mar '16

Hi, everyone. I am the author of liblfds (a lock-free data structure library), which is used by nunc-stans, which is used by 389. By various events of no interest to anyone, I've ended up idling in the #389 channel on Freenode and earlier today had a conversation with richm (who used liblfds in nunc-stans), regarding performance, data structures in use, locking and so on. I was asked by richm to post the substantive content of our conversation to this mailing list. === I am currently working on a benchmark application for liblfds. This benchmarks the lock-free data structures in the library, and also *locking* versions of those same data structures, where the locks tested are all those available in the platform. I've not finished yet, and so currently only have two gnuplots available, from an earlier prototype benchmark application. Note the 16 core image is I think it's 1800x2400 pixels. The file size is still small (~250kb). http://www.liblfds.org/freelist_4core.png http://www.liblfds.org/freelist_16core.png The CPU topology diagram can be found at the top of each image. The four core machine is my laptop, real hardware, real Linux. The sixteen core machine is a dedicated-hardware Amazon VM, so all bets are off. It looks nice, but God knows what the Xen configuration is. I can't even know there's a one-to-one mapping between virtual cores and logical cores - I've seen cloud providers where this is *not* the case. Turning first to the four core plot, what we see is that while we are within the confines of a single physical core, spinlocks win. This is because the physical core knows what its logical cores are up to. As soon as we step outside of a single physical core, lock-free wins, and wins big - an order of magnitude. We see however, looking at the sad, short bars in the final graph, that the freelist in its current form does not scale linearly with cores. This is because of memory contention on the single cache line holding the freelist head pointer. There is a method to improve on this, though, something called an elimination layer, which is based on the insight that when a thread which pushes at or almost at the same time as a thread which pops, those two threads can service each others request, without having to access the freelist head pointer. Turning to the (questionable - VM, don't forget) sixteen core plot, we see much the same. Lock-free wins, but it's not scaling. We do see one or two other items of note; looking at the second graph, we can see a huge imbalance between the spinlock performances on the two cores; in both cases, one core dominated the other and stopped it from getting much work done. The total amount of work done then is fairly high - because we are in fact by this behaviour starting to approach a single-threaded scenario! The pthread mutex absolutely does not display this behaviour, since it serializes access and I believe internally is maintaining a queue of requestors; so everyone gets a fair chance. The spinlocks of course just do their thing, and then we're much more reliant upon the hardware to provide fairness, and as we see, this may not be a safe bet. We also see in the third graph and fifth graphs that the first core performed badly compared to the others. I do not know why, but it needs explanation. Relating to this though, although I don't have other plots, but I did do quite a lot of benchmarking with the prototype benchmark app, and I seemed to find on some machines that some cores *always* did badly, and for *all* lock types. I have a weak suspicion it reflects internal memory access arrangements in the core itself. However, this was on VMs - although they were all dedicated hardware, so I had no neighbours on the box - so it could just be Xen doing strange things. I need to add RW locks to the benchmark, as they are used by nunc-stans/389. The recent new release of liblfds included - and this was done for implementation speed, to at least get some functionality out there ASAP - an *addonly* list, btree and hash. The list will become normal (read-write) soon enough, as I have a safe memory reclamation implementation about to come out. There is now in the literature a lock-free red-black tree, which was invented last year. Cliff Click has had a proper lock-free hash out for some time now. Each of these will take a month to implement, whereas the addonly versions were straightforward, so I got them out first.

1 0

NSACLPlugin warning/error
by ghiureai 07 Mar '16

07 Mar '16

Hi List, I am seeing this warning/error in one of my LDAP log, ( version 1.2.11) " NSACLPlugin - Can't find the acl in the tree for moddn operation:olddnuid= " I would like to know more details , is this related to an aci issue/missing etc? Isabella

2 1

automembership plugin questions
by Frank Munsche 05 Mar '16

05 Mar '16

Hi Guys, I try to get the automembership plugin configured and running in the right way, but it still causes some headaches. I'm running CentOS 6.7 and the following 389 packages: 389-ds-base-1.2.11.15-69.el6_7.x86_64 389-ds-base-libs-1.2.11.15-69.el6_7.x86_64 I followed the guide at http://directory.fedoraproject.org/docs/389ds/design/automember-design.html and created a group, where the members should be automatically added: dn: cn=users,ou=app,ou=groups,dc=example,dc=com description: users group objectClass: top objectClass: groupofnames cn: users And the corresponding auto membership plugin entry: dn: cn=users,cn=Auto Membership Plugin,cn=plugins,cn=config cn: users objectclass: top objectclass: autoMemberDefinition autoMemberScope: dc=example,dc=com autoMemberFilter: Team=ops autoMemberDefaultGroup: cn=users,ou=app,ou=groups,dc=example,dc=com autoMemberGroupingAttr: member:dn After that, I added a user who got the Team attribute (own schema extension) set to 'ops' dn: uid=foo,ou=people,dc=example,dc=com objectClass: top objectClass: posixAccount objectClass: organizationalPerson objectClass: inetorgPerson objectClass: person objectClass: shadowAccount objectClass: ldapPublicKey objectClass: bopsUser objectClass: inetuser uid: foo uidNumber: 2222 gidNumber: 2222 homeDirectory: /home/foo loginShell: /bin/bash cn: foo bar givenName: foo sn: bar gecos: foo bar mail: foo.bar(a)example.com Team: ops A ldapsearch for the groups member attribute shows the user's dn being added automatically: # users, app, groups, bildops.de dn: cn=users,ou=app,ou=groups,dc=example,dc=com member: uid=foo,ou=people,dc=example,dc=com When the user entry is removed, the membership also disappears. But whenever I just changed the Team attribute of an existing user, the membership did not change. I found a page describing a rebuild task implemented in the plugin at: https://fedorahosted.org/389/ticket/20 It worked for me for any existing users, who got the Team attribute set to 'ops'. The task finds them and puts their dn into the group. But the opposite way did not work: I changed the Team's value from 'ops' to 'none' and started the task, but the user's dn is still a member of the group. Is this the expected behavior? thank you very much, cheers, Frank

2 2

ns-slapd memory usage
by Radu Pantiru 04 Mar '16

04 Mar '16

Hi, I am using 1.3.3 and the reserved memory usage it is going up on average ~ 800MB per week and I have a cache hit rate of ~98% on both ldap userRoot and db_stat Is this normal behavior or possibly I have a memory leak? Regards, Radu

4 3

Passwords migration from openldap to 389DS
by wodel youchi 03 Mar '16

03 Mar '16

Hi, We're planning the migration from openldap to 389DS, we did some tests, and we have some problems with users passwords migration. We found this article in 389DS's website http://directory.fedoraproject.org/docs/389ds/design/password-migration-des… Is this implemented or not yet or completely abandoned? the Pass through plugin worked, but we didn't find the "password migration mode option" to catch the passwords. Regards.

2 1

Installation of 389 DS
by wodel youchi 03 Mar '16

03 Mar '16

Hi, I am a newbie on 389 DS, I was following the RDS install document from RedHat Documentation. OS: Centos 7.2 x64 latest updates 389 DS : 389-admin-console-1.1.10-1.el7.noarch 389-ds-base-libs-1.3.4.0-26.el7_2.x86_64 389-ds-base-1.3.4.0-26.el7_2.x86_64 389-console-1.1.9-1.el7.noarch 389-ds-console-1.2.12-1.el7.noarch 389-adminutil-1.1.22-1.el7.x86_64 389-admin-1.1.42-1.el7.x86_64 In the consideration before setting up DS, it's mentioned that we need to add this line to */etc/pam.d/system-authsession required /lib/security/$ISA/pam_limits.so* After adding this line and rebooting the server, I am getting this error when I try to login into it: *Unknown module* in */var/log/secure* I have *login: PAM unable to dlopen(/lib/security/$ISA/pam_limits.so): /lib/security/../../lib64/security/pam_limits.so: cannot open shared object file: No such file or directory* I did read the */etc/pam.d/system-auth* file again, and I found that there is a line like this in it *session required pam_limits.so* My question is : do I need the *session required /lib/security/$ISA/pam_limits.so* for 389 to work properly ? and if yes, how to avoid the above error? if no, does *session required pam_limits.so* do the work? Regards.

2 3

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

389-users March 2016