Hi SSSD Users.
I'm trying to increase the performance of my user's logins, we have a medium sized Active Direcctory.
According to the man page, the enumerate directive:
enumerate (bool) Determines if a domain can be enumerated. This parameter can have one of the following values: TRUE = Users and groups are enumerated FALSE = No enumerations for this domain
However when I start sssd with no cache and simulate an initgroups, it still seems to enumerate many many groups and user accounts.
I'm running sssd v1.8.4:
# pkill sssd # pgrep sssd # pwd /apps/sssd-1.8.4 # rm -f var/lib/sss/db/* # grep enumerate /etc/sssd/sssd.conf enumerate = FALSE # grep ldap_access /etc/sssd/sssd.conf ldap_access_filter = memberOf=cn=xxxgroup,ou=yyyOU,ou=zzzOU,ou=Groups,dc=aaa,dc=bbb,dc=ccc # sbin/sssd -c /etc/sssd/sssd.conf # su - myuser -c "groups | wc" 1 193 1181 # strings var/lib/sss/db/cache_AAA.BBB.CCC.ldb | grep OU=Groups,DC=aaa,DC=bbb,DC=ccc | sort -u | wc -l 522 # strings var/lib/sss/db/cache_AAA.BBB.CCC.ldb | grep OU=Accounts,DC=aaa,DC=bbb,DC=ccc | sort -u | wc -l 1938
Sorry for my use of strings and sort -u, I don't know a better way to interrogate the cache.
Why does it still enumerate so many users and groups (that are not me, and not in my ldap_access_filter) when I log in? Even when I have disabled domain enumeration?
Regards,
Tim.
________________________________
This e-mail is sent by Suncorp Group Limited ABN 66 145 290 124 or one of its related entities "Suncorp". Suncorp may be contacted at Level 18, 36 Wickham Terrace, Brisbane or on 13 11 55 or at suncorp.com.au. The content of this e-mail is the view of the sender or stated author and does not necessarily reflect the view of Suncorp. The content, including attachments, is a confidential communication between Suncorp and the intended recipient. If you are not the intended recipient, any use, interference with, disclosure or copying of this e-mail, including attachments, is unauthorised and expressly prohibited. If you have received this e-mail in error please contact the sender immediately and delete the e-mail and any attachments from your system.
On Thu, May 31, 2012 at 06:50:55AM +0000, GOLLSCHEWSKY, Tim wrote:
Hi SSSD Users.
I'm trying to increase the performance of my user's logins, we have a medium sized Active Direcctory.
According to the man page, the enumerate directive:
enumerate (bool) Determines if a domain can be enumerated. This parameter can have one of the following values: TRUE = Users and groups are enumerated FALSE = No enumerations for this domain
I think there is some confusion with the terminology the SSSD uses. "enumerate=true" for SSSD means "download all data in the directory periodically".
However when I start sssd with no cache and simulate an initgroups, it still seems to enumerate many many groups and user accounts.
I assume that you simulated initgroups with id -G or logging in?
I'm running sssd v1.8.4:
# pkill sssd # pgrep sssd # pwd /apps/sssd-1.8.4 # rm -f var/lib/sss/db/* # grep enumerate /etc/sssd/sssd.conf enumerate = FALSE # grep ldap_access /etc/sssd/sssd.conf ldap_access_filter = memberOf=cn=xxxgroup,ou=yyyOU,ou=zzzOU,ou=Groups,dc=aaa,dc=bbb,dc=ccc
The access filter only tells the SSSD who can log in, it doesn't have effect on the data downloaded.
Since SSSD 1.7, the SSSD allows specifying filter as part of the search base. For example, to only download users that are members of a particular group:
ldap_user_search_base = OU=Accounts,DC=aaa,DC=bbb,DC=ccc?subtree?(memberOf=cn=xxxgroup,ou=yyyOU,ou=zzzOU,ou=Groups,dc=aaa,dc=bbb,dc=ccc)
The format is as per RFC2254. See man sssd-ldap for more details.
# sbin/sssd -c /etc/sssd/sssd.conf # su - myuser -c "groups | wc" 1 193 1181 # strings var/lib/sss/db/cache_AAA.BBB.CCC.ldb | grep OU=Groups,DC=aaa,DC=bbb,DC=ccc | sort -u | wc -l 522 # strings var/lib/sss/db/cache_AAA.BBB.CCC.ldb | grep OU=Accounts,DC=aaa,DC=bbb,DC=ccc | sort -u | wc -l 1938
Sorry for my use of strings and sort -u, I don't know a better way to interrogate the cache.
To poke at the cache, install the ldb-tools package and use the "ldbsearch" utility. It works much like ldapsearch:
ldbsearch -H /var/lib/sss/db/cache_AAA.BBB.CCC.ldb 'objectclass=user'
Why does it still enumerate so many users and groups (that are not me, and not in my ldap_access_filter) when I log in? Even when I have disabled domain enumeration?
Regards,
Tim.
Hi Jakub,
Since SSSD 1.7, the SSSD allows specifying filter as part of the search base. For example, to only download users that are members of a particular group:
ldap_user_search_base = OU=Accounts,DC=aaa,DC=bbb,DC=ccc?subtree?(memberOf=cn=xxxgroup,ou=yyyO U,ou=zzzOU,ou=Groups,dc=aaa,dc=bbb,dc=ccc)
I've been having a play around with this. Strangely it doesn't seem to limit much... and I can't work out why. :/
If I have the ldap_user_search_base set at the OU level, it gets this many user records:
# grep ^ldap_user_search_base /etc/sssd/sssd.conf ldap_user_search_base = ou=Accounts,dc=aaa,dc=bbb,dc=ccc # ldbsearch -H var/lib/sss/db/cache_AAA.BBB.CCC.ldb 'objectclass=user' | grep ^gecos | wc -l asq: Unable to register control with rootdse! 1939
But if I put in the memberOf=cn= in there, it gets:
# grep ^ldap_user_search_base /etc/sssd/sssd.conf ldap_user_search_base = ou=Accounts,dc=aaa,dc=bbb,dc=ccc?subtree?(memberOf=cn=mygroup,ou=UnixGroups,ou=Groups,ou=aaa,ou=bbb,ou=ccc) # ldbsearch -H var/lib/sss/db/cache_AAA.BBB.CCC.ldb 'objectclass=user' | grep ^gecos | wc -l asq: Unable to register control with rootdse! 1938
So not a lot of difference. Just 1 record less.
I'll ensure all my testing from now on is an actual ssh login and not the "groups" command.
I suspect the reason is that SSSD tries to be smart here and differentiates between initgroups operation performed via the NSS interface (id -G) and initgroups performed during auth via the PAM stack. The initgroups that comes from the NSS can be returned from the cache for speed's sake. On the other hand, the PAM responder only keeps a very short-lived in-memory cache - see the pam_id_timeout option in sssd.conf, defaults to 5 seconds. Its purpose is mainly to only contact the server once even if the login program performs multiple initgroups calls during login - SSH is one example, it performs several initgroups operations during various stages of the PAM conversation and even from different processes.
*Maybe* we could do more heuristics if the group and user base DNs are specified and don't overlap, something similar to what's proposed in https://fedorahosted.org/sssd/ticket/1319 ?
If I'm interpreting this correctly (that we can determine if an object is a "user" record based on the DN and not have to do a lookup), and if we can also do this for AD as well as IPA, I think this enhancement would provide a truly remarkable performance improvement for AD-based sites like mine using RFC2307bis with a medium(to large) sized directory.
This goes back to my original question. When I have enumerate disabled, why does SSSD still look up all this fake user information in LDAP? And is there any way to disable it from doing so?
Hm, this is interesting. I'm not seeing this in my test setup -- I tested with the 1.8 branch, with ldap_group_nesting_level=0 against a RFC2307bis server. Would you mind sharing a snippet from your logs?
Sure.
There is probably a better way to do this. :) But this is how I'm doing it: turning on debug, and grepping for ldap_search:
sbin/sssd -c /etc/sssd/sssd.conf -i -d8 2>&1 | fgrep calling\ ldap_search_ext > /tmp/searches.txt
Note in this example I'm using a cleared cache with nesting = 0 and enumerate = false and the following user search base. The login took 31 seconds.
ldap_user_search_base = ou=Accounts,dc=aaa,dc=bbb,dc=ccc?subtree?(memberOf=CN=mygroup,OU=UnixGroups,OU=Groups,DC=aaa,DC=bbb,DC=ccc)
# wc -l /tmp/searches.txt 2980 /tmp/searches.txt # grep objectclass=user /tmp/searches.txt | wc -l 2397
"mygroup" contains 581 users.
This is what some of the searches look like:
# grep objectclass=user /tmp/searches.txt | head (Mon Jun 4 16:25:11 2012) [sssd[be[AAA.BBB.CCC]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(&(&(msSFU30Name=myuser)(objectclass=user))(memberOf=CN=mygroup,OU=UnixGroups,OU=Groups,DC=aaa,DC=bbb,DC=ccc))][ou=Accounts,dc=aaa,dc=corp,dc=ccc]. (Mon Jun 4 16:25:11 2012) [sssd[be[AAA.BBB.CCC]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(&(&(msSFU30Name=myuser)(objectclass=user))(memberOf=CN=mygroup,OU=UnixGroups,OU=Groups,DC=aaa,DC=bbb,DC=ccc))][ou=Accounts,dc=aaa,dc=corp,dc=ccc]. (Mon Jun 4 16:25:12 2012) [sssd[be[AAA.BBB.CCC]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(objectclass=user)][CN=someuser1,OU=Admin Accounts,OU=Accounts,DC=aaa,DC=bbb,DC=ccc]. (Mon Jun 4 16:25:12 2012) [sssd[be[AAA.BBB.CCC]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(objectclass=user)][CN=someuser2,OU=Admin Accounts,OU=Accounts,DC=aaa,DC=bbb,DC=ccc]. (Mon Jun 4 16:25:12 2012) [sssd[be[AAA.BBB.CCC]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(objectclass=user)][CN=someuser3,OU=Admin Accounts,OU=Accounts,DC=aaa,DC=bbb,DC=ccc]. (Mon Jun 4 16:25:12 2012) [sssd[be[AAA.BBB.CCC]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(objectclass=user)][CN=someuser4,OU=Admin Accounts,OU=Accounts,DC=aaa,DC=bbb,DC=ccc]. (Mon Jun 4 16:25:12 2012) [sssd[be[AAA.BBB.CCC]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(objectclass=user)][CN=someuser5,OU=Admin Accounts,OU=Accounts,DC=aaa,DC=bbb,DC=ccc]. (Mon Jun 4 16:25:12 2012) [sssd[be[AAA.BBB.CCC]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(objectclass=user)][CN=someuser6,OU=Admin Accounts,OU=Accounts,DC=aaa,DC=bbb,DC=ccc]. (Mon Jun 4 16:25:12 2012) [sssd[be[AAA.BBB.CCC]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(objectclass=user)][CN=someuser7,OU=Users,OU=Accounts,DC=aaa,DC=bbb,DC=ccc]. (Mon Jun 4 16:25:12 2012) [sssd[be[AAA.BBB.CCC]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(objectclass=user)][CN=someuser8,OU=Users,OU=Accounts,DC=aaa,DC=bbb,DC=ccc].
If more sanitised debug output would be useful, please let me know.
Cheers, Tim.
________________________________
This e-mail is sent by Suncorp Group Limited ABN 66 145 290 124 or one of its related entities "Suncorp". Suncorp may be contacted at Level 18, 36 Wickham Terrace, Brisbane or on 13 11 55 or at suncorp.com.au. The content of this e-mail is the view of the sender or stated author and does not necessarily reflect the view of Suncorp. The content, including attachments, is a confidential communication between Suncorp and the intended recipient. If you are not the intended recipient, any use, interference with, disclosure or copying of this e-mail, including attachments, is unauthorised and expressly prohibited. If you have received this e-mail in error please contact the sender immediately and delete the e-mail and any attachments from your system.
On Thu, 2012-05-31 at 06:50 +0000, GOLLSCHEWSKY, Tim wrote:
Hi SSSD Users.
I'm trying to increase the performance of my user's logins, we have a medium sized Active Direcctory.
According to the man page, the enumerate directive:
enumerate (bool) Determines if a domain can be enumerated. This parameter can have one of the following values: TRUE = Users and groups are enumerated FALSE = No enumerations for this domain
However when I start sssd with no cache and simulate an initgroups, it still seems to enumerate many many groups and user accounts.
I'm running sssd v1.8.4:
# pkill sssd # pgrep sssd # pwd /apps/sssd-1.8.4 # rm -f var/lib/sss/db/* # grep enumerate /etc/sssd/sssd.conf enumerate = FALSE # grep ldap_access /etc/sssd/sssd.conf ldap_access_filter = memberOf=cn=xxxgroup,ou=yyyOU,ou=zzzOU,ou=Groups,dc=aaa,dc=bbb,dc=ccc # sbin/sssd -c /etc/sssd/sssd.conf # su - myuser -c "groups | wc" 1 193 1181 # strings var/lib/sss/db/cache_AAA.BBB.CCC.ldb | grep OU=Groups,DC=aaa,DC=bbb,DC=ccc | sort -u | wc -l 522 # strings var/lib/sss/db/cache_AAA.BBB.CCC.ldb | grep OU=Accounts,DC=aaa,DC=bbb,DC=ccc | sort -u | wc -l 1938
Sorry for my use of strings and sort -u, I don't know a better way to interrogate the cache.
Why does it still enumerate so many users and groups (that are not me, and not in my ldap_access_filter) when I log in? Even when I have disabled domain enumeration?
As Jakub suggested, your investigation is slightly flawed. I'm guessing your version of "simulating an initgroups" is by running 'id username'. This is actually different from initgroups. What this does is an initgroups() call followed by a loop to look up every group that the user is a member of.
The net effect of this is that by doing this, we're also doing a lookup of all the users in those groups (we don't have a choice in this, because RFC2307bis servers can have other groups as a member and we cannot know which we're dealing with until we request it).
Most of what you're seeing in the 'strings' are actually what we call "fake" users though. They're users we've saved a minimal set of data about in the cache so that we can maintain our member/memberOf linkages so that group lookups work properly.
Also, you get many more results when grepping on the search base there because it will appear in multiple places among groups and users. Groups will have it listed as the 'originalDN' attribute, but every user that is a member of those groups will also have an 'originalMemberOf' entry pointing at that subtree as well. This is also the reason you're seeing groups that your initial user doesn't belong to. It's showing up in the originalMemberOf attribue of these fake users.
So if you want to get a better view of what is happening, use the ldb-tools as Jakub said. Also, if you want to test JUST initgroups(), use the 'id -G' command instead of the bare 'id' command. This will skip the individual group lookups and save you a lot of trouble.
Hi Jakub and Stephen,
Thanks for your responses.
Stephen wrote:
Why does it still enumerate so many users and groups (that are not me, and not in my ldap_access_filter) when I log in? Even when I have disabled domain enumeration?
As Jakub suggested, your investigation is slightly flawed. I'm guessing your version of "simulating an initgroups" is by running 'id username'. This is actually different from initgroups. What this does is an initgroups() call followed by a loop to look up every group that the user is a member of.
OK, thanks. I see now I've been using the wrong method to test this.
You guys have suggested "id -G" or actually logging in. "id -G" doesnt seem to have the same effect on SSSD that actually logging in has. "id -G" returns quite quickly, whereas a normal ssh login takes a very long time as SSSD goes nuts performing user lookups on our AD (for hundreds of users that aren't even logging into the server at the time).
I'll ensure all my testing from now on is an actual ssh login and not the "groups" command.
The net effect of this is that by doing this, we're also doing a lookup of all the users in those groups (we don't have a choice in this, because RFC2307bis servers can have other groups as a member and we cannot know which we're dealing with until we request it).
Yes this is what I'm seeing. But is this also the case when I have disabled nesting?
# grep nesting /etc/sssd/sssd.conf ldap_group_nesting_level = 0
All our LDAP groups in the Unix OU have been flattened specifically to negate the need for nested searching and (hopefully) increase performance.
Most of what you're seeing in the 'strings' are actually what we call "fake" users though. They're users we've saved a minimal set of data about in the cache so that we can maintain our member/memberOf linkages so that group lookups work properly.
OK thanks, that makes sense. So when you get the results from a group search, you create "fake" entries for those users.
Can I ask why SSSD still performs an LDAP lookup for each of these "fake" users? Since none of these users are actually logging in at the time, it seems to penalise the user who is logging in because sshd(& pam) blocks waiting for a response from SSSD.
I'm just installed ldb-tools (this is going to make interrogating the cache much easier!) and I can see fullname and gecos information for almost 2000 users after I clear the cache log in with my login user.
# pkill sssd # rm -f var/lib/sss/db/* # sbin/sssd -c /etc/sssd/sssd.conf # ssh myuser@0 date Fri Jun 1 10:48:56 EST 2012 # ldbsearch -H var/lib/sss/db/cache_AAA.BBB.CCC.ldb 'objectclass=user' | grep ^gecos | wc -l asq: Unable to register control with rootdse! 1937
I can confirm SSSD is doing ldap user lookups for all the "fake" users by turning on debug.
This goes back to my original question. When I have enumerate disabled, why does SSSD still look up all this fake user information in LDAP? And is there any way to disable it from doing so?
Thanks for your help.
Tim.
________________________________
This e-mail is sent by Suncorp Group Limited ABN 66 145 290 124 or one of its related entities "Suncorp". Suncorp may be contacted at Level 18, 36 Wickham Terrace, Brisbane or on 13 11 55 or at suncorp.com.au. The content of this e-mail is the view of the sender or stated author and does not necessarily reflect the view of Suncorp. The content, including attachments, is a confidential communication between Suncorp and the intended recipient. If you are not the intended recipient, any use, interference with, disclosure or copying of this e-mail, including attachments, is unauthorised and expressly prohibited. If you have received this e-mail in error please contact the sender immediately and delete the e-mail and any attachments from your system.
On Fri, Jun 01, 2012 at 12:53:36AM +0000, GOLLSCHEWSKY, Tim wrote:
Hi Jakub and Stephen,
Thanks for your responses.
Stephen wrote:
Why does it still enumerate so many users and groups (that are not me, and not in my ldap_access_filter) when I log in? Even when I have disabled domain enumeration?
As Jakub suggested, your investigation is slightly flawed. I'm guessing your version of "simulating an initgroups" is by running 'id username'. This is actually different from initgroups. What this does is an initgroups() call followed by a loop to look up every group that the user is a member of.
OK, thanks. I see now I've been using the wrong method to test this.
You guys have suggested "id -G" or actually logging in. "id -G" doesnt seem to have the same effect on SSSD that actually logging in has. "id -G" returns quite quickly, whereas a normal ssh login takes a very long time as SSSD goes nuts performing user lookups on our AD (for hundreds of users that aren't even logging into the server at the time).
I'll ensure all my testing from now on is an actual ssh login and not the "groups" command.
I suspect the reason is that SSSD tries to be smart here and differentiates between initgroups operation performed via the NSS interface (id -G) and initgroups performed during auth via the PAM stack.
The initgroups that comes from the NSS can be returned from the cache for speed's sake. On the other hand, the PAM responder only keeps a very short-lived in-memory cache - see the pam_id_timeout option in sssd.conf, defaults to 5 seconds. Its purpose is mainly to only contact the server once even if the login program performs multiple initgroups calls during login - SSH is one example, it performs several initgroups operations during various stages of the PAM conversation and even from different processes.
The net effect of this is that by doing this, we're also doing a lookup of all the users in those groups (we don't have a choice in this, because RFC2307bis servers can have other groups as a member and we cannot know which we're dealing with until we request it).
Yes this is what I'm seeing. But is this also the case when I have disabled nesting?
# grep nesting /etc/sssd/sssd.conf ldap_group_nesting_level = 0
All our LDAP groups in the Unix OU have been flattened specifically to negate the need for nested searching and (hopefully) increase performance.
Still, the SSSD needs to check the DNs to see if the DN points to a user entry or not. In theory, the group members can be other objects than users.
*Maybe* we could do more heuristics if the group and user base DNs are specified and don't overlap, something similar to what's proposed in https://fedorahosted.org/sssd/ticket/1319 ?
Most of what you're seeing in the 'strings' are actually what we call "fake" users though. They're users we've saved a minimal set of data about in the cache so that we can maintain our member/memberOf linkages so that group lookups work properly.
OK thanks, that makes sense. So when you get the results from a group search, you create "fake" entries for those users.
Can I ask why SSSD still performs an LDAP lookup for each of these "fake" users? Since none of these users are actually logging in at the time, it seems to penalise the user who is logging in because sshd(& pam) blocks waiting for a response from SSSD.
I'm just installed ldb-tools (this is going to make interrogating the cache much easier!) and I can see fullname and gecos information for almost 2000 users after I clear the cache log in with my login user.
# pkill sssd # rm -f var/lib/sss/db/* # sbin/sssd -c /etc/sssd/sssd.conf # ssh myuser@0 date Fri Jun 1 10:48:56 EST 2012 # ldbsearch -H var/lib/sss/db/cache_AAA.BBB.CCC.ldb 'objectclass=user' | grep ^gecos | wc -l asq: Unable to register control with rootdse! 1937
I can confirm SSSD is doing ldap user lookups for all the "fake" users by turning on debug.
This goes back to my original question. When I have enumerate disabled, why does SSSD still look up all this fake user information in LDAP? And is there any way to disable it from doing so?
Thanks for your help.
Tim.
Hm, this is interesting. I'm not seeing this in my test setup -- I tested with the 1.8 branch, with ldap_group_nesting_level=0 against a RFC2307bis server. Would you mind sharing a snippet from your logs?
On 06/01/2012 03:11 AM, Jakub Hrozek wrote:
On Fri, Jun 01, 2012 at 12:53:36AM +0000, GOLLSCHEWSKY, Tim wrote:
Hi Jakub and Stephen,
Thanks for your responses.
Stephen wrote:
Why does it still enumerate so many users and groups (that are not me, and not in my ldap_access_filter) when I log in? Even when I have disabled domain enumeration?
As Jakub suggested, your investigation is slightly flawed. I'm guessing your version of "simulating an initgroups" is by running 'id username'. This is actually different from initgroups. What this does is an initgroups() call followed by a loop to look up every group that the user is a member of.
OK, thanks. I see now I've been using the wrong method to test this.
You guys have suggested "id -G" or actually logging in. "id -G" doesnt seem to have the same effect on SSSD that actually logging in has. "id -G" returns quite quickly, whereas a normal ssh login takes a very long time as SSSD goes nuts performing user lookups on our AD (for hundreds of users that aren't even logging into the server at the time).
I'll ensure all my testing from now on is an actual ssh login and not the "groups" command.
I suspect the reason is that SSSD tries to be smart here and differentiates between initgroups operation performed via the NSS interface (id -G) and initgroups performed during auth via the PAM stack.
The initgroups that comes from the NSS can be returned from the cache for speed's sake. On the other hand, the PAM responder only keeps a very short-lived in-memory cache - see the pam_id_timeout option in sssd.conf, defaults to 5 seconds. Its purpose is mainly to only contact the server once even if the login program performs multiple initgroups calls during login - SSH is one example, it performs several initgroups operations during various stages of the PAM conversation and even from different processes.
The net effect of this is that by doing this, we're also doing a lookup of all the users in those groups (we don't have a choice in this, because RFC2307bis servers can have other groups as a member and we cannot know which we're dealing with until we request it).
Yes this is what I'm seeing. But is this also the case when I have disabled nesting?
# grep nesting /etc/sssd/sssd.conf ldap_group_nesting_level = 0
All our LDAP groups in the Unix OU have been flattened specifically to negate the need for nested searching and (hopefully) increase performance.
Still, the SSSD needs to check the DNs to see if the DN points to a user entry or not. In theory, the group members can be other objects than users.
*Maybe* we could do more heuristics if the group and user base DNs are specified and don't overlap, something similar to what's proposed in https://fedorahosted.org/sssd/ticket/1319 ?
Are you saying that no when we get a set of the DNs in the memberOf we do not know whether the entry is a user or a group so we have to fetch entries to see if the entry is a user or a group? IMO with there should be a way to regex it in SSSD and fetch only in cases when it is not possible to determine if the member is a user or a group. Use of the regex will help with other 2307 bis servers not just IPA. We need to think about AD too.
Most of what you're seeing in the 'strings' are actually what we call "fake" users though. They're users we've saved a minimal set of data about in the cache so that we can maintain our member/memberOf linkages so that group lookups work properly.
OK thanks, that makes sense. So when you get the results from a group search, you create "fake" entries for those users.
Can I ask why SSSD still performs an LDAP lookup for each of these "fake" users? Since none of these users are actually logging in at the time, it seems to penalise the user who is logging in because sshd(& pam) blocks waiting for a response from SSSD.
I'm just installed ldb-tools (this is going to make interrogating the cache much easier!) and I can see fullname and gecos information for almost 2000 users after I clear the cache log in with my login user.
# pkill sssd # rm -f var/lib/sss/db/* # sbin/sssd -c /etc/sssd/sssd.conf # ssh myuser@0 date Fri Jun 1 10:48:56 EST 2012 # ldbsearch -H var/lib/sss/db/cache_AAA.BBB.CCC.ldb 'objectclass=user' | grep ^gecos | wc -l asq: Unable to register control with rootdse! 1937
I can confirm SSSD is doing ldap user lookups for all the "fake" users by turning on debug.
This goes back to my original question. When I have enumerate disabled, why does SSSD still look up all this fake user information in LDAP? And is there any way to disable it from doing so?
Thanks for your help.
Tim.
Hm, this is interesting. I'm not seeing this in my test setup -- I tested with the 1.8 branch, with ldap_group_nesting_level=0 against a RFC2307bis server. Would you mind sharing a snippet from your logs? _______________________________________________ sssd-users mailing list sssd-users@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/sssd-users
On Sat, 2012-06-02 at 14:28 -0400, Dmitri Pal wrote:
On 06/01/2012 03:11 AM, Jakub Hrozek wrote:
On Fri, Jun 01, 2012 at 12:53:36AM +0000, GOLLSCHEWSKY, Tim wrote:
Hi Jakub and Stephen,
Thanks for your responses.
Stephen wrote:
Why does it still enumerate so many users and groups (that are not me, and not in my ldap_access_filter) when I log in? Even when I have disabled domain enumeration?
As Jakub suggested, your investigation is slightly flawed. I'm guessing your version of "simulating an initgroups" is by running 'id username'. This is actually different from initgroups. What this does is an initgroups() call followed by a loop to look up every group that the user is a member of.
OK, thanks. I see now I've been using the wrong method to test this.
You guys have suggested "id -G" or actually logging in. "id -G" doesnt seem to have the same effect on SSSD that actually logging in has. "id -G" returns quite quickly, whereas a normal ssh login takes a very long time as SSSD goes nuts performing user lookups on our AD (for hundreds of users that aren't even logging into the server at the time).
I'll ensure all my testing from now on is an actual ssh login and not the "groups" command.
I suspect the reason is that SSSD tries to be smart here and differentiates between initgroups operation performed via the NSS interface (id -G) and initgroups performed during auth via the PAM stack.
The initgroups that comes from the NSS can be returned from the cache for speed's sake. On the other hand, the PAM responder only keeps a very short-lived in-memory cache - see the pam_id_timeout option in sssd.conf, defaults to 5 seconds. Its purpose is mainly to only contact the server once even if the login program performs multiple initgroups calls during login - SSH is one example, it performs several initgroups operations during various stages of the PAM conversation and even from different processes.
The net effect of this is that by doing this, we're also doing a lookup of all the users in those groups (we don't have a choice in this, because RFC2307bis servers can have other groups as a member and we cannot know which we're dealing with until we request it).
Yes this is what I'm seeing. But is this also the case when I have disabled nesting?
# grep nesting /etc/sssd/sssd.conf ldap_group_nesting_level = 0
All our LDAP groups in the Unix OU have been flattened specifically to negate the need for nested searching and (hopefully) increase performance.
Still, the SSSD needs to check the DNs to see if the DN points to a user entry or not. In theory, the group members can be other objects than users.
*Maybe* we could do more heuristics if the group and user base DNs are specified and don't overlap, something similar to what's proposed in https://fedorahosted.org/sssd/ticket/1319 ?
Are you saying that no when we get a set of the DNs in the memberOf we do not know whether the entry is a user or a group so we have to fetch entries to see if the entry is a user or a group?
Yes, that's the case, especially with AD, as Users and groups can be stored in the same OU, and there is nothing in the DN that can tell you what is what.
IMO with there should be a way to regex it in SSSD and fetch only in cases when it is not possible to determine if the member is a user or a group. Use of the regex will help with other 2307 bis servers not just IPA. We need to think about AD too.
We can do it for IPA, as long as we force a scheme where we have something in the DN that makes the distinction clear (right now we have both the RDN and part of the DN to hint us, we just need to implement the code to read the smarts instead of blindly searching the LDAp Directory). Unfortunately nothing in AD allows you to do that. However, we should probably smarten up SSSD and make it check the *local* database first. We keep the originalDN attribute so we can search locally to find out if we already know what is the object.
We should also always use the MS-PAC for initgroups when using AD (if Kerberos auth is used), and not search the LDAP directory until some app really asks a for a getgrgid() op, IMO.
Simo.
On Sat, Jun 02, 2012 at 02:28:57PM -0400, Dmitri Pal wrote:
On 06/01/2012 03:11 AM, Jakub Hrozek wrote:
On Fri, Jun 01, 2012 at 12:53:36AM +0000, GOLLSCHEWSKY, Tim wrote:
Hi Jakub and Stephen,
Thanks for your responses.
Stephen wrote:
Why does it still enumerate so many users and groups (that are not me, and not in my ldap_access_filter) when I log in? Even when I have disabled domain enumeration?
As Jakub suggested, your investigation is slightly flawed. I'm guessing your version of "simulating an initgroups" is by running 'id username'. This is actually different from initgroups. What this does is an initgroups() call followed by a loop to look up every group that the user is a member of.
OK, thanks. I see now I've been using the wrong method to test this.
You guys have suggested "id -G" or actually logging in. "id -G" doesnt seem to have the same effect on SSSD that actually logging in has. "id -G" returns quite quickly, whereas a normal ssh login takes a very long time as SSSD goes nuts performing user lookups on our AD (for hundreds of users that aren't even logging into the server at the time).
I'll ensure all my testing from now on is an actual ssh login and not the "groups" command.
I suspect the reason is that SSSD tries to be smart here and differentiates between initgroups operation performed via the NSS interface (id -G) and initgroups performed during auth via the PAM stack.
The initgroups that comes from the NSS can be returned from the cache for speed's sake. On the other hand, the PAM responder only keeps a very short-lived in-memory cache - see the pam_id_timeout option in sssd.conf, defaults to 5 seconds. Its purpose is mainly to only contact the server once even if the login program performs multiple initgroups calls during login - SSH is one example, it performs several initgroups operations during various stages of the PAM conversation and even from different processes.
The net effect of this is that by doing this, we're also doing a lookup of all the users in those groups (we don't have a choice in this, because RFC2307bis servers can have other groups as a member and we cannot know which we're dealing with until we request it).
Yes this is what I'm seeing. But is this also the case when I have disabled nesting?
# grep nesting /etc/sssd/sssd.conf ldap_group_nesting_level = 0
All our LDAP groups in the Unix OU have been flattened specifically to negate the need for nested searching and (hopefully) increase performance.
Still, the SSSD needs to check the DNs to see if the DN points to a user entry or not. In theory, the group members can be other objects than users.
*Maybe* we could do more heuristics if the group and user base DNs are specified and don't overlap, something similar to what's proposed in https://fedorahosted.org/sssd/ticket/1319 ?
Are you saying that no when we get a set of the DNs in the memberOf we do not know whether the entry is a user or a group so we have to fetch entries to see if the entry is a user or a group? IMO with there should be a way to regex it in SSSD and fetch only in cases when it is not possible to determine if the member is a user or a group. Use of the regex will help with other 2307 bis servers not just IPA. We need to think about AD too.
That's only 100% doable with tightly-controlled environments such as IPA where you know for certain that a particular DN points to user or group containers.
In the generic case, the DN might simply look like "cn=name,ou=accounts,dc=example,dc=com" and then only a lookup can tell if it's a user or a group.
What I was telling Tim earlier was that we might want to do the LDAP-search based heuristics also in cases when the group and user containers are distinct, such as "ou=Users,dc=example,dc=com" and "ou=Groups,dc=example,dc=com".
In the generic case, the DN might simply look like "cn=name,ou=accounts,dc=example,dc=com" and then only a lookup can tell if it's a user or a group. What I was telling Tim earlier was that we might want to do the LDAP-search based heuristics also in cases when the group and user containers are distinct, > such as "ou=Users,dc=example,dc=com" and "ou=Groups,dc=example,dc=com".
Just to add to this, I think a lot of organisations with largeish AD directories would have their set up like this (Accounts and Groups in separate OUs). It's just too messy to do it otherwise.
It sounds like this happens by default with IPA, if there was some way we could enable this for AD as well (perhaps with a config option that specified the OUs for Accounts and Groups), this would save SSSD from doing the ldap searches just to distinguish if the record is a user or group, and would be a huge win for performance.
Cheers, Tim.
________________________________
This e-mail is sent by Suncorp Group Limited ABN 66 145 290 124 or one of its related entities "Suncorp". Suncorp may be contacted at Level 18, 36 Wickham Terrace, Brisbane or on 13 11 55 or at suncorp.com.au. The content of this e-mail is the view of the sender or stated author and does not necessarily reflect the view of Suncorp. The content, including attachments, is a confidential communication between Suncorp and the intended recipient. If you are not the intended recipient, any use, interference with, disclosure or copying of this e-mail, including attachments, is unauthorised and expressly prohibited. If you have received this e-mail in error please contact the sender immediately and delete the e-mail and any attachments from your system.
On Tue, 2012-06-05 at 03:21 +0000, GOLLSCHEWSKY, Tim wrote:
In the generic case, the DN might simply look like
"cn=name,ou=accounts,dc=example,dc=com" and then only a lookup can tell if it's a user or a group. > What I was telling Tim earlier was that we might want to do the LDAP-search based heuristics also in cases when the group and user containers are distinct, > such as "ou=Users,dc=example,dc=com" and "ou=Groups,dc=example,dc=com".
Just to add to this, I think a lot of organisations with largeish AD directories would have their set up like this (Accounts and Groups in separate OUs). It's just too messy to do it otherwise.
It sounds like this happens by default with IPA, if there was some way we could enable this for AD as well (perhaps with a config option that specified the OUs for Accounts and Groups), this would save SSSD from doing the ldap searches just to distinguish if the record is a user or group, and would be a huge win for performance.
Well, as Jakub suggested, one thing that might be possible is to examine the user and group search bases[*] to see if there is zero overlap. If this is the case, then we can optimize by checking the member DN against those bases. However, if there is any overlap (e.g. they are both set to the generic ldap_search_base) then we have to take the slower approach.
I've opened https://fedorahosted.org/sssd/ticket/1366 to track this enhancement.
[*] The catch here is that we support multiple search bases for users and groups, so this means we would need to ensure that there was no overlap among ANY of them.
The net effect of this is that by doing this, we're also doing a lookup of all the users in those groups (we don't have a choice in this, because RFC2307bis servers can have other groups as a member and we cannot know which we're dealing with until we request it).
Would the information in the link below provide any help for looking up all members of a group, and all a user group memberships in a single LDAP lookup?
I used the filter "(memberof:1.2.840.113556.1.4.1941:=(cn=Group1,OU=groupsOU,DC=x))" to find all users of a group, including users that we're member of that group through membership of another group some time back.
There seem to be a similar filter for looking up all groups the user is a member of in a single lookup: "(member:1.2.840.113556.1.4.1941:=(cn=user1,cn=users,DC=x))".
See this link for further reference: http://msdn.microsoft.com/en-us/library/windows/desktop/aa746475%28v=vs.85%2...
Rgds, Siggi
On Tue, 2012-06-05 at 16:51 +0200, Sigbjorn Lie wrote:
The net effect of this is that by doing this, we're also doing a lookup of all the users in those groups (we don't have a choice in this, because RFC2307bis servers can have other groups as a member and we cannot know which we're dealing with until we request it).
Would the information in the link below provide any help for looking up all members of a group, and all a user group memberships in a single LDAP lookup?
I used the filter "(memberof:1.2.840.113556.1.4.1941:=(cn=Group1,OU=groupsOU,DC=x))" to find all users of a group, including users that we're member of that group through membership of another group some time back.
There seem to be a similar filter for looking up all groups the user is a member of in a single lookup: "(member:1.2.840.113556.1.4.1941:=(cn=user1,cn=users,DC=x))".
See this link for further reference: http://msdn.microsoft.com/en-us/library/windows/desktop/aa746475%28v=vs.85%2...
Thanks a lot for uncovering this Siggi, it will definitely be useful.
Simo.
On Tue, 2012-06-05 at 11:14 -0400, Simo Sorce wrote:
On Tue, 2012-06-05 at 16:51 +0200, Sigbjorn Lie wrote:
The net effect of this is that by doing this, we're also doing a lookup of all the users in those groups (we don't have a choice in this, because RFC2307bis servers can have other groups as a member and we cannot know which we're dealing with until we request it).
Would the information in the link below provide any help for looking up all members of a group, and all a user group memberships in a single LDAP lookup?
I used the filter "(memberof:1.2.840.113556.1.4.1941:=(cn=Group1,OU=groupsOU,DC=x))" to find all users of a group, including users that we're member of that group through membership of another group some time back.
There seem to be a similar filter for looking up all groups the user is a member of in a single lookup: "(member:1.2.840.113556.1.4.1941:=(cn=user1,cn=users,DC=x))".
See this link for further reference: http://msdn.microsoft.com/en-us/library/windows/desktop/aa746475%28v=vs.85%2...
Thanks a lot for uncovering this Siggi, it will definitely be useful.
I agree, this will be very handy. I've opened https://fedorahosted.org/sssd/ticket/1367 to track its inclusion. I'll be proposing this for 1.9.0 beta 3, I think.
sssd-users@lists.fedorahosted.org
-
Dmitri Pal -
GOLLSCHEWSKY, Tim -
Jakub Hrozek -
Sigbjorn Lie -
Simo Sorce -
Stephen Gallagher