I have a storage appliance that needs local passwd/group files loaded onto it, which need to match the entries we get by using sssd's ldap_id_mapping feature. So I need some way to enumerate or synthesize passwd/group entries, for every user/group object in our domain, using LDIF dumps from AD that includes all users/groups, along with their respective objectSid attributes.
We know (from experience, and from discussion on this list) that enabling enumeration in sssd is problematic, so that's out.
I could just issue individual getpwnam()/getgrnam() calls for every user/group object, and let sssd synthesize the entries. But this would require careful tuning of sssd's cache configuration options to avoid significant delays, and even then, this would pound our AD domain controllers with thousands and thousands of lookup requests every time we regenerate the synthesized passwd/group files (which will probably be hourly).
From digging around in the sssd source code, I see that sssd has a SSS_NSS_GETIDBYSID API call that looks to be exactly what I need. But it's not clear to me whether that's a public or private API, and additionally, it looks like I'd be limited to C for my implementation, as I see no other language bindings for those functions.
Has anyone already rolled (Python, Ruby, Perl, et. al.) bindings for sssd's API calls, specifically the ID-SID mapping calls?
One potential option would be to just re-implement sssd's id mapping code in Python. I could "cheat" in our implementation, because I know that the only options that vary across our domains are (ldap_idmap_range_max, ldap_idmap_range_min, ldap_idmap_range_size). But re-implementation opens the door for a subtle error that would cause my mapping code to return different results from sssd in some corner cases, which I definitely don't want. So leveraging sssd's SSS_NSS_GETIDBYSID API call would be best… if that's possible.
Another option would be to bypass the API and talk directly to the NSS responder via its listening socket, which is easy enough to do in other languages. But this would require me to speak the protocol exactly the way sssd expects, and any API changes would break my code.
Thoughts? Suggestions?
On Tue, Sep 19, 2017 at 4:08 PM, James Ralston ralston@pobox.com wrote:
I could just issue individual getpwnam()/getgrnam() calls for every user/group object, and let sssd synthesize the entries. But this would require careful tuning of sssd's cache configuration options to avoid significant delays, and even then, this would pound our AD domain controllers with thousands and thousands of lookup requests every time we regenerate the synthesized passwd/group files (which will probably be hourly).
It just occurred to me that I can cheat.
For any given user or group, I know that sssd calculates the id from the SID by hashing the non-RID component of the SID to yield a base offset, and then adding the value of the RID. The hashing of the SID component to yield the base offset is the tricky part, because there are many configuration options that can influence how that happens.
But I can derive the base offset that sssd is using by issuing a getgrnam() call for "Domain Users", taking the gid, and subtracting the Domain Users's RID (which I can derive from the SID, but is always 513). And once I have the base offset that sssd is using, I don't need to issue any more getpwnam()/getgrnam() calls, because I can calculate the uid/gid for any object by adding the object's RID (which I can extract from the SID) to the base offset.
Can anyone see any problems or drawbacks to this approach?
On Tue, Sep 19, 2017 at 04:08:34PM -0400, James Ralston wrote:
I have a storage appliance that needs local passwd/group files loaded onto it, which need to match the entries we get by using sssd's ldap_id_mapping feature. So I need some way to enumerate or synthesize passwd/group entries, for every user/group object in our domain, using LDIF dumps from AD that includes all users/groups, along with their respective objectSid attributes.
We know (from experience, and from discussion on this list) that enabling enumeration in sssd is problematic, so that's out.
I could just issue individual getpwnam()/getgrnam() calls for every user/group object, and let sssd synthesize the entries. But this would require careful tuning of sssd's cache configuration options to avoid significant delays, and even then, this would pound our AD domain controllers with thousands and thousands of lookup requests every time we regenerate the synthesized passwd/group files (which will probably be hourly).
From digging around in the sssd source code, I see that sssd has a SSS_NSS_GETIDBYSID API call that looks to be exactly what I need. But it's not clear to me whether that's a public or private API, and additionally, it looks like I'd be limited to C for my implementation, as I see no other language bindings for those functions.
Has anyone already rolled (Python, Ruby, Perl, et. al.) bindings for sssd's API calls, specifically the ID-SID mapping calls?
$ python Python 2.7.12 (default, Sep 29 2016, 12:52:15) [GCC 6.2.1 20160916 (Red Hat 6.2.1-2)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import pysss_nss_idmap pysss_nss_idmap.getidbysid('S-1-5-21-3692237560-1981608775-3610128199-1104')
{'S-1-5-21-3692237560-1981608775-3610128199-1104': {'type': 3, 'id': 1367201104}}
Please see 'pydoc pysss_nss_idmap' for details. On Fedora/RHEL the bindings are in the python[23]-libsss_nss_idmap package.
HTH
bye, Sumit
One potential option would be to just re-implement sssd's id mapping code in Python. I could "cheat" in our implementation, because I know that the only options that vary across our domains are (ldap_idmap_range_max, ldap_idmap_range_min, ldap_idmap_range_size). But re-implementation opens the door for a subtle error that would cause my mapping code to return different results from sssd in some corner cases, which I definitely don't want. So leveraging sssd's SSS_NSS_GETIDBYSID API call would be best… if that's possible.
Another option would be to bypass the API and talk directly to the NSS responder via its listening socket, which is easy enough to do in other languages. But this would require me to speak the protocol exactly the way sssd expects, and any API changes would break my code.
Thoughts? Suggestions? _______________________________________________ sssd-users mailing list -- sssd-users@lists.fedorahosted.org To unsubscribe send an email to sssd-users-leave@lists.fedorahosted.org
On (20/09/17 09:34), Sumit Bose wrote:
On Tue, Sep 19, 2017 at 04:08:34PM -0400, James Ralston wrote:
I have a storage appliance that needs local passwd/group files loaded onto it, which need to match the entries we get by using sssd's ldap_id_mapping feature. So I need some way to enumerate or synthesize passwd/group entries, for every user/group object in our domain, using LDIF dumps from AD that includes all users/groups, along with their respective objectSid attributes.
We know (from experience, and from discussion on this list) that enabling enumeration in sssd is problematic, so that's out.
I could just issue individual getpwnam()/getgrnam() calls for every user/group object, and let sssd synthesize the entries. But this would require careful tuning of sssd's cache configuration options to avoid significant delays, and even then, this would pound our AD domain controllers with thousands and thousands of lookup requests every time we regenerate the synthesized passwd/group files (which will probably be hourly).
From digging around in the sssd source code, I see that sssd has a SSS_NSS_GETIDBYSID API call that looks to be exactly what I need. But it's not clear to me whether that's a public or private API, and additionally, it looks like I'd be limited to C for my implementation, as I see no other language bindings for those functions.
Has anyone already rolled (Python, Ruby, Perl, et. al.) bindings for sssd's API calls, specifically the ID-SID mapping calls?
$ python Python 2.7.12 (default, Sep 29 2016, 12:52:15) [GCC 6.2.1 20160916 (Red Hat 6.2.1-2)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import pysss_nss_idmap pysss_nss_idmap.getidbysid('S-1-5-21-3692237560-1981608775-3610128199-1104')
{'S-1-5-21-3692237560-1981608775-3610128199-1104': {'type': 3, 'id': 1367201104}}
Please see 'pydoc pysss_nss_idmap' for details. On Fedora/RHEL the bindings are in the python[23]-libsss_nss_idmap package.
+ more examples in integration test https://github.com/SSSD/sssd/pull/373/files#diff-b580c91e5e70802720510b5b1a3...
LS
On Wed, Sep 20, 2017 at 3:34 AM, Sumit Bose sbose@redhat.com wrote:
$ python Python 2.7.12 (default, Sep 29 2016, 12:52:15) [GCC 6.2.1 20160916 (Red Hat 6.2.1-2)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import pysss_nss_idmap pysss_nss_idmap.getidbysid('S-1-5-21-3692237560-1981608775-3610128199-1104')
{'S-1-5-21-3692237560-1981608775-3610128199-1104': {'type': 3, 'id': 1367201104}}
Please see 'pydoc pysss_nss_idmap' for details.
Ah; thank you! That's very useful.
On Fedora/RHEL the bindings are in the python[23]-libsss_nss_idmap package.
Alas, RHEL7 only has the python2 bindings, because the only officially supported Python in RHEL7 is 2.7. (Well, there's the python33 package from the RHSCL, but the RHSCL is enormous pain to use, let alone package for.)
On Wed, Sep 20, 2017 at 8:25 AM, Lukas Slebodnik lslebodn@redhat.com wrote:
- more examples in integration test
https://github.com/SSSD/sssd/pull/373/files#diff-b580c91e5e70802720510b5b1a3...
Also useful; thanks.
Unfortunately, all of these functions appear to cause sssd to query the backend data provider (Active Directory, in this case) unless the object being queried is already in the cache. This is true even for the functions that could simply calculate the answer from the question (e.g., getidbysid()). As such, using these functions is equivalent to calling getpwnam()/getgrnam in terms of the amount of load they will place on the backend.
I think for the time being, I'm going to go with my "cheat" approach, which is to perform a lookup, calculate the offset, and then use the offset+RID to synthesize the results sssd would generate. It's not perfect, but it will avoid crushing our AD infrastructure with a complete user/group enumeration every hour.
Thanks, James
On (21/09/17 18:33), James Ralston wrote:
On Wed, Sep 20, 2017 at 3:34 AM, Sumit Bose sbose@redhat.com wrote:
$ python Python 2.7.12 (default, Sep 29 2016, 12:52:15) [GCC 6.2.1 20160916 (Red Hat 6.2.1-2)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import pysss_nss_idmap pysss_nss_idmap.getidbysid('S-1-5-21-3692237560-1981608775-3610128199-1104')
{'S-1-5-21-3692237560-1981608775-3610128199-1104': {'type': 3, 'id': 1367201104}}
Please see 'pydoc pysss_nss_idmap' for details.
Ah; thank you! That's very useful.
On Fedora/RHEL the bindings are in the python[23]-libsss_nss_idmap package.
Alas, RHEL7 only has the python2 bindings, because the only officially supported Python in RHEL7 is 2.7. (Well, there's the python33 package from the RHSCL, but the RHSCL is enormous pain to use, let alone package for.)
On Wed, Sep 20, 2017 at 8:25 AM, Lukas Slebodnik lslebodn@redhat.com wrote:
- more examples in integration test
https://github.com/SSSD/sssd/pull/373/files#diff-b580c91e5e70802720510b5b1a3...
Also useful; thanks.
Unfortunately, all of these functions appear to cause sssd to query the backend data provider (Active Directory, in this case) unless the object being queried is already in the cache. This is true even for the functions that could simply calculate the answer from the question (e.g., getidbysid()). As such, using these functions is equivalent to calling getpwnam()/getgrnam in terms of the amount of load they will place on the backend.
Yes, because libsss_nss_idmap is extension of "nsswitch" interface
I think for the time being, I'm going to go with my "cheat" approach, which is to perform a lookup, calculate the offset, and then use the offset+RID to synthesize the results sssd would generate. It's not perfect, but it will avoid crushing our AD infrastructure with a complete user/group enumeration every hour.
You might check libsss_idmap.so. It does not do any communication. And it is used internally by sssd https://pagure.io/SSSD/sssd/blob/master/f/src/lib/idmap/sss_idmap.h
sh$ rpm -q --whatrequires "libsss_idmap.so.0()(64bit)" sssd-client-1.15.3-4.fc27.x86_64 sssd-common-1.15.3-4.fc27.x86_64 sssd-common-pac-1.15.3-4.fc27.x86_64 sssd-ipa-1.15.3-4.fc27.x86_64 sssd-ad-1.15.3-4.fc27.x86_64 sssd-ldap-1.15.3-4.fc27.x86_64
You might use either libsss_nss_idmap.so or libsss_idmap.so libsss_idmap.so it depends on your use-case. libsss_idmap.so does not have python bindings.
LS
On Fri, Sep 22, 2017 at 7:40 AM, Lukas Slebodnik lslebodn@redhat.com wrote:
You might check libsss_idmap.so. It does not do any communication. And it is used internally by sssd https://pagure.io/SSSD/sssd/blob/master/f/src/lib/idmap/sss_idmap.h
sh$ rpm -q --whatrequires "libsss_idmap.so.0()(64bit)" sssd-client-1.15.3-4.fc27.x86_64 sssd-common-1.15.3-4.fc27.x86_64 sssd-common-pac-1.15.3-4.fc27.x86_64 sssd-ipa-1.15.3-4.fc27.x86_64 sssd-ad-1.15.3-4.fc27.x86_64 sssd-ldap-1.15.3-4.fc27.x86_64
You might use either libsss_nss_idmap.so or libsss_idmap.so libsss_idmap.so it depends on your use-case. libsss_idmap.so does not have python bindings.
Thanks for the pointer.
Right now, I need something I can use with python, but I may go back and try to roll python bindings for one or both of these once our immediate need is met.
James
sssd-users@lists.fedorahosted.org
-
James Ralston -
Lukas Slebodnik -
Sumit Bose