Hello Spike. Sorry that I cannot answer your question directly. I would be interested though: Which search query did you use and which privilege does the account need to use tokengroups? I am also trying to use tokengroups, and may have the same problem. ________________________________ From: Spike White <spikewhitetx(a)gmail.com&gt; Sent: 01 July 2018 22:25:26 To: End-user discussions about the System Security Services Daemon Subject: [SSSD-users] Why is my sssd deployment not doing cross-subdomain AD authentication? sssd subject matter experts, Why is my sssd deployment not doing cross-subdomain AD authentication? Background: I have a parent AD domain DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=http%3A%... with trusted subdomains AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt...;, APAC.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt...;, EMEA.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt... and JAPN.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt... Each subdomain has a transitive trust with DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=http%3A%...;. So all subdomains trust each other. I set up a first test VM deployment using sssd. I set up the cross subdomain auth as in: https://lists.fedorahosted.org/pipermail/sssd-users/2015-February/002648.... It worked great – allowed cross subdomain authentication. The only thing it would not do was use tokengroups. That is, the VM was fully functional, but I had to add ‘ldap_use_tokengroups = false’ to the sssd.conf file. My AD experts have advised me that ‘tokengroups’ are an important AD optimization and I should use them, if at all possible. Using ldapsearch, I was able to verify that machine account didn’t have the necessary privileges to query a user’s tokengroups. Thus, the fault was mine – that this first sssd deployment couldn’t use tokengroups. So I did another sssd deployment, using another test VM. Apparently, I did the realm join command correct this time, as it’s able to use tokengroups. BUT! This second test VM is not allowing cross subdomain authentication and login. How do I fix this so that I have use of both tokengroups and cross subdomain authentication? (BTW -- Both test VMs are still up and operational, as described above.) Details: Here is the realm join command used in the second test VM (spikerealmd02): kinit serviceunixinstall realm join -v --automatic-id-mapping=no --computer-ou='OU=Servers,OU=UNIX,DC=AMER,DC=DELL,DC=COM' --user-principal="host/`hostname --fqdn`@AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2FAMER.DELL.COM&data=01%7C01%7Cjohe%40novozymes.com%7Cebda1b26b8cf4378310108d5df90da3b%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=DBeeCJIp7zh0a5l8YzMMjAU2WWA2kavAB1sZ3RtrP9c%3D&reserved=0>" AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt... Here is the /etc/realmd.conf file from this second test VM: [root@spikerealmd02 etc]# cat realmd.conf [AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt...] computer-ou = OU=SERVERS,OU=UNIX,DC=AMER,DC=DELL,DC=COM automatic-id-mapping = no manage-system = no fully-qualified-names = no # THIS FAILS AT DELL; serviceunixinstall apparently not allowed to create UPNs associated with machine account. # Set the user-prinicpal to yes to create userPrincipalName attributes for the computer account in the realm, in the form host/computer@REALM #user-principal = yes [active-directory] default_client = sssd [service] automatic-install = no [users] # shouldn't need this; should be set in AD for each UNIX-enabled user. default-home = /home/%U # shouldn't need this; should be set in AD for each UNIX-enabled user. default-shell = /bin/bash Here’s the /etc/sssd/sssd.conf file for this second test VM: [root@spikerealmd02 sssd]# cat sssd.conf [sssd] debug_level = 6 domains = amer.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt... domain_resolution_order = amer.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, emea.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, apac.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, japn.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt... config_file_version = 2 services = nss,pam #ldap_user_member_of = member [pam] pam_verbosity = 3 debug_level = 9 [nss] debug_level = 9 filter_groups = root filter_users = root reconnection_retries = 3 #entry_cache_timeout = 300 entry_cache_nowait_percentage = 75 [domain/amer.dell.com<https://emea01.safelinks.protection.outlook.com/?...] debug_level = 9 auto_private_groups = True use_fully_qualified_names = False ad_domain = amer.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt... krb5_realm = AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt... realmd_tags = joined-with-adcli cache_credentials = True id_provider = ad auth_provider = ad krb5_store_password_if_offline = True default_shell = /bin/bash ldap_id_mapping = False fallback_homedir = /home/%u access_provider = simple #access_provider = ad ldap_schema = rfc2307bis ldap_sasl_authid = host/spikerealmd02.us.dell.com@AMER.DELL.COM<mailto:spikerealmd02.us.dell.com@AMER.DELL.COM> #ldap_sasl_authid = SPIKEREALMD02$@AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2FAMER.DELL.COM&data=01%7C01%7Cjohe%40novozymes.com%7Cebda1b26b8cf4378310108d5df90da3b%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=DBeeCJIp7zh0a5l8YzMMjAU2WWA2kavAB1sZ3RtrP9c%3D&reserved=0> #ldap_sasl_authid = spikerealmd02@AMER.DELL.COM<mailto:spikerealmd02@AMER.DELL.COM> ad_enabled_domains = amer.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt... dyndns_update = False subdomains_provider = none ldap_use_tokengroups = true simple_allow_groups = amerlinuxsup@AMER.DELL.COM<mailto:amerlinuxsup@AMER.DELL.COM>, amerlinuxeng@AMER.DELL.COM<mailto:amerlinuxeng@AMER.DELL.COM>, emealinuxsup@EMEA.DELL.COM<mailto:emealinuxsup@EMEA.DELL.COM>, AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt...;, emealinuxeng@EMEA.DELL.COM<mailto:emealinuxeng@EMEA.DELL.COM>, apaclinuxsup@EMEA.DELL.COM<mailto:apaclinuxsup@EMEA.DELL.COM>, apaclinuxeng@EMEA.DELL.COM<mailto:apaclinuxeng@EMEA.DELL.COM> # also look at https://lists.fedorahosted.org/pipermail/sssd-users/2015-February/002648.... [domain/apac.dell.com<https://emea01.safelinks.protection.outlook.com/?...] debug_level = 9 auto_private_groups = True use_fully_qualified_names = False ad_domain = apac.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt... krb5_realm = APAC.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt... cache_credentials = True id_provider = ad auth_provider = ad krb5_store_password_if_offline = True default_shell = /bin/bash ldap_id_mapping = False fallback_homedir = /home/%u access_provider = simple ldap_schema = rfc2307bis ldap_sasl_authid = host/spikerealmd02.us.dell.com@AMER.DELL.COM<mailto:spikerealmd02.us.dell.com@AMER.DELL.COM> #ldap_sasl_authid = SPIKEREALMD02$@AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2FAMER.DELL.COM&data=01%7C01%7Cjohe%40novozymes.com%7Cebda1b26b8cf4378310108d5df90da3b%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=DBeeCJIp7zh0a5l8YzMMjAU2WWA2kavAB1sZ3RtrP9c%3D&reserved=0> #ldap_sasl_authid = spikerealmd02@AMER.DELL.COM<mailto:spikerealmd02@AMER.DELL.COM> ad_enabled_domains = amer.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, apac.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, apac.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, japn.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, dell.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%... dyndns_update = False subdomains_provider = none ldap_use_tokengroups = false simple_allow_groups = apaclinuxsup@APAC.DELL.COM<mailto:apaclinuxsup@APAC.DELL.COM>, apaclinuxeng@APAC.DELL.COM<mailto:apaclinuxeng@APAC.DELL.COM> [domain/emea.dell.com<https://emea01.safelinks.protection.outlook.com/?...] debug_level = 9 auto_private_groups = True use_fully_qualified_names = False ad_domain = emea.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt... krb5_realm = EMEA.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt... cache_credentials = True id_provider = ad auth_provider = ad krb5_store_password_if_offline = True default_shell = /bin/bash ldap_id_mapping = False fallback_homedir = /home/%u access_provider = simple ldap_schema = rfc2307bis ldap_sasl_authid = host/spikerealmd02.us.dell.com@AMER.DELL.COM<mailto:spikerealmd02.us.dell.com@AMER.DELL.COM> #ldap_sasl_authid = SPIKEREALMD02$@AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2FAMER.DELL.COM&data=01%7C01%7Cjohe%40novozymes.com%7Cebda1b26b8cf4378310108d5df90da3b%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=DBeeCJIp7zh0a5l8YzMMjAU2WWA2kavAB1sZ3RtrP9c%3D&reserved=0> #ldap_sasl_authid = spikerealmd02@AMER.DELL.COM<mailto:spikerealmd02@AMER.DELL.COM> ad_enabled_domains = amer.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, apac.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, emea.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, japn.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, dell.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%... dyndns_update = False subdomains_provider = none ldap_use_tokengroups = true simple_allow_groups = emealinuxsup@EMEA.DELL.COM<mailto:emealinuxsup@EMEA.DELL.COM>, emealinuxeng@EMEA.DELL.COM<mailto:emealinuxeng@EMEA.DELL.COM> [domain/japn.dell.com<https://emea01.safelinks.protection.outlook.com/?...] debug_level = 9 auto_private_groups = True use_fully_qualified_names = False ad_domain = japn.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt... krb5_realm = JAPN.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt... cache_credentials = True id_provider = ad auth_provider = ad krb5_store_password_if_offline = True default_shell = /bin/bash ldap_id_mapping = False fallback_homedir = /home/%u access_provider = simple ldap_schema = rfc2307bis ldap_sasl_authid = host/spikerealmd02.us.dell.com@AMER.DELL.COM<mailto:spikerealmd02.us.dell.com@AMER.DELL.COM> #ldap_sasl_authid = SPIKEREALMD02$@AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2FAMER.DELL.COM&data=01%7C01%7Cjohe%40novozymes.com%7Cebda1b26b8cf4378310108d5df90da3b%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=DBeeCJIp7zh0a5l8YzMMjAU2WWA2kavAB1sZ3RtrP9c%3D&reserved=0> #ldap_sasl_authid = spikerealmd02@AMER.DELL.COM<mailto:spikerealmd02@AMER.DELL.COM> ad_enabled_domains = amer.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, apac.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, japn.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, japn.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, dell.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%... dyndns_update = False subdomains_provider = none ldap_use_tokengroups = true simple_allow_groups = japnlinuxsup@JAPN.DELL.COM<mailto:japnlinuxsup@JAPN.DELL.COM>, japnlinuxeng@JAPN.DELL.COM<mailto:japnlinuxeng@JAPN.DELL.COM> and here’s the /etc/krb5.conf file: # Configuration snippets may be placed in this directory as well includedir /etc/krb5.conf.d/ includedir /var/lib/sss/pubconf/krb5.include.d/ [logging] default = FILE:/var/log/krb5libs.log kdc = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmind.log [libdefaults] # SW mod 5/12/2018 # dns_lookup_realm = false dns_lookup_realm = false ticket_lifetime = 24h renew_lifetime = 7d forwardable = true rdns = false # default_realm = EXAMPLE.COM<https://emea01.safelinks.protection.outlook.com/?url=http%... default_ccache_name = KEYRING:persistent:%{uid} default_realm = AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt... [realms] # EXAMPLE.COM<https://emea01.safelinks.protection.outlook.com/?url=http%... = { # kdc = kerberos.example.com<https://emea01.safelinks.protection.outlook.com/?... # admin_server = kerberos.example.com<https://emea01.safelinks.protection.outlook.com/?... # } # AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt... = { # } [domain_realm] # .example.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.com&data=01%7C01%7Cjohe%40novozymes.com%7Cebda1b26b8cf4378310108d5df90da3b%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=OpTGAjaRheIVNjj6q9B%2F25Vx66A%2FUuDse6N%2BDTA2xvw%3D&reserved=0> = EXAMPLE.COM<https://emea01.safelinks.protection.outlook.com/?url=http%... # example.com<https://emea01.safelinks.protection.outlook.com/?url=http%... = EXAMPLE.COM<https://emea01.safelinks.protection.outlook.com/?url=http%... amer.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt... = AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt... .amer.dell.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Famer.dell.com&data=01%7C01%7Cjohe%40novozymes.com%7Cebda1b26b8cf4378310108d5df90da3b%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=CNz3WvJmaG9Nu4O3CY4fakRChgeE4dMIAErMolrchN0%3D&reserved=0> = AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt... Comparing with first VM that does cross subdomain auth: Here’s /etc/realmd.conf of first test VM that does cross subdomain auth (spikerealmd01): [root@spikerealmd01 krb5.include.d]# cat /etc/realmd.conf [AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt...] computer-ou = OU=SERVERS,OU=UNIX,DC=AMER,DC=DELL,DC=COM automatic-id-mapping = no manage-system = no fully-qualified-names = no # THIS FAILS AT DELL; serviceunixinstall apparently not allowed to create UPNs associated with machine account. # Set the user-prinicpal to yes to create userPrincipalName attributes for the computer account in the realm, in the form host/computer@REALM #user-principal = yes [active-directory] default_client = sssd [service] automatic-install = no [users] # shouldn't need this; should be set in AD for each UNIX-enabled user. default-home = /home/%U # shouldn't need this; should be set in AD for each UNIX-enabled user. default-shell = /bin/bash Here’s /etc/sssd/sssd.conf file from same first test VM: [root@spikerealmd01 sssd]# cat sssd.conf [sssd] debug_level = 6 domains = amer.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, apac.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, emea.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, japn.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt... domain_resolution_order = amer.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, emea.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, apac.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, japn.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt... config_file_version = 2 services = nss, pam #ldap_user_member_of = member [pam] pam_verbosity = 3 debug_level = 9 [nss] debug_level = 9 filter_groups = root filter_users = root reconnection_retries = 3 #entry_cache_timeout = 300 entry_cache_nowait_percentage = 75 [domain/amer.dell.com<https://emea01.safelinks.protection.outlook.com/?...] debug_level = 9 auto_private_groups = True use_fully_qualified_names = False ad_domain = amer.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt... krb5_realm = AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt... realmd_tags = joined-with-adcli cache_credentials = True id_provider = ad auth_provider = ad krb5_store_password_if_offline = True default_shell = /bin/bash ldap_id_mapping = False fallback_homedir = /home/%u access_provider = simple ldap_schema = rfc2307bis #ldap_sasl_authid = host/spikerealmd01.us.dell.com<https://emea01.safelinks.protection.out... #ldap_sasl_authid = SPIKEREALMD01$@AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2FAMER.DELL.COM&data=01%7C01%7Cjohe%40novozymes.com%7Cebda1b26b8cf4378310108d5df90da3b%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=DBeeCJIp7zh0a5l8YzMMjAU2WWA2kavAB1sZ3RtrP9c%3D&reserved=0> ldap_sasl_authid = spikerealmd01@AMER.DELL.COM<mailto:spikerealmd01@AMER.DELL.COM> ad_enabled_domains = amer.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, apac.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, emea.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, japn.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, dell.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%... dyndns_update = False subdomains_provider = none ldap_use_tokengroups = false simple_allow_groups = amerlinuxsup@AMER.DELL.COM<mailto:amerlinuxsup@AMER.DELL.COM>, amerlinuxeng@AMER.DELL.COM<mailto:amerlinuxeng@AMER.DELL.COM> # also look at https://lists.fedorahosted.org/pipermail/sssd-users/2015-February/002648.... [domain/apac.dell.com<https://emea01.safelinks.protection.outlook.com/?...] debug_level = 9 auto_private_groups = True use_fully_qualified_names = False ad_domain = apac.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt... krb5_realm = APAC.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt... realmd_tags = joined-with-adcli cache_credentials = True id_provider = ad auth_provider = ad krb5_store_password_if_offline = True default_shell = /bin/bash ldap_id_mapping = False fallback_homedir = /home/%u access_provider = simple ldap_schema = rfc2307bis #ldap_sasl_authid = host/spikerealmd01.us.dell.com<https://emea01.safelinks.protection.out... #ldap_sasl_authid = SPIKEREALMD01$@AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2FAMER.DELL.COM&data=01%7C01%7Cjohe%40novozymes.com%7Cebda1b26b8cf4378310108d5df90da3b%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=DBeeCJIp7zh0a5l8YzMMjAU2WWA2kavAB1sZ3RtrP9c%3D&reserved=0> ldap_sasl_authid = spikerealmd01@AMER.DELL.COM<mailto:spikerealmd01@AMER.DELL.COM> ad_enabled_domains = amer.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, apac.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, emea.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, japn.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, dell.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%... dyndns_update = False subdomains_provider = none ldap_use_tokengroups = false simple_allow_groups = apaclinuxsup@APAC.DELL.COM<mailto:apaclinuxsup@APAC.DELL.COM>, apaclinuxeng@APAC.DELL.COM<mailto:apaclinuxeng@APAC.DELL.COM> [domain/emea.dell.com<https://emea01.safelinks.protection.outlook.com/?...] debug_level = 9 auto_private_groups = True use_fully_qualified_names = False ad_domain = emea.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt... krb5_realm = EMEA.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt... realmd_tags = joined-with-adcli cache_credentials = True id_provider = ad auth_provider = ad krb5_store_password_if_offline = True default_shell = /bin/bash ldap_id_mapping = False fallback_homedir = /home/%u access_provider = simple ldap_schema = rfc2307bis #ldap_sasl_authid = host/spikerealmd01.us.dell.com<https://emea01.safelinks.protection.out... #ldap_sasl_authid = SPIKEREALMD01$@AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2FAMER.DELL.COM&data=01%7C01%7Cjohe%40novozymes.com%7Cebda1b26b8cf4378310108d5df90da3b%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=DBeeCJIp7zh0a5l8YzMMjAU2WWA2kavAB1sZ3RtrP9c%3D&reserved=0> ldap_sasl_authid = spikerealmd01@AMER.DELL.COM<mailto:spikerealmd01@AMER.DELL.COM> ad_enabled_domains = amer.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, apac.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, emea.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, japn.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, dell.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%... dyndns_update = False subdomains_provider = none ldap_use_tokengroups = false simple_allow_groups = emealinuxsup@EMEA.DELL.COM<mailto:emealinuxsup@EMEA.DELL.COM>, emealinuxeng@EMEA.DELL.COM<mailto:emealinuxeng@EMEA.DELL.COM> [domain/japn.dell.com<https://emea01.safelinks.protection.outlook.com/?...] debug_level = 9 auto_private_groups = True use_fully_qualified_names = False ad_domain = japn.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt... krb5_realm = JAPN.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt... realmd_tags = joined-with-adcli cache_credentials = True id_provider = ad auth_provider = ad krb5_store_password_if_offline = True default_shell = /bin/bash ldap_id_mapping = False fallback_homedir = /home/%u access_provider = simple ldap_schema = rfc2307bis #ldap_sasl_authid = host/spikerealmd01.us.dell.com<https://emea01.safelinks.protection.out... #ldap_sasl_authid = SPIKEREALMD01$@AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2FAMER.DELL.COM&data=01%7C01%7Cjohe%40novozymes.com%7Cebda1b26b8cf4378310108d5df90da3b%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=DBeeCJIp7zh0a5l8YzMMjAU2WWA2kavAB1sZ3RtrP9c%3D&reserved=0> ldap_sasl_authid = spikerealmd01@AMER.DELL.COM<mailto:spikerealmd01@AMER.DELL.COM> ad_enabled_domains = amer.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, apac.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, emea.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, japn.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt...;, dell.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%... dyndns_update = False subdomains_provider = none ldap_use_tokengroups = false simple_allow_groups = japnlinuxsup@JAPN.DELL.COM<mailto:japnlinuxsup@JAPN.DELL.COM>, japnlinuxeng@JAPN.DELL.COM<mailto:japnlinuxeng@JAPN.DELL.COM>, linux-core-engineering, amer.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt... Here’s /etc/krb5.conf file: [root@spikerealmd01 etc]# cat krb5.conf # Configuration snippets may be placed in this directory as well includedir /etc/krb5.conf.d/ includedir /var/lib/sss/pubconf/krb5.include.d/ [logging] default = FILE:/var/log/krb5libs.log kdc = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmind.log [libdefaults] # SW mod 5/12/2018 # dns_lookup_realm = false dns_lookup_realm = false ticket_lifetime = 24h renew_lifetime = 7d forwardable = true rdns = false # default_realm = EXAMPLE.COM<https://emea01.safelinks.protection.outlook.com/?url=http%... default_ccache_name = KEYRING:persistent:%{uid} default_realm = AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt... [realms] # EXAMPLE.COM<https://emea01.safelinks.protection.outlook.com/?url=http%... = { # kdc = kerberos.example.com<https://emea01.safelinks.protection.outlook.com/?... # admin_server = kerberos.example.com<https://emea01.safelinks.protection.outlook.com/?... # } # AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt... = { # } [domain_realm] # .example.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.com&data=01%7C01%7Cjohe%40novozymes.com%7Cebda1b26b8cf4378310108d5df90da3b%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=OpTGAjaRheIVNjj6q9B%2F25Vx66A%2FUuDse6N%2BDTA2xvw%3D&reserved=0> = EXAMPLE.COM<https://emea01.safelinks.protection.outlook.com/?url=http%... # example.com<https://emea01.safelinks.protection.outlook.com/?url=http%... = EXAMPLE.COM<https://emea01.safelinks.protection.outlook.com/?url=http%... amer.dell.com<https://emea01.safelinks.protection.outlook.com/?url=htt... = AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt... .amer.dell.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Famer.dell.com&data=01%7C01%7Cjohe%40novozymes.com%7Cebda1b26b8cf4378310108d5df90da3b%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=CNz3WvJmaG9Nu4O3CY4fakRChgeE4dMIAErMolrchN0%3D&reserved=0> = AMER.DELL.COM<https://emea01.safelinks.protection.outlook.com/?url=htt... [root@spikerealmd01 etc]# Other details: If I query group membership of an engineer in APAC: id admjesse_chan on the good VM (spikerealmd01) I see all expected groups and I see this in the /var/log/sssd/sssd_apac.dell.com.log file: … (Sun Jul 1 15:14:30 2018) [sssd[be[apac.dell.com<https://emea01.safelinks.protection.outlook.com/...] [sdap_initgr_rfc2307bis_next_base] (0x0400): Searching for parent groups for user [CN=AdmJesse_Chan,OU=ADMAccounts,DC=apac,DC=dell,DC=com] with base [DC=apac,DC=dell,DC=com] (Sun Jul 1 15:14:30 2018) [sssd[be[apac.dell.com<https://emea01.safelinks.protection.outlook.com/...] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(&(member=CN=AdmJesse_Chan,OU=ADMAccounts,DC=apac,DC=dell,DC=com)(objectClass=group)(sAMAccountName=*))][DC=apac,DC=dell,DC=com]. (Sun Jul 1 15:14:31 2018) [sssd[be[apac.dell.com<https://emea01.safelinks.protection.outlook.com/...] [sdap_initgr_rfc2307bis_process] (0x1000): Found 4 parent groups for user [AdmJesse_Chan@apac.dell.com<mailto:AdmJesse_Chan@apac.dell.com>] (Sun Jul 1 15:14:31 2018) [sssd[be[apac.dell.com<https://emea01.safelinks.protection.outlook.com/...] [sysdb_get_direct_parents] (0x2000): searching sysdb with filter [(&(objectCategory=group)(member=name=AdmJesse_Chan@apac.dell.com<mailto:AdmJesse_Chan@apac.dell.com>,cn=users,cn=apac.dell.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapac.dell.com&data=01%7C01%7Cjohe%40novozymes.com%7Cebda1b26b8cf4378310108d5df90da3b%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=4KAxxKaXc5lCpbwtzakV%2BJCVd%2Bej6MmS5RT0oWoh%2FZo%3D&reserved=0>,cn=sysdb))] (Sun Jul 1 15:14:31 2018) [sssd[be[apac.dell.com<https://emea01.safelinks.protection.outlook.com/...] [sysdb_get_direct_parents] (0x1000): AdmJesse_Chan@apac.dell.com<mailto:AdmJesse_Chan@apac.dell.com> is a member of 4 sysdb groups (Sun Jul 1 15:14:31 2018) [sssd[be[apac.dell.com<https://emea01.safelinks.protection.outlook.com/...] [save_rfc2307bis_user_memberships] (0x2000): Updating memberships for AdmJesse_Chan@apac.dell.com<mailto:AdmJesse_Chan@apac.dell.com> (Sun Jul 1 15:14:31 2018) [sssd[be[apac.dell.com<https://emea01.safelinks.protection.outlook.com/...] [sysdb_set_entry_attr] (0x0200): Entry [name=AdmJesse_Chan@apac.dell.com<mailto:AdmJesse_Chan@apac.dell.com>,cn=users,cn=apac.dell.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapac.dell.com&data=01%7C01%7Cjohe%40novozymes.com%7Cebda1b26b8cf4378310108d5df90da3b%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=4KAxxKaXc5lCpbwtzakV%2BJCVd%2Bej6MmS5RT0oWoh%2FZo%3D&reserved=0>,cn=sysdb] has set [ts_cache] attrs. (Sun Jul 1 15:14:31 2018) [sssd[be[apac.dell.com<https://emea01.safelinks.protection.outlook.com/...] [dp_table_value_destructor] (0x0400): Removing [0:1:0x0001:3::apac.dell.com:name=admjesse_chan@apac.dell.com<mailto:admjesse_chan@apac.dell.com>] from reply table (Sun Jul 1 15:14:35 2018) [sssd[be[apac.dell.com<https://emea01.safelinks.protection.outlook.com/...] [sdap_asq_search_parse_entry] (0x2000): Matched objectclass [user] on DN [CN=AdmJesse_Chan,OU=ADMAccounts,DC=apac,DC=dell,DC=com], will use associated map (Sun Jul 1 15:14:35 2018) [sssd[be[apac.dell.com<https://emea01.safelinks.protection.outlook.com/...] [sdap_parse_entry] (0x1000): OriginalDN: [CN=AdmJesse_Chan,OU=ADMAccounts,DC=apac,DC=dell,DC=com]. (Sun Jul 1 15:14:35 2018) [sssd[be[apac.dell.com<https://emea01.safelinks.protection.outlook.com/...] [sdap_asq_search_parse_entry] (0x2000): DN [CN=AdmJesse_Chan,OU=ADMAccounts,DC=apac,DC=dell,DC=com] did not match the objectClass [group] (Sun Jul 1 15:14:36 2018) [sssd[be[apac.dell.com<https://emea01.safelinks.protection.outlook.com/...] [sdap_nested_group_hash_insert] (0x4000): Inserting [CN=AdmJesse_Chan,OU=ADMAccounts,DC=apac,DC=dell,DC=com] into hash table [users] (Sun Jul 1 15:14:37 2018) [sssd[be[apac.dell.com<https://emea01.safelinks.protection.outlook.com/...] [sdap_get_primary_name] (0x0400): Processing object AdmJesse_Chan (Sun Jul 1 15:14:37 2018) [sssd[be[apac.dell.com<https://emea01.safelinks.protection.outlook.com/...] [sysdb_cache_search_users] (0x2000): Search users with filter: (&(objectCategory=user)(originalDN=CN=AdmJesse_Chan,OU=ADMAccounts,DC=apac,DC=dell,DC=com)) (Sun Jul 1 15:14:37 2018) [sssd[be[apac.dell.com<https://emea01.safelinks.protection.outlook.com/...] [sdap_find_entry_by_origDN] (0x4000): Searching cache for [CN=AdmJesse_Chan,OU=ADMAccounts,DC=apac,DC=dell,DC=com]. (Sun Jul 1 15:14:37 2018) [sssd[be[apac.dell.com<https://emea01.safelinks.protection.outlook.com/...] [sdap_fill_memberships] (0x1000): member #2019 (CN=AdmJesse_Cha ,OU=ADMAccounts,DC=apac,DC=dell,DC=com): [name=AdmJesse_Chan@apac.dell.com<mailto:AdmJesse_Chan@apac.dell.com>,cn=users,cn=apac.dell.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapac.dell.com&data=01%7C01%7Cjohe%40novozymes.com%7Cebda1b26b8cf4378310108d5df90da3b%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=4KAxxKaXc5lCpbwtzakV%2BJCVd%2Bej6MmS5RT0oWoh%2FZo%3D&reserved=0>,cn=sysdb] If I do the same query on the bad VM (spikerealmd02): [root@spikerealmd02 sssd]# id admjesse_chan id: admjesse_chan: no such user and I see nothing in the sssd_apac.dell.com.log file: [root@spikerealmd02 sssd]# grep -i admjesse_chan sssd_apac.dell.com.log [root@spikerealmd02 sssd]# Please help, Spike

Thanks for prompt reply. Yes, user name is strewn throughout sssd_nss.log. Here's the last little bit: (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_ncache] (0x0400): CR #1653: Checking negative cache for [admjesse_chan(a)japn.dell.com] (Sun Jul 1 15:21:12 2018) [sssd[nss]] [sss_ncache_check_str] (0x2000): Checking negative cache for [NCE/USER/ japn.dell.com/admjesse_chan@japn.dell.com] (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_ncache] (0x0400): CR #1653: [admjesse_chan(a)japn.dell.com] is not present in negative cache (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_cache] (0x0400): CR #1653: Looking up [admjesse_chan(a)japn.dell.com] in cache (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_cache] (0x0400): CR #1653: Object [admjesse_chan(a)japn.dell.com] was not found in cache (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_dp] (0x0400): CR #1653: Looking up [admjesse_chan(a)japn.dell.com] in data provider (Sun Jul 1 15:21:12 2018) [sssd[nss]] [sss_dp_issue_request] (0x0400): Issuing request for [0x55f842febb50:1:admjesse_chan@japn.dell.com@ japn.dell.com] (Sun Jul 1 15:21:12 2018) [sssd[nss]] [sss_dp_get_account_msg] (0x0400): Creating request for [japn.dell.com ][0x1][BE_REQ_USER][name=admjesse_chan@japn.dell.com:-] (Sun Jul 1 15:21:12 2018) [sssd[nss]] [sss_dp_internal_get_send] (0x0400): Entering request [0x55f842febb50:1:admjesse_chan@japn.dell.com@japn.dell.com ] (Sun Jul 1 15:21:12 2018) [sssd[nss]] [sss_dp_req_destructor] (0x0400): Deleting request: [0x55f842febb50:1:admjesse_chan@apac.dell.com@ apac.dell.com] (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_cache] (0x0400): CR #1653: Looking up [admjesse_chan(a)japn.dell.com] in cache (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_cache] (0x0400): CR #1653: Object [admjesse_chan(a)japn.dell.com] was not found in cache (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_send] (0x0400): CR #1653: Looking up admjesse_chan(a)dell.com (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_ncache] (0x0400): CR #1653: Checking negative cache for [admjesse_chan(a)dell.com] (Sun Jul 1 15:21:12 2018) [sssd[nss]] [sss_ncache_check_str] (0x2000): Checking negative cache for [NCE/USER/dell.com/admjesse_chan@dell.com] (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_ncache] (0x0400): CR #1653: [admjesse_chan(a)dell.com] is not present in negative cache (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_cache] (0x0400): CR #1653: Looking up [admjesse_chan(a)dell.com] in cache (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_cache] (0x0400): CR #1653: Object [admjesse_chan(a)dell.com] was not found in cache (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_dp] (0x0400): CR #1653: Looking up [admjesse_chan(a)dell.com] in data provider (Sun Jul 1 15:21:12 2018) [sssd[nss]] [sss_dp_issue_request] (0x0400): Issuing request for [0x55f842febb50:1:admjesse_chan@dell.com@dell.com] (Sun Jul 1 15:21:12 2018) [sssd[nss]] [sss_dp_get_account_msg] (0x0400): Creating request for [dell.com ][0x1][BE_REQ_USER][name=admjesse_chan@dell.com:-] (Sun Jul 1 15:21:12 2018) [sssd[nss]] [sss_dp_internal_get_send] (0x0400): Entering request [0x55f842febb50:1:admjesse_chan@dell.com@dell.com] (Sun Jul 1 15:21:12 2018) [sssd[nss]] [sss_dp_req_destructor] (0x0400): Deleting request: [0x55f842febb50:1:admjesse_chan@japn.dell.com@ japn.dell.com] (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_cache] (0x0400): CR #1653: Looking up [admjesse_chan(a)dell.com] in cache (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_cache] (0x0400): CR #1653: Object [admjesse_chan(a)dell.com] was not found in cache (Sun Jul 1 15:21:12 2018) [sssd[nss]] [sss_dp_req_destructor] (0x0400): Deleting request: [0x55f842febb50:1:admjesse_chan@dell.com@dell.com] Specifically, this user is in apac.dell.com, so I want to make sure it's querying that subdomain. It is. [root@spikerealmd02 sssd]# grep 'admjesse_chan.*apac' sssd_nss.log (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_send] (0x0400): CR #1653: Looking up admjesse_chan(a)apac.dell.com (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_ncache] (0x0400): CR #1653: Checking negative cache for [admjesse_chan(a)apac.dell.com] (Sun Jul 1 15:21:12 2018) [sssd[nss]] [sss_ncache_check_str] (0x2000): Checking negative cache for [NCE/USER/ apac.dell.com/admjesse_chan@apac.dell.com] (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_ncache] (0x0400): CR #1653: [admjesse_chan(a)apac.dell.com] is not present in negative cache (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_cache] (0x0400): CR #1653: Looking up [admjesse_chan(a)apac.dell.com] in cache (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_cache] (0x0400): CR #1653: Object [admjesse_chan(a)apac.dell.com] was not found in cache (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_dp] (0x0400): CR #1653: Looking up [admjesse_chan(a)apac.dell.com] in data provider (Sun Jul 1 15:21:12 2018) [sssd[nss]] [sss_dp_issue_request] (0x0400): Issuing request for [0x55f842febb50:1:admjesse_chan@apac.dell.com@ apac.dell.com] (Sun Jul 1 15:21:12 2018) [sssd[nss]] [sss_dp_get_account_msg] (0x0400): Creating request for [apac.dell.com ][0x1][BE_REQ_USER][name=admjesse_chan@apac.dell.com:-] (Sun Jul 1 15:21:12 2018) [sssd[nss]] [sss_dp_internal_get_send] (0x0400): Entering request [0x55f842febb50:1:admjesse_chan@apac.dell.com@apac.dell.com ] (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_cache] (0x0400): CR #1653: Looking up [admjesse_chan(a)apac.dell.com] in cache (Sun Jul 1 15:21:12 2018) [sssd[nss]] [cache_req_search_cache] (0x0400): CR #1653: Object [admjesse_chan(a)apac.dell.com] was not found in cache (Sun Jul 1 15:21:12 2018) [sssd[nss]] [sss_dp_req_destructor] (0x0400): Deleting request: [0x55f842febb50:1:admjesse_chan@apac.dell.com@ apac.dell.com] Yes, sss is in /etc/nsswitch.conf: [root@spikerealmd02 sssd]# grep sss /etc/nsswitch.conf passwd: files sss shadow: files sss group: files sss #initgroups: files sss #netgroup: files sss But I knew that already, as users in the local domain get resolved fine: [root@spikerealmd02 sssd]# id admspike_white uid=2025431(admspike_white(a)amer.dell.com) gid=2025431( admspike_white(a)amer.dell.com) groups=2025431(admspike_white(a)amer.dell.com ),1002(amerlinuxeng@amer.dell.com),1041(linux-core-engineering(a)amer.dell.com ) Curiously, I see these local account lookups both in sssd_nss.log and in sssd_amer.dell.com.log (AMER.DELL.COM is the local domain). But for that APAC user, I see the entries (above) in sssd_nss.log, but no equiv entries in sssd_apac.dell.com.log. Here's the systemctl status for sssd: [root@spikerealmd02 sssd]# systemctl status sssd ● sssd.service - System Security Services Daemon Loaded: loaded (/usr/lib/systemd/system/sssd.service; enabled; vendor preset: disabled) Active: active (running) since Mon 2018-06-25 00:05:10 CDT; 1 weeks 0 days ago Main PID: 82591 (sssd) CGroup: /system.slice/sssd.service ├─82591 /usr/sbin/sssd -i --logger=files ├─82592 /usr/libexec/sssd/sssd_be --domain amer.dell.com --uid 0 --gid 0 --logger=files ├─82593 /usr/libexec/sssd/sssd_be --domain apac.dell.com --uid 0 --gid 0 --logger=files ├─82594 /usr/libexec/sssd/sssd_be --domain emea.dell.com --uid 0 --gid 0 --logger=files ├─82595 /usr/libexec/sssd/sssd_be --domain japn.dell.com --uid 0 --gid 0 --logger=files ├─82596 /usr/libexec/sssd/sssd_nss --uid 0 --gid 0 --logger=files └─82597 /usr/libexec/sssd/sssd_pam --uid 0 --gid 0 --logger=files It's as if sssd_nss is properly invoking sub sssd process 82592 for amer.dell.com lookups. But not invoking sub sssd process 82593 for apac.dell.com lookups. Spike On Mon, Jul 2, 2018 at 2:32 AM Sumit Bose <sbose(a)redhat.com&gt; wrote:

Thanks for responding. what you said was not exactly my situation, but it got me poking around and finally I got the configuration working. I find this interesting item when looking at various sssd subcommands; [root@spikerealmd02 ~]# sssctl domain-list amer.dell.com apac.dell.com emea.dell.com japn.dell.com dell.com apac.dell.com emea.dell.com japn.dell.com *Why are the remote subdomains duplicated? * Ultimately, this was my problem. When I clear my logs: sssctl remove-logs and II do a: getent passwd admjesse_chan which still fails. via 'grep -il admjesse_chan sssd_*.log', I find string only in sssd_amer.dell.com.log and sssd_nss.log. sssd_apac.dell.com.log does the usual site-awareness lookup and identified primary LDAP server optimally. So this sssd_be subprocess is working correctly. Yet this log file never receives a query request for ' admjesse_chan(a)apac.dell.com&#39;. sssd_nss.log gets a query request for admjesse_chan. It knows about remote subdomains: (Wed Jul 4 01:21:34 2018) [sssd[nss]] [cache_req_set_plugin] (0x2000): CR #1753: Setting "User by name" plugin (Wed Jul 4 01:21:34 2018) [sssd[nss]] [cache_req_send] (0x0400): CR #1753: New request 'User by name' (Wed Jul 4 01:21:34 2018) [sssd[nss]] [cache_req_process_input] (0x0400): CR #1753: Parsing input name [admjesse_chan] *(Wed Jul 4 01:21:34 2018) [sssd[nss]] [sss_domain_get_state] (0x1000): Domain apac.dell.com <http://apac.dell.com> is Active* *(Wed Jul 4 01:21:34 2018) [sssd[nss]] [sss_domain_get_state] (0x1000): Domain emea.dell.com <http://emea.dell.com> is Active* *(Wed Jul 4 01:21:34 2018) [sssd[nss]] [sss_domain_get_state] (0x1000): Domain japn.dell.com <http://japn.dell.com> is Active* Since admjesse_chan was specified with domain, it performs a multi-domain search. it searches local domain first. It searches cache for admjesse_chan(a)amer.dell.com and then data_provider first: (Wed Jul 4 01:21:34 2018) [sssd[nss]] [sss_parse_name_for_domains] (0x0200): name 'admjesse_chan' matched without domain, user is admjesse_chan (Wed Jul 4 01:21:34 2018) [sssd[nss]] [cache_req_set_name] (0x0400): CR #1753: Setting name [admjesse_chan] (Wed Jul 4 01:21:34 2018) [sssd[nss]] [cache_req_select_domains] (0x0400): CR #1753: Performing a multi-domain search (Wed Jul 4 01:21:34 2018) [sssd[nss]] [cache_req_search_domains] (0x0400): CR #1753: Search will check the cache and check the data provider ... (Wed Jul 4 01:21:34 2018) [sssd[nss]] [cache_req_search_cache] (0x0400): CR #1753: Object [admjesse_chan(a)amer.dell.com] was not found in cache (Wed Jul 4 01:21:34 2018) [sssd[nss]] [cache_req_search_dp] (0x0400): CR #1753: Looking up [admjesse_chan(a)amer.dell.com] in data provider It dispatches to data_provider (child process sssd_be for amer.dell.com) and receives successful response back (no records found): (Wed Jul 4 01:21:34 2018) [sssd[nss]] [sss_dp_get_account_msg] (0x0400): Creating request for [amer.dell.com ][0x1][BE_REQ_USER][name=admjesse_chan@amer.dell.com:-] (Wed Jul 4 01:21:34 2018) [sssd[nss]] [sbus_add_timeout] (0x2000): 0x55f844a5cac0 (Wed Jul 4 01:21:34 2018) [sssd[nss]] [sss_dp_internal_get_send] (0x0400): Entering request [0x55f842febb50:1:admjesse_chan@amer.dell.com@amer.dell.com ] (Wed Jul 4 01:21:34 2018) [sssd[nss]] [sbus_remove_timeout] (0x2000): 0x55f844a5cac0 (Wed Jul 4 01:21:34 2018) [sssd[nss]] [sbus_dispatch] (0x4000): dbus conn: 0x55f844a330f0 *(Wed Jul 4 01:21:34 2018) [sssd[nss]] [sbus_dispatch] (0x4000): Dispatching.* *(Wed Jul 4 01:21:34 2018) [sssd[nss]] [sss_dp_get_reply] (0x1000): Got reply from Data Provider - DP error code: 0 errno: 0 error message: Success* So next, it searches the next domain -- emea.dell.com: (Wed Jul 4 01:21:34 2018) [sssd[nss]] [sss_dp_get_account_msg] (0x0400): Creating request for [emea.dell.com ][0x1][BE_REQ_USER][name=admjesse_chan@emea.dell.com:-] ... (Wed Jul 4 01:21:34 2018) [sssd[nss]] [sbus_dispatch] (0x4000): Dispatching. *(Wed Jul 4 01:21:34 2018) [sssd[nss]] [sss_dp_get_reply] (0x1000): Got reply from Data Provider - DP error code: 3 errno: 5 error message: Input/output error* *(Wed Jul 4 01:21:34 2018) [sssd[nss]] [cache_req_common_dp_recv] (0x0040): CR #1753: Data Provider Error: 3, 5, Input/output error* *(Wed Jul 4 01:21:34 2018) [sssd[nss]] [cache_req_common_dp_recv] (0x0400): CR #1753: Due to an error we will return cached data* (Wed Jul 4 01:21:34 2018) [sssd[nss]] [cache_req_search_cache] (0x0400): CR #1753: Looking up [admjesse_chan(a)emea.dell.com] in cache It catches the same I/O error when dispatching to the other data_providers: apac.dell.com, japn.dell.com So I focused on -- why are there duplicate entries in sssctl domain-list? Might it be trying to dispatch to some "ghost" data_provider (non-existent sssd_be child process)? Thus I focused on my sssd.conf file. i realized I don't need this line: [sssd] debug_level = 6 domains = amer.dell.com,apac.dell.com,emea.dell.com,japn.dell.com *domain_resolution_order = amer.dell.com <http://amer.dell.com>;, emea.dell.com <http://emea.dell.com>;, apac.dell.com <http://apac.dell.com>;, japn.dell.com <http://japn.dell.com>* As without it, it will search domains in the order listed in "domains" line. Tried that fix, made no difference. Next I realized that https://lists.fedorahosted.org/pipermail/sssd-users/2015-February/002648.... doesn't have these lines in each of his [domain/XXX] stanzas. ad_enabled_domains = amer.dell.com,apac.dell.com,emea.dell.com,japn.dell.com ,dell.com So I removed that line from each [domain/XXX] stanza and did a: sssctl cache-remove (which restarts sssd service) Now sssctl domain-list shows me no dups: [root@spikerealmd02 sssd]# sssctl domain-list amer.dell.com apac.dell.com emea.dell.com japn.dell.com Now the dispatches out of sssd_nss.log to the various data_providers succeed. And I see remote users: [root@spikerealmd02 sssd]# id admjesse_chan uid=525641(admjesse_chan(a)apac.dell.com) gid=525641( admjesse_chan(a)apac.dell.com) groups=525641(admjesse_chan(a)apac.dell.com ),1008(apacunixusers@apac.dell.com),1000(apaclinuxeng(a)apac.dell.com),1001( apaclinuxsup(a)apac.dell.com) I still don't see all expected groups for these users, so my ldap_use_tokengroups is still not 100% right. Or maybe these missing groups are UNIVERSAL groups and I should be additionally querying dell.com. Oh, well -problem for another day. At least now x-subdomain auth is again working and tokengroups is (mostly) working. Spike On Tue, Jul 3, 2018 at 12:21 AM Sumit Bose <sbose(a)redhat.com&gt; wrote:

Yes, most of the groups missing when I set 'ldap_use_tokengroups = true' are universal groups. I don't know where universal groups reside. Whether it's in the parent domain (dell.com), or in the GC or where. (I'm not a AD expert -- I'm a Linux engineer that has has done some AD integration over the years.) But not all the missing groups are universal groups. A few missing groups for certain accounts are domain-local groups. Surprisingly, when I set 'ldap_use_tokengroups = false', all the group memberships for all accounts are 100% correct. But my strong preference is to use tokengroups (& of course, cross-domain authentication is mandatory). It sounds like you're suggesting an alternative configuration that would allow tokengroups to behave as expected. My concern is if that alternative configuration will break cross domain authentication again. I will write up the details of the missing AD groups in this configuration in the next couple of days. Spike On Wed, Jul 4, 2018 at 9:13 AM Sumit Bose <sbose(a)redhat.com&gt; wrote:

Thanks, I'll check it out. On Sun, Jul 8, 2018 at 2:30 PM Michael Ströder <michael(a)stroeder.com&gt; wrote: