Confirmed. I made the following simple change and it allows cb_ping_farm to work with
anonymous binds only enabled for rootdse;
diff -urN a/ldap/servers/plugins/chainingdb/cb_conn_stateless.c
b/ldap/servers/plugins/chainingdb/cb_conn_stateless.c
--- a/ldap/servers/plugins/chainingdb/cb_conn_stateless.c 2020-03-17
04:52:57.000000000 +1000
+++ b/ldap/servers/plugins/chainingdb/cb_conn_stateless.c 2021-03-08
14:04:48.413647052 +1000
@@ -883,7 +883,7 @@
/* NOTE: This will fail if we implement the ability to disable
anonymous bind */
- rc = ldap_search_ext_s(ld, NULL, LDAP_SCOPE_BASE, "objectclass=*", attrs,
1, NULL,
+ rc = ldap_search_ext_s(ld, "", LDAP_SCOPE_BASE, "objectclass=*",
attrs, 1, NULL,
NULL, &timeout, 1, &result);
if (LDAP_SUCCESS != rc) {
slapi_ldap_unbind(ld);
I don't believe this will break any functionality, but since we're running RHEL7,
i'll raise this with Red Hat directly and they can review and/or push upstream.
Regards,
Grant
________________________________
From: Grant Byers <Grant.Byers(a)aarnet.edu.au>
Sent: Monday, March 8, 2021 12:27 PM
To: 389-users(a)lists.fedoraproject.org <389-users(a)lists.fedoraproject.org>
Subject: Re: [389-users] Re: Chain on Update problem
Thanks.
I have tested various combinations of the tuning params without success. I've done
further debugging and confirmed that it always starts after a bind operation timeout.
Looking into the chaining plugin code, I see that on operation timeout results in a call
to cb_ping_farm to see if we can find another server in the pool that is available.
However, it performs this operation (the comment is telling);
/* NOTE: This will fail if we implement the ability to disable
anonymous bind */
rc = ldap_search_ext_s(ld, NULL, LDAP_SCOPE_BASE, "objectclass=*", attrs, 1,
NULL,
NULL, &timeout, 1, &result);
if (LDAP_SUCCESS != rc) {
slapi_ldap_unbind(ld);
cb_update_failed_conn_cpt(cb);
return LDAP_SERVER_DOWN;
}
So basically, because we've disallowed anonymous bind for anything but rootdse, it
will always fail to find another available server. I have confirmed this by allowing
anonymous bind on our masters while the issue was present, then subsequent binds on the
consumers start working again.
I would think it more appropriate for that code to do a search against the rootdse
instead. Is there any good reason why it shouldn't? If not, I might test modifying
it.
Thanks,
Grant
________________________________
From: William Brown <wbrown(a)suse.de>
Sent: Friday, March 5, 2021 3:52 PM
To: 389-users(a)lists.fedoraproject.org <389-users(a)lists.fedoraproject.org>
Subject: [389-users] Re: Chain on Update problem
[External Mail]
On 5 Mar 2021, at 12:03, Grant Byers
<Grant.Byers(a)aarnet.edu.au> wrote:
Hi All,
Version: 1.3.10
In our environment, we'd like to use a chaining backend to push BIND operations up to
masters by way of the consumer (rather than client referral). We'd like to do this to
ensure password lockout attributes are propagated to all consumers equally via our
standard replication agreements. This is described here -
https://directory.fedoraproject.org/docs/389ds/howto/howto-chainonupdate.....
NOTE, we do not have hubs in our topology. Just masters and consumers, so no intermediate
chaining.
We tested this process in our environment and it worked beautifully until we took it to
production. Currently, we have just 2 masters and they are both sitting on some
over-subscribed hardware that suffers from I/O starvation at certain times of the day. The
plan is to scale out our masters eventually, but we're a little hamstrung with other
projects and priorities. It worked extremely well until that time of day when masters
suffered from I/O starvation, and hence, very long I/O wait times. This is generally short
lived and happens at alternate times of the day for each of the masters. However, it seems
that once both nsfarmservers have "failed", there is never any attempt by the
consumer to retry them. This leads to bind errors as follows;
ldapwhoami -x -D "<binddn>" -W
Enter LDAP Password:
ldap_bind: Operations error (1)
additional info: FARM SERVER TEMPORARY UNAVAILABLE
Except it is not temporary. It never recovers, even though all members of nsfarmservers
are now healthy again (and are never unhealthy at the same time). We can confirm this by
performing binds from the consumers directly against the masters. I thought that setting
nsConnectionLife to something larger than 0 (indefinite) would help this, but it has not.
The chain on update appears to use the chaining plugin timeouts, so you could look at
adjusting these parameters which may help.
nsBindTimeout
nsOperationTimeout
nsBindRetryLimit
nsMaxResponseDelay
nsMaxTestResponseDelay
Is this by design, a bug, or an implementation fault on my behalf? Configuration below;
Thanks,
Grant
## On masters, create a dedicated user for chaining backend
dn: cn=proxy,cn=config
objectClass: person
objectClass: top
cn: proxy
sn: admin
## On all consumers, create chaining backend;
dn: cn=chainbe1,cn=chaining database,cn=plugins,cn=config
objectclass: top
objectclass: extensibleObject
objectclass: nsBackendInstance
nsslapd-suffix: <suffix>
nsfarmserverurl: ldaps://<master1>:636 <master2>:636/
nsMultiplexorBindDN: <binddn>>
nsMultiplexorCredentials: <bindpw>
nsCheckLocalACI: on
nsConnectionLife: 30
cn: chainbe1
## On all consumers, add the backend and repl_chain_on_update function
dn: cn="<suffix>",cn=mapping tree,cn=config
changetype: modify
add: nsslapd-backend
nsslapd-backend: chainbe1
-
add: nsslapd-distribution-plugin
nsslapd-distribution-plugin: libreplication-plugin
-
add: nsslapd-distribution-funct
nsslapd-distribution-funct: repl_chain_on_update
## On all servers, enable global pasword policy
dn: cn=config
changetype: modify
replace: passwordIsGlobalPolicy
passwordIsGlobalPolicy: on
_______________________________________________
389-users mailing list -- 389-users(a)lists.fedoraproject.org
To unsubscribe send an email to 389-users-leave(a)lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproje...
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
—
Sincerely,
William Brown
Senior Software Engineer, 389 Directory Server
SUSE Labs, Australia
_______________________________________________
389-users mailing list -- 389-users(a)lists.fedoraproject.org
To unsubscribe send an email to 389-users-leave(a)lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproje...
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure