[389-ds-base] branch 389-ds-base-1.4.3 updated: Issue 51059 - If dbhome directory is set online backup fails
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
spichugi pushed a commit to branch 389-ds-base-1.4.3
in repository 389-ds-base.
The following commit(s) were added to refs/heads/389-ds-base-1.4.3 by this push:
new 7624c59 Issue 51059 - If dbhome directory is set online backup fails
7624c59 is described below
commit 7624c59bcf88d36b5c5e2d2e383631488a931441
Author: Simon Pichugin <spichugi(a)redhat.com>
AuthorDate: Wed Jul 15 21:38:19 2020 +0200
Issue 51059 - If dbhome directory is set online backup fails
Bug Description: If the dbhome directory is set, eg to /dev/shm/instance
then an online backup fails because it looks for the log.000000x file
in the wring diretcory.
This is hidden because the return code is overwritten before checking.
Fix Description: If dblayer_backup function fails - go to error processing
section.
https://pagure.io/389-ds-base/issue/51059
Reviewed by: mreynolds (Thanks!)
---
ldap/servers/slapd/back-ldbm/archive.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/ldap/servers/slapd/back-ldbm/archive.c b/ldap/servers/slapd/back-ldbm/archive.c
index 4eb0eb5..6047e78 100644
--- a/ldap/servers/slapd/back-ldbm/archive.c
+++ b/ldap/servers/slapd/back-ldbm/archive.c
@@ -427,6 +427,11 @@ ldbm_back_ldbm2archive(Slapi_PBlock *pb)
/* tell it to archive */
return_value = dblayer_backup(li, directory, task);
+ if (return_value) {
+ slapi_log_err(SLAPI_LOG_BACKLDBM,
+ "ldbm_back_ldbm2archive", "dblayer_backup failed (%d).\n", return_value);
+ goto err;
+ }
return_value = plugin_call_plugins(pb, SLAPI_PLUGIN_BE_POST_BACKUP_FN);
if (return_value) {
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 8 months
[389-ds-base] branch 389-ds-base-1.4.2 updated: Issue 51059 - If dbhome directory is set online backup fails
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
spichugi pushed a commit to branch 389-ds-base-1.4.2
in repository 389-ds-base.
The following commit(s) were added to refs/heads/389-ds-base-1.4.2 by this push:
new 48572f6 Issue 51059 - If dbhome directory is set online backup fails
48572f6 is described below
commit 48572f6c6f675a8361f900f156baf70d2fdd1965
Author: Simon Pichugin <spichugi(a)redhat.com>
AuthorDate: Wed Jul 15 21:38:19 2020 +0200
Issue 51059 - If dbhome directory is set online backup fails
Bug Description: If the dbhome directory is set, eg to /dev/shm/instance
then an online backup fails because it looks for the log.000000x file
in the wring diretcory.
This is hidden because the return code is overwritten before checking.
Fix Description: If dblayer_backup function fails - go to error processing
section.
https://pagure.io/389-ds-base/issue/51059
Reviewed by: mreynolds (Thanks!)
---
ldap/servers/slapd/back-ldbm/archive.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/ldap/servers/slapd/back-ldbm/archive.c b/ldap/servers/slapd/back-ldbm/archive.c
index 4eb0eb5..6047e78 100644
--- a/ldap/servers/slapd/back-ldbm/archive.c
+++ b/ldap/servers/slapd/back-ldbm/archive.c
@@ -427,6 +427,11 @@ ldbm_back_ldbm2archive(Slapi_PBlock *pb)
/* tell it to archive */
return_value = dblayer_backup(li, directory, task);
+ if (return_value) {
+ slapi_log_err(SLAPI_LOG_BACKLDBM,
+ "ldbm_back_ldbm2archive", "dblayer_backup failed (%d).\n", return_value);
+ goto err;
+ }
return_value = plugin_call_plugins(pb, SLAPI_PLUGIN_BE_POST_BACKUP_FN);
if (return_value) {
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 8 months
[389-ds-base] branch 389-ds-base-1.4.2 updated: Issue 51000 - Separate the BDB backend monitors
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
mreynolds pushed a commit to branch 389-ds-base-1.4.2
in repository 389-ds-base.
The following commit(s) were added to refs/heads/389-ds-base-1.4.2 by this push:
new 88006a6 Issue 51000 - Separate the BDB backend monitors
88006a6 is described below
commit 88006a67cb486bcf645a92304105f5439d215611
Author: Mark Reynolds <mreynolds(a)redhat.com>
AuthorDate: Mon Jul 6 15:18:21 2020 -0400
Issue 51000 - Separate the BDB backend monitors
Bug Description: While trying to remove duplicate code from the backend
and BDB backend code, I found that we were not correctly
separating the BDB monitors from the core backend code.
Fix Description: Move all the monitor registering to the db_layer private
structure. This way we have fully isolated the monitors
for each backend implementation/library. This also removed
some duplicate code from the core backend and BDB code.
relates: https://pagure.io/389-ds-base/issue/51000
Reviewed by: spichugi(Thanks!)
(cherry picked from commit ed5b13cab68f82ae6a1abe03bf41eb7369c30f2c)
---
dirsrvtests/tests/suites/monitor/monitor_test.py | 82 ++++++++++++++++++++++
ldap/servers/slapd/back-ldbm/db-bdb/bdb_config.c | 72 ++++++++++++++++++-
.../slapd/back-ldbm/db-bdb/bdb_instance_config.c | 2 +
ldap/servers/slapd/back-ldbm/db-bdb/bdb_layer.h | 8 ++-
ldap/servers/slapd/back-ldbm/db-bdb/bdb_ldif2db.c | 2 +-
ldap/servers/slapd/back-ldbm/db-bdb/bdb_monitor.c | 28 ++++----
ldap/servers/slapd/back-ldbm/db-bdb/bdb_verify.c | 2 +-
ldap/servers/slapd/back-ldbm/dblayer.c | 3 +-
ldap/servers/slapd/back-ldbm/dblayer.h | 1 +
ldap/servers/slapd/back-ldbm/ldbm_config.c | 32 ---------
.../servers/slapd/back-ldbm/ldbm_instance_config.c | 47 +------------
ldap/servers/slapd/back-ldbm/proto-back-ldbm.h | 8 ---
src/lib389/lib389/backend.py | 4 ++
13 files changed, 186 insertions(+), 105 deletions(-)
diff --git a/dirsrvtests/tests/suites/monitor/monitor_test.py b/dirsrvtests/tests/suites/monitor/monitor_test.py
index 5786903..98dd62d 100644
--- a/dirsrvtests/tests/suites/monitor/monitor_test.py
+++ b/dirsrvtests/tests/suites/monitor/monitor_test.py
@@ -2,6 +2,7 @@ import logging
import pytest
import os
from lib389.monitor import *
+from lib389.backend import Backends, DatabaseConfig
from lib389._constants import *
from lib389.topologies import topology_st as topo
@@ -63,6 +64,87 @@ def test_monitor(topo):
log.info('dtablesize: {0[0]},readwaiters: {0[1]},entriessent: {0[2]},bytessent: {0[3]},currenttime: {0[4]},starttime: {0[5]}'.format(stats))
+pytestmark = pytest.mark.tier1
+def test_monitor_ldbm(topo):
+ """This test is to check if we are getting the correct monitor entry
+
+ :id: e62ba369-32f5-4b03-8865-f597a5bb6a70
+ :setup: Single instance
+ :steps:
+ 1. Get the backend library (bdb, ldbm, etc)
+ 2. Get the database monitor
+ 3. Check for expected attributes in output
+ 4. Check for expected DB library specific attributes
+ :expectedresults:
+ 1. Success
+ 2. Success
+ 3. Success
+ 4. Success
+ """
+
+ # Are we using BDB?
+ db_config = DatabaseConfig(topo.standalone)
+ db_lib = db_config.get_db_lib()
+
+ # Get the database monitor entry
+ monitor = MonitorLDBM(topo.standalone).get_status()
+
+ # Check that known attributes exist (only NDN cache stats)
+ assert 'normalizeddncachehits' in monitor
+
+ # Check for library specific attributes
+ if db_lib == 'bdb':
+ assert 'dbcachehits' in monitor
+ assert 'nsslapd-db-configured-locks' in monitor
+ elif db_lib == 'lmdb':
+ pass
+ else:
+ # Unknown - the server would probably fail to start but check it anyway
+ log.fatal(f'Unknown backend library: {db_lib}')
+ assert False
+
+
+pytestmark = pytest.mark.tier1
+def test_monitor_backend(topo):
+ """This test is to check if we are getting the correct backend monitor entry
+
+ :id: 27b0534f-a18c-4c95-aa2b-936bc1886a7b
+ :setup: Single instance
+ :steps:
+ 1. Get the backend library (bdb, ldbm, etc)
+ 2. Get the backend monitor
+ 3. Check for expected attributes in output
+ 4. Check for expected DB library specific attributes
+ :expectedresults:
+ 1. Success
+ 2. Success
+ 3. Success
+ 4. Success
+ """
+
+ # Are we using BDB?
+ db_config = DatabaseConfig(topo.standalone)
+ db_lib = db_config.get_db_lib()
+
+ # Get the backend monitor
+ be = Backends(topo.standalone).list()[0]
+ monitor = be.get_monitor().get_status()
+
+ # Check for expected attributes
+ assert 'entrycachehits' in monitor
+ assert 'dncachehits' in monitor
+
+ # Check for library specific attributes
+ if db_lib == 'bdb':
+ assert 'dbfilename-0' in monitor
+ elif db_lib == 'lmdb':
+ pass
+ else:
+ # Unknown - the server would probably fail to start but check it anyway
+ log.fatal(f'Unknown backend library: {db_lib}')
+ assert False
+
+
if __name__ == '__main__':
# Run isolated
# -s for DEBUG mode
diff --git a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_config.c b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_config.c
index 7c05d2f..738b841 100644
--- a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_config.c
+++ b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_config.c
@@ -88,6 +88,7 @@ int bdb_init(struct ldbminfo *li, config_info *config_array)
priv->instance_postdel_config_fn = &bdb_instance_post_delete_instance_entry_callback;
priv->instance_cleanup_fn = &bdb_instance_cleanup;
priv->instance_create_fn = &bdb_instance_create;
+ priv->instance_register_monitor_fn = &bdb_instance_register_monitor;
priv->instance_search_callback_fn = &bdb_instance_search_callback;
priv->dblayer_auto_tune_fn = &bdb_start_autotune;
return 0;
@@ -1530,6 +1531,7 @@ bail:
}
return rval;
}
+
/* Reads in any config information held in the dse for the bdb
* implementation of the ldbm plugin.
* Creates dse entries used to configure the ldbm plugin and dblayer
@@ -1640,7 +1642,7 @@ retry:
* Now still using ldbm functions
*/
slapi_config_register_callback(SLAPI_OPERATION_SEARCH, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_BASE, "(objectclass=*)", ldbm_back_monitor_search,
+ LDAP_SCOPE_BASE, "(objectclass=*)", bdb_monitor_search,
(void *)li);
slapi_ch_free_string(&dn);
@@ -1656,7 +1658,7 @@ retry:
goto bail;
}
slapi_config_register_callback(SLAPI_OPERATION_SEARCH, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_BASE, "(objectclass=*)", ldbm_back_dbmonitor_search,
+ LDAP_SCOPE_BASE, "(objectclass=*)", bdb_dbmonitor_search,
(void *)li);
bail:
@@ -1664,6 +1666,72 @@ bail:
return rval;
}
+/* general-purpose callback to deny an operation */
+static int
+bdb_deny_config(Slapi_PBlock *pb __attribute__((unused)),
+ Slapi_Entry *e __attribute__((unused)),
+ Slapi_Entry *entryAfter __attribute__((unused)),
+ int *returncode,
+ char *returntext __attribute__((unused)),
+ void *arg __attribute__((unused)))
+{
+ *returncode = LDAP_UNWILLING_TO_PERFORM;
+ return SLAPI_DSE_CALLBACK_ERROR;
+}
+
+int
+bdb_instance_register_monitor(ldbm_instance *inst) {
+ struct ldbminfo *li = inst->inst_li;
+ char *dn = NULL;
+
+ dn = slapi_create_dn_string("cn=monitor,cn=%s,cn=%s,cn=plugins,cn=config",
+ inst->inst_name, li->li_plugin->plg_name);
+ if (NULL == dn) {
+ slapi_log_err(SLAPI_LOG_ERR,
+ "bdb_instance_register_monitor",
+ "failed create monitor instance dn for plugin %s, "
+ "instance %s\n",
+ inst->inst_li->li_plugin->plg_name, inst->inst_name);
+ return 1;
+ }
+ /* make callback on search; deny add/modify/delete */
+ slapi_config_register_callback(SLAPI_OPERATION_SEARCH, DSE_FLAG_PREOP, dn,
+ LDAP_SCOPE_BASE, "(objectclass=*)", bdb_monitor_instance_search,
+ (void *)inst);
+ slapi_config_register_callback(SLAPI_OPERATION_ADD, DSE_FLAG_PREOP, dn,
+ LDAP_SCOPE_SUBTREE, "(objectclass=*)", bdb_deny_config,
+ (void *)inst);
+ slapi_config_register_callback(SLAPI_OPERATION_MODIFY, DSE_FLAG_PREOP, dn,
+ LDAP_SCOPE_BASE, "(objectclass=*)", bdb_deny_config,
+ (void *)inst);
+ slapi_ch_free_string(&dn);
+
+ return 0;
+}
+
+void
+bdb_instance_unregister_monitor(ldbm_instance *inst) {
+ struct ldbminfo *li = inst->inst_li;
+ char *dn = NULL;
+
+ dn = slapi_create_dn_string("cn=monitor,cn=%s,cn=%s,cn=plugins,cn=config",
+ inst->inst_name, li->li_plugin->plg_name);
+ if (NULL == dn) {
+ slapi_log_err(SLAPI_LOG_ERR,
+ "bdb_instance_unregister_monitor",
+ "Failed create monitor instance dn for plugin %s, "
+ "instance %s\n",
+ inst->inst_li->li_plugin->plg_name, inst->inst_name);
+ return;
+ }
+ slapi_config_remove_callback(SLAPI_OPERATION_SEARCH, DSE_FLAG_PREOP, dn,
+ LDAP_SCOPE_BASE, "(objectclass=*)", bdb_monitor_instance_search);
+ slapi_config_remove_callback(SLAPI_OPERATION_ADD, DSE_FLAG_PREOP, dn,
+ LDAP_SCOPE_SUBTREE, "(objectclass=*)", bdb_deny_config);
+ slapi_config_remove_callback(SLAPI_OPERATION_MODIFY, DSE_FLAG_PREOP, dn,
+ LDAP_SCOPE_BASE, "(objectclass=*)", bdb_deny_config);
+ slapi_ch_free_string(&dn);
+}
/* Utility function used in creating config entries. Using the
* config_info, this function gets info and formats in the correct
diff --git a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_instance_config.c b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_instance_config.c
index 6a21c22..0ac3694 100644
--- a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_instance_config.c
+++ b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_instance_config.c
@@ -197,6 +197,8 @@ bdb_instance_post_delete_instance_entry_callback(struct ldbminfo *li, struct ldb
if (inst_dirp != inst_dir) {
slapi_ch_free_string(&inst_dirp);
}
+ /* unregister the monitor */
+ bdb_instance_unregister_monitor(inst);
} /* non-null pEnv */
return SLAPI_DSE_CALLBACK_OK;
}
diff --git a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_layer.h b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_layer.h
index d729327..bf00d2e 100644
--- a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_layer.h
+++ b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_layer.h
@@ -150,7 +150,6 @@ int bdb_back_ok_to_dump(const char *dn, char **include, char **exclude);
int bdb_back_fetch_incl_excl(Slapi_PBlock *pb, char ***include, char ***exclude);
PRUint64 bdb_get_id2entry_size(ldbm_instance *inst);
-
/* bdb version functions */
int bdb_version_write(struct ldbminfo *li, const char *directory, const char *dataversion, PRUint32 flags);
int bdb_version_read(struct ldbminfo *li, const char *directory, char **ldbmversion, char **dataversion);
@@ -162,3 +161,10 @@ int bdb_instance_post_delete_instance_entry_callback(struct ldbminfo *li, struct
int bdb_instance_add_instance_entry_callback(struct ldbminfo *li, struct ldbm_instance *inst);
int bdb_instance_postadd_instance_entry_callback(struct ldbminfo *li, struct ldbm_instance *inst);
void bdb_config_setup_default(struct ldbminfo *li);
+
+/* monitor functions */
+int bdb_monitor_instance_search(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *entryAfter, int *returncode, char *returntext, void *arg);
+int bdb_monitor_search(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *entryAfter, int *returncode, char *returntext, void *arg);
+int bdb_dbmonitor_search(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *entryAfter, int *returncode, char *returntext, void *arg);
+int bdb_instance_register_monitor(ldbm_instance *inst);
+void bdb_instance_unregister_monitor(ldbm_instance *inst);
diff --git a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_ldif2db.c b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_ldif2db.c
index 542147c..afef5a3 100644
--- a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_ldif2db.c
+++ b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_ldif2db.c
@@ -3144,7 +3144,7 @@ bdb_upgradednformat(Slapi_PBlock *pb)
run_from_cmdline = (task_flags & SLAPI_TASK_RUNNING_FROM_COMMANDLINE);
slapi_pblock_get(pb, SLAPI_PLUGIN_PRIVATE, &li);
if (run_from_cmdline) {
- ldbm_config_load_dse_info(li);
+ bdb_config_load_dse_info(li);
if (bdb_check_and_set_import_cache(li) < 0) {
return -1;
}
diff --git a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_monitor.c b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_monitor.c
index ac3479d..8f954c4 100644
--- a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_monitor.c
+++ b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_monitor.c
@@ -33,12 +33,12 @@
/* DSE callback to monitor stats for a particular instance */
int
-ldbm_back_monitor_instance_search(Slapi_PBlock *pb __attribute__((unused)),
- Slapi_Entry *e,
- Slapi_Entry *entryAfter __attribute__((unused)),
- int *returncode,
- char *returntext __attribute__((unused)),
- void *arg)
+bdb_monitor_instance_search(Slapi_PBlock *pb __attribute__((unused)),
+ Slapi_Entry *e,
+ Slapi_Entry *entryAfter __attribute__((unused)),
+ int *returncode,
+ char *returntext __attribute__((unused)),
+ void *arg)
{
ldbm_instance *inst = (ldbm_instance *)arg;
struct ldbminfo *li = NULL;
@@ -150,8 +150,8 @@ ldbm_back_monitor_instance_search(Slapi_PBlock *pb __attribute__((unused)),
continue;
/* Since the filenames are now relative, we need to construct an absolute version
- * for the purpose of stat() etc below...
- */
+ * for the purpose of stat() etc below...
+ */
slapi_ch_free_string(&absolute_pathname);
absolute_pathname = slapi_ch_smprintf("%s%c%s", inst->inst_parent_dir_name, get_sep(inst->inst_parent_dir_name), mpfstat[i]->file_name);
@@ -159,10 +159,10 @@ ldbm_back_monitor_instance_search(Slapi_PBlock *pb __attribute__((unused)),
if (stat(absolute_pathname, &astat))
continue;
/* If the file has been re-created after been deleted
- * We should show only statistics for the last instance
- * Since SleepyCat returns the statistic of the last open file first,
- * we should only display the first statistic record for a given file
- */
+ * We should show only statistics for the last instance
+ * Since SleepyCat returns the statistic of the last open file first,
+ * we should only display the first statistic record for a given file
+ */
for (j = 0; j < i; j++)
if (!strcmp(mpfstat[i]->file_name, mpfstat[j]->file_name))
break;
@@ -194,7 +194,7 @@ ldbm_back_monitor_instance_search(Slapi_PBlock *pb __attribute__((unused)),
/* monitor global ldbm stats */
int
-ldbm_back_monitor_search(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *entryAfter, int *returncode, char *returntext, void *arg)
+bdb_monitor_search(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *entryAfter, int *returncode, char *returntext, void *arg)
{
struct ldbminfo *li = (struct ldbminfo *)arg;
struct berval val;
@@ -288,7 +288,7 @@ ldbm_back_monitor_search(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *entryAft
/* monitor global ldbm database stats */
int
-ldbm_back_dbmonitor_search(Slapi_PBlock *pb __attribute__((unused)),
+bdb_dbmonitor_search(Slapi_PBlock *pb __attribute__((unused)),
Slapi_Entry *e,
Slapi_Entry *entryAfter __attribute__((unused)),
int *returncode,
diff --git a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_verify.c b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_verify.c
index 32933a8..42d1cd1 100644
--- a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_verify.c
+++ b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_verify.c
@@ -171,7 +171,7 @@ bdb_verify(Slapi_PBlock *pb)
slapi_pblock_get(pb, SLAPI_SEQ_TYPE, &verbose);
slapi_pblock_get(pb, SLAPI_PLUGIN_PRIVATE, &li);
slapi_pblock_get(pb, SLAPI_DBVERIFY_DBDIR, &dbdir);
- ldbm_config_load_dse_info(li);
+ bdb_config_load_dse_info(li);
bdb_config_internal_set(li, CONFIG_DB_TRANSACTION_LOGGING, "off");
/* no write needed; choose EXPORT MODE */
diff --git a/ldap/servers/slapd/back-ldbm/dblayer.c b/ldap/servers/slapd/back-ldbm/dblayer.c
index c9f43c7..05cc5b8 100644
--- a/ldap/servers/slapd/back-ldbm/dblayer.c
+++ b/ldap/servers/slapd/back-ldbm/dblayer.c
@@ -273,10 +273,9 @@ dblayer_setup(struct ldbminfo *li)
}
ldbm_config_load_dse_info(li);
-
priv = (dblayer_private *)li->li_dblayer_private;
-
rc = priv->dblayer_load_dse_fn(li);
+
return rc;
}
diff --git a/ldap/servers/slapd/back-ldbm/dblayer.h b/ldap/servers/slapd/back-ldbm/dblayer.h
index 8e156dd..cd53f25 100644
--- a/ldap/servers/slapd/back-ldbm/dblayer.h
+++ b/ldap/servers/slapd/back-ldbm/dblayer.h
@@ -147,6 +147,7 @@ struct dblayer_private
instance_config_entry_callback_fn_t *instance_postdel_config_fn;
instance_cleanup_fn_t *instance_cleanup_fn;
instance_create_fn_t *instance_create_fn;
+ instance_create_fn_t *instance_register_monitor_fn;
instance_search_callback_fn_t *instance_search_callback_fn;
dblayer_auto_tune_fn_t *dblayer_auto_tune_fn;
};
diff --git a/ldap/servers/slapd/back-ldbm/ldbm_config.c b/ldap/servers/slapd/back-ldbm/ldbm_config.c
index 88c1863..3fe86d5 100644
--- a/ldap/servers/slapd/back-ldbm/ldbm_config.c
+++ b/ldap/servers/slapd/back-ldbm/ldbm_config.c
@@ -1113,38 +1113,6 @@ ldbm_config_load_dse_info(struct ldbminfo *li)
(void *)li);
slapi_ch_free_string(&dn);
- /* setup the dse callback functions for the ldbm backend monitor entry */
- dn = slapi_create_dn_string("cn=monitor,cn=%s,cn=plugins,cn=config",
- li->li_plugin->plg_name);
- if (NULL == dn) {
- slapi_log_err(SLAPI_LOG_ERR,
- "ldbm_config_load_dse_info",
- "failed to create monitor dn for %s\n",
- li->li_plugin->plg_name);
- rval = 1;
- goto bail;
- }
- slapi_config_register_callback(SLAPI_OPERATION_SEARCH, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_BASE, "(objectclass=*)", ldbm_back_monitor_search,
- (void *)li);
- slapi_ch_free_string(&dn);
-
- /* And the ldbm backend database monitor entry */
- dn = slapi_create_dn_string("cn=database,cn=monitor,cn=%s,cn=plugins,cn=config",
- li->li_plugin->plg_name);
- if (NULL == dn) {
- slapi_log_err(SLAPI_LOG_ERR,
- "ldbm_config_load_dse_info",
- "failed create monitor database dn for %s\n",
- li->li_plugin->plg_name);
- rval = 1;
- goto bail;
- }
- slapi_config_register_callback(SLAPI_OPERATION_SEARCH, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_BASE, "(objectclass=*)", ldbm_back_dbmonitor_search,
- (void *)li);
- slapi_ch_free_string(&dn);
-
/* setup the dse callback functions for the ldbm backend instance
* entries */
dn = slapi_create_dn_string("cn=%s,cn=plugins,cn=config",
diff --git a/ldap/servers/slapd/back-ldbm/ldbm_instance_config.c b/ldap/servers/slapd/back-ldbm/ldbm_instance_config.c
index 628ac90..9882f8f 100644
--- a/ldap/servers/slapd/back-ldbm/ldbm_instance_config.c
+++ b/ldap/servers/slapd/back-ldbm/ldbm_instance_config.c
@@ -648,32 +648,6 @@ ldbm_instance_config_load_dse_info(ldbm_instance *inst)
LDAP_SCOPE_BASE, "(objectclass=*)",
ldbm_instance_deny_config, (void *)inst);
/* delete is handled by a callback set in ldbm_config.c */
-
- slapi_ch_free_string(&dn);
-
- /* don't forget the monitor! */
- dn = slapi_create_dn_string("cn=monitor,cn=%s,cn=%s,cn=plugins,cn=config",
- inst->inst_name, li->li_plugin->plg_name);
- if (NULL == dn) {
- slapi_log_err(SLAPI_LOG_ERR,
- "ldbm_instance_config_load_dse_info",
- "failed create monitor instance dn for plugin %s, "
- "instance %s\n",
- inst->inst_li->li_plugin->plg_name, inst->inst_name);
- rval = 1;
- goto bail;
- }
- /* make callback on search; deny add/modify/delete */
- slapi_config_register_callback(SLAPI_OPERATION_SEARCH, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_BASE, "(objectclass=*)", ldbm_back_monitor_instance_search,
- (void *)inst);
- slapi_config_register_callback(SLAPI_OPERATION_ADD, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_SUBTREE, "(objectclass=*)", ldbm_instance_deny_config,
- (void *)inst);
- slapi_config_register_callback(SLAPI_OPERATION_MODIFY, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_BASE, "(objectclass=*)", ldbm_instance_deny_config,
- (void *)inst);
- /* delete is okay */
slapi_ch_free_string(&dn);
/* Callbacks to handle indexes */
@@ -881,6 +855,7 @@ static int
ldbm_instance_generate(struct ldbminfo *li, char *instance_name, Slapi_Backend **ret_be)
{
Slapi_Backend *new_be = NULL;
+ dblayer_private *priv = (dblayer_private *)li->li_dblayer_private;
int rc = 0;
/* Create a new instance, process config info for it,
@@ -895,6 +870,8 @@ ldbm_instance_generate(struct ldbminfo *li, char *instance_name, Slapi_Backend *
}
ldbm_instance_config_load_dse_info(new_be->be_instance_info);
+ priv->instance_register_monitor_fn(new_be->be_instance_info);
+
ldbm_instance_create_default_indexes(new_be);
/* if USN plugin is enabled, set slapi_counter */
@@ -957,24 +934,6 @@ ldbm_instance_unregister_callbacks(ldbm_instance *inst)
ldbm_instance_deny_config);
slapi_ch_free_string(&dn);
- /* now the cn=monitor entry */
- dn = slapi_create_dn_string("cn=monitor,cn=%s,cn=%s,cn=plugins,cn=config",
- inst->inst_name, li->li_plugin->plg_name);
- if (NULL == dn) {
- slapi_log_err(SLAPI_LOG_ERR,
- "ldbm_instance_unregister_callbacks",
- "Failed create monitor instance dn for plugin %s, "
- "instance %s\n",
- inst->inst_li->li_plugin->plg_name, inst->inst_name);
- goto bail;
- }
- slapi_config_remove_callback(SLAPI_OPERATION_SEARCH, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_BASE, "(objectclass=*)", ldbm_back_monitor_instance_search);
- slapi_config_remove_callback(SLAPI_OPERATION_ADD, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_SUBTREE, "(objectclass=*)", ldbm_instance_deny_config);
- slapi_config_remove_callback(SLAPI_OPERATION_MODIFY, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_BASE, "(objectclass=*)", ldbm_instance_deny_config);
- slapi_ch_free_string(&dn);
/* now the cn=index entries */
dn = slapi_create_dn_string("cn=index,cn=%s,cn=%s,cn=plugins,cn=config",
diff --git a/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h b/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h
index 6841dd1..e552ebc 100644
--- a/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h
+++ b/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h
@@ -479,14 +479,6 @@ void ldbm_back_prev_search_results(Slapi_PBlock *pb);
int ldbm_back_isinitialized(void);
/*
- * monitor.c
- */
-
-int ldbm_back_monitor_search(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *entryAfter, int *returncode, char *returntext, void *arg);
-int ldbm_back_monitor_instance_search(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *entryAfter, int *returncode, char *returntext, void *arg);
-int ldbm_back_dbmonitor_search(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *entryAfter, int *returncode, char *returntext, void *arg);
-
-/*
* vlv.c
*/
struct vlv_request
diff --git a/src/lib389/lib389/backend.py b/src/lib389/lib389/backend.py
index 8863ad1..2ebc03f 100644
--- a/src/lib389/lib389/backend.py
+++ b/src/lib389/lib389/backend.py
@@ -1035,6 +1035,10 @@ class DatabaseConfig(DSLdapObject):
vo = vo[0]
self._instance.log.info(f'{k}: {vo}')
+ def get_db_lib(self):
+ """Return the backend library, bdb, lmdb, etc"""
+ return self._db_lib
+
def set(self, value_pairs):
for attr, val in value_pairs:
attr = attr.lower()
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 8 months
[389-ds-base] branch 389-ds-base-1.4.3 updated: Issue 51000 - Separate the BDB backend monitors
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
mreynolds pushed a commit to branch 389-ds-base-1.4.3
in repository 389-ds-base.
The following commit(s) were added to refs/heads/389-ds-base-1.4.3 by this push:
new 7ab8bad Issue 51000 - Separate the BDB backend monitors
7ab8bad is described below
commit 7ab8badd28604063fb7b296d18189b24ba87801d
Author: Mark Reynolds <mreynolds(a)redhat.com>
AuthorDate: Mon Jul 6 15:18:21 2020 -0400
Issue 51000 - Separate the BDB backend monitors
Bug Description: While trying to remove duplicate code from the backend
and BDB backend code, I found that we were not correctly
separating the BDB monitors from the core backend code.
Fix Description: Move all the monitor registering to the db_layer private
structure. This way we have fully isolated the monitors
for each backend implementation/library. This also removed
some duplicate code from the core backend and BDB code.
relates: https://pagure.io/389-ds-base/issue/51000
Reviewed by: spichugi(Thanks!)
(cherry picked from commit ed5b13cab68f82ae6a1abe03bf41eb7369c30f2c)
---
dirsrvtests/tests/suites/monitor/monitor_test.py | 82 ++++++++++++++++++++++
ldap/servers/slapd/back-ldbm/db-bdb/bdb_config.c | 72 ++++++++++++++++++-
.../slapd/back-ldbm/db-bdb/bdb_instance_config.c | 2 +
ldap/servers/slapd/back-ldbm/db-bdb/bdb_layer.h | 8 ++-
ldap/servers/slapd/back-ldbm/db-bdb/bdb_ldif2db.c | 2 +-
ldap/servers/slapd/back-ldbm/db-bdb/bdb_monitor.c | 28 ++++----
ldap/servers/slapd/back-ldbm/db-bdb/bdb_verify.c | 2 +-
ldap/servers/slapd/back-ldbm/dblayer.c | 3 +-
ldap/servers/slapd/back-ldbm/dblayer.h | 1 +
ldap/servers/slapd/back-ldbm/ldbm_config.c | 32 ---------
.../servers/slapd/back-ldbm/ldbm_instance_config.c | 47 +------------
ldap/servers/slapd/back-ldbm/proto-back-ldbm.h | 8 ---
src/lib389/lib389/backend.py | 4 ++
13 files changed, 186 insertions(+), 105 deletions(-)
diff --git a/dirsrvtests/tests/suites/monitor/monitor_test.py b/dirsrvtests/tests/suites/monitor/monitor_test.py
index 5786903..98dd62d 100644
--- a/dirsrvtests/tests/suites/monitor/monitor_test.py
+++ b/dirsrvtests/tests/suites/monitor/monitor_test.py
@@ -2,6 +2,7 @@ import logging
import pytest
import os
from lib389.monitor import *
+from lib389.backend import Backends, DatabaseConfig
from lib389._constants import *
from lib389.topologies import topology_st as topo
@@ -63,6 +64,87 @@ def test_monitor(topo):
log.info('dtablesize: {0[0]},readwaiters: {0[1]},entriessent: {0[2]},bytessent: {0[3]},currenttime: {0[4]},starttime: {0[5]}'.format(stats))
+pytestmark = pytest.mark.tier1
+def test_monitor_ldbm(topo):
+ """This test is to check if we are getting the correct monitor entry
+
+ :id: e62ba369-32f5-4b03-8865-f597a5bb6a70
+ :setup: Single instance
+ :steps:
+ 1. Get the backend library (bdb, ldbm, etc)
+ 2. Get the database monitor
+ 3. Check for expected attributes in output
+ 4. Check for expected DB library specific attributes
+ :expectedresults:
+ 1. Success
+ 2. Success
+ 3. Success
+ 4. Success
+ """
+
+ # Are we using BDB?
+ db_config = DatabaseConfig(topo.standalone)
+ db_lib = db_config.get_db_lib()
+
+ # Get the database monitor entry
+ monitor = MonitorLDBM(topo.standalone).get_status()
+
+ # Check that known attributes exist (only NDN cache stats)
+ assert 'normalizeddncachehits' in monitor
+
+ # Check for library specific attributes
+ if db_lib == 'bdb':
+ assert 'dbcachehits' in monitor
+ assert 'nsslapd-db-configured-locks' in monitor
+ elif db_lib == 'lmdb':
+ pass
+ else:
+ # Unknown - the server would probably fail to start but check it anyway
+ log.fatal(f'Unknown backend library: {db_lib}')
+ assert False
+
+
+pytestmark = pytest.mark.tier1
+def test_monitor_backend(topo):
+ """This test is to check if we are getting the correct backend monitor entry
+
+ :id: 27b0534f-a18c-4c95-aa2b-936bc1886a7b
+ :setup: Single instance
+ :steps:
+ 1. Get the backend library (bdb, ldbm, etc)
+ 2. Get the backend monitor
+ 3. Check for expected attributes in output
+ 4. Check for expected DB library specific attributes
+ :expectedresults:
+ 1. Success
+ 2. Success
+ 3. Success
+ 4. Success
+ """
+
+ # Are we using BDB?
+ db_config = DatabaseConfig(topo.standalone)
+ db_lib = db_config.get_db_lib()
+
+ # Get the backend monitor
+ be = Backends(topo.standalone).list()[0]
+ monitor = be.get_monitor().get_status()
+
+ # Check for expected attributes
+ assert 'entrycachehits' in monitor
+ assert 'dncachehits' in monitor
+
+ # Check for library specific attributes
+ if db_lib == 'bdb':
+ assert 'dbfilename-0' in monitor
+ elif db_lib == 'lmdb':
+ pass
+ else:
+ # Unknown - the server would probably fail to start but check it anyway
+ log.fatal(f'Unknown backend library: {db_lib}')
+ assert False
+
+
if __name__ == '__main__':
# Run isolated
# -s for DEBUG mode
diff --git a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_config.c b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_config.c
index 7c05d2f..738b841 100644
--- a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_config.c
+++ b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_config.c
@@ -88,6 +88,7 @@ int bdb_init(struct ldbminfo *li, config_info *config_array)
priv->instance_postdel_config_fn = &bdb_instance_post_delete_instance_entry_callback;
priv->instance_cleanup_fn = &bdb_instance_cleanup;
priv->instance_create_fn = &bdb_instance_create;
+ priv->instance_register_monitor_fn = &bdb_instance_register_monitor;
priv->instance_search_callback_fn = &bdb_instance_search_callback;
priv->dblayer_auto_tune_fn = &bdb_start_autotune;
return 0;
@@ -1530,6 +1531,7 @@ bail:
}
return rval;
}
+
/* Reads in any config information held in the dse for the bdb
* implementation of the ldbm plugin.
* Creates dse entries used to configure the ldbm plugin and dblayer
@@ -1640,7 +1642,7 @@ retry:
* Now still using ldbm functions
*/
slapi_config_register_callback(SLAPI_OPERATION_SEARCH, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_BASE, "(objectclass=*)", ldbm_back_monitor_search,
+ LDAP_SCOPE_BASE, "(objectclass=*)", bdb_monitor_search,
(void *)li);
slapi_ch_free_string(&dn);
@@ -1656,7 +1658,7 @@ retry:
goto bail;
}
slapi_config_register_callback(SLAPI_OPERATION_SEARCH, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_BASE, "(objectclass=*)", ldbm_back_dbmonitor_search,
+ LDAP_SCOPE_BASE, "(objectclass=*)", bdb_dbmonitor_search,
(void *)li);
bail:
@@ -1664,6 +1666,72 @@ bail:
return rval;
}
+/* general-purpose callback to deny an operation */
+static int
+bdb_deny_config(Slapi_PBlock *pb __attribute__((unused)),
+ Slapi_Entry *e __attribute__((unused)),
+ Slapi_Entry *entryAfter __attribute__((unused)),
+ int *returncode,
+ char *returntext __attribute__((unused)),
+ void *arg __attribute__((unused)))
+{
+ *returncode = LDAP_UNWILLING_TO_PERFORM;
+ return SLAPI_DSE_CALLBACK_ERROR;
+}
+
+int
+bdb_instance_register_monitor(ldbm_instance *inst) {
+ struct ldbminfo *li = inst->inst_li;
+ char *dn = NULL;
+
+ dn = slapi_create_dn_string("cn=monitor,cn=%s,cn=%s,cn=plugins,cn=config",
+ inst->inst_name, li->li_plugin->plg_name);
+ if (NULL == dn) {
+ slapi_log_err(SLAPI_LOG_ERR,
+ "bdb_instance_register_monitor",
+ "failed create monitor instance dn for plugin %s, "
+ "instance %s\n",
+ inst->inst_li->li_plugin->plg_name, inst->inst_name);
+ return 1;
+ }
+ /* make callback on search; deny add/modify/delete */
+ slapi_config_register_callback(SLAPI_OPERATION_SEARCH, DSE_FLAG_PREOP, dn,
+ LDAP_SCOPE_BASE, "(objectclass=*)", bdb_monitor_instance_search,
+ (void *)inst);
+ slapi_config_register_callback(SLAPI_OPERATION_ADD, DSE_FLAG_PREOP, dn,
+ LDAP_SCOPE_SUBTREE, "(objectclass=*)", bdb_deny_config,
+ (void *)inst);
+ slapi_config_register_callback(SLAPI_OPERATION_MODIFY, DSE_FLAG_PREOP, dn,
+ LDAP_SCOPE_BASE, "(objectclass=*)", bdb_deny_config,
+ (void *)inst);
+ slapi_ch_free_string(&dn);
+
+ return 0;
+}
+
+void
+bdb_instance_unregister_monitor(ldbm_instance *inst) {
+ struct ldbminfo *li = inst->inst_li;
+ char *dn = NULL;
+
+ dn = slapi_create_dn_string("cn=monitor,cn=%s,cn=%s,cn=plugins,cn=config",
+ inst->inst_name, li->li_plugin->plg_name);
+ if (NULL == dn) {
+ slapi_log_err(SLAPI_LOG_ERR,
+ "bdb_instance_unregister_monitor",
+ "Failed create monitor instance dn for plugin %s, "
+ "instance %s\n",
+ inst->inst_li->li_plugin->plg_name, inst->inst_name);
+ return;
+ }
+ slapi_config_remove_callback(SLAPI_OPERATION_SEARCH, DSE_FLAG_PREOP, dn,
+ LDAP_SCOPE_BASE, "(objectclass=*)", bdb_monitor_instance_search);
+ slapi_config_remove_callback(SLAPI_OPERATION_ADD, DSE_FLAG_PREOP, dn,
+ LDAP_SCOPE_SUBTREE, "(objectclass=*)", bdb_deny_config);
+ slapi_config_remove_callback(SLAPI_OPERATION_MODIFY, DSE_FLAG_PREOP, dn,
+ LDAP_SCOPE_BASE, "(objectclass=*)", bdb_deny_config);
+ slapi_ch_free_string(&dn);
+}
/* Utility function used in creating config entries. Using the
* config_info, this function gets info and formats in the correct
diff --git a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_instance_config.c b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_instance_config.c
index 6a21c22..0ac3694 100644
--- a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_instance_config.c
+++ b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_instance_config.c
@@ -197,6 +197,8 @@ bdb_instance_post_delete_instance_entry_callback(struct ldbminfo *li, struct ldb
if (inst_dirp != inst_dir) {
slapi_ch_free_string(&inst_dirp);
}
+ /* unregister the monitor */
+ bdb_instance_unregister_monitor(inst);
} /* non-null pEnv */
return SLAPI_DSE_CALLBACK_OK;
}
diff --git a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_layer.h b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_layer.h
index d729327..bf00d2e 100644
--- a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_layer.h
+++ b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_layer.h
@@ -150,7 +150,6 @@ int bdb_back_ok_to_dump(const char *dn, char **include, char **exclude);
int bdb_back_fetch_incl_excl(Slapi_PBlock *pb, char ***include, char ***exclude);
PRUint64 bdb_get_id2entry_size(ldbm_instance *inst);
-
/* bdb version functions */
int bdb_version_write(struct ldbminfo *li, const char *directory, const char *dataversion, PRUint32 flags);
int bdb_version_read(struct ldbminfo *li, const char *directory, char **ldbmversion, char **dataversion);
@@ -162,3 +161,10 @@ int bdb_instance_post_delete_instance_entry_callback(struct ldbminfo *li, struct
int bdb_instance_add_instance_entry_callback(struct ldbminfo *li, struct ldbm_instance *inst);
int bdb_instance_postadd_instance_entry_callback(struct ldbminfo *li, struct ldbm_instance *inst);
void bdb_config_setup_default(struct ldbminfo *li);
+
+/* monitor functions */
+int bdb_monitor_instance_search(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *entryAfter, int *returncode, char *returntext, void *arg);
+int bdb_monitor_search(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *entryAfter, int *returncode, char *returntext, void *arg);
+int bdb_dbmonitor_search(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *entryAfter, int *returncode, char *returntext, void *arg);
+int bdb_instance_register_monitor(ldbm_instance *inst);
+void bdb_instance_unregister_monitor(ldbm_instance *inst);
diff --git a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_ldif2db.c b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_ldif2db.c
index 9ffd877..506c285 100644
--- a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_ldif2db.c
+++ b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_ldif2db.c
@@ -3145,7 +3145,7 @@ bdb_upgradednformat(Slapi_PBlock *pb)
run_from_cmdline = (task_flags & SLAPI_TASK_RUNNING_FROM_COMMANDLINE);
slapi_pblock_get(pb, SLAPI_PLUGIN_PRIVATE, &li);
if (run_from_cmdline) {
- ldbm_config_load_dse_info(li);
+ bdb_config_load_dse_info(li);
if (bdb_check_and_set_import_cache(li) < 0) {
return -1;
}
diff --git a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_monitor.c b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_monitor.c
index ac3479d..8f954c4 100644
--- a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_monitor.c
+++ b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_monitor.c
@@ -33,12 +33,12 @@
/* DSE callback to monitor stats for a particular instance */
int
-ldbm_back_monitor_instance_search(Slapi_PBlock *pb __attribute__((unused)),
- Slapi_Entry *e,
- Slapi_Entry *entryAfter __attribute__((unused)),
- int *returncode,
- char *returntext __attribute__((unused)),
- void *arg)
+bdb_monitor_instance_search(Slapi_PBlock *pb __attribute__((unused)),
+ Slapi_Entry *e,
+ Slapi_Entry *entryAfter __attribute__((unused)),
+ int *returncode,
+ char *returntext __attribute__((unused)),
+ void *arg)
{
ldbm_instance *inst = (ldbm_instance *)arg;
struct ldbminfo *li = NULL;
@@ -150,8 +150,8 @@ ldbm_back_monitor_instance_search(Slapi_PBlock *pb __attribute__((unused)),
continue;
/* Since the filenames are now relative, we need to construct an absolute version
- * for the purpose of stat() etc below...
- */
+ * for the purpose of stat() etc below...
+ */
slapi_ch_free_string(&absolute_pathname);
absolute_pathname = slapi_ch_smprintf("%s%c%s", inst->inst_parent_dir_name, get_sep(inst->inst_parent_dir_name), mpfstat[i]->file_name);
@@ -159,10 +159,10 @@ ldbm_back_monitor_instance_search(Slapi_PBlock *pb __attribute__((unused)),
if (stat(absolute_pathname, &astat))
continue;
/* If the file has been re-created after been deleted
- * We should show only statistics for the last instance
- * Since SleepyCat returns the statistic of the last open file first,
- * we should only display the first statistic record for a given file
- */
+ * We should show only statistics for the last instance
+ * Since SleepyCat returns the statistic of the last open file first,
+ * we should only display the first statistic record for a given file
+ */
for (j = 0; j < i; j++)
if (!strcmp(mpfstat[i]->file_name, mpfstat[j]->file_name))
break;
@@ -194,7 +194,7 @@ ldbm_back_monitor_instance_search(Slapi_PBlock *pb __attribute__((unused)),
/* monitor global ldbm stats */
int
-ldbm_back_monitor_search(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *entryAfter, int *returncode, char *returntext, void *arg)
+bdb_monitor_search(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *entryAfter, int *returncode, char *returntext, void *arg)
{
struct ldbminfo *li = (struct ldbminfo *)arg;
struct berval val;
@@ -288,7 +288,7 @@ ldbm_back_monitor_search(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *entryAft
/* monitor global ldbm database stats */
int
-ldbm_back_dbmonitor_search(Slapi_PBlock *pb __attribute__((unused)),
+bdb_dbmonitor_search(Slapi_PBlock *pb __attribute__((unused)),
Slapi_Entry *e,
Slapi_Entry *entryAfter __attribute__((unused)),
int *returncode,
diff --git a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_verify.c b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_verify.c
index 32933a8..42d1cd1 100644
--- a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_verify.c
+++ b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_verify.c
@@ -171,7 +171,7 @@ bdb_verify(Slapi_PBlock *pb)
slapi_pblock_get(pb, SLAPI_SEQ_TYPE, &verbose);
slapi_pblock_get(pb, SLAPI_PLUGIN_PRIVATE, &li);
slapi_pblock_get(pb, SLAPI_DBVERIFY_DBDIR, &dbdir);
- ldbm_config_load_dse_info(li);
+ bdb_config_load_dse_info(li);
bdb_config_internal_set(li, CONFIG_DB_TRANSACTION_LOGGING, "off");
/* no write needed; choose EXPORT MODE */
diff --git a/ldap/servers/slapd/back-ldbm/dblayer.c b/ldap/servers/slapd/back-ldbm/dblayer.c
index c9f43c7..05cc5b8 100644
--- a/ldap/servers/slapd/back-ldbm/dblayer.c
+++ b/ldap/servers/slapd/back-ldbm/dblayer.c
@@ -273,10 +273,9 @@ dblayer_setup(struct ldbminfo *li)
}
ldbm_config_load_dse_info(li);
-
priv = (dblayer_private *)li->li_dblayer_private;
-
rc = priv->dblayer_load_dse_fn(li);
+
return rc;
}
diff --git a/ldap/servers/slapd/back-ldbm/dblayer.h b/ldap/servers/slapd/back-ldbm/dblayer.h
index 8e156dd..cd53f25 100644
--- a/ldap/servers/slapd/back-ldbm/dblayer.h
+++ b/ldap/servers/slapd/back-ldbm/dblayer.h
@@ -147,6 +147,7 @@ struct dblayer_private
instance_config_entry_callback_fn_t *instance_postdel_config_fn;
instance_cleanup_fn_t *instance_cleanup_fn;
instance_create_fn_t *instance_create_fn;
+ instance_create_fn_t *instance_register_monitor_fn;
instance_search_callback_fn_t *instance_search_callback_fn;
dblayer_auto_tune_fn_t *dblayer_auto_tune_fn;
};
diff --git a/ldap/servers/slapd/back-ldbm/ldbm_config.c b/ldap/servers/slapd/back-ldbm/ldbm_config.c
index 88c1863..3fe86d5 100644
--- a/ldap/servers/slapd/back-ldbm/ldbm_config.c
+++ b/ldap/servers/slapd/back-ldbm/ldbm_config.c
@@ -1113,38 +1113,6 @@ ldbm_config_load_dse_info(struct ldbminfo *li)
(void *)li);
slapi_ch_free_string(&dn);
- /* setup the dse callback functions for the ldbm backend monitor entry */
- dn = slapi_create_dn_string("cn=monitor,cn=%s,cn=plugins,cn=config",
- li->li_plugin->plg_name);
- if (NULL == dn) {
- slapi_log_err(SLAPI_LOG_ERR,
- "ldbm_config_load_dse_info",
- "failed to create monitor dn for %s\n",
- li->li_plugin->plg_name);
- rval = 1;
- goto bail;
- }
- slapi_config_register_callback(SLAPI_OPERATION_SEARCH, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_BASE, "(objectclass=*)", ldbm_back_monitor_search,
- (void *)li);
- slapi_ch_free_string(&dn);
-
- /* And the ldbm backend database monitor entry */
- dn = slapi_create_dn_string("cn=database,cn=monitor,cn=%s,cn=plugins,cn=config",
- li->li_plugin->plg_name);
- if (NULL == dn) {
- slapi_log_err(SLAPI_LOG_ERR,
- "ldbm_config_load_dse_info",
- "failed create monitor database dn for %s\n",
- li->li_plugin->plg_name);
- rval = 1;
- goto bail;
- }
- slapi_config_register_callback(SLAPI_OPERATION_SEARCH, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_BASE, "(objectclass=*)", ldbm_back_dbmonitor_search,
- (void *)li);
- slapi_ch_free_string(&dn);
-
/* setup the dse callback functions for the ldbm backend instance
* entries */
dn = slapi_create_dn_string("cn=%s,cn=plugins,cn=config",
diff --git a/ldap/servers/slapd/back-ldbm/ldbm_instance_config.c b/ldap/servers/slapd/back-ldbm/ldbm_instance_config.c
index 36ec112..56bf35f 100644
--- a/ldap/servers/slapd/back-ldbm/ldbm_instance_config.c
+++ b/ldap/servers/slapd/back-ldbm/ldbm_instance_config.c
@@ -675,32 +675,6 @@ ldbm_instance_config_load_dse_info(ldbm_instance *inst)
LDAP_SCOPE_BASE, "(objectclass=*)",
ldbm_instance_deny_config, (void *)inst);
/* delete is handled by a callback set in ldbm_config.c */
-
- slapi_ch_free_string(&dn);
-
- /* don't forget the monitor! */
- dn = slapi_create_dn_string("cn=monitor,cn=%s,cn=%s,cn=plugins,cn=config",
- inst->inst_name, li->li_plugin->plg_name);
- if (NULL == dn) {
- slapi_log_err(SLAPI_LOG_ERR,
- "ldbm_instance_config_load_dse_info",
- "failed create monitor instance dn for plugin %s, "
- "instance %s\n",
- inst->inst_li->li_plugin->plg_name, inst->inst_name);
- rval = 1;
- goto bail;
- }
- /* make callback on search; deny add/modify/delete */
- slapi_config_register_callback(SLAPI_OPERATION_SEARCH, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_BASE, "(objectclass=*)", ldbm_back_monitor_instance_search,
- (void *)inst);
- slapi_config_register_callback(SLAPI_OPERATION_ADD, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_SUBTREE, "(objectclass=*)", ldbm_instance_deny_config,
- (void *)inst);
- slapi_config_register_callback(SLAPI_OPERATION_MODIFY, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_BASE, "(objectclass=*)", ldbm_instance_deny_config,
- (void *)inst);
- /* delete is okay */
slapi_ch_free_string(&dn);
/* Callbacks to handle indexes */
@@ -908,6 +882,7 @@ static int
ldbm_instance_generate(struct ldbminfo *li, char *instance_name, Slapi_Backend **ret_be)
{
Slapi_Backend *new_be = NULL;
+ dblayer_private *priv = (dblayer_private *)li->li_dblayer_private;
int rc = 0;
/* Create a new instance, process config info for it,
@@ -922,6 +897,8 @@ ldbm_instance_generate(struct ldbminfo *li, char *instance_name, Slapi_Backend *
}
ldbm_instance_config_load_dse_info(new_be->be_instance_info);
+ priv->instance_register_monitor_fn(new_be->be_instance_info);
+
ldbm_instance_create_default_indexes(new_be);
/* if USN plugin is enabled, set slapi_counter */
@@ -984,24 +961,6 @@ ldbm_instance_unregister_callbacks(ldbm_instance *inst)
ldbm_instance_deny_config);
slapi_ch_free_string(&dn);
- /* now the cn=monitor entry */
- dn = slapi_create_dn_string("cn=monitor,cn=%s,cn=%s,cn=plugins,cn=config",
- inst->inst_name, li->li_plugin->plg_name);
- if (NULL == dn) {
- slapi_log_err(SLAPI_LOG_ERR,
- "ldbm_instance_unregister_callbacks",
- "Failed create monitor instance dn for plugin %s, "
- "instance %s\n",
- inst->inst_li->li_plugin->plg_name, inst->inst_name);
- goto bail;
- }
- slapi_config_remove_callback(SLAPI_OPERATION_SEARCH, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_BASE, "(objectclass=*)", ldbm_back_monitor_instance_search);
- slapi_config_remove_callback(SLAPI_OPERATION_ADD, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_SUBTREE, "(objectclass=*)", ldbm_instance_deny_config);
- slapi_config_remove_callback(SLAPI_OPERATION_MODIFY, DSE_FLAG_PREOP, dn,
- LDAP_SCOPE_BASE, "(objectclass=*)", ldbm_instance_deny_config);
- slapi_ch_free_string(&dn);
/* now the cn=index entries */
dn = slapi_create_dn_string("cn=index,cn=%s,cn=%s,cn=plugins,cn=config",
diff --git a/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h b/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h
index 43793f6..5d618a8 100644
--- a/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h
+++ b/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h
@@ -480,14 +480,6 @@ void ldbm_back_prev_search_results(Slapi_PBlock *pb);
int ldbm_back_isinitialized(void);
/*
- * monitor.c
- */
-
-int ldbm_back_monitor_search(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *entryAfter, int *returncode, char *returntext, void *arg);
-int ldbm_back_monitor_instance_search(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *entryAfter, int *returncode, char *returntext, void *arg);
-int ldbm_back_dbmonitor_search(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *entryAfter, int *returncode, char *returntext, void *arg);
-
-/*
* vlv.c
*/
struct vlv_request
diff --git a/src/lib389/lib389/backend.py b/src/lib389/lib389/backend.py
index 8863ad1..2ebc03f 100644
--- a/src/lib389/lib389/backend.py
+++ b/src/lib389/lib389/backend.py
@@ -1035,6 +1035,10 @@ class DatabaseConfig(DSLdapObject):
vo = vo[0]
self._instance.log.info(f'{k}: {vo}')
+ def get_db_lib(self):
+ """Return the backend library, bdb, lmdb, etc"""
+ return self._db_lib
+
def set(self, value_pairs):
for attr, val in value_pairs:
attr = attr.lower()
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 8 months
[389-ds-base] branch 389-ds-base-1.4.3 updated: Issue 49300 - entryUSN is duplicated after memberOf operation
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
spichugi pushed a commit to branch 389-ds-base-1.4.3
in repository 389-ds-base.
The following commit(s) were added to refs/heads/389-ds-base-1.4.3 by this push:
new f7f79ae Issue 49300 - entryUSN is duplicated after memberOf operation
f7f79ae is described below
commit f7f79ae57e61aeba53e91f0b4329dd84b38721a3
Author: Simon Pichugin <spichugi(a)redhat.com>
AuthorDate: Tue Jun 30 15:39:30 2020 +0200
Issue 49300 - entryUSN is duplicated after memberOf operation
Bug Description: When we assign a member to a group we have two
oprations - group modification and user modification.
As a result, they both have the same entryUSN because USN Plugin
assigns entryUSN value in bepreop but increments the counter
in the postop and a lot of things can happen in between.
Fix Description: Increment the counter in bepreop together with
entryUSN assignment. Also, decrement the counter in bepostop if
the failuer has happened.
Add test suite to cover the change.
https://pagure.io/389-ds-base/issue/49300
Reviewed by: tbordaz (Thanks!)
---
dirsrvtests/tests/suites/plugins/entryusn_test.py | 240 ++++++++++++++++++++++
ldap/servers/plugins/usn/usn.c | 109 ++++++----
ldap/servers/slapd/pblock.c | 14 +-
ldap/servers/slapd/pblock_v3.h | 1 +
ldap/servers/slapd/slapi-plugin.h | 3 +
5 files changed, 322 insertions(+), 45 deletions(-)
diff --git a/dirsrvtests/tests/suites/plugins/entryusn_test.py b/dirsrvtests/tests/suites/plugins/entryusn_test.py
new file mode 100644
index 0000000..7213154
--- /dev/null
+++ b/dirsrvtests/tests/suites/plugins/entryusn_test.py
@@ -0,0 +1,240 @@
+# --- BEGIN COPYRIGHT BLOCK ---
+# Copyright (C) 2020 Red Hat, Inc.
+# All rights reserved.
+#
+# License: GPL (version 3 or any later version).
+# See LICENSE for details.
+# --- END COPYRIGHT BLOCK ---
+#
+import ldap
+import logging
+import pytest
+from lib389._constants import DEFAULT_SUFFIX
+from lib389.config import Config
+from lib389.plugins import USNPlugin, MemberOfPlugin
+from lib389.idm.group import Groups
+from lib389.idm.user import UserAccounts
+from lib389.idm.organizationalunit import OrganizationalUnit
+from lib389.tombstone import Tombstones
+from lib389.rootdse import RootDSE
+from lib389.topologies import topology_st, topology_m2
+
+log = logging.getLogger(__name__)
+
+USER_NUM = 10
+GROUP_NUM = 3
+
+
+def check_entryusn_no_duplicates(entryusn_list):
+ """Check that all values in the list are unique"""
+
+ if len(entryusn_list) > len(set(entryusn_list)):
+ raise AssertionError(f"EntryUSN values have duplicates, please, check logs")
+
+
+def check_lastusn_after_restart(inst):
+ """Check that last usn is the same after restart"""
+
+ root_dse = RootDSE(inst)
+ last_usn_before = root_dse.get_attr_val_int("lastusn;userroot")
+ inst.restart()
+ last_usn_after = root_dse.get_attr_val_int("lastusn;userroot")
+ assert last_usn_after == last_usn_before
+
+
+(a)pytest.fixture(scope="module")
+def setup(topology_st, request):
+ """
+ Enable USN plug-in
+ Enable MEMBEROF plugin
+ Add test entries
+ """
+
+ inst = topology_st.standalone
+
+ log.info("Enable the USN plugin...")
+ plugin = USNPlugin(inst)
+ plugin.enable()
+
+ log.info("Enable the MEMBEROF plugin...")
+ plugin = MemberOfPlugin(inst)
+ plugin.enable()
+
+ inst.restart()
+
+ users_list = []
+ log.info("Adding test entries...")
+ users = UserAccounts(inst, DEFAULT_SUFFIX)
+ for id in range(USER_NUM):
+ user = users.create_test_user(uid=id)
+ users_list.append(user)
+
+ groups_list = []
+ log.info("Adding test groups...")
+ groups = Groups(inst, DEFAULT_SUFFIX)
+ for id in range(GROUP_NUM):
+ group = groups.create(properties={'cn': f'test_group{id}'})
+ groups_list.append(group)
+
+ def fin():
+ for user in users_list:
+ try:
+ user.delete()
+ except ldap.NO_SUCH_OBJECT:
+ pass
+ for group in groups_list:
+ try:
+ group.delete()
+ except ldap.NO_SUCH_OBJECT:
+ pass
+ request.addfinalizer(fin)
+
+ return {"users": users_list,
+ "groups": groups_list}
+
+
+def test_entryusn_no_duplicates(topology_st, setup):
+ """Verify that entryUSN is not duplicated after memberOf operation
+
+ :id: 1a7d382d-1214-4d56-b9c2-9c4ed57d1683
+ :setup: Standalone instance, Groups and Users, USN and memberOf are enabled
+ :steps:
+ 1. Add a member to group 1
+ 2. Add a member to group 1 and 2
+ 3. Check that entryUSNs are different
+ 4. Check that lastusn before and after a restart are the same
+ :expectedresults:
+ 1. Success
+ 2. Success
+ 3. Success
+ 4. Success
+ """
+
+ inst = topology_st.standalone
+ config = Config(inst)
+ config.replace('nsslapd-accesslog-level', '260') # Internal op
+ config.replace('nsslapd-errorlog-level', '65536')
+ config.replace('nsslapd-plugin-logging', 'on')
+ entryusn_list = []
+
+ users = setup["users"]
+ groups = setup["groups"]
+
+ groups[0].replace('member', users[0].dn)
+ entryusn_list.append(users[0].get_attr_val_int('entryusn'))
+ log.info(f"{users[0].dn}_1: {entryusn_list[-1:]}")
+ entryusn_list.append(groups[0].get_attr_val_int('entryusn'))
+ log.info(f"{groups[0].dn}_1: {entryusn_list[-1:]}")
+ check_entryusn_no_duplicates(entryusn_list)
+
+ groups[1].replace('member', [users[0].dn, users[1].dn])
+ entryusn_list.append(users[0].get_attr_val_int('entryusn'))
+ log.info(f"{users[0].dn}_2: {entryusn_list[-1:]}")
+ entryusn_list.append(users[1].get_attr_val_int('entryusn'))
+ log.info(f"{users[1].dn}_2: {entryusn_list[-1:]}")
+ entryusn_list.append(groups[1].get_attr_val_int('entryusn'))
+ log.info(f"{groups[1].dn}_2: {entryusn_list[-1:]}")
+ check_entryusn_no_duplicates(entryusn_list)
+
+ check_lastusn_after_restart(inst)
+
+
+def test_entryusn_is_same_after_failure(topology_st, setup):
+ """Verify that entryUSN is the same after failed operation
+
+ :id: 1f227533-370a-48c1-b920-9b3b0bcfc32e
+ :setup: Standalone instance, Groups and Users, USN and memberOf are enabled
+ :steps:
+ 1. Get current group's entryUSN value
+ 2. Try to modify the group with an invalid syntax
+ 3. Get new group's entryUSN value and compare with old
+ 4. Check that lastusn before and after a restart are the same
+ :expectedresults:
+ 1. Success
+ 2. Invalid Syntax error
+ 3. Should be the same
+ 4. Success
+ """
+
+ inst = topology_st.standalone
+ users = setup["users"]
+
+ # We need this update so we get the latest USN pointed to our entry
+ users[0].replace('description', 'update')
+
+ entryusn_before = users[0].get_attr_val_int('entryusn')
+ users[0].replace('description', 'update')
+ try:
+ users[0].replace('uid', 'invalid update')
+ except ldap.NOT_ALLOWED_ON_RDN:
+ pass
+ users[0].replace('description', 'second update')
+ entryusn_after = users[0].get_attr_val_int('entryusn')
+
+ # entryUSN should be OLD + 2 (only two user updates)
+ assert entryusn_after == (entryusn_before + 2)
+
+ check_lastusn_after_restart(inst)
+
+
+def test_entryusn_after_repl_delete(topology_m2):
+ """Verify that entryUSN is incremented on 1 after delete operation which creates a tombstone
+
+ :id: 1704cf65-41bc-4347-bdaf-20fc2431b218
+ :setup: An instance with replication, Users, USN enabled
+ :steps:
+ 1. Try to delete a user
+ 2. Check the tombstone has the incremented USN
+ 3. Try to delete ou=People with users
+ 4. Check the entry has a not incremented entryUSN
+ :expectedresults:
+ 1. Success
+ 2. Success
+ 3. Should fail with Not Allowed On Non-leaf error
+ 4. Success
+ """
+
+ inst = topology_m2.ms["master1"]
+ plugin = USNPlugin(inst)
+ plugin.enable()
+ inst.restart()
+ users = UserAccounts(inst, DEFAULT_SUFFIX)
+
+ try:
+ user_1 = users.create_test_user()
+ user_rdn = user_1.rdn
+ tombstones = Tombstones(inst, DEFAULT_SUFFIX)
+
+ user_1.replace('description', 'update_ts')
+ user_usn = user_1.get_attr_val_int('entryusn')
+
+ user_1.delete()
+
+ ts = tombstones.get(user_rdn)
+ ts_usn = ts.get_attr_val_int('entryusn')
+
+ assert (user_usn + 1) == ts_usn
+
+ user_1 = users.create_test_user()
+ org = OrganizationalUnit(inst, f"ou=People,{DEFAULT_SUFFIX}")
+ org.replace('description', 'update_ts')
+ ou_usn_before = org.get_attr_val_int('entryusn')
+ try:
+ org.delete()
+ except ldap.NOT_ALLOWED_ON_NONLEAF:
+ pass
+ ou_usn_after = org.get_attr_val_int('entryusn')
+ assert ou_usn_before == ou_usn_after
+
+ finally:
+ try:
+ user_1.delete()
+ except ldap.NO_SUCH_OBJECT:
+ pass
+
+
+if __name__ == '__main__':
+ # Run isolated
+ # -s for DEBUG mode
+ CURRENT_FILE = os.path.realpath(__file__)
+ pytest.main("-s %s" % CURRENT_FILE)
diff --git a/ldap/servers/plugins/usn/usn.c b/ldap/servers/plugins/usn/usn.c
index 12ba040..f2cc8a6 100644
--- a/ldap/servers/plugins/usn/usn.c
+++ b/ldap/servers/plugins/usn/usn.c
@@ -333,6 +333,12 @@ _usn_add_next_usn(Slapi_Entry *e, Slapi_Backend *be)
}
slapi_ch_free_string(&usn_berval.bv_val);
+ /*
+ * increment the counter now and decrement in the bepostop
+ * if the operation will fail
+ */
+ slapi_counter_increment(be->be_usn_counter);
+
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
"<-- _usn_add_next_usn\n");
@@ -370,6 +376,12 @@ _usn_mod_next_usn(LDAPMod ***mods, Slapi_Backend *be)
*mods = slapi_mods_get_ldapmods_passout(&smods);
+ /*
+ * increment the counter now and decrement in the bepostop
+ * if the operation will fail
+ */
+ slapi_counter_increment(be->be_usn_counter);
+
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
"<-- _usn_mod_next_usn\n");
return LDAP_SUCCESS;
@@ -420,6 +432,7 @@ usn_betxnpreop_delete(Slapi_PBlock *pb)
{
Slapi_Entry *e = NULL;
Slapi_Backend *be = NULL;
+ int32_t tombstone_incremented = 0;
int rc = SLAPI_PLUGIN_SUCCESS;
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
@@ -441,7 +454,9 @@ usn_betxnpreop_delete(Slapi_PBlock *pb)
goto bail;
}
_usn_add_next_usn(e, be);
+ tombstone_incremented = 1;
bail:
+ slapi_pblock_set(pb, SLAPI_USN_INCREMENT_FOR_TOMBSTONE, &tombstone_incremented);
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
"<-- usn_betxnpreop_delete\n");
@@ -483,7 +498,7 @@ bail:
return rc;
}
-/* count up the counter */
+/* count down the counter */
static int
usn_bepostop(Slapi_PBlock *pb)
{
@@ -493,25 +508,24 @@ usn_bepostop(Slapi_PBlock *pb)
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
"--> usn_bepostop\n");
- /* if op is not successful, don't increment the counter */
+ /* if op is not successful, decrement the counter, else - do nothing */
slapi_pblock_get(pb, SLAPI_RESULT_CODE, &rc);
if (LDAP_SUCCESS != rc) {
- /* no plugin failure */
- rc = SLAPI_PLUGIN_SUCCESS;
- goto bail;
- }
+ slapi_pblock_get(pb, SLAPI_BACKEND, &be);
+ if (NULL == be) {
+ rc = LDAP_PARAM_ERROR;
+ slapi_pblock_set(pb, SLAPI_RESULT_CODE, &rc);
+ rc = SLAPI_PLUGIN_FAILURE;
+ goto bail;
+ }
- slapi_pblock_get(pb, SLAPI_BACKEND, &be);
- if (NULL == be) {
- rc = LDAP_PARAM_ERROR;
- slapi_pblock_set(pb, SLAPI_RESULT_CODE, &rc);
- rc = SLAPI_PLUGIN_FAILURE;
- goto bail;
+ if (be->be_usn_counter) {
+ slapi_counter_decrement(be->be_usn_counter);
+ }
}
- if (be->be_usn_counter) {
- slapi_counter_increment(be->be_usn_counter);
- }
+ /* no plugin failure */
+ rc = SLAPI_PLUGIN_SUCCESS;
bail:
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
"<-- usn_bepostop\n");
@@ -519,13 +533,14 @@ bail:
return rc;
}
-/* count up the counter */
+/* count down the counter on a failure and mod ignore */
static int
usn_bepostop_modify(Slapi_PBlock *pb)
{
int rc = SLAPI_PLUGIN_FAILURE;
Slapi_Backend *be = NULL;
LDAPMod **mods = NULL;
+ int32_t do_decrement = 0;
int i;
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
@@ -534,9 +549,7 @@ usn_bepostop_modify(Slapi_PBlock *pb)
/* if op is not successful, don't increment the counter */
slapi_pblock_get(pb, SLAPI_RESULT_CODE, &rc);
if (LDAP_SUCCESS != rc) {
- /* no plugin failure */
- rc = SLAPI_PLUGIN_SUCCESS;
- goto bail;
+ do_decrement = 1;
}
slapi_pblock_get(pb, SLAPI_MODIFY_MODS, &mods);
@@ -545,25 +558,29 @@ usn_bepostop_modify(Slapi_PBlock *pb)
if (mods[i]->mod_op & LDAP_MOD_IGNORE) {
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
"usn_bepostop_modify - MOD_IGNORE detected\n");
- goto bail; /* conflict occurred.
- skip incrementing the counter. */
+ do_decrement = 1; /* conflict occurred.
+ decrement he counter. */
} else {
break;
}
}
}
- slapi_pblock_get(pb, SLAPI_BACKEND, &be);
- if (NULL == be) {
- rc = LDAP_PARAM_ERROR;
- slapi_pblock_set(pb, SLAPI_RESULT_CODE, &rc);
- rc = SLAPI_PLUGIN_FAILURE;
- goto bail;
+ if (do_decrement) {
+ slapi_pblock_get(pb, SLAPI_BACKEND, &be);
+ if (NULL == be) {
+ rc = LDAP_PARAM_ERROR;
+ slapi_pblock_set(pb, SLAPI_RESULT_CODE, &rc);
+ rc = SLAPI_PLUGIN_FAILURE;
+ goto bail;
+ }
+ if (be->be_usn_counter) {
+ slapi_counter_decrement(be->be_usn_counter);
+ }
}
- if (be->be_usn_counter) {
- slapi_counter_increment(be->be_usn_counter);
- }
+ /* no plugin failure */
+ rc = SLAPI_PLUGIN_SUCCESS;
bail:
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
"<-- usn_bepostop_modify\n");
@@ -573,34 +590,38 @@ bail:
/* count up the counter */
/* if the op is delete and the op was not successful, remove preventryusn */
+/* the function is executed on TXN level */
static int
usn_bepostop_delete(Slapi_PBlock *pb)
{
int rc = SLAPI_PLUGIN_FAILURE;
Slapi_Backend *be = NULL;
+ int32_t tombstone_incremented = 0;
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
"--> usn_bepostop_delete\n");
- /* if op is not successful, don't increment the counter */
+ /* if op is not successful and it is a tombstone entry, decrement the counter */
slapi_pblock_get(pb, SLAPI_RESULT_CODE, &rc);
if (LDAP_SUCCESS != rc) {
- /* no plugin failure */
- rc = SLAPI_PLUGIN_SUCCESS;
- goto bail;
- }
+ slapi_pblock_get(pb, SLAPI_USN_INCREMENT_FOR_TOMBSTONE, &tombstone_incremented);
+ if (tombstone_incremented) {
+ slapi_pblock_get(pb, SLAPI_BACKEND, &be);
+ if (NULL == be) {
+ rc = LDAP_PARAM_ERROR;
+ slapi_pblock_set(pb, SLAPI_RESULT_CODE, &rc);
+ rc = SLAPI_PLUGIN_FAILURE;
+ goto bail;
+ }
- slapi_pblock_get(pb, SLAPI_BACKEND, &be);
- if (NULL == be) {
- rc = LDAP_PARAM_ERROR;
- slapi_pblock_set(pb, SLAPI_RESULT_CODE, &rc);
- rc = SLAPI_PLUGIN_FAILURE;
- goto bail;
+ if (be->be_usn_counter) {
+ slapi_counter_decrement(be->be_usn_counter);
+ }
+ }
}
- if (be->be_usn_counter) {
- slapi_counter_increment(be->be_usn_counter);
- }
+ /* no plugin failure */
+ rc = SLAPI_PLUGIN_SUCCESS;
bail:
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
"<-- usn_bepostop_delete\n");
diff --git a/ldap/servers/slapd/pblock.c b/ldap/servers/slapd/pblock.c
index cb562e9..454ea9c 100644
--- a/ldap/servers/slapd/pblock.c
+++ b/ldap/servers/slapd/pblock.c
@@ -2436,7 +2436,7 @@ slapi_pblock_get(Slapi_PBlock *pblock, int arg, void *value)
(*(char **)value) = NULL;
}
break;
-
+
case SLAPI_SEARCH_CTRLS:
if (pblock->pb_intop != NULL) {
(*(LDAPControl ***)value) = pblock->pb_intop->pb_search_ctrls;
@@ -2479,6 +2479,14 @@ slapi_pblock_get(Slapi_PBlock *pblock, int arg, void *value)
}
break;
+ case SLAPI_USN_INCREMENT_FOR_TOMBSTONE:
+ if (pblock->pb_intop != NULL) {
+ (*(int32_t *)value) = pblock->pb_intop->pb_usn_tombstone_incremented;
+ } else {
+ (*(int32_t *)value) = 0;
+ }
+ break;
+
/* ACI Target Check */
case SLAPI_ACI_TARGET_CHECK:
if (pblock->pb_misc != NULL) {
@@ -4156,6 +4164,10 @@ slapi_pblock_set(Slapi_PBlock *pblock, int arg, void *value)
pblock->pb_intop->pb_paged_results_cookie = *(int *)value;
break;
+ case SLAPI_USN_INCREMENT_FOR_TOMBSTONE:
+ pblock->pb_intop->pb_usn_tombstone_incremented = *((int32_t *)value);
+ break;
+
/* ACI Target Check */
case SLAPI_ACI_TARGET_CHECK:
_pblock_assert_pb_misc(pblock);
diff --git a/ldap/servers/slapd/pblock_v3.h b/ldap/servers/slapd/pblock_v3.h
index 7ec2f37..90498c0 100644
--- a/ldap/servers/slapd/pblock_v3.h
+++ b/ldap/servers/slapd/pblock_v3.h
@@ -161,6 +161,7 @@ typedef struct _slapi_pblock_intop
int pb_paged_results_index; /* stash SLAPI_PAGED_RESULTS_INDEX */
int pb_paged_results_cookie; /* stash SLAPI_PAGED_RESULTS_COOKIE */
+ int32_t pb_usn_tombstone_incremented; /* stash SLAPI_PAGED_RESULTS_COOKIE */
} slapi_pblock_intop;
/* Stuff that is rarely used, but still present */
diff --git a/ldap/servers/slapd/slapi-plugin.h b/ldap/servers/slapd/slapi-plugin.h
index be6c98b..3aadf38 100644
--- a/ldap/servers/slapd/slapi-plugin.h
+++ b/ldap/servers/slapd/slapi-plugin.h
@@ -7482,6 +7482,9 @@ typedef enum _slapi_op_note_t {
#define SLAPI_PAGED_RESULTS_INDEX 1945
#define SLAPI_PAGED_RESULTS_COOKIE 1949
+/* USN Plugin flag for tombstone entries */
+#define SLAPI_USN_INCREMENT_FOR_TOMBSTONE 1950
+
/* ACI Target Check */
#define SLAPI_ACI_TARGET_CHECK 1946
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 8 months
[389-ds-base] branch 389-ds-base-1.4.2 updated: Issue 49300 - entryUSN is duplicated after memberOf operation
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
spichugi pushed a commit to branch 389-ds-base-1.4.2
in repository 389-ds-base.
The following commit(s) were added to refs/heads/389-ds-base-1.4.2 by this push:
new 14b0dcb Issue 49300 - entryUSN is duplicated after memberOf operation
14b0dcb is described below
commit 14b0dcbaf86b99031b368893566e60ab19e156c7
Author: Simon Pichugin <spichugi(a)redhat.com>
AuthorDate: Tue Jun 30 15:39:30 2020 +0200
Issue 49300 - entryUSN is duplicated after memberOf operation
Bug Description: When we assign a member to a group we have two
oprations - group modification and user modification.
As a result, they both have the same entryUSN because USN Plugin
assigns entryUSN value in bepreop but increments the counter
in the postop and a lot of things can happen in between.
Fix Description: Increment the counter in bepreop together with
entryUSN assignment. Also, decrement the counter in bepostop if
the failuer has happened.
Add test suite to cover the change.
https://pagure.io/389-ds-base/issue/49300
Reviewed by: tbordaz (Thanks!)
---
dirsrvtests/tests/suites/plugins/entryusn_test.py | 240 ++++++++++++++++++++++
ldap/servers/plugins/usn/usn.c | 109 ++++++----
ldap/servers/slapd/pblock.c | 14 +-
ldap/servers/slapd/pblock_v3.h | 1 +
ldap/servers/slapd/slapi-plugin.h | 3 +
5 files changed, 322 insertions(+), 45 deletions(-)
diff --git a/dirsrvtests/tests/suites/plugins/entryusn_test.py b/dirsrvtests/tests/suites/plugins/entryusn_test.py
new file mode 100644
index 0000000..7213154
--- /dev/null
+++ b/dirsrvtests/tests/suites/plugins/entryusn_test.py
@@ -0,0 +1,240 @@
+# --- BEGIN COPYRIGHT BLOCK ---
+# Copyright (C) 2020 Red Hat, Inc.
+# All rights reserved.
+#
+# License: GPL (version 3 or any later version).
+# See LICENSE for details.
+# --- END COPYRIGHT BLOCK ---
+#
+import ldap
+import logging
+import pytest
+from lib389._constants import DEFAULT_SUFFIX
+from lib389.config import Config
+from lib389.plugins import USNPlugin, MemberOfPlugin
+from lib389.idm.group import Groups
+from lib389.idm.user import UserAccounts
+from lib389.idm.organizationalunit import OrganizationalUnit
+from lib389.tombstone import Tombstones
+from lib389.rootdse import RootDSE
+from lib389.topologies import topology_st, topology_m2
+
+log = logging.getLogger(__name__)
+
+USER_NUM = 10
+GROUP_NUM = 3
+
+
+def check_entryusn_no_duplicates(entryusn_list):
+ """Check that all values in the list are unique"""
+
+ if len(entryusn_list) > len(set(entryusn_list)):
+ raise AssertionError(f"EntryUSN values have duplicates, please, check logs")
+
+
+def check_lastusn_after_restart(inst):
+ """Check that last usn is the same after restart"""
+
+ root_dse = RootDSE(inst)
+ last_usn_before = root_dse.get_attr_val_int("lastusn;userroot")
+ inst.restart()
+ last_usn_after = root_dse.get_attr_val_int("lastusn;userroot")
+ assert last_usn_after == last_usn_before
+
+
+(a)pytest.fixture(scope="module")
+def setup(topology_st, request):
+ """
+ Enable USN plug-in
+ Enable MEMBEROF plugin
+ Add test entries
+ """
+
+ inst = topology_st.standalone
+
+ log.info("Enable the USN plugin...")
+ plugin = USNPlugin(inst)
+ plugin.enable()
+
+ log.info("Enable the MEMBEROF plugin...")
+ plugin = MemberOfPlugin(inst)
+ plugin.enable()
+
+ inst.restart()
+
+ users_list = []
+ log.info("Adding test entries...")
+ users = UserAccounts(inst, DEFAULT_SUFFIX)
+ for id in range(USER_NUM):
+ user = users.create_test_user(uid=id)
+ users_list.append(user)
+
+ groups_list = []
+ log.info("Adding test groups...")
+ groups = Groups(inst, DEFAULT_SUFFIX)
+ for id in range(GROUP_NUM):
+ group = groups.create(properties={'cn': f'test_group{id}'})
+ groups_list.append(group)
+
+ def fin():
+ for user in users_list:
+ try:
+ user.delete()
+ except ldap.NO_SUCH_OBJECT:
+ pass
+ for group in groups_list:
+ try:
+ group.delete()
+ except ldap.NO_SUCH_OBJECT:
+ pass
+ request.addfinalizer(fin)
+
+ return {"users": users_list,
+ "groups": groups_list}
+
+
+def test_entryusn_no_duplicates(topology_st, setup):
+ """Verify that entryUSN is not duplicated after memberOf operation
+
+ :id: 1a7d382d-1214-4d56-b9c2-9c4ed57d1683
+ :setup: Standalone instance, Groups and Users, USN and memberOf are enabled
+ :steps:
+ 1. Add a member to group 1
+ 2. Add a member to group 1 and 2
+ 3. Check that entryUSNs are different
+ 4. Check that lastusn before and after a restart are the same
+ :expectedresults:
+ 1. Success
+ 2. Success
+ 3. Success
+ 4. Success
+ """
+
+ inst = topology_st.standalone
+ config = Config(inst)
+ config.replace('nsslapd-accesslog-level', '260') # Internal op
+ config.replace('nsslapd-errorlog-level', '65536')
+ config.replace('nsslapd-plugin-logging', 'on')
+ entryusn_list = []
+
+ users = setup["users"]
+ groups = setup["groups"]
+
+ groups[0].replace('member', users[0].dn)
+ entryusn_list.append(users[0].get_attr_val_int('entryusn'))
+ log.info(f"{users[0].dn}_1: {entryusn_list[-1:]}")
+ entryusn_list.append(groups[0].get_attr_val_int('entryusn'))
+ log.info(f"{groups[0].dn}_1: {entryusn_list[-1:]}")
+ check_entryusn_no_duplicates(entryusn_list)
+
+ groups[1].replace('member', [users[0].dn, users[1].dn])
+ entryusn_list.append(users[0].get_attr_val_int('entryusn'))
+ log.info(f"{users[0].dn}_2: {entryusn_list[-1:]}")
+ entryusn_list.append(users[1].get_attr_val_int('entryusn'))
+ log.info(f"{users[1].dn}_2: {entryusn_list[-1:]}")
+ entryusn_list.append(groups[1].get_attr_val_int('entryusn'))
+ log.info(f"{groups[1].dn}_2: {entryusn_list[-1:]}")
+ check_entryusn_no_duplicates(entryusn_list)
+
+ check_lastusn_after_restart(inst)
+
+
+def test_entryusn_is_same_after_failure(topology_st, setup):
+ """Verify that entryUSN is the same after failed operation
+
+ :id: 1f227533-370a-48c1-b920-9b3b0bcfc32e
+ :setup: Standalone instance, Groups and Users, USN and memberOf are enabled
+ :steps:
+ 1. Get current group's entryUSN value
+ 2. Try to modify the group with an invalid syntax
+ 3. Get new group's entryUSN value and compare with old
+ 4. Check that lastusn before and after a restart are the same
+ :expectedresults:
+ 1. Success
+ 2. Invalid Syntax error
+ 3. Should be the same
+ 4. Success
+ """
+
+ inst = topology_st.standalone
+ users = setup["users"]
+
+ # We need this update so we get the latest USN pointed to our entry
+ users[0].replace('description', 'update')
+
+ entryusn_before = users[0].get_attr_val_int('entryusn')
+ users[0].replace('description', 'update')
+ try:
+ users[0].replace('uid', 'invalid update')
+ except ldap.NOT_ALLOWED_ON_RDN:
+ pass
+ users[0].replace('description', 'second update')
+ entryusn_after = users[0].get_attr_val_int('entryusn')
+
+ # entryUSN should be OLD + 2 (only two user updates)
+ assert entryusn_after == (entryusn_before + 2)
+
+ check_lastusn_after_restart(inst)
+
+
+def test_entryusn_after_repl_delete(topology_m2):
+ """Verify that entryUSN is incremented on 1 after delete operation which creates a tombstone
+
+ :id: 1704cf65-41bc-4347-bdaf-20fc2431b218
+ :setup: An instance with replication, Users, USN enabled
+ :steps:
+ 1. Try to delete a user
+ 2. Check the tombstone has the incremented USN
+ 3. Try to delete ou=People with users
+ 4. Check the entry has a not incremented entryUSN
+ :expectedresults:
+ 1. Success
+ 2. Success
+ 3. Should fail with Not Allowed On Non-leaf error
+ 4. Success
+ """
+
+ inst = topology_m2.ms["master1"]
+ plugin = USNPlugin(inst)
+ plugin.enable()
+ inst.restart()
+ users = UserAccounts(inst, DEFAULT_SUFFIX)
+
+ try:
+ user_1 = users.create_test_user()
+ user_rdn = user_1.rdn
+ tombstones = Tombstones(inst, DEFAULT_SUFFIX)
+
+ user_1.replace('description', 'update_ts')
+ user_usn = user_1.get_attr_val_int('entryusn')
+
+ user_1.delete()
+
+ ts = tombstones.get(user_rdn)
+ ts_usn = ts.get_attr_val_int('entryusn')
+
+ assert (user_usn + 1) == ts_usn
+
+ user_1 = users.create_test_user()
+ org = OrganizationalUnit(inst, f"ou=People,{DEFAULT_SUFFIX}")
+ org.replace('description', 'update_ts')
+ ou_usn_before = org.get_attr_val_int('entryusn')
+ try:
+ org.delete()
+ except ldap.NOT_ALLOWED_ON_NONLEAF:
+ pass
+ ou_usn_after = org.get_attr_val_int('entryusn')
+ assert ou_usn_before == ou_usn_after
+
+ finally:
+ try:
+ user_1.delete()
+ except ldap.NO_SUCH_OBJECT:
+ pass
+
+
+if __name__ == '__main__':
+ # Run isolated
+ # -s for DEBUG mode
+ CURRENT_FILE = os.path.realpath(__file__)
+ pytest.main("-s %s" % CURRENT_FILE)
diff --git a/ldap/servers/plugins/usn/usn.c b/ldap/servers/plugins/usn/usn.c
index 12ba040..f2cc8a6 100644
--- a/ldap/servers/plugins/usn/usn.c
+++ b/ldap/servers/plugins/usn/usn.c
@@ -333,6 +333,12 @@ _usn_add_next_usn(Slapi_Entry *e, Slapi_Backend *be)
}
slapi_ch_free_string(&usn_berval.bv_val);
+ /*
+ * increment the counter now and decrement in the bepostop
+ * if the operation will fail
+ */
+ slapi_counter_increment(be->be_usn_counter);
+
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
"<-- _usn_add_next_usn\n");
@@ -370,6 +376,12 @@ _usn_mod_next_usn(LDAPMod ***mods, Slapi_Backend *be)
*mods = slapi_mods_get_ldapmods_passout(&smods);
+ /*
+ * increment the counter now and decrement in the bepostop
+ * if the operation will fail
+ */
+ slapi_counter_increment(be->be_usn_counter);
+
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
"<-- _usn_mod_next_usn\n");
return LDAP_SUCCESS;
@@ -420,6 +432,7 @@ usn_betxnpreop_delete(Slapi_PBlock *pb)
{
Slapi_Entry *e = NULL;
Slapi_Backend *be = NULL;
+ int32_t tombstone_incremented = 0;
int rc = SLAPI_PLUGIN_SUCCESS;
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
@@ -441,7 +454,9 @@ usn_betxnpreop_delete(Slapi_PBlock *pb)
goto bail;
}
_usn_add_next_usn(e, be);
+ tombstone_incremented = 1;
bail:
+ slapi_pblock_set(pb, SLAPI_USN_INCREMENT_FOR_TOMBSTONE, &tombstone_incremented);
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
"<-- usn_betxnpreop_delete\n");
@@ -483,7 +498,7 @@ bail:
return rc;
}
-/* count up the counter */
+/* count down the counter */
static int
usn_bepostop(Slapi_PBlock *pb)
{
@@ -493,25 +508,24 @@ usn_bepostop(Slapi_PBlock *pb)
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
"--> usn_bepostop\n");
- /* if op is not successful, don't increment the counter */
+ /* if op is not successful, decrement the counter, else - do nothing */
slapi_pblock_get(pb, SLAPI_RESULT_CODE, &rc);
if (LDAP_SUCCESS != rc) {
- /* no plugin failure */
- rc = SLAPI_PLUGIN_SUCCESS;
- goto bail;
- }
+ slapi_pblock_get(pb, SLAPI_BACKEND, &be);
+ if (NULL == be) {
+ rc = LDAP_PARAM_ERROR;
+ slapi_pblock_set(pb, SLAPI_RESULT_CODE, &rc);
+ rc = SLAPI_PLUGIN_FAILURE;
+ goto bail;
+ }
- slapi_pblock_get(pb, SLAPI_BACKEND, &be);
- if (NULL == be) {
- rc = LDAP_PARAM_ERROR;
- slapi_pblock_set(pb, SLAPI_RESULT_CODE, &rc);
- rc = SLAPI_PLUGIN_FAILURE;
- goto bail;
+ if (be->be_usn_counter) {
+ slapi_counter_decrement(be->be_usn_counter);
+ }
}
- if (be->be_usn_counter) {
- slapi_counter_increment(be->be_usn_counter);
- }
+ /* no plugin failure */
+ rc = SLAPI_PLUGIN_SUCCESS;
bail:
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
"<-- usn_bepostop\n");
@@ -519,13 +533,14 @@ bail:
return rc;
}
-/* count up the counter */
+/* count down the counter on a failure and mod ignore */
static int
usn_bepostop_modify(Slapi_PBlock *pb)
{
int rc = SLAPI_PLUGIN_FAILURE;
Slapi_Backend *be = NULL;
LDAPMod **mods = NULL;
+ int32_t do_decrement = 0;
int i;
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
@@ -534,9 +549,7 @@ usn_bepostop_modify(Slapi_PBlock *pb)
/* if op is not successful, don't increment the counter */
slapi_pblock_get(pb, SLAPI_RESULT_CODE, &rc);
if (LDAP_SUCCESS != rc) {
- /* no plugin failure */
- rc = SLAPI_PLUGIN_SUCCESS;
- goto bail;
+ do_decrement = 1;
}
slapi_pblock_get(pb, SLAPI_MODIFY_MODS, &mods);
@@ -545,25 +558,29 @@ usn_bepostop_modify(Slapi_PBlock *pb)
if (mods[i]->mod_op & LDAP_MOD_IGNORE) {
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
"usn_bepostop_modify - MOD_IGNORE detected\n");
- goto bail; /* conflict occurred.
- skip incrementing the counter. */
+ do_decrement = 1; /* conflict occurred.
+ decrement he counter. */
} else {
break;
}
}
}
- slapi_pblock_get(pb, SLAPI_BACKEND, &be);
- if (NULL == be) {
- rc = LDAP_PARAM_ERROR;
- slapi_pblock_set(pb, SLAPI_RESULT_CODE, &rc);
- rc = SLAPI_PLUGIN_FAILURE;
- goto bail;
+ if (do_decrement) {
+ slapi_pblock_get(pb, SLAPI_BACKEND, &be);
+ if (NULL == be) {
+ rc = LDAP_PARAM_ERROR;
+ slapi_pblock_set(pb, SLAPI_RESULT_CODE, &rc);
+ rc = SLAPI_PLUGIN_FAILURE;
+ goto bail;
+ }
+ if (be->be_usn_counter) {
+ slapi_counter_decrement(be->be_usn_counter);
+ }
}
- if (be->be_usn_counter) {
- slapi_counter_increment(be->be_usn_counter);
- }
+ /* no plugin failure */
+ rc = SLAPI_PLUGIN_SUCCESS;
bail:
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
"<-- usn_bepostop_modify\n");
@@ -573,34 +590,38 @@ bail:
/* count up the counter */
/* if the op is delete and the op was not successful, remove preventryusn */
+/* the function is executed on TXN level */
static int
usn_bepostop_delete(Slapi_PBlock *pb)
{
int rc = SLAPI_PLUGIN_FAILURE;
Slapi_Backend *be = NULL;
+ int32_t tombstone_incremented = 0;
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
"--> usn_bepostop_delete\n");
- /* if op is not successful, don't increment the counter */
+ /* if op is not successful and it is a tombstone entry, decrement the counter */
slapi_pblock_get(pb, SLAPI_RESULT_CODE, &rc);
if (LDAP_SUCCESS != rc) {
- /* no plugin failure */
- rc = SLAPI_PLUGIN_SUCCESS;
- goto bail;
- }
+ slapi_pblock_get(pb, SLAPI_USN_INCREMENT_FOR_TOMBSTONE, &tombstone_incremented);
+ if (tombstone_incremented) {
+ slapi_pblock_get(pb, SLAPI_BACKEND, &be);
+ if (NULL == be) {
+ rc = LDAP_PARAM_ERROR;
+ slapi_pblock_set(pb, SLAPI_RESULT_CODE, &rc);
+ rc = SLAPI_PLUGIN_FAILURE;
+ goto bail;
+ }
- slapi_pblock_get(pb, SLAPI_BACKEND, &be);
- if (NULL == be) {
- rc = LDAP_PARAM_ERROR;
- slapi_pblock_set(pb, SLAPI_RESULT_CODE, &rc);
- rc = SLAPI_PLUGIN_FAILURE;
- goto bail;
+ if (be->be_usn_counter) {
+ slapi_counter_decrement(be->be_usn_counter);
+ }
+ }
}
- if (be->be_usn_counter) {
- slapi_counter_increment(be->be_usn_counter);
- }
+ /* no plugin failure */
+ rc = SLAPI_PLUGIN_SUCCESS;
bail:
slapi_log_err(SLAPI_LOG_TRACE, USN_PLUGIN_SUBSYSTEM,
"<-- usn_bepostop_delete\n");
diff --git a/ldap/servers/slapd/pblock.c b/ldap/servers/slapd/pblock.c
index cb562e9..454ea9c 100644
--- a/ldap/servers/slapd/pblock.c
+++ b/ldap/servers/slapd/pblock.c
@@ -2436,7 +2436,7 @@ slapi_pblock_get(Slapi_PBlock *pblock, int arg, void *value)
(*(char **)value) = NULL;
}
break;
-
+
case SLAPI_SEARCH_CTRLS:
if (pblock->pb_intop != NULL) {
(*(LDAPControl ***)value) = pblock->pb_intop->pb_search_ctrls;
@@ -2479,6 +2479,14 @@ slapi_pblock_get(Slapi_PBlock *pblock, int arg, void *value)
}
break;
+ case SLAPI_USN_INCREMENT_FOR_TOMBSTONE:
+ if (pblock->pb_intop != NULL) {
+ (*(int32_t *)value) = pblock->pb_intop->pb_usn_tombstone_incremented;
+ } else {
+ (*(int32_t *)value) = 0;
+ }
+ break;
+
/* ACI Target Check */
case SLAPI_ACI_TARGET_CHECK:
if (pblock->pb_misc != NULL) {
@@ -4156,6 +4164,10 @@ slapi_pblock_set(Slapi_PBlock *pblock, int arg, void *value)
pblock->pb_intop->pb_paged_results_cookie = *(int *)value;
break;
+ case SLAPI_USN_INCREMENT_FOR_TOMBSTONE:
+ pblock->pb_intop->pb_usn_tombstone_incremented = *((int32_t *)value);
+ break;
+
/* ACI Target Check */
case SLAPI_ACI_TARGET_CHECK:
_pblock_assert_pb_misc(pblock);
diff --git a/ldap/servers/slapd/pblock_v3.h b/ldap/servers/slapd/pblock_v3.h
index 7ec2f37..90498c0 100644
--- a/ldap/servers/slapd/pblock_v3.h
+++ b/ldap/servers/slapd/pblock_v3.h
@@ -161,6 +161,7 @@ typedef struct _slapi_pblock_intop
int pb_paged_results_index; /* stash SLAPI_PAGED_RESULTS_INDEX */
int pb_paged_results_cookie; /* stash SLAPI_PAGED_RESULTS_COOKIE */
+ int32_t pb_usn_tombstone_incremented; /* stash SLAPI_PAGED_RESULTS_COOKIE */
} slapi_pblock_intop;
/* Stuff that is rarely used, but still present */
diff --git a/ldap/servers/slapd/slapi-plugin.h b/ldap/servers/slapd/slapi-plugin.h
index b78395d..a71050f 100644
--- a/ldap/servers/slapd/slapi-plugin.h
+++ b/ldap/servers/slapd/slapi-plugin.h
@@ -7471,6 +7471,9 @@ typedef enum _slapi_op_note_t {
#define SLAPI_PAGED_RESULTS_INDEX 1945
#define SLAPI_PAGED_RESULTS_COOKIE 1949
+/* USN Plugin flag for tombstone entries */
+#define SLAPI_USN_INCREMENT_FOR_TOMBSTONE 1950
+
/* ACI Target Check */
#define SLAPI_ACI_TARGET_CHECK 1946
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 8 months
[389-ds-base] branch 389-ds-base-1.4.2 updated: Issue 50984 - Fix disk_mon_check_diskspace types
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
spichugi pushed a commit to branch 389-ds-base-1.4.2
in repository 389-ds-base.
The following commit(s) were added to refs/heads/389-ds-base-1.4.2 by this push:
new 6023396 Issue 50984 - Fix disk_mon_check_diskspace types
6023396 is described below
commit 6023396b7cdc149ce42f8a50558dccbf76a502cc
Author: Simon Pichugin <spichugi(a)redhat.com>
AuthorDate: Wed Jul 15 21:25:00 2020 +0200
Issue 50984 - Fix disk_mon_check_diskspace types
Description: Function parameters are inconsistence.
Documentation states that threshold should be from 0 to 2^63 - 1
so we can use uint64_t for that.
https://pagure.io/389-ds-base/issue/50984
Reviewed by: firstyear (Thanks!)
---
ldap/servers/slapd/daemon.c | 6 +++---
ldap/servers/slapd/libglobs.c | 6 +++---
ldap/servers/slapd/main.c | 2 +-
ldap/servers/slapd/proto-slap.h | 3 ++-
4 files changed, 9 insertions(+), 8 deletions(-)
diff --git a/ldap/servers/slapd/daemon.c b/ldap/servers/slapd/daemon.c
index d75ca9e..81a7098 100644
--- a/ldap/servers/slapd/daemon.c
+++ b/ldap/servers/slapd/daemon.c
@@ -345,8 +345,8 @@ disk_mon_check_diskspace(char **dirs, uint64_t threshold, uint64_t *disk_space)
uint64_t freeBytes = 0;
uint64_t blockSize = 0;
char *worst_dir = NULL;
- int hit_threshold = 0;
- int i = 0;
+ int32_t hit_threshold = 0;
+ int32_t i = 0;
for (i = 0; dirs && dirs[i]; i++) {
if (statvfs(dirs[i], &buf) != -1) {
@@ -396,7 +396,7 @@ disk_monitoring_thread(void *nothing __attribute__((unused)))
char *dirstr = NULL;
uint64_t previous_mark = 0;
uint64_t disk_space = 0;
- int64_t threshold = 0;
+ uint64_t threshold = 0;
uint64_t halfway = 0;
time_t start = 0;
time_t now = 0;
diff --git a/ldap/servers/slapd/libglobs.c b/ldap/servers/slapd/libglobs.c
index d9225a3..cc88536 100644
--- a/ldap/servers/slapd/libglobs.c
+++ b/ldap/servers/slapd/libglobs.c
@@ -1981,7 +1981,7 @@ config_set_disk_threshold(const char *attrname, char *value, char *errorbuf, int
if (apply) {
CFG_LOCK_WRITE(slapdFrontendConfig);
- slapdFrontendConfig->disk_threshold = threshold;
+ slapdFrontendConfig->disk_threshold = (uint64_t)threshold;
CFG_UNLOCK_WRITE(slapdFrontendConfig);
}
@@ -5107,11 +5107,11 @@ config_get_disk_grace_period()
return retVal;
}
-PRInt64
+uint64_t
config_get_disk_threshold()
{
slapdFrontendConfig_t *slapdFrontendConfig = getFrontendConfig();
- PRInt64 retVal;
+ uint64_t retVal;
CFG_LOCK_READ(slapdFrontendConfig);
retVal = slapdFrontendConfig->disk_threshold;
diff --git a/ldap/servers/slapd/main.c b/ldap/servers/slapd/main.c
index 818970b..8e07f2d 100644
--- a/ldap/servers/slapd/main.c
+++ b/ldap/servers/slapd/main.c
@@ -894,7 +894,7 @@ main(int argc, char **argv)
char **dirs = NULL;
char *dirstr = NULL;
uint64_t disk_space = 0;
- int64_t threshold = 0;
+ uint64_t threshold = 0;
uint64_t halfway = 0;
threshold = config_get_disk_threshold();
halfway = threshold / 2;
diff --git a/ldap/servers/slapd/proto-slap.h b/ldap/servers/slapd/proto-slap.h
index ac02618..7134b69 100644
--- a/ldap/servers/slapd/proto-slap.h
+++ b/ldap/servers/slapd/proto-slap.h
@@ -543,7 +543,7 @@ void config_set_auditlog_enabled(int value);
void config_set_auditfaillog_enabled(int value);
int config_get_accesslog_logging_enabled(void);
int config_get_disk_monitoring(void);
-PRInt64 config_get_disk_threshold(void);
+uint64_t config_get_disk_threshold(void);
int config_get_disk_grace_period(void);
int config_get_disk_logging_critical(void);
int config_get_ndn_cache_count(void);
@@ -1499,6 +1499,7 @@ void slapd_do_nothing(int);
void slapd_wait4child(int);
void disk_mon_get_dirs(char ***list);
int32_t disk_get_info(char *dir, uint64_t *total_space, uint64_t *avail_space, uint64_t *used_space);
+char *disk_mon_check_diskspace(char **dirs, uint64_t threshold, uint64_t *disk_space);
/*
* main.c
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 8 months
[389-ds-base] branch 389-ds-base-1.4.3 updated: Issue 50984 - Fix disk_mon_check_diskspace types
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
spichugi pushed a commit to branch 389-ds-base-1.4.3
in repository 389-ds-base.
The following commit(s) were added to refs/heads/389-ds-base-1.4.3 by this push:
new 668475a Issue 50984 - Fix disk_mon_check_diskspace types
668475a is described below
commit 668475ae474524a8023515d794db22fa8d2f540d
Author: Simon Pichugin <spichugi(a)redhat.com>
AuthorDate: Wed Jul 15 21:25:00 2020 +0200
Issue 50984 - Fix disk_mon_check_diskspace types
Description: Function parameters are inconsistence.
Documentation states that threshold should be from 0 to 2^63 - 1
so we can use uint64_t for that.
https://pagure.io/389-ds-base/issue/50984
Reviewed by: firstyear (Thanks!)
---
ldap/servers/slapd/daemon.c | 6 +++---
ldap/servers/slapd/libglobs.c | 6 +++---
ldap/servers/slapd/main.c | 2 +-
ldap/servers/slapd/proto-slap.h | 2 +-
4 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/ldap/servers/slapd/daemon.c b/ldap/servers/slapd/daemon.c
index 772b496..0c5585e 100644
--- a/ldap/servers/slapd/daemon.c
+++ b/ldap/servers/slapd/daemon.c
@@ -345,8 +345,8 @@ disk_mon_check_diskspace(char **dirs, uint64_t threshold, uint64_t *disk_space)
uint64_t freeBytes = 0;
uint64_t blockSize = 0;
char *worst_dir = NULL;
- int hit_threshold = 0;
- int i = 0;
+ int32_t hit_threshold = 0;
+ int32_t i = 0;
for (i = 0; dirs && dirs[i]; i++) {
if (statvfs(dirs[i], &buf) != -1) {
@@ -396,7 +396,7 @@ disk_monitoring_thread(void *nothing __attribute__((unused)))
char *dirstr = NULL;
uint64_t previous_mark = 0;
uint64_t disk_space = 0;
- int64_t threshold = 0;
+ uint64_t threshold = 0;
uint64_t halfway = 0;
time_t start = 0;
time_t now = 0;
diff --git a/ldap/servers/slapd/libglobs.c b/ldap/servers/slapd/libglobs.c
index 97af7bb..7d5374c 100644
--- a/ldap/servers/slapd/libglobs.c
+++ b/ldap/servers/slapd/libglobs.c
@@ -2031,7 +2031,7 @@ config_set_disk_threshold(const char *attrname, char *value, char *errorbuf, int
if (apply) {
CFG_LOCK_WRITE(slapdFrontendConfig);
- slapdFrontendConfig->disk_threshold = threshold;
+ slapdFrontendConfig->disk_threshold = (uint64_t)threshold;
CFG_UNLOCK_WRITE(slapdFrontendConfig);
}
@@ -5164,11 +5164,11 @@ config_get_disk_grace_period()
return retVal;
}
-PRInt64
+uint64_t
config_get_disk_threshold()
{
slapdFrontendConfig_t *slapdFrontendConfig = getFrontendConfig();
- PRInt64 retVal;
+ uint64_t retVal;
CFG_LOCK_READ(slapdFrontendConfig);
retVal = slapdFrontendConfig->disk_threshold;
diff --git a/ldap/servers/slapd/main.c b/ldap/servers/slapd/main.c
index 9e5219c..694375b 100644
--- a/ldap/servers/slapd/main.c
+++ b/ldap/servers/slapd/main.c
@@ -894,7 +894,7 @@ main(int argc, char **argv)
char **dirs = NULL;
char *dirstr = NULL;
uint64_t disk_space = 0;
- int64_t threshold = 0;
+ uint64_t threshold = 0;
uint64_t halfway = 0;
threshold = config_get_disk_threshold();
halfway = threshold / 2;
diff --git a/ldap/servers/slapd/proto-slap.h b/ldap/servers/slapd/proto-slap.h
index 41308f1..3acc24f 100644
--- a/ldap/servers/slapd/proto-slap.h
+++ b/ldap/servers/slapd/proto-slap.h
@@ -545,7 +545,7 @@ void config_set_auditfaillog_enabled(int value);
int config_get_accesslog_logging_enabled(void);
int config_get_disk_monitoring(void);
int config_get_disk_threshold_readonly(void);
-PRInt64 config_get_disk_threshold(void);
+uint64_t config_get_disk_threshold(void);
int config_get_disk_grace_period(void);
int config_get_disk_logging_critical(void);
int config_get_ndn_cache_count(void);
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 8 months
[389-ds-base] branch 389-ds-base-1.4.2 updated: Bump version to 1.4.2.16
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
mreynolds pushed a commit to branch 389-ds-base-1.4.2
in repository 389-ds-base.
The following commit(s) were added to refs/heads/389-ds-base-1.4.2 by this push:
new 92afa2e Bump version to 1.4.2.16
92afa2e is described below
commit 92afa2ea7d1b8c890300e0eaa1d0de7f04614e63
Author: Mark Reynolds <mreynolds(a)redhat.com>
AuthorDate: Wed Jul 8 18:02:38 2020 -0400
Bump version to 1.4.2.16
---
VERSION.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/VERSION.sh b/VERSION.sh
index 0dce236..ce7b5a7 100644
--- a/VERSION.sh
+++ b/VERSION.sh
@@ -10,7 +10,7 @@ vendor="389 Project"
# PACKAGE_VERSION is constructed from these
VERSION_MAJOR=1
VERSION_MINOR=4
-VERSION_MAINT=2.15
+VERSION_MAINT=2.16
# NOTE: VERSION_PREREL is automatically set for builds made out of a git tree
VERSION_PREREL=
VERSION_DATE=$(date -u +%Y%m%d)
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 8 months
[389-ds-base] branch 389-ds-base-1.4.3 updated: Bump version to 1.4.3.11
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
mreynolds pushed a commit to branch 389-ds-base-1.4.3
in repository 389-ds-base.
The following commit(s) were added to refs/heads/389-ds-base-1.4.3 by this push:
new 82796f1 Bump version to 1.4.3.11
82796f1 is described below
commit 82796f172237a07935149b7cd1dc43e7f257dcac
Author: Mark Reynolds <mreynolds(a)redhat.com>
AuthorDate: Wed Jul 8 17:30:22 2020 -0400
Bump version to 1.4.3.11
---
VERSION.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/VERSION.sh b/VERSION.sh
index d868350..2b66655 100644
--- a/VERSION.sh
+++ b/VERSION.sh
@@ -10,7 +10,7 @@ vendor="389 Project"
# PACKAGE_VERSION is constructed from these
VERSION_MAJOR=1
VERSION_MINOR=4
-VERSION_MAINT=3.10
+VERSION_MAINT=3.11
# NOTE: VERSION_PREREL is automatically set for builds made out of a git tree
VERSION_PREREL=
VERSION_DATE=$(date -u +%Y%m%d)
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 8 months