On 4 Aug 2019, at 23:55, Lutz Berger lutz.berger@multigrid.de wrote:
Hello,
I've come across a web site that claims that an "equality index" is only allowed for attributes that have "EQUALITY" in their description, "otherwise terrible things will happen".
For example
attributeTypes: ( 1.3.6.1.4.1.13299.2.3.7.xxx NAME 'abCLZ' EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'user defined' )
For the sake of correctness, I've tried to build an equality-index for an attribute missing such description, e.g.
attributeTypes: ( 1.3.6.1.4.1.13299.2.3.7.xyz NAME 'abID' SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'user defined' )
Withour EQ-Index: [root@ur1 slapd-ur1devims]# fgrep "conn=34" access [03/Aug/2019:14:21:00 +0200] conn=34 fd=65 slot=65 connection from 192.168.69.152 to 192.168.69.152 [03/Aug/2019:14:21:00 +0200] conn=34 op=0 BIND dn="cn=Directory Manager" method=128 version=3 [03/Aug/2019:14:21:00 +0200] conn=34 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager" [03/Aug/2019:14:21:00 +0200] conn=34 op=1 SRCH base="ou=users,c=de,o=creditreform" scope=2 filter="(abID=777777777022544)" attrs=ALL [03/Aug/2019:14:21:26 +0200] conn=34 op=1 RESULT err=0 tag=101 nentries=1 etime=26 notes=A [03/Aug/2019:14:21:26 +0200] conn=34 op=2 UNBIND [03/Aug/2019:14:21:26 +0200] conn=34 op=2 fd=65 closed - U1 [root@ur1 slapd-ur1devims]#
With EQ-Index: [root@ur1 slapd-ur1devims]# fgrep "conn=35" access [03/Aug/2019:14:24:18 +0200] conn=35 fd=65 slot=65 connection from 192.168.69.152 to 192.168.69.152 [03/Aug/2019:14:24:18 +0200] conn=35 op=0 BIND dn="cn=Directory Manager" method=128 version=3 [03/Aug/2019:14:24:18 +0200] conn=35 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager" [03/Aug/2019:14:24:18 +0200] conn=35 op=1 SRCH base="ou=users,c=de,o=creditreform" scope=2 filter="(abID=777777777022544)" attrs=ALL [03/Aug/2019:14:24:18 +0200] conn=35 op=1 RESULT err=0 tag=101 nentries=1 etime=0 [03/Aug/2019:14:24:18 +0200] conn=35 op=2 UNBIND [03/Aug/2019:14:24:18 +0200] conn=35 op=2 fd=65 closed - U1
My question is now, is the EQUALITY part of the schema description really necessary for building equality-indexes on attributes, since I couldn't reproduce any obvious problem.
From the access pattern I see in the access log, building such an index is definitely beneficial in sense of performance.
I think that without the equality syntax defined, you will fall back on a literal memcmp between the two values as byte arrays. This of course has lots of issues with UTF8 and many other types.
Generally if possible, it's *always* best to give a proper equality type to ensure that types will compare in an expected manner, rather than giving surprising results. Be explicit :)
I hope that helps,
Thanks for your efforts!
You are most welcome, and thanks for your question!
Best regards, Lutz
-- Dipl.-Inform. Univ. Lutz Berger Email: lutz.berger@multigrid.de
389-users mailing list -- 389-users@lists.fedoraproject.org To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject....
— Sincerely,
William Brown
Senior Software Engineer, 389 Directory Server SUSE Labs
On 8/4/19 9:55 AM, Lutz Berger wrote:
Hello,
I've come across a web site that claims that an "equality index" is only allowed for attributes that have "EQUALITY" in their description, "otherwise terrible things will happen".
For example
*attributeTypes: ( 1.3.6.1.4.1.13299.2.3.7.xxx NAME 'abCLZ'
EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'user defined' )**
For the sake of correctness, I've tried to build an equality-index for an attribute missing such description, e.g.
*>> attributeTypes: ( 1.3.6.1.4.1.13299.2.3.7.xyz NAME 'abID' SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'user defined' )*
So what is happening is that the first example has a "matching rule" defined: EQUALITY caseIgnoreMatch. If you define such an attribute with this syntax you MUST have an equality index for that attribute. Otherwise the server has to manually perform this matching - which is VERY expensive. Hence why you see an etime of 26 seconds. Once its indexed for equality the matching rule can efficiently be processed. But you do not NEED to use this matching rule: EQUALITY caseIgnoreMatch, unless you have a requirement for it. But you should always index your attribute for how you plan to use it. In this case you are doing an equality search: <ATTR>=<some exact value> so you would want an equality index (regardless of the presence of a matching rule).
HTH,
Mark
*Withour EQ-Index:* [root@ur1 slapd-ur1devims]# fgrep "conn=34" access [03/Aug/2019:14:21:00 +0200] conn=34 fd=65 slot=65 connection from 192.168.69.152 to 192.168.69.152 [03/Aug/2019:14:21:00 +0200] conn=34 op=0 BIND dn="cn=Directory Manager" method=128 version=3 [03/Aug/2019:14:21:00 +0200] conn=34 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager" [03/Aug/2019:14:21:00 +0200] conn=34 op=1 SRCH base="ou=users,c=de,o=creditreform" scope=2 filter="(abID=777777777022544)" attrs=ALL *[03/Aug/2019:14:21:26 +0200] conn=34 op=1 RESULT err=0 tag=101 nentries=1 etime=26 notes=A* [03/Aug/2019:14:21:26 +0200] conn=34 op=2 UNBIND [03/Aug/2019:14:21:26 +0200] conn=34 op=2 fd=65 closed - U1 [root@ur1 slapd-ur1devims]#
*With EQ-Index:* [root@ur1 slapd-ur1devims]# fgrep "conn=35" access [03/Aug/2019:14:24:18 +0200] conn=35 fd=65 slot=65 connection from 192.168.69.152 to 192.168.69.152 [03/Aug/2019:14:24:18 +0200] conn=35 op=0 BIND dn="cn=Directory Manager" method=128 version=3 [03/Aug/2019:14:24:18 +0200] conn=35 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager" [03/Aug/2019:14:24:18 +0200] conn=35 op=1 SRCH base="ou=users,c=de,o=creditreform" scope=2 filter="(abID=777777777022544)" attrs=ALL *[03/Aug/2019:14:24:18 +0200] conn=35 op=1 RESULT err=0 tag=101 nentries=1 etime=0* [03/Aug/2019:14:24:18 +0200] conn=35 op=2 UNBIND [03/Aug/2019:14:24:18 +0200] conn=35 op=2 fd=65 closed - U1
*My question is now, is the EQUALITY part of the schema description really necessary** **for building equality-indexes on attributes, since I couldn't reproduce any**obvious problem.*
From the access pattern I see in the access log, building such an index is definitely beneficial in sense of performance.
Thanks for your efforts!
Best regards, Lutz
-- Dipl.-Inform. Univ. Lutz Berger Email: lutz.berger@multigrid.de
Multigrid-Logo http://multigrid.de/
389-users mailing list -- 389-users@lists.fedoraproject.org To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject....
On 8/5/19 8:57 AM, Mark Reynolds wrote:
On 8/4/19 9:55 AM, Lutz Berger wrote:
Hello,
I've come across a web site that claims that an "equality index" is only allowed for attributes that have "EQUALITY" in their description, "otherwise terrible things will happen".
For example
*attributeTypes: ( 1.3.6.1.4.1.13299.2.3.7.xxx NAME 'abCLZ'
EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'user defined' )**
For the sake of correctness, I've tried to build an equality-index for an attribute missing such description, e.g.
*>> attributeTypes: ( 1.3.6.1.4.1.13299.2.3.7.xyz NAME 'abID' SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'user defined' )*
So what is happening is that the first example has a "matching rule" defined: EQUALITY caseIgnoreMatch. If you define such an attribute with this syntax you MUST have an equality index for that attribute. Otherwise the server has to manually perform this matching - which is VERY expensive. Hence why you see an etime of 26 seconds. Once its indexed for equality the matching rule can efficiently be processed.
Sorry this is actually incorrect. The matching rule is not the problem in the case, since you are not using the matching rule in the search filter. See https://access.redhat.com/documentation/en-us/red_hat_directory_server/10/ht... So this is just an issue of basic indexing. You are doing an equality search, so you need an equality index, otherwise the server has to sequentially scan the database for matching entries.
Regards,
Mark
But you do not NEED to use this matching rule: EQUALITY caseIgnoreMatch, unless you have a requirement for it. But you should always index your attribute for how you plan to use it. In this case you are doing an equality search: <ATTR>=<some exact value> so you would want an equality index (regardless of the presence of a matching rule).
HTH,
Mark
*Withour EQ-Index:* [root@ur1 slapd-ur1devims]# fgrep "conn=34" access [03/Aug/2019:14:21:00 +0200] conn=34 fd=65 slot=65 connection from 192.168.69.152 to 192.168.69.152 [03/Aug/2019:14:21:00 +0200] conn=34 op=0 BIND dn="cn=Directory Manager" method=128 version=3 [03/Aug/2019:14:21:00 +0200] conn=34 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager" [03/Aug/2019:14:21:00 +0200] conn=34 op=1 SRCH base="ou=users,c=de,o=creditreform" scope=2 filter="(abID=777777777022544)" attrs=ALL *[03/Aug/2019:14:21:26 +0200] conn=34 op=1 RESULT err=0 tag=101 nentries=1 etime=26 notes=A* [03/Aug/2019:14:21:26 +0200] conn=34 op=2 UNBIND [03/Aug/2019:14:21:26 +0200] conn=34 op=2 fd=65 closed - U1 [root@ur1 slapd-ur1devims]#
*With EQ-Index:* [root@ur1 slapd-ur1devims]# fgrep "conn=35" access [03/Aug/2019:14:24:18 +0200] conn=35 fd=65 slot=65 connection from 192.168.69.152 to 192.168.69.152 [03/Aug/2019:14:24:18 +0200] conn=35 op=0 BIND dn="cn=Directory Manager" method=128 version=3 [03/Aug/2019:14:24:18 +0200] conn=35 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager" [03/Aug/2019:14:24:18 +0200] conn=35 op=1 SRCH base="ou=users,c=de,o=creditreform" scope=2 filter="(abID=777777777022544)" attrs=ALL *[03/Aug/2019:14:24:18 +0200] conn=35 op=1 RESULT err=0 tag=101 nentries=1 etime=0* [03/Aug/2019:14:24:18 +0200] conn=35 op=2 UNBIND [03/Aug/2019:14:24:18 +0200] conn=35 op=2 fd=65 closed - U1
*My question is now, is the EQUALITY part of the schema description really necessary** **for building equality-indexes on attributes, since I couldn't reproduce any**obvious problem.*
From the access pattern I see in the access log, building such an index is definitely beneficial in sense of performance.
Thanks for your efforts!
Best regards, Lutz
-- Dipl.-Inform. Univ. Lutz Berger Email: lutz.berger@multigrid.de
Multigrid-Logo http://multigrid.de/
389-users mailing list --389-users@lists.fedoraproject.org To unsubscribe send an email to389-users-leave@lists.fedoraproject.org Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject....
--
389 Directory Server Development Team
389-users mailing list -- 389-users@lists.fedoraproject.org To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject....
On 08/05/2019 03:05 PM, Mark Reynolds wrote:
On 8/5/19 8:57 AM, Mark Reynolds wrote:
On 8/4/19 9:55 AM, Lutz Berger wrote:
Hello,
I've come across a web site that claims that an "equality index" is only allowed for attributes that have "EQUALITY" in their description, "otherwise terrible things will happen".
For example
*attributeTypes: ( 1.3.6.1.4.1.13299.2.3.7.xxx NAME 'abCLZ'
EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'user defined' )**
For the sake of correctness, I've tried to build an equality-index for an attribute missing such description, e.g.
*>> attributeTypes: ( 1.3.6.1.4.1.13299.2.3.7.xyz NAME 'abID' SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'user defined' )*
So what is happening is that the first example has a "matching rule" defined: EQUALITY caseIgnoreMatch. If you define such an attribute with this syntax you MUST have an equality index for that attribute. Otherwise the server has to manually perform this matching - which is VERY expensive. Hence why you see an etime of 26 seconds. Once its indexed for equality the matching rule can efficiently be processed.
Sorry this is actually incorrect. The matching rule is not the problem in the case, since you are not using the matching rule in the search filter. See https://access.redhat.com/documentation/en-us/red_hat_directory_server/10/ht... So this is just an issue of basic indexing. You are doing an equality search, so you need an equality index, otherwise the server has to sequentially scan the database for matching entries.
and regarding the question of defining the matching rule in the schema I think there is a default matching rule used for the syntax
Regards,
Mark
But you do not NEED to use this matching rule: EQUALITY caseIgnoreMatch, unless you have a requirement for it. But you should always index your attribute for how you plan to use it. In this case you are doing an equality search: <ATTR>=<some exact value> so you would want an equality index (regardless of the presence of a matching rule).
HTH,
Mark
*Withour EQ-Index:* [root@ur1 slapd-ur1devims]# fgrep "conn=34" access [03/Aug/2019:14:21:00 +0200] conn=34 fd=65 slot=65 connection from 192.168.69.152 to 192.168.69.152 [03/Aug/2019:14:21:00 +0200] conn=34 op=0 BIND dn="cn=Directory Manager" method=128 version=3 [03/Aug/2019:14:21:00 +0200] conn=34 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager" [03/Aug/2019:14:21:00 +0200] conn=34 op=1 SRCH base="ou=users,c=de,o=creditreform" scope=2 filter="(abID=777777777022544)" attrs=ALL *[03/Aug/2019:14:21:26 +0200] conn=34 op=1 RESULT err=0 tag=101 nentries=1 etime=26 notes=A* [03/Aug/2019:14:21:26 +0200] conn=34 op=2 UNBIND [03/Aug/2019:14:21:26 +0200] conn=34 op=2 fd=65 closed - U1 [root@ur1 slapd-ur1devims]#
*With EQ-Index:* [root@ur1 slapd-ur1devims]# fgrep "conn=35" access [03/Aug/2019:14:24:18 +0200] conn=35 fd=65 slot=65 connection from 192.168.69.152 to 192.168.69.152 [03/Aug/2019:14:24:18 +0200] conn=35 op=0 BIND dn="cn=Directory Manager" method=128 version=3 [03/Aug/2019:14:24:18 +0200] conn=35 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager" [03/Aug/2019:14:24:18 +0200] conn=35 op=1 SRCH base="ou=users,c=de,o=creditreform" scope=2 filter="(abID=777777777022544)" attrs=ALL *[03/Aug/2019:14:24:18 +0200] conn=35 op=1 RESULT err=0 tag=101 nentries=1 etime=0* [03/Aug/2019:14:24:18 +0200] conn=35 op=2 UNBIND [03/Aug/2019:14:24:18 +0200] conn=35 op=2 fd=65 closed - U1
*My question is now, is the EQUALITY part of the schema description really necessary** **for building equality-indexes on attributes, since I couldn't reproduce any**obvious problem.*
From the access pattern I see in the access log, building such an index is definitely beneficial in sense of performance.
Thanks for your efforts!
Best regards, Lutz
-- Dipl.-Inform. Univ. Lutz Berger Email: lutz.berger@multigrid.de
Multigrid-Logo http://multigrid.de/
389-users mailing list --389-users@lists.fedoraproject.org To unsubscribe send an email to389-users-leave@lists.fedoraproject.org Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject....
--
389 Directory Server Development Team
389-users mailing list --389-users@lists.fedoraproject.org To unsubscribe send an email to389-users-leave@lists.fedoraproject.org Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject....
--
389 Directory Server Development Team
389-users mailing list -- 389-users@lists.fedoraproject.org To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject....
Hallo Lutz,
bist Du's :-)
Servus, Ludwig
On 08/04/2019 03:55 PM, Lutz Berger wrote:
Hello,
I've come across a web site that claims that an "equality index" is only allowed for attributes that have "EQUALITY" in their description, "otherwise terrible things will happen".
For example
*attributeTypes: ( 1.3.6.1.4.1.13299.2.3.7.xxx NAME 'abCLZ'
EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'user defined' )**
For the sake of correctness, I've tried to build an equality-index for an attribute missing such description, e.g.
*>> attributeTypes: ( 1.3.6.1.4.1.13299.2.3.7.xyz NAME 'abID' SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'user defined' )*
*Withour EQ-Index:* [root@ur1 slapd-ur1devims]# fgrep "conn=34" access [03/Aug/2019:14:21:00 +0200] conn=34 fd=65 slot=65 connection from 192.168.69.152 to 192.168.69.152 [03/Aug/2019:14:21:00 +0200] conn=34 op=0 BIND dn="cn=Directory Manager" method=128 version=3 [03/Aug/2019:14:21:00 +0200] conn=34 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager" [03/Aug/2019:14:21:00 +0200] conn=34 op=1 SRCH base="ou=users,c=de,o=creditreform" scope=2 filter="(abID=777777777022544)" attrs=ALL *[03/Aug/2019:14:21:26 +0200] conn=34 op=1 RESULT err=0 tag=101 nentries=1 etime=26 notes=A* [03/Aug/2019:14:21:26 +0200] conn=34 op=2 UNBIND [03/Aug/2019:14:21:26 +0200] conn=34 op=2 fd=65 closed - U1 [root@ur1 slapd-ur1devims]#
*With EQ-Index:* [root@ur1 slapd-ur1devims]# fgrep "conn=35" access [03/Aug/2019:14:24:18 +0200] conn=35 fd=65 slot=65 connection from 192.168.69.152 to 192.168.69.152 [03/Aug/2019:14:24:18 +0200] conn=35 op=0 BIND dn="cn=Directory Manager" method=128 version=3 [03/Aug/2019:14:24:18 +0200] conn=35 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager" [03/Aug/2019:14:24:18 +0200] conn=35 op=1 SRCH base="ou=users,c=de,o=creditreform" scope=2 filter="(abID=777777777022544)" attrs=ALL *[03/Aug/2019:14:24:18 +0200] conn=35 op=1 RESULT err=0 tag=101 nentries=1 etime=0* [03/Aug/2019:14:24:18 +0200] conn=35 op=2 UNBIND [03/Aug/2019:14:24:18 +0200] conn=35 op=2 fd=65 closed - U1
*My question is now, is the EQUALITY part of the schema description really necessary** **for building equality-indexes on attributes, since I couldn't reproduce any**obvious problem.*
From the access pattern I see in the access log, building such an index is definitely beneficial in sense of performance.
Thanks for your efforts!
Best regards, Lutz
-- Dipl.-Inform. Univ. Lutz Berger Email: lutz.berger@multigrid.de
Multigrid-Logo http://multigrid.de/
389-users mailing list -- 389-users@lists.fedoraproject.org To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject....
389-users@lists.fedoraproject.org
-
Ludwig Krispenz -
Lutz Berger -
Mark Reynolds -
William Brown