Hello
We are using 18 389-ds-base-1.3.10.2,
Recently, one server (A) was set up to log to a pipe.
Afterwards, a phenomenon occurs where all server connections are occupied by an external system.
The external system has stopped.
After this incident, the servers returned to normal condition, but
Only server A had intermittent port 389 inaccessibility and delays.
After this, we determined that the problem on server A was due to logging in the pipe.
After changing the pipe to a file, I restarted the server.
After that the problem went away.
We want to know the exact cause of the delay and inaccessibility that occurred on server A.
How can logging to pipe affect directory server port 389 access?
Could it be a file descriptor related issue? Why were the other 17 servers able to recover to normal?
Thanks
Hi,
It would be helpful to have some details how you configured the log pipe and did the tests. I wonder if it could be related to https://github.com/389ds/389-ds-base/issues/198.
regards thierry
On 12/7/23 09:06, Nyquist wrote:
Hello
We are using 18 389-ds-base-1.3.10.2,
Recently, one server (A) was set up to log to a pipe.
Afterwards, a phenomenon occurs where all server connections are occupied by an external system.
The external system has stopped.
After this incident, the servers returned to normal condition, but
Only server A had intermittent port 389 inaccessibility and delays.
After this, we determined that the problem on server A was due to logging in the pipe.
After changing the pipe to a file, I restarted the server.
After that the problem went away.
We want to know the exact cause of the delay and inaccessibility that occurred on server A.
How can logging to pipe affect directory server port 389 access?
Could it be a file descriptor related issue? Why were the other 17 servers able to recover to normal?
Thanks
389-users mailing list -- 389-users@lists.fedoraproject.org To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
1. Stop directory server and remove access log file. 2. Run logpipe.py. Python create access.pipe (pipe) 3. Change directory server settings nsslapd-accesslog-maxlogsperdir: 1 nsslapd-accesslog-logexpirationtime: -1 nsslapd-accesslog-logrotationtime: -1 nsslapd-accesslog: /var/log/dirsrv/slapd-localhost/access.pipe nsslapd-accesslog-logbuffering: off 4. Run directory server => Check normal execution, logpipe.py and custom.py are operating normally.
The problem arises after this. This server has consumed all connections. A forced shutdown of the external system returned the connection, and we expected it to work normally. However, port 389 health check failed intermittently. Joining and removing from the L4 switch is repeated. Could this be caused by my logging settings to pipe? Or should I think it's a problem with the Linux file system?
Hi,
I see two possible reasons why 389ds could intermittently stop listening on 389 port: [1] nearly all connections are exhausted. [2] the server is blocked (probably trying to write in the pipe)
[1] is possible if most of the connections did not get released when the external system was stopped. (It can happen if the ip connection gets silently broken (typically by rebooting a switch) ) In that case it is the fact of restarting 389ds that solved the issue. [2] Would mean that the application that read the pipe was not able to read the data fast enough. (Either too much data were logged or there was an issue on that application)
Regards, Pierre
On Thu, Dec 7, 2023 at 11:25 AM Nyquist dongminic@gmail.com wrote:
- Stop directory server and remove access log file.
- Run logpipe.py. Python create access.pipe (pipe)
- Change directory server settings
nsslapd-accesslog-maxlogsperdir: 1 nsslapd-accesslog-logexpirationtime: -1 nsslapd-accesslog-logrotationtime: -1 nsslapd-accesslog: /var/log/dirsrv/slapd-localhost/access.pipe nsslapd-accesslog-logbuffering: off 4. Run directory server => Check normal execution, logpipe.py and custom.py are operating normally.
The problem arises after this. This server has consumed all connections. A forced shutdown of the external system returned the connection, and we expected it to work normally. However, port 389 health check failed intermittently. Joining and removing from the L4 switch is repeated. Could this be caused by my logging settings to pipe? Or should I think it's a problem with the Linux file system? -- _______________________________________________ 389-users mailing list -- 389-users@lists.fedoraproject.org To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Hello.
If a log cannot be written (lack of space in the case of a file, lack of reader in the case of a pipe, etc.), does the directory server block the port? Will I be in a state where I cannot perform operations such as BIND, SEARCH?
Thank you
On Thu, Dec 7, 2023 at 11:25 AM Nyquist dongminic@gmail.com wrote:
- Stop directory server and remove access log file.
- Run logpipe.py. Python create access.pipe (pipe)
- Change directory server settings
nsslapd-accesslog-maxlogsperdir: 1 nsslapd-accesslog-logexpirationtime: -1 nsslapd-accesslog-logrotationtime: -1 nsslapd-accesslog: /var/log/dirsrv/slapd-localhost/access.pipe nsslapd-accesslog-logbuffering: off 4. Run directory server => Check normal execution, logpipe.py and custom.py are operating normally.
The problem arises after this. This server has consumed all connections. A forced shutdown of the external system returned the connection, and we expected it to work normally. However, port 389 health check failed intermittently. Joining and removing from the L4 switch is repeated. Could this be caused by my logging settings to pipe? Or should I think it's a problem with the Linux file system?
Judging by the version, it's a system with systemd, where logpipe was not working properly and the server would refuse to start. Some time ago I investigated issues with logpipe and systemd and wrote these instructions to make it work: https://bugzilla.redhat.com/show_bug.cgi?id=1520373#c5 (you can skip dsconf part as you already did have this configuration in cn=config). Could you please try this approach?
Thanks.
-- _______________________________________________ 389-users mailing list -- 389-users@lists.fedoraproject.org To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
As a result of the cause analysis, it was confirmed that the pipe reader process performance was poor. However, what was more problematic than that was the log buffering off option among the LDAP options. When buffering was not used, a block on port 389 occurred on all servers in response to large requests. Here's what I need to do: 1. Increasing the performance of the pipe reader, 2. If the log buffer is not used, measure the limitations of the LDAP server and reflect them in my architecture.
Thanks
389-users@lists.fedoraproject.org
-
Nyquist -
Pierre Rogier -
Thierry Bordaz -
Viktor Ashirov