FYI. El9, latest stable.
Mar 04 22:46:05 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: ipaserver.dnssec.bindmgr: INFO attrs: <ldap.cidict.cidict object at 0x7f283b53f7f0> Mar 04 22:46:05 registry1.1.quietfountain.com named[1463]: client @0x7fd61803ae68 10.12.112.3#49016: received notify for zone '0.0.0.0.0.0.a.0.0.8.9.2.4.0.6.2.ip6.arpa' Mar 04 22:46:05 registry1.1.quietfountain.com named[1463]: client @0x7fd618191538 fc00:1002:c7::3#41560: received notify for zone '0.0.0.0.0.0.a.0.0.8.9.2.4.0.6.2.ip6.arpa' Mar 04 22:46:05 registry1.1.quietfountain.com dnssec-keyfromlabel[75984]: Configuration.cpp(96): Missing log.level in configuration. Using default value: INFO Mar 04 22:46:05 registry1.1.quietfountain.com dnssec-keyfromlabel[75984]: Configuration.cpp(96): Missing slots.mechanisms in configuration. Using default value: ALL Mar 04 22:46:05 registry1.1.quietfountain.com dnssec-keyfromlabel[75984]: Configuration.cpp(124): Missing slots.removable in configuration. Using default value: false Mar 04 22:46:05 registry1.1.quietfountain.com systemd[1]: ipa-ods-exporter.service: Deactivated successfully. Mar 04 22:46:05 registry1.1.quietfountain.com systemd[1]: ipa-ods-exporter.service: Consumed 4.047s CPU time. Mar 04 22:46:05 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: ipaserver.dnssec.bindmgr: INFO attrs: <ldap.cidict.cidict object at 0x7f283b53faf0> Mar 04 22:46:06 registry1.1.quietfountain.com dnssec-keyfromlabel[75991]: Configuration.cpp(96): Missing log.level in configuration. Using default value: INFO Mar 04 22:46:06 registry1.1.quietfountain.com dnssec-keyfromlabel[75991]: Configuration.cpp(96): Missing slots.mechanisms in configuration. Using default value: ALL Mar 04 22:46:06 registry1.1.quietfountain.com dnssec-keyfromlabel[75991]: Configuration.cpp(124): Missing slots.removable in configuration. Using default value: false Mar 04 22:46:06 registry1.1.quietfountain.com named[1463]: client @0x7fd61803ae68 10.12.112.3#49016: received notify for zone '0.0.0.0.0.0.f.0.0.8.9.2.4.0.6.2.ip6.arpa' Mar 04 22:46:06 registry1.1.quietfountain.com named[1463]: client @0x7fd618191538 fc00:1002:c7::3#41560: received notify for zone '0.0.0.0.0.0.f.0.0.8.9.2.4.0.6.2.ip6.arpa' Mar 04 22:46:06 registry1.1.quietfountain.com kernel: dnssec-keyfroml[75991]: segfault at 18 ip 00007f7fba586284 sp 00007ffff3e834a8 error 4 in libsofthsm2.so[7f7fba4ef000+9b000] likely on CPU 0 (core 0, socket 0) Mar 04 22:46:06 registry1.1.quietfountain.com kernel: Code: 75 f4 48 8b 53 30 49 89 4c 24 20 49 89 44 24 10 49 89 54 24 28 48 83 c4 08 4c 89 e0 5b 41 5c c3 66 0f 1f 44 00 00 f3 0f 1e fa <48> 8b 47 18 48 85 c0 74 44 4c 8d 47 10 4c 89 c7 eb 12 66 2e 0f 1f Mar 04 22:46:06 registry1.1.quietfountain.com systemd[1]: Started Process Core Dump (PID 75992/UID 0). Mar 04 22:46:06 registry1.1.quietfountain.com systemd-coredump[75993]: Resource limits disable core dumping for process 75991 (dnssec-keyfroml). Mar 04 22:46:06 registry1.1.quietfountain.com systemd-coredump[75993]: [🡕] Process 75991 (dnssec-keyfroml) of user 984 dumped core. Mar 04 22:46:06 registry1.1.quietfountain.com systemd[1]:systemd-coredump@40-75992-0.service: Deactivated successfully. Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: Traceback (most recent call last): Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/libexec/ipa/ipa-dnskeysyncd", line 130, in <module> Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: while ldap_connection.syncrepl_poll(all=1, msgid=ldap_search): Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/lib64/python3.9/site-packages/ldap/syncrepl.py", line 464, in syncrepl_poll Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: self.syncrepl_refreshdone() Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipaserver/dnssec/keysyncer.py", line 128, in syncrepl_refreshdone Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: self.bindmgr.sync(self.dnssec_zones) Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipaserver/dnssec/bindmgr.py", line 232, in sync Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: self.sync_zone(zone) Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipaserver/dnssec/bindmgr.py", line 205, in sync_zone Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: self.install_key(zone, uuid, attrs, tempdir) Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipaserver/dnssec/bindmgr.py", line 146, in install_key Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: result = ipautil.run(cmd, capture_output=True) Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipapython/ipautil.py", line 599, in run Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: raise CalledProcessError( Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: ipapython.ipautil.CalledProcessError: CalledProcessError(Command ['/usr/sbin/dnssec-keyfromlabel', '-E', 'pkcs11', '-K', '/var/named/dyndb-ldap/ipa/master/_domainkey.rockstable.systems/XXXXXX', '-a', b'RSASHA256', '-l', b'pkcs11:object=YYYYY;pin-source=/var/lib/ipa/dnssec/softhsm_pin', '-P', b'20231101161909', '-A', 'none', '-I', 'none', '-D', 'none', '-E', 'pkcs11', '_domainkey.rockstable.systems.'] returned non-zero exit status -11: 'Found uninitialized token\nFound uninitialized token\nKey not found.\nPKCS11_load_public_key returned NULL\nFound uninitialized token\nKey not found.\nPKCS11_get_private_key returned NULL\ndnssec-keyfromlabel: warning: ENGINE_load_private_key failed (not found)\ndnssec-keyfromlabel: fatal: failed to get key _domainkey.rockstable.systems/RSASHA256: not found\n') Mar 04 22:46:06 registry1.1.quietfountain.com named[1463]: dispatch 0x7fd618009200: shutting down due to TCP receive error: 2001:13c7:7002:3000::11#53: connection reset
Here's ChatGPT feedback regarding your server logs...
Analyzing the provided log messages from a RHEL 9 Linux server, we can identify several events. Here's a breakdown of each event, including its probable cause and some examples:
1. ipa-dnskeysyncd LDAP Operations: - Messages from `ipa-dnskeysyncd` show LDAP operations related to DNSSEC key synchronization. - Example: `ipa-dnskeysyncd[75834]: ipaserver.dnssec.bindmgr: INFO attrs: <ldap.cidict.cidict object at 0x7f283b53f7f0>`. - Probable Cause: Routine operations of DNSSEC synchronization with IPA server.
2. DNS Notify Messages: - Messages from `named` indicate it received DNS notify messages for specific zones. - Example: `named[1463]: client @0x7fd61803ae68 10.12.112.3#49016: received notify for zone '0.0.0.0.0.0.a.0.0.8.9.2.4.0.6.2.ip6.arpa'`. - Probable Cause: Notification of changes in DNS zones, likely part of routine DNS zone updates.
3. dnssec-keyfromlabel Configuration Warnings: - Warnings from `dnssec-keyfromlabel` about missing configuration parameters. - Example: `dnssec-keyfromlabel[75984]: Configuration.cpp(96): Missing log.level in configuration. Using default value: INFO`. - Probable Cause: Default configuration being used due to missing specific configuration settings.
4. Systemd Service Messages: - `systemd` messages about `ipa-ods-exporter.service`. - Example: `systemd[1]: ipa-ods-exporter.service: Consumed 4.047s CPU time`. - Probable Cause: Normal systemd logging of service status.
5. Kernel Segmentation Fault (Core Dump): - Critical: A segmentation fault in `dnssec-keyfroml`. - Example: `kernel: dnssec-keyfroml[75991]: segfault at 18 ip 00007f7fba586284 sp 00007ffff3e834a8 error 4 in libsofthsm2.so[7f7fba4ef000+9b000]`. - Probable Cause: A crash in the `dnssec-keyfromlabel` process, possibly due to an issue with `libsofthsm2`.
6. Systemd Core Dump Processing: - Systemd handling of the core dump created by the crash. - Example: `systemd-coredump[75993]: Process 75991 (dnssec-keyfroml) of user 984 dumped core`. - Indicates that a core dump was generated for further analysis.
7. ipa-dnskeysyncd Python Traceback: - A Python traceback from `ipa-dnskeysyncd`, indicating an error in the script. - Example: `ipa-dnskeysyncd[75834]: ipapython.ipautil.CalledProcessError: Command [...] returned non-zero exit status -11`. - Probable Cause: The error seems related to the `dnssec-keyfromlabel` command failure.
8. Named TCP Receive Error: - Named server shutting down a TCP connection due to an error. - Example: `named[1463]: dispatch 0x7fd618009200: shutting down due to TCP receive error: connection reset`. - Probable Cause: Network-related issue or remote client reset the connection.
### Steps to Troubleshoot Core Dump
To investigate the core dump caused by `dnssec-keyfroml`:
1. Locate the Core Dump: - Core dumps are typically stored in `/var/lib/systemd/coredump/` or `/var/crash/` on RHEL systems. - Use `coredumpctl list` to list all core dumps and find the one related to `dnssec-keyfroml`.
2. Analyze the Core Dump: - Use `coredumpctl info <PID>` or `gdb` to analyze the core dump. Replace `<PID>` with the process ID of `dnssec-keyfroml`. - This will provide stack traces and other details about the state of the program at the time of the crash.
3. Check Dependencies: - Ensure that all dependencies, especially `libsofthsm2`, are correctly installed and not corrupted.
4. Review Configuration and Logs: - Check the configuration files for `dnssec-keyfromlabel` and related services. - Review system logs for any anomalies or errors leading up to the crash.
5. Test in a Controlled Environment: - If possible, replicate the issue in a test environment to better understand the conditions leading to the creation of the core dump file.
On Mon, Mar 4, 2024 at 10:54 PM Harry G Coin via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
FYI. El9, latest stable.
Mar 04 22:46:05 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: ipaserver.dnssec.bindmgr: INFO attrs: <ldap.cidict.cidict object at 0x7f283b53f7f0> Mar 04 22:46:05 registry1.1.quietfountain.com named[1463]: client @0x7fd61803ae68 10.12.112.3#49016: received notify for zone '0.0.0.0.0.0.a.0.0.8.9.2.4.0.6.2.ip6.arpa' Mar 04 22:46:05 registry1.1.quietfountain.com named[1463]: client @0x7fd618191538 fc00:1002:c7::3#41560: received notify for zone '0.0.0.0.0.0.a.0.0.8.9.2.4.0.6.2.ip6.arpa' Mar 04 22:46:05 registry1.1.quietfountain.com dnssec-keyfromlabel[75984]: Configuration.cpp(96): Missing log.level in configuration. Using default value: INFO Mar 04 22:46:05 registry1.1.quietfountain.com dnssec-keyfromlabel[75984]: Configuration.cpp(96): Missing slots.mechanisms in configuration. Using default value: ALL Mar 04 22:46:05 registry1.1.quietfountain.com dnssec-keyfromlabel[75984]: Configuration.cpp(124): Missing slots.removable in configuration. Using default value: false Mar 04 22:46:05 registry1.1.quietfountain.com systemd[1]: ipa-ods-exporter.service: Deactivated successfully. Mar 04 22:46:05 registry1.1.quietfountain.com systemd[1]: ipa-ods-exporter.service: Consumed 4.047s CPU time. Mar 04 22:46:05 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: ipaserver.dnssec.bindmgr: INFO attrs: <ldap.cidict.cidict object at 0x7f283b53faf0> Mar 04 22:46:06 registry1.1.quietfountain.com dnssec-keyfromlabel[75991]: Configuration.cpp(96): Missing log.level in configuration. Using default value: INFO Mar 04 22:46:06 registry1.1.quietfountain.com dnssec-keyfromlabel[75991]: Configuration.cpp(96): Missing slots.mechanisms in configuration. Using default value: ALL Mar 04 22:46:06 registry1.1.quietfountain.com dnssec-keyfromlabel[75991]: Configuration.cpp(124): Missing slots.removable in configuration. Using default value: false Mar 04 22:46:06 registry1.1.quietfountain.com named[1463]: client @0x7fd61803ae68 10.12.112.3#49016: received notify for zone '0.0.0.0.0.0.f.0.0.8.9.2.4.0.6.2.ip6.arpa' Mar 04 22:46:06 registry1.1.quietfountain.com named[1463]: client @0x7fd618191538 fc00:1002:c7::3#41560: received notify for zone '0.0.0.0.0.0.f.0.0.8.9.2.4.0.6.2.ip6.arpa' Mar 04 22:46:06 registry1.1.quietfountain.com kernel: dnssec-keyfroml[75991]: segfault at 18 ip 00007f7fba586284 sp 00007ffff3e834a8 error 4 in libsofthsm2.so[7f7fba4ef000+9b000] likely on CPU 0 (core 0, socket 0) Mar 04 22:46:06 registry1.1.quietfountain.com kernel: Code: 75 f4 48 8b 53 30 49 89 4c 24 20 49 89 44 24 10 49 89 54 24 28 48 83 c4 08 4c 89 e0 5b 41 5c c3 66 0f 1f 44 00 00 f3 0f 1e fa <48> 8b 47 18 48 85 c0 74 44 4c 8d 47 10 4c 89 c7 eb 12 66 2e 0f 1f Mar 04 22:46:06 registry1.1.quietfountain.com systemd[1]: Started Process Core Dump (PID 75992/UID 0). Mar 04 22:46:06 registry1.1.quietfountain.com systemd-coredump[75993]: Resource limits disable core dumping for process 75991 (dnssec-keyfroml). Mar 04 22:46:06 registry1.1.quietfountain.com systemd-coredump[75993]: [🡕] Process 75991 (dnssec-keyfroml) of user 984 dumped core. Mar 04 22:46:06 registry1.1.quietfountain.com systemd[1]: systemd-coredump@40-75992-0.service: Deactivated successfully. Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: Traceback (most recent call last): Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/libexec/ipa/ipa-dnskeysyncd", line 130, in <module> Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: while ldap_connection.syncrepl_poll(all=1, msgid=ldap_search): Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/lib64/python3.9/site-packages/ldap/syncrepl.py", line 464, in syncrepl_poll Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: self.syncrepl_refreshdone() Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipaserver/dnssec/keysyncer.py", line 128, in syncrepl_refreshdone Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: self.bindmgr.sync(self.dnssec_zones) Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipaserver/dnssec/bindmgr.py", line 232, in sync Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: self.sync_zone(zone) Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipaserver/dnssec/bindmgr.py", line 205, in sync_zone Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: self.install_key(zone, uuid, attrs, tempdir) Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipaserver/dnssec/bindmgr.py", line 146, in install_key Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: result = ipautil.run(cmd, capture_output=True) Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipapython/ipautil.py", line 599, in run Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: raise CalledProcessError( Mar 04 22:46:07 registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: ipapython.ipautil.CalledProcessError: CalledProcessError(Command ['/usr/sbin/dnssec-keyfromlabel', '-E', 'pkcs11', '-K', '/var/named/dyndb-ldap/ipa/master/_domainkey.rockstable.systems/XXXXXX', '-a', b'RSASHA256', '-l', b'pkcs11:object=YYYYY;pin-source=/var/lib/ipa/dnssec/softhsm_pin', '-P', b'20231101161909', '-A', 'none', '-I', 'none', '-D', 'none', '-E', 'pkcs11', '_domainkey.rockstable.systems.'] returned non-zero exit status -11: 'Found uninitialized token\nFound uninitialized token\nKey not found.\nPKCS11_load_public_key returned NULL\nFound uninitialized token\nKey not found.\nPKCS11_get_private_key returned NULL\ndnssec-keyfromlabel: warning: ENGINE_load_private_key failed (not found)\ndnssec-keyfromlabel: fatal: failed to get key _domainkey.rockstable.systems/RSASHA256: not found\n') Mar 04 22:46:06 registry1.1.quietfountain.com named[1463]: dispatch 0x7fd618009200: shutting down due to TCP receive error: 2001:13c7:7002:3000::11#53: connection reset
-- _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Update:
This same core dump pattern appears across all freeipa masters running dns in this sandbox, ruling out hardware causes. Ideas?
On Mon, Mar 4, 2024 at 10:54 PM Harry G Coin via FreeIPA-users freeipa-users@lists.fedorahosted.org wrote:
FYI. El9, latest stable. Mar 04 22:46:05registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> ipa-dnskeysyncd[75834]: ipaserver.dnssec.bindmgr: INFO attrs: <ldap.cidict.cidict object at 0x7f283b53f7f0> Mar 04 22:46:05registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> named[1463]: client @0x7fd61803ae68 10.12.112.3#49016: received notify for zone '0.0.0.0.0.0.a.0.0.8.9.2.4.0.6.2.ip6.arpa' Mar 04 22:46:05registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> named[1463]: client @0x7fd618191538 fc00:1002:c7::3#41560: received notify for zone '0.0.0.0.0.0.a.0.0.8.9.2.4.0.6.2.ip6.arpa' Mar 04 22:46:05registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> dnssec-keyfromlabel[75984]: Configuration.cpp(96): Missing log.level in configuration. Using default value: INFO Mar 04 22:46:05registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> dnssec-keyfromlabel[75984]: Configuration.cpp(96): Missing slots.mechanisms in configuration. Using default value: ALL Mar 04 22:46:05registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> dnssec-keyfromlabel[75984]: Configuration.cpp(124): Missing slots.removable in configuration. Using default value: false Mar 04 22:46:05registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> systemd[1]: ipa-ods-exporter.service: Deactivated successfully. Mar 04 22:46:05registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> systemd[1]: ipa-ods-exporter.service: Consumed 4.047s CPU time. Mar 04 22:46:05registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> ipa-dnskeysyncd[75834]: ipaserver.dnssec.bindmgr: INFO attrs: <ldap.cidict.cidict object at 0x7f283b53faf0> Mar 04 22:46:06registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> dnssec-keyfromlabel[75991]: Configuration.cpp(96): Missing log.level in configuration. Using default value: INFO Mar 04 22:46:06registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> dnssec-keyfromlabel[75991]: Configuration.cpp(96): Missing slots.mechanisms in configuration. Using default value: ALL Mar 04 22:46:06registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> dnssec-keyfromlabel[75991]: Configuration.cpp(124): Missing slots.removable in configuration. Using default value: false Mar 04 22:46:06registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> named[1463]: client @0x7fd61803ae68 10.12.112.3#49016: received notify for zone '0.0.0.0.0.0.f.0.0.8.9.2.4.0.6.2.ip6.arpa' Mar 04 22:46:06registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> named[1463]: client @0x7fd618191538 fc00:1002:c7::3#41560: received notify for zone '0.0.0.0.0.0.f.0.0.8.9.2.4.0.6.2.ip6.arpa' Mar 04 22:46:06registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> kernel: dnssec-keyfroml[75991]: segfault at 18 ip 00007f7fba586284 sp 00007ffff3e834a8 error 4 in libsofthsm2.so[7f7fba4ef000+9b000] likely on CPU 0 (core 0, socket 0) Mar 04 22:46:06registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> kernel: Code: 75 f4 48 8b 53 30 49 89 4c 24 20 49 89 44 24 10 49 89 54 24 28 48 83 c4 08 4c 89 e0 5b 41 5c c3 66 0f 1f 44 00 00 f3 0f 1e fa <48> 8b 47 18 48 85 c0 74 44 4c 8d 47 10 4c 89 c7 eb 12 66 2e 0f 1f Mar 04 22:46:06registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> systemd[1]: Started Process Core Dump (PID 75992/UID 0). Mar 04 22:46:06registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> systemd-coredump[75993]: Resource limits disable core dumping for process 75991 (dnssec-keyfroml). Mar 04 22:46:06registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> systemd-coredump[75993]: [🡕] Process 75991 (dnssec-keyfroml) of user 984 dumped core. Mar 04 22:46:06registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> systemd[1]:systemd-coredump@40-75992-0.service: Deactivated successfully. Mar 04 22:46:07registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> ipa-dnskeysyncd[75834]: Traceback (most recent call last): Mar 04 22:46:07registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> ipa-dnskeysyncd[75834]: File "/usr/libexec/ipa/ipa-dnskeysyncd", line 130, in <module> Mar 04 22:46:07registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> ipa-dnskeysyncd[75834]: while ldap_connection.syncrepl_poll(all=1, msgid=ldap_search): Mar 04 22:46:07registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> ipa-dnskeysyncd[75834]: File "/usr/lib64/python3.9/site-packages/ldap/syncrepl.py", line 464, in syncrepl_poll Mar 04 22:46:07registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> ipa-dnskeysyncd[75834]: self.syncrepl_refreshdone() Mar 04 22:46:07registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipaserver/dnssec/keysyncer.py", line 128, in syncrepl_refreshdone Mar 04 22:46:07registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> ipa-dnskeysyncd[75834]: self.bindmgr.sync(self.dnssec_zones) Mar 04 22:46:07registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipaserver/dnssec/bindmgr.py", line 232, in sync Mar 04 22:46:07registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> ipa-dnskeysyncd[75834]: self.sync_zone(zone) Mar 04 22:46:07registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipaserver/dnssec/bindmgr.py", line 205, in sync_zone Mar 04 22:46:07registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> ipa-dnskeysyncd[75834]: self.install_key(zone, uuid, attrs, tempdir) Mar 04 22:46:07registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipaserver/dnssec/bindmgr.py", line 146, in install_key Mar 04 22:46:07registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> ipa-dnskeysyncd[75834]: result = ipautil.run(cmd, capture_output=True) Mar 04 22:46:07registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipapython/ipautil.py", line 599, in run Mar 04 22:46:07registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> ipa-dnskeysyncd[75834]: raise CalledProcessError( Mar 04 22:46:07registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> ipa-dnskeysyncd[75834]: ipapython.ipautil.CalledProcessError: CalledProcessError(Command ['/usr/sbin/dnssec-keyfromlabel', '-E', 'pkcs11', '-K', '/var/named/dyndb-ldap/ipa/master/_domainkey.rockstable.systems/XXXXXX', '-a', b'RSASHA256', '-l', b'pkcs11:object=YYYYY;pin-source=/var/lib/ipa/dnssec/softhsm_pin', '-P', b'20231101161909', '-A', 'none', '-I', 'none', '-D', 'none', '-E', 'pkcs11', '_domainkey.rockstable.systems.'] returned non-zero exit status -11: 'Found uninitialized token\nFound uninitialized token\nKey not found.\nPKCS11_load_public_key returned NULL\nFound uninitialized token\nKey not found.\nPKCS11_get_private_key returned NULL\ndnssec-keyfromlabel: warning: ENGINE_load_private_key failed (not found)\ndnssec-keyfromlabel: fatal: failed to get key _domainkey.rockstable.systems/RSASHA256: not found\n') Mar 04 22:46:06registry1.1.quietfountain.com <http://registry1.1.quietfountain.com> named[1463]: dispatch 0x7fd618009200: shutting down due to TCP receive error: 2001:13c7:7002:3000::11#53: connection reset -- _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue-- _______________________________________________ FreeIPA-users mailing list --freeipa-users@lists.fedorahosted.org To unsubscribe send an email tofreeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste... Do not reply to spam, report it:https://pagure.io/fedora-infrastructure/new_issue
On Аўт, 05 сак 2024, Harry G Coin via FreeIPA-users wrote:
Update:
This same core dump pattern appears across all freeipa masters running dns in this sandbox, ruling out hardware causes. Ideas?
This is most likely atexit callback issue in softhsmv2. Or, an issue with openssl callbacks being run automatically. I updated softhsm in Rawhide recently to include a patch which supposed to fix most of these problems, contributed by an OpenSSL's developer. We haven't seen issues like that in rawhide after that yet.
I am planning to add this patch to CentOS 9 Stream soon.
If you have actual coredump, can you send it to me off-list, along with the details on the packages you used?
On Mon, Mar 4, 2024 at 10:54 PM Harry G Coin via FreeIPA-users freeipa-users@lists.fedorahosted.org wrote:
FYI. El9, latest stable.
Mar 04 22:46:05registry1.1.quietfountain.com http://registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: ipaserver.dnssec.bindmgr: INFO attrs: <ldap.cidict.cidict object at 0x7f283b53f7f0> Mar 04 22:46:05registry1.1.quietfountain.com http://registry1.1.quietfountain.com named[1463]: client @0x7fd61803ae68 10.12.112.3#49016: received notify for zone '0.0.0.0.0.0.a.0.0.8.9.2.4.0.6.2.ip6.arpa' Mar 04 22:46:05registry1.1.quietfountain.com http://registry1.1.quietfountain.com named[1463]: client @0x7fd618191538 fc00:1002:c7::3#41560: received notify for zone '0.0.0.0.0.0.a.0.0.8.9.2.4.0.6.2.ip6.arpa' Mar 04 22:46:05registry1.1.quietfountain.com http://registry1.1.quietfountain.com dnssec-keyfromlabel[75984]: Configuration.cpp(96): Missing log.level in configuration. Using default value: INFO Mar 04 22:46:05registry1.1.quietfountain.com http://registry1.1.quietfountain.com dnssec-keyfromlabel[75984]: Configuration.cpp(96): Missing slots.mechanisms in configuration. Using default value: ALL Mar 04 22:46:05registry1.1.quietfountain.com http://registry1.1.quietfountain.com dnssec-keyfromlabel[75984]: Configuration.cpp(124): Missing slots.removable in configuration. Using default value: false Mar 04 22:46:05registry1.1.quietfountain.com http://registry1.1.quietfountain.com systemd[1]: ipa-ods-exporter.service: Deactivated successfully. Mar 04 22:46:05registry1.1.quietfountain.com http://registry1.1.quietfountain.com systemd[1]: ipa-ods-exporter.service: Consumed 4.047s CPU time. Mar 04 22:46:05registry1.1.quietfountain.com http://registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: ipaserver.dnssec.bindmgr: INFO attrs: <ldap.cidict.cidict object at 0x7f283b53faf0> Mar 04 22:46:06registry1.1.quietfountain.com http://registry1.1.quietfountain.com dnssec-keyfromlabel[75991]: Configuration.cpp(96): Missing log.level in configuration. Using default value: INFO Mar 04 22:46:06registry1.1.quietfountain.com http://registry1.1.quietfountain.com dnssec-keyfromlabel[75991]: Configuration.cpp(96): Missing slots.mechanisms in configuration. Using default value: ALL Mar 04 22:46:06registry1.1.quietfountain.com http://registry1.1.quietfountain.com dnssec-keyfromlabel[75991]: Configuration.cpp(124): Missing slots.removable in configuration. Using default value: false Mar 04 22:46:06registry1.1.quietfountain.com http://registry1.1.quietfountain.com named[1463]: client @0x7fd61803ae68 10.12.112.3#49016: received notify for zone '0.0.0.0.0.0.f.0.0.8.9.2.4.0.6.2.ip6.arpa' Mar 04 22:46:06registry1.1.quietfountain.com http://registry1.1.quietfountain.com named[1463]: client @0x7fd618191538 fc00:1002:c7::3#41560: received notify for zone '0.0.0.0.0.0.f.0.0.8.9.2.4.0.6.2.ip6.arpa' Mar 04 22:46:06registry1.1.quietfountain.com http://registry1.1.quietfountain.com kernel: dnssec-keyfroml[75991]: segfault at 18 ip 00007f7fba586284 sp 00007ffff3e834a8 error 4 in libsofthsm2.so[7f7fba4ef000+9b000] likely on CPU 0 (core 0, socket 0) Mar 04 22:46:06registry1.1.quietfountain.com http://registry1.1.quietfountain.com kernel: Code: 75 f4 48 8b 53 30 49 89 4c 24 20 49 89 44 24 10 49 89 54 24 28 48 83 c4 08 4c 89 e0 5b 41 5c c3 66 0f 1f 44 00 00 f3 0f 1e fa <48> 8b 47 18 48 85 c0 74 44 4c 8d 47 10 4c 89 c7 eb 12 66 2e 0f 1f Mar 04 22:46:06registry1.1.quietfountain.com http://registry1.1.quietfountain.com systemd[1]: Started Process Core Dump (PID 75992/UID 0). Mar 04 22:46:06registry1.1.quietfountain.com http://registry1.1.quietfountain.com systemd-coredump[75993]: Resource limits disable core dumping for process 75991 (dnssec-keyfroml). Mar 04 22:46:06registry1.1.quietfountain.com http://registry1.1.quietfountain.com systemd-coredump[75993]: [🡕] Process 75991 (dnssec-keyfroml) of user 984 dumped core. Mar 04 22:46:06registry1.1.quietfountain.com http://registry1.1.quietfountain.com systemd[1]:systemd-coredump@40-75992-0.service: Deactivated successfully. Mar 04 22:46:07registry1.1.quietfountain.com http://registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: Traceback (most recent call last): Mar 04 22:46:07registry1.1.quietfountain.com http://registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/libexec/ipa/ipa-dnskeysyncd", line 130, in <module> Mar 04 22:46:07registry1.1.quietfountain.com http://registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: while ldap_connection.syncrepl_poll(all=1, msgid=ldap_search): Mar 04 22:46:07registry1.1.quietfountain.com http://registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/lib64/python3.9/site-packages/ldap/syncrepl.py", line 464, in syncrepl_poll Mar 04 22:46:07registry1.1.quietfountain.com http://registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: self.syncrepl_refreshdone() Mar 04 22:46:07registry1.1.quietfountain.com http://registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipaserver/dnssec/keysyncer.py", line 128, in syncrepl_refreshdone Mar 04 22:46:07registry1.1.quietfountain.com http://registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: self.bindmgr.sync(self.dnssec_zones) Mar 04 22:46:07registry1.1.quietfountain.com http://registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipaserver/dnssec/bindmgr.py", line 232, in sync Mar 04 22:46:07registry1.1.quietfountain.com http://registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: self.sync_zone(zone) Mar 04 22:46:07registry1.1.quietfountain.com http://registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipaserver/dnssec/bindmgr.py", line 205, in sync_zone Mar 04 22:46:07registry1.1.quietfountain.com http://registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: self.install_key(zone, uuid, attrs, tempdir) Mar 04 22:46:07registry1.1.quietfountain.com http://registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipaserver/dnssec/bindmgr.py", line 146, in install_key Mar 04 22:46:07registry1.1.quietfountain.com http://registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: result = ipautil.run(cmd, capture_output=True) Mar 04 22:46:07registry1.1.quietfountain.com http://registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File "/usr/lib/python3.9/site-packages/ipapython/ipautil.py", line 599, in run Mar 04 22:46:07registry1.1.quietfountain.com http://registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: raise CalledProcessError( Mar 04 22:46:07registry1.1.quietfountain.com http://registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: ipapython.ipautil.CalledProcessError: CalledProcessError(Command ['/usr/sbin/dnssec-keyfromlabel', '-E', 'pkcs11', '-K', '/var/named/dyndb-ldap/ipa/master/_domainkey.rockstable.systems/XXXXXX', '-a', b'RSASHA256', '-l', b'pkcs11:object=YYYYY;pin-source=/var/lib/ipa/dnssec/softhsm_pin', '-P', b'20231101161909', '-A', 'none', '-I', 'none', '-D', 'none', '-E', 'pkcs11', '_domainkey.rockstable.systems.'] returned non-zero exit status -11: 'Found uninitialized token\nFound uninitialized token\nKey not found.\nPKCS11_load_public_key returned NULL\nFound uninitialized token\nKey not found.\nPKCS11_get_private_key returned NULL\ndnssec-keyfromlabel: warning: ENGINE_load_private_key failed (not found)\ndnssec-keyfromlabel: fatal: failed to get key _domainkey.rockstable.systems/RSASHA256: not found\n') Mar 04 22:46:06registry1.1.quietfountain.com http://registry1.1.quietfountain.com named[1463]: dispatch 0x7fd618009200: shutting down due to TCP receive error: 2001:13c7:7002:3000::11#53: connection reset
-- _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
-- _______________________________________________ FreeIPA-users mailing list --freeipa-users@lists.fedorahosted.org To unsubscribe send an email tofreeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste... Do not reply to spam, report it:https://pagure.io/fedora-infrastructure/new_issue
freeipa-users@lists.fedorahosted.org
-
Alexander Bokovoy -
Carlos Eduardo Porter -
Harry G Coin