Thanks for the info.
I have loaded 389-ds-base-debuginfo and attached the stack trace.
Hopefully this will be more telling.
Excellent. Yes, very nice.
This looks like
Thanks.
On Mon, Jul 22, 2013 at 8:18 AM, Rich Megginson <rmeggins(a)redhat.com
<mailto:rmeggins@redhat.com>> wrote:
On 07/22/2013 09:16 AM, Jeffrey Dunham wrote:
> Uploaded the stacktrace as specified by
>
http://port389.org/wiki/FAQ#Debugging_Hangs
> I will look into upgrading to 1.2.11, but it would be to know if
> the issue will actually be fixed there. Does anything stand out
> here?
No. Not sure what happened, but this stack trace is useless.
Looks like you don't have the right versions of the debuginfo
packages. Please check the directions at
http://port389.org/wiki/FAQ#Debugging_Crashes and Debugging_Hangs
>
>
> On Mon, Jul 22, 2013 at 8:02 AM, Rich Megginson
> <rmeggins(a)redhat.com <mailto:rmeggins@redhat.com>> wrote:
>
> On 07/22/2013 08:46 AM, Jeffrey Dunham wrote:
>> Here's thread 31:
>> (gdb) thread 31
>> [Switching to thread 31 (Thread 0x2b0e92901940 (LWP
>> 10739))]#0 0x00002b0df2847654 in __lll_lock_wait () from
>> /lib64/libpthread.so.0
>> (gdb) bt
>> #0 0x00002b0df2847654 in __lll_lock_wait () from
>> /lib64/libpthread.so.0
>> #1 0x00002b0df2842f80 in _L_lock_1233 () from
>> /lib64/libpthread.so.0
>> #2 0x00002b0df2842f03 in pthread_mutex_lock () from
>> /lib64/libpthread.so.0
>> #3 0x00002b0df2204289 in PR_Lock () from /usr/lib64/libnspr4.so
>> #4 0x0000000000411e88 in ber_sockbuf_free ()
>> #5 0x00002b0df04d652a in ?? () from
>> /usr/lib64/dirsrv/libslapd.so.0
>> #6 0x00002b0df04d6f1e in send_ldap_result_ext () from
>> /usr/lib64/dirsrv/libslapd.so.0
>> #7 0x00002b0df04d7421 in send_ldap_result () from
>> /usr/lib64/dirsrv/libslapd.so.0
>> #8 0x00002b0df04c58c4 in slapi_send_ldap_result () from
>> /usr/lib64/dirsrv/libslapd.so.0
>> #9 0x000000000042524e in ?? ()
>> #10 0x00002b0df04c70f7 in plugin_call_exop_plugins () from
>> /usr/lib64/dirsrv/libslapd.so.0
>> #11 0x000000000041841c in ber_sockbuf_free ()
>> #12 0x00000000004137ad in ber_sockbuf_free ()
>> #13 0x00002b0df22098cd in ?? () from /usr/lib64/libnspr4.so
>> #14 0x00002b0df284083d in start_thread () from
>> /lib64/libpthread.so.0
>> #15 0x00002b0df2b2af8d in clone () from /lib64/libc.so.6
>
> For hangs/deadlocks, please follow this:
>
>
http://port389.org/wiki/FAQ#Debugging_Hangs
>
> 1.2.11 has improved deadlock detection and handling
>
>
>>
>>
>> On Mon, Jul 22, 2013 at 7:42 AM, Ludwig Krispenz
>> <lkrispen(a)redhat.com <mailto:lkrispen@redhat.com>> wrote:
>>
>> could you add the bt for thread 31 ?
>>
>>
>> On 07/22/2013 03:58 PM, Jeffrey Dunham wrote:
>>> Hello,
>>>
>>> Starting a day ago we have been noticing deadlocks on
>>> select 389 servers. It does not happen on all of them
>>> (10% of servers have been affected currently, and no
>>> repeat offenders).
>>>
>>> Just looking for any hints on where to look or what
>>> could be causing this issue.
>>>
>>> Version: 389-Directory/1.2.10.14 <
http://1.2.10.14>
>>>
>>> From GDB:
>>> 0x00002b0df2847654 in __lll_lock_wait () from
>>> /lib64/libpthread.so.0
>>> (gdb) info threads
>>> 40 Thread 0x2b0e86c91940 (LWP 10689)
>>> 0x00002b0df2b24122 in select () from /lib64/libc.so.6
>>> 39 Thread 0x2b0e87692940 (LWP 10690)
>>> 0x00002b0df2b24122 in select () from /lib64/libc.so.6
>>> 38 Thread 0x2b0e88093940 (LWP 10691)
>>> 0x00002b0df2b24122 in select () from /lib64/libc.so.6
>>> 37 Thread 0x2b0e88a94940 (LWP 10692)
>>> 0x00002b0df2b24122 in select () from /lib64/libc.so.6
>>> 36 Thread 0x2b0e89495940 (LWP 10734)
>>> 0x00002b0df2845019 in pthread_cond_wait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_wait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 35 Thread 0x2b0e89e96940 (LWP 10735)
>>> 0x00002b0df2845019 in pthread_cond_wait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_wait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 34 Thread 0x2b0e8a897940 (LWP 10736)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 33 Thread 0x2b0e8b298940 (LWP 10737)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 32 Thread 0x2b0e91f00940 (LWP 10738)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 31 Thread 0x2b0e92901940 (LWP 10739)
>>> 0x00002b0df2847654 in __lll_lock_wait () from
>>> /lib64/libpthread.so.0
>>> 30 Thread 0x2b0e93302940 (LWP 10740)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 29 Thread 0x2b0e93d03940 (LWP 10741)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 28 Thread 0x2b0e94704940 (LWP 10742)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 27 Thread 0x2b0e95105940 (LWP 10743)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 26 Thread 0x2b0e95b06940 (LWP 10744)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 25 Thread 0x2b0e96507940 (LWP 10745)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 24 Thread 0x2b0e96f08940 (LWP 10746)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 23 Thread 0x2b0e97909940 (LWP 10747)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 22 Thread 0x2b0e9830a940 (LWP 10748)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 21 Thread 0x2b0e98d0b940 (LWP 10749)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 20 Thread 0x2b0e9970c940 (LWP 10750)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 19 Thread 0x2b0e9a10d940 (LWP 10751)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 18 Thread 0x2b0e9ab0e940 (LWP 10752)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 17 Thread 0x2b0e9b50f940 (LWP 10753)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 16 Thread 0x2b0e9bf10940 (LWP 10754)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 15 Thread 0x2b0e9c911940 (LWP 10755)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 14 Thread 0x2b0e9d312940 (LWP 10756)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 13 Thread 0x2b0e9dd13940 (LWP 10757)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 12 Thread 0x2b0e9e714940 (LWP 10758)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 11 Thread 0x2b0e9f115940 (LWP 10759)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 10 Thread 0x2b0e9fb16940 (LWP 10760)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 9 Thread 0x2b0ea0517940 (LWP 10761)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 8 Thread 0x2b0ea0f18940 (LWP 10762)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 7 Thread 0x2b0ea1919940 (LWP 10763)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 6 Thread 0x2b0ea231a940 (LWP 10764)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 5 Thread 0x2b0ea2d1b940 (LWP 10765)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 4 Thread 0x2b0ea371c940 (LWP 10766)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 3 Thread 0x2b0ea411d940 (LWP 10767)
>>> 0x00002b0df2845280 in
>>> pthread_cond_timedwait@(a)GLIBC_2.3.2
>>> <mailto:pthread_cond_timedwait@@GLIBC_2.3.2> () from
>>> /lib64/libpthread.so.0
>>> 2 Thread 0x2b0ea4b1e940 (LWP 10768)
>>> 0x00002b0df2b24122 in select () from /lib64/libc.so.6
>>> * 1 Thread 0x2b0df452a7e0 (LWP 10666)
>>> 0x00002b0df2847654 in __lll_lock_wait () from
>>> /lib64/libpthread.so.0
>>> (gdb) bt
>>> #0 0x00002b0df2847654 in __lll_lock_wait () from
>>> /lib64/libpthread.so.0
>>> #1 0x00002b0df2842f80 in _L_lock_1233 () from
>>> /lib64/libpthread.so.0
>>> #2 0x00002b0df2842f03 in pthread_mutex_lock () from
>>> /lib64/libpthread.so.0
>>> #3 0x00002b0df2204289 in PR_Lock () from
>>> /usr/lib64/libnspr4.so
>>> #4 0x0000000000415955 in ber_sockbuf_free ()
>>> #5 0x00000000004166f1 in ber_sockbuf_free ()
>>> #6 0x000000000041d7d4 in ?? ()
>>> #7 0x00002b0df2a739c4 in __libc_start_main () from
>>> /lib64/libc.so.6
>>> #8 0x000000000040c9b9 in ber_sockbuf_free ()
>>> #9 0x00007fff5cd57a98 in ?? ()
>>> #10 0x0000000000000000 in ?? ()
>>>
>>>
>>>
>>> --
>>> 389 users mailing list
>>> 389-users(a)lists.fedoraproject.org
<mailto:389-users@lists.fedoraproject.org>
>>>
https://admin.fedoraproject.org/mailman/listinfo/389-users
>>
>>
>> --
>> 389 users mailing list
>> 389-users(a)lists.fedoraproject.org
>> <mailto:389-users@lists.fedoraproject.org>
>>
https://admin.fedoraproject.org/mailman/listinfo/389-users
>>
>>
>>
>>
>> --
>> 389 users mailing list
>> 389-users(a)lists.fedoraproject.org
<mailto:389-users@lists.fedoraproject.org>
>>
https://admin.fedoraproject.org/mailman/listinfo/389-users
>
>