On Mon, Dec 05, 2016 at 03:54:46AM -0000, zfnoctis@gmail.com wrote:
I've created some flamegraphs of sssd_be, hovering your mouse over the elements should show more information.
This is of sssd_be while running "id <domain user>" with the caches completely flushed: http://svgshare.com/i/MJ.svg
This is of sssd_be while running "sudo -l" as a domain user with sudo privileges: http://svgur.com/i/MX.svg
Clearly there are some symbols missing, but I wasn't sure how to resolve this. If someone could point me to the package(s) needed to fix this, I'd be happy to rerun them.
First, sorry for not following up sooner, but there was quite a bit of different new cases and bugs coming in internally last week, so the sssd-users list responses are a bit delayed. But some quick thoughts on the systemtap output and the flame graphs: 1) I think it would be nice (and this is a work for us) to decorate the paths in sssd_be a bit more with systemtap probes because there the probes just indicate that sssd started parsing entries which takes a long time, but we need to break down that part better into smaller parts to see which exact part of code is slow
2) Thank you very much for the flame graphs. I'm also a fan of Brendan Gregg's work :-) Could you tell us how exactly did you generate those? And to resolve the missing symbols, normally debuginfo-install sssd should be enough. If you already did that, maybe 'dnf --enablerepo=*debuginfo upgrade' would sync the debuginfo packages in sync with the non-debuginfo ones.
btw especially the flame graphs that captures the sudo output IMO correlatest with the theory that parsing the entries we receive from LDAP should be optimized, because the strcasecmp() function appears so often in the flame graph and I suspect this is coming from us finding the attributes and the values we are interested in.
I will try to find a bit more time this week to follow up (or ask some other developer :-) and decorate the hot pats with more systemtap probes.
Thank you for working on us on investigating the performance.