389-devel October 2018

389-devel@lists.fedoraproject.org

8 participants
36 discussions

Start a nNew thread

389 DS nightly 2018-10-31 - 89% PASS
by vashirov＠redhat.com 31 Oct '18

31 Oct '18

https://fedorapeople.org/groups/389ds/ci/nightly/2018/10/31/report-389-ds-b…

1 0

389 DS nightly 2018-10-30 - 90% PASS
by vashirov＠redhat.com 30 Oct '18

30 Oct '18

https://fedorapeople.org/groups/389ds/ci/nightly/2018/10/30/report-389-ds-b…

1 0

389 DS nightly 2018-10-29 - 90% PASS
by vashirov＠redhat.com 29 Oct '18

29 Oct '18

https://fedorapeople.org/groups/389ds/ci/nightly/2018/10/29/report-389-ds-b…

1 0

389 DS nightly 2018-10-28 - 91% PASS
by vashirov＠redhat.com 28 Oct '18

28 Oct '18

https://fedorapeople.org/groups/389ds/ci/nightly/2018/10/28/report-389-ds-b…

1 0

please review: PR 49996 - Revised internal operation logging format to include nesting level
by Mark Reynolds 26 Oct '18

26 Oct '18

https://pagure.io/389-ds-base/pull-request/49996

1 0

389 DS nightly 2018-10-26 - 91% PASS
by vashirov＠redhat.com 26 Oct '18

26 Oct '18

https://fedorapeople.org/groups/389ds/ci/nightly/2018/10/26/report-389-ds-b…

1 0

Profiling discussion
by William Brown 26 Oct '18

26 Oct '18

Hi there, In a ticket Thierry and I mentioned that we should have a quick discussion about ideas for profiling and what we want it to look like and what we need. I think it’s important we improve our observation into the server so that we can target improvements correctly, I think we should know: * Who is the target audience to run our profiling tools? * What kind of information we do want? * Potential solution for the above. With those in mind I think that Thierry suggested STAP scripts. * Target audience - developers (us) and some “highly experienced” admins (STAP is not the easiest thing to run). * Information - STAP would largely tell us timing and possibly allows some variable/struct extraction. STAP does allow us to look at connection info too a bit easier. I would suggest an “event” struct, and logging service At the start of an operation we create an event struct. As we enter - exit a plugin we can append timing information, and the plugin itself can add details (for example, backend could add idl performance metrics or other). At the end of the operation, we log the event struct as a json blob to our access log associated to the conn/op. * Target - anyone, it’s a log level. Really easy to enable (Think mailing list or user support, can easily send us diagnostic logs) * Information - we need a bit more work to structure the “event” struct internally for profiling, but we’d get timings and possibly internal variable data as well in the event. I think these are two possible approaches. STAP is less invasive, easier to start now, but harder to extend later. Logging is more accessible to users/admins, easier to extend later, but more work to add now. What do we think? — Sincerely, William

3 9

389 DS nightly 2018-10-25 - 91% PASS
by vashirov＠redhat.com 25 Oct '18

25 Oct '18

https://fedorapeople.org/groups/389ds/ci/nightly/2018/10/25/report-389-ds-b…

1 0

Nunc-stans status
by thierry bordaz 24 Oct '18

24 Oct '18

Hi, C10K is a scalability problem that a server can face when dealing with events of thousands of connections (i.e. clients) at the same time. Events can be new connections, new operations on the established connections, closure of connection (from client or server) For 389-ds, C10K problem was resolved with a new framework *Nunc-Stans* [1]. Nunc-stans was first enabled in RHDS 7.4 and improved/fixed in 7.5. Robustness issues [2] and [3] were reported in 7.5 and it was decided to disable Nunc-stans. It is not known if those issues exist or not in 7.4. William posted a PR to fix those two issues [4]. Nunc-stans is a complex framework, with its own dynamic. Review of this PR is not easy and even a careful review may not guaranty it will fix [2] and [3] and may not introduce others unexpected side effects. From there we discussed two options (but there may be others): 1. Review and merge the PR [4], then later run some intensive tests aiming to verify [2],[3] and checking the robustness in order to reenable NS 2. Build some tests for 1. measure the benefit of NS as [2] and [3] do not prevent some performance tests 2. identify possible reproducers for [2] and [3] 3. create robustness and long duration NS specific tests 4. review and merge the PR [4] As PR [4] is not intended for perf improvement, the step 2.1 will impact the priority according to the performance benefits. Comments are welcomed Regarding 2.1 plan we made the following notes for the test plan: /The benefit of Nunc-Stans can only be measure with a large number of connections (i.e. client) above 1000. That means a set of clients (sometime all) should keep their connection //*opened*//. Clients should run on several hosts so that clients are not the bootleneck./ /For the two types of events (new connection and new operations), the measurement could be/ * /Event: New connections / o /Start all clients in parallel to establish connections (keeping them opened) take the duration to get 1000, 2000, ... 10000 connections and check there are drop or not/ o /Establish 1000 connections and monitor during to open 100 more, the same starting with 2000, 10000/ o /Client should not run any operations during the monitoring/ * /Event: New operations / o /Start all clients and when 1000 connections are established, launch simple operations (e.g. search -s base -b "" objectclass) and monitor how many of them can be handled. The same with 2000, ... 10000./ o /response time and workqueue length could be monitored to be sure the bottleneck are not the worker./ [1] http://www.port389.org/docs/389ds/design/nunc-stans.html [2]https://bugzilla.redhat.com/show_bug.cgi?id=1608746 deadlock [3] https://bugzilla.redhat.com/show_bug.cgi?id=1605554 connection leaks [4]https://pagure.io/389-ds-base/pull-request/49636

3 5

What are we currently doing with docs?
by William Brown 24 Oct '18

24 Oct '18

I remember that Simon did some docs work for lib389. What are we currently using for that? I would like to extend this to the server core so that our man pages are easier to update, more extensive, and are kept up-to-date along with our releases. Sphinx was recommended and the name rings a bell …. — Sincerely, William

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

389-devel October 2018