[PATCH] postgresql_server / db-koji01: Adjust a bunch more for performance.
by kevin@scrye.com
From: Kevin Fenzi <kevin(a)scrye.com>
The current settings cause database dumps to drive the load way up
and make the entire application slow, so we need to adjust.
Using pgtune, these values might well be better.
shared_buffers + effective_cache_size should = total memory.
random_page_cost should be lowered a bunch since we are on ssds there.
1.1 is only slightly more than 1.0 for sequential.
effective_io_concurrency should also be raised a bunch for ssds.
a few other values should be higher based on memory.
Signed-off-by: Kevin Fenzi <kevin(a)scrye.com>
---
inventory/host_vars/db-koji01.phx2.fedoraproject.org | 4 ++--
roles/postgresql_server/templates/postgresql.conf | 12 +++++++-----
2 files changed, 9 insertions(+), 7 deletions(-)
diff --git a/inventory/host_vars/db-koji01.phx2.fedoraproject.org b/inventory/host_vars/db-koji01.phx2.fedoraproject.org
index 49a19d8..587e561 100644
--- a/inventory/host_vars/db-koji01.phx2.fedoraproject.org
+++ b/inventory/host_vars/db-koji01.phx2.fedoraproject.org
@@ -45,5 +45,5 @@ nrpe_procs_warn: 600
nrpe_procs_crit: 700
db_backup_dir: ['/backups']
-shared_buffers: "32GB"
-effective_cache_size: "32GB"
+shared_buffers: "30GB"
+effective_cache_size: "90GB"
diff --git a/roles/postgresql_server/templates/postgresql.conf b/roles/postgresql_server/templates/postgresql.conf
index b148442..cbaaaae 100644
--- a/roles/postgresql_server/templates/postgresql.conf
+++ b/roles/postgresql_server/templates/postgresql.conf
@@ -121,8 +121,8 @@ shared_buffers = {{ shared_buffers }} # min 128kB or max_connections*16kB
# 8 MB is probably on the high side. We can probably do with 4MB. But we
# were seeing a problem and we have the RAM so we're going to try this.
-work_mem = 4MB # min 64kB
-maintenance_work_mem = 1024MB # min 1MB
+work_mem = 157286kB
+maintenance_work_mem = 2GB # min 1MB
#max_stack_depth = 2MB # min 100kB
# - Free Space Map -
@@ -169,7 +169,7 @@ wal_sync_method = fdatasync # the default is the first option
# fsync_writethrough
# open_sync
#full_page_writes = on # recover from partial page writes
-wal_buffers = 64kB # min 32kB
+wal_buffers = 16MB # min 32kB
# (change requires restart)
#wal_writer_delay = 200ms # 1-10000 milliseconds
@@ -212,7 +212,7 @@ checkpoint_warning = 180s # 0 is off
# - Planner Cost Constants -
#seq_page_cost = 1.0 # measured on an arbitrary scale
-random_page_cost = 3.0 # same scale as above
+random_page_cost = 1.1 # same scale as above
#cpu_tuple_cost = 0.01 # same scale as above
#cpu_index_tuple_cost = 0.005 # same scale as above
#cpu_operator_cost = 0.0025 # same scale as above
@@ -502,4 +502,6 @@ default_text_search_config = 'pg_catalog.english'
#------------------------------------------------------------------------------
#custom_variable_classes = '' # list of custom variable class names
-
+#
+# Number of concurrent i/o operations at the same time. The default is 1.
+effective_io_concurrency = 100
--
1.8.3.1
4 years, 5 months
FBR : update robosignatory
by Aurelien Bompard
Hey folks,
Last Monday, before the freeze, we updated Robosignatory in prod with a few
new features, some of which could not be tested in staging as thoroughly as
we wanted to. As a result, the version currently in prod has issues with
the CoreOS artifacts. We've worked on that and our tests in staging are now
entirely successful, that why I'm asking for a freeze break to update
robosignatory to the latest code (0.6.5) and make some adjustments to the
configuration file in ansible (patch attached).
What say you?
Thanks,
Aurélien
4 years, 5 months
Emergency FBR: Lower concurrency on postgres to help load
by Stephen John Smoogen
Currently the load on db-koji is 140+ and it is slow and
non-responsive to users. I believe it is the below change which has
affected this. I would like to lower down io to 32 to see if that
affects things and tune further if needed. I am not doing this until
Kevin or Patrick are around as it will affect koji and there is an
order needed
diff --git a/roles/postgresql_server/templates/postgresql.conf
b/roles/postgresql_server/templates/postgresql.conf
index cbaaaae..341340c 100644
--- a/roles/postgresql_server/templates/postgresql.conf
+++ b/roles/postgresql_server/templates/postgresql.conf
@@ -504,4 +504,4 @@ default_text_search_config = 'pg_catalog.english'
#custom_variable_classes = '' # list of custom variable class names
#
# Number of concurrent i/o operations at the same time. The default is 1.
-effective_io_concurrency = 100
+effective_io_concurrency = 32
--
Stephen J Smoogen.
4 years, 5 months
FBR: subscribe fedora-messaging consumers to zmq.topic
by Aurelien Bompard
Hey folks,
The fedora-messaging consumers are currently subscribed to the amq.topic
exchange where they get all messages sent over AMQP. However, the bridges
that forward messages from fedmsg publish to the zmq.topic exchange,
therefore consumers need to subscribe to that one too to benefit from the
bridges.
It's just a matter of changing the rabbit/queue role, which should always
have had that double binding in place. It is not necessary to update every
consumer's config file. If the app did not have an issue with this missing
binding until now, it means it's not consuming from unmigrated fedmsg
messages, and it does not need an ansible run to fix it. Otherwise running
the playbook will add the missing bindings.
What do you think?
Aurélien
4 years, 5 months
Fedora 31 Final Release Readiness Meeting
by Ben Cotton
Dear all,
Join us on irc.freenode.net in #fedora-meeting-1 for the Fedora 31
Final Release Readiness meeting. This meeting will be held on
Thursday, 2019-10-17 at 19:00 UTC.
We will meet to make sure we are coordinated and ready for the release
of Fedora 31 Final. Please note that this meeting will be held even if
the release is delayed at the Go/No-Go meeting on the same day two
hours earlier.
You may receive this message several times in order to open this
meeting to the teams and to raise awareness, so hopefully more team
representatives will come to this meeting. This meeting works best
when we have representatives from all of the teams.
For more information, see
https://fedoraproject.org/wiki/Release_Readiness_Meetings.
View the meeting on Fedocal:
https://apps.fedoraproject.org/calendar/Fedora%20release/2019/10/17/#m9630
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
4 years, 5 months
Testing request: fpaste 0.4.0.0 (uses paste.centos.org)
by Ankur Sinha
Hello,
I've updated fpaste to use paste.centos.org and cut a new release. I've
tested it out and it works fine here, but since a few options changed,
could more people please test it for a bit before I push it out to
users?
The bodhi updates are here:
https://bodhi.fedoraproject.org/updates/?packages=fpaste
Changes include:
- update options to match what Stikked allows:
* expiry
* languages
* author
* title
- the URL option has been removed---since the API key is hardcoded, it
can't be used for other Stikked servers anyway.
(Making it configurable is on my list, but it is quite low priority.)
Users should not notice any changes in normal usage (sysinfo etc). If
you notice any weirdness, please let me know.
--
Thanks,
Regards,
Ankur Sinha "FranciscoD" (He / Him / His) | https://fedoraproject.org/wiki/User:Ankursinha
Time zone: Europe/London
4 years, 5 months
[PATCH] bodhi / backend: No longer sync or make aarch64 for epel7.
by kevin@scrye.com
From: Kevin Fenzi <kevin(a)scrye.com>
Since rhel dropped alt arch aarch64 with 7.7, we have to drop it in epel too.
Signed-off-by: Kevin Fenzi <kevin(a)scrye.com>
---
roles/bodhi2/backend/files/new-updates-sync | 4 ++--
roles/bodhi2/backend/templates/variants.rpm.xml.j2 | 4 +++-
2 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/roles/bodhi2/backend/files/new-updates-sync b/roles/bodhi2/backend/files/new-updates-sync
index fde72e3..d08c893 100755
--- a/roles/bodhi2/backend/files/new-updates-sync
+++ b/roles/bodhi2/backend/files/new-updates-sync
@@ -171,12 +171,12 @@ RELEASES = {'f31': {'topic': 'fedora',
'modules': ['epel'],
'repos': {'epel-testing': {
'from': 'epel7-testing',
- 'to': [{'arches': ['x86_64', 'aarch64', 'ppc64le', 'source'],
+ 'to': [{'arches': ['x86_64', 'ppc64le', 'source'],
'dest': os.path.join(EPELDEST, 'testing', '7')}
]},
'epel': {
'from': 'epel7',
- 'to': [{'arches': ['x86_64', 'aarch64', 'ppc64le', 'source'],
+ 'to': [{'arches': ['x86_64', 'ppc64le', 'source'],
'dest': os.path.join(EPELDEST, '7')}
]}}
},
diff --git a/roles/bodhi2/backend/templates/variants.rpm.xml.j2 b/roles/bodhi2/backend/templates/variants.rpm.xml.j2
index 062a63e..e848632 100644
--- a/roles/bodhi2/backend/templates/variants.rpm.xml.j2
+++ b/roles/bodhi2/backend/templates/variants.rpm.xml.j2
@@ -10,8 +10,10 @@
[% if release.id_prefix == "FEDORA" %]
<arch>armhfp</arch>
[% endif %]
- [% if release.version_int >= 26 or release.version_int >= 7 %]
+ [% if release.version_int >= 26 or release.version_int >= 8 %]
<arch>aarch64</arch>
+ [% endif %]
+ [% if release.version_int >= 26 or release.version_int >= 7 %]
<arch>ppc64le</arch>
[% endif %]
<arch>s390x</arch>
--
1.8.3.1
4 years, 5 months
Meeting Agenda Item: Introduction Dudley Burrows
by pyratebeard
Greetings,
I have been a Fedora user since 2009 and a full time Linux systems administrator since 2013. I am currently residing in Dublin, Ireland.
I have been a member of the Dublin LUG for about 6 months.
My knowledge covers managing large RHEL and CentOS estates and all the usual sysadmin work that brings. More recently I have been working with Openshift, Openstack, containers and orchestration with Kubernetes. I have worked with some cloud providers, mainly Azure. I am comfortable with shell scripting and have some experience with Python.
It has long been a plan of mine to join the Fedora community, but only recently feel that I have the time to give to the project. I look forward to improving my infrastructure knowledge as well as meeting like minded people.
My freenode IRC handle is pyratebeard.
Regards
Dudley
4 years, 5 months