11:18 a.m.
https://fedoraproject.org/wiki/Changes/Plocate_as_the_default_locate_impl...
== Summary ==
The venerable `mlocate` program is replaced by `plocate` — a
compatible reimplementation that is faster and uses less disk space.
== Owner ==
* Name: [[User:Zbyszek| Zbigniew Jędrzejewski-Szmek]]
* Email: zbyszek at in.waw.pl
* Name: [[User:Msekleta| Michal Sekletár]]
* Email: msekleta at redhat.com
== Detailed Description ==
Plocate is a newer implementation of `locate`/`mlocate` that using
`liburing` and `libzstd` for speed.
The database it creates on disk is also smaller.
Debian recently switched to `plocate` as the default implementation
(https://lwn.net/Articles/846405/).
It doesn't seem useful to maintain multiple locate implementations.
Thus the new package Conflicts with the old one, so they cannot be
installed in parallel.
The plan is:
# F35: `plocate` is made available for testing
# F36: `mlocate` is replaced by `plocate` in comps
# F37 or F38: `mlocate` will be retired (or given away, if somebody
wants to pick it up)
== Benefit to Fedora ==
We save some cpu cycles and disk sectors by using a more modern
implementation of a common tool.
== Scope ==
* Proposal owners:
** package `mlocate` (Review bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1931141, DONE)
** submit a pull request to comps with `s/mlocate/plocate/`
* Other developers: install plocate locally and test if it works as
expected on F35 and other versions
* Release engineering: n/a
* Policies and guidelines: n/a
* Trademark approval: n/a
* Alignment with Objectives: none
== Upgrade/compatibility impact ==
The upgrade should be mostly invisible. It is possible that somebody
might be relying on some very specific `mlocate` behaviour or parsing
the `mlocate` database directly, but no such cases are currently
known.
== How To Test ==
# Install `plocate` (`sudo dnf install plocate --allowerasing`)
# Wait for `plocate-updatedb.service` to finish (`sudo systemctl start
plocate-updatedb.service`)
# Use `plocate pattern` or `plocate -r <regexp>` to search for files.
== User Experience ==
Users should not notice the difference. Installing `plocate`
automatically removes `mlocate`. The new implementation is generally
compatible with the old one in all common cases, and provides some
additional options.
== Dependencies ==
None.
== Contingency Plan ==
* Contingency mechanism: (What to do? Who will do it?) N/A (not a
System Wide Change)
* Contingency deadline: N/A (not a System Wide Change)
* Blocks release? N/A (not a System Wide Change)
== Documentation ==
N/A (not a System Wide Change)
== Release Notes ==
`plocate` is now used as the default provider of `/usr/bin/locate`
instead of `mlocate`.
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
12:52 p.m.
On Tue, Nov 23, 2021 at 12:19 PM Ben Cotton <bcotton(a)redhat.com> wrote:
== How To Test ==
# Install `plocate` (`sudo dnf install plocate --allowerasing`)
# Wait for `plocate-updatedb.service` to finish (`sudo systemctl start
plocate-updatedb.service`)
# Use `plocate pattern` or `plocate -r <regexp>` to search for files.
Works for me. Also fixes this old bug where mlocate pruned bind mount
locations, which resulted in /home on Btrfs not being indexed.
https://bugzilla.redhat.com/show_bug.cgi?id=906591
Note for laptop users, plocate-updatedb.service contains
ConditionACPower=true, so if you're getting errors locating files, it
might be because the initial database isn't being created because no
AC power.
== User Experience ==
Users should not notice the difference. Installing `plocate`
automatically removes `mlocate`. The new implementation is generally
compatible with the old one in all common cases, and provides some
additional options.
I'm definitely noticing a difference. I'm not sure it was ever correct
that updatedb.conf was pruning bind mounts on Fedora. It's not the
mlocate default, but it is what Fedora has been using for years
(defacto default on Fedora).
--
Chris Murphy
3:53 p.m.
On Tue, Nov 23, 2021 at 01:52:43PM -0500, Chris Murphy wrote:
On Tue, Nov 23, 2021 at 12:19 PM Ben Cotton
<bcotton(a)redhat.com> wrote:
> == How To Test ==
> # Install `plocate` (`sudo dnf install plocate --allowerasing`)
> # Wait for `plocate-updatedb.service` to finish (`sudo systemctl start
> plocate-updatedb.service`)
> # Use `plocate pattern` or `plocate -r <regexp>` to search for files.
Works for me. Also fixes this old bug where mlocate pruned bind mount
locations, which resulted in /home on Btrfs not being indexed.
https://bugzilla.redhat.com/show_bug.cgi?id=906591
Note for laptop users, plocate-updatedb.service contains
ConditionACPower=true, so if you're getting errors locating files, it
might be because the initial database isn't being created because no
AC power.
Ack. (I assume that most people have the laptop on AC at least some
of the time, so the db will get updated sooner or later. In the worst
case, if the laptop is only charged while off or suspended, it would be
necessary to remove the conditional. But I think that's a rare case, and
the default is good.)
> == User Experience ==
> Users should not notice the difference. Installing `plocate`
> automatically removes `mlocate`. The new implementation is generally
> compatible with the old one in all common cases, and provides some
> additional options.
I'm definitely noticing a difference. I'm not sure it was ever correct
that updatedb.conf was pruning bind mounts on Fedora. It's not the
mlocate default, but it is what Fedora has been using for years
(defacto default on Fedora).
That's a known-fixed bug. I didn't mention it because, well, it was
considered a bug in mlocate. ¯\_(ツ)_/¯
Zbyszek
4:52 p.m.
New subject: F36 Change: Plocate as the default locate implementation (Self-Contained Change proposal)
On Tuesday, November 23, 2021 4:53:12 PM EST Zbigniew Jędrzejewski-Szmek wrote:
On Tue, Nov 23, 2021 at 01:52:43PM -0500, Chris Murphy wrote:
> I'm definitely noticing a difference. I'm not sure it was ever correct
> that updatedb.conf was pruning bind mounts on Fedora. It's not the
> mlocate default, but it is what Fedora has been using for years
> (defacto default on Fedora).
That's a known-fixed bug.
Not fixed as far as I know[*]. I've been patching my
/etc/updatedb.conf ever since I installed with btrfs. (The second
time I installed with btrfs, f33, that is. Not years ago. :-))
This change does render the bug moot, though. Nice.
_____________________
[*] The bug claims a fix for Silverblue, but not Fedora Workstation or
KDE.
--
Garry T. Williams
2:24 a.m.
On Tue, Nov 23, 2021 at 05:52:26PM -0500, Garry T. Williams wrote:
On Tuesday, November 23, 2021 4:53:12 PM EST Zbigniew
Jędrzejewski-Szmek wrote:
> On Tue, Nov 23, 2021 at 01:52:43PM -0500, Chris Murphy wrote:
> > I'm definitely noticing a difference. I'm not sure it was ever correct
> > that updatedb.conf was pruning bind mounts on Fedora. It's not the
> > mlocate default, but it is what Fedora has been using for years
> > (defacto default on Fedora).
>
> That's a known-fixed bug.
Not fixed as far as I know[*].
known-fixed-in-plocate ;)
This was explicitly discussed somewhere: the gist is that plocate has
better handling of duplicates, so it doesn't need to filter out bind mounts.
But I can't find that discussion. If I do, I'll add it to the change page.
I've been patching my
/etc/updatedb.conf ever since I installed with btrfs. (The second
time I installed with btrfs, f33, that is. Not years ago. :-))
This change does render the bug moot, though. Nice.
Zbyszek
1:15 p.m.
On Wed, Nov 24, 2021 at 3:25 AM Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl> wrote:
On Tue, Nov 23, 2021 at 05:52:26PM -0500, Garry T. Williams wrote:
> On Tuesday, November 23, 2021 4:53:12 PM EST Zbigniew Jędrzejewski-Szmek wrote:
> > On Tue, Nov 23, 2021 at 01:52:43PM -0500, Chris Murphy wrote:
> > > I'm definitely noticing a difference. I'm not sure it was ever
correct
> > > that updatedb.conf was pruning bind mounts on Fedora. It's not the
> > > mlocate default, but it is what Fedora has been using for years
> > > (defacto default on Fedora).
> >
> > That's a known-fixed bug.
>
> Not fixed as far as I know[*].
known-fixed-in-plocate ;)
OK super. (I should have read the whole thread...)
--
Chris Murphy
1:13 p.m.
On Tue, Nov 23, 2021 at 4:54 PM Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl> wrote:
On Tue, Nov 23, 2021 at 01:52:43PM -0500, Chris Murphy wrote:
> On Tue, Nov 23, 2021 at 12:19 PM Ben Cotton <bcotton(a)redhat.com> wrote:
>
> > == How To Test ==
> > # Install `plocate` (`sudo dnf install plocate --allowerasing`)
> > # Wait for `plocate-updatedb.service` to finish (`sudo systemctl start
> > plocate-updatedb.service`)
> > # Use `plocate pattern` or `plocate -r <regexp>` to search for files.
>
> Works for me. Also fixes this old bug where mlocate pruned bind mount
> locations, which resulted in /home on Btrfs not being indexed.
> https://bugzilla.redhat.com/show_bug.cgi?id=906591
>
> Note for laptop users, plocate-updatedb.service contains
> ConditionACPower=true, so if you're getting errors locating files, it
> might be because the initial database isn't being created because no
> AC power.
Ack. (I assume that most people have the laptop on AC at least some
of the time, so the db will get updated sooner or later. In the worst
case, if the laptop is only charged while off or suspended, it would be
necessary to remove the conditional. But I think that's a rare case, and
the default is good.)
> > == User Experience ==
> > Users should not notice the difference. Installing `plocate`
> > automatically removes `mlocate`. The new implementation is generally
> > compatible with the old one in all common cases, and provides some
> > additional options.
>
> I'm definitely noticing a difference. I'm not sure it was ever correct
> that updatedb.conf was pruning bind mounts on Fedora. It's not the
> mlocate default, but it is what Fedora has been using for years
> (defacto default on Fedora).
That's a known-fixed bug. I didn't mention it because, well, it was
considered a bug in mlocate. ¯\_(ツ)_/¯
It's not a fixed bug. It's an intentional departure from the default,
and it's still a problem in fresh Fedora installations today.
i.e. the mlocate default in updatedb.conf
#PRUNE_BIND_MOUNTS = "yes"
But in Fedora it ships uncommented, so /home is not indexed on default
installs because Btrfs "home" subvolume mounted at /home is a bind
mount behind the scenes. Recently Silverblue fixed this, as it makes
considerable use of bind mounts, by (re)commenting the pruning of bind
mounts.
--
Chris Murphy
2:55 p.m.
On Wed, Dec 01, 2021 at 02:13:07PM -0500, Chris Murphy wrote:
On Tue, Nov 23, 2021 at 4:54 PM Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl> wrote:
>
> On Tue, Nov 23, 2021 at 01:52:43PM -0500, Chris Murphy wrote:
> > On Tue, Nov 23, 2021 at 12:19 PM Ben Cotton <bcotton(a)redhat.com> wrote:
> >
> > > == How To Test ==
> > > # Install `plocate` (`sudo dnf install plocate --allowerasing`)
> > > # Wait for `plocate-updatedb.service` to finish (`sudo systemctl start
> > > plocate-updatedb.service`)
> > > # Use `plocate pattern` or `plocate -r <regexp>` to search for
files.
> >
> > Works for me. Also fixes this old bug where mlocate pruned bind mount
> > locations, which resulted in /home on Btrfs not being indexed.
> > https://bugzilla.redhat.com/show_bug.cgi?id=906591
> >
> > Note for laptop users, plocate-updatedb.service contains
> > ConditionACPower=true, so if you're getting errors locating files, it
> > might be because the initial database isn't being created because no
> > AC power.
>
> Ack. (I assume that most people have the laptop on AC at least some
> of the time, so the db will get updated sooner or later. In the worst
> case, if the laptop is only charged while off or suspended, it would be
> necessary to remove the conditional. But I think that's a rare case, and
> the default is good.)
>
> > > == User Experience ==
> > > Users should not notice the difference. Installing `plocate`
> > > automatically removes `mlocate`. The new implementation is generally
> > > compatible with the old one in all common cases, and provides some
> > > additional options.
> >
> > I'm definitely noticing a difference. I'm not sure it was ever correct
> > that updatedb.conf was pruning bind mounts on Fedora. It's not the
> > mlocate default, but it is what Fedora has been using for years
> > (defacto default on Fedora).
>
> That's a known-fixed bug. I didn't mention it because, well, it was
> considered a bug in mlocate. ¯\_(ツ)_/¯
It's not a fixed bug.
Sorry, I meant that it's fixed in plocate, which simply does exhibit
this behaviour.
Zbyszek
2:56 p.m.
On Wed, Dec 01, 2021 at 08:55:44PM +0000, Zbigniew Jędrzejewski-Szmek wrote:
On Wed, Dec 01, 2021 at 02:13:07PM -0500, Chris Murphy wrote:
> On Tue, Nov 23, 2021 at 4:54 PM Zbigniew Jędrzejewski-Szmek
> <zbyszek(a)in.waw.pl> wrote:
> >
> > On Tue, Nov 23, 2021 at 01:52:43PM -0500, Chris Murphy wrote:
> > > On Tue, Nov 23, 2021 at 12:19 PM Ben Cotton <bcotton(a)redhat.com>
wrote:
> > >
> > > > == How To Test ==
> > > > # Install `plocate` (`sudo dnf install plocate --allowerasing`)
> > > > # Wait for `plocate-updatedb.service` to finish (`sudo systemctl
start
> > > > plocate-updatedb.service`)
> > > > # Use `plocate pattern` or `plocate -r <regexp>` to search for
files.
> > >
> > > Works for me. Also fixes this old bug where mlocate pruned bind mount
> > > locations, which resulted in /home on Btrfs not being indexed.
> > > https://bugzilla.redhat.com/show_bug.cgi?id=906591
> > >
> > > Note for laptop users, plocate-updatedb.service contains
> > > ConditionACPower=true, so if you're getting errors locating files, it
> > > might be because the initial database isn't being created because no
> > > AC power.
> >
> > Ack. (I assume that most people have the laptop on AC at least some
> > of the time, so the db will get updated sooner or later. In the worst
> > case, if the laptop is only charged while off or suspended, it would be
> > necessary to remove the conditional. But I think that's a rare case, and
> > the default is good.)
> >
> > > > == User Experience ==
> > > > Users should not notice the difference. Installing `plocate`
> > > > automatically removes `mlocate`. The new implementation is generally
> > > > compatible with the old one in all common cases, and provides some
> > > > additional options.
> > >
> > > I'm definitely noticing a difference. I'm not sure it was ever
correct
> > > that updatedb.conf was pruning bind mounts on Fedora. It's not the
> > > mlocate default, but it is what Fedora has been using for years
> > > (defacto default on Fedora).
> >
> > That's a known-fixed bug. I didn't mention it because, well, it was
> > considered a bug in mlocate. ¯\_(ツ)_/¯
>
> It's not a fixed bug.
Sorry, I meant that it's fixed in plocate, which simply does exhibit
this behaviour.
Ah, OK, this has been cleared up in another part of the thread ;)
Zbyszek
1:13 p.m.
Dne 23. 11. 21 v 18:18 Ben Cotton napsal(a):
== User Experience ==
Users should not notice the difference. Installing `plocate`
automatically removes `mlocate`. The new implementation is generally
compatible with the old one in all common cases, and provides some
additional options.
Any reason why `plocate` does not own `/etc/updatedb.conf` and does not provide the config
with a sane defaults?
Additionally, providing a man page for `locate` as alias to `plocate` will be nice.
Miroslav
3:54 p.m.
On Tue, Nov 23, 2021 at 08:13:02PM +0100, Miroslav Suchý wrote:
Dne 23. 11. 21 v 18:18 Ben Cotton napsal(a):
>== User Experience ==
>Users should not notice the difference. Installing `plocate`
>automatically removes `mlocate`. The new implementation is generally
>compatible with the old one in all common cases, and provides some
>additional options.
Any reason why `plocate` does not own `/etc/updatedb.conf` and does
not provide the config with a sane defaults?
The file isn't needed, the defaults are OK. I'll add '%ghost
/etc/updatedb.conf'.
Additionally, providing a man page for `locate` as alias to `plocate`
will be nice.
I'll put that on the list too.
Zbyszek
4:28 p.m.
Ben Cotton wrote on Tue, Nov 23, 2021 at 12:18:56PM -0500:
== Summary ==
The venerable `mlocate` program is replaced by `plocate` — a
compatible reimplementation that is faster and uses less disk space.
Thanks for doing this!
For what it's worth, I've been running plocate locally since it was
brought up to the list earlier this year(?), and while it's much faster
I have one small complain: the indexing phase is way too power-hungry.
This is not actually a complain about power draw (there's the onACpower
conditional as rightfully pointed out), but about kicking in the fans
a few times a week (because of the randomized delay, it happens when I'm
at desk regularily) when I'm at desk.
I see now that it has IOSchedulingClass=IDLE but nothing equivalent
about CPU, but I don't think that'll help: if the computer is mostly
idle it'll still use all the cores (and increase cpufreq automatically)
for long enough to heat up and make the fans kick in.
I don't think there is a laptop friendly scheduler that says if this
task is niced then keep the cpufreq at its minimum, is there?
I've been manually fiddling with my cpufreqs' scaling_max_freq just low
enough to not trigger the fans but it sounds like overengineering to
me...
(feel free to ignore me, I'm overall very happy about this change)
--
Dominique
5:34 p.m.
(replying to myself as I got a message on irc from the author)
Dominique Martinet wrote on Wed, Nov 24, 2021 at 07:28:25AM +0900:
I see now that it has IOSchedulingClass=IDLE but nothing equivalent
about CPU, but I don't think that'll help: if the computer is mostly
idle it'll still use all the cores (and increase cpufreq automatically)
for long enough to heat up and make the fans kick in.
I seem to have misremembered this part: it is single core.
(doesn't change much as far as cpufreq/heat is concerned though)
Also, it had been a while but it has actually gotten much better after I
added /.snapshots and similar paths to PRUNEPATHS, so this isn't as much
of a problem as I remembered, but could still be a problem for large
filesystems
I don't think there is a laptop friendly scheduler that says if
this
task is niced then keep the cpufreq at its minimum, is there?
I've been manually fiddling with my cpufreqs' scaling_max_freq just low
enough to not trigger the fans but it sounds like overengineering to
me...
Still interested if anyone knows how to do this!
--
Dominique
1:58 a.m.
On Wed, Nov 24, 2021 at 07:28:25AM +0900, Dominique Martinet wrote:
Ben Cotton wrote on Tue, Nov 23, 2021 at 12:18:56PM -0500:
> == Summary ==
> The venerable `mlocate` program is replaced by `plocate` — a
> compatible reimplementation that is faster and uses less disk space.
Thanks for doing this!
The positive responses make me very happy ;)
For what it's worth, I've been running plocate locally since
it was
brought up to the list earlier this year(?), and while it's much faster
I have one small complain: the indexing phase is way too power-hungry.
This is not actually a complain about power draw (there's the onACpower
conditional as rightfully pointed out), but about kicking in the fans
a few times a week (because of the randomized delay, it happens when I'm
at desk regularily) when I'm at desk.
My understanding is that in order to minimize the overall energy consumption,
the scheduler will try to maximize the cpu frequency and minimize the total
time of computation… So a cpu scheduler will not help much here, as they
will all schedule 100% of the cpu. AFAIK, there is no scheduler that
allows saying "schedule this so that so that the fans don't go on".
From the amount of complaints about loud fans, I think it would be a
popular feature. (There is something similar with "lap protection", where
heating is avoided if the laptop is detected as not-on-a-surface).
Please try with the following file:
# /etc/systemd/system/plocate-updatedb.service.d/limits.conf
[Service]
CPUAccounting=yes
CPUQuota=20%
(then 'sudo systemctl daemon-reload && sudo systemctl restart
plocate-updatedb').
I think this could work.
Zbyszek
3:10 a.m.
Zbigniew Jędrzejewski-Szmek wrote on Wed, Nov 24, 2021 at 07:58:47AM +0000:
My understanding is that in order to minimize the overall energy
consumption,
the scheduler will try to maximize the cpu frequency and minimize the total
time of computation… So a cpu scheduler will not help much here, as they
will all schedule 100% of the cpu. AFAIK, there is no scheduler that
allows saying "schedule this so that so that the fans don't go on".
From the amount of complaints about loud fans, I think it would be a
popular feature. (There is something similar with "lap protection", where
heating is avoided if the laptop is detected as not-on-a-surface).
Right, from a total power consumption point of view this probably makes
sense, but it's not necessarily what people are after.
Please try with the following file:
# /etc/systemd/system/plocate-updatedb.service.d/limits.conf
[Service]
CPUAccounting=yes
CPUQuota=20%
(then 'sudo systemctl daemon-reload && sudo systemctl restart
plocate-updatedb').
I think this could work.
That's a nice one.
For my particular laptop, with governor=performance I'm cutting it too
close to fan being audible (and firefox etc also regularily kicks it
in), so I'm not surprised even with CPUQuota=20% it's enough to be
audible, albeit at a lower level than full throttle (something like +5°
instead of +10-15 from where core temp usually stands, that makes a
difference to fan RPM)
With governor=powersave though it's really worlds appart, no CPUQuota
immediately scales up to close to the max and heats just like
performance would but with this I barely see any difference from not
doing anything, cpufreq stays < 1GHz and heat stays way down.
The consequence of that is it takes much longer to complete because the
clock is down: what previously normally took ~55s real for ~27s of CPU
time now takes 7m10 for 85s of CPU time -- but honestly I don't care how
long this takes if it's not noticeable, this is perfect. Thanks again!
Steinar would you consider adding this to the upstream service file?
(Zbigniew, does systemd just ignore the setting on systems where cpu
accounting has not been enabled? iirc some distros still have it off and
upstream might want to provide defaults that work everywhere)
I would have suggested also adding Nice=5 or something but I don't think
it's required with this.
Cheers,
--
Dominique
3:23 a.m.
On Wed, Nov 24, 2021 at 06:10:18PM +0900, Dominique Martinet wrote:
> Please try with the following file:
>
> # /etc/systemd/system/plocate-updatedb.service.d/limits.conf
> [Service]
> CPUAccounting=yes
> CPUQuota=20%
>
> (then 'sudo systemctl daemon-reload && sudo systemctl restart
plocate-updatedb').
> I think this could work.
That's a nice one.
For my particular laptop, with governor=performance I'm cutting it too
close to fan being audible (and firefox etc also regularily kicks it
in), so I'm not surprised even with CPUQuota=20% it's enough to be
audible, albeit at a lower level than full throttle (something like +5°
instead of +10-15 from where core temp usually stands, that makes a
difference to fan RPM)
With governor=powersave though it's really worlds appart, no CPUQuota
immediately scales up to close to the max and heats just like
performance would but with this I barely see any difference from not
doing anything, cpufreq stays < 1GHz and heat stays way down.
The consequence of that is it takes much longer to complete because the
clock is down: what previously normally took ~55s real for ~27s of CPU
time now takes 7m10 for 85s of CPU time -- but honestly I don't care how
long this takes if it's not noticeable, this is perfect. Thanks again!
That's a really long time… 55/27 s seems fairly standard, e.g. I get 43/22 s
with a 256 GB SATA disk. But 440/85 s is quite a bit worse.
Steinar would you consider adding this to the upstream service file?
I don't think this qualifies as a good default. I would consider it a
hack to fix a local hardware problem. Fans in a laptop simply must not
be loud enough to be annoying.
(Zbigniew, does systemd just ignore the setting on systems where cpu
accounting has not been enabled? iirc some distros still have it off and
upstream might want to provide defaults that work everywhere)
I would have suggested also adding Nice=5 or something but I don't think
it's required with this.
I think that with IOSchedulingClass=idle, niceness
doesn't matter anymore.
Zbyszek
4:14 a.m.
Zbigniew Jędrzejewski-Szmek wrote on Wed, Nov 24, 2021 at 09:23:37AM +0000:
> The consequence of that is it takes much longer to complete
because the
> clock is down: what previously normally took ~55s real for ~27s of CPU
> time now takes 7m10 for 85s of CPU time -- but honestly I don't care how
> long this takes if it's not noticeable, this is perfect. Thanks again!
That's a really long time… 55/27 s seems fairly standard, e.g. I get 43/22 s
with a 256 GB SATA disk. But 440/85 s is quite a bit worse.
I really don't think it matters: this is a background job.
If I want to refresh the locate database for a one-time thing,
e.g. because I just updated a lot of files and look for something new,
I'd run `sudo updatedb` not `sudo systemctl start plocate-updatedb`...
> Steinar would you consider adding this to the upstream service
file?
I don't think this qualifies as a good default. I would consider it a
hack to fix a local hardware problem. Fans in a laptop simply must not
be loud enough to be annoying.
Well, there's annoying and annoying - I'm sure it could be better but I
can accept hearing laptop fans if it means I can build a kernel slightly
faster (because getting more heat out means thermal throttle doesn't
need to hold back as much); but I don't want to hear the fan at random
times when I'm not doing anything, even if it's not a horrible sound.
Even for a fanless system, the fan itself won't be annoying, but
say you want to compile something just as plocate has heated up the CPU
and you'll get much lower performance for what you actually wanted to do
at this time -- so it makes sense for real background stuff to stay
close to idle.
(I'm not sure how stuff like intel/amd's "turbo" mode works, but I
believe it won't work if your cpu is hot, and I'd take this even for my
servers or other non-noisy systems)
> (Zbigniew, does systemd just ignore the setting on systems where
cpu
> accounting has not been enabled? iirc some distros still have it off and
> upstream might want to provide defaults that work everywhere)
>
> I would have suggested also adding Nice=5 or something but I don't think
> it's required with this.
I think that with IOSchedulingClass=idle, niceness doesn't matter anymore.
Hmm, I'm not sure what makes you think that.
From what I can tell, if nice has been set, the ioprio will derive a
value from it automatically (ionice = (nice + 20) / 5 for best effort,
and also ioprio set to idle if set to SCHED_IDLE -- see
include/linux/ioprio.h in the kernel)
I however don't see anything that would hint that the other way around
is true, it really seems to be specific to the IO scheduler to me.
I don't see anything in block/ioprio.c that would also affect task
scheduling.
I'll grant you that if no IO can be done, the process will be stuck
waiting for IO and not consume a lot of CPU, but that's not really the
same as being niced.
--
Dominique
10:30 a.m.
On Wed, Nov 24, 2021 at 07:14:47PM +0900, Dominique Martinet wrote:
Zbigniew Jędrzejewski-Szmek wrote on Wed, Nov 24, 2021 at 09:23:37AM
+0000:
> > The consequence of that is it takes much longer to complete because the
> > clock is down: what previously normally took ~55s real for ~27s of CPU
> > time now takes 7m10 for 85s of CPU time -- but honestly I don't care how
> > long this takes if it's not noticeable, this is perfect. Thanks again!
>
> That's a really long time… 55/27 s seems fairly standard, e.g. I get 43/22 s
> with a 256 GB SATA disk. But 440/85 s is quite a bit worse.
I really don't think it matters: this is a background job.
It matters in the sense that the total energy consumption will be higher.
I don't think we should try to work around hardware issues at the
level of individual programs trying to influence decisions. This can
never work, except as a local hack for some issue.
> > I would have suggested also adding Nice=5 or something but
I don't think
> > it's required with this.
>
> I think that with IOSchedulingClass=idle, niceness doesn't matter anymore.
Hmm, I'm not sure what makes you think that.
Brainfart, I was thinking about CPUSchedulingPolicy=. I think it would
make sense to set CPUSchedulingPolicy=batch.
Zbyszek
2:05 p.m.
On Tue, Nov 23, 2021 at 12:18:56PM -0500, Ben Cotton wrote:
https://fedoraproject.org/wiki/Changes/Plocate_as_the_default_locate_impl...
== Summary ==
The venerable `mlocate` program is replaced by `plocate` — a
compatible reimplementation that is faster and uses less disk space.
== Owner ==
* Name: [[User:Zbyszek| Zbigniew Jędrzejewski-Szmek]]
* Email: zbyszek at in.waw.pl
* Name: [[User:Msekleta| Michal Sekletár]]
* Email: msekleta at redhat.com
Thanks for doing this! One long-standing issue (that's becoming more
relevant now that Workstation and desktop spins switched to Btrfs in
F33, and Cloud in F35) with our mlocate is that it's configured to skip
bind mounts by default: https://bugzilla.redhat.com/show_bug.cgi?id=906591
which affects both Btrfs-based installs and rpm-ostree ones, because
`/home` is on a Btrfs subvolume, and on a bind mount on Silverblue
even predating Btrfs being default, and both look like bind mounts.
This was overridden for Silverblue and other rpm-ostree based
spins:
https://github.com/fedora-silverblue/issue-tracker/issues/76#issuecomment...
but using a RPM scriptlet, and I'm glad with plocate we can just use the
default settings rather than having to hack in yet another exception for
Btrfs systems. Users likely expect `/home` to be indexed if they bother
using `locate`.
Best regards,
--
Michel Alexandre Salim
profile: https://keyoxide.org/michel@michel-slm.name
8:10 a.m.
On Tue, 23 Nov 2021 at 17:20, Ben Cotton wrote:
== Upgrade/compatibility impact ==
The upgrade should be mostly invisible. It is possible that somebody
might be relying on some very specific `mlocate` behaviour or parsing
the `mlocate` database directly, but no such cases are currently
known.
== How To Test ==
# Install `plocate` (`sudo dnf install plocate --allowerasing`)
# Wait for `plocate-updatedb.service` to finish (`sudo systemctl start
plocate-updatedb.service`)
# Use `plocate pattern` or `plocate -r <regexp>` to search for files.
== User Experience ==
Users should not notice the difference. Installing `plocate`
automatically removes `mlocate`. The new implementation is generally
compatible with the old one in all common cases, and provides some
additional options.
I have just installed plocate to try it on F34, and got:
warning: /etc/updatedb.conf saved as /etc/updatedb.conf.rpmsave
I reviewed the changes, and the new file is empty. Users will notice
the difference. Is that expected? Is it because I'm still on F34?
Also, the plocate-updatedb service was not started after installing
it, so I can't use locate until the DB is created. Is that expected?
Does it wait until the timer runs it?
9:36 a.m.
On Wed, Dec 01, 2021 at 02:10:33PM +0000, Jonathan Wakely wrote:
I have just installed plocate to try it on F34, and got:
warning: /etc/updatedb.conf saved as /etc/updatedb.conf.rpmsave
I reviewed the changes, and the new file is empty. Users will notice
the difference. Is that expected? Is it because I'm still on F34?
I think rpm is saving the file because if was declared as
%config(noreplace) by mlocate. So when you install plocate, mlocate gets
uninstalled, and rpm moves the file out of the way.
plocate just declares %ghost %{_sysconfdir}/updatedb.conf, because there
is no need for the file. If I understand correctly, rpm created an
empty file for you. I don't know why it would do that.
Anyway, if the the %ghost is not appropriate, or it should be some
more complex incantation, suggests are welcome.
Also, the plocate-updatedb service was not started after installing
it, so I can't use locate until the DB is created. Is that expected?
Does it wait until the timer runs it?
Do you have fedora-release-common-34-39 installed? The service should
run once immediately after the installation.
Zbyszek
2 p.m.
New subject: F36 Change: Plocate as the default locate implementation (Self-Contained Change proposal)
On Wednesday, December 1, 2021 9:10:33 AM EST Jonathan Wakely wrote:
Also, the plocate-updatedb service was not started after installing
it, so I can't use locate until the DB is created. Is that expected?
Does it wait until the timer runs it?
I believe that is normal. Earlier releases of mlocate were never
enabled to run -- I always had to enable (the timer) and start them.
--
Garry T. Williams
2:59 p.m.
On Wed, Dec 01, 2021 at 03:00:34PM -0500, Garry T. Williams wrote:
On Wednesday, December 1, 2021 9:10:33 AM EST Jonathan Wakely wrote:
> Also, the plocate-updatedb service was not started after installing
> it, so I can't use locate until the DB is created. Is that expected?
> Does it wait until the timer runs it?
I believe that is normal. Earlier releases of mlocate were never
enabled to run -- I always had to enable (the timer) and start them.
Define "normal". I set up the *p*locate scripts to start the service
immediately. Obviously this only works on a live system, with systemd
installed and active. My assumption is that if you're installing plocate
on a running system, you probably want to use it, and this way after
approx. 30—60 seconds you can.
Zbyszek
12:07 p.m.
`plocate` does not have an option which `mlocate` [1] has:
-A, --all
Print only names which match all non-option arguments, not
those matching one or more non-option arguments.
The same could be achieved using extended regular expressions i.e.
`--regex`, but it's cumbersome.
Say I want to find out all documents about Fedora IoT security. Compare:
mlocate -i -A fedora iot security
with:
plocate -i --regex
'fedora.*iot.*security|fedora.*security.*iot|iot.*fedora.*security|iot.*security.*fedora|security.*fedora.*iot|security.*iot.*fedora'
This is needed because the order of the words is not fixed; results
could include for example `Security/Fedora IoT.pdf` or
`Fedora/IoT/Security.pdf`.
Speaking of security, `systemd-analyze security plocate-updatedb.service`
suggests a couple of improvements which could be made, e.g.
LockPersonality or ProtectClock.
[1]: https://man7.org/linux/man-pages/man1/locate.1.html
7:22 a.m.
On 23. 11. 21 18:18, Ben Cotton wrote:
https://fedoraproject.org/wiki/Changes/Plocate_as_the_default_locate_impl...
== Summary ==
The venerable `mlocate` program is replaced by `plocate` — a
compatible reimplementation that is faster and uses less disk space.
== Owner ==
* Name: [[User:Zbyszek| Zbigniew Jędrzejewski-Szmek]]
* Email: zbyszek at in.waw.pl
* Name: [[User:Msekleta| Michal Sekletár]]
* Email: msekleta at redhat.com
...
Hey folks,
I've just noticed that `ausearch -m AVC,USER_AVC -ts recent` lists a lot of:
type=AVC msg=audit(1642770831.616:31668): avc: denied { read } for
pid=866275 comm="locate" name="plocate.db" dev="dm-0"
ino=5268062
scontext=system_u:system_r:setroubleshootd_t:s0
tcontext=system_u:object_r:var_lib_t:s0 tclass=file permissive=0
Is this a bug, or some kind of error in configuration?
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
10:03 a.m.
On Fri, Jan 21, 2022 at 2:24 PM Miro Hrončok <mhroncok(a)redhat.com> wrote:
On 23. 11. 21 18:18, Ben Cotton wrote:
>
https://fedoraproject.org/wiki/Changes/Plocate_as_the_default_locate_impl...
>
> == Summary ==
> The venerable `mlocate` program is replaced by `plocate` — a
> compatible reimplementation that is faster and uses less disk space.
>
>
> == Owner ==
> * Name: [[User:Zbyszek| Zbigniew Jędrzejewski-Szmek]]
> * Email: zbyszek at in.waw.pl
> * Name: [[User:Msekleta| Michal Sekletár]]
> * Email: msekleta at redhat.com
> ...
Hey folks,
I've just noticed that `ausearch -m AVC,USER_AVC -ts recent` lists a lot
of:
type=AVC msg=audit(1642770831.616:31668): avc: denied { read } for
pid=866275 comm="locate" name="plocate.db" dev="dm-0"
ino=5268062
scontext=system_u:system_r:setroubleshootd_t:s0
tcontext=system_u:object_r:var_lib_t:s0 tclass=file permissive=0
Is this a bug, or some kind of error in configuration?
plocate uses /var/lib/plocate as its data directory, so it needs to have
assigned the correct label in selinux-policy. I've already submitted a PR
for the change.
https://github.com/fedora-selinux/selinux-policy/pull/1018
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
--
Zdenek Pytela
Security SELinux team
10:14 a.m.
On 21. 01. 22 17:03, Zdenek Pytela wrote:
Hey folks,
I've just noticed that `ausearch -m AVC,USER_AVC -ts recent` lists a lot of:
type=AVC msg=audit(1642770831.616:31668): avc: denied { read } for
pid=866275 comm="locate" name="plocate.db" dev="dm-0"
ino=5268062
scontext=system_u:system_r:setroubleshootd_t:s0
tcontext=system_u:object_r:var_lib_t:s0 tclass=file permissive=0
Is this a bug, or some kind of error in configuration?
plocate uses /var/lib/plocate as its data directory, so it needs to have
assigned the correct label in selinux-policy. I've already submitted a PR for
the change.
https://github.com/fedora-selinux/selinux-policy/pull/1018
<https://github.com/fedora-selinux/selinux-policy/pull/1018>
Thanks!
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
825
days inactive
884
days old
26 comments
11 participants
participants (11)
-
Ben Cotton -
Chris Murphy -
Cristian Ciupitu -
Dominique Martinet -
Garry T. Williams -
Jonathan Wakely -
Michel Alexandre Salim -
Miro Hrončok -
Miroslav Suchý -
Zbigniew Jędrzejewski-Szmek -
Zdenek Pytela