Hi everyone!
RPM 4.19 added automatic sysuser handling [1]. In Fedora 39 this feature was not enabled right away [2] as it requires some care to properly transition to it. Also going back to 4.18 was technically still the fallback option during this change.
I just noticed in an issue in the RPM upstream repository [3] that the sysuser feature is still not enabled. May be right now might be a good time to get this going for Fedora 41. I am happy to help with the technical details but would prefer if this effort was driven from within Fedora.
Currently users are either done manually by calling useradd in scriptlets or using the macros in systemd-rpm-macros which is a sub package of the systemd package. RPM's mechanism is switched off by rpm-4.18.92-disable-sysusers.patch in the rpm package.
This whole thing probably needs to be a Global Change involving a change to the Packaging Guidelines [4] and may be an Mass Package Change (although that might be avoided by changing the macros in systemd-rpm-macros to NOPs).
Anyone interested in picking this up? I remember quite a few people being exited about this when it was announced with the rpm-4.19 Change.
Florian
[1] https://rpm-software-management.github.io/rpm/manual/users_and_groups.html [2] https://fedoraproject.org/wiki/Changes/RPM-4.19 [3] https://github.com/rpm-software-management/rpm/issues/3073 [4] https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups
On Fri, May 10, 2024 at 01:28:07PM +0200, Florian Festi wrote:
Anyone interested in picking this up? I remember quite a few people being exited about this when it was announced with the rpm-4.19 Change.
I would be interested in making this happen.
You mentioned that the transition "requires some care". What are the problems?
This whole thing probably needs to be a Global Change involving a change to the Packaging Guidelines [4] and may be an Mass Package Change (although that might be avoided by changing the macros in systemd-rpm-macros to NOPs).
The macros are written in a way that if the user/group exist, no operation is done. Thus, naively, I would think that if rpm starts to create users and groups on its own, then the existing scriptlets would become noops. That would mean that we could enable the feature in rpm without any mass package changes first.
If the rpm approach works, I think it'd make sense to a) change the macros to be noops, b) do a mass package change to strip the scriptlets from all packages. That's probably the right thing to do for the rawhide branch, but as usual, the question becomes how to handle packages that use a common branch for older releases. But the Mass Change process is intended to deal with such cases.
Zbyszek
On 5/11/24 12:56, Zbigniew Jędrzejewski-Szmek wrote:
On Fri, May 10, 2024 at 01:28:07PM +0200, Florian Festi wrote:
Anyone interested in picking this up? I remember quite a few people being exited about this when it was announced with the rpm-4.19 Change.
I would be interested in making this happen.
You mentioned that the transition "requires some care". What are the problems?
There are Requires created for the users and groups. To make this work the Provides need to be there first - obviously. So one will probably need to set %_use_weak_usergroup_deps for a transition period. At least until the first mass rebuild.
There are also a large number of packages that are using useradd: grep useradd *.spec | cut -d: -f1 | sort -u | wc 281 281 4090
We need to think what to do with them.
The sysusers macros are much less used actually: grep sysusers_requires_compat *.spec | cut -d: -f1 | sort -u | wc 53 53 725 grep sysusers_create_compat *.spec | cut -d: -f1 | sort -u | wc 101 101 1476
This whole thing probably needs to be a Global Change involving a change to the Packaging Guidelines [4] and may be an Mass Package Change (although that might be avoided by changing the macros in systemd-rpm-macros to NOPs).
The macros are written in a way that if the user/group exist, no operation is done. Thus, naively, I would think that if rpm starts to create users and groups on its own, then the existing scriptlets would become noops. That would mean that we could enable the feature in rpm without any mass package changes first.
That might work, but I have not looked deep enough into that to do that blindly.
If the rpm approach works, I think it'd make sense to a) change the macros to be noops, b) do a mass package change to strip the scriptlets from all packages. That's probably the right thing to do for the rawhide branch, but as usual, the question becomes how to handle packages that use a common branch for older releases. But the Mass Change process is intended to deal with such cases.
One way to deal with this is to keep the noop macros until all current Fedora versions are using the new method.
It's probably not super complicated it just wasn't something we wanted to do during the actual RPM update - which already was a lot of "fun" without also enabling the new user handling.
Then there is ofc all the paper work needed.
Florian
PS: Note that the upcoming RPM 4.20 release is extending the support to "m" lines in sysuser files (which adds users to existing groups)
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On 5/13/24 13:07, Florian Festi wrote:
On 5/11/24 12:56, Zbigniew Jędrzejewski-Szmek wrote:
On Fri, May 10, 2024 at 01:28:07PM +0200, Florian Festi wrote:
Anyone interested in picking this up? I remember quite a few people being exited about this when it was announced with the rpm-4.19 Change.
I would be interested in making this happen.
You mentioned that the transition "requires some care". What are the problems?
There are Requires created for the users and groups. To make this work the Provides need to be there first - obviously. So one will probably need to set %_use_weak_usergroup_deps for a transition period. At least until the first mass rebuild.
Fedora has already been through at least one mass rebuild under 4.19 so the provides (and weak requires) are already there.
There are also a large number of packages that are using useradd: grep useradd *.spec | cut -d: -f1 | sort -u | wc 281 281 4090
We need to think what to do with them.
The sysusers macros are much less used actually: grep sysusers_requires_compat *.spec | cut -d: -f1 | sort -u | wc 53 53 725 grep sysusers_create_compat *.spec | cut -d: -f1 | sort -u | wc 101 101 1476
This whole thing probably needs to be a Global Change involving a change to the Packaging Guidelines [4] and may be an Mass Package Change (although that might be avoided by changing the macros in systemd-rpm-macros to NOPs).
The macros are written in a way that if the user/group exist, no operation is done. Thus, naively, I would think that if rpm starts to create users and groups on its own, then the existing scriptlets would become noops. That would mean that we could enable the feature in rpm without any mass package changes first.
That might work, but I have not looked deep enough into that to do that blindly.
I'm quite sure there's *some* exception to the rule, but in general even the useradd era scripts are written in a way to allow pre-existing users. They kinda have to.
I outlined the migration process last year in https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... but failed to follow-up, so I'm glad to see this getting revisited.
- Panu -
On Mon, May 13, 2024 at 01:37:11PM +0300, Panu Matilainen wrote:
I outlined the migration process last year in https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... but failed to follow-up, so I'm glad to see this getting revisited.
I started looking into this, and I think we need to start at the bottom, i.e. in the setup package.
It currently provides /etc/{passwd,group} with a bunch of ids (23 groups) and /usr/lib/sysusers.d/20-setup-{users,groups} with a bunch of entries, but some of the groups listed in sysusers are not listed in the /etc files. IIUC, once we enable the rpm stuff, rpm will create /etc/{passwd,group} automatically, and the file provided by setup will be ignored. (It's specified as %config(noreplace).)
Should be drop the static /etc/{passwd,group} from setup?
Zbyszek
On 5/14/24 13:39, Zbigniew Jędrzejewski-Szmek wrote:
On Mon, May 13, 2024 at 01:37:11PM +0300, Panu Matilainen wrote:
I outlined the migration process last year in https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... but failed to follow-up, so I'm glad to see this getting revisited.
I started looking into this, and I think we need to start at the bottom, i.e. in the setup package.
It currently provides /etc/{passwd,group} with a bunch of ids (23 groups) and /usr/lib/sysusers.d/20-setup-{users,groups} with a bunch of entries, but some of the groups listed in sysusers are not listed in the /etc files. IIUC, once we enable the rpm stuff, rpm will create /etc/{passwd,group} automatically, and the file provided by setup will be ignored. (It's specified as %config(noreplace).)
Should be drop the static /etc/{passwd,group} from setup?
The static files aren't harmful as long as they're not duplicated in other packages.
I seem to recall seeing systemd-sysusers error out if those files were not present, but I might be misremembering and/or it might've changed since then. The default mechanism uses useradd/groupadd though, I don't know if those support non-existent /etc/{passwd,group}.
- Panu -
On Tue, May 14, 2024 at 02:01:09PM +0300, Panu Matilainen wrote:
On 5/14/24 13:39, Zbigniew Jędrzejewski-Szmek wrote:
On Mon, May 13, 2024 at 01:37:11PM +0300, Panu Matilainen wrote:
I outlined the migration process last year in https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... but failed to follow-up, so I'm glad to see this getting revisited.
I started looking into this, and I think we need to start at the bottom, i.e. in the setup package.
It currently provides /etc/{passwd,group} with a bunch of ids (23 groups) and /usr/lib/sysusers.d/20-setup-{users,groups} with a bunch of entries, but some of the groups listed in sysusers are not listed in the /etc files. IIUC, once we enable the rpm stuff, rpm will create /etc/{passwd,group} automatically, and the file provided by setup will be ignored. (It's specified as %config(noreplace).)
Should be drop the static /etc/{passwd,group} from setup?
The static files aren't harmful as long as they're not duplicated in other packages.
Harmful — no, but unnecessary and confusing. If we go decide to switch to the rpm sysusers mechanism, then I think we should go all-in on it. It doesn't make sense to ship a file in setup that would never be installed.
I seem to recall seeing systemd-sysusers error out if those files were not present, but I might be misremembering and/or it might've changed since then. The default mechanism uses useradd/groupadd though, I don't know if those support non-existent /etc/{passwd,group}.
There might have been bugs for some specific cases, but in general sysusers was always intended for starting with empty /etc. We certainly test that case in our tests.
Zbyszek
Hi,
I think all the issues wrt. sysusers in systemd and setup have been resolved.
On Tue, May 14, 2024 at 11:34:51AM +0000, Zbigniew Jędrzejewski-Szmek wrote:
On Tue, May 14, 2024 at 02:01:09PM +0300, Panu Matilainen wrote:
On 5/14/24 13:39, Zbigniew Jędrzejewski-Szmek wrote:
On Mon, May 13, 2024 at 01:37:11PM +0300, Panu Matilainen wrote:
I outlined the migration process last year in https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... but failed to follow-up, so I'm glad to see this getting revisited.
I started looking into this, and I think we need to start at the bottom, i.e. in the setup package.
It currently provides /etc/{passwd,group} with a bunch of ids (23 groups) and /usr/lib/sysusers.d/20-setup-{users,groups} with a bunch of entries, but some of the groups listed in sysusers are not listed in the /etc files. IIUC, once we enable the rpm stuff, rpm will create /etc/{passwd,group} automatically, and the file provided by setup will be ignored. (It's specified as %config(noreplace).)
I was confused here. setup generates its two sysusers files from the passwd/groups file that it distributes, so they will always match.
We added the missing group defintions that systemd-udev relies on to default groups file distributed by setup (in setup-2.15.0-3). The next build of systemd (256~rc4-1) will drop its sysusers.d/basic.conf file.
Please carry on with the enablement of rpm sysusers handling ;)
Zbyszek
P.S. While at it, Martin Osvald and I implemented a move of the content from the the "upstream" setup repo (https://pagure.io/setup/) into the dist-git repo (https://src.fedoraproject.org/rpms/pagure). The "upstream" was only used by a single "downstream", managed by the same people, and the separation was just generating busywork. setup >= 2.15 has all the content in dist-git.
Dne 06. 06. 24 v 22:25 Zbigniew Jędrzejewski-Szmek napsal(a):
Hi,
I think all the issues wrt. sysusers in systemd and setup have been resolved.
On Tue, May 14, 2024 at 11:34:51AM +0000, Zbigniew Jędrzejewski-Szmek wrote:
On Tue, May 14, 2024 at 02:01:09PM +0300, Panu Matilainen wrote:
On 5/14/24 13:39, Zbigniew Jędrzejewski-Szmek wrote:
On Mon, May 13, 2024 at 01:37:11PM +0300, Panu Matilainen wrote:
I outlined the migration process last year in https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... but failed to follow-up, so I'm glad to see this getting revisited.
I started looking into this, and I think we need to start at the bottom, i.e. in the setup package.
It currently provides /etc/{passwd,group} with a bunch of ids (23 groups) and /usr/lib/sysusers.d/20-setup-{users,groups} with a bunch of entries, but some of the groups listed in sysusers are not listed in the /etc files. IIUC, once we enable the rpm stuff, rpm will create /etc/{passwd,group} automatically, and the file provided by setup will be ignored. (It's specified as %config(noreplace).)
I was confused here. setup generates its two sysusers files from the passwd/groups file that it distributes, so they will always match.
We added the missing group defintions that systemd-udev relies on to default groups file distributed by setup (in setup-2.15.0-3). The next build of systemd (256~rc4-1) will drop its sysusers.d/basic.conf file.
Please carry on with the enablement of rpm sysusers handling ;)
Zbyszek
P.S. While at it, Martin Osvald and I implemented a move of the content from the the "upstream" setup repo (https://pagure.io/setup/) into the dist-git repo (https://src.fedoraproject.org/rpms/pagure).
https://src.fedoraproject.org/rpms/setup
was likely the intended link ...
Vít
The "upstream" was only used by a single "downstream", managed by the same people, and the separation was just generating busywork. setup >= 2.15 has all the content in dist-git. -- _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Now that the dust from 4.20 landing has settled a bit...
On 6/6/24 23:25, Zbigniew Jędrzejewski-Szmek wrote:
Hi,
I think all the issues wrt. sysusers in systemd and setup have been resolved.
On Tue, May 14, 2024 at 11:34:51AM +0000, Zbigniew Jędrzejewski-Szmek wrote:
On Tue, May 14, 2024 at 02:01:09PM +0300, Panu Matilainen wrote:
On 5/14/24 13:39, Zbigniew Jędrzejewski-Szmek wrote:
On Mon, May 13, 2024 at 01:37:11PM +0300, Panu Matilainen wrote:
I outlined the migration process last year in https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... but failed to follow-up, so I'm glad to see this getting revisited.
I started looking into this, and I think we need to start at the bottom, i.e. in the setup package.
It currently provides /etc/{passwd,group} with a bunch of ids (23 groups) and /usr/lib/sysusers.d/20-setup-{users,groups} with a bunch of entries, but some of the groups listed in sysusers are not listed in the /etc files. IIUC, once we enable the rpm stuff, rpm will create /etc/{passwd,group} automatically, and the file provided by setup will be ignored. (It's specified as %config(noreplace).)
I was confused here. setup generates its two sysusers files from the passwd/groups file that it distributes, so they will always match.
We added the missing group defintions that systemd-udev relies on to default groups file distributed by setup (in setup-2.15.0-3). The next build of systemd (256~rc4-1) will drop its sysusers.d/basic.conf file.
Please carry on with the enablement of rpm sysusers handling ;)
Excellent :) With the duplicates gone from systemd basic.conf gone, a logical next step would be turning on hard dependencies for users and groups before the F41 mass rebuild (by dropping rpm-4.19.91-weak-user-group.patch from Fedora rpm).
All the necessary user/group provides should already be in place since F40 mass rebuild, and it shouldn't matter which mechanism actually creates the users, so it's not committing to any changes in user/group handling as such, this is just an extra packaging hygiene step in the process.
I'll be out for the entire July, but Florian and Michal will be around.
- Panu -
On Fri, May 10, 2024 at 01:28:07PM +0200, Florian Festi wrote:
Hi everyone!
RPM 4.19 added automatic sysuser handling [1]. In Fedora 39 this feature was not enabled right away [2] as it requires some care to properly transition to it. Also going back to 4.18 was technically still the fallback option during this change.
I just noticed in an issue in the RPM upstream repository [3] that the sysuser feature is still not enabled. May be right now might be a good time to get this going for Fedora 41. I am happy to help with the technical details but would prefer if this effort was driven from within Fedora.
Currently users are either done manually by calling useradd in scriptlets or using the macros in systemd-rpm-macros which is a sub package of the systemd package. RPM's mechanism is switched off by rpm-4.18.92-disable-sysusers.patch in the rpm package.
This whole thing probably needs to be a Global Change involving a change to the Packaging Guidelines [4] and may be an Mass Package Change (although that might be avoided by changing the macros in systemd-rpm-macros to NOPs).
Anyone interested in picking this up? I remember quite a few people being exited about this when it was announced with the rpm-4.19 Change.
IIRC with the current sysusers impl from systemd-rpm-macros, there's an hard constraint, that the sysusers file had to be explicitly listed as a separate "SourceNN" file. You could not point %sysusers_create_compat to a file from the either the build root, or the unpacked source tarball, as the macro appeared to load its argument at spec parse time.
Does the native RPM implementation avoid this limitation ? It looks like it probably does...
With regards, Daniel
On 5/13/24 13:24, Daniel P. Berrangé wrote:
On Fri, May 10, 2024 at 01:28:07PM +0200, Florian Festi wrote:
Hi everyone!
RPM 4.19 added automatic sysuser handling [1]. In Fedora 39 this feature was not enabled right away [2] as it requires some care to properly transition to it. Also going back to 4.18 was technically still the fallback option during this change.
I just noticed in an issue in the RPM upstream repository [3] that the sysuser feature is still not enabled. May be right now might be a good time to get this going for Fedora 41. I am happy to help with the technical details but would prefer if this effort was driven from within Fedora.
Currently users are either done manually by calling useradd in scriptlets or using the macros in systemd-rpm-macros which is a sub package of the systemd package. RPM's mechanism is switched off by rpm-4.18.92-disable-sysusers.patch in the rpm package.
This whole thing probably needs to be a Global Change involving a change to the Packaging Guidelines [4] and may be an Mass Package Change (although that might be avoided by changing the macros in systemd-rpm-macros to NOPs).
Anyone interested in picking this up? I remember quite a few people being exited about this when it was announced with the rpm-4.19 Change.
IIRC with the current sysusers impl from systemd-rpm-macros, there's an hard constraint, that the sysusers file had to be explicitly listed as a separate "SourceNN" file. You could not point %sysusers_create_compat to a file from the either the build root, or the unpacked source tarball, as the macro appeared to load its argument at spec parse time.
Does the native RPM implementation avoid this limitation ? It looks like it probably does...
Yup, no such limitation in the RPM implementation. All you need to do is package sysusers file(s) in their native location.
- Panu -
-
Daniel P. Berrangé -
Florian Festi -
Panu Matilainen -
Vít Ondruch -
Zbigniew Jędrzejewski-Szmek