Hi,
I'm dabbling in writing a small Python application (further details below to provide some context). This application:
* Has no dependencies other than the Python standard library * Makes only sense to be installed using your distribution's package manager * Would initially support *only* EL8, although later on it could be expanded to support other distributions * Proper testing requires testing for each of the supported Linux distributions for the tool. The application should be packaged and deployed on a system running each supported distro and then certain commands need to be run to verify it is working fine
I would like:
* Making it easy for developers to hack in the package. They should be able to use any Linux system to develop. I would provide scripts using containers (podman/docker) to run tests and create packages for all supported distros. * Keep the packaging in the main source Git repo * Packaging should be fully automatized in CI/CD. All commits should generate packages for all supported distros (with a timestamped version) and push it to a repo (I would use COPR, ppa, etc.). Latest master should be easily installable using dnf, other branches should be easily installable for testing purposes. * I also like using Poetry for development (e.g. while the application itself would have 0 dependencies, development will require testing/style-checking/etc. and setting that up with venvs is friendlier to devs, instead of requiring the tools to be installed manually).
My questions (as a relative noob to RPM packaging- I've created and automated some RPM packaging, but mostly by winging it):
* Is any of the above a terrible idea I should reconsider? * Is there any project out there with similar goals doing things "correctly" I could "copy"? * pyproject-rpm-macros looks like something useful for what I want to do, can it be used on to package for EL8? (I only see Fedora branches)
Cheers,
Álex
------------------------- Context:
Software such as the EL8 MariaDB server RPM requires additional steps to be configured, such as running mysql_secure_installation. If you are using MariaDB as the back end database for an application (e.g. NextCloud), you will also typically want to create a database and a user for the application.
It is fairly easy to perform those steps manually. Under certain practices (such as container-heavy setups or very cattle-like deployments) it can also be easy to do that.
However, if you are using automated configuration management based on idempotency, such as Ansible, Puppet, etc. it is a bit more complex to perform that.
To cover that need, typically modules are developed for those configuration management tools which solve those issues.
However, a module developed for MariaDB/Puppet is not ideal for Ansible users, etc.
A better way would be to develop a command-line tool which can set up MariaDB using declarative configuration (e.g. a file that describes which databases/users/etc. are desired). Then this command-line tool can be easily executed using whatever configuration management tool. This tool could offer a generic way to implement idempotency (e.g. a --check flag whose return code indicates whether changes need to be performed).
On 29. 08. 20 13:55, Alex Corcoles wrote:
Hi,
Hi Alex.
I'm dabbling in writing a small Python application (further details below to provide some context). This application:
- Has no dependencies other than the Python standard library
- Makes only sense to be installed using your distribution's package manager
Why? This is a requirement I don't understand.
...
I would like:
- Making it easy for developers to hack in the package. They should be able to use any Linux system to develop. I would provide scripts using containers (podman/docker) to run tests and create packages for all supported distros.
Making it pip installable makes it very easy to run. Using poetry makes it very easy to hack on.
- Keep the packaging in the main source Git repo
The RPM packaging or the Python (pip/poetry packaging)?
- Packaging should be fully automatized in CI/CD. All commits should generate packages for all supported distros (with a timestamped version) and push it to a repo (I would use COPR, ppa, etc.). Latest master should be easily installable using dnf, other branches should be easily installable for testing purposes.
That should be easily achievable with pip-installable packages with the help of setuptools_scm (possibly poetry also has something like this). Not sure about dnf/rpm packages. Copr has some CI abilities.
...
My questions (as a relative noob to RPM packaging- I've created and automated some RPM packaging, but mostly by winging it):
- Is any of the above a terrible idea I should reconsider?
That depends on the answer for my main question.
- Is there any project out there with similar goals doing things "correctly" I could "copy"?
I don't know any Python project with automatic Copr builds from git, but maybe look at a non-Python one?
https://copr.fedorainfracloud.org/coprs/dcantrell/rpminspect/package/rpminsp... https://github.com/rpminspect/rpminspect/blob/master/.copr/Makefile
- pyproject-rpm-macros looks like something useful for what I want to do, can it be used on to package for EL8? (I only see Fedora branches)
It cannot. pyproject-rpm-macros heavaily relies on technology not yet (fully) available in EPEL 8 which is unfortunately now an ancient distribution when it comes to leading edge stuff :(
- RPM buildrequires generators are missing (old RPM) - new Python RPM dependency generators would have to be backported ideally with parametric generators from Fedora 33+'s RPM - RHEL 8 has an ancient pip version 9, it might work, however nobody was brave enough to try it) - tox 3.13+ is needed (EPEL 8 has tox 3.4) for %tox
Hi Miro,
- Makes only sense to be installed using your distribution's package
manager
Why? This is a requirement I don't understand.
That might be an overstatement. This is software to help install and configure other software, so it doesn't make sense if it has a complex installation procedure.
Of course, installing using pip from an URL is a simple installation procedure, so I could go for that.
- Keep the packaging in the main source Git repo
The RPM packaging or the Python (pip/poetry packaging)?
The RPM packaging. I don't often see many examples of software which is package-friendly out there.
- Is there any project out there with similar goals doing things
"correctly" I could "copy"?
I don't know any Python project with automatic Copr builds from git, but maybe look at a non-Python one?
https://copr.fedorainfracloud.org/coprs/dcantrell/rpminspect/package/rpminsp... https://github.com/rpminspect/rpminspect/blob/master/.copr/Makefile
Oh, that's most useful, thanks!
- pyproject-rpm-macros looks like something useful for what I want to
do, can it be used on to package for EL8? (I only see Fedora branches)
It cannot. pyproject-rpm-macros heavaily relies on technology not yet (fully) available in EPEL 8 which is unfortunately now an ancient distribution when it comes to leading edge stuff :(
- RPM buildrequires generators are missing (old RPM)
- new Python RPM dependency generators would have to be backported ideally with parametric generators from Fedora 33+'s RPM
- RHEL 8 has an ancient pip version 9, it might work, however nobody was brave enough to try it)
- tox 3.13+ is needed (EPEL 8 has tox 3.4) for %tox
No worries, I just wanted to know if it was worth investigating that.
Thanks!
I might for the moment make something pip-installable from a URL, so I have something "useful" as soon as possible, and postpone RPM packaging to after that.
Thanks!
Álex
On 30. 08. 20 11:40, Alex Corcoles wrote:
I might for the moment make something pip-installable from a URL, so I have something "useful" as soon as possible, and postpone RPM packaging to after that.
I'd recommend making it pip installable from PyPI. That way you can have some CI (such as GitHub Action if GitHub is your git forge) to publish sdist / wheels to PyPI on every push to the default branch. With tools like setuptools_scm, you ensure that each newly created version on PyPI is automatically bumped. With git tags, you can control what versions are considered "stable" and what versions are considered "prereleases".
python-devel@lists.fedoraproject.org