I've finally finished writing patches to integrate zchunk support into dnf/libsolv/librepo[1], and I'd greatly appreciate some code review. A vast majority of the code is in librepo, but libsolv has been expanded to support zchunk files and dnf has a tiny patch that passes the base cache directory to librepo to find source zchunk files to delta against.
With these patches and a zchunk-enabled repository, you will download only the differences in your primary/filelists/other metadata. Partial downloads are validated and then continued, and each chunk is validated as it's downloaded, ending the download and moving to a new mirror if any chunk is corrupt.
I would love to get these changes into Fedora 29, and the code is testable now, but with only three weeks until System-Wide change proposals are due, I'm not sure if I'm being ambitious.
On another note, I would also like to finalize zchunk's API and make a stable ABI promise, but before I take that step I'd really love some feedback on its usability.
Jonathan
[1] https://github.com/rpm-software-management/dnf/pull/1107 https://github.com/openSUSE/libsolv/pull/270 https://github.com/rpm-software-management/librepo/pull/127
On Tue, Jun 12, 2018 at 5:21 AM Jonathan Dieter jdieter@gmail.com wrote:
I've finally finished writing patches to integrate zchunk support into dnf/libsolv/librepo[1], and I'd greatly appreciate some code review. A vast majority of the code is in librepo, but libsolv has been expanded to support zchunk files and dnf has a tiny patch that passes the base cache directory to librepo to find source zchunk files to delta against.
This is awesome, but we're missing patches for libdnf and createrepo_c. PackageKit and microdnf rely on libdnf for all of this, and no one can create zck rpmmd without a suitably enhanced createrepo_c. Could you please make PRs against both for that? :)
On Tue, 2018-06-12 at 05:24 -0400, Neal Gompa wrote:
On Tue, Jun 12, 2018 at 5:21 AM Jonathan Dieter jdieter@gmail.com wrote:
I've finally finished writing patches to integrate zchunk support into dnf/libsolv/librepo[1], and I'd greatly appreciate some code review. A vast majority of the code is in librepo, but libsolv has been expanded to support zchunk files and dnf has a tiny patch that passes the base cache directory to librepo to find source zchunk files to delta against.
This is awesome, but we're missing patches for libdnf and createrepo_c. PackageKit and microdnf rely on libdnf for all of this, and no one can create zck rpmmd without a suitably enhanced createrepo_c. Could you please make PRs against both for that? :)
Here's createrepo_c. It was failing the python testcases and I wanted those fixed before putting out the pull request.
https://github.com/rpm-software-management/createrepo_c/pull/92
I'm working on libdnf.
Jonathan
On Tue, 2018-06-12 at 05:24 -0400, Neal Gompa wrote:
On Tue, Jun 12, 2018 at 5:21 AM Jonathan Dieter jdieter@gmail.com wrote:
I've finally finished writing patches to integrate zchunk support into dnf/libsolv/librepo[1], and I'd greatly appreciate some code review. A vast majority of the code is in librepo, but libsolv has been expanded to support zchunk files and dnf has a tiny patch that passes the base cache directory to librepo to find source zchunk files to delta against.
This is awesome, but we're missing patches for libdnf and createrepo_c. PackageKit and microdnf rely on libdnf for all of this, and no one can create zck rpmmd without a suitably enhanced createrepo_c. Could you please make PRs against both for that? :)
And I've done libdnf:
https://github.com/rpm-software-management/libdnf/pull/478
Jonathan
On Tue, 2018-06-12 at 12:21 +0300, Jonathan Dieter wrote:
I would love to get these changes into Fedora 29, and the code is testable now, but with only three weeks until System-Wide change proposals are due, I'm not sure if I'm being ambitious.
FWIW, I have a COPR available for F28 and Rawhide with zchunk-enabled dnf/libdnf and the supporting libraries.
https://copr.fedorainfracloud.org/coprs/jdieter/dnf-zchunk/
Obviously, you'll need a zchunk-enabled repository to test it.
Jonathan
infrastructure@lists.fedoraproject.org
-
Jonathan Dieter -
Neal Gompa