I installed all versions of yum-presto DeltaRPM package for Fedora Core 6 since 0.34 and they all worked great! I didn't have any issues. Now I run 0.38 and it works perfectly.
I can't wait to test it on Fedora 7 test 3 I have on my other machine becuase I saw that it still needs a bit of polish to start running fine on Fedora 7.
RPM build speed is also much better that SUSE updater! Bravo for yum-presto team!
What experience do others have regarding yum-presto? Do you like it? Do you have any feature ideas?
I would like to see the yum-presto status after update with bandwidth savings in human readable notation. If I'm not wrong there was that status in 0.34 but I didn't see it later version - correct me if I'm overlooking it.
Thank you, Valent from Croatia.
On Wed, 2007-04-11 at 08:46 +0200, Valent Turkovic wrote: <snip>
I would like to see the yum-presto status after update with bandwidth savings in human readable notation. If I'm not wrong there was that status in 0.34 but I didn't see it later version - correct me if I'm overlooking it.
Thank you, Valent from Croatia.
Just a heads up - if you're not getting the screen with the bandwidth savings, then yum-presto hasn't downloaded any deltarpms.
If this happens consistently, it's probably because your .repo files don't point to the yum-presto repository. The easiest way to fix it is: in fedora-updates.repo, add: deltaurl=http://www.lesbg.com/jdieter/updates/fc$releasever/$basearch and in fedora-extras.repo, add: deltaurl=http://www.lesbg.com/jdieter/extras/fc$releasever/$basearch
These should become automatic once repository makers start creating presto repositories themselves.
Thanks for the positive feedback.
Jonathan
Le mercredi 11 avril 2007 à 10:44 +0300, Jonathan Dieter a écrit :
On Wed, 2007-04-11 at 08:46 +0200, Valent Turkovic wrote:
<snip> > I would like to see the yum-presto status after update with bandwidth > savings in human readable notation. If I'm not wrong there was that > status in 0.34 but I didn't see it later version - correct me if I'm > overlooking it. > > Thank you, > Valent from Croatia. > Just a heads up - if you're not getting the screen with the bandwidth savings, then yum-presto hasn't downloaded any deltarpms.
If this happens consistently, it's probably because your .repo files don't point to the yum-presto repository. The easiest way to fix it is: in fedora-updates.repo, add: deltaurl=http://www.lesbg.com/jdieter/updates/fc$releasever/$basearch and in fedora-extras.repo, add: deltaurl=http://www.lesbg.com/jdieter/extras/fc$releasever/$basearch
These should become automatic once repository makers start creating presto repositories themselves.
Thanks for the positive feedback.
You have to put deltaurl in fedora-updates.repo and fedora-extras.repo and not in /etc/yum/pluginconf.d/presto.conf ?
When the original repo will create also deltarepo ? When you update your system soon the deltarepo is not created and you download the packages not the delta ones.
Is it possible to integrate presto with the gui updater ? (that you can read the bandwidth savings ?) Thanks
Eric
I wonder if it is time to discuss whether we want deltarpm's to be distributed by default on the mirrors, or to enable it by default in Fedora's yum.
Perhaps it is too late to do this for Fedora 7, and adding it to the mirrors might be best to do when the mirror layout changes for the merged distribution.
Warren Togami wtogami@redhat.com
On Wednesday 11 April 2007 11:39:00 Warren Togami wrote:
I wonder if it is time to discuss whether we want deltarpm's to be distributed by default on the mirrors, or to enable it by default in Fedora's yum.
Perhaps it is too late to do this for Fedora 7, and adding it to the mirrors might be best to do when the mirror layout changes for the merged distribution.
Correct me if I'm wrong, but deltarpms don't yet support the Fedora 7 platform do they?
Jesse Keating wrote:
On Wednesday 11 April 2007 11:39:00 Warren Togami wrote:
I wonder if it is time to discuss whether we want deltarpm's to be distributed by default on the mirrors, or to enable it by default in Fedora's yum.
Perhaps it is too late to do this for Fedora 7, and adding it to the mirrors might be best to do when the mirror layout changes for the merged distribution.
Correct me if I'm wrong, but deltarpms don't yet support the Fedora 7 platform do they?
There are obviously some fixes that are required at the client level: yum presto plugin. However we can encourage the mirrors to presto enable all the repositories right away.
Since the client is in extras users can choose to install it for FC5, FC6 and rawhide (this one is esp interesting right now) and we should look at installing the plugin by default in F8.
Rahul
On Wednesday 11 April 2007 11:55:13 Rahul Sundaram wrote:
There are obviously some fixes that are required at the client level: yum presto plugin. However we can encourage the mirrors to presto enable all the repositories right away.
Well, we don't ask the mirrors to generate things, we generate things at the source and the mirrors just pick it up. If the mirrors have to do something themselves, we've lost.
Since the client is in extras users can choose to install it for FC5, FC6 and rawhide (this one is esp interesting right now) and we should look at installing the plugin by default in F8.
I really don't want to add this midstream to a released product. The right process here is to get it working in rawhide during the open development cycle, start producing the content for it at the source, have it mirrored out and get more and more people beating on it during rawhide. If it survives and makes it into the released product as a default, that's awesome, from that point on we use it. Trying to start at a released product and port forward just seems backwards to me, and not something I want to expose all of our "stable" Fedora 6 users to.
On Mi April 11 2007, Jesse Keating wrote:
Well, we don't ask the mirrors to generate things, we generate things at the source and the mirrors just pick it up. If the mirrors have to do something themselves, we've lost.
The delta rpms only need to be created at the source.
I really don't want to add this midstream to a released product. The right process here is to get it working in rawhide during the open development cycle, start producing the content for it at the source, have it mirrored out and get more and more people beating on it during rawhide. If it
The yum plugin is already in Fedora Extras 6, only the official delta rpms are missing. They can be generated without interfering with the full rpms, so they cannot break anything on peoples machines, that do not use yum-presto (the delta rpms plugin). For this reason, I see no harm done, when they get already distributed by Fedora / generated at the source. Only the directory layout needs to be decided upon.
Regards, Till
Jesse Keating wrote:
I really don't want to add this midstream to a released product. The right process here is to get it working in rawhide during the open development cycle, start producing the content for it at the source, have it mirrored out and get more and more people beating on it during rawhide. If it survives and makes it into the released product as a default, that's awesome, from that point on we use it. Trying to start at a released product and port forward just seems backwards to me, and not something I want to expose all of our "stable" Fedora 6 users to.
You don't expose them really to any additional risks by default. If the delta rpms are already mirrored, yum just ignores them without the plugin. Since the plugin is already in Fedora Extras users just would have install them explicitly to use the delta rpms. The plugin falls back to downloading the full rpm if the delta rpm fails or is not available.
Rahul
On Wednesday 11 April 2007 13:38:15 Rahul Sundaram wrote:
You don't expose them really to any additional risks by default. If the delta rpms are already mirrored, yum just ignores them without the plugin. Since the plugin is already in Fedora Extras users just would have install them explicitly to use the delta rpms. The plugin falls back to downloading the full rpm if the delta rpm fails or is not available.
It is still not something I want to do to a released product, or even Fedora 7 after the feature freeze. The generation of the delta rpms, the layout, the use of the plugin, etc.. these are all codepaths that need more exposure/testing during an open development phase. I think it's great that you guys have gotten it to the point it is now, and I really look forward to seeing it get wider use once we start up Fedora 8. I just don't want to add new features/functionality into 7 and 6.
Jesse Keating wrote:
On Wednesday 11 April 2007 13:38:15 Rahul Sundaram wrote:
You don't expose them really to any additional risks by default. If the delta rpms are already mirrored, yum just ignores them without the plugin. Since the plugin is already in Fedora Extras users just would have install them explicitly to use the delta rpms. The plugin falls back to downloading the full rpm if the delta rpm fails or is not available.
It is still not something I want to do to a released product, or even Fedora 7 after the feature freeze. The generation of the delta rpms, the layout, the use of the plugin, etc.. these are all codepaths that need more exposure/testing during an open development phase. I think it's great that you guys have gotten it to the point it is now, and I really look forward to seeing it get wider use once we start up Fedora 8. I just don't want to add new features/functionality into 7 and 6.
So let me ask. Who or which team decides this? What about application defaults or which packages to install by default? Is this supposed to be handled by release engineering or FESCo?
I have a suggestion for a package to be installed by default that I would like to see someone or a team take ownership and give me a decisive response.
https://www.redhat.com/archives/fedora-desktop-list/2007-March/msg00032.html
Rahul
On Wed, 2007-04-11 at 23:42 +0530, Rahul Sundaram wrote:
Jesse Keating wrote:
On Wednesday 11 April 2007 13:38:15 Rahul Sundaram wrote:
You don't expose them really to any additional risks by default. If the delta rpms are already mirrored, yum just ignores them without the plugin. Since the plugin is already in Fedora Extras users just would have install them explicitly to use the delta rpms. The plugin falls back to downloading the full rpm if the delta rpm fails or is not available.
It is still not something I want to do to a released product, or even Fedora 7 after the feature freeze. The generation of the delta rpms, the layout, the use of the plugin, etc.. these are all codepaths that need more exposure/testing during an open development phase. I think it's great that you guys have gotten it to the point it is now, and I really look forward to seeing it get wider use once we start up Fedora 8. I just don't want to add new features/functionality into 7 and 6.
+10. I am in complete agreement with Jesse here.
So let me ask. Who or which team decides this? What about application defaults or which packages to install by default? Is this supposed to be handled by release engineering or FESCo?
Things that get into the spins are handled by rel-eng. If there's dispute, it goes up to FESCo.
I have a suggestion for a package to be installed by default that I would like to see someone or a team take ownership and give me a decisive response.
https://www.redhat.com/archives/fedora-desktop-list/2007-March/msg00032.html
You should start a separate thread on that. At first glance, it doesn't make any sense to me at all as to how that would be useful on a liveCD.
josh
Josh Boyer wrote:
https://www.redhat.com/archives/fedora-desktop-list/2007-March/msg00032.html
You should start a separate thread on that. At first glance, it doesn't make any sense to me at all as to how that would be useful on a liveCD.
It is especially useful in the live cd if we do the configuration by default since NetworkManager and front ends are turned on by default in the GNOME live cd unlike the prime spin.
Rahul
On Wednesday 11 April 2007 14:12:39 Rahul Sundaram wrote:
So let me ask. Who or which team decides this? What about application defaults or which packages to install by default? Is this supposed to be handled by release engineering or FESCo?
Good question. In the past it was the "core cabal" who is mostly now on FESCo as well as the release team, however these two teams have many more people on them than what was the cabal before.
I have a suggestion for a package to be installed by default that I would like to see someone or a team take ownership and give me a decisive response.
Given that both the bugs you referenced are still in NEW state, I find it unlikely that we'd be adding this package to the default list in comps. Perhaps you haven't gotten any responses because the bugs that should be fixed for its inclusion are not fixed yet.
Jesse Keating wrote:
On Wednesday 11 April 2007 14:12:39 Rahul Sundaram wrote:
So let me ask. Who or which team decides this? What about application defaults or which packages to install by default? Is this supposed to be handled by release engineering or FESCo?
Good question. In the past it was the "core cabal" who is mostly now on FESCo as well as the release team, however these two teams have many more people on them than what was the cabal before.
Should FESCo be the deciding body here? I would like to have a team of people look at decisions rather than a single person so that we have a chance of a debate and different outlooks.
I have a suggestion for a package to be installed by default that I
would like to see someone or a team take ownership and give me a decisive response.
Given that both the bugs you referenced are still in NEW state, I find it unlikely that we'd be adding this package to the default list in comps. Perhaps you haven't gotten any responses because the bugs that should be fixed for its inclusion are not fixed yet.
Actually I did get positive responses from the maintainer, upstream developer and others. The enhancements are "nice to have" but the package is still very functional in its current state.
Rahul
On Wednesday 11 April 2007 15:09:10 Rahul Sundaram wrote:
Should FESCo be the deciding body here? I would like to have a team of people look at decisions rather than a single person so that we have a chance of a debate and different outlooks.
For lack of a better body, sure. I'd like to see SIGs really decide what goes into their comps group as mandatory/default/optional, and the release team + FESCo would only have to get involved if there is a dispute about what they're trying to do. Of course that would mean defining the stake holders for the various groups within comps, but that's somewhat needed anyway.
I have a suggestion for a package to be installed by default that I
would like to see someone or a team take ownership and give me a decisive response.
Given that both the bugs you referenced are still in NEW state, I find it unlikely that we'd be adding this package to the default list in comps. Perhaps you haven't gotten any responses because the bugs that should be fixed for its inclusion are not fixed yet.
Actually I did get positive responses from the maintainer, upstream developer and others. The enhancements are "nice to have" but the package is still very functional in its current state.
"We have two bugs open against pam_keyring now both of which are very important to fix if we need to put this in by default." Sounds to me like we should get those fixed first. Maybe I'm misreading...
On Mi April 11 2007, Jesse Keating wrote:
It is still not something I want to do to a released product, or even Fedora 7 after the feature freeze. The generation of the delta rpms, the layout, the use of the plugin, etc.. these are all codepaths that need more exposure/testing during an open development phase. I think it's great that you guys have gotten it to the point it is now, and I really look forward to seeing it get wider use once we start up Fedora 8. I just don't want to add new features/functionality into 7 and 6.
With the plugin already beeing in Fedora Extras 6, the feature is already there. Also Nearly every new Extras package is also built for the supported releases and not only for devel, so there are always new features and functionality in stable releases. And when the plugin is tested in devel, then the delta rpms and mirror layout has to be generated, too. So the only differences are, whether the Fedora 6 / 7 or devel rpms are used for the generation of the delta rpms and an extra directory for the different releases.
Regards, Till
On Wed, 2007-04-11 at 20:25 +0200, Till Maas wrote:
On Mi April 11 2007, Jesse Keating wrote:
It is still not something I want to do to a released product, or even Fedora 7 after the feature freeze. The generation of the delta rpms, the layout, the use of the plugin, etc.. these are all codepaths that need more exposure/testing during an open development phase. I think it's great that you guys have gotten it to the point it is now, and I really look forward to seeing it get wider use once we start up Fedora 8. I just don't want to add new features/functionality into 7 and 6.
With the plugin already beeing in Fedora Extras 6, the feature is already there. Also Nearly every new Extras package is also built for the supported
But it isn't really used if the deltarpms don't exist. You still need a repo with those in it. Without the actual deltarpms in the official repos, you aren't really enabling the feature. So for people to actual _use_ it, they'd have to manually add a repo containing the deltarpms.
josh
On Mi April 11 2007, Josh Boyer wrote:
But it isn't really used if the deltarpms don't exist. You still need a repo with those in it. Without the actual deltarpms in the official repos, you aren't really enabling the feature. So for people to actual _use_ it, they'd have to manually add a repo containing the deltarpms.
Even if the repos contain deltarpms, one does not need to activate the yum-presto plugin, it can also be installed with "enabled=0" in its configuration.
Regards, Till
On Wed, 2007-04-11 at 22:20 +0200, Till Maas wrote:
On Mi April 11 2007, Josh Boyer wrote:
But it isn't really used if the deltarpms don't exist. You still need a repo with those in it. Without the actual deltarpms in the official repos, you aren't really enabling the feature. So for people to actual _use_ it, they'd have to manually add a repo containing the deltarpms.
Even if the repos contain deltarpms, one does not need to activate the yum-presto plugin, it can also be installed with "enabled=0" in its configuration.
Yes. From where I sit though, having deltarpms in the official repos is an advertisement that the feature is available, tested, and fairly stable. If users see that the repos contain the deltarpms, why wouldn't they enable the plugin?
While yum-presto looks really promising, I agree with Jesse in that it really needs to be put through the paces in rawhide before we enable deltarpm creation in the offical repos.
josh
Josh Boyer wrote:
While yum-presto looks really promising, I agree with Jesse in that it really needs to be put through the paces in rawhide before we enable deltarpm creation in the offical repos.
Um, how is such testing to take place if there exist no (official) presto-enabled repos?
I'm of the mind to "just do it", regardless of whether it is enabled by default or not. fwiw, I don't think the tools to generate preso-compatible repos exist yet, so it may be a moot point, for now.
-- Rex
On Thu, 2007-04-12 at 06:18 -0500, Rex Dieter wrote:
Um, how is such testing to take place if there exist no (official) presto-enabled repos?
I'm of the mind to "just do it", regardless of whether it is enabled by default or not. fwiw, I don't think the tools to generate preso-compatible repos exist yet, so it may be a moot point, for now.
-- Rex
Sorry I've been off fedora-devel the last few days, but we're in the middle of Easter vacation here and my wife's sister is visiting, so we're showing her the sights in Lebanon.
First off, as Rex mentioned, the tools to generate presto-compatible repos are very much in the "alpha" stage, though that's what my focus is on at the moment. So, in many ways, this whole thing is moot, at least for the next few weeks.
Jesse, I totally understand where you're coming from on the "it should have been started in Rawhide". I'm afraid that's my fault, though I'll beg extenuating circumstances. I live and work in Beirut where broadband means anything but (thus my interest in the problem in the first place).
Downloading FC6 took three days of tying up the school's internet. Downloading Rawhide will take just as long. That's why I haven't done it yet. All that to say, I will do what I can to get Rawhide ASAP. I've even got 20 GB on my laptop's HD set aside for it.
Finally, when we do get to the point of "do we put deltarpms in the master mirror", *if* we decide *not* to (for the many reasons mentioned by those against the idea), is it possible for yum-presto in Extras to point to the test server in the default .conf file?
I would obviously prefer to put the deltarpms directly on the master mirror, but if that's not a possibility, I would at least like to make it easy for people to install yum-presto and not worry about manually editing .conf (or .repo) files. (And yes, I do realize that this "a bad thing".)
Jonathan
On Thursday 12 April 2007 14:27:45 Jonathan Dieter wrote:
Jesse, I totally understand where you're coming from on the "it should have been started in Rawhide". I'm afraid that's my fault, though I'll beg extenuating circumstances. I live and work in Beirut where broadband means anything but (thus my interest in the problem in the first place).
Well, even if development started in fc6, that's fine. I think DEPLOYMENT should begin with rawhide, and work its way down.
Finally, when we do get to the point of "do we put deltarpms in the master mirror", *if* we decide *not* to (for the many reasons mentioned by those against the idea), is it possible for yum-presto in Extras to point to the test server in the default .conf file?
Typically we don't allow packages other than fedora-release to add repo files. Maybe we could grant an exception for this package for a temporary time being, but once a repo file is on, it is difficult to get it off, so wherever you point you might want to be willing to at least maintain a redirect to somewhere where content lives.
On 4/12/07, Jesse Keating jkeating@redhat.com wrote:
On Thursday 12 April 2007 14:27:45 Jonathan Dieter wrote:
Jesse, I totally understand where you're coming from on the "it should have been started in Rawhide". I'm afraid that's my fault, though I'll beg extenuating circumstances. I live and work in Beirut where broadband means anything but (thus my interest in the problem in the first place).
Well, even if development started in fc6, that's fine. I think DEPLOYMENT should begin with rawhide, and work its way down.
Finally, when we do get to the point of "do we put deltarpms in the master mirror", *if* we decide *not* to (for the many reasons mentioned by those against the idea), is it possible for yum-presto in Extras to point to the test server in the default .conf file?
Typically we don't allow packages other than fedora-release to add repo files. Maybe we could grant an exception for this package for a temporary time being, but once a repo file is on, it is difficult to get it off, so wherever you point you might want to be willing to at least maintain a redirect to somewhere where content lives.
-- Jesse Keating Release Engineer: Fedora
Hi, can you please update us with the status of yum-presto and Fedora 7?
I installed Fedora 7 test 4 and I don't see yum-prest and deltarpm installed by default.
Will this change in the mean time so it is installed by default when Fedora 7 ships?
I can only say that deltarpm is a great idea, and works great in practice (on Fedora Core 6 - I still haven't tested it on Fedora 7).
It is maybe little less needed when people install new pacgakes but for updates it is a godsend...
You should think how to enable more and more people to use linux, and great features like fast updates with small bandwidth cost is a really big benefit to linux users.
There are millions of people with dialup and other slow connections on our planet and updates for them are almost impossible. Or students with laptops who can do some big installs when they are at their university but at home have also slow connection.
For all of them bandwidth is scarce and deltarpms is a great solution to that problem so please do all you can with implementing it.
Thank you in advance.
Valent from Croatia.
On Tue, 2007-05-08 at 20:49 +0200, Valent Turkovic wrote:
Hi, can you please update us with the status of yum-presto and Fedora 7?
I installed Fedora 7 test 4 and I don't see yum-prest and deltarpm installed by default.
It won't be.
Will this change in the mean time so it is installed by default when Fedora 7 ships?
No.
I can only say that deltarpm is a great idea, and works great in practice (on Fedora Core 6 - I still haven't tested it on Fedora 7).
It is maybe little less needed when people install new pacgakes but for updates it is a godsend...
You should think how to enable more and more people to use linux, and great features like fast updates with small bandwidth cost is a really big benefit to linux users.
There are millions of people with dialup and other slow connections on our planet and updates for them are almost impossible. Or students with laptops who can do some big installs when they are at their university but at home have also slow connection.
For all of them bandwidth is scarce and deltarpms is a great solution to that problem so please do all you can with implementing it.
These are all perfectly valid reasons. However, it is entirely too late to enable this by default for Fedora 7. And, realistically, it should be tested throughout a development cycle in rawhide before being enabled by default for a released distro.
josh
Please look at this:
Size of all updates downloaded from Presto-enabled repositories: 14M Size of updates that would have been downloaded if Presto wasn't enabled: 32M This is a savings of 56 percent
Updated: gimp.i386 2:2.2.14-5.fc6 gimp-libs.i386 2:2.2.14-5.fc6 gnome-media.i386 0:2.16.1-4.fc6 gstreamer.i386 0:0.10.11-1.fc6 gstreamer-plugins-base.i386 0:0.10.11-1.fc6 gstreamer-tools.i386 0: 0.10.11-1.fc6 gtk2.i386 0:2.10.8-3.fc6 hpijs.i386 1:1.7.2-3.fc6 hplip.i3860: 1.7.2-3.fc6 httpd.i386 0:2.2.4-2.fc6 httpd-manual.i386 0:2.2.4-2.fc6 Dependency Updated: mod_ssl.i386 1:2.2.4-2.fc6 Complete!
Full update gave me even better results!!! 75% savings !!!
Size of all updates downloaded from Presto-enabled repositories: 10M Size of updates that would have been downloaded if Presto wasn't enabled: 41M This is a savings of 75 percent
Dependency Installed: freeglut.i386 0:2.4.0-11.fc6 libcaca.i386 0: 0.99-0.1.beta11.fc6 libfame.i386 0:0.9.1-12.fc6 xine-lib-moles.i386 0: 1.1.6-1.fc6 Updated: coreutils.i386 0:5.97-12.5.fc6 dbus.i386 0:1.0.1-12.fc6 dbus-devel.i386 0:1.0.1-12.fc6 dbus-x11.i386 0:1.0.1-12.fc6 elfutils.i386 0: 0.127-1.fc6 elfutils-libelf.i386 0:0.127-1.fc6 elfutils-libelf-devel.i386 0: 0.127-1.fc6 elfutils-libelf-devel-static.i386 0:0.127-1.fc6 elfutils-libs.i386 0:0.127-1.fc6 evolution-data-server.i386 0:1.8.3-6.fc6 iputils.i386 0:20070202-3.fc6 kernel-headers.i386 0:2.6.20-1.2948.fc6 krename.i386 0:3.0.14-1.fc6 lftp.i386 0:3.5.9-0.fc6 libsane-hpaio.i386 0: 1.7.2-3.fc6 libupnp.i386 0:1.4.6-1.fc6 libxml2.i386 0:2.6.28-1.fc6 libxml2-devel.i386 0:2.6.28-1.fc6 libxml2-python.i386 0:2.6.28-1.fc6 m4.i3860: 1.4.8-2 php.i386 0:5.1.6-3.5.fc6 php-cli.i386 0:5.1.6-3.5.fc6 php-common.i386 0:5.1.6-3.5.fc6 php-ldap.i386 0:5.1.6-3.5.fc6 php-mysql.i3860: 5.1.6-3.5.fc6 php-pdo.i386 0:5.1.6-3.5.fc6 policycoreutils.i386 0: 1.34.1-9.fc6 popt.i386 0:1.10.2-33.fc6 pygobject2.i386 0:2.12.3-2.fc6 python-mutagen.noarch 0:1.11-1.fc6 rpm.i386 0:4.4.2-33.fc6 rpm-build.i386 0: 4.4.2-33.fc6 rpm-devel.i386 0:4.4.2-33.fc6 rpm-libs.i386 0:4.4.2-33.fc6 rpm-python.i386 0:4.4.2-33.fc6 selinux-policy.noarch 0:2.4.6-62.fc6 selinux-policy-targeted.noarch 0:2.4.6-62.fc6 smartmontools.i386 1: 5.37-1.1.fc6 subversion.i386 0:1.4.3-2.fc6 system-config-date.noarch 0: 1.8.12-2.fc6 tar.i386 2:1.15.1-25.fc6 tcp_wrappers.i386 0:7.6-40.3.fc6 vim-common.i386 2:7.0.235-1.fc6 vim-enhanced.i386 2:7.0.235-1.fc6 vim-minimal.i386 2:7.0.235-1.fc6 xine.i386 0:0.99.5-1.fc6 xine-lib.i386 0: 1.1.6-2.fc6 xsane.i386 0:0.994-2.fc6 xsane-gimp.i386 0:0.994-2.fc6 xterm.i386 0:225-1.fc6 Complete!
On 05/08/2007 10:06 PM, Valent Turkovic wrote:
Full update gave me even better results!!! 75% savings !!!
Size of all updates downloaded from Presto-enabled repositories: 10M Size of updates that would have been downloaded if Presto wasn't enabled: 41M This is a savings of 75 percent
No one said that it would not be great to have this tool. Unfortunately it a) is still in beta stage and more important b)it arrived too late to be included in the release. Even the tools to generated the deltarpm enabled repos have just been released. It will be included in rawhide and after proper testing might be pushed as an update to FC6/FC7 too. Or at least that's the picture I got.
On Wed, 2007-05-09 at 02:09 +0300, Manuel Wolfshant wrote:
It will be included in rawhide and after proper testing might be
pushed as an update to FC6/FC7 too. Or at least that's the picture I got.
The packages for yum-presto and deltarpm certainly can be. Whether the official Fedora repositories have deltarpms enabled is a different matter.
It would be good to do this for rawhide, however that is something we'll have to evaluate post-F7. Just too much stuff to look at for the time being.
josh
On May 8, 2007, at 8:58 PM, Josh Boyer wrote:
On Wed, 2007-05-09 at 02:09 +0300, Manuel Wolfshant wrote:
It will be included in rawhide and after proper testing might be
pushed as an update to FC6/FC7 too. Or at least that's the picture I got.
The packages for yum-presto and deltarpm certainly can be. Whether the official Fedora repositories have deltarpms enabled is a different matter.
It would be good to do this for rawhide, however that is something we'll have to evaluate post-F7. Just too much stuff to look at for the time being.
Don't forget - F8 is only 6 or 7 months away. If enough people put time and effort into developing, maintaining, and testing Presto, there's no reason[1] it can't be installed by default for F8..
-w
[1] well, assuming the mirror maintainers are OK with carrying it..
On Tue, 2007-05-08 at 19:58 -0500, Josh Boyer wrote:
On Wed, 2007-05-09 at 02:09 +0300, Manuel Wolfshant wrote:
It will be included in rawhide and after proper testing might be
pushed as an update to FC6/FC7 too. Or at least that's the picture I got.
The packages for yum-presto and deltarpm certainly can be. Whether the official Fedora repositories have deltarpms enabled is a different matter.
It would be good to do this for rawhide, however that is something we'll have to evaluate post-F7. Just too much stuff to look at for the time being.
josh
+1
Just want to note that both yum-presto and deltarpm are already in extras for FC6 and in F7, though I don't think yum-presto will be in any spins.
Jonathan
On Thu, 2007-04-12 at 14:37 -0400, Jesse Keating wrote: <snip>
Finally, when we do get to the point of "do we put deltarpms in the master mirror", *if* we decide *not* to (for the many reasons mentioned by those against the idea), is it possible for yum-presto in Extras to point to the test server in the default .conf file?
Typically we don't allow packages other than fedora-release to add repo files. Maybe we could grant an exception for this package for a temporary time being, but once a repo file is on, it is difficult to get it off, so wherever you point you might want to be willing to at least maintain a redirect to somewhere where content lives.
I have a proposal that I would like to run by all interested parties.
As has been covered elsewhere, FI isn't ready to start generating deltarpms for <insert name of new repository that replaces Core+Extras here>. At the moment yum-presto in Extras (for FC6 and F7) doesn't point to any Presto-enabled repositories. It does have some commented out deltaurls in presto.conf that, when uncommented, point to the test server at http://www.lesbg.com.
The "right" way to set a deltaurl for a repository in yum-presto is to put it in the repository's .repo file. ATM, you can also put the deltaurl in presto.conf, though I keep on saying that that method will be removed soon.
So, my proposal is this: I would like to be able to uncomment the deltaurls in presto.conf, even though they point to a non-Fedora Project url (yes, I know this is against policy). When (or if) FI starts generating deltarpms, I would then remove the ability to set deltaurls in presto.conf from the next version of yum-presto.
All people who upgrade to the new version of yum-presto would no longer be pointed to the test server and would instead grab the deltarpms directly from the official repositories. I would maintain a redirect on the test server pointing to the official repositories for a long period of time after the transition has been made.
I would appreciate any comments, flames, or large rocks aimed at my head.
Jonathan
On Wed, May 09, 2007 at 10:41:53AM +0300, Jonathan Dieter wrote:
So, my proposal is this: I would like to be able to uncomment the deltaurls in presto.conf, even though they point to a non-Fedora Project url (yes, I know this is against policy).
I think it is a bad idea to continue supporting deltaurls in presto.conf, even now. It is also a bad idea to have any default configuration point to non-Fedora repos.
The documentation should instruct the users to add the deltaurls to the main *.repo files themselves. This avoids the situation where non-Fedora URLs are being added automatically--the user has to do it themselves.
When (or if) FI starts generating deltarpms, I would then remove the ability to set deltaurls in presto.conf from the next version of yum-presto.
Deltaurls-in-presto.conf support should be removed ASAP.
All people who upgrade to the new version of yum-presto would no longer be pointed to the test server and would instead grab the deltarpms directly from the official repositories. I would maintain a redirect on the test server pointing to the official repositories for a long period of time after the transition has been made.
A redirect would still be fine if you would like to do that.
On Wed, May 09, 2007 at 10:41:53AM +0300, Jonathan Dieter wrote:
The "right" way to set a deltaurl for a repository in yum-presto is to put it in the repository's .repo file. ATM, you can also put the deltaurl in presto.conf, though I keep on saying that that method will be removed soon.
Would it be possible to have transparent configs, e.g. a canonical folder in the repo which presto checks for existance and uses it if available?
Ideally, if deltarpms are a success as they promise to be why touch the repofiles at all and not simply just download the plugin and ready you are?
This would also allow repo to slide in presto support w/o having to notify their users to modify repofiles etc. So today you install presto and only a couple of repos support it and automagically you get more and more repos that spare your bandwidth w/o the user doing anything more.
On Do Mai 10 2007, Axel Thimm wrote:
Would it be possible to have transparent configs, e.g. a canonical folder in the repo which presto checks for existance and uses it if available?
Afaik on presto enabled repositories, this is written in the repo metadata,
Ideally, if deltarpms are a success as they promise to be why touch the repofiles at all and not simply just download the plugin and ready you are?
so this is already the case. But as long as the delta rpms are in another repository / not mentioned in the repo metadata, manual intervention is needed.
Regards, Till
On Thu, 2007-05-10 at 00:58 +0200, Axel Thimm wrote:
Would it be possible to have transparent configs, e.g. a canonical folder in the repo which presto checks for existance and uses it if available?
<snip>
This is already implemented in yum-presto, but not yet in createprestorepo. The prestomd format is actually identical to the repomd format with a datatype of "deltas" for presto.xml.gz file. Yum-presto will look in repomd.xml for "deltas" if there is no prestomd.xml file available.
It should be quite simple to put the small amount of code in createprestorepo that is different from createrepo back into createrepo and add a -d option to createrepo that generates presto.xml.gz and adds it to repomd.xml
Jonathan
Jonathan Dieter wrote:
On Thu, 2007-05-10 at 00:58 +0200, Axel Thimm wrote:
Would it be possible to have transparent configs, e.g. a canonical folder in the repo which presto checks for existance and uses it if available?
<snip>
This is already implemented in yum-presto, but not yet in createprestorepo. The prestomd format is actually identical to the repomd format with a datatype of "deltas" for presto.xml.gz file. Yum-presto will look in repomd.xml for "deltas" if there is no prestomd.xml file available.
It should be quite simple to put the small amount of code in createprestorepo that is different from createrepo back into createrepo and add a -d option to createrepo that generates presto.xml.gz and adds it to repomd.xml
You could submit a patch to do this then and get any other relevant packages reviewed, included and then request Fedora Infrastructure team to run new create repo for the Fedora devel as soon as we move beyond Fedora 7. I don't think letting the package connect to a different repo by default is desirable. That doesn't scale.
Rahul
On Thu, 2007-05-10 at 09:50 +0530, Rahul Sundaram wrote:
You could submit a patch to do this then and get any other relevant packages reviewed, included and then request Fedora Infrastructure team to run new create repo for the Fedora devel as soon as we move beyond Fedora 7. I don't think letting the package connect to a different repo by default is desirable. That doesn't scale.
+1.
This needs to be in createrepo and/or modify the repo using modifyrepo. Contact me on tuesday (I'm out of town this week) and I'll see what we can get merged.
-sv
On Thu, 2007-05-10 at 12:39 -0400, seth vidal wrote:
+1.
This needs to be in createrepo and/or modify the repo using modifyrepo. Contact me on tuesday (I'm out of town this week) and I'll see what we can get merged.
-sv
I took a look at createrepo 0.4.8 this morning, and it looks like it does everything I wanted 0.4.4 to do, but it couldn't. Woo-hoo!
So we can either (1) have createprestorepo work alongside createrepo without any modification to createrepo or (2) add a "-d" switch to createrepo and merge the few additional steps needed to create presto.xml.gz (which isn't much).
If we're moving towards presto integration in F8, I would probably encourage (2) otherwise (1) would be easier for you. I'll contact you again on Tuesday.
Jonathan
On Thu, 10 May 2007, Jonathan Dieter wrote:
On Thu, 2007-05-10 at 12:39 -0400, seth vidal wrote:
This needs to be in createrepo and/or modify the repo using modifyrepo. Contact me on tuesday (I'm out of town this week) and I'll see what we can get merged.
I took a look at createrepo 0.4.8 this morning, and it looks like it does everything I wanted 0.4.4 to do, but it couldn't. Woo-hoo!
So we can either (1) have createprestorepo work alongside createrepo without any modification to createrepo or (2) add a "-d" switch to createrepo and merge the few additional steps needed to create presto.xml.gz (which isn't much).
If we're moving towards presto integration in F8, I would probably encourage (2) otherwise (1) would be easier for you. I'll contact you again on Tuesday.
Not that my opinion means all that much, but I (still) think this would be a great feature for F8, and don't see any reason (upstream willing) that this functionality shouldn't be merged into createrepo. I think it'd be best if the existing Fedora mirrors could offer DRPMs alongside regular RPMs, and left the preference up to the client. Of course, in this ideal world, the mirrors would have all the disk space they needed, and wouldn't have to care about the extra space of the DRPMs, but...we'll see.
Jima
On Wednesday 11 April 2007 14:25:23 Till Maas wrote:
With the plugin already beeing in Fedora Extras 6, the feature is already there.
Which is kind of a shame since it doesn't work for devel and thus there is a broken feature path. This is why new things should go into devel first, and be backported/built if possible. If we set an expectation that this works in FC6 and then it stops working, perhaps badly in F7 that is a very poor user experience.
Also Nearly every new Extras package is also built for the supported releases and not only for devel, so there are always new features and functionality in stable releases.
Which is another thing I'm not exactly comfortable with. Especially for things like yum plugins which have the potential to break many things on the system in very user unfriendly ways.
And when the plugin is tested in devel, then the delta rpms and mirror layout has to be generated, too.
Not necessarily. Testing can be continued with with another test repo until the plugin and the delta generation code is more stable and less likely to break, then we can start applying it to the master mirror.
Jesse Keating wrote:
On Wednesday 11 April 2007 13:38:15 Rahul Sundaram wrote:
You don't expose them really to any additional risks by default. If the delta rpms are already mirrored, yum just ignores them without the plugin. Since the plugin is already in Fedora Extras users just would have install them explicitly to use the delta rpms. The plugin falls back to downloading the full rpm if the delta rpm fails or is not available.
It is still not something I want to do to a released product, or even Fedora 7 after the feature freeze. The generation of the delta rpms, the layout, the use of the plugin, etc.. these are all codepaths that need more exposure/testing during an open development phase. I think it's great that you guys have gotten it to the point it is now, and I really look forward to seeing it get wider use once we start up Fedora 8. I just don't want to add new features/functionality into 7 and 6.
Agreed that it is too late to enable DeltaRPM by default in Fedora 7. However, the yum plugin will be available in the distro as an optional add-on.
We should begin discussion of things like: - standardize a future directory structure and location for deltarpms within the new mirror format. - where and how deltarpms are generated to land on the master mirror. - Are we satisfied with the existing metadata format for deltarpm?
After we have settled on standards for all this, then we can add production deltarpm to the mirrors for Fedora 7 users to optionally use. Perhaps by Fedora 8 we can enable the plugin by default.
Warren Togami wtogami@redhat.com
Warren Togami wrote:
Agreed that it is too late to enable DeltaRPM by default in Fedora 7. However, the yum plugin will be available in the distro as an optional add-on.
Correction: the yum plugin *IS* in the repo today.
- Are we satisfied with the existing metadata format for deltarpm?
Not just the format... but the means in which it is handled. If the deltarpms live on the same mirrors as the original packages, then shouldn't the metadata live in the same place as the original packages? Maybe even that metadata should be grabbed at the same time as the other metadata?
Warren
On Wed, Apr 11, 2007 at 12:05:43PM -0400, Jesse Keating wrote:
On Wednesday 11 April 2007 11:55:13 Rahul Sundaram wrote:
There are obviously some fixes that are required at the client level: yum presto plugin. However we can encourage the mirrors to presto enable all the repositories right away.
Well, we don't ask the mirrors to generate things, we generate things at the source and the mirrors just pick it up. If the mirrors have to do something themselves, we've lost.
Since the client is in extras users can choose to install it for FC5, FC6 and rawhide (this one is esp interesting right now) and we should look at installing the plugin by default in F8.
I really don't want to add this midstream to a released product. The right process here is to get it working in rawhide during the open development cycle, start producing the content for it at the source, have it mirrored out and get more and more people beating on it during rawhide. If it survives and makes it into the released product as a default, that's awesome, from that point on we use it. Trying to start at a released product and port forward just seems backwards to me, and not something I want to expose all of our "stable" Fedora 6 users to.
It is great no doubt and I have been using it for a few days now. Updating is no more that big a drain on the bandwidth. Here an excerpt:
-------------------------------------------------------------------------- release 100% |=========================| 951 B 00:00 Loading mirror speeds from cached hostfile Setting up Presto extras 100% |=========================| 383 B 00:00 updates 100% |=========================| 383 B 00:00 Reading Presto metadata in from local files presto.xml.gz 100% |=========================| 54 kB 00:02 Reading repository metadata in from local files primary.xml.gz 100% |=========================| 1.6 MB 01:18 extras : ################################################## 5139/5139 <snip> Size of all updates downloaded from Presto-enabled repositories: 156K Size of updates that would have been downloaded if Presto wasn't enabled: 7.3M This is a savings of 98 percent
Installed: tomcat4-servlet-2.3-api.noarch 0:4.1.31-7jpp Updated: blobwars.i386 0:1.06-1.fc6 gaim-encryption.i386 0:3.0-0.2.beta8.fc6 Replaced: servletapi4.noarch 0:4.0.4-4jpp Complete! [root@fc6host ~]# --------------------------------------------------------------------------
Is it possible to produce delta primary.xml files just as delta rpms?
The situation has changed now to xml files being bigger downloads than the packages themselves.
Regards!
On Thu, Apr 12, 2007 at 07:36:16PM +0530, Vikram Goyal wrote:
Is it possible to produce delta primary.xml files just as delta rpms?
The situation has changed now to xml files being bigger downloads than the packages themselves.
Have a look at the zsync program, maybe it can be used for that purpose. zsync works like rsync, but it run's the delta algorithm on the client, thus it doesn't need a special server.
Cheers, Michael.
On Wed, 2007-04-11 at 11:39 -0400, Warren Togami wrote:
I wonder if it is time to discuss whether we want deltarpm's to be distributed by default on the mirrors, or to enable it by default in Fedora's yum.
Perhaps it is too late to do this for Fedora 7, and adding it to the mirrors might be best to do when the mirror layout changes for the merged distribution.
It's too late for F7.
-sv
On Wed, Apr 11, 2007 at 08:46:30AM +0200, Valent Turkovic wrote:
RPM build speed is also much better that SUSE updater! Bravo for yum-presto team!
What's rpm build speed? If you mean the applydeltarpm time, it's because SUSE uses bzip2 for payload compression, which is much slower than gzip.
Cheers, Michael.
Michael Schroeder wrote:
On Wed, Apr 11, 2007 at 08:46:30AM +0200, Valent Turkovic wrote:
RPM build speed is also much better that SUSE updater! Bravo for yum-presto team!
What's rpm build speed? If you mean the applydeltarpm time, it's because SUSE uses bzip2 for payload compression, which is much slower than gzip.
That probably does explain the speed difference. Looks like we are trading in disk space for better performance. Why was bzip2 compression chosen in SUSE? How much disk space do you save on disk by using bzip2?
Rahul
On Wed, Apr 11, 2007 at 04:09:37PM +0530, Rahul Sundaram wrote:
That probably does explain the speed difference. Looks like we are trading in disk space for better performance. Why was bzip2 compression chosen in SUSE?
To squeeze a couple of packages more on the CDs.
How much disk space do you save on disk by using bzip2?
I think about 10%. But in retrospect it IMHO was a bad decission, bzip2 is much to slow. I hope things will be better if we switch to new compression schemes like 7zip.
Cheers, Michael.