Shyam said the following on 06/17/2008 12:10 PM Pacific Time: > I've been using Fedora since fedora 6 and it is good to see improvements > in yum. When compared with ubuntu in the packages which is around > 25000 and fedora which has around 10000, the time taken to download the > package list during the refresh in ubuntu takes lesser time than Fedora. > Is it possible to speed this process.? > > -- > Shyam > Just out of curiosity... exactly what is the time different downloading and installing the same package set on Fedora and Ubuntu?

From: Patrick O'Callaghan <pocallaghan(a)gmail.com&gt; Subject: Re: Is it possible to speed up yum. To: fedora-test-list(a)redhat.com Date: Tuesday, June 17, 2008, 10:11 AM On Tue, 2008-06-17 at 12:18 -0400, John Poelstra wrote: > Shyam said the following on 06/17/2008 12:10 PM Pacific Time: > > I've been using Fedora since fedora 6 and it is good to see improvements > > in yum. When compared with ubuntu in the packages which is around > > 25000 and fedora which has around 10000, the time taken to download the > > package list during the refresh in ubuntu takes lesser time than Fedora. > > Is it possible to speed this process.? > > > > -- > > Shyam > > > > Just out of curiosity... exactly what is the time different downloading > and installing the same package set on Fedora and Ubuntu? I don't know that it makes much sense to compare download times since in general the packages are coming from different servers (and aren't even the same sizes). What I do notice is that apt-get and friends on Ubuntu are much faster at resolving dependencies, so it may be something about the database implementation. For example if I do a "yum update" and a minute later do another one, yum takes a while to tell there's nothing to be done (even when using the cache), whereas apt-get is almost instantaneous. Here's a quick comparison. Both machines are Intel Core 2 Duos with 2GB of RAM. One is Fedora 9, the other is Ubuntu Gutsy. Both are up to date with their respective repos, so no network activity is going on here: Fedora: # time yum update Loaded plugins: fastestmirror, protectbase, refresh-packagekit Loading mirror speeds from cached hostfile * livna-development: mirror.atrpms.net * livna: mirror.atrpms.net * google: dl.google.com * fedora: fedora.c3sl.ufpr.br * adobe-linux-i386: linuxdownload.adobe.com * localrepo: * updates: ftp.usf.edu 0 packages excluded due to repository protections Setting up Update Process No Packages marked for Update real 0m5.376s user 0m3.611s sys 0m0.278s Gutsy: $ time sudo apt-get upgrade Reading package lists... Done Building dependency tree Reading state information... Done 0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded. real 0m0.430s user 0m0.420s sys 0m0.008s poc --

Antonio Olivares said the following on 06/17/2008 01:37 PM Pacific Time: > Anyway, the behavior is hard to determine why things are the way they are. The size of the packages matters, also the servers from where the packages are fetched is also important, which plugins are present in yum for instance, fastest mirror, presto, etc. A one-on-one comparison does not say much since other factors kick into the equation. > Right. I was just curious as I've seen this said a lot, but little data to back up the assertion except that "apt-get/ubuntu is faster". I think someone in Fedora pointed out that in some comparisons the difference was only a few seconds which hardly seemed worth getting excited about.

Read my post again. I deliberately set this up so that neither machine was fetching new information from its repos, so network considerations do not enter into it. Obviously a proper benchmark comparison would need to ensure that both machines have essentially the same set of packages installed and I didn't bother doing that, but the wallclock time difference in this specific example is more than 10 to 1 and I can assure you that the Fedora machine does not have even twice as many packages as the Ubuntu one. Does this cosmically speaking matter? I've no idea. I've no agenda to promote apt-get over yum or vice versa. It's just a data point. poc --

This topic is neverending story. Yes yum is feeling slower compare to apt but just try to compare its capabilities.

Also try to run "yum -d 10" to see yum timing.

On Tue, 2008-06-17 at 12:18 -0400, John Poelstra wrote: > Shyam said the following on 06/17/2008 12:10 PM Pacific Time: > > I've been using Fedora since fedora 6 and it is good to see improvements > > in yum. When compared with ubuntu in the packages which is around > > 25000 and fedora which has around 10000, the time taken to download the > > package list during the refresh in ubuntu takes lesser time than Fedora. > > Is it possible to speed this process.? > > > > -- > > Shyam > > > > Just out of curiosity... exactly what is the time different downloading > and installing the same package set on Fedora and Ubuntu? I don't know that it makes much sense to compare download times since in general the packages are coming from different servers (and aren't even the same sizes). What I do notice is that apt-get and friends on Ubuntu are much faster at resolving dependencies, so it may be something about the database implementation. For example if I do a "yum update" and a minute later do another one, yum takes a while to tell there's nothing to be done (even when using the cache), whereas apt-get is almost instantaneous. Here's a quick comparison. Both machines are Intel Core 2 Duos with 2GB of RAM. One is Fedora 9, the other is Ubuntu Gutsy. Both are up to date with their respective repos, so no network activity is going on here: Fedora: # time yum update Loaded plugins: fastestmirror, protectbase, refresh-packagekit Loading mirror speeds from cached hostfile * livna-development: mirror.atrpms.net * livna: mirror.atrpms.net * google: dl.google.com * fedora: fedora.c3sl.ufpr.br * adobe-linux-i386: linuxdownload.adobe.com * localrepo: * updates: ftp.usf.edu 0 packages excluded due to repository protections Setting up Update Process No Packages marked for Update real 0m5.376s user 0m3.611s sys 0m0.278s Gutsy: $ time sudo apt-get upgrade

It doesn't. It refreshes it if metadata_expire seconds (defined in /etc/yum.conf) have elapsed since the last time it was run. Since I deliberately updated both systems just before running the test, there was no network activity. This is explained in the post.

> > and send the output along. # yum -d3 info yum | grep 'time:' Config time: 0.182 rpmdb time: 0.000 pkgsack time: 7.505

These runs were one after the other in quick succession. What's notable is the large variance in pkgsack time.

On Wed, 2008-06-18 at 23:43 -0400, seth vidal wrote: > On Wed, 2008-06-18 at 16:19 -0430, Patrick O'Callaghan wrote: > > > > > > and send the output along. > > > > # yum -d3 info yum | grep 'time:' > > Config time: 0.182 > > rpmdb time: 0.000 > > pkgsack time: 7.505 > > It's taking 7.5 seconds to put the package sack together and it's not > downloading ANYTHING? Not even the mirrorlists again or rechecking the > repomd.xml? Yes. > I ask b/c the same version on my not-exceptionally-fast laptop is: > > yum -d3 info yum | grep 'time:' > Config time: 0.106 > rpmdb time: 0.000 > pkgsack time: 0.345 Interesting. I ran it again a few times, first doing an update: [root@bree S3]# yum update Loaded plugins: fastestmirror, protectbase, refresh-packagekit Loading mirror speeds from cached hostfile * livna-development: mirror.atrpms.net * livna: mirror.atrpms.net * google: dl.google.com * fedora: ftp.usf.edu * adobe-linux-i386: linuxdownload.adobe.com * localrepo: * updates: ftp.usf.edu 0 packages excluded due to repository protections Setting up Update Process No Packages marked for Update [root@bree S3]# yum -d3 info yum | grep 'time:' Config time: 0.163 rpmdb time: 0.000 pkgsack time: 2.526 [root@bree S3]# yum -d3 info yum | grep 'time:' Config time: 0.168 rpmdb time: 0.000 pkgsack time: 1.690 [root@bree S3]# yum -d3 info yum | grep 'time:' Config time: 0.162 rpmdb time: 0.000 pkgsack time: 6.976 [root@bree S3]# yum -d3 info yum | grep 'time:' Config time: 0.163 rpmdb time: 0.000 pkgsack time: 1.774 [root@bree S3]# These runs were one after the other in quick succession. What's notable is the large variance in pkgsack time.

On Thu, 2008-06-19 at 06:39 -0400, seth vidal wrote: > On Thu, 2008-06-19 at 00:53 -0430, Patrick O'Callaghan wrote: > > On Wed, 2008-06-18 at 23:43 -0400, seth vidal wrote: > > > On Wed, 2008-06-18 at 16:19 -0430, Patrick O'Callaghan wrote: > > > > > > > > > > and send the output along. > > > > > > > > # yum -d3 info yum | grep 'time:' > > > > Config time: 0.182 > > > > rpmdb time: 0.000 > > > > pkgsack time: 7.505 > > > > > > It's taking 7.5 seconds to put the package sack together and it's not > > > downloading ANYTHING? Not even the mirrorlists again or rechecking the > > > repomd.xml? > > > > Yes. > > > > > I ask b/c the same version on my not-exceptionally-fast laptop is: > > > > > > yum -d3 info yum | grep 'time:' > > > Config time: 0.106 > > > rpmdb time: 0.000 > > > pkgsack time: 0.345 > > > > Interesting. > > > > I ran it again a few times, first doing an update: > > > > [root@bree S3]# yum update > > Loaded plugins: fastestmirror, protectbase, refresh-packagekit > > Loading mirror speeds from cached hostfile > > * livna-development: mirror.atrpms.net > > * livna: mirror.atrpms.net > > * google: dl.google.com > > * fedora: ftp.usf.edu > > * adobe-linux-i386: linuxdownload.adobe.com > > * localrepo: > > * updates: ftp.usf.edu > > 0 packages excluded due to repository protections > > Setting up Update Process > > No Packages marked for Update > > [root@bree S3]# yum -d3 info yum | grep 'time:' > > Config time: 0.163 > > rpmdb time: 0.000 > > pkgsack time: 2.526 > > [root@bree S3]# yum -d3 info yum | grep 'time:' > > Config time: 0.168 > > rpmdb time: 0.000 > > pkgsack time: 1.690 > > [root@bree S3]# yum -d3 info yum | grep 'time:' > > Config time: 0.162 > > rpmdb time: 0.000 > > pkgsack time: 6.976 > > [root@bree S3]# yum -d3 info yum | grep 'time:' > > Config time: 0.163 > > rpmdb time: 0.000 > > pkgsack time: 1.774 > > [root@bree S3]# > > > > These runs were one after the other in quick succession. What's notable > > is the large variance in pkgsack time. > > try it like this: > yum --noplugins -d3 info yum | grep 'time:' [root@bree S3]# yum update Loaded plugins: fastestmirror, protectbase, refresh-packagekit Loading mirror speeds from cached hostfile * livna-development: mirror.atrpms.net * livna: mirror.atrpms.net * google: dl.google.com * fedora: ftp.usf.edu * adobe-linux-i386: linuxdownload.adobe.com * localrepo: * updates: ftp.usf.edu 0 packages excluded due to repository protections Setting up Update Process No Packages marked for Update [root@bree S3]# yum --noplugins -d3 info yum | grep 'time:' Config time: 0.156 rpmdb time: 0.000 pkgsack time: 0.427 [root@bree S3]# yum --noplugins -d3 info yum | grep 'time:' Config time: 0.148 rpmdb time: 0.000 pkgsack time: 0.385 [root@bree S3]# yum --noplugins -d3 info yum | grep 'time:' Config time: 0.156 rpmdb time: 0.000 pkgsack time: 0.413 [root@bree S3]# yum --noplugins -d3 info yum | grep 'time:' Config time: 0.149 rpmdb time: 0.000 pkgsack time: 0.391 [root@bree S3]# yum --noplugins -d3 info yum | grep 'time:' Config time: 0.151 rpmdb time: 0.000 pkgsack time: 0.404 [root@bree S3]# yum --noplugins -d3 info yum | grep 'time:' Config time: 0.158 rpmdb time: 0.000 pkgsack time: 0.403 So it's clear that the large variance I saw before is being caused by one or more of the plugins. However I repeated the experiment using just one plugin at a time and the variance doesn't come back, so it looks like some combination. Just to be sure, I did it yet again with all plugins enabled, and sure enough it's still there.

The plugin that I'd point to first as having an impact pre-sack-setup is fastestmirror. If you disable onlt that one do you see it?

On Thu, 2008-06-19 at 09:46 -0430, Patrick O'Callaghan wrote: > On Thu, 2008-06-19 at 09:21 -0400, seth vidal wrote: > > The plugin that I'd point to first as having an impact pre-sack-setup > > is fastestmirror. If you disable onlt that one do you see it? > > Looks fairly conclusive I would say. > However /var/cache/yum/timedhosts.txt has only 126 entries, which > shouldn't take more than a jiffy to read and parse. May be worth a look. > okay - it looks like fastestmirror isn't doing a 'good enough' check on its mirror sorting. So on EVERY run it is rechecking those mirrors, which means connecting to those 126 mirrors and determining the response times.

So - on the original subject. With fastestmirror out of the list- how are your result times from yum?

On Thu, Jun 19, 2008 at 10:24 AM, Patrick O'Callaghan <pocallaghan(a)gmail.com&gt; wrote: > What's the cache for then? The plugin seems to have no documentation (at > least in the package) so it's hard to interpret what's going on without > reading the code. Read the Source, Luke. ;-) In addition to code, the source file provides some superb documentation (in this case).

You can't compare this way it make no sense at all, you are comparing apples and oranges. it like saying that drinking a glass of milk take 5 seconds and drinking a big bottle of water takes 50 seconds.