On Tue, July 31, 2007 10:04 am, Karl Larsen wrote: > I am back from running Umbutu which failed to live up to it's press > releases. It does have all my wanted applications and apt-get install > works faster than yum install. I guess there storage computer is closer. > The major problem is X Windows. I tried several ways to focus the > screen but nothing worked. And it was too bright. Also I was able to get > the VLC media player but no way to get the codex lib so it worked. Also > it takes over the computer. > So back to F7 and while some things do not work perhaps in a while they > will. At least the screen looks real nice. Seriously... the scope of this mailing list is not for it to be Karl Larsen's personal journal. These posts do not comply with the purpose of this mailing list. Please please please start using a blog.

- Erich

I am back from running Umbutu which failed to live up to it's press releases. It does have all my wanted applications and apt-get install works faster than yum install. I guess there storage computer is closer. The major problem is X Windows. I tried several ways to focus the screen but nothing worked. And it was too bright. Also I was able to get the VLC media player but no way to get the codex lib so it worked. Also it takes over the computer. So back to F7 and while some things do not work perhaps in a while they will. At least the screen looks real nice.

Your remarks are beyond repulsive, your behavior unconscionable, and you and the rest of the wing nuts piling on Karl do NOT speak for me. Until you guys begin to exhibit a little tolerance and practice a little common courtesy, you can put me on your list of "not your friends" too! Let it also be stated for the public record that not one of you keyboard cowards has the least standing as moderator of this list.

On Tuesday July 31 2007 3:19:21 pm Steve Searle wrote: > And there has been plenty of tolerance and politeness shown. >  I suspect a number of people felt that after the pleasant > change while Karl has been away, his return was a good > opportunity to try and keep things that way. > > Steve SS -- I've observed your behavior in this regard and I consider you one of the faction to which I directed my comments so you can count me out of your "number of people" suppositions. Get something straight here, I'm not defending Karl, I'm attacking your behavior... You're perfectly entitled to be an utter boor, and I don't control that, nor to I wish to, but, you don't speak for me.

Your remarks are beyond repulsive, your behavior unconscionable, and you and the rest of the wing nuts piling on Karl do NOT speak for me. Until you guys begin to exhibit a little tolerance and practice a little common courtesy, you can put me on your list of "not your friends" too! Let it also be stated for the public record that not one of you keyboard cowards has the least standing as moderator of this list.

> 2. Your generalizations of people who have previously > responded to this thread is highly insulting to me. how perceptive of you - that was my intention

On Tuesday July 31 2007 3:44:27 pm Erich Zigler wrote: > Your actions are as bad if not worse then the individuals you > are attacking. Your feeble attempt to put a logical face on the behavior of five or so intemperate people falls short; your attempt to equate my behavior with theirs is ludicrous. Do you behave the same way towards people face-to-face? If your answer to that is yes, I take back the keyboard coward characterization for you, in favor of something far worse...

Now, I do not deny that "In My Opinion" Karl is abusing this list as his personal diary, and "I" would really appreciate it if he didn't.

On Tuesday July 31 2007 12:13:38 pm Tony Nelson wrote: > Now that you mention it, the list has been quieter lately, > with fewer foolish noise threads, and, thinking back, it's > clearly becase /you/ were not posting. Now be quiet. This is > not your personal blog, and we are not your friends. Your remarks are beyond repulsive, your behavior unconscionable, and you and the rest of the wing nuts piling on Karl do NOT speak for me. Until you guys begin to exhibit a little tolerance and practice a little common courtesy, you can put me on your list of "not your friends" too! Let it also be stated for the public record that not one of you keyboard cowards has the least standing as moderator of this list. -- Claude Jones Levit & James, Inc. Leesburg, VA 703-771-1549 -- fedora-list mailing list fedora-list(a)redhat.com To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list -- No virus found in this incoming message. Checked by AVG Free Edition. Version: 7.5.476 / Virus Database: 269.11.0/927 - Release Date: 7/30/2007 5:02 PM

I am back from running Umbutu which failed to live up to it's press releases. It does have all my wanted applications and apt-get install works faster than yum install. I guess there storage computer is closer.

Sam Varshavchik wrote: > Nah, it's not closer. It's just that rpm is getting crappier every > year, and is long overdue for replacement. I could easily be mistaken, but AFAIK, the main difference in speed that end users notice between yum and apt is due to the fact that apt caches it's metadata. In between runs of apt-get update, calls to apt-get use the data on disk without hitting the network. With yum, the update and upgrade steps from apt-get are both done in the update.

The rpm included in Fedora hasn't really changed much in the past few years, so I'm not sure how it would be getting crappier[1]. Care to elaborate?

[1] rpm is finally getting some love and updates in rawhide.

> I don't know if you've ever upgraded Fedora from one release > to the next. The upgrade process is as slow as molasses, > even though all the metadata is right there. Do you know just why an upgrade of a system 6 months old, or more, takes longer than a fresh install of a new release? You should study that situation. Start with package dependencies and then think about just what you might have changed and added from third party sites. Then think some more. Care for a really stupid example? Take a 2006 automobile. Examine it very closely. Then with a garage full of new 2007 parts make it a 2007 automobile. All the time making sure that everything fits and still works. Email us when you're finished.

The new kid on the block is conary as used by rPath which claims to have some big advantages (rollbacks, binary diffs, sandboxes etc.) http://www.linux.com/articles/60500 I don't have any experience with it, but conceptually you want to do exactly the same things as a version control system so it makes sense to use something that provides those functions.

David Boles writes: > on 7/31/2007 7:03 PM, Sam Varshavchik wrote: >> Todd Zullinger writes: >> >>> Sam Varshavchik wrote: >>>> Nah, it's not closer. It's just that rpm is getting crappier every >>>> year, and is long overdue for replacement. >>> I could easily be mistaken, but AFAIK, the main difference in speed >>> that end users notice between yum and apt is due to the fact that apt >>> caches it's metadata. In between runs of apt-get update, calls to >>> apt-get use the data on disk without hitting the network. With yum, >>> the update and upgrade steps from apt-get are both done in the update. >> I don't know if you've ever upgraded Fedora from one release to the next. >> The upgrade process is as slow as molasses, even though all the metadata is >> right there. > > Do you know just why an upgrade of a system 6 months old, or more, takes > longer than a fresh install of a new release? You should study that > situation. Start with package dependencies and then think about just what > you might have changed and added from third party sites. Then think some more. Well, I did think. The system does not have anything beyond Fedora and Fedora Extras, plus my own RPMs. But why does it matter, anyway? Why does the presence of a foreign RPM cause such a nervous breakdown? At most it should result in an unsatisifed dependency. But why would should this result in rpm spinning its wheels, to such an extent? > Care for a really stupid example? Take a 2006 automobile. Examine it very > closely. Then with a garage full of new 2007 parts make it a 2007 > automobile. All the time making sure that everything fits and still works. > > Email us when you're finished. No matter which parts you do have in your automobile and where they came from, when you have to compare its part with a fixed list of two thousand other parts, from a reference model, it should take the exact same amount of time whether all your parts are OEM or aftermarket. It's the same number of parts in your car, whether original or replacement, after all. So why would it matter? At most, the complexity of what RPM has to do would be O(N), and it should really be O(log N). But it seems, though, that RPM's actual complexity is at least O(N^2), unscientifically. I tell you this. I mentioned before that I use my own package management tool internally to manage some homebrewed software. I have a compatilibity shim that sucks out pretty much the entire contents of the system RPM database, and imports all of the dependencies into my internal package database. This is to allow my own packages, which might have, say, a dependency on something.so, have the dependencies satisified by an RPM. Basically, I read all RPM resources, and create a dummy package that provides those resources, then install the dummy package, so my internal package database contains all the RPM-provided resources. Each time I update some RPMs, I rerun the import script and upgrade the old dummy RPM compatibility package to a new one. This operation, you understand of course, is analogous to your example -- taking an old snapshot of the entire RPM database, comparing it to a new one, and reconciling any differences against resources required by my internal packages, to make sure that they don't break. This operation is also equivalent, to what Anaconda has to do when it's about to upgrade the Fedora distro -- take the current RPM database, and reconcile it with the RPM database from the release you're updating to. It takes me, oh, maybe a minute or so to crunch everything together. The analogous step in Anaconda -- "Preparing transaction" -- takes aout 5-10 minutes. And I actually have more work to do. RPM has, I believe, three resources classes to reconcile against each other -- provided resources, required resources, and conflicting resources. My internal package database has six resource classes to reconcile, so I actually have more work to do. The performance degradation that I see in Anaconda is far more pronounced on less-robust hardware. On my less-than one year old laptop, with a fairly speedy Pentium, and 2 gigs of RAM, Anaconda is about 2-3 times slower than my homegrown code. On an old box that I have, running a pair of decade-old (approx) 500 Mhz Celerons, with 256MB RAM, rpm is dreadfully slow -- about 10-15 times slower than my homegrown code. There's something terribly inefficient in the way that Anaconda goes about its business. It should /not/ take that long to do its duty. Some of it might be due to Anaconda being Python code, and my homegrown code being C++.

On Tue, 2007-07-31 at 22:03 -0400, Sam Varshavchik wrote: > With a large repository, like Fedora, even a compressed XML file is > going to end up being rather huge. Then, you have to uncompress it and > parse it. And, XML parsing is also not exactly a light task. I thought there was supposed to be a move towards using a proper database scheme, a long time ago, that would have sped up all of that?

Sam Varshavchik: >>> With a large repository, like Fedora, even a compressed XML file is >>> going to end up being rather huge. Then, you have to uncompress it >>> and >>> parse it. And, XML parsing is also not exactly a light task. I was under the impression that part of the reason for using something SQL based (see listing, below) was to do with it being faster to parse than the rather free-for-all structure of an XML file. Supposedly being able to use a pre-existing databasing technique, rather than a custom job on this special XML?

Sam Varshavchik: >>> With a large repository, like Fedora, even a compressed XML file is >>> going to end up being rather huge. Then, you have to uncompress it >>> and >>> parse it. And, XML parsing is also not exactly a light task. I was under the impression that part of the reason for using something SQL based (see listing, below) was to do with it being faster to parse than the rather free-for-all structure of an XML file. Supposedly being able to use a pre-existing databasing technique, rather than a custom job on this special XML?

Thank you for the detailed reply Sam. :) Excuse me for asking more questions (potentially dumb questions at that). Sam Varshavchik wrote: > I don't know if you've ever upgraded Fedora from one release to the > next. The upgrade process is as slow as molasses, even though all > the metadata is right there. No, I avoid upgrades. I always do fresh installs as a matter of habit. Point taken though. I have read a lot of complaints of slow upgrades at the dependency resolution stage. > A few years ago the base distro was much smaller than it it now. The > size of a typical Linux distro has really balooned. Some of the > algorithms in rpm scale horribly. It wasn't such a big deal when a > typical linux distro was only a few hundred packages, but now it's a > few thousand packages, with dependencies that are much more > complicated, and rpm is now really blowing apart at the seams. I haven't looked at the code, but is it rpm or yum that's really bogging down? Or aren't you making much of a distinction when you say rpm?

> Furthermore, rpm, as is, does not implement remote repositories. Does it need to? Does dpkg do this? > With a large repository, like Fedora, even a compressed XML file is > going to end up being rather huge. Then, you have to uncompress it > and parse it. And, XML parsing is also not exactly a light task. But somehow or another you need to deal with a sizable chunk of data to make reasonable decisions regarding dependencies. The tough part about rpm development is trying to be backward compatible and still make forward progress. I don't envy the guys hacking on rpm.

From what I see yum is doing, it download the primary, the other file, and

> Remote package repositories could've been implemented much better. > When I had some free time some time ago, I quickly hacked up a > package manager for some of my internally-developed software. I > found that I could do similar kind of package metadata > synchronization much more efficiently than yum/rpm. Isn't the harder part doing this in a way that doesn't completely break backward compatibility though? And then you have to spend a bunch of years adding new code to deal with the odd sorts of deps that packagers come up with in the wild (versioned obsoletes on a multilib system sounds fun :). Someone posted to the fedora-devel list a month or so ago saying they'd created a super fast depsolver using php and mysql. Once all of the various cases they'd missed were explained, things didn't go much further. (And no, I'm not at all suggesting that applies to your work -- it's obvious that you know more than that and that you actually created a working system. :)

> metadata file you want to download, you can use HTTP 1.1 partial > chunk request feature to download just the bits of the metadata file > that you want. Perhaps you should bribe someone to implement this in yum as a proof of concept?

> But then, after all is said and done, no amount of tweaks to rpm can > compensate for stupid and broken packaging. Right now, due to > indirect dependencies, grub requires *GTK* runtime libraries to be > installed. On my headless machine, I now have to plop down a > crapload of x.org and GTK RPMs, because grub requires them, due to > its intermediate dependencies. Yeah. This was caused by policy more than by incompetence. The folks at Red Hat's legal department asked that all of the trademarked logos be kept in one package, for easier tracking and removal by downstream users of Fedora's packages (or something like that).

The annoying parts for me are that after finding the things I want, yum repeats the metadata download for every install step and goes out of its way to avoid allowing the site proxy cache from supplying it or the rpms.

Sam Varshavchik wrote: > You do /not/ need that much info in the first step. My first step is usually a 'yum search'.

> All you need is a > just a list of names of packages available on the remote repository. You > reconcile that against the list of packages you already have downloaded > the metadata for, and you then know what's new. The annoying parts for me are that after finding the things I want, yum repeats the metadata download for every install step and goes out of its way to avoid allowing the site proxy cache from supplying it or the rpms.

Tony Nelson wrote:

> Yum defaults to spreading the load > over all the mirrors by randomly choosing a mirror each time,

> but it can be > configured to use a baseurl list instead, which allows you to use the same > mirror each time, with fallbacks when that mirror is not available. Please translate that into the commands that people served by the same proxy should use to update their machines, keeping in mind that they may not know about each other or have anything else in common. If that's not possible, translate it into what one user has to do to update 2 machines, keeping in mind that yum was supposed to make things easier.

> Ditto for the epoch hack -- my solution fixes the original > underlying reason for having an epoch in the first place. Eh? So how do you handle the sometimes retarded versioning schemes of upstream sources? Or the occasional need to push an older version of something as an update?

Sam Varshavchik wrote: > > You have to break the built-in assumption that the newer version of a > package always carries a later version number. The whole reason for the > epoch is the automatic assumption regarding version and release numbers. Wouldn't that mean you'd have to keep and parse all the metatdata for all versions of packages in a repository which can contain many updates of every package instead of only considering the highest version package name when a version number is not specified in the command?

> From what I see yum is doing, it download the primary, the other file, > and possibly filelists, /every/ time a single package gets added to the repository. Even though 99% of the content is the same as before. This, in my opinion, does not really such an optimum design to me. You should /not/ have to download /everything/ every time a single package changes.

>> From what I see yum is doing, it download the primary, the other file, >> and > possibly filelists, /every/ time a single package gets added to the > repository. Even though 99% of the content is the same as before. > > This, in my opinion, does not really such an optimum design to me. You > should /not/ have to download /everything/ every time a single package > changes. Out of interest, how are these files downloaded ? Would it be possible to use something like rsync to avoid downloading the whole file, and only get the 'bit' that changed ? For me this is the only bit of yum I still find a little annoying. The new sqlite format may be bigger, but it is much faster to parse. If the download time could be improved, then for me yum wouldn't be OK.

I don't know if you've ever upgraded Fedora from one release to the next.

The upgrade process is as slow as molasses, even though all the metadata is right there.

I am back from running Umbutu which failed to live up to it's press releases. It does have all my wanted applications and apt-get install works faster than yum install. I guess there storage computer is closer. The major problem is X Windows. I tried several ways to focus the screen but nothing worked. And it was too bright. Also I was able to get the VLC media player but no way to get the codex lib so it worked. Also it takes over the computer. So back to F7 and while some things do not work perhaps in a while they will. At least the screen looks real nice.

I am back from running Umbutu which failed to live up to it's press releases. It does have all my wanted applications and apt-get install works faster than yum install. I guess there storage computer is closer. The major problem is X Windows. I tried several ways to focus the screen but nothing worked. And it was too bright. Also I was able to get the VLC media player but no way to get the codex lib so it worked. Also it takes over the computer. So back to F7 and while some things do not work perhaps in a while they will. At least the screen looks real nice.

and I did notice that their repos tended to lag behind the release curve after some time - security patches would come through, but many other packages would remain with older versions for much longer periods than with other distros.