imgcreate/yuminst.py
by Jeremy Katz
imgcreate/yuminst.py | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
New commits:
commit ce58f625e8215cface69ddc7f12cfbded0e3c66f
Author: Jeremy Katz <katzj(a)redhat.com>
Date: Mon Feb 11 14:16:37 2008 -0500
Ensure removing packages (-foo) really removes them
-foo works as long as the package is a regular package, but it intentionally
doesn't remove them from being pulled in for deps. We don't, though, want
them to be pulled in as a result of the conditional requirements. So
go through and remove them from the conditionals dict. Yes, this is
ugly but conditionals are generally ugly
diff --git a/imgcreate/yuminst.py b/imgcreate/yuminst.py
index d15ed2f..14f4806 100644
--- a/imgcreate/yuminst.py
+++ b/imgcreate/yuminst.py
@@ -100,7 +100,15 @@ class LiveCDYum(yum.YumBase):
txmbrs.append(p)
if len(txmbrs) > 0:
- map(lambda x: self.tsInfo.remove(x.pkgtup), txmbrs)
+ for x in txmbrs:
+ self.tsInfo.remove(x.pkgtup)
+ # we also need to remove from the conditionals
+ # dict so that things don't get pulled back in as a result
+ # of them. yes, this is ugly. conditionals should die.
+ for req, pkgs in self.tsInfo.conditionals:
+ if x in pkgs:
+ pkgs.remove(x)
+ self.tsInfo.conditionals[req] = pkgs
else:
print >> sys.stderr, "No such package %s to remove" %(pkg,)
16 years, 2 months
Woohoo! 31% boot speedup (*so far*), and 4.5MB free...
by Douglas McClendon
Disclaimer: take these claims with the proverbial grain of salt, as I
can on occasion be a bit over-optimistic or over-committal...
Second Disclaimer: read the first disclaimer again.
BUT THAT SAID...
The Bad News: my "SuperDeviceMapperCaching" technique does not (yet[1])
appear to actually significantly improve LiveCD boot speed.
The Good News: Some of the things that I did which were prerequisites of
SDMC, appear to have the potential to speed up LiveCD boot by at least
31%, and even shrink the LiveCD by 4.5MB.
The Better News: those prerequisite things are actually far less
invasive (non-invasive really) than SDMC.
Review: The SDMC idea was to use a virtualized boot of a LiveCD to
determine which files were accessed during boot, then remaster the root
filesystem so that those files would be grouped together, so that they
could be read into a portion of the rootfs that would temporarily live
in ram, via a devicemapper linear combination of the 'cached/preloaded'
portion, and the normal rest of the rootfs on CDROM.
The Reality: For expediency, I opted to gather the list of files via a
simple find atime command after a virtual boot settled. I then sorted
the files (and perhaps more importantly, directories and symlinks) by
size, and then populated a filesystem, starting with the smallest
boot-accessed file, and working my way up to the largest boot accessed
file, and then just doing the rest.
The realization: At some point I realized that this was actually
gathering the files at inner part of the cdrom, which is read half as
fast as the outter part.
The sick future: I will get those files pushed to the outter edge... no
matter what sick devicemapper magic it takes ;)
The already optimistic numbers, that I really hope I'm not
misinterpreting or overhyping-
I timed a fedora-8 livecd boot on my laptop.
I got about 2:30s to gdm, 3:30s to gnome desktop fully rendered, and
3:50s to firefox launched and fully rendered.
I then (slight lie, did this last), timed a build of my very fedora-like
livecd, built with my VirOS tools, but for all practical purposes, in no
way radically different than the fedora livecd.
What I got was 2:06s to the first post rhgb cursor (I'm using gdm
autologin, so this is either a gdm mouse arrow, or the gnome mouse
arrow). Then 3:19s to gnome fully rendered, and 3:23s to firefox fully
rendered. (firefox was autostarted and fighting with gnome a bit).
Then, I tried my SDMC livecd, but without SDMC, just the file
reordering- and got 1:48s to first gdm mouse arrow, 2:29s to gnome fully
rendered, and 2:34s to firefox fully rendered.
I think I was able to get my SDMC mechanism to shave 2-5 seconds off of
that, but not really a terribly significant difference.
Another interesting fact, is that the reordered filesystem was 4.5MB
smaller than unordered. This is no doubt because when I sort those
files by size, a lot of very similar files end up spatially closer
together, and are thus better compressed by squashfs.
Thus, what I tenatively see, beyond the 4.5M of free space, is a 31%
speedup (203s/154s)
So, going forward, I plan to
[1]A) do some sick stuff to get those early files on the outter rim of
the cdrom, which I think may get further significant gains by itself,
and perhaps even push my SDMC method into the significantly profitable
area (though I don't much care at this point).
B) perform all of this semi-manually on the official F8-livecd, and see
how much I can really improve 'the real deal'.
C) do boot file access data gathering on a pair of dissimilar physical
hosts, and use the file diffs to determine classes of files (hw drivers
mostly) which should get added to the list. Probably /lib/modules and a
few other obvious directories should just get thrown into the list.
D) I don't really care enough at this point to do a full systemtap or
similar boot profile. atime seems sufficient for now (though I wonder,
do I miss a lot of files that get touched when root is still mounted
readonly?)
E) do all of this for ubuntu, where I think my SDMC techniques, combined
with their native squashfs usage, will yield as much, or perhaps even
more benefit.
-dmc/jdog
Disclaimer: I really hope this all doesn't turn out to be fruitless...
but I'm doing a pretty good job convincing myself that it all makes
sense... (I'm just disappointed that I failed to really eliminate 95%
of all seeks during boot... I still think that might be possible... but
I need to go write a little C program to visualize the population of a
sparse file... or instrument some filesystems.... Nah... I'm pretty
sure I've got lower hanging fruit within my grasp...)
P.S.- When I say I 'did' all of this, I of course mean completely
scripted from a single VirOS commandline that just takes nearly half a
day to run :)
16 years, 2 months
2 commits - imgcreate/creator.py imgcreate/kickstart.py tools/mayflower
by Jeremy Katz
imgcreate/creator.py | 4 ++--
imgcreate/kickstart.py | 6 ++++++
tools/mayflower | 2 +-
3 files changed, 9 insertions(+), 3 deletions(-)
New commits:
commit ce1a7bb55abb8ae9c04351555bb28edc9e6c6285
Author: Jeremy Katz <katzj(a)redhat.com>
Date: Thu Feb 7 01:02:32 2008 -0500
Don't explicitly specify the filesystem type to mount so that ext2 can
get mounted as well
diff --git a/tools/mayflower b/tools/mayflower
index f77b2be..2e46098 100755
--- a/tools/mayflower
+++ b/tools/mayflower
@@ -609,7 +609,7 @@ do_live_from_base_loop() {
rm -f /dev/root
ln -s /dev/mapper/live-rw /dev/root
- mount -n -t ext3 /dev/mapper/live-rw /sysroot
+ mount -n /dev/mapper/live-rw /sysroot
# here you can modify the rw ext3 fs for testing if you don't want to
# respin the entire rootfs (which takes ages). Example
#
commit 4f761b21c6ab98a2421d7706c12860c55c26fa69
Author: Jeremy Katz <katzj(a)redhat.com>
Date: Thu Feb 7 01:02:09 2008 -0500
Actually follow the specified filesystem from the ks.cfg
diff --git a/imgcreate/creator.py b/imgcreate/creator.py
index b1766b6..40dcead 100644
--- a/imgcreate/creator.py
+++ b/imgcreate/creator.py
@@ -206,7 +206,7 @@ class ImageCreator(object):
A sensible default implementation is provided.
"""
- s = "/dev/root / ext3 defaults,noatime 0 0\n"
+ s = "/dev/root / %s defaults,noatime 0 0\n" %(self._fstype)
s += "devpts /dev/pts devpts gid=5,mode=620 0 0\n"
s += "tmpfs /dev/shm tmpfs defaults 0 0\n"
s += "proc /proc proc defaults 0 0\n"
@@ -697,7 +697,7 @@ class LoopImageCreator(ImageCreator):
self.__minsize_KB = 0
self.__blocksize = 4096
- self.__fstype = "ext3"
+ self.__fstype = kickstart.get_image_fstype(self.ks, "ext3")
self.__instloop = None
self.__imgdir = None
diff --git a/imgcreate/kickstart.py b/imgcreate/kickstart.py
index 4c6f48f..a7e0723 100644
--- a/imgcreate/kickstart.py
+++ b/imgcreate/kickstart.py
@@ -396,6 +396,12 @@ def get_image_size(ks, default = None):
return int(p.size) * 1024L * 1024L
return default
+def get_image_fstype(ks, default = None):
+ for p in ks.handler.partition.partitions:
+ if p.mountpoint == "/" and p.fstype:
+ return p.fstype
+ return default
+
def get_modules(ks):
devices = []
if isinstance(ks.handler.device, kscommands.device.FC3_Device):
16 years, 2 months
Re: [Fedora-livecd-list] live fedora 8 cd - login: no shell: Permission denied
by ltx＠charter.net
Yeah, I can't seem to find any docs that mention what the
password is.
I did put the rootpw line in and it does appear that the
log in works since I get the 'Last Login' message and
I get the message from /etc/motd, but then it complains
that there is 'no shell: Permission denied'
Jerry
Tim Wood wrote:
> Good question that I don't have the answer to. Since I always forget, I
> started overiding it by adding this to the kickstart:
>
> # Set root password
> rootpw iamroot
>
> If it's not a space issue, modify /etc/inittab during post and change
> the runlevel from 5 to 3. If /etc/inittab doesn't make sense to you,
> post again and I'll copy and paste the particular line.
>
> Tim
>
>
>
> ltx wrote:
>>
>> Hi Tim,
>>
>> Thanks for the quick reply. I was hoping to avoid all the
>> graphics since this is more for utility work and will be
>> booted often.
>>
>> Even if I went to the KDE desktop kickstart file what would
>> the root password be?
>>
>> Thanks.
>> Jerry
>>
>> Tim Wood wrote:
>>> One 'feature' of the minimal is that login is disabled. The kde
>>> desktop is a good starter.
>>>
>>> Tim Wood
>>>
>>>
>>> ltx wrote:
>>>>
>>>>
>>>> Hi,
>>>>
>>>> I'm trying to build a basic Fedora 8 livecd to use as
>>>> a vehicle to flash a card I am working on. The flash
>>>> utility works from linux running from the hard drive
>>>> so that is not a concern at the moment.
>>>>
>>>> My problem - I can't log in! The iso builds fine with
>>>> command;
>>>>
>>>> livecd-creator --config=./livecd-fedora-minimal.ks --fslabel=Fedora8
>>>>
>>>> When I boot from it, or use qemu to test, I can't log into root
>>>> or another id that I create in the kiskstart file. I followed
>>>> the instructions at
>>>> http://fedoraproject.org/wiki/FedoraLiveCD/LiveCDHowTo
>>>> which are great, but they even point out that the
>>>> livecd-fedora-minimal.ks
>>>> kickstart file found in /usr/share/livecd-tools will not let you log
>>>> in.
>>>> (unless you go to all full desktop environment - which I don't
>>>> need/want)
>>>>
>>>> I noticed that the minimal kickstart file disabled the root id,
>>>> so I removed that and scoured the net and found that kickstart command
>>>> rootpw allows you to set the root password. That gave me some
>>>> progress. Now when I try to log in to root I get these messages;
>>>>
>>>> Last login: time date...
>>>> Welcome to my world (this is what I put in /etc/motd)
>>>> login: no shell: Permission denied.
>>>>
>>>> So, several questions:
>>>>
>>>> 1 - What is wrong with my kickstart file (below) that prevents
>>>> me from logging in?
>>>>
>>>> 2 - Is there any collections of kickstart files available (other than
>>>> those that come with the livecd-tools packages?
>>>>
>>>> 3 - Where can I find the kickstart file options documented?
>>>>
>>>> Oh, my build environment is Fedora 8 (x86_64) with all the latest
>>>> updates. I point to the x86 mirrors so I can use the CD on older
>>>> machines.
>>>>
>>>> I'd appreciate any help you can offer.
>>>>
>>>> Thanks.
>>>> Jerry
>>>>
>>>>
>>>> lang en_US.UTF-8
>>>> keyboard us
>>>> timezone US/Eastern
>>>> #auth --useshadow --enablemd5
>>>> selinux --disabled
>>>> firewall --disabled
>>>> firstboot --disable
>>>> #root password
>>>> rootpw emulex
>>>> part / --size 1024
>>>>
>>>> repo --name=released
>>>> --mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=fedora-8&arch=i386
>>>>
>>>> repo --name=updates
>>>> --mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=updates-released-f8&arch=i386
>>>>
>>>>
>>>> %packages
>>>> @core
>>>> bash
>>>> kernel
>>>> passwd
>>>> policycoreutils
>>>> chkconfig
>>>> authconfig
>>>> rootfiles
>>>>
>>>> %post
>>>> # FIXME: it'd be better to get this installed from a package
>>>> cat > /etc/rc.d/init.d/fedora-live << EOF
>>>> #!/bin/bash
>>>> #
>>>> # live: Init script for live image
>>>> #
>>>> # chkconfig: 345 00 99
>>>> # description: Init script for live image.
>>>>
>>>> . /etc/init.d/functions
>>>>
>>>> if ! strstr "\`cat /proc/cmdline\`" liveimg || [ "\$1" != "start" ]
>>>> || [ -e /.liveimg-configured ] ; then
>>>> exit 0
>>>> fi
>>>>
>>>> exists() {
>>>> which \$1 >/dev/null 2>&1 || return
>>>> \$*
>>>> }
>>>>
>>>> touch /.liveimg-configured
>>>>
>>>> # mount live image
>>>> if [ -b /dev/live ]; then
>>>> mkdir -p /mnt/live
>>>> mount -o ro /dev/live /mnt/live
>>>> fi
>>>>
>>>> # add a user
>>>> useradd -c "Jerry" jerry
>>>> echo 'password' | passwd --stdin jerry
>>>>
>>>> # read some variables out of /proc/cmdline
>>>> for o in \`cat /proc/cmdline\` ; do
>>>> case \$o in
>>>> ks=*)
>>>> ks="\${o#ks=}"
>>>> ;;
>>>> xdriver=*)
>>>> xdriver="--set-driver=\${o#xdriver=}"
>>>> ;;
>>>> esac
>>>> done
>>>>
>>>> # Stopgap fix for RH #217966; should be fixed in HAL instead
>>>> touch /media/.hal-mtab
>>>> EOF
>>>>
>>>> chmod 755 /etc/rc.d/init.d/fedora-live
>>>> #/sbin/restorecon /etc/rc.d/init.d/fedora-live
>>>> /sbin/chkconfig --add fedora-live
>>>>
>>>> echo "Welcome to my world" > /etc/motd
>>>>
>>>> %end
>>>>
>>>>
>>>> --
>>>> Fedora-livecd-list mailing list
>>>> Fedora-livecd-list(a)redhat.com
>>>> https://www.redhat.com/mailman/listinfo/fedora-livecd-list
>>>>
>>>
>>>
>>
>
>
16 years, 2 months
liveinst failing on images made with livecd-creator
by Brian Varney
I'm having problems getting the liveinst to work with images that I create
with livecd-creator.
The problem only occurs when I make the livecd iso file. Here's the command
I'm using to create the iso.
"livecd-creator -c /usr/share/livecd-tools/livecd-fedora-8-desktop.ks"
liveinst works great until the very end when it says "performing
post-installation filesystem changes". This step never finishes. It stays
on this step forever and I've waited days. When I try to boot to the
installed image, I get a grub prompt but no menu options.
I've tried liveinst on the same system using a an iso file straight from
http://torrent.fedoraproject.org/torrents//Fedora-8-Live-i686.torrent and
liveinst completes with no issues.
Any ideas on how to get some more debug information out of liveinst to see
what is going wrong?
Thanks for any help,
Brian Varney
16 years, 2 months
Virt-p2v live CD, comments and packaging
by Richard W.M. Jones
Virt-p2v is a physical-to-virtual migration tool, for turning physical
computers into virtual machines running under some virtualization
environment like Xen.
The tool is packaged as a live CD, based on a Fedora 8 distribution, and
using livecd-creator to build the ISO.
Main page: http://et.redhat.com/~rjones/virt-p2v/
Source repo: http://hg.et.redhat.com/virt/applications/virt-p2v--devel
I'm posting here to see if anyone has any comments about the way I did
this live CD. The kickstart script uses a rather complex %post section,
which adds a few files to the filesystem, instead of building RPMs or
modifying existing RPMs.
I've also investigated how to attach stuff to the end of the ISO image,
and the way I've come up with allows me to update an ISO with a new
script quite easily, and much more quickly than waiting for
livecd-creator to rerun. This great (a) for rapid development and (b)
to send new updates to users without forcing them to download another
170 MB ISO.
I'd like to package this for Fedora. Obviously I don't want to package
the binary live CD, but instead package the kickstart file and some
tools to allow people to easily build custom live CDs. If anyone has
any ideas or examples of this I'd be interested to hear.
Thanks,
Rich.
--
Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/
Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod
Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in
England and Wales under Company Registration No. 03798903
16 years, 2 months
RPMs for RHEL 4
by Holter, Kenneth
Hi.
I need to make a Live CD for disaster recovery purposes, and found this
link on the web:
http://www.informit.com/articles/article.aspx?p=1157197
... and thought I'd give it a try.
I'm running Red Hat Enterprise Linux 4, and can't seem to find any RPMs
for my distro. Does such exist? If yes: where can I find them?
Regards,
Kenneth Holter
16 years, 2 months
Re: Review? LiveUSB persistence
by Jeremy Katz
Moving to the more appropriate list, original is at
https://www.redhat.com/archives/fedora-devel-list/2008-January/msg03188.html for those not on fedora-devel
On Thu, 2008-01-31 at 16:11 -0600, Douglas McClendon wrote:
> I keep hearing these crazy rumors that RedHat actually wants to subject
> actual users to my LiveUSB persistence feature next/this month.
Lots of people want lots of different things. I've definitely left this
sitting hanging too long and I apologize. I just didn't have time to
poke over the holidays and my January was a bitty nutty. But I should
be back to just my normal mail lag now ;-)
> I for one think the feature is really cool, useful, and know of no
> showstopping or even known bugs. But I also would not subject actual
> users to it, without having had more people review it and sign off on
> it. Anyway, to absolve myself of any responsibility, I once again
> request code review and feedback-
Overall, I think that it looks good. I think the big thing to work out
is how best to integrate the initscripts changes. Especially as we're
in parallel switching things over to upstart for F9. Just to make sure
that I'm clear about what's there so that when we try to bribe Bill, we
can do so effectively...
* The unmount loops in /etc/init.d/functions are modified so that we
don't unmount the underlying fs of the overlay
* We don't want to sync the hwclock on shutdown of a live image
* Do an overlay teardown on halt to replace the rw snapshot with a ro
version
The latter feels like we're going to want a generic hook to be able to
run, perhaps from back in the initramfs. The middle feels largely
unrelated to persistence in general, but just a "needs fixing" which
will fit in nicely with other hwclock changes that are underway in
rawhide. And the first I'm less clear on I guess.
Outside of those changes, everything looks pretty straight-forward and
reasonable. We'll want a little more error checking in
livecd-iso-to-disk and it might also be nice to have another tool that
lets you create your usb stick to use with the actual cd form (I did
this by hand and it was pretty nice that I could easily get it
working!). As for the findoverlay bits, for right now, leaving them as
a separate script is fine. In the longer term, we're hopefully going to
kill mayflower and be able to use more "normal" initrds[1], at which
point pulling it in more directly would be nice.
As far as reliability... shutting down cleanly, I can't cause a problem.
I pulled the usb stick from a box and got some garbage in a file I had
written. I want to do some more playing here, but I also wanted to get
this mail out today. Worst case, if we say "you need to shut down
cleanly", then so be it. It might be nice if we flag clean vs not
shutdown so that we can force a fsck in unclean cases. But no clue if
that's even feasible; I'll take a look, though.
Jeremy
[1] One mkinitrd to rule them all... :)
16 years, 2 months
%post and errors
by Richard W.M. Jones
Shouldn't %post (and hence livecd-creator) exit if a shell command fails?
At the moment the %post script prints an error message, but continues,
resulting in a bad ISO.
Rich.
--
Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/
Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod
Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in
England and Wales under Company Registration No. 03798903
16 years, 2 months
Making content available on both sides of the live CD
by Asheesh Laroia
I was talking earlier about putting content on the ISO to be read by e.g.
Windows users. If I want to make that content available in the live
Fedora system also, what's the best way to do that?
What I'm after is a directory on the live Fedora desktop that doesn't
waste space but shows the same content as the
There were a couple of "obvious" ways I could think of, which I'll refer
to by one would achieve them:
* mount --bind /media/cdrom/my_directory
/home/liveuser/Desktop/my_directory
* ln -s /media/cdrom/my_directory /home/liveuser/Desktop/my_directory
Both these can only expose the content read-only, which is acceptable but
less than amazing.
* have a unionfs/aufs mount that exposes the ISO's content and lets users
write using "copy-up" semantics
The downside of unionfs is that it adds complications.
Given these choices, I'd go for the symlink. Is that what you guys would
do? Is there something I'm missing?
One used to be able to make "hybrid CDs" (c.f.
http://en.wikipedia.org/wiki/Hybrid_CD ) that shared data between the HFS
filesystems and ISO9660 filesystems on them, but I imagine that's not
possible for the ISO9660 filesystem at the base of the LiveCD-Creator disc
and the squashfs inside it - right?
Thanks for the quick and sharp replies!
-- Asheesh.
--
A witty saying proves nothing.
-- Voltaire
16 years, 2 months
