3:14 p.m.
Moving to the more appropriate list, original is at
https://www.redhat.com/archives/fedora-devel-list/2008-January/msg03188.html for those not
on fedora-devel
On Thu, 2008-01-31 at 16:11 -0600, Douglas McClendon wrote:
I keep hearing these crazy rumors that RedHat actually wants to
subject
actual users to my LiveUSB persistence feature next/this month.
Lots of people want lots of different things. I've definitely left this
sitting hanging too long and I apologize. I just didn't have time to
poke over the holidays and my January was a bitty nutty. But I should
be back to just my normal mail lag now ;-)
I for one think the feature is really cool, useful, and know of no
showstopping or even known bugs. But I also would not subject actual
users to it, without having had more people review it and sign off on
it. Anyway, to absolve myself of any responsibility, I once again
request code review and feedback-
Overall, I think that it looks good. I think the big thing to work out
is how best to integrate the initscripts changes. Especially as we're
in parallel switching things over to upstart for F9. Just to make sure
that I'm clear about what's there so that when we try to bribe Bill, we
can do so effectively...
* The unmount loops in /etc/init.d/functions are modified so that we
don't unmount the underlying fs of the overlay
* We don't want to sync the hwclock on shutdown of a live image
* Do an overlay teardown on halt to replace the rw snapshot with a ro
version
The latter feels like we're going to want a generic hook to be able to
run, perhaps from back in the initramfs. The middle feels largely
unrelated to persistence in general, but just a "needs fixing" which
will fit in nicely with other hwclock changes that are underway in
rawhide. And the first I'm less clear on I guess.
Outside of those changes, everything looks pretty straight-forward and
reasonable. We'll want a little more error checking in
livecd-iso-to-disk and it might also be nice to have another tool that
lets you create your usb stick to use with the actual cd form (I did
this by hand and it was pretty nice that I could easily get it
working!). As for the findoverlay bits, for right now, leaving them as
a separate script is fine. In the longer term, we're hopefully going to
kill mayflower and be able to use more "normal" initrds[1], at which
point pulling it in more directly would be nice.
As far as reliability... shutting down cleanly, I can't cause a problem.
I pulled the usb stick from a box and got some garbage in a file I had
written. I want to do some more playing here, but I also wanted to get
this mail out today. Worst case, if we say "you need to shut down
cleanly", then so be it. It might be nice if we flag clean vs not
shutdown so that we can force a fsck in unclean cases. But no clue if
that's even feasible; I'll take a look, though.
Jeremy
[1] One mkinitrd to rule them all... :)
5:34 p.m.
New subject: Review? LiveUSB persistence
Jeremy Katz wrote:
Moving to the more appropriate list, original is at
https://www.redhat.com/archives/fedora-devel-list/2008-January/msg03188.html for those
not on fedora-devel
On Thu, 2008-01-31 at 16:11 -0600, Douglas McClendon wrote:
> I keep hearing these crazy rumors that RedHat actually wants to subject
> actual users to my LiveUSB persistence feature next/this month.
Lots of people want lots of different things. I've definitely left this
sitting hanging too long and I apologize. I just didn't have time to
poke over the holidays and my January was a bitty nutty. But I should
be back to just my normal mail lag now ;-)
> I for one think the feature is really cool, useful, and know of no
> showstopping or even known bugs. But I also would not subject actual
> users to it, without having had more people review it and sign off on
> it. Anyway, to absolve myself of any responsibility, I once again
> request code review and feedback-
Overall, I think that it looks good. I think the big thing to work out
is how best to integrate the initscripts changes. Especially as we're
in parallel switching things over to upstart for F9. Just to make sure
that I'm clear about what's there so that when we try to bribe Bill, we
can do so effectively...
* The unmount loops in /etc/init.d/functions are modified so that we
don't unmount the underlying fs of the overlay
Correct. And the method currently in use is adding any subcomponents of
the rootfs to /dev/.fstab.live.special. I chose that location just
because it is available from both the initramfs and then later during
shutdown.
* We don't want to sync the hwclock on shutdown of a live image
* Do an overlay teardown on halt to replace the rw snapshot with a ro
version
Yup, basically the old phase of 'remount rootfs readonly' just gets
quite a bit more complicated, but still essentially just that.
The latter feels like we're going to want a generic hook to be
able to
run, perhaps from back in the initramfs. The middle feels largely
unrelated to persistence in general, but just a "needs fixing" which
will fit in nicely with other hwclock changes that are underway in
rawhide. And the first I'm less clear on I guess.
Outside of those changes, everything looks pretty straight-forward and
reasonable. We'll want a little more error checking in
livecd-iso-to-disk and it might also be nice to have another tool that
lets you create your usb stick to use with the actual cd form (I did
this by hand and it was pretty nice that I could easily get it
working!). As for the findoverlay bits, for right now, leaving them as
a separate script is fine. In the longer term, we're hopefully going to
kill mayflower and be able to use more "normal" initrds[1], at which
point pulling it in more directly would be nice.
I have no preference where the code lives. The main thing I would
highlight is how originally I wrote findoverlay, with the mindset that
the overlay file/layer could be located anywhere. But then, to focus on
getting something really usable, I honed in on the assumption of the
overlay file living together with the LiveOS on a LiveUSB. As a result
the findoverlay script definitely does not look like entirely grade A
code right now. But I have no objection to it getting merged in any
form, and perhaps later having cleanup/refactoring passes, perhaps with
even later passes that add back the functionality I originally had in mind.
As far as reliability... shutting down cleanly, I can't cause a
problem.
I pulled the usb stick from a box and got some garbage in a file I had
written. I want to do some more playing here, but I also wanted to get
this mail out today. Worst case, if we say "you need to shut down
cleanly", then so be it. It might be nice if we flag clean vs not
shutdown so that we can force a fsck in unclean cases. But no clue if
that's even feasible; I'll take a look, though.
My vague understanding right now would that it would be treated just
like a normal rootfs. I.e. the same things happening if it is flagged
as dirty, as if a normal system. But... I guess then there is the
issue of fscking the host filesystem, e.g. vfat(ext3?) of the LiveUSB. Ugh.
One immediate enhancement I thought of, is that perhaps the interface
for initialization can be taken out of livecd-iso-to-disk, and put into
the initramfs. I.e. a new kernel cmdline parameter of
"initoverlay=256MB"
would just invoke the dd if=/dev/zero... during that boot. Maybe even a
default value that just uses half the free space on the LiveUSB.
Thus perhaps livecd-iso-to-disk just by default adds another couple menu
entries to syslinux, one for 'boot with persistence' and one for
'initialize persistence'. Or perhaps the former automatically
initializes if none is found, and the latter, or some post-boot system
command could 'reinitialize persistence'...
-dmc
6:51 p.m.
New subject: Review? LiveUSB persistence
Jeremy Katz wrote:
Outside of those changes, everything looks pretty straight-forward and
reasonable. We'll want a little more error checking in
livecd-iso-to-disk and it might also be nice to have another tool that
lets you create your usb stick to use with the actual cd form (I did
this by hand and it was pretty nice that I could easily get it
working!).
I didn't really slow down and read this bit at first- So you actually
used the livecd+usb form? I probably tried that once when I first wrote
the code, but haven't even considered it since then. I guess you must
have by hand added something like overlay=UUID=bla to the kernel
cmdline. Of course the original incarnation of the patch supported the
syntax overlay=auto which would scan all drives. But that code scares
my responsibility-averse nature, and I was happy to remove it when
focusing on the liveusb case. And also there was a bunch of related
bash junk for dealing with finding multiple overlays.
-dmc
-dmc
7:54 p.m.
New subject: Review? LiveUSB persistence
On Mon, 2008-02-04 at 17:34 -0600, Douglas McClendon wrote:
Jeremy Katz wrote:
> Moving to the more appropriate list, original is at
> https://www.redhat.com/archives/fedora-devel-list/2008-January/msg03188.html for
those not on fedora-devel
>
> On Thu, 2008-01-31 at 16:11 -0600, Douglas McClendon wrote:
>> I keep hearing these crazy rumors that RedHat actually wants to subject
>> actual users to my LiveUSB persistence feature next/this month.
>
> Lots of people want lots of different things. I've definitely left this
> sitting hanging too long and I apologize. I just didn't have time to
> poke over the holidays and my January was a bitty nutty. But I should
> be back to just my normal mail lag now ;-)
>
>> I for one think the feature is really cool, useful, and know of no
>> showstopping or even known bugs. But I also would not subject actual
>> users to it, without having had more people review it and sign off on
>> it. Anyway, to absolve myself of any responsibility, I once again
>> request code review and feedback-
>
> Overall, I think that it looks good. I think the big thing to work out
> is how best to integrate the initscripts changes. Especially as we're
> in parallel switching things over to upstart for F9. Just to make sure
> that I'm clear about what's there so that when we try to bribe Bill, we
> can do so effectively...
> * The unmount loops in /etc/init.d/functions are modified so that we
> don't unmount the underlying fs of the overlay
Correct. And the method currently in use is adding any subcomponents of
the rootfs to /dev/.fstab.live.special. I chose that location just
because it is available from both the initramfs and then later during
shutdown.
*nod*
> * We don't want to sync the hwclock on shutdown of a live
image
> * Do an overlay teardown on halt to replace the rw snapshot with a ro
> version
Yup, basically the old phase of 'remount rootfs readonly' just gets
quite a bit more complicated, but still essentially just that.
Right. I really am leaning towards thinking that the teardown of the rw
mount to be ro probably needs to be from initramfs land as that allows
it to not require changes anywhere else when adding a new rootfs scheme.
I'll have to see what we can cobble together to make that work (it has
the added bonus of we'd be able to eject the cd on reboot from a livecd)
> The latter feels like we're going to want a generic hook to
be able to
> run, perhaps from back in the initramfs. The middle feels largely
> unrelated to persistence in general, but just a "needs fixing" which
> will fit in nicely with other hwclock changes that are underway in
> rawhide. And the first I'm less clear on I guess.
>
> Outside of those changes, everything looks pretty straight-forward and
> reasonable. We'll want a little more error checking in
> livecd-iso-to-disk and it might also be nice to have another tool that
> lets you create your usb stick to use with the actual cd form (I did
> this by hand and it was pretty nice that I could easily get it
> working!). As for the findoverlay bits, for right now, leaving them as
> a separate script is fine. In the longer term, we're hopefully going to
> kill mayflower and be able to use more "normal" initrds[1], at which
> point pulling it in more directly would be nice.
I have no preference where the code lives. The main thing I would
highlight is how originally I wrote findoverlay, with the mindset that
the overlay file/layer could be located anywhere. But then, to focus on
getting something really usable, I honed in on the assumption of the
overlay file living together with the LiveOS on a LiveUSB. As a result
the findoverlay script definitely does not look like entirely grade A
code right now. But I have no objection to it getting merged in any
form, and perhaps later having cleanup/refactoring passes, perhaps with
even later passes that add back the functionality I originally had in mind.
*nod* I think it's the best way so that we get things in sooner rather
than later. You've had it basically working for quite a while and we
should just bite the bullet and say "good enough" rather than waiting
for "perfect"
> As far as reliability... shutting down cleanly, I can't
cause a problem.
> I pulled the usb stick from a box and got some garbage in a file I had
> written. I want to do some more playing here, but I also wanted to get
> this mail out today. Worst case, if we say "you need to shut down
> cleanly", then so be it. It might be nice if we flag clean vs not
> shutdown so that we can force a fsck in unclean cases. But no clue if
> that's even feasible; I'll take a look, though.
My vague understanding right now would that it would be treated just
like a normal rootfs. I.e. the same things happening if it is flagged
as dirty, as if a normal system. But... I guess then there is the
issue of fscking the host filesystem, e.g. vfat(ext3?) of the LiveUSB. Ugh.
Right. It's ... less than pretty. But could easily be a second pass
once we get things merged in.
One immediate enhancement I thought of, is that perhaps the interface
for initialization can be taken out of livecd-iso-to-disk, and put into
the initramfs. I.e. a new kernel cmdline parameter of "initoverlay=256MB"
would just invoke the dd if=/dev/zero... during that boot. Maybe even a
default value that just uses half the free space on the LiveUSB.
This would definitely make things simpler for Luke who's working on a
Win32 version of livecd-iso-to-disk ;-) I'm not really sure where I
stand on this one, I could probably be moved in either direction
Thus perhaps livecd-iso-to-disk just by default adds another couple
menu
entries to syslinux, one for 'boot with persistence' and one for
'initialize persistence'. Or perhaps the former automatically
initializes if none is found, and the latter, or some post-boot system
command could 'reinitialize persistence'...
The problem with adding entries is that conceivably, you could have
multiple kernels included and editing the syslinux.cfg at that point
gets tricky. But I can think of a few ways to handle that, including
having the persistence option in CD form and letting people choose (with
a timeout) a location for their persistent bits.
I'm not going to get hung up on figuring that out today, though. I kind
of like the fact that to easily use it, you use livecd-iso-to-disk and
it sets things up for you. Since if you're going to have persistence,
then a USB stick is a pretty good idea. And by using USB instead, you
end up getting better performance
Jeremy
7:55 p.m.
New subject: Review? LiveUSB persistence
On Mon, 2008-02-04 at 18:51 -0600, Douglas McClendon wrote:
Jeremy Katz wrote:
> Outside of those changes, everything looks pretty straight-forward and
> reasonable. We'll want a little more error checking in
> livecd-iso-to-disk and it might also be nice to have another tool that
> lets you create your usb stick to use with the actual cd form (I did
> this by hand and it was pretty nice that I could easily get it
> working!).
I didn't really slow down and read this bit at first- So you actually
used the livecd+usb form? I probably tried that once when I first wrote
the code, but haven't even considered it since then. I guess you must
have by hand added something like overlay=UUID=bla to the kernel
cmdline. Of course the original incarnation of the patch supported the
syntax overlay=auto which would scan all drives. But that code scares
my responsibility-averse nature, and I was happy to remove it when
focusing on the liveusb case. And also there was a bunch of related
bash junk for dealing with finding multiple overlays.
Yep, I remember the auto bits as well. While at the time, I think I was
pushing you towards it, I think I'm now with you on the being
conservative. It was pretty easy to set up to use an external drive,
though. I had a partition with a dos filesystem, created the
LiveOS/Fedora-8-Live-i686- file and booted with overlay=LABEL=foo (which
was the label of my filesystem).
Jeremy
11:42 p.m.
New subject: Review? LiveUSB persistence
Jeremy Katz wrote:
On Mon, 2008-02-04 at 18:51 -0600, Douglas McClendon wrote:
> Jeremy Katz wrote:
>> Outside of those changes, everything looks pretty straight-forward and
>> reasonable. We'll want a little more error checking in
>> livecd-iso-to-disk and it might also be nice to have another tool that
>> lets you create your usb stick to use with the actual cd form (I did
>> this by hand and it was pretty nice that I could easily get it
>> working!).
> I didn't really slow down and read this bit at first- So you actually
> used the livecd+usb form? I probably tried that once when I first wrote
> the code, but haven't even considered it since then. I guess you must
> have by hand added something like overlay=UUID=bla to the kernel
> cmdline. Of course the original incarnation of the patch supported the
> syntax overlay=auto which would scan all drives. But that code scares
> my responsibility-averse nature, and I was happy to remove it when
> focusing on the liveusb case. And also there was a bunch of related
> bash junk for dealing with finding multiple overlays.
Yep, I remember the auto bits as well. While at the time, I think I was
pushing you towards it, I think I'm now with you on the being
conservative. It was pretty easy to set up to use an external drive,
though. I had a partition with a dos filesystem, created the
LiveOS/Fedora-8-Live-i686- file and booted with overlay=LABEL=foo (which
was the label of my filesystem).
So now that you're threatening to merge the code... reviewing it myself
I see that-
One obvious error is that in the halt teardown code, I assume the
overlay device is /dev/live. This is only true for the LiveUSB case,
and my seemingly pointless use of a variable to store /dev/live, shows
that I was somewhat aware of the issue. The right answer might be the
need to add another udev rule to the initramfs so that the overlayfs
device can be known at halt time. I would say that because I think
there might be no guaranteed way to determine the FS holding the file
associated with a loop device (because losetup -a output may be
truncated, and I don't know of another way?????)
Other than that, I don't actually see anything too ugly. Probably the
selinux permissifying in the teardown should be tested to see if it is
really needed. It's possible I just messed up my labels in some prior
version. Or, it's possible that the right answer is to change the
selinux policy to allow whatever it was disallowing.
As far as your statements about doing teardown from initramfs... One
thing I had wanted to do was to make the initramfs mount visible later
on. This was to be able to watch the size of the overlay sparse file.
My way around the need for that was to create a new tmpfs just for it.
But what you see in the code ('nobwa') is that currently there is a
devicemapper bug I'm hitting if I try to do that. (I have a
reproducibly indeterminite example I need to make the effort of
attaching to the bug report). But digressing a bit... To see the
initramfs, I think you might have to change nash so that it goes back to
using pivotroot instead of mount --move. And then presumably hook into
halt so that at the end, it fires off something from the initramfs
again. Or I guess, you could copy your initramfs stuff to a new tmpfs,
and not need the pivotroot thing. But the pivotroot thing seems more
natural. But I have no idea why that got changed to a movemount (just
to hide the mountpoint, or some other issue?)
-dmc
7:54 a.m.
New subject: Review? LiveUSB persistence
Douglas McClendon wrote:
attaching to the bug report). But digressing a bit... To see the
initramfs, I think you might have to change nash so that it goes back to
using pivotroot instead of mount --move. And then presumably hook into
halt so that at the end, it fires off something from the initramfs
again. Or I guess, you could copy your initramfs stuff to a new tmpfs,
and not need the pivotroot thing. But the pivotroot thing seems more
natural. But I have no idea why that got changed to a movemount (just
to hide the mountpoint, or some other issue?)
http://lxr.linux.no/linux/fs/namespace.c#L1642
you can't pivot_root initramfs=rootfs
10:04 a.m.
New subject: Review? LiveUSB persistence
On Mon, 2008-02-04 at 23:42 -0600, Douglas McClendon wrote:
Jeremy Katz wrote:
> On Mon, 2008-02-04 at 18:51 -0600, Douglas McClendon wrote:
>> Jeremy Katz wrote:
>>> Outside of those changes, everything looks pretty straight-forward and
>>> reasonable. We'll want a little more error checking in
>>> livecd-iso-to-disk and it might also be nice to have another tool that
>>> lets you create your usb stick to use with the actual cd form (I did
>>> this by hand and it was pretty nice that I could easily get it
>>> working!).
>> I didn't really slow down and read this bit at first- So you actually
>> used the livecd+usb form? I probably tried that once when I first wrote
>> the code, but haven't even considered it since then. I guess you must
>> have by hand added something like overlay=UUID=bla to the kernel
>> cmdline. Of course the original incarnation of the patch supported the
>> syntax overlay=auto which would scan all drives. But that code scares
>> my responsibility-averse nature, and I was happy to remove it when
>> focusing on the liveusb case. And also there was a bunch of related
>> bash junk for dealing with finding multiple overlays.
>
> Yep, I remember the auto bits as well. While at the time, I think I was
> pushing you towards it, I think I'm now with you on the being
> conservative. It was pretty easy to set up to use an external drive,
> though. I had a partition with a dos filesystem, created the
> LiveOS/Fedora-8-Live-i686- file and booted with overlay=LABEL=foo (which
> was the label of my filesystem).
So now that you're threatening to merge the code... reviewing it myself
I see that-
My plan is working ;-) Just kidding...
One obvious error is that in the halt teardown code, I assume the
overlay device is /dev/live. This is only true for the LiveUSB case,
and my seemingly pointless use of a variable to store /dev/live, shows
that I was somewhat aware of the issue. The right answer might be the
need to add another udev rule to the initramfs so that the overlayfs
device can be known at halt time. I would say that because I think
there might be no guaranteed way to determine the FS holding the file
associated with a loop device (because losetup -a output may be
truncated, and I don't know of another way?????)
Yeah, if we can't go back into the initramfs (which, yes, may not be
possible given our use of mount --move), then we'll have to come up with
some better trickery for the halt side. I need to corner notting this
afternoon and see what thoughts he has on it as initscripts/init is all
his territory and he's in the middle of reworking some of it for upstart
anyway
As far as your statements about doing teardown from initramfs... One
thing I had wanted to do was to make the initramfs mount visible later
on. This was to be able to watch the size of the overlay sparse file.
My way around the need for that was to create a new tmpfs just for it.
But what you see in the code ('nobwa') is that currently there is a
devicemapper bug I'm hitting if I try to do that. (I have a
reproducibly indeterminite example I need to make the effort of
attaching to the bug report). But digressing a bit... To see the
initramfs, I think you might have to change nash so that it goes back to
using pivotroot instead of mount --move. And then presumably hook into
halt so that at the end, it fires off something from the initramfs
again. Or I guess, you could copy your initramfs stuff to a new tmpfs,
and not need the pivotroot thing. But the pivotroot thing seems more
natural. But I have no idea why that got changed to a movemount (just
to hide the mountpoint, or some other issue?)
You can't (successfully) do a pivotroot from an initramfs iirc.
Jeremy
5944
days inactive
5945
days old
livecd@lists.fedoraproject.org
7 comments
3 participants
participants (3)
-
apevec＠redhat.com -
Douglas McClendon -
Jeremy Katz