7:19 p.m.
I have a rather large set of RPMs and such on my host I want to install
on my guest using libguestfs. The normal way to do this would be to
upload the package(s), install them, then (optionally) remove the RPMs.
However, I don't particularly want to bloat the qcow2 file with the
changes made via uploading files which are only going to be deleted when
no longer in use. Someday (*sigh*) we'll have 9p-over-virtio support
built into qemu; until then, a few ways to get around this present
themselves:
- Add a temporary file as a disk to the guestfs session on startup;
within the guest, create a new filesystem there, transfer temporary
files to it, and delete when done.
Pro: No external dependencies.
Con: Not sure of a safe and correct way to find the name of the
temporary disk [to ensure that the mkfs operation being scripted refers
only to it], particularly when trying to script everything into a single
guestfish invocation with a variable number of other disks exposed (ie.
using a command line generated by virt-inspect). Upload needs to be
file-by-file (or have everything bundled in a tarball beforehand).
- Run a tmpfs mount on the guest; upload RPMs there.
Pro: Fast. No need to worry about device names.
Con: Potential for running out of memory. Upload needs to be
file-by-file (or have everything bundled in a tarball beforehand).
- Stuff the files to be added into a cramfs filesystem, running mkcramfs
on the host.
Pro: Relatively fast. Easy to transfer a full directory structure to the
guest at one stroke.
Con: Device names an issue. Requires cramfs.ko to be added to the
whitelist in appliance/update-initramfs.sh.in.
- Support a FUSE-based mechanism for exporting files, ie.
ccgfs-over-vmchannel
Pro: Potential to be the cleanest approach from an end-user perspective.
Con: Implementation difficulty; no Fedora or EPEL packages are available
for ccgfs.
11:38 p.m.
New subject: libguestfs best practices: Exposing files from the host for the duration of a session
On Thu, May 28, 2009 at 8:19 PM, Charles Duffy <charles(a)dyfis.net> wrote:
I have a rather large set of RPMs and such on my host I want to
install on
my guest using libguestfs. The normal way to do this would be to upload the
package(s), install them, then (optionally) remove the RPMs.
However, I don't particularly want to bloat the qcow2 file with the changes
made via uploading files which are only going to be deleted when no longer
in use. Someday (*sigh*) we'll have 9p-over-virtio support built into qemu;
until then, a few ways to get around this present themselves:
What about setting up a temporary NFS export on the host and mounting
that on the guest? I've done installs using yum/rpm directly from the
NFS mount, eliminating the need to ever copy the RPMs into anything
more than working memory. If network security is an issue, I'm sure
you could use a combination of iptables rules and NFS export options
to limit access to that single box as well (though, I haven't done
that one).
Here's to hoping I didn't miss something blatantly obvious...
Michael
11:51 p.m.
Michael Ansel wrote:
On Thu, May 28, 2009 at 8:19 PM, Charles Duffy
<charles(a)dyfis.net> wrote:
> I have a rather large set of RPMs and such on my host I want to install on
> my guest using libguestfs. The normal way to do this would be to upload the
> package(s), install them, then (optionally) remove the RPMs.
>
> However, I don't particularly want to bloat the qcow2 file with the changes
> made via uploading files which are only going to be deleted when no longer
> in use. Someday (*sigh*) we'll have 9p-over-virtio support built into qemu;
> until then, a few ways to get around this present themselves:
>
What about setting up a temporary NFS export on the host and mounting
that on the guest? I've done installs using yum/rpm directly from the
NFS mount, eliminating the need to ever copy the RPMs into anything
more than working memory. If network security is an issue, I'm sure
you could use a combination of iptables rules and NFS export options
to limit access to that single box as well (though, I haven't done
that one).
As libguestfs (and the software I'm building with it) runs
unprivileged
and unattended, an added requirement of NFS server support on the host
would mean a pretty massive increase in security exposure and
administrative load. (Also, the libguestfs appliance doesn't included
nfs-utils at present, and I don't know that NFS works over qemu's
SLiRP-based "-net user" mode anyhow).
If I were going to go that route, I'd probably lean towards using qemu's
existing SMB server support -- but there would still be changes to the
guest needed, making it less than pain-free.
[For the moment, I ended up using squashfs; cramfs, mentioned in my
previous email, was a Really Bad Idea on account of the size limitations]
1:27 a.m.
On Fri, May 29, 2009 at 12:38:46AM -0400, Michael Ansel wrote:
On Thu, May 28, 2009 at 8:19 PM, Charles Duffy
<charles(a)dyfis.net> wrote:
> I have a rather large set of RPMs and such on my host I want to install on
> my guest using libguestfs. The normal way to do this would be to upload the
> package(s), install them, then (optionally) remove the RPMs.
>
> However, I don't particularly want to bloat the qcow2 file with the changes
> made via uploading files which are only going to be deleted when no longer
> in use. Someday (*sigh*) we'll have 9p-over-virtio support built into qemu;
> until then, a few ways to get around this present themselves:
>
What about setting up a temporary NFS export on the host and mounting
that on the guest? I've done installs using yum/rpm directly from the
NFS mount, eliminating the need to ever copy the RPMs into anything
more than working memory. If network security is an issue, I'm sure
you could use a combination of iptables rules and NFS export options
to limit access to that single box as well (though, I haven't done
that one).
The original libguestfs plan was to use NFS for all file transfers.
The reason why we don't intend to do this anymore is twofold: NFS
requires root on the host side, and NFS requires either matching UIDs
or complex UID-UID mappings. If we were going to implement any
protocol, it'd be FTP, because FTP clients are common, can be run
non-root, and don't need complicated UID mappings.
Rich.
--
Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into Xen guests.
http://et.redhat.com/~rjones/virt-p2v
2:34 a.m.
New subject: libguestfs best practices: Exposing files from the host for the duration of a session
On Fri, May 29, 2009 at 2:27 AM, Richard W.M. Jones <rjones(a)redhat.com> wrote:
The original libguestfs plan was to use NFS for all file transfers.
The reason why we don't intend to do this anymore is twofold: NFS
requires root on the host side, and NFS requires either matching UIDs
or complex UID-UID mappings. If we were going to implement any
protocol, it'd be FTP, because FTP clients are common, can be run
non-root, and don't need complicated UID mappings.
Okay, I think I'm not completely understanding how libguestfs works,
so that is probably why I'm getting confused. *adds that to to-do
list* I was looking at it figuring:
1) root (UID=0, everywhere) is the only one installing packages, so
the UID mapping lines up perfectly every time
2) you only need root access on the host once when you are building
the VM; you don't need NFS after it is already built
I have a feeling this will all become clear to me once I do some
research *self-chastisement for speaking before reading*, but for now
I need sleep, so that reading shall come tomorrow.
Also, on a side note: Is there a reason all of my replies are
automatically directed at individuals vs the list? (I keep having to
retype fedora-virt(a)redhat.com) Or is this another one of those things
where I didn't read something I was supposed to...
Thanks!
Michael
2:46 a.m.
Michael Ansel wrote:
1) root (UID=0, everywhere) is the only one installing packages, so
the UID mapping lines up perfectly every time
libguestfs in general, and host
filesystem access in particular, is good
for more than just installing packages, though admittedly that's my
immediate use case.
2) you only need root access on the host once when you are building
the VM; you don't need NFS after it is already built
Even if there were no use case for access to files off the host other
than early installation --
Why is it acceptable for libguestfs to require root access *ever*, when
so many alternate transport mechanisms (9p, ccgfs, FTP, qemu's built in
smb, etc) are available which wouldn't create this constraint?
If I'm building a piece of software invoked by untrusted users in a lab
environment, I don't want that software to need, or have any means to
access, root. Ever. For anything, at all.
By the way -- I'm tired too (normal wakeup time in ~3.25 hours), so
please excuse my tone if it's a bit harsh.
4:44 a.m.
On Fri, May 29, 2009 at 03:34:21AM -0400, Michael Ansel wrote:
1) root (UID=0, everywhere) is the only one installing packages, so
the UID mapping lines up perfectly every time
No - the concept of UIDs in libguestfs is non-intuitive.
Firstly the normal mode of operation is to run libguestfs as non-root.
There are plenty of reasons why this is a good idea, I think Charles
covered a few of them. So everything we do is oriented around making
sure that we don't need to be root to do libguestfs operations.
However, libguestfs runs qemu/kvm (also as non-root) but that boots a
mini appliance. *Inside* the appliance the appliance kernel thinks
everything is running as root. It's not really from the point of view
of the host machine, but inside the appliance that's how it looks.
If we were to put an NFS server inside the appliance and tunnel the
connections out to the host, then the host would have to mount the
drive as root (or we'd have to fake it somehow - non-trivial with
NFS).
If we put an FTP server inside the appliance, things are better. FTP
clients are widely available, either standalone or as libraries for
many programming languages. They just need to be able to make a TCP
connection, so there is no requirement for special privileges. On the
server side (inside the appliance) the FTP server is running as "root"
so it can make arbitrary changes to the filesystem.
2) you only need root access on the host once when you are building
the VM; you don't need NFS after it is already built
With libguestfs currently, you never need root at all, not to build
libguestfs, not to run it, not for any operations using it.
Rich.
5:13 a.m.
On May 29, 2009, at 2:44, Richard W.M. Jones wrote:
If we put an FTP server inside the appliance, things are better.
FTP
clients are widely available, either standalone or as libraries for
many programming languages. They just need to be able to make a TCP
connection, so there is no requirement for special privileges. On the
server side (inside the appliance) the FTP server is running as "root"
so it can make arbitrary changes to the filesystem.
Please make FTP die already! :-)
How about HTTP (WebDAV) or - better - just use the ssh file transfer
protocol?
- ask
--
http://develooper.com/ - http://askask.com/
6:10 a.m.
On Fri, May 29, 2009 at 03:13:10AM -0700, Ask Bjørn Hansen wrote:
On May 29, 2009, at 2:44, Richard W.M. Jones wrote:
> If we put an FTP server inside the appliance, things are better. FTP
> clients are widely available, either standalone or as libraries for
> many programming languages. They just need to be able to make a TCP
> connection, so there is no requirement for special privileges. On the
> server side (inside the appliance) the FTP server is running as "root"
> so it can make arbitrary changes to the filesystem.
Please make FTP die already! :-)
How about HTTP (WebDAV) or - better - just use the ssh file transfer
protocol?
ssh isn't too widely supported (from programming language libraries).
Does libssh2 even support scp?
WebDAV is possible, but it comes down to how much crap it will depend
upon, all of which needs to go into the appliance.
FTP is widely supported and well understood, and the servers are old,
small, self-contained code. I really think it's better for this, and
not just because I once wrote an FTP server ...
Rich.
6:13 a.m.
Richard W.M. Jones wrote:
FTP is widely supported and well understood, and the servers are
old,
small, self-contained code. I really think it's better for this, and
not just because I once wrote an FTP server ...
Just to weigh in -- my only concern with FTP is that it's limited in
what metadata it will transport. ccgfs, for instance, supports full
POSIX ACLs and xattrs. Certainly, that's overkill for the common use
cases here -- but maybe something to keep in mind.
6:18 a.m.
On Fri, May 29, 2009 at 06:13:05AM -0500, Charles Duffy wrote:
Richard W.M. Jones wrote:
>FTP is widely supported and well understood, and the servers are old,
>small, self-contained code. I really think it's better for this, and
>not just because I once wrote an FTP server ...
>
Just to weigh in -- my only concern with FTP is that it's limited in
what metadata it will transport. ccgfs, for instance, supports full
POSIX ACLs and xattrs. Certainly, that's overkill for the common use
cases here -- but maybe something to keep in mind.
I don't think you'd want to transfer ACLs/xattrs from the host into
the guest because that implies some fairly close alignment between
user / permissioning model between guest & host, cf why NFS isn't
a good idea for this. It likeyl be more useful just let the uploaded
files get the default ACL/xattr and ue a guestfs command to change
them after the fact if needed
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
6:21 a.m.
On Fri, May 29, 2009 at 06:13:05AM -0500, Charles Duffy wrote:
Richard W.M. Jones wrote:
> FTP is widely supported and well understood, and the servers are old,
> small, self-contained code. I really think it's better for this, and
> not just because I once wrote an FTP server ...
>
Just to weigh in -- my only concern with FTP is that it's limited in
what metadata it will transport. ccgfs, for instance, supports full
POSIX ACLs and xattrs. Certainly, that's overkill for the common use
cases here -- but maybe something to keep in mind.
In addition to Dan's reply, as long as 'tar' supports it, you should
be able to upload it using the 'tar-in' or 'tgz-in' commands:
http://et.redhat.com/~rjones/libguestfs/guestfs.3.html#guestfs_tgz_in
I have also added support for cramfs & squashfs to the appliance,
which is yet another way to 'get stuff inside' without needing time-
consuming uploads.
Rich.
6:20 a.m.
On Fri, May 29, 2009 at 12:10:05PM +0100, Richard W.M. Jones wrote:
On Fri, May 29, 2009 at 03:13:10AM -0700, Ask Bjørn Hansen wrote:
> On May 29, 2009, at 2:44, Richard W.M. Jones wrote:
>> If we put an FTP server inside the appliance, things are better. FTP
>> clients are widely available, either standalone or as libraries for
>> many programming languages. They just need to be able to make a TCP
>> connection, so there is no requirement for special privileges. On the
>> server side (inside the appliance) the FTP server is running as
"root"
>> so it can make arbitrary changes to the filesystem.
>
> Please make FTP die already! :-)
>
> How about HTTP (WebDAV) or - better - just use the ssh file transfer
> protocol?
ssh isn't too widely supported (from programming language libraries).
Does libssh2 even support scp?
WebDAV is possible, but it comes down to how much crap it will depend
upon, all of which needs to go into the appliance.
FTP is widely supported and well understood, and the servers are old,
small, self-contained code. I really think it's better for this, and
not just because I once wrote an FTP server ...
I tend to agree with this - ssh is overkill here - the host <->guest
channel is already secure by virtue of using vmchannel, so using ssh
is not really required. FTP is a really easy to implement, low overhead
protocol that'd fit nicely here. I can't really think of what WebDAV
would do for this use case that FTP can't do in a simpler manner.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
7:57 a.m.
"Daniel P. Berrange" <berrange(a)redhat.com> writes:
On Fri, May 29, 2009 at 12:10:05PM +0100, Richard W.M. Jones wrote:
> On Fri, May 29, 2009 at 03:13:10AM -0700, Ask Bjørn Hansen wrote:
> > On May 29, 2009, at 2:44, Richard W.M. Jones wrote:
> >> If we put an FTP server inside the appliance, things are better. FTP
> >> clients are widely available, either standalone or as libraries for
> >> many programming languages. They just need to be able to make a TCP
> >> connection, so there is no requirement for special privileges. On the
> >> server side (inside the appliance) the FTP server is running as
"root"
> >> so it can make arbitrary changes to the filesystem.
> >
> > Please make FTP die already! :-)
> >
> > How about HTTP (WebDAV) or - better - just use the ssh file transfer
> > protocol?
>
> ssh isn't too widely supported (from programming language libraries).
> Does libssh2 even support scp?
>
> WebDAV is possible, but it comes down to how much crap it will depend
> upon, all of which needs to go into the appliance.
>
> FTP is widely supported and well understood, and the servers are old,
> small, self-contained code. I really think it's better for this, and
> not just because I once wrote an FTP server ...
I tend to agree with this - ssh is overkill here - the host <->guest
channel is already secure by virtue of using vmchannel, so using ssh
is not really required. FTP is a really easy to implement, low overhead
protocol that'd fit nicely here. I can't really think of what WebDAV
would do for this use case that FTP can't do in a simpler manner.
Daniel
Sounds like even TFTP would do.
12:40 p.m.
On May 29, 2009, at 4:10, Richard W.M. Jones wrote:
> just use the ssh file transfer protocol?
ssh isn't too widely supported (from programming language libraries).
Does libssh2 even support scp?
libcurl does. On the server side running sshd (and having sftp-server
installed) seems pretty manageable.
WebDAV is possible, but it comes down to how much crap it will
depend
upon, all of which needs to go into the appliance.
nginx has a reasonable WebDAV implementation and few dependencies
IIRC. I don't think WebDAV is worth it though; the only real benefit
is that it'd make it easy to make a "browse this disk from your web
browser" feature.
FTP is widely supported and well understood, and the servers are
old,
small, self-contained code. I really think it's better for this, and
not just because I once wrote an FTP server ...
As long as it's just host->guest communication I agree, it makes no
difference. If it's ever extended to remote host -> guest
communication FTP will just bring tears and misery.
- ask
12:03 p.m.
New subject: libguestfs best practices: Exposing files from the host for the duration of a session
On Fri, May 29, 2009 at 5:44 AM, Richard W.M. Jones <rjones(a)redhat.com> wrote:
On Fri, May 29, 2009 at 03:34:21AM -0400, Michael Ansel wrote:
> 1) root (UID=0, everywhere) is the only one installing packages, so
> the UID mapping lines up perfectly every time
No - the concept of UIDs in libguestfs is non-intuitive.
Firstly the normal mode of operation is to run libguestfs as non-root.
There are plenty of reasons why this is a good idea, I think Charles
covered a few of them. So everything we do is oriented around making
sure that we don't need to be root to do libguestfs operations.
However, libguestfs runs qemu/kvm (also as non-root) but that boots a
mini appliance. *Inside* the appliance the appliance kernel thinks
everything is running as root. It's not really from the point of view
of the host machine, but inside the appliance that's how it looks.
> 2) you only need root access on the host once when you are
building
> the VM; you don't need NFS after it is already built
With libguestfs currently, you never need root at all, not to build
libguestfs, not to run it, not for any operations using it.
First, I just want to say, you guys are amazing: when do you sleep?!
I also want to apologize to the list for my somewhat mis- and
un-informed comments last night/early this morning. I do however think
I have figured out where my confusion was: I was under the impression
that running a virtual machine required root access and libguestfs
worked by launching said VM (in order to "run commands in the context
of the guest"). It seems I was grossly mistaken on both: virtual
machines can run as non-root (though, KVM is currently root only --
possibly changing), and libguestfs does only a miniature launch of the
system (? I'm still a little unclear on how the virtual appliance in
libguestfs works). I also did not realize that libguestfs was designed
with an explicit goal of not requiring root access for anything. In
hindsight, I *knew* you didn't have to be root to launch a VM: I've
done it countless times to test boot CDs; for some reason (probably
because I was up later than normal), I just forgot that fact.
Again, I'm very sorry for my ill-informed comments, and have
officially decided I'm no longer allowed to post on mailing lists
after midnight!
Michael Ansel
1:26 a.m.
On Thu, May 28, 2009 at 07:19:06PM -0500, Charles Duffy wrote:
- Add a temporary file as a disk to the guestfs session on startup;
within the guest, create a new filesystem there, transfer temporary
files to it, and delete when done.
Pro: No external dependencies.
Con: Not sure of a safe and correct way to find the name of the
temporary disk [to ensure that the mkfs operation being scripted refers
only to it], particularly when trying to script everything into a single
guestfish invocation with a variable number of other disks exposed (ie.
using a command line generated by virt-inspect). Upload needs to be
file-by-file (or have everything bundled in a tarball beforehand).
Or you could prepare the disk image in advance using a
separate guestfish operation:
http://et.redhat.com/~rjones/libguestfs/recipes.html#tar2vm
When you attach a new disk it has a predictable name, ie. /dev/hda,
/dev/hdb and so on. If you know that your guest has only one real
disk image, then your prepared image will always be /dev/hdb.
This is what our V2V scripts do in order to upload new RPMs into the
host, although at the moment it's not very well tested.
- Run a tmpfs mount on the guest; upload RPMs there.
Pro: Fast. No need to worry about device names.
Con: Potential for running out of memory. Upload needs to be
file-by-file (or have everything bundled in a tarball beforehand).
The amount of memory available is quite limited.
- Stuff the files to be added into a cramfs filesystem, running
mkcramfs
on the host.
Pro: Relatively fast. Easy to transfer a full directory structure to the
guest at one stroke.
Con: Device names an issue. Requires cramfs.ko to be added to the
whitelist in appliance/update-initramfs.sh.in.
We should add this anyway -- please file a bug in bugzilla.
- Support a FUSE-based mechanism for exporting files, ie.
ccgfs-over-vmchannel
Pro: Potential to be the cleanest approach from an end-user perspective.
Con: Implementation difficulty; no Fedora or EPEL packages are available
for ccgfs.
And the other option would be:
Implement the FTP server!
Rich.
--
Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into Xen guests.
http://et.redhat.com/~rjones/virt-p2v
5457
days inactive
5457
days old
16 comments
6 participants
participants (6)
-
Ask Bjørn Hansen -
Charles Duffy -
Daniel P. Berrange -
Markus Armbruster -
Michael Ansel -
Richard W.M. Jones