Need some setup pointers for using Xen in a HA system
by Jeremy Utley
Hello to everyone on the list. We're looking at making our datacenter
monitoring system (traffic graphs, nagios, etc) run under Xen so a
machine failure doesn't kill off our entire monitoring system. Right
now, all our monitoring runs on a single machine, so if that fails,
we're dead in the water, which is why we want to do this.
Here's what we've thought of so far.
2 identical machines running Xen-enabled
All monitoring services running within domU
Shared filesystem provided by GFS (Sun T3 array connected to both
machines with a brocade switch)
domU root filesystem is a loopback-style EXT3 on the GFS filesystem
The sticky point is getting set up so that we can move the domU from
one machine to the other in a live migration scenario, or activate on
the other machine in the case of a crash. In the second case (crash),
we'd like to automate the process so it will automatically come up
properly.
The distro in use for both dom0 and domU is planned to be FC6. Can
any of you who have done this type of thing in a real-life scenario
think of anything I've missed, or any issues with what we're thinking?
Especially suggestions for automating in the case of a crash would be
welcome!
Thanks for any input any of you can give!
Jeremy
17 years, 5 months
Re: [Fedora-xen] FC6 w updates: xenguest-install.py hangs on Dell poweredge
by . .
On Mon, 20 Nov 2006, Paul Wouters wrote:
>>xenguest-install -n ftp.xtdnet.nl -f
>>/children/ftp.xtdnet.nl/ftp.xtdnet.nl.img --nographics -p -l
>> >ftp://dl.xs4all.nl/pub/mirror/fedora/core/6/i386/os/ -x "noacpi noapic"
>>-r 1024
>>
>>
>>(the noapic noacpi was a test in the hope tht it would fix my issue).
>>
>>Also, I can still do xm shutdown and then I see the system halting.
>Using kpartx and losetup, I checked the filesystems. They hasn't been
>any writing apart from formatting done to the disk. So no logfiles of
>the anaconda installer to see what went wrong either :
>Paul
Hi,
I'm having exactly the same issue trying to load a new Xen guest system. My
setup is:
AMD64, 1Gig RAM, SATA Hard disk
Fedora Core 6 x86_64 - Dom0
I'm using FC6 for the guest system also, installing to a partition of the
hard disk - /dev/sda15
The installation source is an NFS mount of the FC6 DVD media (in the local
DVD drive).
It gets to the "Starting install process. This may take several minutes..."
and just sits there. Every couple of minutes, the DVD drives spins up and
reads but nothing else happens - like it's stuck in a loop of some kind. The
log files in /var/log/xen don't give any clues.
Ian.
_________________________________________________________________
Advertisement: House hunt online now!
http://a.ninemsn.com.au/b.aspx?URL=http%3A%2F%2Fninemsn%2Erealestate%2Eco...
17 years, 5 months
: "Guest Install Error" while creating a para virtualized guest with virt manager on FC6
by Ian Brown
Hello,
I have fc6 which runs on AMD x86_64. I have SVM support in this
processor; still I want
to create x86_64 fully virtualized guest OS , based on FC6.
So I ran virt-mananger (I have virt-manager-0.2.3-2.fc6 rpm , which
came with the installation).
After I enter parameters and click "finish" it starts doing something,
but after about a minute or two it stops giving the following error:
"Guest install error"
ERROR: VirDomainCreateLinux() failed
I had tried this 3 times, each time getting the same error.
The parameters I had chosen are:
MachineName: guestFC6
Virtulaization Method: Paravirtualized
installation source:
http://download.fedora.redhat.com/pub/fedora/linux/core/6/x86_64/os/
Disk Image: /work/xenGuest/guest
Disk Size: 2000MB
Maximum memory: 500 MB
Initial memory: 500 MB
Virtual CPUs: 1
Am I doing something wrong ? is there some log where the errors are
more detailed ?
Regards,
IB
17 years, 5 months
Howto enable sHype/ACM security for Xen with FC6 Xen sources and FC6 kernels
by Reiner Sailer
I have been approached for help in enabling sHype/ACM for Xen on an FC6
system using Fedora sources only. Since sHype/ACM is still disabled by
default in Xen, you need to recompile and re-install Xen to enable it. I
have attached a short howto, since this procedure might not be
straight-forward for the general user.
sHype/ACM is part of the core Xen distribution and includes mandatory
access control in the Xen hypervisor. sHype controls sharing between user
domains (controls which domains can communicate with each other and which
domains can access which resources) and enforces anti-collocation rules
(controls which domains can run simultaneously on the same platform) with
simple formal security policies. Please refer to the Xen user guide
section about sHype/ACM for more details and for usage/test examples and
current limitations.
You do not need to follow this howto if you choose to install the
original Xensource.com Xen version and the 2.6.16.29 Xen kernel. In this
case, the Xen user guide for sHype/ACM includes all information needed for
configuration, installation, and usage examples.
Feedback / corrections / improvements are welcome (I am not an FC6
specialist!).
Regards
Reiner
=======================================HOWTO
BUILD AND INSTALL SHYPE/ACM XEN FROM FEDORA CORE 6 SOURCES
***********************************************************
Foreword: You can use the official Xen source install from Xensource.com
and configure ACM (see Xen user guide). However, Xen comes with a
2.6.16.29 kernel by default. If you depend on a FC6 2.6.18 kernel running
on Xen and you want the sHype ACM security extension, then the following
document describes how to get there from a clean non-virtualized FC6
install.
The following step-wise description shows how to get sources, configure
them, and install them so that sHype/ACM security is enabled in Xen on FC6
for the latest FC6 kernel.
Once you run sHype/ACM Xen, you can refer to the Xen user guide manual
chapter (found in: ) 10 to walk through usage examples.
A) Get source Xen/Kernel for FC6 (from any FC6 mirror)
======================================================
ftp://ftp.linux.ncsu.edu/pub/fedora/linux/core/updates/6/SRPMS
download:
kernel-2.6.18-1.2849.fc6.src.rpm
ftp://ftp.linux.ncsu.edu/pub/fedora/linux/core/6/source/SRPMS
download:
xen-3.0.3-0.1.rc3.src.rpm
B) Unpack source rpm
====================
rpm -ihv kernel-2.6.18-1.2849.fc6.src.rpm
rpm -ihv xen-3.0.3-0.1.rc3.src.rpm
C) Create sources and configure sHype Access Control Module for Xen
===================================================================
(this step creates the sources into /usr/src/redhat/BUILD)
cd /usr/src/redhat/SPECS
rpmbuild -bp xen.spec
rpmbuild -bp kernel-2.6.spec
D) Build/Install Xen
====================
Note: it appears that most problems in this stage stem from inconsistent
PAE settings in Xen and Kernel (must be the same).
i) Configure + install security enabled Xen and tools:
cd /usr/src/redhat/BUILD/xen-3.0.3-rc3
edit Config.mk and set following variables for PAE/no PAE:
i.a) if you DON'T want PAE support (<4GB on x386):
XEN_TARGET_X86_PAE ?= n
ACM_SECURITY ?= y
i.b) if you DO want PAE support:
XEN_TARGET_X86_PAE ?= y
ACM_SECURITY ?= y
ii) Now save Config.mk and exit editor.
iii) in the current xen-3.0.3-rc3 directory:
root# (cd LibVNCServer-0.8.2; make install)
root# make xen tools
Note: do not just 'make' because it will take a long time to build the
kernel and you are not going to use it (see below)
iv)
root# make install-xen; make install-tools
v) Install wxPython for ez-Security Policy tool
root# yum install wxPython
Test: /usr/sbin/xensec_ezpolicy should bring up a GUI (close it)
E) BUILD/INSTALL FC6 Kernel for Xen
===================================
We only use the 2.6.18.i386 kernel from this install. Not xen.
i) Configure + install FC6 Kernel for Xen:
root# cd /usr/src/redhat/BUILD/kernel-2.6.18/linux-2.6.18.i386
root# cp configs/kernel-2.6.18.i686-xen.config .config
use 'make menuconfig' or 'make gconfig' to configure the following
variables for PAE/no PAE:
i.a) if you DON'T want PAE support (<4GB on x386):
In submenu: Processor_type_and_features->High_Memory_Support
set HIMEM to 4GB
i.b) if you DO want PAE support:
In submenu: Processor_type_and_features->High_Memory_Support
set HIMEM to 64GB
ii) Compile + Install kernel (currently, the kernel is not ACM specific)
Note: If you already have a proprietary kernel installed, you might want
to name the kernel by setting the LOCALVERSION config parameter.
root# make all
root# make modules_install
root# make install
F) CREATE BOOT ENTRY
====================
Mine looks as follows (using xen/kernel that were built/installed above):
title XEN sHype/ACM (2.6.18-1.2849-xen)
root (hd0,0)
kernel /xen-3.0.3-rc3.gz
module /vmlinuz-2.6.18-prep ro root=/dev/hda3 rhgb
module /initrd-2.6.18-prep.img
Make sure you have the initrd and that you have the proper file prefix for
the files. This example assumes that you mount /boot. You might need to
build the initrd manually if it does not show up in the /boot directory
after the kernel make install:
root #cd /boot
root #mkinitrd initrd-2.6.18-prep.img 2.6.18-prep
G) WHERE DO I GO FROM HERE
==========================
If you boot into sHype/ACM XEN, then you need to label resources and
domains. For this, you need a policy. Without it, you can start domain 0
but no other domains. Please refer to the Xen User Guide (currently
chapter 10) for further information.
=======================================END
__________________________________________________________
Reiner Sailer, Research Staff Member, Secure Systems Department
IBM T J Watson Research Ctr, 19 Skyline Drive, Hawthorne NY 10532
Phone: 914 784 6280 (t/l 863) Fax: 914 784 6205, sailer(a)us.ibm.com
http://www.research.ibm.com/people/s/sailer/
17 years, 5 months
Windows Vista RC1 on Xen 3.0.3 FC6
by Ian Patton
Hello,
I now have Windows Vista RC1 attempting to boot using the following hvm
file:
# Automatically generated xen config file
name = "WindowsVistaRC1"
builder = "hvm"
memory = "784"
disk = [ 'phy:/dev/hdc1,hda,w', 'file:/tmp/WINDOWS_VISTA.iso,hdc:cdrom,r' ]
vif = [ 'type=ioemu, mac=00:16:3e:0f:a7:08, bridge=xenbr0', ]
uuid = "7232d84d-f579-a4f1-d190-b31ebb66faf9"
device_model = "/usr/lib/xen/bin/qemu-dm"
kernel = "/usr/lib/xen/boot/hvmloader"
vnc=1
vncunused=1
#sdl=1
apic=1
acpi=1
pae=1
boot="d"
vcpus=1
serial = "pty" # enable serial console
on_reboot = 'restart'
on_crash = 'destroy'
I can see it boot from the DVD image and start to load the installer. It
gets to the point where the black screen progress bar is at 100% and then I
see the DomU processor drop to 0% and it does nothing.
xend.log shows nothing useful
the qemu-dm.*.log shows:
domid: 28
qemu: the number of cpus is 1
shared page at pfn:30fff, mfn: 3a0af
buffered io page at pfn:30ffd, mfn: 3a0b1
char device redirected to /dev/pts/1
False I/O request ... in-service already: 0, pvalid: 0, port: 0, data: 0,
count: 0, size: 0
and the xend-debug.log shows this:
Nothing to flush.
Nothing to flush.
File "/usr/lib/python2.4/site-packages/xen/xend/server/SrvDomainDir.py",
line 77, in op_create
dominfo = self.xd.domain_create(config)
File "/usr/lib/python2.4/site-packages/xen/xend/XendDomain.py", line 228,
in domain_create
dominfo = XendDomainInfo.create(config)
File "/usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py", line
195, in create
vm.initDomain()
File "/usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py", line
1335, in initDomain
balloon.free(memory + shadow)
File "/usr/lib/python2.4/site-packages/xen/xend/balloon.py", line 166, in
free
raise VmError(
VmError: I need 1070492 KiB, but dom0_min_mem is 262144 and shrinking to
262144 KiB would leave only 773804 KiB free.
I have tried chaning the APIC, ACPI and PAE settings to every possible combo
and it only made it worse.
I did create a partition for this install... Actually I gave it it's own
whole disk drive. The partition is /dev/hdc1 but it is not formatted. I
assumed Windows would format it as needed. Does anyone know if it should be
formatted as ext3 for it to work?
--
Ian Patton
17 years, 5 months
Installation says to less space on disk
by Roger Grosswiler
Hi,
I try to install XEN on FC6 on a machine having 512 mb of ram and enough
diskspace.
i created a DOMU, having 2.5GB of Diskspace (Image-File), 256 MB Ram with
64 at start and ballooned Dom0 to 200 MB Ram.
Everything goes well, until the installation starts to get its packages.
When the download should start, i get the error message, that / just has
284 MB's of Disk available.
I then tried to change the partition-scheme not to use LVM and wanted to
create the scheme manually. This was mission impossible, as i had no / to
indicate the mountpoints.
How could i handle this? It was also strange, that i could not use the
GUI-installer.
btw. i tried to install via virt-manager.
Thanks,
Roger
17 years, 5 months
Bringing up second physical interface in Dom0
by Tan Chee Sin
Hi,
I have two physical network interfaces on my machine and I'm trying to
do VLAN on a DomU via eth1 instead of eth0.
On Dom0, I see peth0 but not peth1, how do I bring up peth1?
cheesin
17 years, 5 months
Updated 'xorg-x11-server' + kernel-xen = system reboot.
by Naoki
Howdy,
I was pleased to hear Dan B's announcement of a Xorg fix that would stop
the high frequency random crashing under certain intel chipsets but here
are my results on my Dell GX620.
Booting up is fine, all the way to GDM. At this point, without logging
in, I can SSH to the machine and check no application failures, and "xm
list" shows my dom0 as expected. However immediately upon successful
login the system reboots. I tried three times and have now fallen back
to the non-xen kernel. I was trying with "selinux=0" if that's of use.
There is nothing in the messages file to indicate what the problem might
be, the messages file simply contains the next kernel boot message.
>From the sounds of things debugging the intel / xorg issue was a
nightmare, hopefully this gremlin will be easier to tame. But is
anybody else finding this issue, or just my lucky self?
17 years, 5 months
renaming vg for lvm backed domU
by Robert Story
Hi,
I've got a FC5 dom0 where I've created a lvm backed domU I want to use as a
template. The domU works fine. If I create a new lv in the dom0 and dd the
orginal lv over to the new one, copy the xen conf file, update the path and
mac addr, the domU clone boots fine too.
However, I want the domU vg name to be different in the new domU. So, I used
kpartx to get to the new vg, and used vgrename to rename it. This works great
in the dom0, and I can get to all the lv's. However, when I try to reboot the
domU (after updating it's grub.conf and fstab, deleting it's /etc/lvm/cache,
and renaming the files and vg name in /etc/lvm/(backup|archive)/*), something
is still looking for the old vg name.
On boot, I see:
...
Kernel command line: ro root=/dev/new/root s
...
Loading dm-mod.ko module
device-mapper: ioctl: 4.7.0-ioctl (2006-06-24) initialised: dm-devel(a)redhat.com
Loading dm-mirror.ko module
Loading dm-zero.ko module
Loading dm-snapshot.ko module
Making device-mapper control node
Scanning logical volumes
Reading all physical volumes. This may take a while...
Found volume group "new" using metadata type lvm2
Activating logical volumes
Unable to find volume group "old"
Creating root device.
Mounting root filesystem.
mount: could not find filesystem '/dev/root'
Setting up other filesystems.
Setting up new root fs
setuproot: moving /dev failed: No such file or directory
no fstab.sys, mounting internal defaults
setuproot: error mounting /proc: No such file or directory
setuproot: error mounting /sys: No such file or directory
Switching to new root and running init.
unmounting old /dev
unmounting old /proc
unmounting old /sys
switchroot: mount failed: No such file or directory
Kernel panic - not syncing: Attempted to kill init!
In dom0:
# kpartx -a /dev/vm/new
# pvscan
PV /dev/dm-15 VG new lvm2 [9.88 GB / 1.94 GB free]
# lvscan
inactive '/dev/new/root' [1.00 GB] inherit
inactive '/dev/new/usr' [5.94 GB] inherit
inactive '/dev/new/swap' [1.00 GB] inherit
I mounted /dev/new/root and did a find 'etc -type f|xargs grep old', and
couldn't find any files that still referenced the 'old' vg. Anyone have any
idea what I'm missing?
17 years, 5 months
Xen + LVM + OS image
by turner＠ftn.net
Hello all,
I have a setup that I have been testing to create a FC6 guest image on a
lvm logical volume (lv) and then use that to deploy new a new guest. I
wanted to get feedback about anything that I might have overlooked.
In a nutshell, each xen guest install has it's own lv for a root partition
and swap partition provided by the host. The guest installs are not using
lvm, they use the whole 'disk' as they see it, which is of course managed
by the host lvm.
To deploy a new FC6 guest, lvcreate new swap and data partitions and dd
the image lv to the new guest partition.
Then a copy of the /etc/xen/image is made, the mac address, uuid and name
are updated. Furthermore, in the new guest os the mac address and
networking settings must be updated.
An external USB2.0 drive that is managed by lvm is used for backups. Use
pvcreate and vgcreate to create a backup volume group (first time only)
and create logical volumes, shutdown the guests and dd the guest lv to
the backup volume group. Then use vgchange and vgexport to remove the
backup volume (pvscan and vgimport are used to re-attach the volume in the
future).
The following is a edited version of my more detailed notes.
I would really like to hear any suggestions or any major problems with
this,
Thanks!
Gordon Turner.
1. Host setup
-------------
- Install FC6 and xen on the host, using LVM and leaving space for the
guest logical volumes.
- Create logical volumes for xen hosts and create swap space:
lvcreate -L 1G -n ImageSwap00 VolGroup00
lvcreate -L 3G -n ImageData00 VolGroup00
2. Create image domain
----------------------
- Run installer:
[root@xen ~]# /usr/sbin/xenguest-install
What is the name of your virtual machine? image
How much RAM should be allocated (in megabytes)? 256
What would you like to use as the disk (path)?
/dev/VolGroup00/ImageData00
Would you like to enable graphics support? (yes or no) no
What is the install location? http://192.168.1.199/fc6/
- During the guest install, remove default lvm partitioning and create
one filesystem that completely fills the available 'drive'. Ignore the
swap warning, it will be added later.
- After the first boot of the guest image, update fstab in the image to
include the swap space:
vi /etc/fstab
/dev/xvdb swap swap defaults 0 0
- Then halt the guest os:
halt
- In the host xen system, edit the /etc/xen/image file to shrink the
ram(optional) and add the swap:
vi /etc/xen/image
...
memory = "64"
disk = [ 'phy:/dev/VolGroup00/ImageData00,xvda,w',
'phy:/dev/VolGroup00/ImageSwap00,xvdb,w' ]
- Create the swap space:
mkswap /dev/VolGroup00/ImageSwap00
- Boot and test guest image, halt when done.
3. Create a new guest from image
--------------------------------
- Create lvm partitions:
lvcreate -L 1G -n ExampleSwap00 VolGroup00
lvcreate -L 3G -n ExampleData00 VolGroup00
- Map the ImageData00 partition:
kpartx -av /dev/VolGroup00/ImageData00
- Use dd to copy to the new partition:
dd if=/dev/mapper/ImageData00p1 of=/dev/VolGroup00/ExampleData00
- Create swap space:
mkswap /dev/VolGroup00/ExampleSwap00
- Unmap the ImageData00 partition:
kpartx -dv /dev/VolGroup00/ImageData00
- Mount the new data partition:
mount /dev/VolGroup00/ExampleData00 /mnt/Data
- In new host, edit the following files and change 'image' to new name:
vi /mnt/Data/etc/hosts
vi /mnt/Data/etc/sysconfig/network
- Also on new host, set the correct ip address and MAC address:
vi /mnt/Data/etc/sysconfig/network-scripts/ifcfg-eth0
- Unmount the new data partition:
umount /mnt/Data
- Copy the 'image' xen configuration file and edit it:
cp /etc/xen/image /etc/xen/example
vi /etc/xen/example
4. Create new Backup Volume Group
---------------------------------
- One time deal:
pvcreate /dev/sda
vgcreate VolBackup00 /dev/sda
vgchange -a n VolBackup00
vgexport VolBackup00
5. Backup Logical Volume
------------------------
- Plugin USB2.0 drive, wait for initialization(check dmesg) and import the
Volume Group:
pvscan
vgimport VolBackup00
- To create and backup to new logical volume:
lvcreate -L3G -n YYYYMMDD_NAME VolBackup00
dd if=/dev/VolGroup00/ImageData00 of=/dev/VolBackup00/YYYYMMDD_NAME
- To export the Volume Group and disconnect the drive:
vgchange -a n VolBackup00
vgexport VolBackup00
6. Notes and questions
----------------------
- My understanding is that using dd to copy lv around is safe as long as
the physical extents are the same, ie lvcreate should create volumes of
exactly the same size.
- Not sure the exact process of increasing a guest lv from the host, but
it should be possible.
- When a new copy of an image is copied, the ssh keys should be recreated
so that they are unique. Haven't looked at how to do that yet.
17 years, 5 months
