Greetings, The development of DVD auto install test script is coming to the end,it can work both in runlevel 5 and 3. If you are interested, please get the attachments and do some tests,any comments which can help to correct mistakes will be welcome. James Laska gives me lots of help during writing this test.Many thanks.It's very simple to run this test, just run dvd_install.py --help to get how to use it. This test needs a kickstart file, I will attach it for your convenience. Regards Liam _______________________________________________ autoqa-devel mailing list autoqa-devel(a)lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/autoqa-devel

On 03/04/2010 05:29 PM, He Rui wrote: > On Thu, 2010-03-04 at 16:02 +0800, Li Ming wrote: >> Greetings, >> >> The development of DVD auto install test script is coming to the >> end,it can work both in runlevel 5 and 3. If you are interested, please >> get the attachments and do some tests,any comments which can help to >> correct mistakes will be welcome. James Laska gives me lots of help >> during writing this test.Many thanks.It's very simple to run this test, >> just run dvd_install.py --help to get how to use it. This test needs a >> kickstart file, I will attach it for your convenience. >> >> Regards >> Liam >> _______________________________________________ >> autoqa-devel mailing list >> autoqa-devel(a)lists.fedorahosted.org >> https://fedorahosted.org/mailman/listinfo/autoqa-devel > > Wow, it's awesome!! I didn't expect it could be so automatic and visible > like this in such a short time. Well done Liam! I like the "usage" and > "example" part particularly when no input parameters given in the > command. It's a helpful guidance for a new user as me. > > I also found out that it must have the same arch as the currently used > system. Just wondering if it's possible to run both i386 and x86_64 on > one system. > Thanks for your test Hurry,what you said gives me great encouragement. :). At the beginning, I wanted the i386 and x86_64 to be tested on the same host.This will save lots of time for testers.But current KVM does not support that,for example, the i386 host does not support x86_64 guest on F12. But I will play attention on this, and will make it work for both i386 and x86_64 on same host once the KVM supports that, like the VMware,VMware's i386 and x86_64 guest can run on i386 or x86_64 host if the hardware is x86_64.

Greetings, The development of DVD auto install test script is coming to the end,it can work both in runlevel 5 and 3. If you are interested, please get the attachments and do some tests,any comments which can help to correct mistakes will be welcome. James Laska gives me lots of help during writing this test.Many thanks.It's very simple to run this test, just run dvd_install.py --help to get how to use it. This test needs a kickstart file, I will attach it for your convenience.

On Thu, 2010-03-04 at 16:02 +0800, Li Ming wrote: > Greetings, > > The development of DVD auto install test script is coming to the > end,it can work both in runlevel 5 and 3. If you are interested, please > get the attachments and do some tests,any comments which can help to > correct mistakes will be welcome. James Laska gives me lots of help > during writing this test.Many thanks.It's very simple to run this test, > just run dvd_install.py --help to get how to use it. This test needs a > kickstart file, I will attach it for your convenience. Liam and I have had some discussion on further refinements to the test privately. I'll bring them to the list for review from a wider audience. = system_sanity() = After some more thought, I think it makes sense to move the rpm package checks into the autotest wrapper layer (see http://git.fedorahosted.org/git/autoqa.git?p=autoqa.git;a=blob;f=tests/ra...). Then the autotest setup() method will include 'yum -y install' commands for each of the packages your test needs for successful execution. As for just the test script, it is only responsible for checking that the binaries or libraries needed are available. This allows for running the test in varied environments (non-rpm, chroot etc...). So for example, you might try ... * For xorg-x11-drv-Xvfb - just run 'which Xvfb' and examine the return code? Or a helper function to scan through os.environ("PATH") searching for os.path.exists(prefix + 'Xvfb')? * For qemu-kvm - Mimic what wwoods does in install.py. We want to avoid relying on the underlying virt technology as much as possible. Use the libvirt layer when you can (aka virsh and virt-install). The --accelerate option you use with virt-install will ensure you get KVM and not slow qemu. * For dogtail and python-libguestfs ... you can wrap the import for those methods in a try/except to catch ImportError. Then check for pass/fail in your system_sanity() test. For a similar example, see handling ImportError (http://git.fedorahosted.org/git/snake?p=snake;a=blob;f=snake-ks;h=38bf57b...) and checking for module presence (http://git.fedorahosted.org/git/snake?p=snake;a=blob;f=snake-ks;h=38bf57b...).

= Setting DISPLAY = Hmm, I have mixed feelings on this. I'd prefer that this test always do the same thing ... Xvfb. But I see the value in using a running DESKTOP for testing the dogtail functionality. Howabout this ... By default, always use create a new X session using Xvfb, and destroy it when the test is complete. However, your test will accept an option -d,--display parameter so you can tell it to use an existing DESKTOP? Also, don't forget to move the package_check('xorg-x11-server-Xvfb') call into system_sanity() too. I like how you are checking for an existing Xvfb using process_find('Xvfb'). However, as noted above, I think it will be simpler if your test always starts it's own Xvfb ... and kills it when the test has completed. Careful, you're display won't always be DISPLAY=:1.0. You'll need to somehow find the next available DISPLAY value that you can then supply when calling Xvfb (for example, `Xvfb :8`). Or, if a -d,--display value is provided at the command-line, you'll just use that and won't create/destroy you're own X server.

= Graphical vs Serial = The test now boots the DVD graphically and then passes console=ttyS0 so that your subsequent checks can monitor install progress. Something we'll need to think about for the future is doing a fully automated graphical install (using kickstart not dogtail). I don't think it would be too much of a change from what your test does now, I think we'll just need to: 1. change the boot args to "console=ttyS0 console=tty0". This sets /dev/console to tty0 but also sends kernel output to ttyS0. This lets us do a graphical install, but also creates a serial device in the guest. 2. Alter kernel_boot_test() to look for the appropriate kernel lines on the serial device (you'll see when you try ... you just get less output than before). 3. Start a script from %pre to send status to the serial device - Your test script will continue monitoring the serial device for install progress ... but the install continues as a graphical install 4. Later ... investigate using the kickstart options 'interactive' and 'autostep --autoscreenshot' to ensure each UI step gets run.

Added cc+ wwoods for a question below. On Tue, 2010-03-09 at 15:59 +0800, Li Ming wrote: > On 03/04/2010 10:31 PM, James Laska wrote: > > On Thu, 2010-03-04 at 16:02 +0800, Li Ming wrote: > >> Greetings, > >> > >> The development of DVD auto install test script is coming to the > >> end,it can work both in runlevel 5 and 3. If you are interested, please > >> get the attachments and do some tests,any comments which can help to > >> correct mistakes will be welcome. James Laska gives me lots of help > >> during writing this test.Many thanks.It's very simple to run this test, > >> just run dvd_install.py --help to get how to use it. This test needs a > >> kickstart file, I will attach it for your convenience. > > > > Liam and I have had some discussion on further refinements to the test > > privately. I'll bring them to the list for review from a wider > > audience. > > > > = system_sanity() = > > > > After some more thought, I think it makes sense to move the rpm package > > checks into the autotest wrapper layer (see > > http://git.fedorahosted.org/git/autoqa.git?p=autoqa.git;a=blob;f=tests/ra...). Then the autotest setup() method will include 'yum -y install' commands for each of the packages your test needs for successful execution. > > > > As for just the test script, it is only responsible for checking that > > the binaries or libraries needed are available. This allows for running > > the test in varied environments (non-rpm, chroot etc...). So for > > example, you might try ... > > > > * For xorg-x11-drv-Xvfb - just run 'which Xvfb' and examine the > > return code? Or a helper function to scan through > > os.environ("PATH") searching for os.path.exists(prefix + > > 'Xvfb')? > > * For qemu-kvm - Mimic what wwoods does in install.py. We want to > > avoid relying on the underlying virt technology as much as > > possible. Use the libvirt layer when you can (aka virsh and > > virt-install). The --accelerate option you use with > > virt-install will ensure you get KVM and not slow qemu. > > * For dogtail and python-libguestfs ... you can wrap the import > > for those methods in a try/except to catch ImportError. Then > > check for pass/fail in your system_sanity() test. For a similar > > example, see handling ImportError > > (http://git.fedorahosted.org/git/snake?p=snake;a=blob;f=snake-ks;h=38bf57b...) and checking for module presence (http://git.fedorahosted.org/git/snake?p=snake;a=blob;f=snake-ks;h=38bf57b...). > > I added a method command_check() to seach the command, this method like > command "which", it will seach the PATH variable to check whether the > command exists in system, this will support both rpm and non-rpm > install. I also add try/except to handle import errors by referring the > link above > > > = Setting DISPLAY = > > > > Hmm, I have mixed feelings on this. I'd prefer that this test always do > > the same thing ... Xvfb. But I see the value in using a running > > DESKTOP for testing the dogtail functionality. Howabout this ... > > > > By default, always use create a new X session using Xvfb, and destroy it > > when the test is complete. However, your test will accept an option > > -d,--display parameter so you can tell it to use an existing DESKTOP? > > > > Also, don't forget to move the package_check('xorg-x11-server-Xvfb') > > call into system_sanity() too. > > > > I like how you are checking for an existing Xvfb using > > process_find('Xvfb'). However, as noted above, I think it will be > > simpler if your test always starts it's own Xvfb ... and kills it when > > the test has completed. Careful, you're display won't always be > > DISPLAY=:1.0. You'll need to somehow find the next available DISPLAY > > value that you can then supply when calling Xvfb (for example, > > `Xvfb :8`). Or, if a -d,--display value is provided at the > > command-line, you'll just use that and won't create/destroy you're own X > > server. > > I added a -d/--display option for advanced user to enable Xvfb in > runlevel 5.but I prefer the default option is Xorg, because this will > not confuse the new user and let them know what happened during testing. > Now,this script should work for runlevel3(with Xvfb),renlevel5(both Xvfb > and Xorg). > > > = Graphical vs Serial = > > > > The test now boots the DVD graphically and then passes console=ttyS0 so > > that your subsequent checks can monitor install progress. Something > > we'll need to think about for the future is doing a fully automated > > graphical install (using kickstart not dogtail). I don't think it would > > be too much of a change from what your test does now, I think we'll just > > need to: > > > > 1. change the boot args to "console=ttyS0 console=tty0". This > > sets /dev/console to tty0 but also sends kernel output to ttyS0. > > This lets us do a graphical install, but also creates a serial > > device in the guest. > > 2. Alter kernel_boot_test() to look for the appropriate kernel > > lines on the serial device (you'll see when you try ... you just > > get less output than before). > > 3. Start a script from %pre to send status to the serial device - > > Your test script will continue monitoring the serial device for > > install progress ... but the install continues as a graphical > > install > > 4. Later ... investigate using the kickstart options 'interactive' > > and 'autostep --autoscreenshot' to ensure each UI step gets run. > > > > I tested this method today,but I did not get useful data from ttyS0,and > the output stopped when anaconda started. During installation, we can > send key Ctrl+Alt+F3, the information in this /dev/tty3 is useful, but I > did not figure out how to get it via ttyS0 till now. I will keep > exploring this. It seemed to work okay here when testing, but we may be doing things slightly differently. 1. Start up a graphical DVD install $ virt-install --name RATS --ram 1024 --vcpus 1 --os-type linux --os-variant fedora13 --disk path=/var/lib/libvirt/images/RATS.img,size=10,sparse=false,bus=virtio --network network=default --accelerate --vnc --extra-args "console=ttyS0 console=tty0" --cdrom /var/lib/libvirt/images/Fedora-12-x86_64-DVD.iso 2. Proceed to stage#2 install (graphical) 3. From another terminal, connect to the serial console. $ virsh console RATS 4. From the graphical install, change to tty2 and monitor the console ... $ tail -fn100 /tmp/anaconda.log >> /dev/ttyS0 5. From the graphical install, return to tty6 and proceed with install I wouldn't recommend the exact above procedure in your script. This was just to demonstrate the method of using a serial device to communicate install progress back to the virt host. What I'd like to see is some form of the minimon script that support communicating over a device like ttyS0 (instead of the network). Will, do you have any thoughts here?

On 03/10/2010 05:13 AM, James Laska wrote: > On Tue, 2010-03-09 at 08:15 -0500, James Laska wrote: >> Added cc+ wwoods for a question below. >> >> On Tue, 2010-03-09 at 15:59 +0800, Li Ming wrote: >>> On 03/04/2010 10:31 PM, James Laska wrote: >>>> On Thu, 2010-03-04 at 16:02 +0800, Li Ming wrote: >>>>> Greetings, >>>>> >>>>> The development of DVD auto install test script is coming to the >>>>> end,it can work both in runlevel 5 and 3. If you are interested, please >>>>> get the attachments and do some tests,any comments which can help to >>>>> correct mistakes will be welcome. James Laska gives me lots of help >>>>> during writing this test.Many thanks.It's very simple to run this test, >>>>> just run dvd_install.py --help to get how to use it. This test needs a >>>>> kickstart file, I will attach it for your convenience. >>>> >>>> Liam and I have had some discussion on further refinements to the test >>>> privately. I'll bring them to the list for review from a wider >>>> audience. >>>> >>>> = system_sanity() = >>>> >>>> After some more thought, I think it makes sense to move the rpm package >>>> checks into the autotest wrapper layer (see >>>> http://git.fedorahosted.org/git/autoqa.git?p=autoqa.git;a=blob;f=tests/ra...). Then the autotest setup() method will include 'yum -y install' commands for each of the packages your test needs for successful execution. >>>> >>>> As for just the test script, it is only responsible for checking that >>>> the binaries or libraries needed are available. This allows for running >>>> the test in varied environments (non-rpm, chroot etc...). So for >>>> example, you might try ... >>>> >>>> * For xorg-x11-drv-Xvfb - just run 'which Xvfb' and examine the >>>> return code? Or a helper function to scan through >>>> os.environ("PATH") searching for os.path.exists(prefix + >>>> 'Xvfb')? >>>> * For qemu-kvm - Mimic what wwoods does in install.py. We want to >>>> avoid relying on the underlying virt technology as much as >>>> possible. Use the libvirt layer when you can (aka virsh and >>>> virt-install). The --accelerate option you use with >>>> virt-install will ensure you get KVM and not slow qemu. >>>> * For dogtail and python-libguestfs ... you can wrap the import >>>> for those methods in a try/except to catch ImportError. Then >>>> check for pass/fail in your system_sanity() test. For a similar >>>> example, see handling ImportError >>>> (http://git.fedorahosted.org/git/snake?p=snake;a=blob;f=snake-ks;h=38bf57b...) and checking for module presence (http://git.fedorahosted.org/git/snake?p=snake;a=blob;f=snake-ks;h=38bf57b...). >>> >>> I added a method command_check() to seach the command, this method like >>> command "which", it will seach the PATH variable to check whether the >>> command exists in system, this will support both rpm and non-rpm >>> install. I also add try/except to handle import errors by referring the >>> link above >>> >>>> = Setting DISPLAY = >>>> >>>> Hmm, I have mixed feelings on this. I'd prefer that this test always do >>>> the same thing ... Xvfb. But I see the value in using a running >>>> DESKTOP for testing the dogtail functionality. Howabout this ... >>>> >>>> By default, always use create a new X session using Xvfb, and destroy it >>>> when the test is complete. However, your test will accept an option >>>> -d,--display parameter so you can tell it to use an existing DESKTOP? >>>> >>>> Also, don't forget to move the package_check('xorg-x11-server-Xvfb') >>>> call into system_sanity() too. >>>> >>>> I like how you are checking for an existing Xvfb using >>>> process_find('Xvfb'). However, as noted above, I think it will be >>>> simpler if your test always starts it's own Xvfb ... and kills it when >>>> the test has completed. Careful, you're display won't always be >>>> DISPLAY=:1.0. You'll need to somehow find the next available DISPLAY >>>> value that you can then supply when calling Xvfb (for example, >>>> `Xvfb :8`). Or, if a -d,--display value is provided at the >>>> command-line, you'll just use that and won't create/destroy you're own X >>>> server. >>> >>> I added a -d/--display option for advanced user to enable Xvfb in >>> runlevel 5.but I prefer the default option is Xorg, because this will >>> not confuse the new user and let them know what happened during testing. >>> Now,this script should work for runlevel3(with Xvfb),renlevel5(both Xvfb >>> and Xorg). >>> >>>> = Graphical vs Serial = >>>> >>>> The test now boots the DVD graphically and then passes console=ttyS0 so >>>> that your subsequent checks can monitor install progress. Something >>>> we'll need to think about for the future is doing a fully automated >>>> graphical install (using kickstart not dogtail). I don't think it would >>>> be too much of a change from what your test does now, I think we'll just >>>> need to: >>>> >>>> 1. change the boot args to "console=ttyS0 console=tty0". This >>>> sets /dev/console to tty0 but also sends kernel output to ttyS0. >>>> This lets us do a graphical install, but also creates a serial >>>> device in the guest. >>>> 2. Alter kernel_boot_test() to look for the appropriate kernel >>>> lines on the serial device (you'll see when you try ... you just >>>> get less output than before). >>>> 3. Start a script from %pre to send status to the serial device - >>>> Your test script will continue monitoring the serial device for >>>> install progress ... but the install continues as a graphical >>>> install >>>> 4. Later ... investigate using the kickstart options 'interactive' >>>> and 'autostep --autoscreenshot' to ensure each UI step gets run. >>>> >>> >>> I tested this method today,but I did not get useful data from ttyS0,and >>> the output stopped when anaconda started. During installation, we can >>> send key Ctrl+Alt+F3, the information in this /dev/tty3 is useful, but I >>> did not figure out how to get it via ttyS0 till now. I will keep >>> exploring this. >> >> It seemed to work okay here when testing, but we may be doing things >> slightly differently. >> >> 1. Start up a graphical DVD install >> >> $ virt-install --name RATS --ram 1024 --vcpus 1 --os-type linux >> --os-variant fedora13 --disk >> path=/var/lib/libvirt/images/RATS.img,size=10,sparse=false,bus=virtio --network network=default --accelerate --vnc --extra-args "console=ttyS0 console=tty0" --cdrom /var/lib/libvirt/images/Fedora-12-x86_64-DVD.iso >> >> 2. Proceed to stage#2 install (graphical) >> >> 3. From another terminal, connect to the serial console. >> >> $ virsh console RATS >> >> 4. From the graphical install, change to tty2 and monitor the >> console ... >> >> $ tail -fn100 /tmp/anaconda.log>> /dev/ttyS0 >> >> 5. From the graphical install, return to tty6 and proceed with install >> >> >> I wouldn't recommend the exact above procedure in your script. This was >> just to demonstrate the method of using a serial device to communicate >> install progress back to the virt host. What I'd like to see is some >> form of the minimon script that support communicating over a device like >> ttyS0 (instead of the network). >> >> Will, do you have any thoughts here? > > Will and I talked about this a bit in person. Certainly seems like a > worthwhile effort. If we can solve this for the network-less DVD and > CDROM tests, we'd likely use the same mechanism for network-enabled > scenarios like how RATS works. Why not share the code :) I don't think > things will be too different from how minimon works now. Perhaps just > transport will change (not over http or something). I'm not sure yet. > > I'm going to investigate more, but I think one option we should consider > is http://fedoraproject.org/wiki/Features/VirtioSerial > > Thanks, > James > I have some concerns about CD graphical install, use kickstart to install CD, it will not prompt user to swap disc, it hangs up there.

Even we get the method to successfully control the graphical install, we will still be unable to swap CD.Without dogtail,how can we give instruction to anaconda,assuming that anaconda gives some information to swap CD.