This page is here for historical reference purposes only.
( This is just a copy/paste from the centos-devel mailing list post, to be used as a starting point. [KB] )
http://www.karan.org/blog/index.php/2010/03/30/autmated-install-testing-for-centos has something to start with, it is very basic but it's something. There have been a few posts about that on the mailing list as well. But I'll give you enough with which to get started.
The tests themselves can be of two types :
a) installer based ( where one would write a kickstart that sets up the machine and does its tests or sets up the tests in %post ). Kickstarts are well, kickstarts. Look here for some examples if you need to : https://nazar.karan.org/cgit/bluecain/tree/
b) post-install scripts. These are the ones where one would do functional tests. Each of these tests is a single script which must - at this time - be self contained. And the only real concern for the testing system is what the exit code was ( 0 = pass, 1 = fail ). They can be written in any language you like. Here is a very simple example:
echo -n 'Test that all updates can be applied to this machine cleanly' yum -d0 -y upgrade > /dev/null 2>&1 if [ $? -eq 0 ]; then echo ': PASS' else echo ': Fail' exit 1 fi
here is another which does something similar :
echo -n 'Test that all 32 rpms can be removed' # only run this test on x86_64 machines! is64=$(uname -m|grep x86_64) if [ "$?" -ne '0' ]; then echo ' Skip' exit 1 fi yum -d0 -y erase *.i?86 > /dev/null 2>&1 if [ $? -eq 0 ]; then echo ' PASS' else echo ' Fail' exit 1 fi # note, this does not imply the machine is still usable after # the remove! need to test that independently
Things that you *do* need to be concerned about :
- kickstarts should make no assumptions about storage ( since we run them on xen and kvm hosts ). If you need specific names and labels, use lvm.
- Getting the tests into the machine post install is not something you need to worry about, thats done by the test harness.
- Write a test for specific functionality to make sure that it works
- Also consider writing tests for things that should fail, eg. 1 must never be equal to 2. or with iptables blocking :22 there should be no way to connect to that port etc.
- ideally, tests will be in filenames that reflect the package or the nature of the test they are hoping to work against. eg: yum_CanBlindUpgrade.sh and yum_CanRemove32bit.sh
- if you are not writing a kickstart to cover the scope of tests, then you need to assume packages being tested are not installed. eg. php tests should start with a 'yum -d0 install php'
- test real functionality first, then work your way down if you want to. Eg:
echo -n 'Test to see if dns works' echo 'nameserver 126.96.36.199' > /etc/resolv.conf # its important we dont hit a dns record with a wildcard like centos.org ping -c 1 www.google.com > /dev/null 2>&1 if [ $? -eq 0 ]; then echo ' PASS' else echo ' Fail' exit 1 fi # implied results: # - network works # - default route is really honoured # - atleast one network link on the machine is working # - ipv4 is functional
So in this case, its testing if DNS is functional ( something a user would care about ) and you get some free test wins ( network works, route etc ).
Questions? Then ask away. Maybe a few people can get together and even start working on a set of wiki pages for this stuff!
The tests are all inside a git repo, but for the first few that you write - how about posting them here to the list and we can take it from there.