QA Process for CentOS 5
Describe general ideas/solutions/requirements here.
Like "what should be tested", "who is going to test it", which QA methods should be used, and so on.
what kind of arch/hardware is available (list: cpu/motherboard/chipset/NIC/video card/...) , what test has been done (install bare bone, along with another distrib/upgrade/...)
Tru available hardware: - 32 bits legacy athlon, Abit KT7A, 3com, plain IDE (PATA) hard drive, cdrom - 32 bits legacy P4 1.8 GHz - new laptop core duo 2 32/64 bits sony SZ3XP/C - vmware server box
RHEL5b2 x86_64 iso available somewhere on centos.org machine?
Tests Completed So Far
What needs to be done for the Beta?
Tru: - plain install (cdrom/dvd) - linux rescue? - network install/upgrade - re-partitionning works? i.e. LABEL=/1 if LABEL=/ exists?
What needs to be done before Release
A lot of this is based on the steps I used for CentOS-2. Much of the testing can and should be automated checking. There is documentation to write. This might be short term pain but should be a long term gain. At the end there will be an answer to 'How do I build my own CentOS-5'. (Not that I encourage this, but it is a measure of safety that if CentOS was to disappear, anyone else could pick it up and run with it).
Per package checks:
- Are there any changes from upstream? If so,
- are they in SVN?
- Will there be ongoing changes to build updates?
- Does the release have the correct centos tags? (or a good reason not to?)
- Are any special build instructions documented?
- Does the package have correct vendor, distro tags?
- Is the package correctly signed?
- Is the package contents the same as the upstream package
- Are any libraries the same as the upstream package (special care is needed for compat packages)
- If there is a compat style package which is not being built from source, how is this handled?
Alternate packages to the upstream versions
Does the replacement have the correct rpm tags to allow up/downgrades of OS releases (ie, Red Hat -> CentOS conversion)
Packages not included from upstream
- Are any packages being dropped from the upstream version? If so why?
CentOS only packages
- Are any packages being added which are not from upstream?
- Where do they come from
- Who will maintain them
Things which need to be tested by developers and agreed upon:
- Contents of /etc/redhat-release
- Contents of /etc/centos-release
- Source of yum config files
Contents of yum config files (mirrors & plugins)
- Source of up2date config files
- Contents of up2date config files (what channels are selected by default?)
- Changes to anaconda
- Changes to runfirst
- Changes to initscripts
Build environment Dependencies in the build environment should not be a large issue but for consistent quality, a definitive build environment would be good. I propose a vmware image which can be easily shared between developers and/or offered as a download to interested users. This may not be suitable for all arch
Some packages will require special build environments, these should be documented.
Fixing upstream bugs For the most part, this is not the goal or job of CentOS. Some bugs however can become show stoppers and may easily be addressed if the package in question is already being altered. One example is changes to the installer. Changes may be made for i586 installs, rebranding etc. Preventing DOS partitions over 2Tb might be an easy fix to add given that the package is already being customised. (That bug may have been fixed already, It is just an example)
Release Candidates Once the core developers are happy that their release meets all the requirements listed above, the beta testers should test the diverse types of installs that people go through and test all the changes from the upstream product. A list of the tests will need to be made and each test checked off as it is performed. Such a list might include
- Install from CD
- Install from DVD
- Install via PXE/HTTP
- Install on i586
- (And combinations, ie does i586 from DVD work?)
- yum update
- default yum config (will it include mirror selection?)
- How does a mortal configure ntp without firstboot?
Do the steps to convert Red Hat -> CentOS work?
- Do the instructions for enabling rpmforge work? (should yum ship with protectbase?)
There was a report of a comps duplication on 3U8 on a couple of the ISOs, with differing comps file -- This needs to be scanned for. Dunno if it has a bug yet