QA Process for CentOS 5

Describe general ideas/solutions/requirements here.

Like "what should be tested", "who is going to test it", which QA methods should be used, and so on.

what kind of arch/hardware is available (list: cpu/motherboard/chipset/NIC/video card/...) , what test has been done (install bare bone, along with another distrib/upgrade/...)

Tru available hardware: - 32 bits legacy athlon, Abit KT7A, 3com, plain IDE (PATA) hard drive, cdrom - 32 bits legacy P4 1.8 GHz - new laptop core duo 2 32/64 bits sony SZ3XP/C - vmware server box

RHEL5b2 x86_64 iso available somewhere on centos.org machine?

Tests Completed So Far

i386 Hardware

x86_64 Hardware

Beta

What needs to be done for the Beta?

Tru: - plain install (cdrom/dvd) - linux rescue? - network install/upgrade - re-partitionning works? i.e. LABEL=/1 if LABEL=/ exists?

Gold

What needs to be done before Release

John's Thoughts

A lot of this is based on the steps I used for CentOS-2. Much of the testing can and should be automated checking. There is documentation to write. This might be short term pain but should be a long term gain. At the end there will be an answer to 'How do I build my own CentOS-5'. (Not that I encourage this, but it is a measure of safety that if CentOS was to disappear, anyone else could pick it up and run with it).

Per package checks:

Are there any changes from upstream? If so,
- are they in SVN?
- Will there be ongoing changes to build updates?
- Does the release have the correct centos tags? (or a good reason not to?)
Are any special build instructions documented?
Does the package have correct vendor, distro tags?
Is the package correctly signed?
Is the package contents the same as the upstream package
Are any libraries the same as the upstream package (special care is needed for compat packages)
If there is a compat style package which is not being built from source, how is this handled?

Alternate packages to the upstream versions

Does the replacement have the correct rpm tags to allow up/downgrades of OS releases (ie, Red Hat -> CentOS conversion)

Packages not included from upstream

Are any packages being dropped from the upstream version? If so why?

CentOS only packages

Are any packages being added which are not from upstream?
Where do they come from
Who will maintain them

Things which need to be tested by developers and agreed upon:

Contents of /etc/redhat-release
Contents of /etc/centos-release
Source of yum config files
Contents of yum config files (mirrors & plugins)
Source of up2date config files
Contents of up2date config files (what channels are selected by default?)
Changes to anaconda
Changes to runfirst
Changes to initscripts

Build environment Dependencies in the build environment should not be a large issue but for consistent quality, a definitive build environment would be good. I propose a vmware image which can be easily shared between developers and/or offered as a download to interested users. This may not be suitable for all arch

Some packages will require special build environments, these should be documented.

Fixing upstream bugs For the most part, this is not the goal or job of CentOS. Some bugs however can become show stoppers and may easily be addressed if the package in question is already being altered. One example is changes to the installer. Changes may be made for i586 installs, rebranding etc. Preventing DOS partitions over 2Tb might be an easy fix to add given that the package is already being customised. (That bug may have been fixed already, It is just an example)

Release Candidates Once the core developers are happy that their release meets all the requirements listed above, the beta testers should test the diverse types of installs that people go through and test all the changes from the upstream product. A list of the tests will need to be made and each test checked off as it is performed. Such a list might include

Install from CD
Install from DVD
Install via PXE/HTTP
Install on i586
(And combinations, ie does i586 from DVD work?)
yum update
default yum config (will it include mirror selection?)
up2date
How does a mortal configure ntp without firstboot?
Do the steps to convert Red Hat -> CentOS work?
Do the instructions for enabling rpmforge work? (should yum ship with protectbase?)
etc

RussHerrold thoughts