[FrontPage] [TitleIndex] [WordIndex] 

This is a read-only archived version of wiki.centos.org

We should be able to expand on this wide enough that it could potentially become a standard checklist that we work through for all Releases

Things we should test for

This is a list of things that I ( KaranbirSingh ) feel we should be testing for. Perhaps we can take this doc, merge it with the tests already in the 5.x and 4.x QA series and come up with a master template. It could also serve, as a byproduct, a qa-testing endpoint. So once we have ticks on each of these, we'd know that the product is in a 'ship' stage.

At the end of this document I will attempt to create a test-matrix and define how we might be able to also create some software ( if nothing public is found ) to manage and track these.

Distro Stuff

Tree

Well, this is the basics really. All these tasks can be done at the buildsystem level, and should be done there. Packages that dont meet these specs should not be moved into the QA repo's at all. However, if required, notes on what packages failed these sanity tests should be posted somewhere.

Branding

During the entire testing process, it should be an aim to list places that are potential upstream Branding points. Even if they are not to be changed, a mention could be made somewhere on the QA Wiki ( private please ? ) about the places where these exist. Special care should be taken to work with the packages not seen before this release.

Also, if possible we should identify places where the branding is revealed as a part of the installed OS. eg. the apache Vendor string, Firefox and Thunderbird have something similar. We want to make every reasonable effort to identify and track these.

The Clustering ( conga ) packages need special attention to check for branding removal and testing. Fabian has offered to help further define these tests and write them up for the QA process.

Installer

We need to make sure that these things are tested for each of the installer interfaces ( via Kickstart, via GUI and via TUI ). We could even automate some of these things via dogtail, at some stage. And each test for the installer should be run, both in text-mode as well as graphical mode, for each of the potential install-media options, including :

And then repeating each of those test mechanisms for both real iron and virtualised environments we care about. At the moment, real iron + vmware + xen + kvm + VirtualBox seem to be the 'cared about environments', but add to taste. Before removing anything, I'd like to request people to initiate a specific focused conversation on the qa-list before doing the removal, to make sure that were not stepping on anyones toes or removing a test-case that others might be relying on.

Package Management

Most of the tests under this could / would be to test any specific changes in yum/rpm and the mirror network behavior.

Existing installs

Once the things above this have been tested, we should then move onto actually deploying the new QA tree into our own testing / production / Desktop machines - or clones of such machines, to see how the new distro impacts issues. If required, we could come up with some basic steps to recommend how people might clone their existing setups for these tests, however there is some merit in also using a different / diverse set of process's to get better coverage.

Functionality

( we need details on what is considered a test for things missing those )

Bug triaging / review

Once the basic package tree's are available, nominate one or maybe two people who are going to be available for a few days to do QA work ( outside work hours even ), to track and identify bugs reported on bugs.centos.org and ensure anything / everything that we are able to fix has already gone in. We will need a process for this, and that would need a bit more thought however, something like this can work right now :

So we end up with 2 bug's that all the people doing qa need to track and can comment on. It would also be good if 1 person was to take ownership of this task, and perhaps have another person along to help with. If its left upto the generic 'someone will take it', its almost certain that 'noone will take the task'.

Pre release Testing

Once the tree's are put into the production mirror.centos.org, run through some basic sanity tests. Exactly what those tests are needs to be defined, but should include:

2023-09-11 07:23