Contents
Updates At Bottom of Document as Needed
CentOS 7.9.2009 Rough Status Page
Red Hat Enterprise Linux 7.9 was released on 2020-09-29. We are currently building the CentOS-7.9.2003 release from those sources.
One major issue with this release is that the aarch64 and power9 (ALT) architectures are no longer being done as part of RHEL-7.9. The CentOS project is going to TRY to maintain our aarch64 and power9 arches in 7.9.2009 and throughout the entire CentOS 7 Linux life. Since the kernel-alt is no longer being maintained by Red Hat for RHEL-7.9 we are going to try to backport the currently maintained EL8 kernels for ppc64le and aarch64 into CentOS-7.9.2009.
General Steps
Steps needed to make a CentOS rebuild usually follow these steps:
- Red Hat makes the sources available.
This used to be done via src.rpms but 7 changed to git repos (at https://git.centos.org/)
- The pushes from Red Hat happened on 2020-09-29.
- CentOS needs to look at the sources and determine what is needed to build these packages
- There are always packaging loops which need to be worked out. You need golang-(X-1) to build golang-X but golang-(X-1) doesn't exist yet.. how do you build a version and break the loop (ie - bootstrapping new packages)?
- CentOS needs to go through the source code and find out all the places where debranding is needed.
- No you can't just "sed s'/Red Hat/CentOS/' (someone always offers that)
- There are times when you do replace and times when you don't.
- Builds can start occurring through the system
- This usually requires a multi-loop as you do a build to get started
- Then rebuild that core with itself
Then add some more and possibly repeat 1 && 2 a couple of times.
- Then you can start building out the rest of the packages
- The installer usually takes a certain amount of work to get packaged together.
- Some things need additional patches
- Some things just need to be ordered correctly
- Some things like shim signatures for Secure Boot take outside review by signing authority
- Continuous Release (CR) Repository
The Continuous Release (CR) Repository is where updates that will be CentOS 7.9.2009 will initially be released to CentOS Users. Here is a detailed explanation of what the CR repo is.
- There is QA work for the CR rpms. Once we are satisfied with CR, we will release CR.
- Final ISO / Install Tree and Final QA work can begin
- Full trees for all arches will be assembled from the new CR packages and any older packages that were already part of the 7.8.2003 CentOS Linux release.
- The installer tree and the install media are created.
- Full QA testing of the Install Tree, Install Media is conducted.
- Zero Day updates and the extra repositories (extras, centosplus, all Special Interest group content, etc.) for the new release are compiled.
- These new repositories are released to the QA team for testing.
- A final build is released
The above is a 'you asked the people trying to build the train when it will arrive' guide. That said, every release is different and the order and additional steps get found and added each time.
Current Build Status
|
NOTE : I must build llvm-toolset-9.0 and rust-toolset-1.41 and build updates for devtoolset-7, devtoolset-8, and devtoolset-9 before I can start the main builds (some packages are built against each of those, even though CentOS does not release them in base) |
In the above roadmap, we are currently somewhere between steps 1 and 2.
|
The current status of all builds can be seen on CentOS Build Logs Server, look in the c7.2009.* directories |
Major issues hit that require solving during this build:
1. Several Gnome packages have a Hidden BuildRoot requirement for meson. Had to build and add meson the c7.2009.00 directories for all arches. Complicating this issue .. el7 requires meson-0.45 to build .. el7-extras requires meson-0.47 to build. Meson is not in base el7.
2. webkitgtk4 requires EOL'ed devtoolset-7 combined with specifically llvm-toolset-7, needed a unique buildroot/configuration specifically for that package on all arches. Neither of these toolsets are in base el7.
3. exiv2 requires llvm-toolset-7, needed a unique buildroot/configuration specifically for that package on all arches. This is not in base el7.
4. cockpit requires go-toolset-1.12. This is not in base el7.
5. The standard CentOS debranding for ipa and sos have failed, they need to be refactored.
6. We need to convert the aarch64 and ppc64le (power9) kernel, kernel-4.18.0-193.19.1.el8_2, to el7 and get it to build.
7. We had an issue with the ppc, ppc64, ppc64le gcc hanging on build which we solved using rngd on the builders.
8. We need a new set of shim packages (for x86_64, the kernel, shim-*, and grub2 packages). This requires some packages be built and submitted to a Trusted Certificate Authority for signature. The CentOS Project has no control over the amount of time it will take to get back signed keys or the priority of that remote CA.
9. The devtoolset-7-gcc needs a patch for arm building (Patch)
Current Status:
We are validating current builds for CR and getting ready to push all GA items to CR except items that need secureboot signatures (kernel, shim, grub2, fwupd) and also not centos-release, anaconda which need to be tested / part of the full tree and not CR
[10/15/2020] We have pushed a signed CR tree for x86_64 to QA testing .. testing has started. ppc64le tree in progress and it should also be done by the end of the day. aarch64 tree will be started after that. See the BIG 3 note in the AltArch section of this page.
[10/20/2020] We have released a signed CR tree for x86_64 , aarch64, and ppc64le now live. The other 3 arches (i386, ppc64, armhfp) still in progress. Announcements here:
[10/23/2020] New altarch kernels for aarch64 and power9 repositories (kernel-4.18.0-193.28.1.el7.src.rpm) released to CR .. Full GA CR tree for power9 released. armhfp GA CR tree pushed to QA for testing. Currently working on i686 and ppc64 GA CR trees.
[10/26/2020] Full GA CR trees for All Arches complete. Began working on all Install trees.
[10/28/2020] Completely working x86_64 , i386 , aarch64 lorax installer trees for GA 7.9.2009/os trees created and on the way to QA for testing. Other arches (ppc64, armhfp, ppc64le, power9) in progress now. For those remaining arches, trees created .. just running the lorax to create the installers. Note: armhfp and aarch64 uboot installs do not use lorax or the anaconda installer .. those are also being worked on separately.
[10/30/2020] Good install trees for all arches. Now working on Cloud Images and Installer ISO for all arches.
[11/03/2020] All install media and cloud images have been generated and have been sent to QA for testing. All isos have undergone basic testing, as have cloud images. Still doing some advanced iso installer tests.
[11/12/2020] Released
Architectures
Main architectures
The following arches are built automatically in parallel.
x86_64 (w/ i686 multilib) Owner: CentOS Core SIG
AltArch
ppc64le (Little Endian) Owner : CentOS SIG-AltArch
aarch64 (ARM64, ARMv8) Owner : CentOS SIG-AltArch
armhfp (ARM32, ARMv7 - aka armv7hl) Owner : CentOS SIG-AltArch
i386 ( x86 , 32 Bit - aka i686) Owner : CentOS SIG-AltArch
ppc64 (Power PC Big Endian) Owner : CentOS SIG-AltArch
|
NOTE : Because many SIGs build content for x86_64, ppc64le, and aarch64 .. those are the BIG 3 (even though 2 of them are altarch). We must release CR for the BIG 3 at the same time so as not to break the Community Build System CBS |
Current Timeline
Item |
Started |
Ended |
Status |
Sources pushed to CentOS Git |
2020-09-29 |
2020-09-30 |
Complete |
Source code evaluation |
2020-09-29 |
2020-10-02 |
Complete |
Debranding patches added |
2020-10-01 |
... |
Complete |
Build Loop 0 |
2020-10-01 |
2020-10-02 |
Complete |
Build Loop 1 |
2020-10-02 |
2020-10-15 |
Complete |
Build Loop N |
2020-10-15 |
2020-10-26 |
Complete |
QA CR |
Mid October |
2020-10-20 |
Complete |
CR Release (BIG 3) |
Mid to Late October |
2020-10-20 |
Complete |
CR Release (Rest of Arches) |
Mid to Late October |
2020-10-26 |
Complete |
Installer work |
2020-10-28 |
2020-10-30 |
Complete |
ISO Images, Cloud Images |
2020-10-30 |
2020-11-08 |
Complete |
Updates, Extras, Plus Repos |
2020-11-04 |
2020-11-08 |
Complete |
Final QA work |
2020-11-08 |
2020-11-09 |
Complete |
Release work |
2020-11-09 |
2020-11-12 |
Complete |
Updates
Definitions
- DONE - Step is completed
- Ongoing - Work is progressing
- Not started - This step may require work from another step to reach some completion or requires time from a person who is working on other parts.
- Build loop 0 - Getting an initial set of packages together which can then build more packages
- Build loop 1 - Using the smallest set of packages to build a larger set which can then build more packages
- Build loop N - I really don't know how many loops this is going to be.. so we will use N nomenclature.
- Installer work - there is usually some work needed to make the installer actually work with various things like secure boot or different fonts or 'oh they fixed that in an upgrade.img., let us just short cut and put that here.'