Software Collections SIG
- Resource Requirements
- Mailing lists and Communications
- SIG Content
- SIG Membership
- Getting involved
- Roadmap and initial actions
- Task list
- Steps for adding a new collection
- Requesting CBS tags and targets
- CBS tag inheritance
- Building packages for SCLo
- Testing SCLo collections in CI
- The collection naming breakdown
- Further information
Software Collections SIG
SIG Status: Approved
Board Member Helping Bootstrap: JimPerrin
The Software Collections SIG will provide an upstream development area for various software collections and related tools. Developers can build on and extend existing SCLs, so they don't need to re-invent the wheel or take responsibility for packaging unnecessary dependencies.
Act as upstream git repository for SoftwareCollections.org
- Work with developers to accept contributions for newer software collections.
- Establish release management duties to act as gateway to build/sign/distribute packages.
Build and distribute software collections packages via centos.org as well as SoftwareCollections.org
Work with upstreams like OpenShift Origin and foreman to provide compatibility.
Full control of project space on http://git.centos.org - sig-sclo or similar.
Rights for SIG release engineering to SIG branch under the /rpms namespace on http://git.centos.org for builder access / control
Build privileges and related targets for http://cbs.centos.org/koji
Ability to sign packages for distribution (being done after requests like https://bugs.centos.org/view.php?id=9819)
Possible use of http://bugs.centos.org for tracking
Mailing lists and Communications
Work with the CentOS resources and any discussions around that will take place on the centos-devel mailing list. All work related to SCL, user interactions and upstream development will take place on the existing software collections mailing list.
The SIG is usually available also on irc at #centos-devel on irc.freenode.net.
- at least in the beginning we hold (almost) weekly meeting on irc at #centos-meeting on irc.freenode.net
- currently the meeting is held on Wednesday, 4pm UTC
- invitation to the irc meeting is sent to both mailing lists above usually one day before
See Collections List for whole portfolio and EOL information.
For more information about particular Software Collection, search for the collection on http://softwarecollections.org.
The SCLo SIG will have a steering committee and committers. The steering committee will initially consist of:
List of other members is available at https://accounts.centos.org/group/members/sig-sclo.
New committers and members may be added by the steering committee. Steering committee members may interface with the CentOS board via the bootstrapping board member.
Committer privileges, once earned, do not expire unless revoked by the steering committee.
- simple contribution doesn't require access to the dist-git and CentOS Build System (CBS)
have you some bug reports, patches to contribute, other things to discuss, use the email@example.com mailing list until there are some better ways (still working on this)
Some tips for active contributors
- send introduction to the mailing list above (share some basic info about yourself, your intentions within SCLo SIG, which collection you plan to help with)
for gaining access to CentOS Build System, follow steps at http://wiki.centos.org/HowTos/CommunityBuildSystem
Roadmap and initial actions
- Define initial package set and deliver binaries.
- Establish git workflow
- Define criteria for contributions
- Provide documentation for community users as well as contributors
- Establish policy/practice for bug submissions
- Set release cadence and upgrade policies
The following are things that are on the todo list:
prepare release package for RHEL, so RHEL users can install SCLs from sclo namespace
- implement authtag to be able to work with dist-git to store SRPMs for SCLo packages
until it is ready, we plan to import packages manually as requested here: https://www.redhat.com/archives/sclorg/2015-September/msg00005.html
- after dist-git is ready, we want to be able to build SRPMs from SCM, which is not possible from sclo/ namespace, so we'll need some tooling around this, so importing into rpms/ namespace is not manual
- create a sub-page for docker images that are based on SCL
Steps for adding a new collection
Get familiar with Packaging Guide about SCL packaging
Introduce yourself and propose adding a new SCL on the mailing list firstname.lastname@example.org
Request SIG membership by following Getting Involved steps
Request tags as described bellow
Follow steps for building packages
Consider writing tests for your collection and put them into CI
Add entry to http://softwarecollections.org
Update SCL list on SCL List Page
- Do some marketing for your new content on blogs, relevant (upstream) mailing lists or social media
(talk to other SIG members for getting help in any step)
Requesting CBS tags and targets
If you want to create tags and targets for a new collection follow these steps:
Check whether the tag is not already created: http://cbs.centos.org
follow the example to ask for the tags/targets: https://bugs.centos.org/view.php?id=9661
CBS tag inheritance
Sometimes an SCL wants to use packages from another SCL (either for building or for running). The buildroots for particular SCL don't usually contain any other packages than packages from the same SCL, so we need to request specific tags inheritance (e.g. when MongoDB SCL needs Maven SCL for building some of the packages, MongoDB tags need to inherit packages from Maven SCL). To request tags inheritance, a maintainer is required to submit a request like this: https://bugs.centos.org/view.php?id=10525
List of actual tag inheritance is here: https://git.centos.org/blob/sig-core!cbs-tools.git/master/scripts!sigs!sclo!sclo-inheritance.sh
Building packages for SCLo
Currently there is still no dist-git repositories available for the SCLo packages, but lot of package sources are hosted on https://github.com/sclorg-distgit
We can build packages directly like this:
cbs add-pkg sclo7-rh-mariadb100-rh-candidate --owner=sclo rh-mariadb100-mariadb cbs build sclo7-rh-mariadb100-rh-el7 rh-mariadb100-mariadb-10.0.18-1.el7.src.rpm
Once build, the CI should run the sanity tests and then we may tag them to the -testing repository, like this:
cbs add-pkg sclo7-rh-mariadb100-rh-testing --owner=sclo rh-mariadb100-mariadb cbs tag-build sclo7-rh-mariadb100-rh-testing rh-mariadb100-mariadb-10.0.18-1.el7
Putting packages to the testing repository
There is a workflow, that grabs all packages tagged with -testing tag every few hours and make them available at the testing repository at http://buildlogs.centos.org/centos/7/sclo/x86_64/.
In order to happen that way, every collection must be added to the workflow, only for the first time, by creating a ticket like this: https://bugs.centos.org/view.php?id=10260. Then, users may install it by installing and enabling one of the the following repos:
yum-config-manager --enable centos-sclo-rh-testing yum-config-manager --enable centos-sclo-sclo-testing
Releasing the packages
When we want to release the packages, the process is similar as releasing in the testing repository. The packages should be spent at least a week or two -testing tag/repository before release.
- tag builds into -release tag:
cbs add-pkg sclo7-rh-mariadb100-rh-release --owner=sclo rh-mariadb100-mariadb cbs tag-build sclo7-rh-mariadb100-rh-release rh-mariadb100-mariadb-10.0.18-1.el7
The next steps are necessary only if you are releasing a new collection, and should be omitted on updates:
- we need to announce separately for CentOS 6 and CentOS 7
- we must test properly that the steps mentioned in the announcement work
Using EPEL packages for SCLo
This is exclusive for sclo- namespace, i.e. Software Collections not being rebases of RHSCL collections.
It is not possible to use EPEL packages directly for SCLo builds, since only CentOS packages are available in CBS. So, EPEL packages required need to be built in CBS. However, there are more variants how those packages can be built (proposal for now):
We want to avoid dependency hell which would be caused by conflicts of packages in SCLo with EPEL packages (different packages would depend on different versions of the same package).
1. convert EPEL package into SCL and include it in particular SCL
- the technology ensures avoiding with EPEL
2. packaged as normal package
- we must ensure the version is the same as in EPEL
- might be tricky if we want the same version for EPEL 6 and 7 (see mixed cases bellow)
- available in sclo-common tag (sclo7-common-el7 target)
3. rename EPEL package in SCLo repo
- by renaming we ensure conflicts are avoided with EPEL package
- the packages are still common for more SCLs
- all SCLs must be fine with one version
4. mixed cases (e.g. renamed for EPEL 6 and normal package for EPEL 7)
Testing SCLo collections in CI
Each collection has a set of tests to check it meets the expected package lists, is installable and that basic functionality works. These are run automatically on the CentOS CI service against the candidate tags, i.e. newly built packages.
tests for the packages live at https://github.com/sclorg/sclo-ci-tests
test jobs run at https://ci.centos.org/view/SCLo/
Tests can be run locally on or remotely against an EL6/7 host. Helpers are provided in the repository to set up virtual machines under KVM.
New collections should add files to sclo-ci-tests via a pull request (example), adding the collection to the lists, new package lists, tests and a Jenkins config file. Many of the tests and scripts are symlinked, particularly for multiple versions of the same interpreter or collection.
Once a collection is added, the Jenkins Job Builder script (jenkins/run.sh) should be executed to update the jobs in CI. See jenkins/README for more details.
The collection naming breakdown
Software collection's name is cohesive identification that was chosen by author and is not supposed to be broke down to separate parts.
However, since the naming usually follows some guidelines, there is an example of sclo-vagrant1 software collection:
<sclo> : The goal of prefix in the SCL name is *not* to scope or brand the software collection, rather it is to "namespace" it. Prefix "rh-" is used in case of Red Hat's collections available in Red Hat Software Collections product; or "sclo-" in case of collections created by the community.
A prefix in the collection name:
- allows "others" (users in particular) to have a distinguishing name per their requirement
- SCL authors have one "collection name" wherever the collection will appear
- users (open source projects, paying, whatever) have one collection name to target irrelevant of platform
More on the prefix available at: https://www.redhat.com/archives/sclorg/2015-February/msg00022.html
<vagrant> : This is the app which is being shipped as the primary component. A collection will often include many support components. Eg on the php collections, many php extensions and pear modules are also shipped in the same collection, under the same collection name.
<1> : Denotes the major version and the level at which the abi will be tracked. Examples might include ruby197 ruby200 etc