The SIG Guide: Purpose

CentOS Special Interest Groups are smaller groups within the CentOS community that focus feature work on or awareness around a topic related to CentOS Linux. This guide is for anyone interested in starting, managing, or contributing to a CentOS Special Interest Group. Below are templates and best practices for each step in the process.

CentOS Properties

accounts.centos.org

URL: http://accounts.centos.org

Description: ACO is the place to ask for access to the Community Build System. Right now these accounts are only active for CBS

Technology: accounts.centos.org is an instance of FAS2

Contact: The CentOS Infrastructure Team (#centos-devel on freenode) or The centos-devel mailing list.

Notes:

git.centos.org

URL: https://git.centos.org

Description: git.c.o is the authoritative repository of package sources that go into CentOS Linux 7. git.c.o also contains repositories for SIG use and sources for other parts of the CentOS Project

Technology: git.c.o is an instance of Gitblit

Contact: The CentOS Sysadmin Team - (#centos-devel on Freenode) or the centos-devel mailing list.

Notes:

cbs.centos.org

URL: http://cbs.centos.org

Description: The Community Buildsystem is where Special Interest Groups build and manage packages for inclusion in their released repos

Technology: CBS is an instance of the Koji buildsystem

Contact: The CentOS Infrastructure Team (#centos-devel on Freenode) or the centos-devel mailing list.

Notes:

  • Scratch builds on cbs.centos.org currently persist and are not expired, unlike the Fedora Project hosted Koji.
  • All image builds, must be run as scratch builds at this time

mirror.centos.org

Describe Properties/mirror.centos.org here.

Devcloud

Describe Properties/Devcloud here.

The SIG Process

Anyone may propose and/or participate in a Special Interest Group, here are some guidelines and tips related to the workings of a SIG.

Proposal

Creating a new SIG requires participation from a member of the CentOS Governing Board, and a SIG must meet some requirements:

Requirements

  1. The topic must be related to CentOS, or a use scenario for CentOS
  2. There must be adequate control and feedback into the CentOS community
  3. Generally all communication as to the work of the SIG should be public, understanding that sometimes a matter may need to be private. In such cases, please consult with the sponsoring Governing Board member
  4. All code produced within the SIG must be compatible with a FOSS license presently used by CentOS
  5. All documentation produced within the SIG Must be compatible with the license of this wiki
  6. SIGs should be mindful of general CentOS directions from the Governing Board
  7. One member of the SIG should be a member of the Governing Board/Devteam

The Proposal Process

  1. Check to see if the topic of collaboration is already covered by an existing SIG
  2. Post an introductory 'RFC' email to the centos-devel mailing list and ask for comments
  3. Find a CentOS Governing board member to join the effort
  4. The board member will:
    1. Request any initial resources be created
    2. List the SIG on the SpecialInterestGroup page in the wiki

Acceptance

The sponsoring member of the Governing board will put the proposal at a regularly scheduled board meeting. The Board will, if the proposal is accepted, give its charter to the SIG to begin its work.

SIG founders should stay in close contact with their sponsor through this process to work out any questions arising from the proposal.

Account Setup

Community Buildsystem

Prerequisites

We provide a set of tools in the CentOS-Extras repository for building SIG packages in the Community Build System. If your development workstation is running CentOS 7:

    yum install centos-packager

Many of these tools will become the building blocks for Centpkg (HowTos/Centpkg).

If your workstation is on Fedora (23/24/25) there is a Copr available:

    dnf copr enable bstinson/centos-packager
    dnf install centos-packager

Step 1: Signing up for an account
  1. Visit the Accounts System

  2. Select 'New Account'
  3. Fill out the form with your information

Step 2: Join a Special Interest Group

Your account will not be active in CBS until you are a member of a SIG

  1. After signing in to accounts.centos.org, choose 'Group List' and locate the SIG you are interested in joining
  2. Apply for membership
  3. Speak with your SIG Chair to have your request approved

Step 3: Generating your user certificate

Your user certificate bundle comes in the form of 3 files:

Filename

Purpose

~/.centos.cert

PEM file with your X509 Client Certificate and Key

~/.centos-ca.cert

The CA Certificate from FAS

~/.centos-upload-ca.cert

The CA Certificate for the lookaside

To generate your certificate you can use the 'centos-cert' tool included in the centos-packager package:

   Usage: centos-cert [OPTIONS]

   Options:
     -h, --help            show this help message and exit
     -u USERNAME, --username=USERNAME
                           FAS Username.
     -n, --new-cert        Generate a new Fedora Certificate.
     -f CERTFILE, --file=CERTFILE
                           User Certificate.
     -v, --verify-cert     Verify Certificate.

If you've signed up with the account name tuser, you can generate your new certificate like this:

    [tuser@myworkstation]$ centos-cert -u tuser -n
    FAS Password: <password_goes_here>

Attention that centos-cert -u tuser -n will request a new certificate, so that will automatically revoke any other certificate you had in the past. If you need to use cbs/koji on multiple machines, just copy the files mentioned above on the other machine.

Step 3.1: Renewing your certificate

/!\ Your user certificate is valid for 6 months. If your certificate expires and you do not renew it after another 4 months, your account will be disabled in accounts.centos.org

To renew your certificate:

    [tuser@myworkstation]$ centos-cert -u tuser -n
    FAS Password: <password_goes_here>

ci.centos.org

Open a Bug
  1. Visit https://bugs.centos.org/

  2. Report an Issue under the buildsys project
  3. Select the 'Ci.centos.org Ecosystem Testing' Category
  4. Include the following information in your report:
    • Your Name
    • The project you are working with
    • Your Desired Username
    • Your Email Address
    • Your ssh pubkey (attach this to the bug please)

Account Approval

Special Interest Group Members: Contact your SIG Chair to comment his/her approval on the bug

Upstream Projects: We will work with you to designate a coordinator to approve new members of your project in ci

Devcloud

Requesting Resources

Content Signing Key

Mailing Lists

IRC Channels

Bug 'Projects' in bugs.centos.org

The SIG Sponsor (board member) handles requests for a SIG Project in https://bugs.centos.org. A 'project' is a collection of bug categories that can be assigned to members of the SIG.

Some basic strategies for assigning categories:

  • By Subtopic: For example, the Atomic SIG might have separate categories for docker, rpm-ostree, and kubernetes each with separate default assignees

  • By Release Component: For example, a SIG might keep separate categories for each release repo shipped on http://mirror.centos.org

The SIG should agree on this at a regularly scheduled meeting and contact the sponsor for next steps.

SIG Bot accounts for CBS

Some SIGs may want to use a bot account for automated builds in CBS from CICO or other infrastructures.

Requirements
  1. The account name is the shortname of the sig (cloud, configmanagement, cloudinstance) etc
  2. The email on the account _must_ be deliverable to someone who can change the certificate in the production environment

The account approval process follows the usual sponsorship model. Be sure to notify one of the ACO admins and they will sponsor the account into the appropriate group.

CBS Tags

To request new tags in CBS open a bug Project: Buildsys Category: community buildsys

Be sure to include the following information:

  • The name of the SIG
  • The SIG project
  • The release string of the project (if any)

Tags in CBS follow the following format: <signame><centos_version>-<project>-<release_string>-{candidate,release,testing}

Example: cloud7-openstack-kilo-testing

SIG:

Cloud

Project:

Openstack

Release String:

Kilo

If the requestor is not the SIG Chair, the Chair should comment on the bug with a +1 or -1 to approve or deny the new tags.

Daily Workings (Meetings)

Running a SIG Meeting can be much easier with an agenda, and a few snippets for calling out things for the minutes.

Meetbot Commands

We'll use some examples from previous CBS meetings.

Command

Description

Sample usage

#startmeeting <name_of_your_meeting>

Tells centbot to begin a meeting

#startmeeting CBS/Infra

#endmeeting

Ends the currently running meeting

#endmeeting

#chair <list_of_nicks>

Meetbot chairs can do administrative commands (like #undo, #agreed, #topic). The person who starts the meeting is automatically a chair

#chair alphacc Arrfab kbsingh

#topic <text>

Starts a topic for discussion. This shows up as a section in the Meeting Summary portion of the minutes

#topic Status Updates

#info <some_text>

Adds a line in the Meeting Summary under the currently active topic

#info First Config Management SIG Meeting: Wed Mar 23 16:00:00 UTC 2016 in #centos-devel

#action <nick_of_someone> <todo_item_they_should_do>

Registers a todo item for the named person

#todo bstinson to follow up on the arm64 builder

#agreed <something_the_group_decided>

This announces a decision of the group, reached by either a vote or by lazy consensus

#agreed The meeting time will be 14h00 UTC

Publishing the Minutes

Contact a member of the Core SIG in #centos-devel soon after your meeting to publish the meeting minutes to the web (Arrfab usually handles these).

After publishing, your minutes will show up under: http://www.centos.org/minutes/

Managing Content

Importing to Version Control

Package sources checked into git.centos.org are in exploded SRPM format. This means that the package working directory should have at least a SPECS/ subdirectory.

New Package (from source)

To work on a package that is not already checked into git.centos.org from upstream sources:

# Let's start on a package called new-package by creating the rpm structure
[bstinson@localhost]$ mkdir -p ~/src/rpms/new-package/{SOURCES,SPECS}

[bstinson@localhost]$ cd ~/src/rpms/new-package/

# Write your spec file. Starting from scratch is ok, but rpmdevtools provides a skeleton
[bstinson@localhost new-package]$ rpmdev-newspec -o SPECS/new-package.spec

New Package (from existing SRPM)

To work on a package that is not already checked into git.centos.org from an existing Source RPM:

# Let's import new-package-1.0.1-2.el7 from a source RPM into an empty workspace
[bstinson@localhost new-package]$ rpm --define "%_topdir `pwd`" -Uvh ~/Downloads/new-package-1.0.1-2.el7.src.rpm

Building In CBS

The current workflow for submitting builds to CBS is to generate a Source RPM in the developer's working directory for submission to the builders. A developer needs an active CBS account and the client tools installed on their workstation (see SIGGuide#CBSAccount).

Generating a Source RPM

[bstinson@localhost new-package]$ rpmbuild --define "%_topdir `pwd`" -bs SPECS/new-package.spec

Short detour to describe Tags and Targets

When building in CBS it helps to understand where to send the sources to start a build, and where the built packages will come out the other end.

Definitions
  • Build Target: Specifies the buildroot and destination of a package for submission. This is specified on the /usr/bin/cbs command line to direct the package into the correct place

  • Tag: A place for packages to be grouped together. Built packages can be added to a tag automatically (e.g. just after a build) or manually placed in a tag with the tag-build operation to /usr/bin/cbs

Build Targets

Build targets are named according to the CentOS version, SIG, Project, Project Version, and Disttag they represent. Taking cloud7-openstack-kilo-el7 as an example:

SIG

cloud

CentOS Version

7

Project

openstack

Version

kilo

Disttag

el7

The build target will be assigned when a new set of tags is requested, and a list of current targets can be found at http://cbs.centos.org/koji/buildtargets

Tags

When a chairperson requests a new set of build tags, the SIG is encouraged to adopt the default workflow:

[build] -> cloud7-openstack-kilo-candidate -> cloud7-openstack-kilo-testing -> cloud7-openstack-kilo-release

The buildroot for new packages includes repositories from:

  • The CentOS Linux base operating system
  • A collection of the latest builds of every package in the candidate tag.

This allows developers to satisfy dependencies by relying on content in the base OS or by making sure a proper version is tagged in to -candidate (either by building it through the corresponding build target, or by issuing the tag-build command to include an already-existing build).

Build!

New Packages:

If your package has not been built before add it to the package list on your destination tags:

[bstinson@localhost new-package]$ cbs add-pkg --owner=bstinson cloud7-openstack-kilo-candidate new-package
[bstinson@localhost new-package]$ cbs add-pkg --owner=bstinson cloud7-openstack-kilo-testing new-package
[bstinson@localhost new-package]$ cbs add-pkg --owner=bstinson cloud7-openstack-kilo-release new-package

Scratch Build:

# Sending a source RPM to a scratch build
[bstinson@localhost new-package]$ cbs build --scratch cloud7-openstack-kilo-el7 SRPMS/new-package-1.0.1-2.el7.src.rpm

Scratch builds will show up in the CBS web interface, and the built RPMS will be downloadable from there, but the package will not be included in any of the CBS tags (or the repos generated from the tags). Scratch builds are used to test functionality before doing a proper tagged build.

Tagged (Normal) Build:

# Sending a source RPM to a build target 
[bstinson@localhost new-package]$ cbs build cloud7-openstack-kilo-el7 SRPMS/new-package-1.0.1-2.el7.src.rpm

Tagged builds will end up in the -candidate tag after it's finished building, and will be available in the buildroot in case other packages depend on it.

/!\ NOTE: There is a restriction in CBS that a package's Name-Version-Release must be globally unique. This means that if anyone else (even from another SIG) has built new-package-1.0.1-2.el7 you must use that build, or build a different version

Frequent Error Messages
  • FAILED: BuildError: package new-package not in list for tag cloud7-openstack-kilo-candidate

    • Be sure you add the package to the list be sure to add it to the package list: See the New Packages section under the Build step

Automated Testing in ci.centos.org

Describe SIGGuide/Content/Testing here.

Releasing to the Mirror Network

There are 2 points in the process where packages can be promoted for wider consumption. Testing content can be distributed to the buildlogs.centos.org content delivery network for developers and CI systems to consume. Release-ready content can be promoted to a SIG-specific directory on mirror.centos.org for consumption by end-users.

centos-release Packages

To deliver SIG packages to users, the yum repo files must be available in a package named centos-release-<component>

For example the cloud sig releases Openstack Liberty using the repo definitions in centos-release-openstack-liberty

Building a centos-release-* package
  1. Decide on the final release location of the repo on mirror.centos.org and build that into a .repo file
  2. If this is the first build, add the new centos-release-mycomponent to the Extras tags
    [bstinson@localhost centos-release-mycomponent]$ cbs add-pkg --owner=bstinson core7-extras-common-candidate centos-release-mycomponent
    [bstinson@localhost centos-release-mycomponent]$ cbs add-pkg --owner=bstinson core7-extras-common-testing centos-release-mycomponent
    [bstinson@localhost centos-release-mycomponent]$ cbs add-pkg --owner=bstinson core7-extras-common-release centos-release-mycomponent
    
  3. Build the package in CBS against the Extras tag
    [bstinson@localhost centos-release-openstack-mycomponent]$ cbs build core7-extras-common-el7.centos centos-release-mycomponent-0.0.1-1.rpm
    
  4. File a bug to request content to be synced to mirror.centos.org (See SIGGuide#MirrorSpace)

  5. Tag the build to the testing tag
    [bstinson@localhost centos-release-openstack-mycomponent]$ cbs tag-build core7-extras-common-testing centos-release-mycomponent-0.0.1-1
    
  6. Do a test yum install of a package from the new repo

  7. When ready, tag the build to the release tag
    [bstinson@localhost centos-release-openstack-liberty]$ cbs tag-build core7-extras-common-release centos-release-mycomponent-0.0.1-1
    

Some Guidelines for centos-release-* packages
  • centos-release-* packages should be built as noarch RPMs

Pushing Test Content to Buildlogs

Submit the data in this format:

<tag>|<destination path>|<dir to run createrepo in>|<destination on buildlogs>

Example:
sclo7-rh-nodejs4-rh-testing/|7/sclo/x86_64/rh/rh-nodejs4/|7/sclo/x86_64/rh/|7/sclo/x86_64/rh/
sclo7-rh-mongodb32-rh-testing/|7/sclo/x86_64/rh/rh-mongodb32/|7/sclo/x86_64/rh/|7/sclo/x86_64/rh/
sclo7-rh-mongodb30upg-rh-testing/|7/sclo/x86_64/rh/rh-mongodb30upg/|7/sclo/x86_64/rh/|7/sclo/x86_64/rh/
sclo7-rh-python35-rh-testing/|7/sclo/x86_64/rh/rh-python35/|7/sclo/x86_64/rh/|7/sclo/x86_64/rh/
sclo7-rh-ruby23-rh-testing/|7/sclo/x86_64/rh/rh-ruby23/|7/sclo/x86_64/rh/|7/sclo/x86_64/rh/
sclo7-rh-ror42-rh-testing/|7/sclo/x86_64/rh/rh-ror42/|7/sclo/x86_64/rh/|7/sclo/x86_64/rh/

  • Once setup, the CBS tag will be checked every 2 hours for new content, and when available, will be pushed to buildlogs.centos.org to the destination repository setup. This requires no manual intervention, beyond you tagging the right rpms to the -testing repositories.

Releasing content to mirror.centos.org

Steps to request space on the mirror network

  1. Open a bug on https://bugs.centos.org in the 'Community Build Service' component of the 'Buildsys' project

  2. Include the following information:
    • SIG Name
    • CBS Release Tag(s) to combine into the final repo
    • Final directory location on mirror.centos.org

Collaborating with other SIGs

SIGGuide (last edited 2017-07-26 17:14:25 by AkemiYagi)