[FrontPage] [TitleIndex] [WordIndex

This is a read-only archived version of wiki.centos.org

"Onboarding" new projects within CI environment



This wiki page will try to list all the mandatory steps to "onboard" new projects allowed to run tests within the CI Environment. Target audience is CI admins (from the CentOS side), having "power" in Jenkins and other machines.

Ensure all users being onboarded are signed up to the ci-users list ( https://lists.centos.org/mailman/listinfo/ci-users )


Jenkins master

There is a jenkins master running on the jenkins.ci.centos.org (and known both internally and externally as https://ci.centos.org too). For a new project, we need to :

Jenkins slave

It was decided to not let projects directly access the central jenkins master, but instead use some Virtual Machines (shared - like slave01.ci.centos.org - , or dedicated, depending on the justification/need) Each time that a jenkins admin create a slave, it gets a specific key to authenticate against the master and announce itself as "online". To know that key, in Jenkins, just use manage/manage nodes/click on the node (like $project-ci-slave01)/status and notice the command listed with -secret

At the agent side, one needs first to download the jenkins slave jar, and start it : (assuming that jenkins remote root dir is /home/$project/)

cd /home/$project
wget https://ci.centos.org/jnlpJars/slave.jar #only once
java -jar slave.jar -jnlpUrl https://ci.centos.org/computer/${project_slave_name}/slave-agent.jnlp -secret ${secret key from above}

TODO : write :

Machine access

Each $project can get access to either the "shared" VM (actually slave01.ci.centos.org) or a dedicated VM for the jenkins slave (TBD on a project basis) All shell access are granted to

Everything can be granted via Ansible playbooks, actually available from the admin.ci.centos.org:/srv/upstream.git/CentOS_CI/ git repo The playbooks/config-users.yml can be used against specific VMs to add the users, once they are added in the local_users_list, to be modified at the hosts/host_vars/jump.ci.centos.org (and VM name) Attention, be sure that nobody has sudo right on the jump host !

Accounts for the Shared VM

Duffy credentials

Projects will have to use the Duffy middleware (see http://wiki.centos.org/QaWiki/CI/Duffy) to be able to request CentOS 5/6/7 i386/x86_64 nodes to run their jobs (nothing will run directly on the slave VM). The dedicated wiki page list how to use the Duffy middleware and request a node, or give it back to duffy (meaning that Duffy will then reinstall it and put it back in the allocation pool) Each project though need to be "known" by Duffy, and so needs :

Duffy uses mysql DB to store all those informations. So a CI admin just needs to add it in the duffy.users db/table (project being replaced with real name):

insert into users values(uuid(),'$project','$project',now(),10);

For the ssh keys, the /home/duffy/user-keys has to contain the $project file containing the ssh pub keys. Duffy will then inject dynamically those public keys into each machine given back through the api call.


Projects can publish artifacts from their builds (e.g. VM images) to a publicly accessible folder (available at http://artifacts.ci.centos.org/). This folder can be created using the storage role in ansible. Each project should be added to the rsync.secrets file (in ansible) using the unique part of their duffy key as a password, for example if the duffy key is 9c67d9c6-b5e2-11e4-b2af-525400ea212d their rsync password would be 9c67d9c6-b5e2. An rsync target should be created using ansible named according to the project.

Projects can sync whatever they wish to their new folder:

RSYNC_PASSWORD='9c67d9c6-b5e2' rsync -av files-in-workdir-to-sync/ <project>@artifacts.ci.centos.org::<project>/

2023-09-11 07:23