Ideas for GSoC 2015
Ideas for GSoC can be of direct benefit to the CentOS Project, such as tools to help contributors. They can be of benefit to the wider world, such as 'yum' plugins or packaging to help SIG variants. They can be of use to an upstream that is in CentOS Linux and a benefit to all free/open source software and Linux users (as long as a mentor can be found).
Students: Use this page to give you a specific idea or generate an idea for your proposal. You will need to ask on the centos-gsoc list to get in touch with mentors and start conversations around the areas that interest you most.
Mentors: Write up your ideas here, and let us know who is interested in being a mentor; one main mentor is preferred but it's fine to have extra/back-up mentors. It is also good when an idea is associated with a full team.
Contents
- Idea: Develop a kpatch delivery mechanism
- Idea: Extend package management to consume content at a definable confidence level
- Idea: CentOS Cloud Storage components
- Idea: Build on premise Vagrant Hosted Index
- Idea: Extend buildsystem to host triggers
- Idea: Lightweight Cloud instance contextualisation tool
- Idea: yum plugin to handle docker /container images
- Idea: Xen based Hypervisor in a Box
- Idea: Extendable system hardening tool
- Idea: App to trim containers
- Idea: Cloud in a box
- Idea: Source validation and mirroring app
- Idea: Rootfs build factory
- Idea: Clean-up of rpkg
- Idea: Replace sub-shells with library calls in rpkg
- Idea: API on top of reposync for repo sync management
- Idea: Implement and create new documentation toolchain
1. Idea: Develop a kpatch delivery mechanism
Make it possible to a user audience to easily consume live patching in the Linux Kernel.
The CentOS Linux 7 kernel has kpatch support enabled and functional, however there is no easy way for users to consume a central set of kpatch diffs; the aim of this effort is to try to deliver a system like rpm/yum that can be used on the client machines to consume a set of kpatch diffs from a central resource (not unlike a yum repository).
Details
The challenge involves working out a metadata format that encapsulates the kpatch content, and map it to the user architecture as well as kernels installed. Then build a control repository structure to be hosted by the project. Finally the user interface needs to be developed that allows the users to consume these kpatch payloads. At the very least the features that CLI must implement:
- Check for updates.
- Download and test update.
- Setup a patch queue (i.e., if the user is running a kernel that is 3 updates back, the kpatch queue needs to reapply all these 3 updates on system bootup.)
- Remove kpatch payloads and replace with the new kernel as needed (this would involve interfacing with yum as a backend.)
Mentor:
- Name: Corey Henderson
Email: centos-gsoc@centos.org
2. Idea: Extend package management to consume content at a definable confidence level
Facilitate layered components on the baseline platform to define and then deliver their own confidence level in the platform.
Some applications, specially in the infrastructure space, e.g. OpenAFS and the SCST project, deliver kernel enhancing code. The goal of this project idea is to extend the package management layers and build the support mechanisms needed on the buildsystem side to allow the projects/applications to deliver a tested user experience.
Details
Typically, only the base OS is tested to make sure that updates can be applied and that the base OS itself is functional. In many cases, various open source projects layer on top of the base OS with their own components, some of which might replace some functionality in the base OS. An example might be a library or a kernel module. For these projects, offering a yum repo isn't enough, since in many cases they are unable to test the base OS components and their own integration with these components before the components are released.
The goal of this effort would be to extend the package management components in CentOS Linux, to facilitate this level of control. This would involve users getting and consuming content downstream from the external project (or a CentOS Special Interest Group) only after it's been tested, approved, and the external project has validated their code is functional on the new updates. An example workflow here would be:
- CentOS releases the base OS.
- External project (e.g. OpenAFS, SCST) do their development and release code for CentOS Linux.
- Everytime there is an update issued by CentOS Linux, the downstream projects are able to rebuild, test, and validate their code against the new updates.
- Users who have opted into the content hosted from these external projects would then only see the updates once the external projects have done their own testing and approval (this might include builds, CI runs, various release modifications, etc.)
Mentor:
- Name: Howard Johnson
Email: centos-gsoc@centos.org
- SIG: Primarily from the Storage SIG, but others will also benefit
3. Idea: CentOS Cloud Storage components
Deliver a set of cloud specific images that auto build a software defined storage network.
Make it trivial to deploy cloud hosted distributed storage arrays for either on-premise or public hosted clouds.
Details
The deliverable is a set of images, pre-setup and pre-configured, for various cloud vendors and ecosystems that can be deployed trivially, that use auto-discovery, and have pre-included monitoring and availability options. It should then be possible to deliver these from either inside the cloud itself, or from a user machine off-cloud. A typical setup, built around Gluster should be able to:
- Autoscale to consume all storage available per node.
- Be able to scale to reasonable node density (64 would be a good benchmark to hit.)
Be vendor neutral (AMI and OpenStack images for public clouds, and qcow2 files for on premise.)
- There would be a need to develop a single unified CLI that then allows the users to do this deployment, with the following basic functions:
- Deploy an arbitary number of nodes (and have them autodiscover based on a pre-setup criteria.)
- Tear down part of or a complete storage cluster.
- Be able to run status and monitoring reports on a schedule.
- Be easy to use.
- Be well documented.
As an extended goal, the effort could use a non-storage technology specific configuration management interface to deliver the abstraction.
Mentor:
- Name: Lalatendu Mohanty
Email: centos-gsoc@centos.org
- SIG: Storage SIG, Cloud Instance SIG
4. Idea: Build on premise Vagrant Hosted Index
Facilitate groups of people to self host vagrant boxes for their own groups' consumption.
Vagrant is a popular tool used in the developer communities. We would like to develop an application that allows for Vagrant images to be hosted in small registry-like apps, accessible over the web, and develop the tooling needed around it to allow anyone to implement it on site.
Details
Vagrant uses the idea of prebuilt VM images called boxes, and allows developers to get started with developing and testing in those boxes rapidly. The key value being that these boxes can be shared, and are often shared between various teams (sysadmins will often create them, maintain them, then send them out to the developers in the same company, who when done with a feature want to test the instances and then deploy them). However, there is no easy way to do this sharing of VM's other than copying them around from user to user via shared media (USB keys are often used, NFS sometimes, or just scp to copy between machines).
A single app that could be hosted behind a website that allows users to upload boxes, and other uses to search and download these boxs would make the entire process much easier and help improve the overall experience. We would also like to host an instance of this app to allow SIGs and other layered projects in CentOS to publish their own development and testing images for use within their groups.
The basic functionality needed would be:
- User management (register, add some personal details etc., login / logout).
- Ability for users to upload boxes.
- Ability to track different backend providers for the boxes.
- Add tags and keywords to every box uploaded.
- List the boxes availale using different criteria.
- Download and easily consume the boxes from this index. Either from within the vagrant file itself, or by using a wrapper script around Vagrant.
As an additional goal: The ability to host Vagrant files in a similar manner could be developed.
Mentor:
- Name: Ian Mcleod
Email: centos-gsoc@centos.org
- SIG: Cloud Instange SIG
5. Idea: Extend buildsystem to host triggers
Build the ability for Koji and related tooling to execute on conditionals and triggers.
Allow projects that consume CentOS Linux and various components to setup and rely on conditional triggers to execute builds, tests, CI runs, and delivery scripts.
Details
The CentOS Community Build Service (cbs.centos.org) facilitates various build options, but does not integrate with the CI facilities available to the community (and hosted from ci.centos.org). The aim of this effort is to extend Koji (the software used to run CBS) to deliver the following features:
- Ability for builds to be run based on conditionals (i.e., other rpms being built).
- Ability for builds to be run based on time stamp (e.g. run a build job every day, every week, every month).
- Ability to collect and submit a collection of packages as a single entity to the CI infra.
- Trigger CI runs.
- Collect artifacts from the CI runs. For example, on success the resulting RPMS could be designted as 'releaseable' and hosted in a repository that other projects can consume. E.g. Libvirt might want to rerun their CI process everytime a new Qemu build passes tests.
- Be able to do some level of notifications via emails or via URL triggers.
Mentor:
- Name: Mike Mclean
Email: centos-gsoc@centos.org
- SIG: Core SIG
6. Idea: Lightweight Cloud instance contextualisation tool
Develop a lightweight, zero dependancy cloud instance contextualisation tool.
Virtual Machines running in the cloud (or virtualised environments) need some way to have their instance contextualised to run within that environment. This allows providers to build a single instance image that can be used in different places.
Details
All existing cloud instance contextualisation tools are heavyweight, have long dependancy chains, are hard to customise and extend, and are slow to execute. The aim here is to deliver something written in 'C' or 'Go' so it has no dependancy and delivers on the key basic functionality that is needed by a majority of situations. The functions include:
- Setup a user.
- Setup a ssh key for the user.
- Set a password, or block passwords from being used.
- Manipulate the firewall as needed.
- Be able to resize the disk backing image, based on disk blocks available and policy.
- Be able to do some basic notifications.
- Be able to run an arbitary script.
The basic formats this tool needs to be able to consume:
- AWS EC2 Metadata over network.
- Generic Config drive.
- Google Compute Engine metadata.
The main focal points are to deliver the basic functionality with the fastest and the most lightweight solution possible.
Mentor:
- Name: Colin Walters
Email: centos-gsoc@centos.org
- SIG: Cloud Instance SIG, Atomic SIG
7. Idea: yum plugin to handle docker /container images
A single yum command that can download and install container images.
In this effort, we would setup a server side repository of pre-built and pre-configured images that are consumable directly under Docker; we would then write the yum code needed to expose that repository to the users and a few simple commands to download and install the images users want.
Details
The problem space we are trying to address here is that at the moment, there is no way to easily mass build and promote container images (a problem easily solved in the RPM world with yum and the server metadata for RPM/yum repositories). Given that the mechanism to facilitate this already exists in yum, it makes most sense to just consume that interface along with the backend support, and run it for container images. We need to be able to implement 'yum list container', 'yum install container', and 'yum erase container'. Due to the nature of how containers work, we will not need to implement a 'yum upgrade container' command since the instance would update using its own yum mechanism inside the container.
This extends just delivering dockerfiles in an RPM, it enables existing system management tooling to work with an evolving container set, by being treated much like an RPM repository would be.
Mentor:
- Name: Jim Perrin
Email: centos-gsoc@centos.org
- SIG: Cloud Instance SIG
8. Idea: Xen based Hypervisor in a Box
A Single installer ISO that delivers a functional Xen4 stack on CentOS.
The Xen4CentOS effort already builds packages for using Xen dom0 on CentOS-6, this effort would be to extend that into delivering a consumeable ISO that does not need CentOS to be preinstalled on the machine. The ISO would also contain CentOS-5 and CentOS-6 vm images that can be instantiated easily.
Details
Consuming Xen as a hypervisor on CentOS-6 requires a few cycles, including needing an existing CentOS-6 installed base, and requires a running kernel upgrade that can cause corner case issues hard to deal with. For example, firmware versions for drivers changing and device naming changing in a way that the system is hard to recover post install. The aim of this task is to build an installer instance that uses the same kernel as the xen4centos repositories, and sets up networking as well as storage on a machine making it possible to consume Xen out of the box. The storage should be setup to consume either filebacked or LVM backed storage for the VM's, and the network should be setup to be either self-hosted or completely bridged.
For self hosted network we would create a local bridge, managed via dnsmasq and setup to NAT all VM traffic, much like how the default libvirt install is. For fully bridged, the installer should bridge the selected physical network device and ensure that Xen creates all virtual network interfaces for that bridge. In this scenario we would not do DHCP, NAT, or any other network management.
A key followup goal from this task should be to facilitate upstream additions and downstream extensions on user scenarios. E.g. it should be possible for OpenStack or OpenNebula efforts to consume this hypervisor. We don't need to deliver the mechanics for these extensions, just ensure that the work we do is open and extensible easily.
Mentor:
- Name: George Dunlap
Email: centos-gsoc@centos.org
- SIG: Virtualisation SIG
9. Idea: Extendable system hardening tool
A single tool to audit and implement system hardening recipes.
Create an application that can be easily deployed on CentOS machines, which can consume central hosted metadata and recipes on hardening and securing the CentOS Linux install.
Details
This project is to build an easily extendable tool that can implement policy and recipes to secure a machine, and if needed audit the machine against a given set of recipes. This makes it easier for a user-contributed, community-driven, security-focused Special Interest Group to come together and help the larger, newer userbase implement best practises for various roles. The tool should be able to communicate with a central server, and access information from the RPM metadata repositories, as well as custom recipes. This might be a wrapper around a config management tool.
The aim would be to deliver the tooling needed, the client side application, and a reference set of recipes. Examples of some recipes might be:
- Role: Webserver, ensure no firewall ports other than those needed by the webserver are enabled.
- Role: MySQL Server, ensure mysql is not accessible as user 'root' without a password, ensure the remote ports are closed.
Mentor:
- Name: Fabian Arrotin
Email: centos-gsoc@centos.org
- SIG: Core SIG
10. Idea: App to trim containers
Trim content inside a container to just be enough to run the app it is meant to host.
Containers are typically used to run applications, but the container images delivered are still done as complete machine instances, in many cases carrying redundant libraries, docs, and other content that is never used from the container itself and has no impact on the application being run from the container. We want to lose all that excess content.
Details
Linux applications have a predictable dependancy tree, and it's possible to trim everything else away from the container images that are going to used to host and run these apps. Typically, this could mean 80 - 85% savings on disk sizes of these container images, and also dramatically reduce the resources need to ship them around over the Internet.
We would need to define scope, define the goals, and also identify automation opportunities in order to deliver on this.
Mentor:
- Name: Adam Miller
Email: centos-gsoc@centos.org
- SIG: Virtualisation SIG, Atomic SIG
11. Idea: Cloud in a box
Preconfigured, assumption driven, easily installed single node cloud controller for development and Proof of Concept testing.
Cloud infrastructure is typically hard to install and requires quite a lot of early study. Depending on how well a user is able to prepare the result of a cloud evaluation can vary. The goal would be to accumulate best practises into a single location, and then deliver a live image (either as an ISO or a qcow2 file), that implements these ideas, ready to be instantiated or installed as a single machine cloud in a box.
Details
Installing a cloud controller requires a fair bit of prep, which a lot of users prefer to skip when evaluating a cloud infrastructure. Having a single ISO or a Live Image that pre-implements best practices and sets up a usable single node would make it simpler and faster to productivity for people looking at evaluating or running simple workloads on a single node cloud.
As an additional goal, the same setup could be extended to be hosted on an installer image, allowing people to expand their cloud easily onto multiple nodes.
Mentor:
- Name: Rich Bowen
Email: centos-gsoc@centos.org
- SIG: Cloud SIG
12. Idea: Source validation and mirroring app
Mirror, audit, and trigger tasks based around source changes in arbitary locations upstream.
Developers prefer to write code in areas they are most confortable in, using tools they are most comfortable with. We need the ability to manage code hosted in git.centos.org, build a relationship with the upstream content it feeds from (either github.com or bitbucket.org or any arbitary place), and mirror content as needed. Sometimes this might be a two-way process where content from git.centos.org is made available on various other places, making it easier for contributors to interact and develop. The goal would be to develop the applications needed to make this happen, but also to validate content and ensure that no changes take place in past history.
This project also needs to build functionality that allows further actions to take place depending on the code that is changing and what the intended results would. For RPM payloads, we need to track the tarballs, the spec files, and the patches - with every change triggering a build and CI run. For kickstart and metadata around images, we need to ensure that content validates for syntax etc., triggers relevant builds, and collect a diff of content changes included in the resulting artifacts.
Mentor:
- Name: Karanbir Singh
Email: centos-gsoc@centos.org
- SIG: Core SIG
13. Idea: Rootfs build factory
ARMv7 and other platforms require rootfs images, but there is no easy way to build them.
Deliver a set of scripts and tools needed to unpack stock CentOS Linux Arm 32bit images and rebuild / rebundle them based on the target machine the image needs to be dropped into.
Details
Every ARM32 board manufacturer has their own device tree, bootloader, firmware, and kernel expectations. It's increasingly difficult for the CentOS Project, or the community around it, to deliver a rootfs image that targets every possible board. However, having a tool that is able to consume some baseline metadata and rebuild a generic rootfs image to suit the boards needed would reduce the user barrier to entry and also make it much easier for people to rebase their images after critical updates in the kernel and the bootloader stack.
Mentor:
- Name: Johnny Hughes Jr
Email: centos-gsoc@centos.org
- SIG: Core SIG
14. Idea: Clean-up of rpkg
Clean-up and refactoring of rpkg for easier customization.
The rpkg application is used by CentOS Project to build community-supported software packages. This project focuses on cleaning-up and refactoring the code to organize by components for easier customization and sub-classing.
Details
CentOS Project uses a tool derived from the upstream rpkg called centpkg. This project would refactor methods to be organized by components, such as the buildsystem, SCM layout on the filesystem, and local build tools. The purpose of making these components is to make it easier to customize and sub-class from the main rpkg methods.
Part of this project will be to work with the rpkg upstream project to work out a good plan so that the changes can be carried in the upstream project for all to use.
Mentor:
- Name: Brian Stinson
Email: centos-gsoc@centos.org
- SIG: Community Build System (CBS) SIG
15. Idea: Replace sub-shells with library calls in rpkg
Clean-up and refactoring of rpkg for easier customization.
The rpkg application is used by CentOS Project to build community-supported software packages. This project focuses on replacing sub-shells with proper library calls, especially for git.
Details
CentOS Project uses a tool derived from the upstream rpkg called centpkg. This project focuses on general cleanup and replacing a lot of the subshells with proper library calls, especially the git operations.
Part of this project will be to work with the rpkg upstream project to work out a good plan so that the changes can be carried in the upstream project for all to use.
Mentor:
- Name: Brian Stinson
Email: centos-gsoc@centos.org
- SIG: Community Build System (CBS) SIG
16. Idea: API on top of reposync for repo sync management
Build an API on top of reposync to manage and delegate repo synchronization.
Have an API on top of reposync to manage and delegate repository synchronization to the special interest group (SIG) users.
Details
CentOS SIGs need to be able to run reposync for their packages in the CentOS package repository without having to request administrative access or others to do the re-synchronization manually.
The API should support:
- Basic reposync configuration creation/modification.
- Workflow ( force source and destination, who, when, etc.)
- Pluggable auth (x509, pass, kerberos).
- ACLs (ldap, file, etc.)
- No external dependencies.
libpq as a queue to allow reuse of PostgreSQL. (The project does not want to maintain another daemon.)
Mentor:
- Name: Thomas Oulevey
Email: centos-gsoc@centos.org
- SIG: Community Build System (CBS) SIG
17. Idea: Implement and create new documentation toolchain
Implement and partially create a toolchain that lowers the barriers to contributing.
The CentOS Project needs more short-form contributions of content that focuses on how-to do things on top of a CentOS Linux installation, with a method to push changes in to appropriate and related upstream open source projects.
Details
There is a lot of content scattered across the Internet on how to do things with CentOS Linux. The goal of this toolchain is to make it easy for people to contribute new, short-form content articles to the Project with an ability to push them outward to relevant upstream projects.
The toolchain should:
- Synchronize content from git.centos.org to github.com/CentOS.
Enable authors to write using a common markup format such as MarkDown or AsciiDoc.
- Convert from the markup to relevant target platforms
- Some target formats are HTML, PDF, EPUB, and so forth.
- Might include converting to another source used by an upstream project.
- Tag content in git.centos.org to be associated with the relevant upstream projects.
- Write up documentation on how to use the system.
Authors can contribute via GitHub and a pull request.
- Authors can edit, modify, and fork for new content from a set of examples in github.com/CentOS.
Project maintainers (CentOS Docs SIG) can accept content from GitHub in to git.centos.org, tagging it appropriately for related upstreams.
- Content can be broken up and submitted upstream to relevant projects.
Mentor:
- Name: Karsten Wade
Email: centos-gsoc@centos.org
- SIG: Documentation
Category:GSoC2015