Run Kubernetes on CentOS Atomic Host with Kubeadm
Kubeadm is a tool for bootstrapping Kubernetes clusters that offers a really simple method of getting up and running with a single or multi-node cluster. The CentOS Virtualization SIG provides a kubernetes-kubeadm that can be used with CentOS Atomic Host via rpm-ostree package layering.
1. Configure the Virt SIG kubernetes repo on each host
cat <<EOF > /etc/yum.repos.d/virt7-kubernetes-110-candidate.repo [virt7-kubernetes-110-candidate] name=virt7-kubernetes-110-candidate baseurl=http://cbs.centos.org/repos/virt7-kubernetes-110-candidate/x86_64/os enabled=1 gpgcheck=0 EOF
2. Use package layering to install kubeadm on each host
# rpm-ostree install kubernetes-kubeadm -r
3. SELinux labelling
In order to use kubeadm with selinux in enforcing mode, create and set the context of /var/lib/etcd, /etc/kubernetes/pki, and /etc/cni/net.d:
# for i in {/var/lib/etcd,/etc/kubernetes/pki,/etc/kubernetes/pki/etcd,/etc/cni/net.d}; do mkdir -p $i && chcon -Rt svirt_sandbox_file_t $i; done
4. Initialize the cluster
Start the kubelet and initialize the kubernetes cluster. We specify a pod-network-cidr because flannel, which we'll use in this test, requires it, and we ignore preflight errors because because kubeadm looks in the wrong place for kernel config. Also, we'll start by running kubeadm reset to ensure that we're working with from a clean slate:
# kubeadm reset # systemctl enable --now kubelet # kubeadm init --pod-network-cidr=10.244.0.0/16 --ignore-preflight-errors=all
5. Configure kubectl
Follow the directions in the resulting output to configure kubectl:
# mkdir -p $HOME/.kube # sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config # sudo chown $(id -u):$(id -g) $HOME/.kube/config
6. Add hosts or configure master to run pods
By default, your cluster will not schedule pods on the master for security reasons. If you want to be able to schedule pods on the master, e.g. for a single-machine Kubernetes cluster run:
# kubectl taint nodes --all node-role.kubernetes.io/master-
If desired, join additional nodes to the master using the kubeadm join command provided in the kubeadm init output. For instance:
# kubeadm reset # systemctl enable kubelet --now # kubeadm join cah-1.osas.lab:6443 --token jlav0u.73r45r8votgtwazx --discovery-token-ca-cert-hash sha256:9eac6fdf9e8823bba9399079cb06f41ee2cda6c932d0964707f06a369a2e55a0 --ignore-preflight-errors=all
7. Deploy the flannel network plugin
# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
8. Check on the install
# kubectl get nodes NAME STATUS ROLES AGE VERSION cah-1.osas.lab Ready master 6m v1.10.3 cah-2.osas.lab Ready <none> 2m v1.10.3 cah-3.osas.lab Ready <none> 2m v1.10.3
# kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system etcd-cah-1.osas.lab 1/1 Running 0 10m kube-system kube-apiserver-cah-1.osas.lab 1/1 Running 0 10m kube-system kube-controller-manager-cah-1.osas.lab 1/1 Running 0 10m kube-system kube-dns-86f4d74b45-thjqw 3/3 Running 0 10m kube-system kube-flannel-ds-8nwgm 1/1 Running 0 5m kube-system kube-flannel-ds-bcrxd 1/1 Running 0 5m kube-system kube-flannel-ds-x4kq7 1/1 Running 0 5m kube-system kube-proxy-7q92q 1/1 Running 0 7m kube-system kube-proxy-927dm 1/1 Running 0 10m kube-system kube-proxy-rb6pz 1/1 Running 0 7m kube-system kube-scheduler-cah-1.osas.lab 1/1 Running 0 10m
Run some test apps
# kubectl run nginx --image=nginx --port=80 --replicas=3
deployment "nginx" created
# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE
nginx-768979984b-69jht 1/1 Running 0 19s 10.244.1.2 cah-2.osas.lab
nginx-768979984b-ctbgv 1/1 Running 0 19s 10.244.1.3 cah-2.osas.lab
nginx-768979984b-mwlwx 1/1 Running 0 19s 10.244.2.3 cah-3.osas.lab
# kubectl expose deployment nginx --type NodePort
service "nginx" exposed
# kubectl get svc
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes 10.254.0.1 <none> 443/TCP 40m
nginx 10.254.52.120 <nodes> 80:32681/TCP 14s
# curl http://cah-1.osas.lab:32681
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>