Centos7.6 使用kubeadm快速部署kubernetes v1.12.1集群

1.1.服务器规划

1.2.Master节点

1.2.1.确认系统版本，修改系统主机名

1.查看系统版本
[root@iZ2ze7ftggknd1fplnxygqZ ~]# cat /etc/redhat-release 
CentOS Linux release 7.6.1810 (Core)
2.修改主机名
hostnamectl set-hostname kubernetes01
别忘了改/etc/hosts文件
[root@kubernetes01 ~]# cat /etc/hosts
127.0.0.1       localhost       localhost.localdomain   localhost4      localhost4.localdomain4
::1     localhost       localhost.localdomain   localhost6      localhost6.localdomain6
10.5.0.206 kubernetes01

1.2.2.关闭防火墙

systemctl stop firewalld && systemctl disable firewalld

1.2.3.检查selinux是否关闭

[root@kubernetes-master ~]# setenforce 0
setenforce: SELinux is disabled

1.2.4.提前处理路由问题

vim /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1    
vm.swappiness=0
sysctl --system

1.2.5.安装docker-ce, 一定要注意docker-ce版本和kubernetes版本的兼容性！

[root@kubernetes01 ~]# yum -y install yum-utils device-mapper-persistent-data lvm2
[root@kubernetes01 ~]# yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
[root@kubernetes01 ~]# yum install docker-ce-18.06.1.ce
[root@kubernetes01 ~]# service docker start 
[root@kubernetes01 ~]# docker --version 
Docker version 18.06.1-ce, build e68fc7a

1.2.6.安装kubelet kubeadm kubectl注意版本1.12.1！

1.配置yum源
[root@kubernetes01 ~]# cat /etc/yum.repos.d/kubernetes.repo 
[kubernetes]
name=kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
gpgcheck=0
enable=1
2.yum安装
[root@kubernetes-master ~]# yum install kubelet-1.12.1 kubeadm-1.12.1  kubectl-1.12.1

1.2.7.检查版本

[root@kubernetes01 ~]# kubeadm version 
kubeadm version: &version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState:"clean", BuildDate:"2018-10-05T16:43:08Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}
[root@kubernetes01 ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState:"clean", BuildDate:"2018-10-05T16:46:06Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState:"clean", BuildDate:"2018-10-05T16:36:14Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}

该版本kubeadm需要的kubernetes组件的docker镜像版本:
k8s.gcr.io/kube-apiserver:v1.12.1
k8s.gcr.io/kube-controller-manager:v1.12.1
k8s.gcr.io/kube-scheduler:v1.12.1
k8s.gcr.io/kube-proxy:v1.12.1
k8s.gcr.io/pause:3.1
k8s.gcr.io/etcd:3.2.24
k8s.gcr.io/coredns:1.2.2

1.2.8.下载kubernetes相关组件的docker镜像

这里由于国内网络环境的特殊性，选择使用脚本另辟蹊径。
[root@kubernetes-master ~]# vim pull_k8s_images.sh 
#!/bin/bash
images=(kube-proxy:v1.12.1 kube-scheduler:v1.12.1 kube-controller-manager:v1.12.1
kube-apiserver:v1.12.1
etcd:3.2.24 coredns:1.2.2 pause:3.1 )
for imageName in ${images[@]} ; do
docker pull anjia0532/google-containers.$imageName
docker tag anjia0532/google-containers.$imageName k8s.gcr.io/$imageName
docker rmi anjia0532/google-containers.$imageName
done

1.2.9.查看镜像信息

[root@kubernetes01 ~]# docker images 
REPOSITORY                           TAG                 IMAGE ID            CREATED             SIZE
k8s.gcr.io/coredns                   1.2.6               f59dcacceff4        4 weeks ago         40MB
k8s.gcr.io/kube-proxy                v1.12.1             61afff57f010        2 months ago        96.6MB
k8s.gcr.io/kube-scheduler            v1.12.1             d773ad20fd80        2 months ago        58.3MB
k8s.gcr.io/kube-controller-manager   v1.12.1             aa2dd57c7329        2 months ago        164MB
k8s.gcr.io/kube-apiserver            v1.12.1             dcb029b5e3ad        2 months ago        194MB
k8s.gcr.io/etcd                      3.2.24              3cab8e1b9802        2 months ago        220MB
k8s.gcr.io/coredns                   1.2.2               367cdc8433a4        3 months ago        39.2MB
k8s.gcr.io/pause                     3.1                 da86e6ba6ca1        11 months ago       742kB

1.2.10.使用kubeadm部署kubernetes集群master节点

为什么要用kubeadm，因为它是kubernetes原生的部署工具，并且简单快捷，部署出来的集群环境和一个跟二进制文件搭建起来的集群环境基本上没什么区别。

[root@kubernetes01 ~]# cat kubeadm.yaml 
apiVersion: kubeadm.k8s.io/v1alpha3
kind: InitConfiguration
controllerManagerExtraArgs:
  horizontal-pod-autoscaler-use-rest-clients: "true"
  horizontal-pod-autoscaler-sync-period: "10s"
  node-monitor-grace-period: "10s"
apiServerExtraArgs:
  runtime-config: "api/all=true"
kubernetesVersion: "v1.12.1"
[root@kubernetes-master ~]# kubeadm init --config kubeadm.yaml

安装成功后看到提示算是完成了Kubernetes Master的部署了！
Your Kubernetes master has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of machines by running the following on each node
as root:

  kubeadm join 10.5.0.206:6443 --token l8m3xp.y1w2u4psngfqliiv --discovery-token-ca-cert-hash sha256:3338de637d46b4d24e7295be4b4fa9acce099a48ce818e9052a9319fca74260c

在开始使用集群之前，需要以常规用户身份运行以下命令: 上面那段英文！
so
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

1.2.11.健康检查

[root@kubernetes01 ~]# kubectl get cs
NAME                 STATUS    MESSAGE              ERROR
controller-manager   Healthy   ok                   
scheduler            Healthy   ok                   
etcd-0               Healthy   {"health": "true"}   
查看节点状态信息
[root@kubernetes01 ~]# kubectl get nodes
NAME                STATUS     ROLES    AGE   VERSION
kubernetes-master   NotReady   master   13m   v1.12.1

 #####1.2.12.部署网络插件Weave

[root@kubernetes-master ~]# kubectl apply -f https://git.io/weave-kube-1.6
serviceaccount/weave-net created
clusterrole.rbac.authorization.k8s.io/weave-net created
clusterrolebinding.rbac.authorization.k8s.io/weave-net created
role.rbac.authorization.k8s.io/weave-net created
rolebinding.rbac.authorization.k8s.io/weave-net created
daemonset.extensions/weave-net created

再次查看节点状态信息，STATUS已经变了，因为我们部署了网络
[root@kubernetes01 ~]# kubectl get nodes
NAME                STATUS   ROLES    AGE   VERSION
kubernetes-master   Ready    master   21m   v1.12.1

1.2.13查看这个节点上相关pod的状态

[root@kubernetes01 ~]# kubectl get pods -n kube-system -l name=weave-net -o wide
NAME              READY   STATUS    RESTARTS   AGE     IP           NODE                NOMINATED NODE
weave-net-vhs56   2/2     Running   0          6m59s   10.5.0.206   kubernetes-master   <none>

1.2.14部署可视化插件

docker pull anjia0532/google-containers.kubernetes-dashboard-amd64:v1.10.0
docker tag  anjia0532/google-containers.kubernetes-dashboard-amd64:v1.10.0   k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1
docker rmi  anjia0532/google-containers.kubernetes-dashboard-amd64:v1.10.0 
修改yaml文件，后期通过token登陆，这里需要注意暴露了30001端口这在生产环境是极不安全的！
[root@kubernetes01 ~]# cat kubernetes-dashboard.yaml 
kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kube-system
spec:
  type: NodePort
  ports:
    - port: 443
      targetPort: 8443
      nodePort: 30001
  selector:
    k8s-app: kubernetes-dashboard
部署可视化插件
[root@kubernetes01 ~]# kubectl apply -f   kubernetes-dashboard.yaml
查看插件对应的Pod的状态
[root@kubernetes01 ~]# kubectl get pods -n kube-system |  grep dash
kubernetes-dashboard-77fd78f978-w4twx       1/1     Running   0          106m

获取token值登陆可视化控制面板
kubectl -n kube-system describe $(kubectl -n kube-system get secret -n kube-system -o name | grep namespace) | grep token
用token值访问https://10.5.0.206:30001 就可以登陆控制面板了！

1.3.Node节点

使用ansible playbook配合shell脚本安装
cat install_dockerce.sh 
#!/bin/bash
yum -y install yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum -y install docker-ce-18.06.1.ce

cat install_kubectl.sh
#!/bin/bash
# install kubelet and kubeadm and kubectls
yum -y install kubelet-1.12.1 kubeadm-1.12.1  kubectl-1.12.1
# install kube-proxy and pause
images=(kube-proxy:v1.12.1 pause:3.1 )
for imageName in ${images[@]} ; do
docker pull anjia0532/google-containers.$imageName
docker tag anjia0532/google-containers.$imageName k8s.gcr.io/$imageName
docker rmi anjia0532/google-containers.$imageName
done
# join cluster
kubeadm join 10.5.0.206:6443 --token l8m3xp.y1w2u4psngfqliiv --discovery-token-ca-cert-hash sha256:3338de637d46b4d24e7295be4b4fa9acce099a48ce818e9052a9319fca74260c

1.4其它

我遇到的一些问题
节点报错[ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]:或者是从k8s.gcr.io拉取镜像失败，都很好解决！