通过kubeadm快速部署K8S集群
kubeadm是官方社区推出的一个用于快速部署kubernetes集群的工具。
这个工具能通过两条指令完成一个kubernetes集群的部署：
# 创建一个 Master 节点
$ kubeadm init
# 将一个 Node 节点加入到当前集群中
$ kubeadm join <Master节点的IP和端口 >
-
一台或多台机器，操作系统 CentOS7.x-86_x64
-
硬件配置：2GB或更多RAM，2个CPU或更多CPU，硬盘30GB或更多
-
可以访问外网，需要拉取镜像，如果服务器不能上网，需要提前下载镜像并导入节点
-
2. 准备环境
| IP | |
|---|---|
| master | 192.168.1.1 |
| node1 | 192.168.1.2 |
| node2 |
# 关闭防火墙 systemctl stop firewalld systemctl disable firewalld # 关闭selinux sed -i 's/enforcing/disabled/' /etc/selinux/config # 永久 setenforce 0 # 临时 # 关闭swap swapoff -a # 临时 sed -ri 's/.*swap.*/#&/' /etc/fstab # 永久 # 根据规划设置主机名 hostnamectl set-hostname <hostname> # 在master添加hosts cat >> /etc/hosts << EOF 192.168.44.146 k8smaster 192.168.44.145 k8snode1 192.168.44.144 k8snode2 EOF # 将桥接的IPv4流量传递到iptables的链 cat > /etc/sysctl.d/k8s.conf << EOF net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 EOF sysctl --system # 生效 # 时间同步 yum install ntpdate -y ntpdate time.windows.com
3.1 安装Docker、配置加速器
#添加docker官方的repo
yum-config-manager –add-repo https://download.docker.com/linux/centos/docker-ce.repo
#安装docker
yum install -y docker-ce-18.09.9-3.el7
#配置镜像加速器
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
"registry-mirrors": ["https://lcs5rvt6.mirror.aliyuncs.com"]
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker
#查看docker信息检查加速器地址
sudo docker info
Registry Mirrors:
https://lcs5rvt6.mirror.aliyuncs.com/
Live Restore Enabled: false
Product License: Community Engine
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
$ yum install -y kubelet-1.18.0 kubeadm-1.18.0 kubectl-1.18.0
$ systemctl enable kubelet && systemctl start kubelet
$ kubeadm init \ --apiserver-advertise-address=192.168.1.1 \ --image-repository registry.aliyuncs.com/google_containers \ --kubernetes-version v1.18.0 \ --service-cidr=10.96.0.0/12 \ --pod-network-cidr=10.244.0.0/16 #192.168.1.1是apiserver的地址
#上述这步骤解释：
下载k8s的组件images，稍等，可以再开一个终端查看images变化。
1 [root@k8s-master ~]# kubeadm init \ 2 > --apiserver-advertise-address=192.168.1.1 \ 3 > --image-repository registry.aliyuncs.com/google_containers \ 4 > --kubernetes-version v1.18.0 \ 5 > --service-cidr=10.96.0.0/12 \ 6 > --pod-network-cidr=10.244.0.0/16 7 W0304 10:07:21.552884 67589 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io] 8 [init] Using Kubernetes version: v1.18.0 9 [preflight] Running pre-flight checks 10 [WARNING Service-Docker]: docker service is not enabled, please run 'systemctl enable docker.service' 11 [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/ 12 [preflight] Pulling images required for setting up a Kubernetes cluster 13 [preflight] This might take a minute or two, depending on the speed of your internet connection 14 [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' 15 [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" 16 [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" 17 [kubelet-start] Starting the kubelet 18 [certs] Using certificateDir folder "/etc/kubernetes/pki" 19 [certs] Generating "ca" certificate and key 20 [certs] Generating "apiserver" certificate and key 21 [certs] apiserver serving cert is signed for DNS names [k8s-master kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.1.1] 22 [certs] Generating "apiserver-kubelet-client" certificate and key 23 [certs] Generating "front-proxy-ca" certificate and key 24 [certs] Generating "front-proxy-client" certificate and key 25 [certs] Generating "etcd/ca" certificate and key 26 [certs] Generating "etcd/server" certificate and key 27 [certs] etcd/server serving cert is signed for DNS names [k8s-master localhost] and IPs [192.168.1.1 127.0.0.1 ::1] 28 [certs] Generating "etcd/peer" certificate and key 29 [certs] etcd/peer serving cert is signed for DNS names [k8s-master localhost] and IPs [192.168.1.1 127.0.0.1 ::1] 30 [certs] Generating "etcd/healthcheck-client" certificate and key 31 [certs] Generating "apiserver-etcd-client" certificate and key 32 [certs] Generating "sa" key and public key 33 [kubeconfig] Using kubeconfig folder "/etc/kubernetes" 34 [kubeconfig] Writing "admin.conf" kubeconfig file 35 [kubeconfig] Writing "kubelet.conf" kubeconfig file 36 [kubeconfig] Writing "controller-manager.conf" kubeconfig file 37 [kubeconfig] Writing "scheduler.conf" kubeconfig file 38 [control-plane] Using manifest folder "/etc/kubernetes/manifests" 39 [control-plane] Creating static Pod manifest for "kube-apiserver" 40 [control-plane] Creating static Pod manifest for "kube-controller-manager" 41 W0304 10:08:28.041239 67589 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC" 42 [control-plane] Creating static Pod manifest for "kube-scheduler" 43 W0304 10:08:28.041875 67589 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC" 44 [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" 45 [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s 46 [apiclient] All control plane components are healthy after 15.002231 seconds 47 [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace 48 [kubelet] Creating a ConfigMap "kubelet-config-1.18" in namespace kube-system with the configuration for the kubelets in the cluster 49 [upload-certs] Skipping phase. Please see --upload-certs 50 [mark-control-plane] Marking the node k8s-master as control-plane by adding the label "node-role.kubernetes.io/master=''" 51 [mark-control-plane] Marking the node k8s-master as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule] 52 [bootstrap-token] Using token: 39sbs2.rnezmmzq6k3nfazh 53 [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles 54 [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes 55 [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials 56 [bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token 57 [bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster 58 [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace 59 [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key 60 [addons] Applied essential addon: CoreDNS 61 [addons] Applied essential addon: kube-proxy 62 63 Your Kubernetes control-plane has initialized successfully! 64 65 To start using your cluster, you need to run the following as a regular user: 66 67 mkdir -p $HOME/.kube 68 sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config 69 sudo chown $(id -u):$(id -g) $HOME/.kube/config 70 71 You should now deploy a pod network to the cluster. 72 Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: 73 https://kubernetes.io/docs/concepts/cluster-administration/addons/ 74 75 Then you can join any number of worker nodes by running the following on each as root: 76 77 kubeadm join 192.168.1.1:6443 --token 39sbs2.rnezmmzq6k3nfazh \ 78 --discovery-token-ca-cert-hash sha256:5618c53639da0a3aebe95209614612a200ae084c04678737b9e18f2c523531f2
init初始化过程
#在初始的时候尾部会生成一个秘钥，作用就是将各个node节点加入k8s群集当中
kubeadm join 192.168.1.1:6443 --token 39sbs2.rnezmmzq6k3nfazh
--discovery-token-ca-cert-hash sha256:5618c53639da0a3aebe95209614612a200ae084c04678737b9e18f2c523531f2
#images的变化，下载的各个组件 [root@k8s-master ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE registry.aliyuncs.com/google_containers/kube-proxy v1.18.0 43940c34f24f 11 months ago 117MB registry.aliyuncs.com/google_containers/kube-apiserver v1.18.0 74060cea7f70 11 months ago 173MB registry.aliyuncs.com/google_containers/kube-controller-manager v1.18.0 d3e55153f52f 11 months ago 162MB registry.aliyuncs.com/google_containers/kube-scheduler v1.18.0 a31f78c7c8ce 11 months ago 95.3MB registry.aliyuncs.com/google_containers/pause 3.2 80d28bedfe5d 12 months ago 683kB registry.aliyuncs.com/google_containers/coredns 1.6.7 67da37a9a360 13 months ago 43.8MB registry.aliyuncs.com/google_containers/etcd 3.4.3-0 303ce5db0e90 16 months ago 288MB
#在初始的完成的时候会有这三行命令，作用是开启kubectl工具
mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config
5. 加入Kubernetes Node
在192.168.1.2/5（Node）执行。
向集群添加新节点，执行在kubeadm init输出的kubeadm join命令：
kubeadm join 192.168.1.1:6443 –token 39sbs2.rnezmmzq6k3nfazh
–discovery-token-ca-cert-hash sha256:5618c53639da0a3aebe95209614612a200ae084c04678737b9e18f2c523531f2
默认token有效期为24小时，当过期之后，该token就不可用了。这时就需要重新创建token，操作如下：
kubeadm token create --print-join-command
#查看一下集群信息 [root@k8s-master ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master NotReady master 14m v1.18.0 k8s-worker01 NotReady <none> 103s v1.18.0 k8s-worker02 NotReady <none> 9m46s v1.18.0
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
默认镜像地址无法访问，sed命令修改为docker hub镜像仓库。
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
[root@k8s-master ~]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-7ff77c879f-8kf97 1/1 Running 0 20m
coredns-7ff77c879f-qbmqm 1/1 Running 0 20m
etcd-k8s-master 1/1 Running 0 20m
kube-apiserver-k8s-master 1/1 Running 0 20m
kube-controller-manager-k8s-master 1/1 Running 0 20m
kube-flannel-ds-68ptv 1/1 Running 0 2m9s
kube-flannel-ds-7p6d9 1/1 Running 0 2m9s
kube-flannel-ds-zm5wh 1/1 Running 0 2m9s
kube-proxy-4nzft 1/1 Running 0 7m41s
kube-proxy-bc5rl 1/1 Running 0 20m
kube-proxy-g6msv 1/1 Running 0 15m
kube-scheduler-k8s-master 1/1 Running 0 20m
[root@k8s-master ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready master 21m v1.18.0
k8s-worker01 Ready <none> 8m34s v1.18.0
k8s-worker02 Ready <none> 16m v1.18.0
#这是所有的节点都处于ready准备状态
7. 测试kubernetes集群
在Kubernetes集群中创建一个pod，验证是否正常运行：
#通过kubectl创建部署一个名称nginx镜像是nginx的pod
[root@k8s-master ~]# kubectl create deployment nginx –image=nginx
deployment.apps/nginx created
#通过kubectl部署nginx暴露80端口，类型为节点端口
[root@k8s-master ~]# kubectl expose deployment nginx –port=80 –type=NodePort
service/nginx exposed
#查看rc和service的列表
[root@k8s-master ~]# kubectl get pod,svc
NAME READY STATUS RESTARTS AGE
pod/nginx-f89759699-24xd5 1/1 Running 0 36s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 23m
service/nginx NodePort 10.101.245.166 <none> 80:30067/TCP 12s
访问地址：http://NodeIP:Port