#云原生征文#k8s高可用三台master部署图文并茂 _云原生

少年辛苦终身事，莫向光阴惰寸功。这篇文章主要讲述#云原生征文#k8s高可用三台master部署图文并茂相关的知识，希望能为你提供帮助。

每台机器都要部署nginx
1.前提：k8s相关服务必须安装完关闭每台机器防火墙,postfix，selinux，swap

systemctl disable firewalld & & systemctl stop firewalld
systemctl disable postfix & & systemctl stop postfix
setenforce 0 & & sed -i "s/^SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config
swapoff -a & & sed -i s/.*swap.*/#& / /etc/fstab

修改k8s文件，将桥接的IPv4流量传递到iptables的链

cat > /etc/sysctl.d/k8s.conf < < EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

# 生效
sysctl --system

安装配置docker
mkdir -p /etc/docker/

vim /etc/docker/daemon.json
#添加如下配置

"hosts":[
"tcp://0.0.0.0:9998",
"unix:///var/run/docker.sock"
],
"insecure-registries":["192.168.146.101:5005"],
"registry-mirrors": ["https://b9pmyelo.mirror.aliyuncs.com"]

yum install -y yum-utils

yum-config-manager --add-repo https://mirrors.tuna.tsinghua.edu.cn/docker-ce/linux/centos/docker-ce.repo

yum install -y docker-ce docker-ce-cli containerd.io

systemctl enable docker & & systemctl start docker

配置kubernetes源

vim /etc/yum.repos.d/kubernetes.repo
#添加如下配置
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg

安装kubeadm，kubelet和kubectl

yum install -y kubelet-1.18.6 kubeadm-1.18.6 kubectl-1.18.6

systemctl enable kubelet

k8s三台master部署

10.0.0.128? 10.0.0.215? 10.0.0.29? #### 重新生成新的api-server证书? 在master节点下执行下列操作：? ```shell? # 导出线上kubeadm配置? kubectl -n kube-system get configmap kubeadm-config -o jsonpath=.data.ClusterConfiguration > kubeadm.yaml? ? 增加apiServer参数certSANs? ```yaml? apiServer:? certSANs:? - localhost? - 10.? - 10.? - 10.? - hw-? - hw-? - hw-? extraArgs:? authorization-mode: Node,RBAC? timeoutForControlPlane: 4m0s? apiVersion: kubeadm.k8s.io/v1beta2? certificatesDir: /etc/kubernetes/pki? clusterName: kubernetes? controllerManager: ? dns:? type: CoreDNS? etcd:? local:? dataDir: /var/lib/etcd? imageRepository: registry.aliyuncs.com/google_containers? kind: ClusterConfiguration? kubernetesVersion: v1.18.6? networking:? dnsDomain: cluster.local? podSubnet: 10.244.0.0/16? serviceSubnet: 10.96.0.0/12? scheduler: ? ```?

更新证书

更新证书? ? ```shell? #把整个/etc/kubernetes做备份? cp -r /etc/kubernetes ~/backups? ? #删除老的api-server证书? rm /etc/kubernetes/pki/apiserver.crt,key? ? #直接使用 kubeadm 命令生成一个新的证书? kubeadm init phase certs apiserver --config kubeadm.yaml? ? #重启 APIServer 来接收新的证书，最简单的方法是直接杀死 APIServer 的容器? docker kill $(docker ps \| grep kube-apiserver \| grep -v pause \| cut -d-f1)? #验证证书? openssl x509 -in /etc/kubernetes/pki/apiserver.crt -text? ? ? #将上面的集群配置信息保存到集群的 kubeadm-config 这个 ConfigMap 中去? kubeadm config upload from-file --config kubeadm.yaml? ? ? ? #验证是否保存成功? kubectl -n kube-system get configmap kubeadm-config -o yaml?

负载均衡?

k8s高可用部署?已部署完k8s相关操作?

在所有节点上执行如下操作：?

安装组件nginx、keepalived?
```shell?
yum install nginx keepalived -y?
```?
在所有节点上使用 nginx 来作为一个负载均衡器?
```shell?
vim /etc/kubernetes/nginx.conf?
```?

添加如下内容?

```shell?
error_log stderr notice; ?

worker_processes 2; ?
worker_rlimit_nofile 130048; ?
worker_shutdown_timeout 10s; ?

events ?
multi_accept on; ?
use epoll; ?
worker_connections 16384; ?
?

stream ?
upstream kube_apiserver ?
least_conn; ?
server 10.0.0.215:6443; ?
server 10.0.0.128:6443; ?
server 10.0.0.29:6443; ?
?

server ?
listen 8443; ?
proxy_pass kube_apiserver; ?
proxy_timeout 10m; ?
proxy_connect_timeout 1s; ?
?
?

http ?
aio threads; ?
aio_write on; ?
tcp_nopush on; ?
tcp_nodelay on; ?

keepalive_timeout 5m; ?
keepalive_requests 100; ?
reset_timedout_connection on; ?
server_tokens off; ?
autoindex off; ?

server ?
listen 8081; ?
location /stub_status ?
stub_status on; ?
access_log off; ?
?
?
?
```?

部署keepalived服务?
yum install keepalived -y?
2.更新master节点配置?**修改 kubelet 配置：**?
```shell?
vim /etc/kubernetes/kubelet.conf?
```?
将原有的ip改成nginx的代理配置?
```yaml?
......?
server: https://localhost:8443?
name: kubernetes?
......?

```?
重启服务?

```shell?
systemctl restart kubelet?
```?
**修改 controller-manager 配置：**?
```shell?
vim /etc/kubernetes/controller-manager.conf?
```?

将原有的ip改成nginx的代理配置?
```yaml?
......?
server: https://localhost:8443?
name: kubernetes?
......?
```?
重启服务?
```shell?
docker kill $(docker ps | grep kube-controller-manager | grep -v pause | cut -d-f1)?
```?

**修改 scheduler 配置：**?

```shell?
vim /etc/kubernetes/scheduler.conf?
```?

将原有的ip改成nginx的代理配置?

```yaml?
......?
server: https://localhost:8443?
name: kubernetes?
......?
```?

重启服务?

```shell?
docker kill $(docker ps | grep kube-scheduler | grep -v pause | cut -d-f1)?
```?
**更新kube客户端配置**?
```?
vim ~/.kube/config?
```?
将原有的ip改成nginx的代理配置?

```yaml?
......?
server: https://localhost:8443?
name: kubernetes?
......?
```?
**更新 kube-proxy 配置**?
```shell?
kubectl -n kube-system edit cm kube-proxy?
```?
将原有的ip改成nginx的代理配置?
```yaml?
......?
kubeconfig.conf: |-?
apiVersion: v1?
kind: Config?
clusters:?
- cluster:?
certificate-authority: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt?
server: https://localhost:8443?
name: default?
......?

```?
重启各个节点的 kube-proxy?

3.更新控制平面(master)配置?从集群中的 ConfigMap 中获取当前配置?
```shell?
kubectl -n kube-system get configmap kubeadm-config -o jsonpath=.data.ClusterConfiguration > kubeadm.yaml?
```?

然后在当前配置文件里面里面添加 `controlPlaneEndpoint` 属性，用于指定控制面板的负载均衡器的地址。?
```yaml?
controlPlaneEndpoint: localhost:8443 #在首行添加该配置?
```?
使用以下命令将其上传回集群?
```shell?
kubeadm config upload from-file --config kubeadm.yaml?
```?
然后需要在 `kube-public` 命名空间中更新 `cluster-info` 这个 ConfigMap，该命名空间包含一个Kubeconfig 文件，该文件的 `server:` 一行指向单个控制平面节点。只需使用`kubectl -n kube-public edit cm cluster-info` 更新该 `server:` 行以指向控制平面的负载均衡器即可。?
```shell?
kubectl -n kube-public edit cm cluster-info?
```?
将原有的ip改成nginx的代理配置?

```yaml?
......?
server: https://localhost:8443?
name: ""?
......?
```?
更新完成就可以看到 cluster-info 的信息变成了负载均衡器的地址了。?
```shell?
kubectl cluster-info?
```?
4.生成token?kubeadm init phase upload-certs --upload-certs?
kubeadm token create --print-join-command --config kubeadm.yaml?
5.添加master节点?kubeadm reset?
rm -rf /var/lib/etcd?
kubeadm join localhost:8443 --token 4pi1b4.ngn8krw0aonwpnzd --discovery-token-ca-cert-hash sha256:e94427a152103d795535f5ec783f5f4dbaf2f92419682326d8716332d493f683 --control-plane --certificate-key 653c8a46198e675bee0b7b0183049b7e9ee08a2ff567bc5c36b82c28553ad484?
```?

6.修改etcd组件配置?登录各个master节点，修改etcd配置?
```shell?
vim /etc/kubernetes/manifests/etcd.yaml?
```?
增加所有master的连接?
```yaml?
......?
- --initial-cluster=hw-prd-dtp-hue-server-10-4-46-215=https://10.4.46.215:2380,hw-prd-dtp-k8s-master-10-4-46-128=https://10.4.46.128:2380,hw-prd-dtp-k8s-master-10-4-46-29=https://10.4.46.29:2380?

......