侧边栏壁纸
博主头像
cplinux98

  • 累计撰写 112 篇文章
  • 累计创建 31 个标签
  • 累计收到 0 条评论
标签搜索

目 录CONTENT

文章目录
DevOps CloudNative

Kubeadm管理k8s集群系列03-环境准备

cplinux98
cplinux98
2022-09-06 / 0 评论 / 0 点赞 / 837 阅读 / 1,817 字 / 正在检测是否收录...

00：文章简介

介绍k8s集群部署的环境准备。

01: 了解k8s基础环境组成及相关要求

1.1：基本环境组成

OS：Ubuntu 20.04.2 LTS

容器技术：docker

批量管理工具：ansible

内网DNS环境：bind

SSL证书：阿里云

相关域名： linux98.com

资源配置文件存放目录： /root/mykube

1.2：系统详情

网络配置

network:
  version: 2
  ethernets:
    eth0:
      addresses: [172.20.200.201/16]
      gateway4: 172.20.1.1
      nameservers:
        search: [linux98.com]
        addresses: [172.20.254.10, 172.20.254.11]

软件源配置

deb http://mirrors.aliyun.com/ubuntu/ focal main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-backports main restricted universe multiverse

1.3：相关要求

关闭swap分区

echo "vm.swappiness=0" >> /etc/sysctl.d/k8s.conf
sed -i 's/.*swap/#&/' /etc/fstab
swapoff /swap.img

配置内核参数

echo "net.bridge.bridge-nf-call-ip6tables=1" >> /etc/sysctl.d/k8s.conf
echo "net.bridge.bridge-nf-call-iptables=1" >> /etc/sysctl.d/k8s.conf
echo "net.ipv4.ip_forward=1" >> /etc/sysctl.d/k8s.conf

启动相关模块

modprobe br_netfilter
modprobe overlay

02：使用cobbler批量部署基础架构

这里只安装底层操作系统，初始化配置在下面手动做

03：配置master节点环境

3.1：基础环境

cat >> /etc/sysctl.d/k8s.conf <<EOF
vm.swappiness=0
net.bridge.bridge-nf-call-ip6tables=1
net.bridge.bridge-nf-call-iptables=1
net.ipv4.ip_forward=1
EOF

modprobe br_netfilter
modprobe overlay
sysctl -p /etc/sysctl.d/k8s.conf
swapoff /swap.img

3.2：docker环境

apt-get -y install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | sudo apt-key add -
add-apt-repository "deb [arch=amd64] https://mirrors.aliyun.com/docker-ce/linux/ubuntu $(lsb_release -cs) stable"
apt-get -y update
apt-get -y install docker-ce

修改docker启动方式和设置本地镜像仓库

# setting docker
cat >> /etc/docker/daemon.json << EOF
{
 "registry-mirrors": ["https://73yi6cz9.mirror.aliyuncs.com"],
 "insecure-registries": ["harbor.linux98.com"],
 "exec-opts": ["native.cgroupdriver=systemd"]
}
EOF

systemctl restart docker

3.3：安装k8s软件源

apt-get update && apt-get install -y apt-transport-https
curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add -
cat >/etc/apt/sources.list.d/kubernetes.list << EOF
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl

04：相关要求编排为playbook

gitee(后续增加ansible设置的playbook)

这里先使用shell脚本快速配置

#!/bin/bash

# settings apt source
> /etc/apt/sources.list
cat >> /etc/apt/sources.list << EOF
deb http://mirrors.aliyun.com/ubuntu/ focal main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-backports main restricted universe multiverse
EOF

apt update

# off swap
echo "vm.swappiness=0" >> /etc/sysctl.d/k8s.conf
sed -i 's/.*swap/#&/' /etc/fstab
swapoff /swap.img

# setting kernel params
echo "net.bridge.bridge-nf-call-ip6tables=1" >> /etc/sysctl.d/k8s.conf
echo "net.bridge.bridge-nf-call-iptables=1" >> /etc/sysctl.d/k8s.conf
echo "net.ipv4.ip_forward=1" >> /etc/sysctl.d/k8s.conf
sysctl -p /etc/sysctl.d/k8s.conf

# open model
modprobe br_netfilter
modprobe overlay

# install docker
apt-get -y install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | sudo apt-key add -
add-apt-repository "deb [arch=amd64] https://mirrors.aliyun.com/docker-ce/linux/ubuntu $(lsb_release -cs) stable"
apt-get -y update
apt-get -y install docker-ce


# setting docker
cat >> /etc/docker/daemon.json << EOF
{
 "registry-mirrors": ["https://73yi6cz9.mirror.aliyuncs.com"],
 "insecure-registries": ["harbor.linux98.com"],
 "exec-opts": ["native.cgroupdriver=systemd"]
}
EOF

systemctl restart docker


# install k8s soft source
apt-get update && apt-get install -y apt-transport-https
curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add -
cat >/etc/apt/sources.list.d/kubernetes.list << EOF
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl

05：在harbor节点上部署harbor镜像仓库

apt-get -y install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | sudo apt-key add -
add-apt-repository "deb [arch=amd64] https://mirrors.aliyun.com/docker-ce/linux/ubuntu $(lsb_release -cs) stable"
apt-get -y update
apt-get -y install docker-ce


apt-get install docker-compose -y
wget https://github.com/goharbor/harbor/releases/download/v2.3.2/harbor-offlineinstaller-v2.3.2.tgz
# 这里需要修改为harbor的当前版本，可以去https://github.com/goharbor/harbor/releases中查看
tar -xf harbor-offlineinstaller-v2.3.2.tgz -C /usr/local/
cd /usr/local/harbor

docker load < harbor.v2.3.2.tar.gz
cp harbor.yml.tmpl harbor.yml

# 修改如下部分
hostname: harbor.linux98.com
https:
  port: 443
  # nginx ssl认证
  certificate: /usr/local/harbor/cert/6442545_harbor.linux98.com 
  private_key: /usr/local/harbor/cert/6442545_harbor.linux98.com.key
harbor_admin_password: 123456
data_volume: /data/harbor

# 执行检查
./prepare

# 安装
./install.sh

# systemd文件
cat > /lib/systemd/system/harbor.service << EOF
[Unit]
Description=Harbor
After=docker.service systemd-networkd.service systemd-resolved.service
Requires=docker.service
Documentation=http://github.com/vmware/harbor

[Service]
Type=simple
Restart=on-failure
RestartSec=5
#需要注意harbor的安装位置
ExecStart=/usr/bin/docker-compose -f /usr/local/harbor/docker-compose.yml up
ExecStop=/usr/bin/docker-compose -f /usr/local/harbor/docker-compose.yml down

[Install]
WantedBy=multi-user.target
EOF

# 设置开机自启动
systemctl start harbor
systemctl enable harbor

使用浏览器登陆harbor.linux98l.com，用户名：admin，密码：123456

左侧导航->系统管理->用户管理->创建用户-> 创建普通用户：用户名cpli，密码：A12345678a

使用普通用户登陆创建公开项目：google_containers、appimages

image

06：在临时节点下载并上传k8s的镜像

查看k8s v1.22.1所依赖的镜像及版本

# 查看
kubeadm config images list --kubernetes-version=v1.22.1

k8s.gcr.io/kube-apiserver:v1.22.1
k8s.gcr.io/kube-controller-manager:v1.22.1
k8s.gcr.io/kube-scheduler:v1.22.1
k8s.gcr.io/kube-proxy:v1.22.1
k8s.gcr.io/pause:3.5
k8s.gcr.io/etcd:3.5.0-0
k8s.gcr.io/coredns/coredns:v1.8.4

登陆harbor并使用脚本下载上面的镜像，然后上传到harbor

下面的脚步需要先安装expect: apt install expect

# 执行脚本
#!/bin/bash
#login
expect -c "
spawn docker login harbor.linux98.com
        expect {
                \"*Username:*\" {send \"cpli\r\"; exp_continue}
                \"*Password:*\" {send \"A12345678a\r\"; exp_continue}
        } "

# download and upload
images=$(kubeadm config images list --kubernetes-version=v1.22.1 | grep -o '[a-z|-]*:.*')

for i in ${images}
do
    docker pull registry.aliyuncs.com/google_containers/$i
    docker tag registry.aliyuncs.com/google_containers/$i harbor.linux98.com/google_containers/$i
    docker rmi registry.aliyuncs.com/google_containers/$i
    docker push harbor.linux98.com/google_containers/$i
done

下载dashboard和flannel插件的镜像，上传到harbor

flannel的github主页：https://github.com/flannel-io/flannel 这里使用的是v0.14.0

dashboard的github主页：https://github.com/kubernetes/dashboard

# 下载flannel和dashboard镜像并上传
# 这里的flannel的版本可能会出现问题，如有问题，请访问github查看版本号
# wget https://github.com/flannel-io/flannel/blob/v0.14.0/Documentation/kube-flannel.yml

docker pull quay.io/coreos/flannel:v0.14.0
docker tag quay.io/coreos/flannel:v0.14.0 harbor.linux98.com/google_containers/flannel:v0.14.0
docker rmi quay.io/coreos/flannel:v0.14.0

# dashboard-github https://github.com/kubernetes/dashboard
# wget https://github.com/kubernetes/dashboard/blob/v2.3.1/aio/deploy/recommended.yaml

docker pull kubernetesui/dashboard:v2.3.1
docker pull kubernetesui/metrics-scraper:v1.0.6

docker tag kubernetesui/dashboard:v2.3.1 harbor.linux98.com/google_containers/dashboard:v2.3.1
docker tag kubernetesui/metrics-scraper:v1.0.6 harbor.linux98.com/google_containers/metrics-scraper:v1.0.6

docker rmi kubernetesui/dashboard:v2.3.1
docker rmi kubernetesui/metrics-scraper:v1.0.6

最后登陆harbor查看效果

image

0
kubernetes kubeadm
版权归属： cplinux98
本文链接： https://www.linux98.com/archives/kubeadm-03
许可协议： 本文使用《署名-非商业性使用-相同方式共享 4.0 国际 (CC BY-NC-SA 4.0)》协议授权

评论区