jenkins发布Kubernetes(K8s)集群(基于containerd)

news/2025/2/12 21:35:09/

一、实验环境

1、k8s环境

版本v1.26.5，容器为containerd
二进制安装Kubernetes(K8s)集群(基于containerd)—从零安装教程（带证书）

主机名IP系统版本安装服务
master0110.10.10.21rhel7.5nginx、etcd、api-server、scheduler、controller-manager、kubelet、proxy
master0210.10.10.22rhel7.5nginx、etcd、api-server、scheduler、controller-manager、kubelet、proxy
master0310.10.10.23rhel7.5nginx、etcd、api-server、scheduler、controller-manager、kubelet、proxy
node0110.10.10.24rhel7.5nginx、kubelet、proxy
node0210.10.10.25rhel7.5nginx、kubelet、proxy

2、jenkins环境

jenkins入门与安装
容器为docker

主机IP系统版本
jenkins10.10.10.10rhel7.5

二、docker-compose安装

jenkins服务器上面安装

1、下载

https://github.com/docker/compose/releases/
下载版本：v2.18.0

在这里插入图片描述

2、安装

[root@jenkins ~]# cp docker-compose-linux-x86_64 /usr/local/bin/docker-compose
[root@jenkins ~]# chmod +x /usr/local/bin/docker-compose

3、查看版本

[root@jenkins ~]# docker-compose --version
Docker Compose version v2.18.0

三、cfssl证书生成

此处记录使用cfssl工具生成harbor私有证书，并使用证书搭建Harbor仓库，此证书使用按照kubernetes时使用的ca证书安装

1、安装cfssl

https://imroc.cc/kubernetes/trick/certs/sign-certs-with-cfssl.html
安装包下载地址：https://github.com/cloudflare/cfssl/releases

[root@jenkins ~]# ls cfssl*
cfssl_1.6.2_linux_amd64  cfssl-certinfo_1.6.2_linux_amd64  cfssljson_1.6.2_linux_amd64
[root@jenkins ~]# mv cfssl_1.6.2_linux_amd64 /usr/bin/cfssl
[root@jenkins ~]# mv cfssl-certinfo_1.6.2_linux_amd64  /usr/bin/cfssl-certinfo
[root@jenkins ~]# mv cfssljson_1.6.2_linux_amd64 /usr/bin/cfssljson
[root@jenkins ~]# chmod +x /usr/bin/cfssl*

2、ca生成证书

[root@jenkins ~]# mkdir -p pki &&  cd pki
[root@jenkins pki]# cat > ca-csr.json   << EOF 
{"CN": "kubernetes","key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","ST": "Beijing","L": "Beijing","O": "Kubernetes","OU": "Kubernetes-manual"}],"ca": {"expiry": "876000h"}
}
EOF

[root@jenkins pki]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca
[root@jenkins pki]# ls
ca.csr  ca-csr.json  ca-key.pem  ca.pem

3、过期时间查看

[root@jenkins pki]# openssl x509 -noout -text -in ca.pem|grep -A 5 ValidityValidityNot Before: Jun  4 12:32:00 2023 GMTNot After : May 11 12:32:00 2123 GMTSubject: C=CN, ST=Beijing, L=Beijing, O=Kubernetes, OU=Kubernetes-manual, CN=kubernetesSubject Public Key Info:Public Key Algorithm: rsaEncryption

4、创建Harbor证书

[root@jenkins pki]# cat > ca-config.json << EOF 
{"signing": {"default": {"expiry": "438000h"},"profiles": {"kubernetes": {"usages": ["signing","key encipherment","server auth","client auth"],"expiry": "438000h"}}}
}
EOF

[root@jenkins pki]# cat > harbor-csr.json  << EOF 
{"CN": "harbor","hosts": ["127.0.0.1","10.10.10.10",  "harbor.wielun.com"],"key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","ST": "Beijing","L": "Beijing","O": "Kubernetes","OU": "Kubernetes-manual"}]
}
EOF

[root@jenkins pki]# cfssl gencert \
-ca=ca.pem \
-ca-key=ca-key.pem \
-config=ca-config.json \
-profile=kubernetes harbor-csr.json | cfssljson -bare harbor

[root@jenkins pki]# mkdir -p /etc/harbor/pki
[root@jenkins pki]# cp harbor.pem harbor-key.pem /etc/harbor/pki

四、安装harbor

jenkins服务器上面安装

1、下载

下载地址：https://github.com/goharbor/harbor/releases#install
安装官网：https://goharbor.io/docs/2.8.0/install-config/

在这里插入图片描述

2、安装

（1）解压文件

[root@jenkins ~]#  tar xf harbor-offline-installer-v2.8.1.tgz -C /usr/local

（2）修改harbor.yml

[root@jenkins ~]# cd /usr/local/harbor/
[root@jenkins harbor]# cp harbor.yml.tmpl harbor.yml
[root@jenkins harbor]# vim harbor.yml

在这里插入图片描述
（3）启动

[root@jenkins harbor]# docker load -i harbor.v2.8.1.tar.gz
[root@jenkins harbor]# ./prepare
[root@jenkins harbor]# ./install.sh
[root@jenkins harbor]# docker-compose up -d     #手动启动命令

3、创建登录证书

[root@jenkins ~]# mkdir -p /etc/docker/certs.d/10.10.10.10
[root@jenkins ~]# mkdir -p /etc/docker/certs.d/harbor.wielun.com
[root@jenkins ~]# cp pki/ca.pem /etc/docker/certs.d/10.10.10.10
[root@jenkins ~]# cp pki/ca.pem /etc/docker/certs.d/harbor.wielun.com

4、修改daemon.json

[root@jenkins ~]# cat /etc/docker/daemon.json
{"registry-mirrors": ["https://docker.mirrors.ustc.edu.cn"],"insecure-registries": ["10.10.10.10","harbor.wielun.com"]
}[root@jenkins ~]# systemctl restart docker

5、添加hosts

[root@jenkins ~]# vim /etc/hosts
10.10.10.10 harbor.wielun.com

6、登录验证

账号密码：admin/Harbor12345

（1）docker login

[root@jenkins ~]# docker login 10.10.10.10
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeeded[root@jenkins ~]# docker login harbor.wielun.com
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeeded

（2）浏览器登录

https://10.10.10.10/

在这里插入图片描述

7、docker上传镜像测试

（1）拉取镜像

[root@jenkins ~]# docker pull nginx
[root@jenkins ~]# docker images|grep nginx
nginx                           latest              f9c14fe76d50        10 days ago         143MB
goharbor/nginx-photon           v2.8.1              cea1bb2450ee        3 weeks ago         127MB

（2）打包上传

[root@jenkins ~]# docker tag nginx:latest harbor.wielun.com/library/nginx:latest
[root@jenkins ~]# docker push harbor.wielun.com/library/nginx

（3）浏览器中查看
在这里插入图片描述

五、K8s（containerd）拉取镜像（每台机器）

选择一种即可，这边我使用得是跳过证书

1、K8s（containerd）拉取镜像（跳过证书）

（1）删除之前containerd配置

[root@master01 ~]# rm -rf /etc/containerd/config.toml
[root@master01 ~]# containerd config default | tee /etc/containerd/config.toml
[root@master01 ~]# sed -i "s#SystemdCgroup\ \=\ false#SystemdCgroup\ \=\ true#g" /etc/containerd/config.toml
[root@master01 ~]# sed -i "s#registry.k8s.io#registry.cn-hangzhou.aliyuncs.com/chenby#g" /etc/containerd/config.toml
[root@master01 ~]# sed -i "s#config_path\ \=\ \"\"#config_path\ \=\ \"/etc/containerd/certs.d\"#g" /etc/containerd/config.toml

（2）配置hosts.toml

[root@master01 ~]# mkdir -p /etc/containerd/certs.d/harbor.wielun.com
[root@master01 ~]# cat > /etc/containerd/certs.d/harbor.wielun.com/hosts.toml << EOF
server = "https://harbor.wielun.com"
[host."https://harbor.wielun.com"]capabilities = ["pull", "resolve", "push"]skip_verify = true
EOF

（3）重启containerd

[root@master01 ~]# systemctl restart containerd

（4）添加hosts

[root@master01 ~]# cat /etc/hosts
10.10.10.10 harbor.wielun.com

2、K8s（containerd）拉取镜像（通过证书）

（1）证书配置

[root@master01 ~]# mkdir -p /etc/containerd/certs.d/harbor.wielun.com[root@jenkins ~]# cd pki/
[root@jenkins pki]# scp ca.pem harbor.pem harbor-key.pem root@10.10.10.21:/etc/containerd/certs.d/harbor.wielun.com

（2）删除之前containerd配置

[root@master01 ~]# rm -rf /etc/containerd/config.toml
[root@master01 ~]# containerd config default | tee /etc/containerd/config.toml
[root@master01 ~]# sed -i "s#SystemdCgroup\ \=\ false#SystemdCgroup\ \=\ true#g" /etc/containerd/config.toml
[root@master01 ~]# sed -i "s#registry.k8s.io#registry.cn-hangzhou.aliyuncs.com/chenby#g" /etc/containerd/config.toml

（3）配置config.toml

[root@master01 ~]# vim /etc/containerd/config.toml[plugins."io.containerd.grpc.v1.cri".image_decryption]key_model = "node"[plugins."io.containerd.grpc.v1.cri".registry]config_path = ""[plugins."io.containerd.grpc.v1.cri".registry.auths][plugins."io.containerd.grpc.v1.cri".registry.configs][plugin."io.containerd.grpc.v1.cri".registry.configs."harbor.wielun.com".tls]ca_file = "/etc/containerd/certs.d/harbor.wielun.com/ca.pem"cert_file = "/etc/containerd/certs.d/harbor.wielun.com/harbor.pem"key_file = "/etc/containerd/certs.d/harbor.wielun.com/harbor-key.pem"[plugins."io.containerd.grpc.v1.cri".registry.configs."harbor.wielun.com".auth]username = "admin"password = "Harbor12345"[plugins."io.containerd.grpc.v1.cri".registry.headers][plugins."io.containerd.grpc.v1.cri".registry.mirrors][plugins."io.containerd.grpc.v1.cri".registry.mirrors."harbor.wielun.com"]endpoint = ["https://harbor.wielun.com"][plugins."io.containerd.grpc.v1.cri".x509_key_pair_streaming]tls_cert_file = ""tls_key_file = ""

在这里插入图片描述

（4）重启containerd

[root@master01 ~]# systemctl restart containerd

（5）添加hosts

[root@master01 ~]# cat /etc/hosts
10.10.10.10 harbor.wielun.com

3、测试拉取镜像

（1）拉取镜像

# -k：跳过证书认证
[root@master01 ~]# ctr -n harbor.wielun.com  images  pull harbor.wielun.com/library/nginx:latest -k
harbor.wielun.com/library/nginx:latest:                                           resolved       |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:6b06964cdbbc517102ce5e0cef95152f3c6a7ef703e4057cb574539de91f72e6: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:831f51541d386c6d0d86f6799fcfabb48e91e9e5aea63c726240dd699179f495:    done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:f9c14fe76d502861ba0939bc3189e642c02e257f06f4c0214b1f8ca329326cda:   done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:f03b40093957615593f2ed142961afb6b540507e0b47e3f7626ba5e02efbbbf1:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:eed12bbd64949353649476b59d486ab4c5b84fc5ed2b2dc96384b0b892b6bf7e:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:fa7eb8c8eee8792b8db1c0043092b817376f096e3cc8feeea623c6e00211dad1:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:7ff3b2b12318a41d4b238b643d7fcf1fe6da400ca3e02aa61766348f90455354:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:0f67c7de5f2c7e0dc408ce685285419c1295f24b7a01d554517c7a72374d4aeb:    done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 0.1 s                                                                    total:   0.0 B (0.0 B/s)
unpacking linux/amd64 sha256:6b06964cdbbc517102ce5e0cef95152f3c6a7ef703e4057cb574539de91f72e6...[root@master01 ~]# ctr -n harbor.wielun.com images pull harbor.wielun.com/library/nginx:latest --tlscacert  /etc/containerd/certs.d/harbor.wielun.com/ca.pem --tlscert   /etc/containerd/certs.d/harbor.wielun.com/harbor.pem  --tlskey  /etc/containerd/certs.d/harbor.wielun.com/harbor-key.pem
harbor.wielun.com/library/nginx:latest:                                           resolved       |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:6b06964cdbbc517102ce5e0cef95152f3c6a7ef703e4057cb574539de91f72e6: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:831f51541d386c6d0d86f6799fcfabb48e91e9e5aea63c726240dd699179f495:    done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:f9c14fe76d502861ba0939bc3189e642c02e257f06f4c0214b1f8ca329326cda:   done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:f03b40093957615593f2ed142961afb6b540507e0b47e3f7626ba5e02efbbbf1:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:eed12bbd64949353649476b59d486ab4c5b84fc5ed2b2dc96384b0b892b6bf7e:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:fa7eb8c8eee8792b8db1c0043092b817376f096e3cc8feeea623c6e00211dad1:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:7ff3b2b12318a41d4b238b643d7fcf1fe6da400ca3e02aa61766348f90455354:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:0f67c7de5f2c7e0dc408ce685285419c1295f24b7a01d554517c7a72374d4aeb:    done           |++++++++++++++++++++++++++++++++++++++|

（2）查看镜像

[root@master01 ~]# ctr -n harbor.wielun.com images ls
REF                                    TYPE                                                 DIGEST                                                                  SIZE     PLATFORMS   LABELS
harbor.wielun.com/library/nginx:latest application/vnd.docker.distribution.manifest.v2+json sha256:6b06964cdbbc517102ce5e0cef95152f3c6a7ef703e4057cb574539de91f72e6 54.5 MiB linux/amd64 -

（3）crictl拉取镜像

[root@master01 ~]# crictl  pull harbor.wielun.com/library/nginx:latest
Image is up to date for sha256:f9c14fe76d502861ba0939bc3189e642c02e257f06f4c0214b1f8ca329326cda[root@master01 ~]# crictl images ls|grep harbor.wielun.com/library/nginx
harbor.wielun.com/library/nginx                                                latest              f9c14fe76d502       57.2MB

六、jenkins发布到K8S

1、推送tomcat到harbor

这里用Java项目做演示，一般项目打包成jar包，我们这里打包成war包

[root@jenkins ~]# docker pull tomcat:8.5.59
[root@jenkins ~]# docker tag tomcat:8.5.59 harbor.wielun.com/library/tomcat:8.5.59
[root@jenkins ~]# docker push harbor.wielun.com/library/tomcat:8.5.59

2、创建项目

这里我们使用Jenkinsfile

在这里插入图片描述
在这里插入图片描述

3、查看项目文件

在这里插入图片描述
在这里插入图片描述

（1）配置Dockerfile

FROM harbor.wielun.com/library/tomcat:8.5.59
MAINTAINER Wielun
RUN rm -rf /usr/local/tomcat/webapps/*
ADD target/*.war /usr/local/tomcat/webapps/ROOT.war

（2）配置Jenkinsfile

pipeline {agent anyenvironment {harborUser = 'admin'harborPasswd = 'Harbor12345'HarborAddress = 'harbor.wielun.com'harborRepo = 'library'}stages {stage('git拉取代码') {steps {git credentialsId: '0c71c0f9-8277-493b-xxxx-540a9324cf08', url: 'https://jihulab.com/xxxx/java-demo.git'}}stage('maven编译') { steps {sh '''JAVA_HOME=/usr/local/jdkPATH=$PATH:$JAVA_HOME/bin/usr/local/maven/bin/mvn clean package -Dmaven.test.skip=true'''}}stage('生成自定义镜像') { steps {sh '''docker build -t ${JOB_NAME}:latest .'''}}stage('上传自定义镜像到harbor') { steps {sh '''docker login -u ${harborUser} -p ${harborPasswd} ${HarborAddress}docker tag ${JOB_NAME}:latest ${HarborAddress}/${harborRepo}/${JOB_NAME}:latestdocker push ${HarborAddress}/${harborRepo}/${JOB_NAME}:latest'''}}stage('发送yaml到k8s-master并部署') { steps {sshPublisher(publishers: [sshPublisherDesc(configName: 'k8s-master', transfers: [sshTransfer(cleanRemote: false, excludes: '', execCommand: '''/usr/local/bin/kubectl apply -f /tmp/${JOB_NAME}/pipeline.yaml''', execTimeout: 120000, flatten: false, makeEmptyDirs: false, noDefaultExcludes: false, patternSeparator: '[, ]+', remoteDirectory: '${JOB_NAME}', remoteDirectorySDF: false, removePrefix: '', sourceFiles: 'pipeline.yaml')], usePromotionTimestamp: false, useWorkspaceInPromotion: false, verbose: false)])}} }
}

（3）配置pipeline.yaml

apiVersion: apps/v1
kind: Deployment
metadata:namespace: testname: pipelinelabels:app: pipeline
spec:replicas: 2selector:matchLabels:app: pipelinetemplate:metadata:labels:app: pipelinespec:containers:- name: pipelineimage: harbor.wielun.com/library/java-k8s:latestimagePullPolicy: Alwaysports:- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:namespace: testname: pipelinelabels:app: pipeline
spec:ports:- port: 8081targetPort: 8080selector:app: pipelinetype: NodePort
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:namespace: testname: pipeline
spec:ingressClassName: nginxrules:- host: "harbor.wielun.com"http:paths:- pathType: Prefixpath: /backend:service:name: pipelineport:number: 8081

4、创建namespace

[root@master01 ~]# kubectl create ns test

5、构建并查看结果

[root@master01 ~]# kubectl get pod -n test
NAME                        READY   STATUS    RESTARTS   AGE
pipeline-556759f7b4-7x8ml   1/1     Running   0          11s
pipeline-556759f7b4-zwdgr   1/1     Running   0          11s

http://www.ppmy.cn/news/363949.html

相关文章

iperf网络测速

iperf网络测速

启动服务端&#xff1a; -s 服务端模式 .\iperf3.exe -s 启动客户端&#xff1a; .\iperf3.exe -c 10.10.4.120 -u -b 100M -t 10000 -l 1380 -i 1 -c host-ip 客户端模式&#xff0c;连接主机IP地址host-ip -i seconds 间隔多少秒统计一次 -l 每次收发的包大小 -h 查看…
阅读更多...
DNS测速最优

DNS测速最优

DNS影响网速快慢&#xff0c;因此&#xff0c;我们在选择DNS的时候要先测网速。 如何测DNS的快慢&#xff0c;最简单的方法用命令提示符 如图所示&#xff0c;pingDNS代号&#xff08;PING与DNS代号之间有空格&#xff09; C:\Users\&#xff1f;>ping 119.29.29.29正在 P…
阅读更多...
Linux网络测速工具Speedtest

Linux网络测速工具Speedtest

安装speedtest-cli yum install python-pip –y pip install speedtest-cli执行网络测速 speedtest #执行结果 Retrieving speedtest.net configuration... Testing from Tencent cloud computing (140.143.139.14)... Retrieving speedtest.net server list... Selecting be…
阅读更多...
网速测速

网速测速

一、可以测上行速度、下行速度、ip地址等 上行速度&#xff1a; 是指上传数据的速度&#xff0c;比如上传云存储的文档、图片等; 下行速度&#xff1a; 是指下载数据的速度&#xff0c;比如浏览网页、下载电影等。 平时我们说的多少带宽一般是指下行速度。日常用网络我们多…
阅读更多...
网络带宽 (网速) 在线测试

网络带宽 (网速) 在线测试

网络带宽 (网速) 在线测试 https://www.speedtest.net/ https://www.speedtest.cn/ 1. 测速 2. 获取到下载和上传带宽 3. bits (比特) Kilobit per second (Kbit/s, Kb/s or Kbps) 千比特每秒 Megabit per second (Mbit/s, Mb/s or Mbps) 兆比特每秒 Gigabit per second (G…
阅读更多...
Linux网络测速

Linux网络测速

speedtest speedtest是一个用Python编写的轻量级Linux命令行工具&#xff0c;在Python2.4至3.4版本下均可运行。它基于Speedtest.net的基础架构来测量网络的上传和下载速率。安装speedtest很简单——只需要下载其Python脚本文件即可。 新建文件 vim speedtest.py将此网站内容…
阅读更多...
宽带测速

宽带测速

文章目录 1. 第三方测速网站2. 中国移动家庭宽带质量检测中心3. 电信3.1. 广东电信 1. 第三方测速网站 https://www.speedtest.cn/ 2. 中国移动家庭宽带质量检测中心 仅限中国移动宽带用户访问 http://211.138.20.246:28080http://cm.coco56.top:28080/ 需要电脑登陆进行测…
阅读更多...
Linux Command iperf3网络测速工具

Linux Command iperf3网络测速工具

Linux Command iperf3网络测速工具 文章目录 Linux Command iperf3网络测速工具1. 简介2. 安装3. 功能4. 参数5. 示例5.1 测试TCP吞吐量5.2 测试UDP吞吐量 1. 简介 Iperf是一款基于TCP/IP和UDP/IP的网络性能测试工具&#xff0c;可以用来测量网络带宽和网络质量&#xff0c;提…
阅读更多...
最新文章

Copyright @ 2022~2023