企业级实战 Spring Boot + K8S 中的滚动发布、优雅停机、弹性伸缩、应用监控、配置分离

news/2024/9/18 8:07:18/

下面为大家介绍我司生产环境使用了3年的基于K8S的dev ops 配置实现 K8s + SpringCloud实现零宕机发版，优雅重启：健康检查+滚动更新+优雅停机+弹性伸缩+Prometheus监控+配置分离（镜像复用）

汇总配置

业务层面

项目依赖 pom.xml

使用 spring-boot-starter-actuator 镜像存活、就绪检查

使用 prometheus 进行应用监控

<!-- 引入Spring boot的监控机制-->
<dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency><groupId>io.micrometer</groupId><artifactId>micrometer-registry-prometheus</artifactId>
</dependency>

定义访问端口、路径及权限 application.yaml

spring:application:name: project-sampleprofiles:active: @profileActive@lifecycle:timeout-per-shutdown-phase: 30s     # 停机过程超时时长设置30s，超过30s，直接停机server:port: 8080shutdown: graceful                    # 默认为IMMEDIATE，表示立即关机；GRACEFUL表示优雅关机management:server:port: 50000                         # 启用独立运维端口metrics:tags:application: ${spring.application.name}endpoint:                             # 开启shutdown和health、serviceregistry（用于nacos下线）端点shutdown:enabled: trueserviceregistry：enabled: truehealth:probes:enabled: trueendpoints:web:exposure:base-path: /actuator            # 指定上下文路径，启用相应端点include: health,shutdown,metrics,prometheus

运维层面

打包属于我们自己的dockerfile基础镜像

FROM openjdk:8-jdk-alpineLABEL Author  andanyang
## 时区环境
ENV TZ=Asia/Shanghai
ENV JAVA_OPT
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezoneWORKDIR /root
## 使用 skywalking 不用可忽略
COPY skywalking-agent /root/agent## 解决EasyExcel 缺少字体问题
RUN apk --update add curl bash ttf-dejavu && \rm -rf /var/cache/apk/*
RUN apk --update --no-cache add tini
## 使用 tini 启动应用 让容器里也可以使用jps等命令
ENTRYPOINT ["tini"]

docker build -t common/jdk8 .

打包各个服务的镜像

FROM common/jdk8
## 不使用 skywalking 可移除
CMD /usr/bin/java -Djava.security.egd=file:/dev/./urandom -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/root/log/ -javaagent:/root/agent/skywalking-agent.jar ${JAVA_OPT} -jar ${app_name}.jarCOPY ${app_name}-impl/target/${app_name}-impl.jar ${app_name}.jar

k8s部署模版deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:name: {APP_NAME}labels:app: {APP_NAME}
spec:selector:matchLabels:app: {APP_NAME}replicas: {REPLICAS}                            # Pod副本数strategy:type: RollingUpdate                           # 滚动更新策略rollingUpdate:maxSurge: 1maxUnavailable: 0template:metadata:name: {APP_NAME}labels:app: {APP_NAME}annotations:timestamp: {TIMESTAMP}prometheus.io/port: "50000"               # 不能动态赋值prometheus.io/path: /actuator/prometheusprometheus.io/scrape: "true"              # 基于pod的服务发现spec:affinity:                                   # 设置调度策略，采取多主机/多可用区部署podAntiAffinity:preferredDuringSchedulingIgnoredDuringExecution:- weight: 100podAffinityTerm:labelSelector:matchExpressions:- key: appoperator: Invalues:- {APP_NAME}topologyKey: "kubernetes.io/hostname" # 多可用区为"topology.kubernetes.io/zone"terminationGracePeriodSeconds: 30             # 优雅终止宽限期containers:- name: {APP_NAME}image: {IMAGE_URL}imagePullPolicy: Alwaysports:- containerPort: {APP_PORT}- name: management-portcontainerPort: 50000         # 应用管理端口readinessProbe:                # 就绪探针httpGet:path: /actuator/health/readinessport: management-portinitialDelaySeconds: 30      # 延迟加载时间periodSeconds: 10            # 重试时间间隔timeoutSeconds: 1            # 超时时间设置successThreshold: 1          # 健康阈值failureThreshold: 9          # 不健康阈值livenessProbe:                 # 存活探针httpGet:path: /actuator/health/livenessport: management-portinitialDelaySeconds: 30      # 延迟加载时间periodSeconds: 10            # 重试时间间隔timeoutSeconds: 1            # 超时时间设置successThreshold: 1          # 健康阈值failureThreshold: 6          # 不健康阈值resources:                     # 容器资源管理limits:                      # 资源限制（监控使用情况）cpu: 0.5memory: 1Girequests:                    # 最小可用资源（灵活调度）cpu: 0.1memory: 200Milifecycle:preStop:exec:command:               # 结束回调钩子,在实战中SpringClould 中防止注册中心没有及时下掉服务，最好需要手动下线服务。再安全下线- /bin/sh- "-c"- >-curl -X "POST" "http://localhost:50000/actuator/service-registry?status=DOWN" -H "Content-Type: application/vnd.spring-boot.actuator.v2+json;charset=UTF-8";sleep10s;curl -X "POST" "http://localhost:50000/actuator/shutdown" -H "Content-Type: application/vnd.spring-boot.actuator.v2+json;charset=UTF-8";env:- name: TZvalue: Asia/Shanghai- name: JAVA_OPT              # 配置JAVA启动参数value: "-Xmx4g -Xms4g"
---
kind: HorizontalPodAutoscaler            # 弹性伸缩控制器
apiVersion: autoscaling/v2beta2
metadata:name: {APP_NAME}
spec:scaleTargetRef:apiVersion: apps/v1kind: Deploymentname: {APP_NAME}minReplicas: {REPLICAS}                # 缩放范围maxReplicas: 6metrics:- type: Resourceresource:name: cpu                        # 指定资源指标target:type: UtilizationaverageUtilization: 50