1、jenkins在k8s内部署(请参考其他人的文章)
2、jenkins安装kubenents相关插件
3、配置k8s云
非常重要,目的是实现jenkins可以远程调用k8s进行部署,并可实现安装jenkins-slave进行构建。使得不再依赖jenkins单机能力进行构建,比较适合一定规模的公司。
插件安装后,打开jenkins,点击系统管理-->系统配置-->Clouds,点击新增一个kubernetes。
具体配置见如下:
3.1 首先配置K8S地址
3.2 为K8S云创建Pod Templates
目的是在jenkins构建的时候,可以自动在K8S内创建一个POD服务于本次构建。
以JAVA为例,构建时需要使用maven并docker构建,然后连接至K8S Master进行远程发布。
在POD容器列表中放了2个Docker镜像,一个是Maven负责java的build,一个是jnlp创建一个jenkins的slave。
3.3 挂载存储卷
3.3.1、主要是把maven的repository挂载到宿主机上,以方便多次构建后复用。
3.3.2、宿主机上安装了docker,把docker命令挂载到K8S的POD内,使得POD具备了
docker in docker的能力,如此就可以在该POD内进行docker构建。
3.3.3、宿主机一般都是被K8S控制的Node,因此一般都装有kubernets,因此可以把kubectl命令关在到POD中,使得POD具备执行kubectl命令能力,以控制K8S。
请用如下代码给k8s内的jenkins账号授权
---
apiVersion: v1
kind: ServiceAccount
metadata:name: jenkinsnamespace: zo-jenkins
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: jenkinsnamespace: zo-jenkins
rules:- apiGroups: [""]resources: ["pods"]verbs: ["create","delete","get","list","patch","update","watch"]- apiGroups: [""]resources: ["pods/portforward"]verbs: ["*"]- apiGroups: [""]resources: ["pods/exec"]verbs: ["create","delete","get","list","patch","update","watch"]- apiGroups: [""]resources: ["pods/log"]verbs: ["get","list","watch"]- apiGroups: [""]resources: ["secrets"]verbs: ["get"]- apiGroups: ["apps"]resources: ["deployments"]verbs: ["*"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:name: jenkinsnamespace: zo-jenkins
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: jenkins
subjects:- kind: ServiceAccountname: jenkinsnamespace: zo-jenkins---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: jenkinsClusterRolenamespace: zo-jenkins
rules:- apiGroups: [""]resources: ["pods"]verbs: ["create","delete","get","list","patch","update","watch"]- apiGroups: [""]resources: ["pods/exec"]verbs: ["create","delete","get","list","patch","update","watch"]- apiGroups: [""]resources: ["pods/log"]verbs: ["get","list","watch"]- apiGroups: [""]resources: ["secrets"]verbs: ["get"]- apiGroups: [ "apps" ]resources: ["deployments"]verbs: ["*"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: jenkinsClusterRuleBinding
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: jenkinsClusterRole
subjects:- kind: ServiceAccountname: jenkinsnamespace: zo-jenkins
4、JAVA应用中增加pipeline脚本
#!groovy
pipeline {agent {kubernetes {cloud "k8s-deploy" //选择名字是kubernetes1的cloud,这里不要修改label "jenkins-slave"//yamlFile '' //jenkins-slave部署文件位置,内容不要修改}}stages {stage('初始化环境') {steps {script {env.APP_NAME = "report-server" //修改为自己的应用名称env.APP_CODE = "https://codeup.aliyun.com/xxxxxxx/outsourced/excleToReport/xxx-xxxx.git"env.CODE_BRANCH = "master" //修改为实际部署的分支env.DOCKER_IMAGE_URL = "registry.cn-hangzhou.aliyuncs.com/xx-xx/xx-xxx" // 修改DOCKER镜像地址env.DOCKER_TAG = "1.0.0"env.K8S_NAME_SPACE = "zo-home-prd" // 修改应用在K8S内的实际部署的命名空间}}}stage('拉取代码') {steps {echo "1.Git Clone Code"git branch: "$CODE_BRANCH", credentialsId: 'zo-git-yun', url: "$APP_CODE"}}stage('maven构建') {steps {container('maven') {sh 'mvn -s jenkins/settings.xml clean package -e -U -Dmaven.test.skip=true -Dautoconfig.skip'}}}stage('docker镜像构建') {steps {container('maven') { // maven容器确保与宿主机的docker实现docker in docker的能力sh 'docker build -t $APP_NAME:$DOCKER_TAG .'sh 'docker tag $APP_NAME:$DOCKER_TAG $DOCKER_IMAGE_URL:$DOCKER_TAG'sh 'docker push $DOCKER_IMAGE_URL:$DOCKER_TAG'}}}// 部署到K8sstage('K8S部署-开发环境') {steps {container('jenkins-jnlp-slave') {withKubeConfig(serverUrl: 'https://192.168.10.200:6443') {// 这里配置K8S Master的API地址echo 'begin k8s deploy!'sh 'chmod 744 k8s/home-prd/deployment.sh'sh 'k8s/home-prd/deployment.sh' // 执行部署// --record 用作记录滚动更新的信息, 后面方便版本回退//sh 'kubectl set image deployment/$APP_NAME $APP_NAME=$IMAGE_URL --namespace $NAME_SPACE --record'}}}}}options {buildDiscarder(logRotator(numToKeepStr: '5', artifactNumToKeepStr: '5'))}
}