Kubernetes(k8s)离线部署DolphinScheduler3.2.2

1.环境准备

1.1 集群规划

本次安装环境为：3台k8s+现有的postgreSql数据库+zookeeper服务

1.2 下载及介绍

DolphinScheduler-3.2.2官网：https://dolphinscheduler.apache.org/zh-cn/docs/3.2.2
官网安装文档：https://dolphinscheduler.apache.org/zh-cn/docs/3.2.2/guide/installation/kubernetes

2.前置工作

默认k8s集群已经安装完成，本次以kubesphere为例介绍
参考kubesphere官方文档：https://kubesphere.io/zh/

2.1 Helm配置

下载 Helm 客户端
下载地址：https://github.com/helm/helm/releases

# 下载
wget https://get.helm.sh/helm-v3.14.3-linux-amd64.tar.gz# 解压
tar -zxvf helm-v3.14.3-linux-amd64.tar.gz# 移动到可执行目录
mv linux-amd64/helm /usr/local/bin/helm# 查看 helm 版本（确保可正常运行）
helm version
#version.BuildInfo{Version:"v3.14.3", GitCommit:"f03cc04caaa8f6d7c3e67cf918929150cf6f3f12", GitTreeState:"clean", GoVersion:"go1.21.7"}

helm常用命令举例

#helm 删除 官方仓库
helm repo remove  stable  
#仓库管理)查看添加的chart仓库，可在这些chart仓库中拉取chart
helm repo list#将chart包发布到k8s集群中安装部署
helm install releaseName chartName  
#列出所有已发布的版本
helm list
helm list -n test
helm list -A

2.2 下载dolphinscheduler镜像

dolphinscheduler镜像

# 拉取镜像
docker pull hub.rat.dev/apache/dolphinscheduler-tools:latest
docker pull hub.rat.dev/apache/dolphinscheduler-api:latest
docker pull hub.rat.dev/apache/dolphinscheduler-alter-server:latest
docker pull hub.rat.dev/apache/dolphinscheduler-master:latest
docker pull hub.rat.dev/apache/dolphinscheduler-worker:latest
# 导出镜像
docker  save -o api.tar hub.rat.dev/apache/dolphinscheduler-api:latest
docker  save -o tools.tar hub.rat.dev/apache/dolphinscheduler-tools:latest
docker  save -o alter-server.tar hub.rat.dev/apache/dolphinscheduler-alter-server:latest
docker  save -o master.tar hub.rat.dev/apache/dolphinscheduler-master:latest
docker  save -o worker.tar hub.rat.dev/apache/dolphinscheduler-worker:latest

其他镜像

如果没有zookeeper与postgreSql需要用这下面的镜像进行安装

 docker pull  hub.rat.dev/bitnami/zookeeper:3.7.1docker pull  hub.rat.dev/bitnami/postgresql:15.2.0

2.3 导入dolphinscheduler镜像

以dolphinscheduler-api为例，其他镜像请参照api的例子

# 导入镜像
docker load --input api.tar 
# 重新打标签
docker tag hub.rat.dev/apache/dolphinscheduler-api:latest registry.flow.cn/apache/dolphinscheduler-api:latest
# 推送镜像
docker push registry.flow.cn/apache/dolphinscheduler-api:latest

3.dolphinscheduler集群安装

集群使用会单独启动一个psql以及zookeeper
请下载源码包 https://github.com/apache/dolphinscheduler/archive/refs/tags/3.2.2.zip，下载
发布一个名为 dolphinscheduler 的版本(release)，请执行以下命令：

因为是单独启动的psql和zookeeper则需要把Chat.yaml的dependencies注释掉

cd apache-dolphinscheduler-3.2.2-src/deploy/kubernetes/dolphinscheduler
vim Chart.yaml

在这里插入图片描述
修改镜像仓库地址

vim values.yaml

在这里插入图片描述

将名为 dolphinscheduler 的版本(release) 发布到 test 的命名空间中：

helm install dolphinscheduler . -n test

提示: 如果名为 test 的命名空间被使用, 选项参数 -n test 需要添加到 helm 和 kubectl 命令中

提示: 列出所有已发布的版本，使用 helm list，PostgreSQL (用户 root, 密码 root, 数据库 dolphinscheduler) 和 ZooKeeper 服务将会默认启动

查看运行状态
在这里插入图片描述

创建服务外部访问端口

名称随便填然后点击下一步

端口配置12345

外部访问选择nodePort，然后点击创建

访问前端页面：http://k8s节点ip:32312/dolphinscheduler/ui，如果有需要请修改成对应的 IP 地址(k8s节点ip)

在这里插入图片描述

默认的用户是admin，默认的密码是dolphinscheduler123
在这里插入图片描述

请参考用户手册章节的 快速上手 查看如何使用 DolphinScheduler

4. 问题解决

4.1 数据裤初始化报错

因为没有修改zookeeper、postgreSql地址

kind: Job
apiVersion: batch/v1
metadata:name: dolphinscheduler-db-init-jobnamespace: testlabels:app.kubernetes.io/instance: dolphinschedulerapp.kubernetes.io/managed-by: Helmannotations:helm.sh/hook: 'post-install,post-upgrade,post-rollback'helm.sh/hook-weight: '1'revisions: >-{"1":{"status":"running","desire":1,"uid":"cd3b4532-7abd-470e-885f-e42dc4c6dea0","start-time":"2024-12-19T14:43:47+08:00","completion-time":"0001-01-01T00:00:00Z"}}
spec:parallelism: 1completions: 1backoffLimit: 6selector:matchLabels:#batch.kubernetes.io/controller-uid: cd3b4532-7abd-470e-885f-e42dc4c6dea0template:metadata:creationTimestamp: nulllabels:app.kubernetes.io/instance: dolphinschedulerapp.kubernetes.io/managed-by: Helm#batch.kubernetes.io/controller-uid: cd3b4532-7abd-470e-885f-e42dc4c6dea0batch.kubernetes.io/job-name: dolphinscheduler-db-init-job#controller-uid: cd3b4532-7abd-470e-885f-e42dc4c6dea0job-name: dolphinscheduler-db-init-jobspec:initContainers:- name: wait-for-databaseimage: 'registry.flow.cn/library/busybox:latest'command:- sh- '-xc'- >-for i in $(seq 1 180); do nc -z -w3 dolphinscheduler-postgresql-f68v.data-center.svc.cluster.local5432 && exit 0 || sleep 5; done; exit 1resources: {}terminationMessagePath: /dev/termination-logterminationMessagePolicy: FileimagePullPolicy: IfNotPresentcontainers:- name: dolphinscheduler-db-init-jobimage: 'registry.flow.cn/apache/dolphinscheduler-tools:latest'args:- tools/bin/upgrade-schema.shenvFrom:- configMapRef:name: dolphinscheduler-commonenv:- name: TZvalue: Asia/Shanghai- name: SPRING_JACKSON_TIME_ZONEvalue: Asia/Shanghai- name: DATABASEvalue: postgresql- name: SPRING_DATASOURCE_URLvalue: >-jdbc:postgresql://dolphinscheduler-postgresql-f68v.data-center.svc.cluster.local:5432/dolphinscheduler?characterEncoding=utf8- name: SPRING_DATASOURCE_USERNAMEvalue: postgres- name: SPRING_DATASOURCE_PASSWORDvalue: postgres- name: SPRING_DATASOURCE_DRIVER-CLASS-NAMEvalue: org.postgresql.Driver- name: REGISTRY_TYPEvalue: zookeeper- name: REGISTRY_ZOOKEEPER_CONNECT_STRINGvalue: 'dolphinscheduler-zookeepe-headless.data-center.svc.cluster.local:2181'- name: DATABASE_TYPEvalue: postgresql- name: JAVA_OPTSvalue: -server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE} -Dspring.profiles.active=postgresql -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprofresources: {}terminationMessagePath: /dev/termination-logterminationMessagePolicy: FileimagePullPolicy: IfNotPresentrestartPolicy: NeverterminationGracePeriodSeconds: 30dnsPolicy: ClusterFirstsecurityContext: {}imagePullSecrets:- name: registry-configschedulerName: default-schedulercompletionMode: NonIndexedsuspend: false

需要修改REGISTRY_ZOOKEEPER_CONNECT_STRING和SPRING_DATASOURCE_URL的值，然后重新创建任务。
在这里插入图片描述
数据库初始完成之后会创建表

4.2 `Caused by: java.net.UnknownHostException: s3`

修改common.properties，加上下面3行

aws.s3.endpoint=<minio地址>
aws.s3.access.key.id=<your id>
aws.s3.access.key.secret=<your secret>

然后重启

4.3 zookeeper和psql报错

需要修改pod环境的变量，修改为正确的地址
在这里插入图片描述

4.4 worker节点 `No qualifying bean of type 'org.apache.dolphinscheduler.meter.metrics.MetricsProvider' available`

错误信息：

2024-12-21T22:50:53.537397234+08:00 Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'workerRegistryClient': Unsatisfied dependency expressed through field 'metricsProvider'; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'org.apache.dolphinscheduler.meter.metrics.MetricsProvider' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}2024-12-21T22:50:53.537401243+08:00 	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:659)2024-12-21T22:50:53.537408792+08:00 	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:639)2024-12-21T22:50:53.537412721+08:00 	at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:119)2024-12-21T22:50:53.537416353+08:00 	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399)2024-12-21T22:50:53.537418799+08:00 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1431)2024-12-21T22:50:53.537421259+08:00 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:619)2024-12-21T22:50:53.537428244+08:00 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)2024-12-21T22:50:53.537430992+08:00 	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)2024-12-21T22:50:53.537433562+08:00 	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)2024-12-21T22:50:53.537436073+08:00 	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)2024-12-21T22:50:53.537438469+08:00 	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)2024-12-21T22:50:53.537441051+08:00 	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)2024-12-21T22:50:53.537443569+08:00 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1391)2024-12-21T22:50:53.537446141+08:00 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311)2024-12-21T22:50:53.537448881+08:00 	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:656)2024-12-21T22:50:53.537451540+08:00 	... 20 common frames omitted2024-12-21T22:50:53.537454771+08:00 Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'org.apache.dolphinscheduler.meter.metrics.MetricsProvider' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}2024-12-21T22:50:53.537457334+08:00 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.raiseNoMatchingBeanFound(DefaultListableBeanFactory.java:1801)2024-12-21T22:50:53.537459878+08:00 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1357)2024-12-21T22:50:53.537462469+08:00 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311)2024-12-21T22:50:53.537465066+08:00 	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:656)2024-12-21T22:50:53.537467531+08:00 	... 34 common frames omitted

进入到worker节点源码目录
把resources下面的文件全部挂载在到/opt/dolphinscheduler/conf目录下面
在这里插入图片描述

新增worker配置，然后把resources目录下面的文件内容复制进来

挂载到/opt/dolphinscheduler/conf路径

挂载了之后点击确定即可。