Seatunnel-2.3.3 自打包 docker部署(含web)

devtools/2024/10/18 0:25:51/

前言

此篇重点是,自己将源码编译后,将打包文件部署在docker里(也可以直接用官网的)

如果也有人是希望,将自己打包的源码部署了,可以参考可乐的这篇文章,这篇文章详细介绍了2.3.3的serve和web的源码启动流程

Apache Seatunnel本地源码构建编译运行调试

前期准备

  1. docker环境
  2. seatunnel-2.3.3  二进制执行文件(apache-seatunnel-2.3.4-bin.tar.gz解压后的文件)
  3. seatunnel-web-1.0.0 二进制可执行文件
  4. cp /opt/seatunnel/apache-seatunnel-2.3.3/config/hazelcast-client.yaml /opt/seatunnel/apache-seatunnel-web-1.0.0-bin/conf/

4的命令不用执行,只要是给大家一个参考,记得吧hazelcast-client.yaml文件覆盖到二进制里面去,同时hazelcast-client.yaml的内容改一下,主要把127或者localhost改为本机宿主机ip

提示:  这里我就默认大家,已经解决了所有,可乐文章里的疑难杂症,也就是说,

源码是可以添加数据源,可以添加任务执行单表操作的。

如果!!! 不能保障!!!请先看可乐文章后,再阅读本篇,不然的话,真的会很痛苦,要不断编译,不断部署!!!

文件目录介绍

--docer-compose.yml

--seatunnel

        --Dockerfile

        --Dockerfile_web

        --apache-seatunnel-2.3.3

        --apache-seatunnel-web-1.0.0

文件内容

docker-compose.yml

version: '3'
services:seatunnel-serve:build:context: ./seatunnel/dockerfile: Dockerfilecontainer_name: seatunnel_serveports:- 5801:5801command: nohup sh bin/seatunnel-cluster.sh 2>&1 &volumes:- ./seatunnel/apache-seatunnel-2.3.3/:/opt/seatunnel/restart: alwaysnetworks:- appseatunnel-web:build:context: ./seatunnel/dockerfile: Dockerfile_webvolumes:- ./seatunnel/apache-seatunnel-web-1.0.0/:/opt/seatunnel_web/container_name: seatunnel_webentrypoint: /opt/seatunnel_web/bin/chmod-daemon.shports:- 8801:8801restart: alwaysnetworks:- app
volumes:data:
networks:app:external: true

Dockerfile

这个其实也可以不用,但是有个小瑕疵没解决,先预留

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#FROM openjdk:8
ENV SEATUNNEL_HOME="/opt/seatunnel"
RUN  mkdir -p SEATUNNEL_HOME/logs
WORKDIR /opt/seatunnel

Dockerfile_web

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#FROM openjdk:8
user root
ENV SEATUNNEL_HOME="/opt/seatunnel_web"
COPY ./apache-seatunnel-web-1.0.0/ ${SEATUNNEL_HOME}/
RUN  mkdir -p SEATUNNEL_HOME/logs
RUN chmod +x /opt/seatunnel_web/bin/seatunnel-backend-daemon.sh
RUN chmod +x /opt/seatunnel_web/bin/chmod-daemon.sh
WORKDIR /opt/seatunnel_web
#CMD ["chmod"," +x", "bin/seatunnel-backend-daemon.sh"]
#CMD ["sh", "/opt/seatunnel_web/bin/seatunnel-backend-daemon.sh", "start"]CMD ["tail"," -f", "nohup.out"]

chmod-daemon.sh

这里有个小瑕疵

 sh /opt/seatunnel_web/bin/seatunnel-backend-daemon.sh start权限问题一直没有解决

所以我在seatunnel-backend-daemon.sh打了echo,把参数都打印出来了,打印出来后直接贴到了这里,这个后期我解决了,我会上报解决方案

注意!!!chmod-daemon.sh会存在sh文件格式的问题

映射进去的时候可能是CRLF的格式,linux识别不来,所以在docker-compose里的entrypoint查找文件的时候,会报no such file or directory

注意文件格式是LF还是CRLF

LF   linux

CRLF  window

#!/bin/sh#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
#echo test111111111
chmod 777 /opt/seatunnel_web/bin/seatunnel-backend-daemon.sh
echo 111111111111111111111111123/usr/local/openjdk-8/bin/java -server -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof -Dseatunnel-web.logs.path=/opt/seatunnel_web/bin/../logs -cp /opt/seatunnel_web/bin/../conf:/opt/seatunnel_web/bin/../libs/*:/opt/seatunnel_web/bin/../datasource/* -Dspring.config.name=application.yml -Dspring.config.location=classpath:application.yml org.apache.seatunnel.app.SeatunnelApplication
# sh /opt/seatunnel_web/bin/seatunnel-backend-daemon.sh start

还有一种解决方案(这是可乐提供的解决方案,不过没有测试,先贴在这里)

sh 如果是在windows下直接打开复制过去的,会有让shell文件的格式有问题,镜像运行的时候报初始化脚本\r问题,所以需要在wsl进入centOs子系统中执行如下:
sed -i ‘s/\r$//’ filename
将xxxx.sh格式转为linux下的格式


http://www.ppmy.cn/devtools/23441.html

相关文章

Spark持久化、broadcast广播变量和accumulator累加器

持久化操作 什么是持久化,为什么要持久化 Spark中最重要的功能之一是跨操作在内存中持久化(或缓存)数据集。当您持久化RDD时,每个节点将其计算的任何分区存储在内存中,并在该数据集(或从该数据集派生的数…

OpenWRT设置自动获取IP,作为二级路由器

前言 上一期咱们讲了在OpenWRT设置PPPoE拨号的教程,在光猫桥接的模式下,OpenWRT如果不设置PPPoE拨号,就无法正常上网。 OpenWRT设置PPPoE拨号教程 但现在很多新装的宽带,宽带师傅为了方便都会把光猫设置为路由模式。如果你再外…

C语言趣味代码(四)

这一篇主要编写几个打字练习的小程序,然后通过这些小程序的实现来回顾复习我们之前学过的知识,然后通过这写打字练习的小程序来提升我们的打字技术和编程技术。 1. 打字练习 1.1 基本打字练习 1.1.1 基本实现 首先我们来制作一个用于计算并显示输入一…

优化ES搜索性能

最近做的舆情系统用到了ES,比数据库的效率要高得多,不过我也一直在顾虑经过长时间的运行之后数据编的特别多,到时ES也变得吃力,最近也有一些了解 优化ES主要从两方面考虑,硬件和软件 集群和硬件优化 使用更好的硬件肯定是能快速解决问题的,但是这需要钱,我们说了不算,所以主…

第6篇:创建Nios II工程之控制LED<一>

Q:还记得第1篇吗?设计简单的逻辑电路,控制DE2-115开发板上LED的亮与熄灭,一行Verilog HDL的assign赋值语句即可实现。本期开始创建Nios II工程,用C语言代码控制DE2-115开发板上的LED实现流水灯效果。 A:在…

TP8 利用jwt 生成token

使用Composer安装依赖库&#xff0c;终端切换到项目根目录输入如下 composer require firebase/php-jwt 下面是示例代码&#xff1a; <?php namespace app\common\base;use Firebase\JWT\JWT; use Firebase\JWT\Key;class Token {/*** 创建 token* param array $data 必填…

Java8 Stream常见用法

Stream流的常见用法&#xff1a; 1.利用stream流特性把数组转list集合 //定义一个数组Integer[] array {5,2,1,6,4,3};//通过stream特性把数组转list集合List<Integer> list Arrays.stream(array).collect(Collectors.toList());//打印结果System.out.println(list);…

计算机系统概述试题

01&#xff0e;完整的计算机系统应包括()。 A.运算器、存储器、控制器 B.外部设备和主机 C.主机和应用程序 D.配套的硬件设备和软件系统 02.冯诺依曼机的基本工作方式是()。 A.控制流驱动方式 B.多指令多数据流方式 C.微程序控制方式 D.数据流驱动方式 03.下列()是冯诺依曼机…