背景:
注:在kubernetes集群上部署单机版的zookeeper+kafka服务,是采用了kubernetes中的deploment组件+service组件+pvc存储组件
1、部署zookeeper服务:
注:这里时候的镜像是:dockerhub.jiang.com/jiang-public/zookeeper:3.5.9
1. 镜像下载地址:
registry.cn-hangzhou.aliyuncs.com/images-speed-up/zookeeper:3.5.9
2. 开发deploment控制器的配置yaml:
kind: Deployment
metadata:name: zookeeper-kultznamespace: sitlabels:app: zookeeper-kultzname: zookeeperversion: v3.5.9
spec:replicas: 1selector:matchLabels:app: zookeeper-kultzname: zookeepertemplate:metadata:labels:app: zookeeper-kultzname: zookeeperversion: v3.5.9spec:volumes:- name: zookeeper-pvcpersistentVolumeClaim:claimName: zookeeper-pvccontainers:- name: zookeeperimage: 'dockerhub.jiang.com/jiang-public/zookeeper:3.5.9'ports:- containerPort: 2181protocol: TCPenv:- name: ALLOW_ANONYMOUS_LOGINvalue: 'yes'resources:limits:cpu: '1'memory: 2Girequests:cpu: 800mmemory: 2GivolumeMounts:- name: zookeeper-pvcmountPath: /bitnami/zookeeper/dataterminationMessagePath: /dev/termination-logterminationMessagePolicy: FileimagePullPolicy: IfNotPresentsecurityContext:privileged: falserestartPolicy: AlwaysterminationGracePeriodSeconds: 30dnsPolicy: ClusterFirstserviceAccountName: defaultserviceAccount: defaultsecurityContext:runAsUser: 0fsGroup: 0imagePullSecrets:- name: user-1-registrysecretaffinity: {}schedulerName: default-schedulerstrategy:type: RecreateminReadySeconds: 10revisionHistoryLimit: 10progressDeadlineSeconds: 600
注:这里有两点需要注意的,
1、是env的配置, ALLOW_ANONYMOUS_LOGIN='yes'
2、securityContext:的配置,需要设置 runAsUser: 0和fsGroup: 0,否则提示报错:
mkdir: cannot create directory '/bitnami/zookeeper/data': Permission denied不然zookeeper服务会无法启动起来。
pvc存储是挂载了/bitnami/zookeeper/data位置,这个地址是zoo.cfg里的配置。
3. pvc存储yaml配置:
apiVersion: storage.k8s.io/v1
kind: PersistentVolumeClaim
metadata:name: zookeeper-pvcnamespace: sitfinalizers:- kubernetes.io/pvc-protection
spec:accessModes:- ReadWriteOnceresources:requests:storage: 10GistorageClassName: hpe-sanvolumeMode: Filesystem
注:这里是用storageclaas控制器,所以这里就不多介绍了。
4.zookeeper的service控制器配置:
kind: Service
metadata:name: zookeepernamespace: sitlabels:name: zookeepersystem/appName: nginxdemo0516
spec:ports:- name: tcp-port-0protocol: TCPport: 2181targetPort: 2181selector:name: zookeepertype: ClusterIPsessionAffinity: None
status:loadBalancer: {}
5.运行zookeeper容器放服务:
先创建pvc存储:
# kubectl apply -f zookeeper-pvc.yaml
再次创建deployment:
# kubectl apply -f zookeeper-deploy.yaml
最后创建zookeeper的service服务:
# kubectl apply -f zookeeper-svc.yaml
2、部署kafka服务:
注:这里时候的镜像是:dockerhub.jiang.com/jiang-public/kafka:3.2.1
1. 镜像下载地址:
registry.cn-hangzhou.aliyuncs.com/images-speed-up/kafka:3.2.1
2. 开发deploment控制器的配置yaml:
kind: Deployment
metadata:name: kafka-jbhpbnamespace: sitlabels:app: kafka-jbhpbname: kafkaversion: v3.2.1
spec:replicas: 1selector:matchLabels:app: kafka-jbhpbname: kafkatemplate:metadata:labels:app: kafka-jbhpbname: kafkaversion: v3.2.1spec:volumes:- name: kafka-pvcpersistentVolumeClaim:claimName: kafka-pvccontainers:- name: kafkaimage: 'dockerhub.jiang.com/jiang-public/kafka:3.2.1'ports:- containerPort: 9092protocol: TCPenv:- name: KAFKA_ZOOKEEPER_CONNECTvalue: 'zookeeper.sit:2181'- name: ALLOW_PLAINTEXT_LISTENERvalue: 'yes'resources:limits:cpu: '1'memory: 2Girequests:cpu: 800mmemory: 2GivolumeMounts:- name: kafka-pvcmountPath: /bitnami/kafka/data/terminationMessagePath: /dev/termination-logterminationMessagePolicy: FileimagePullPolicy: IfNotPresentsecurityContext:privileged: falserestartPolicy: AlwaysterminationGracePeriodSeconds: 30dnsPolicy: ClusterFirstserviceAccountName: defaultserviceAccount: defaultsecurityContext:runAsUser: 0fsGroup: 0imagePullSecrets:- name: user-1-registrysecretaffinity: {}schedulerName: default-schedulerstrategy:type: RecreateminReadySeconds: 10revisionHistoryLimit: 10progressDeadlineSeconds: 600
注:这里有两点需要注意的,
1、是env的配置,
KAFKA_ZOOKEEPER_CONNECT='zookeeper.sit:2181'
ALLOW_PLAINTEXT_LISTENER=yes2、securityContext:的配置,需要设置 runAsUser: 0和fsGroup: 0,不然挂载存储时,会提示报错:
mkdir: cannot create directory '/bitnami/kafka/data': Permission denied不然kafka服务会无法启动起来。
pvc存储是挂载了/bitnami/kafka/data位置,这个地址是server.properties里的配置。
3. pvc存储yaml配置:
apiVersion: storage.k8s.io/v1
kind: PersistentVolumeClaim
metadata:name: kafka-pvcnamespace: sitfinalizers:- kubernetes.io/pvc-protection
spec:accessModes:- ReadWriteOnceresources:requests:storage: 5GistorageClassName: hpe-sanvolumeMode: Filesystem
4.kafka的service控制器配置:
kind: Service
metadata:name: kafkanamespace: sitlabels:name: kafkasystem/appName: nginxdemo0516
spec:ports:- name: tcp-port-0protocol: TCPport: 9092targetPort: 9092selector:name: kafkatype: ClusterIPsessionAffinity: None
status:loadBalancer: {}
5.运行kafka容器放服务:
先创建pvc存储:
# kubectl apply -f kafka-pvc.yaml
再次创建deployment:
# kubectl apply -f kafka-deploy.yaml
最后创建zookeeper的service服务:
# kubectl apply -f kafka-svc.yaml
3、测试kafka功能:
在kafak容器里操作:
创建topic:
root@kafka-jbhpb-78bb6df4dc-xhmp6:/opt/bitnami/kafka/bin# ./kafka-topics.sh --bootstrap-server localhost:9092 --create --topic my-topic --partitions 1 --replication-factor 1
Created topic my-topic.
查看topic:
root@kafka-jbhpb-78bb6df4dc-xhmp6:/opt/bitnami/kafka/bin# ./kafka-topics.sh --bootstrap-server localhost:9092 --list
my-topic
这里就是成功了。