19.3 打镜像部署到k8s中，prometheus配置采集并在grafana看图

ops/2024/10/19 1:42:16/

本节重点介绍 :

  • 打镜像，导出镜像，传输到各个节点并导入
  • 运行该项目
  • 配置prometheus和grafana

打镜像

本地build


docker build -t ink8s-pod-metrics:v1 .

build过程

image.png

导出镜像


docker save  ink8s-pod-metrics > ink8s-pod-metrics.tar

传输到各个node节点上

scp ink8s-pod-metrics.tar k8s-node01:~

各个node节点上导入镜像

使用docker

docker load < ink8s-pod-metrics.tar

使用containerd

ctr --namespace k8s.io images import ink8s-pod-metrics.tar

运行该项目

 kubectl apply -f rbac.yamlkubectl apply -f deployment.yaml

检查

[root@k8s-master01 ink8s-pod-metrics]# kubectl get pod -o wide 
NAME                                           READY   STATUS    RESTARTS   AGE    IP              NODE         NOMINATED NODE   READINESS GATES
grafana-d5d85bcd6-f74ch                        1/1     Running   0          3d9h   10.100.85.199   k8s-node01   <none>           <none>
grafana-d5d85bcd6-l44mx                        1/1     Running   0          3d9h   10.100.85.198   k8s-node01   <none>           <none>
ink8s-pod-metrics-deployment-85d9795d6-95lsp   1/1     Running   0          13m    10.100.85.207   k8s-node01   <none>           <none>
  • 日志
[root@k8s-master01 ink8s-pod-metrics]# kubectl logs -l app=ink8s-pod-metrics  -f  
2021-08-23 20:34:35.377256 INFO app/get_k8s_objs.go:128 [pod.label:map[component:etcd tier:control-plane]]
2021-08-23 20:34:35.377266 INFO app/get_k8s_objs.go:128 [pod.label:map[component:kube-apiserver tier:control-plane]]
2021-08-23 20:34:35.377274 INFO app/get_k8s_objs.go:128 [pod.label:map[component:kube-controller-manager tier:control-plane]]
2021-08-23 20:34:35.377292 INFO app/get_k8s_objs.go:128 [pod.label:map[controller-revision-hash:85c698c6d4 k8s-app:kube-proxy pod-template-generation:1]]
2021-08-23 20:34:35.377299 INFO app/get_k8s_objs.go:128 [pod.label:map[controller-revision-hash:85c698c6d4 k8s-app:kube-proxy pod-template-generation:1]]
2021-08-23 20:34:35.377317 INFO app/get_k8s_objs.go:128 [pod.label:map[component:kube-scheduler tier:control-plane]]
2021-08-23 20:34:35.377324 INFO app/get_k8s_objs.go:128 [pod.label:map[app.kubernetes.io/name:kube-state-metrics app.kubernetes.io/version:v1.9.7 pod-template-hash:564668c858]]
2021-08-23 20:34:35.377331 INFO app/get_k8s_objs.go:128 [pod.label:map[k8s-app:metrics-server pod-template-hash:7dbf6c4558]]
2021-08-23 20:34:35.377336 INFO app/get_k8s_objs.go:128 [pod.label:map[controller-revision-hash:prometheus-5b9cdcfd6c k8s-app:prometheus statefulset.kubernetes.io/pod-name:prometheus-0]]
2021-08-23 20:34:35.377358 INFO app/get_k8s_objs.go:143 server_pod_ips_result][num_pod:11][time_took_seconds:6.189551999]
2021-08-23 20:34:39.197614 INFO app/get_k8s_objs.go:107 server_node_ips_result][num_node:2][time_took_seconds:0.009575987]
2021-08-23 20:34:39.200824 INFO app/get_k8s_objs.go:128 [pod.label:map[k8s-app:kube-dns pod-template-hash:68b9d7b887]]
2021-08-23 20:34:39.200857 INFO app/get_k8s_objs.go:128 [pod.label:map[k8s-app:kube-dns pod-template-hash:68b9d7b887]]
2021-08-23 20:34:39.200871 INFO app/get_k8s_objs.go:128 [pod.label:map[component:etcd tier:control-plane]]
2021-08-23 20:34:39.200889 INFO app/get_k8s_objs.go:128 [pod.label:map[component:kube-apiserver tier:control-plane]]
2021-08-23 20:34:39.200903 INFO app/get_k8s_objs.go:128 [pod.label:map[component:kube-controller-manager tier:control-plane]]
2021-08-23 20:34:39.200920 INFO app/get_k8s_objs.go:128 [pod.label:map[controller-revision-hash:85c698c6d4 k8s-app:kube-proxy pod-template-generation:1]]
2021-08-23 20:34:39.200934 INFO app/get_k8s_objs.go:128 [pod.label:map[controller-revision-hash:85c698c6d4 k8s-app:kube-proxy pod-template-generation:1]]
2021-08-23 20:34:39.200947 INFO app/get_k8s_objs.go:128 [pod.label:map[component:kube-scheduler tier:control-plane]]
2021-08-23 20:34:39.200961 INFO app/get_k8s_objs.go:128 [pod.label:map[app.kubernetes.io/name:kube-state-metrics app.kubernetes.io/version:v1.9.7 pod-template-hash:564668c858]]
2021-08-23 20:34:39.200981 INFO app/get_k8s_objs.go:128 [pod.label:map[k8s-app:metrics-server pod-template-hash:7dbf6c4558]]
2021-08-23 20:34:39.200992 INFO app/get_k8s_objs.go:128 [pod.label:map[controller-revision-hash:prometheus-5b9cdcfd6c k8s-app:prometheus statefulset.kubernetes.io/pod-name:prometheus-0]]
2021-08-23 20:34:39.201022 INFO app/get_k8s_objs.go:143 server_pod_ips_result][num_pod:11][time_took_seconds:0.013052527]

node上请求 pod 的metrics

  • curl pod的ip:8080/metrics
[root@k8s-master01 ink8s-pod-metrics]# curl -s 10.100.85.207:8080/metrics |grep ink8s
# HELP ink8s_pod_metrics_get_node_detail k8s node detail each
# TYPE ink8s_pod_metrics_get_node_detail gauge
ink8s_pod_metrics_get_node_detail{containerRuntimeVersion="containerd://1.4.4",hostname="k8s-master01",ip="172.20.70.205",kubeletVersion="v1.20.1"} 1
ink8s_pod_metrics_get_node_detail{containerRuntimeVersion="containerd://1.4.4",hostname="k8s-node01",ip="172.20.70.215",kubeletVersion="v1.20.1"} 1
# HELP ink8s_pod_metrics_get_node_last_duration_seconds get node last duration seconds
# TYPE ink8s_pod_metrics_get_node_last_duration_seconds gauge
ink8s_pod_metrics_get_node_last_duration_seconds 0.008066143
# HELP ink8s_pod_metrics_get_pod_control_plane_pod_detail k8s pod detail of control plane
# TYPE ink8s_pod_metrics_get_pod_control_plane_pod_detail gauge
ink8s_pod_metrics_get_pod_control_plane_pod_detail{component="etcd",ip="172.20.70.205",pod_name="etcd-k8s-master01"} 1
ink8s_pod_metrics_get_pod_control_plane_pod_detail{component="kube-apiserver",ip="172.20.70.205",pod_name="kube-apiserver-k8s-master01"} 1
ink8s_pod_metrics_get_pod_control_plane_pod_detail{component="kube-controller-manager",ip="172.20.70.205",pod_name="kube-controller-manager-k8s-master01"} 1
ink8s_pod_metrics_get_pod_control_plane_pod_detail{component="kube-scheduler",ip="172.20.70.205",pod_name="kube-scheduler-k8s-master01"} 1
# HELP ink8s_pod_metrics_get_pod_last_duration_seconds get pod last duration seconds
# TYPE ink8s_pod_metrics_get_pod_last_duration_seconds gauge
ink8s_pod_metrics_get_pod_last_duration_seconds 0.01159838

prometheus_targetpod_119">prometheus target页面检查pod

image.png

  • 发现pod已经发现到了，但是 报错：向http的server发送https的请求

prometheusgrafana_125">prometheus和grafana配置

kubernetespodsjob_127">检查 kubernetes-pods的job

  • 如果之前配置的https，需要改为http的
- job_name: kubernetes-podshonor_timestamps: truescrape_interval: 30sscrape_timeout: 10smetrics_path: /metricsscheme: httpfollow_redirects: truerelabel_configs:- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]separator: ;regex: "true"replacement: $1action: keep- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]separator: ;regex: (.+)target_label: __metrics_path__replacement: $1action: replace- source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]separator: ;regex: ([^:]+)(?::\d+)?;(\d+)target_label: __address__replacement: $1:$2action: replace- separator: ;regex: __meta_kubernetes_pod_label_(.+)replacement: $1action: labelmap- source_labels: [__meta_kubernetes_namespace]separator: ;regex: (.*)target_label: kubernetes_namespacereplacement: $1action: replace- source_labels: [__meta_kubernetes_pod_name]separator: ;regex: (.*)target_label: kubernetes_pod_namereplacement: $1action: replacekubernetes_sd_configs:- role: podkubeconfig_file: ""follow_redirects: true

prometheus_179">在prometheus中检查指标

  • 查询 ink8s_pod_metrics_get_node_detail
  • image.png
ink8s_pod_metrics_get_node_detail{app="ink8s-pod-metrics", containerRuntimeVersion="containerd://1.4.4", hostname="k8s-master01", instance="10.100.85.207:8080", ip="172.20.70.205", job="kubernetes-pods", kubeletVersion="v1.20.1", kubernetes_namespace="default", kubernetes_pod_name="ink8s-pod-metrics-deployment-85d9795d6-95lsp", pod_template_hash="85d9795d6"}
1
ink8s_pod_metrics_get_node_detail{app="ink8s-pod-metrics", containerRuntimeVersion="containerd://1.4.4", hostname="k8s-node01", instance="10.100.85.207:8080", ip="172.20.70.215", job="kubernetes-pods", kubeletVersion="v1.20.1", kubernetes_namespace="default", kubernetes_pod_name="ink8s-pod-metrics-deployment-85d9795d6-95lsp", pod_template_hash="85d9795d6"}

grafana_190">配置grafana

  • 举例图片
  • image.png
  • image.png
  • image.png
  • image.png

本节重点总结 :

  • 打镜像，导出镜像，传输到各个节点并导入
  • 运行该项目
  • 配置prometheus和grafana

http://www.ppmy.cn/ops/121970.html

相关文章

10.2今日错题解析（软考）

10.2今日错题解析（软考）

目录 前言面向对象技术——设计模式的应用场景系统开发基础——概要设计与详细设计 前言 这是用来记录我备考软考设计师的错题的&#xff0c;今天知识点为设计模式的应用场景、概要设计与详细设计&#xff0c;大部分错题摘自希赛中的题目&#xff0c;但相关解析是原创&#xf…
阅读更多...
HarmonyOS/OpenHarmony 自定义弹窗页面级层级控制解决方案

HarmonyOS/OpenHarmony 自定义弹窗页面级层级控制解决方案

关键词&#xff1a;CuntomDialog自定义弹窗、SubWindow子窗口、页面级、弹窗层级控制、鸿蒙、弹窗展示层级异常 问题存在API版本&#xff1a;API10 - API12&#xff08;该问题已反馈&#xff0c;期望后续官方能增加页面级控制能力&#xff09; 在正常的鸿蒙app开发过程中&…
阅读更多...
Cyber Weekly #27

Cyber Weekly #27

赛博新闻 1、ChatGPT迎来交互界面大改 10月4日凌晨&#xff0c;ChatGPT推出了Canvas功能&#xff0c;这是一个全新的交互界面&#xff0c;允许用户在写作和编码方面与ChatGPT进行更紧密的协作。Canvas被设计为一个更大的协作空间&#xff0c;能够随着用户的偏好不断演变&…
阅读更多...
计数排序(counting sort)

计数排序(counting sort)

计算排序是一种通过计算每个元素出现的次数来进行排序的算法。它不比较元素&#xff0c;而是利用数组来“计数”&#xff0c;所以在某些情况下&#xff0c;它能比传统的比较排序快。 计数排序首先要找到要排序数组的最大值&#xff0c;然后创建一个统计数组存储每个排序元素的…
阅读更多...
rk3566开发之rknn npu 部署

rk3566开发之rknn npu 部署

目录 NPU使用 RKNN 模型 非 RKNN 模型 RKNN-Toolkit2工具 RKNN NPU 测试代码如下 main.cc ssd.cc 调用 ssd模型进行目标检测测试 ssd.h qt 中调用 rknn npu 接口 NPU使用 RK3566 内置 NPU 模块。使用该NPU需要下载RKNN SDK，RKNN SDK 为带有 NPU 的 RK3566/RK3568 芯片…
阅读更多...
【数据结构】【链表代码】移除链表元素

【数据结构】【链表代码】移除链表元素

移除链表元素 /*** Definition for singly-linked list.* struct ListNode {* int val;* struct ListNode *next;* };*/struct ListNode* removeElements(struct ListNode* head, int val) { // 创建一个虚拟头节点&#xff0c;以处理头节点可能被删除的情况 struct…
阅读更多...
WPF后台创建控件和绑定

WPF后台创建控件和绑定

WPF后台创建控件和绑定 一、创建一个类&#xff1a; public class MyData : INotifyPropertyChanged {private string text;public string Text{get { return text; }set { text value;OnPropertyChanged(nameof(Text));}}public event PropertyChangedEventHandler Propert…
阅读更多...
蓝桥杯省赛真题打卡day4

蓝桥杯省赛真题打卡day4

[蓝桥杯 2013 省 A] 大臣的旅费 题目描述 很久以前&#xff0c;T 王国空前繁荣。为了更好地管理国家&#xff0c;王国修建了大量的快速路&#xff0c;用于连接首都和王国内的各大城市。 为节省经费&#xff0c;T 国的大臣们经过思考&#xff0c;制定了一套优秀的修建方案&am…
阅读更多...
最新文章

Copyright @ 2022~2023