【云原生】Prometheus+Grafana on K8s 环境部署

• 一、概述
• 二、使用 Helm 安装 Prometheus
• 1）配置源
• 2）下载 prometheus 包
• 3）修改镜像
• 4）安装 prometheus
• 5）访问web
• 6）配置https并更新
  • 1、生成证书（有证书可忽略）
  • 2、修改配置
  • 3、upgrade 更新
• 7）卸载
• 三、使用 Helm 安装 Grafana
• 1）配置源
• 2）下载grafana包
• 3）修改镜像
• 4）安装 grafana
• 5）访问web
• 6）配置https并更新
  • 1、修改配置
  • 2、upgrade 更新
• 7）卸载

一、概述

Prometheus 最开始是由 SoundCloud 开发的开源监控告警系统，是 Google BorgMon 监控系统的开源版本。在 2016 年，Prometheus 加入 CNCF，成为继 Kubernetes 之后第二个被 CNCF 托管的项目。随着 Kubernetes 在容器编排领头羊地位的确立，Prometheus 也成为 Kubernetes 容器监控的标配。

二、使用 Helm 安装 Prometheus

地址：https://artifacthub.io/packages/helm/prometheus-community/prometheus

1）配置源

添加repo
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update prometheus-community
helm search repo prometheus-community/prometheus

2）下载 prometheus 包

拉包
helm pull prometheus-community/prometheus
解包
tar -xf prometheus-15.12.2.tgz

3）修改镜像

grep -A3 'image:' prometheus/values.yaml

### 1、alertmanager
docker search alertmanager
docker pull quay.io/prometheus/alertmanager
docker tag  quay.io/prometheus/alertmanager myharbor.com/monitoring/alertmanager:v0.24.0
docker push myharbor.com/monitoring/alertmanager:v0.24.0

### 2、configmap-reload
docker search configmap-reload
docker pull jimmidyson/configmap-reload:v0.5.0
docker tag jimmidyson/configmap-reload:v0.5.0 myharbor.com/monitoring/configmap-reload:v0.5.0
docker push myharbor.com/monitoring/configmap-reload:v0.5.0

### 3、node-exporter
docker search node-exporter
docker pull quay.io/prometheus/node-exporter:v1.3.1
docker tag quay.io/prometheus/node-exporter:v1.3.1 myharbor.com/monitoring/node-exporter:v1.3.1
docker push myharbor.com/monitoring/node-exporter:v1.3.1

### 4、prometheus
docker search prometheus
docker pull quay.io/prometheus/prometheus:v2.36.2
docker tag  quay.io/prometheus/prometheus:v2.36.2 myharbor.com/monitoring/prometheus:v2.36.2
docker push myharbor.com/monitoring/prometheus:v2.36.2

### 5、pushgateway
docker search pushgateway
docker pull prom/pushgateway:v1.4.3
docker tag prom/pushgateway:v1.4.3 myharbor.com/monitoring/pushgateway:v1.4.3
docker push myharbor.com/monitoring/pushgateway:v1.4.3

### 6、kube-state-metrics
charts/kube-state-metrics/values.yaml
docker pull bitnami/kube-state-metrics
docker tag bitnami/kube-state-metrics:latest myharbor.com/monitoring/kube-state-metrics:latest
docker push myharbor.com/monitoring/kube-state-metrics:latest

修改镜像 values.yaml， charts/kube-state-metrics/values.yaml

4）安装 prometheus

--dry-run --debug
helm install prometheus ./ \
  -n prometheus \
  --create-namespace \
  --set server.ingress.enabled=true \
  --set server.ingress.hosts='{prometheus.k8s.local}' \
  --set server.ingress.paths='{/}' \
  --set server.ingress.pathType=Prefix \
  --set alertmanager.ingress.enabled=true \
  --set alertmanager.ingress.hosts='{alertmanager.k8s.local}' \
  --set alertmanager.ingress.paths='{/}' \
  --set alertmanager.ingress.pathType=Prefix \
  --set grafana.ingress.enabled=true \
  --set grafana.ingress.hosts='{grafana.k8s.local}' \
  --set grafana.ingress.paths='{/}' \
  --set grafana.ingress.pathType=Prefix

NOTES

NAME: prometheus
LAST DEPLOYED: Sat Sep 17 10:06:04 2022
NAMESPACE: prometheus
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The Prometheus server can be accessed via port 80 on the following DNS name from within your cluster:
prometheus-server.prometheus.svc.cluster.local

Get the Prometheus server URL by running these commands in the same shell:
  export POD_NAME=$(kubectl get pods --namespace prometheus -l "app=prometheus,component=server" -o jsonpath="{.items[0].metadata.name}")
  kubectl --namespace prometheus port-forward $POD_NAME 9090

The Prometheus alertmanager can be accessed via port 80 on the following DNS name from within your cluster:
prometheus-alertmanager.prometheus.svc.cluster.local

From outside the cluster, the alertmanager URL(s) are:
http://alertmanager.k8s.local
#################################################################################
######   WARNING: Pod Security Policy has been moved to a global property.  #####
######            use .Values.podSecurityPolicy.enabled with pod-based      #####
######            annotations                                               #####
######            (e.g. .Values.nodeExporter.podSecurityPolicy.annotations) #####
#################################################################################

The Prometheus PushGateway can be accessed via port 9091 on the following DNS name from within your cluster:
prometheus-pushgateway.prometheus.svc.cluster.local

Get the PushGateway URL by running these commands in the same shell:
  export POD_NAME=$(kubectl get pods --namespace prometheus -l "app=prometheus,component=pushgateway" -o jsonpath="{.items[0].metadata.name}")
  kubectl --namespace prometheus port-forward $POD_NAME 9091

For more information on running Prometheus, visit:
https://prometheus.io/

查看

kubectl get pods,svc,ingress -n prometheus

5）访问web

prometheus：http://prometheus.k8s.local/

6）配置https并更新

1、生成证书（有证书可忽略）

cd /opt/k8s/prometheus/artifacthub/prometheus
mkdir tls ; cd tls

生成 CA 证书私钥
openssl genrsa -out ca.key 4096
生成 CA 证书
openssl req -x509 -new -nodes -sha512 -days 3650 \
 -subj "/C=CN/ST=Guangdong/L=Shenzhen/O=k8s.local/OU=k8s.local/CN=k8s.local" \
 -key ca.key \
 -out ca.crt
创建域名证书，生成私钥
openssl genrsa -out k8s.local.key 4096
生成证书签名请求 CSR
openssl req -sha512 -new \
    -subj "/C=CN/ST=Guangdong/L=Shenzhen/O=k8s.local/OU=k8s.local/CN=k8s.local" \
    -key k8s.local.key \
    -out k8s.local.csr
生成 x509 v3 扩展
cat > v3.ext <

2、修改配置

alertmanager:
...

  ingress:
  ...

    tls:
      - secretName: prometheus-alerts-tls
        hosts:
          - alertmanager.k8s.local
  ...

server:
...

  ingress:
  ...

    tls:
      - secretName: prometheus-alerts-tls
        hosts:
          - alertmanager.k8s.local
...

secrets:
  - name: prometheus-alerts-tls
    cert: tls/k8s.local.crt
    key: tls/k8s.local.key

新增一个 templates/tls-secret.yaml文件

{{ range .Values.secrets }}
apiVersion: v1
kind: Secret
metadata:
  name: {{ .name }}
data:
  tls.crt: {{ $.Files.Get .cert | b64enc }}
  tls.key: {{ $.Files.Get .key | b64enc }}
type: kubernetes.io/tls
{{ end }}

2、upgrade 更新

helm upgrade grafana ./ -n grafana

查看

kubectl get pods,svc,ingress -n grafana

web 访问：https://grafana.k8s.local/
账号： admin，密码通过下面命令获取 0D0NfEWWFx9qsBiKR8PuFVxf6PPa9o8YGhZZaNXY

kubectl get secret --namespace grafana grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo

7）卸载

helm uninstall grafana -n grafana

kubectl delete pod -n grafana kubectl get pod -n grafana|awk 'NR>1{print $1}' --force
kubectl patch ns grafana -p '{"metadata":{"finalizers":null}}'
kubectl delete ns grafana --force

Prometheus on K8s 环境部署就先到这里了，下一篇文章讲具体怎么使用Prometheus+grafana监控k8s资源，请小伙伴耐心等待哦，有任何疑问欢迎给我留言哦~

Original: https://www.cnblogs.com/liugp/p/16702342.html
Author: 大数据老司机
Title: 【云原生】Prometheus+Grafana on K8s 环境部署