北青永恒 - Service-Monitor
type: channel
link: http://kococ.cn/tag/Service-Monitor/
items: Prometheus-Operator配置自动发现新增ServiceMonitor的NameSpace

Prometheus-Operator配置自动发现新增ServiceMonitor的NameSpace
type: item
date: 2021-11-24T16:23:00+08:00
description: 最近在监控k8s当中的Jenkins 的时候,新增ServiceMonitor总是不显示,这里的不显示指的是不在Prometheus-Operator中显示,我认为这其中一定有问题,于是查阅了大量的资料,终于从中找到了答案, 原因竟是Prometheus-Operator在安装的时候默认只会自动发现default,kube-system,monitoring这三个名称空间下新建的ServiceMonitor,而我的ServiceMonitor又是新建在别的名称空间下所以他无法进行自动发现更奇怪的是,网上对这个问题的记录文章几乎为0,我根本找不到,所以才写下这个笔记,供看到的人借鉴!1. 监控其他 namespace 中的 endpoint 资源需要做的操作创建新增命名空间的 role， 用于获取监控的信息。将创建的role绑定到 monitoring 命名空间中的 prometheus-k8s SA。kubectl create ns test namespace=test cat <<EOF | kubectl apply -f - apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: prometheus-k8s namespace: ${namespace} roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: prometheus-k8s subjects: - kind: ServiceAccount name: prometheus-k8s namespace: monitoring --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: prometheus-k8s namespace: ${namespace} rules: - apiGroups: - "" resources: - services - endpoints - pods verbs: - get - list - watch - apiGroups: - extensions resources: - ingresses verbs: - get - list - watch EOF COPY2. 监控其他 namespace 中的 serviceMonitor 资源serviceMonitorNamespaceSelector 命名空间的标签匹配，不指定时，只匹配自身命名空间的资源serviceMonitorSelector serviceMonitor的标签匹配， 不指定时，只匹配自身命名空间的资源修改 Prometheus 资源配置apiVersion: monitoring.coreos.com/v1 kind: Prometheus metadata: labels: prometheus: k8s name: k8s namespace: monitoring spec: ... # 增加 ns 匹配的标签 serviceMonitorNamespaceSelector: matchLabels: serviceMonitor: prometheus # 或者 增加下面的匹配，用来匹配 serviceMonitor serviceMonitorSelector: matchLabels: serviceMonitor: prometheus COPY添加命名空间的标签for ns in default kube-system monitoring test; do kubectl patch ns $ns --patch '{"metadata":{"labels":{"serviceMonitor": "prometheus" } } }' done COPY添加 serviceMonitor 的标签kubectl patch -n test servicemonitor demo-app --patch '{"metadata":{"labels":{"serviceMonitor":"prometheus"}}}' --type=merge COPY测试cat <<EOF | kubectl-test apply -f - apiVersion: apps/v1 kind: Deployment metadata: name: ingress-demo-app namespace: test labels: app: ingress-demo-app spec: repli…