亚洲免费在线视频-亚洲啊v-久久免费精品视频-国产精品va-看片地址-成人在线视频网

您的位置:首頁技術(shù)文章
文章詳情頁

Zabbix對Kafka topic積壓數(shù)據(jù)監(jiān)控的問題(bug優(yōu)化)

瀏覽:171日期:2023-04-06 15:09:24
目錄
  • 簡述
  • 分區(qū)自動(dòng)發(fā)現(xiàn)
  • 獲取監(jiān)控項(xiàng)“test-group/test/分區(qū)X”的Lag
  • 最終優(yōu)化后腳本
  • 接入Zabbix
    • 1.Zabbix配置文件
    • 2.Zabbix自動(dòng)發(fā)現(xiàn)
    • 3.監(jiān)控項(xiàng)配置
    • 4.告警信息

簡述

分區(qū)自動(dòng)發(fā)現(xiàn)

# 未優(yōu)化前的計(jì)算方式:# 自動(dòng)發(fā)現(xiàn)配置文件vim consumer-groups.conf#按消費(fèi)者組(Group)|Topic格式,寫入自動(dòng)發(fā)現(xiàn)配置文件test-group|test# 執(zhí)行腳本自動(dòng)發(fā)現(xiàn)指定消費(fèi)者和topic的分區(qū)bash consumer-groups.sh discovery{    "data": [{ "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"0" },{ "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"1" },{ "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"3" },{ "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"2" }    ]}

經(jīng)過上線驗(yàn)證,當(dāng)自動(dòng)發(fā)現(xiàn)配置文件只有一個(gè)test-group|test是沒有問題的,但當(dāng)我們按需求再接入test-group|test1 (即test-group消費(fèi)者組的第二個(gè)Topic)時(shí),自動(dòng)發(fā)現(xiàn)的結(jié)果如下:

# 未優(yōu)化前的計(jì)算方式:vim consumer-groups.conf#按消費(fèi)者組(Group)|Topic格式,寫入自動(dòng)發(fā)現(xiàn)配置文件test-group|testtest-group|test1# 執(zhí)行腳本自動(dòng)發(fā)現(xiàn)指定消費(fèi)者和topic的分區(qū)bash consumer-groups.sh discovery{    "data": [{ "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"0" },{ "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"1" },{ "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"3" },{ "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"2" }{ "{#GROUP}":"test-group", "{#TOPICP}":"test1", "{#PARTITION}":"0" },{ "{#GROUP}":"test-group", "{#TOPICP}":"test2", "{#PARTITION}":"1" },{ "{#GROUP}":"test-group", "{#TOPICP}":"test3", "{#PARTITION}":"2" }    ]}

了解Zabbix自動(dòng)發(fā)現(xiàn)格式的同學(xué)會(huì)發(fā)現(xiàn),每個(gè)Topic的Partition會(huì)出現(xiàn)',',這種格式是不符合規(guī)范,這就是導(dǎo)致我們的監(jiān)控項(xiàng)會(huì)出現(xiàn)問題,因此我們需要進(jìn)一步修改腳本。

經(jīng)修改后,最終效果應(yīng)該如下:

# 優(yōu)化后的計(jì)算方式:vim consumer-groups.conf#按消費(fèi)者組(Group)|Topic格式,寫入自動(dòng)發(fā)現(xiàn)配置文件test-group|testtest-group|test1# 執(zhí)行腳本自動(dòng)發(fā)現(xiàn)指定消費(fèi)者和topic的分區(qū)bash consumer-groups.sh discovery{    "data": [{ "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"0" },{ "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"1" },{ "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"3" },{ "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"2" },{ "{#GROUP}":"test-group", "{#TOPICP}":"test1", "{#PARTITION}":"0" },{ "{#GROUP}":"test-group", "{#TOPICP}":"test1", "{#PARTITION}":"1" },{ "{#GROUP}":"test-group", "{#TOPICP}":"test1", "{#PARTITION}":"2" }    ]}

獲取監(jiān)控項(xiàng)“test-group/test/分區(qū)X”的Lag

經(jīng)過自動(dòng)發(fā)現(xiàn)后的數(shù)據(jù),我們可以進(jìn)一步獲取不同分區(qū)的lag

# 優(yōu)化后的計(jì)算方式:# test-group test分區(qū)0 lagbash consumer-groups.sh lag test-group test 0# test-group test分區(qū)1 lagbash consumer-groups.sh lag test-group test 1# test-group test1分區(qū)0 lagbash consumer-groups.sh lag test-group test1 0

通過命令可以看到,我們的參數(shù)通過消費(fèi)者組、Topic、Partition來獲取最終的lag值,如果不加消費(fèi)者區(qū)分,那么無法區(qū)分不同消費(fèi)者組和不同Topic相應(yīng)的lag結(jié)果:

# 未優(yōu)化前的計(jì)算方式:# 獲取分區(qū)0 lagbash consumer-groups.sh lag 0# 獲取分區(qū)1 lagbash consumer-groups.sh lag 1# 獲取分區(qū)2 lagbash consumer-groups.sh lag 2# 獲取分區(qū)3 lagbash consumer-groups.sh lag 3

最終優(yōu)化后腳本

# 自動(dòng)發(fā)現(xiàn)配置文件vim consumer-groups.conf#按消費(fèi)者組(Group)|Topic格式,寫入自動(dòng)發(fā)現(xiàn)配置文件test-group|testtest-group|test1# 自動(dòng)發(fā)現(xiàn)、lag計(jì)算腳本vim consumer-groups.sh#!/bin/bash##comment: 根據(jù)消費(fèi)者組監(jiān)控topic lag,進(jìn)行監(jiān)控告警#配置文件說明#消費(fèi)者組|Topic#test-group|test#獲取topic 信息cal_topic() {    if [ $# -ne 2 ]; thenecho "parameter num error, 讀取topic信息失敗"exit 1    else/usr/local/kafka/bin/./kafka-consumer-groups.sh --bootstrap-server 192.168.3.55:9092 --describe --group $1 |grep -w $2|grep -v none     fi}#topic+分區(qū)自動(dòng)發(fā)現(xiàn)topic_discovery() {    printf "{\n"    printf "\t\"data\": [\n"    m=0    num=`cat /etc/zabbix/monitor_scripts/consumer-groups.conf|wc -l`    for line in `cat /etc/zabbix/monitor_scripts/consumer-groups.conf`    do  m=`expr $m + 1`group=`echo ${line} | awk -F"|" "{print $1}"`topic=`echo ${line} | awk -F"|" "{print $2}"`cal_topic $group $topic > /tmp/consumer-group-tmpcount=`cat /tmp/consumer-group-tmp|wc -l`n=0while read linedo     n=`expr  $n + 1`     #判斷最后一行     if [ $n -eq $count ] && [ $m -eq $num ]; then topicp=`echo $line | awk "{print $1}"` partition=`echo $line  | awk "{print $2}"` printf "\t\t{ \"{#GROUP}\":\"${group}\", \"{#TOPICP}\":\"${topicp}\", \"{#PARTITION}\":\"${partition}\" }\n"     else topicp=`echo $line | awk "{print $1}"` partition=`echo $line  | awk "{print $2}"` printf "\t\t{ \"{#GROUP}\":\"${group}\", \"{#TOPICP}\":\"${topicp}\", \"{#PARTITION}\":\"${partition}\" },\n"     fidone < /tmp/consumer-group-tmp    done    printf "\t]\n"    printf "}\n"}if [ $1 == "discovery" ]; then    topic_discoveryelif [ $1 == "lag" ];then    cal_topic $2 $3 > /tmp/consumer-group    cat /tmp/consumer-group |awk -v t=$3 -v p=$4 "{if($1==t && $2==p ){print $5}}"else    echo "Usage: /data/scripts/consumer-group.sh discovery | lag"fi# 手動(dòng)運(yùn)行## 自動(dòng)發(fā)現(xiàn)bash consumer-groups.sh discovery## test-group test分區(qū)0 lagbash consumer-groups.sh lag test-group test 0

接入Zabbix

1.Zabbix配置文件

vim userparameter_kafka.confUserParameter=topic_discovery,bash /data/scripts/consumer-groups.sh discoveryUserParameter=topic_log[*],bash /data/scripts/consumer-groups.sh lag "$1" "$2" "$3"

2.Zabbix自動(dòng)發(fā)現(xiàn)

3.監(jiān)控項(xiàng)配置

4.告警信息

告警主機(jī):Kafka_192.168.3.55主機(jī)IP:192.168.3.55主機(jī)組:Kafka告警時(shí)間:2022.03.21 00:23:10告警等級(jí):Average告警信息:test-group/test/分區(qū)1:數(shù)據(jù)積壓100告警項(xiàng)目:topic_lag[test-group,test,1]問題詳情:test-group/test/1: 62

到此這篇關(guān)于Zabbix對Kafka topic積壓數(shù)據(jù)監(jiān)控的文章就介紹到這了,更多相關(guān)Zabbix積壓數(shù)據(jù)監(jiān)控內(nèi)容請搜索以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持!

標(biāo)簽: Zabbix
相關(guān)文章:
主站蜘蛛池模板: 亚洲爽妇网| 精品亚洲视频在线观看 | 国产成人精品视频一区二区不卡 | 女人扒开双腿让男人捅 | 在线视频 中文字幕 | 免费看久久 | 韩国porno xxxx| 久草在线视频新时代视频 | 欧美午夜精品久久久久久黑人 | 美女黄色三级 | 中文国产成人精品久久96 | 成人国产一区 | ririai99在线视频观看 | 国产成人高清亚洲一区久久 | 97国产成人精品免费视频 | 牛牛本精品99久久精品88m | 久久国内精品自在自线软件 | 青青热在线精品视频免费 | vvvv99日韩精品亚洲 | 男人天堂社区 | 奇米色88欧美一区二区 | 成人久久久久久 | 午夜宅男在线永远免费观看网 | 女人又黄的视频网站 | 手机看片手机在线看片 | 国产成人一区免费观看 | 性色aⅴ在线观看swag | 杨幂丝袜国产福利视频 | 久久国产精品最新一区 | 亚洲男人的天堂在线视频 | 亚洲综合久久久久久中文字幕 | 欧美 日韩 国产在线 | 久久视频在线免费观看 | 特级黄色毛片在放 | 国产精品福利午夜h视频 | 精品丝袜国产自在线拍亚洲 | 免费国产a国产片高清 | 亚洲黄视频在线观看 | 粉嫩jk制服美女啪啪 | 欧美成人免费一级人片 | 欧美三级真做在线观看 |