PMM出问题排查

本文永久地址:
http://fuxkdb.com/2017/09/29/PMM%E5%87%BA%E9%97%AE%E9%A2%98%E6%8E%92%E6%9F%A5/

PMM出问题排查

看各种日志
monitoring service

[root@node4 ~]# ll /var/log/pmm-*-rw-r--r--. 1 root root 1880 Sep 27 18:02 /var/log/pmm-linux-metrics-42000.log-rw-r--r--. 1 root root  783 Sep 27 18:02 /var/log/pmm-mysql-metrics-42002.log-rw-r--r--. 1 root root 7143 Sep 27 18:08 /var/log/pmm-mysql-queries-0.log

docker里的日志

[root@node4 log]# docker ps -aCONTAINER ID        IMAGE                      COMMAND                CREATED            STATUS              PORTS                        NAMESe4916410b314        percona/pmm-server:latest  "/opt/entrypoint.sh"  2 hours ago        Up 31 minutes      0.0.0.0:80->80/tcp, 443/tcp  pmm-server948a9aeb047e        percona/pmm-server:latest  "/bin/true"            2 hours ago        Created                                          pmm-data[root@node4 log]# docker logs e4916410b3142017-09-27 08:39:47,175 CRIT Supervisor running as root (no user in config file)2017-09-27 08:39:47,175 WARN Included extra file "/etc/supervisord.d/pmm.ini" during parsingUnlinking stale socket /var/run/supervisor/supervisor.sock2017-09-27 08:39:47,527 INFO RPC interface 'supervisor' initialized2017-09-27 08:39:47,528 INFO supervisord started with pid 12017-09-27 08:39:48,536 INFO spawned: 'mysql' with pid 152017-09-27 08:39:48,543 INFO spawned: 'consul' with pid 162017-09-27 08:39:48,552 INFO spawned: 'grafana' with pid 172017-09-27 08:39:48,563 INFO spawned: 'nginx' with pid 182017-09-27 08:39:48,610 INFO spawned: 'cron' with pid 192017-09-27 08:39:48,612 INFO spawned: 'qan-api' with pid 20

进去容器看

docker exec -it e4916410b314 /bin/bash/var/log 下面各种日志/var/log/grafana/grafana.log

pmm-admin list是yes不代表没问题,check-network看看

[root@node4 ~]# pmm-admin check-networkPMM Network StatusServer Address | 172.16.83.103Client Address | 172.16.83.103 * System TimeNTP Server (0.pool.ntp.org)        | 2017-09-27 17:13:58 +0800 CSTPMM Server                          | 2017-09-27 09:13:58 +0000 GMTPMM Client                          | 2017-09-27 17:13:58 +0800 CSTPMM Server Time Drift              | OKPMM Client Time Drift              | OKPMM Client to PMM Server Time Drift | OK* Connection: Client --> Server-------------------- -------      SERVER SERVICE      STATUS      -------------------- -------      Consul API          OKPrometheus API      OKQuery Analytics API  OKConnection duration | 551.133µsRequest duration    | 2.467879msFull round trip    | 3.019012ms* Connection: Client <-- Server-------------- ------ -------------------- ------- ---------- ---------SERVICE TYPE  NAME  REMOTE ENDPOINT      STATUS  HTTPS/TLS  PASSWORD -------------- ------ -------------------- ------- ---------- ---------linux:metrics  node4  172.16.83.103:42000  OK      YES        -        mysql:metrics  node4  172.16.83.103:42002  OK      YES        -        

---

PMM系列文章:

1.Percona Monitoring and Management架构
2.PMM部署
3.PMM部署遇到的坑
4.PMM设置grafana登录用户
5.PMM出问题排查
6.PMM升级