查看磁盘IO负载 - 看哪些进程在读写磁盘

来源：互联网发布：windows怎样打日语编辑：程序博客网时间：2024/06/05 10:21

转载：http://www.cnblogs.com/cloudstorage/archive/2012/11/11/2764623.html

今天晚上发现服务器io有点高，顺带看看哪些进程在读写磁盘。

1. lsof

用 lsof 查看某块磁盘上的读写进程。

gddg:~ # lsof /dev/xvda2 |headCOMMAND     PID       USER   FD   TYPE DEVICE SIZE/OFF   NODE NAMEinit          1       root  cwd    DIR  202,2     4096      2 /init          1       root  rtd    DIR  202,2     4096      2 /init          1       root  txt    REG  202,2    40784 193218 /sbin/initinit          1       root  mem    REG  202,2    19114   8063 /lib64/libdl-2.11.1.soinit          1       root  mem    REG  202,2  1661454   8057 /lib64/libc-2.11.1.soinit          1       root  mem    REG  202,2   236384   8114 /lib64/libsepol.so.1init          1       root  mem    REG  202,2   113904   8115 /lib64/libselinux.so.1init          1       root  mem    REG  202,2   149797   8050 /lib64/ld-2.11.1.sokthreadd      2       root  cwd    DIR  202,2     4096      2 /

然后可以通过 lsof -p $pid 查看详情

gddg:~ # lsof -p 32597COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF   NODE NAMEbash    32597 root  cwd    DIR  202,2     4096  16097 /rootbash    32597 root  rtd    DIR  202,2     4096      2 /bash    32597 root  txt    REG  202,2   584016  32203 /bin/bashbash    32597 root  mem    REG  202,2   293936   8125 /lib64/libncurses.so.5.6bash    32597 root  mem    REG  202,2  1661454   8057 /lib64/libc-2.11.1.sobash    32597 root  mem    REG  202,2    19114   8063 /lib64/libdl-2.11.1.sobash    32597 root  mem    REG  202,2   263568   8153 /lib64/libreadline.so.5.2bash    32597 root  mem    REG  202,2   149797   8050 /lib64/ld-2.11.1.sobash    32597 root  mem    REG  202,2   217016  16498 /var/run/nscd/passwdbash    32597 root  mem    REG  202,2   256324 149503 /usr/lib/locale/en_US.utf8/LC_CTYPEbash    32597 root  mem    REG  202,2       54 149490 /usr/lib/locale/en_US.utf8/LC_NUMERICbash    32597 root  mem    REG  202,2     2454 133112 /usr/lib/locale/en_US.utf8/LC_TIMEbash    32597 root  mem    REG  202,2  1163682 149504 /usr/lib/locale/en_US.utf8/LC_COLLATEbash    32597 root  mem    REG  202,2      286 133111 /usr/lib/locale/en_US.utf8/LC_MONETARYbash    32597 root  mem    REG  202,2       57 149408 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGESbash    32597 root  mem    REG  202,2       34 149428 /usr/lib/locale/en_US.utf8/LC_PAPERbash    32597 root  mem    REG  202,2       77 149438 /usr/lib/locale/en_US.utf8/LC_NAMEbash    32597 root  mem    REG  202,2      155 133108 /usr/lib/locale/en_US.utf8/LC_ADDRESSbash    32597 root  mem    REG  202,2       59 149407 /usr/lib/locale/en_US.utf8/LC_TELEPHONEbash    32597 root  mem    REG  202,2       23 149429 /usr/lib/locale/en_US.utf8/LC_MEASUREMENTbash    32597 root  mem    REG  202,2    26050 149293 /usr/lib64/gconv/gconv-modules.cache

2. cat /proc/$pid/io

如果内核版本大于2.6.20，通过cat /proc/pid/io 便可以获取进程的io信息。详细解释

gddg:~ # cat /proc/4140/iorchar: 197448798054        // 读出的总字节数，read()或者pread()中的长度参数总和(pagecache中统计而来，不代表实际磁盘的读入)wchar: 209896059897        // 写入的总字节数，write()或者pwrite()中的长度参数总和syscr: 6491904             // read()或者pread()总的调用次数syscw: 13633940            // write()或者pwrite()总的调用次数read_bytes: 49616125952    // 实际从磁盘中读取的字节总数write_bytes: 14038130688   // 实际写入到磁盘中的字节总数cancelled_write_bytes: 2473984     // 由于截断pagecache导致应该发生而没有发生的写入字节数

3. block_dump

通过echo 1 > /proc/sys/vm/block_dump ，来把 block 读写（WRITE/READ/DIRTY）状况 dump 到日志里，通过 dmesg 命令来查看

#!/bin/sh/etc/init.d/syslog stopecho 1 > /proc/sys/vm/block_dumpsleep 60dmesg | awk '/(READ|WRITE|dirtied)/ {process[$1]++} END {for (x in process) \print process[x],x}' |sort -nr |awk '{print $2 " " $1}' | \head -n 10

#dmesg | egrep "READ|WRITE|dirtied" | egrep -o '([a-zA-Z]*)' | sort | uniq -c | sort -rn | headecho 0 > /proc/sys/vm/block_dump/etc/init.d/syslog start

输出如下：

pdflush(10423): 4000nginx(1167): 179nginx(1229): 172nginx(1187): 111nginx(1243): 105nginx(1213): 92nginx(1233): 69nginx(1157): 61nginx(1161): 50nginx(1155): 32

或者参考这段代码吧：
http://stackoverflow.com/questions/249570/how-can-i-record-what-process-or-kernel-activity-is-using-the-disk-in-gnu-linux

sudo -sdmesg -c/etc/init.d/klogd stopecho 1 > /proc/sys/vm/block_dumprm /tmp/disklogwatch "dmesg -c >> /tmp/disklog"   CTRL-C when you're done collecting dataecho 0 > /proc/sys/vm/block_dump/etc/init.d/klogd startexit (quit root shell)cat /tmp/disklog | awk -F"[() \t]" '/(READ|WRITE|dirtied)/ {activity[$1]++} END {for (x in activity) print x, activity[x]}'| sort -nr -k2

4. iotop类脚本

python版

#!/usr/bin/python# Monitoring per-process disk I/O activity# written by http://www.vpsee.com import sys, os, time, signal, reclass DiskIO:    def __init__(self, pname=None, pid=None, reads=0, writes=0):        self.pname = pname         self.pid = pid        self.reads = 0        self.writes = 0def main():    argc = len(sys.argv)    if argc != 1:        print "usage: ./iotop"        sys.exit(0)    if os.getuid() != 0:        print "must be run as root"        sys.exit(0)    signal.signal(signal.SIGINT, signal_handler)    os.system('echo 1 > /proc/sys/vm/block_dump')    print "TASK              PID       READ      WRITE"    while True:        os.system('dmesg -c > /tmp/diskio.log')        l = []          f = open('/tmp/diskio.log', 'r')        line = f.readline()        while line:            m = re.match(\                '^(\S+)\((\d+)\): (READ|WRITE) block (\d+) on (\S+)', line)            if m != None:                if not l:                           l.append(DiskIO(m.group(1), m.group(2)))                    line = f.readline()                     continue                            found = False                   for item in l:                      if item.pid == m.group(2):                        found = True                                    if m.group(3) == "READ":                            item.reads = item.reads + 1                         elif m.group(3) == "WRITE":                            item.writes = item.writes + 1                if not found:                       l.append(DiskIO(m.group(1), m.group(2)))            line = f.readline()        time.sleep(1)        for item in l:            print "%-10s %10s %10d %10d" % \                (item.pname, item.pid, item.reads, item.writes)def signal_handler(signal, frame):    os.system('echo 0 > /proc/sys/vm/block_dump')    sys.exit(0)if __name__=="__main__":    main()

perl版，强烈推荐打开下面链接

http://www.xaprb.com/blog/2009/08/23/how-to-find-per-process-io-statistics-on-linux/

#!/usr/bin/env perl# This program is part of Aspersa (http://code.google.com/p/aspersa/)=pod=head1 NAMEiodump - Compute per-PID I/O stats for Linux when iotop/pidstat/iopp are not available.=head1 SYNOPSISPrepare the system:  dmesg -c  /etc/init.d/klogd stop  echo 1 > /proc/sys/vm/block_dumpStart the reporting:  while true; do sleep 1; dmesg -c; done | perl iodump  CTRL-CStop the system from dumping these messages:  echo 0 > /proc/sys/vm/block_dump  /etc/init.d/klogd start=head1 AUTHORBaron Schwartz=cutuse strict;use warnings FATAL => 'all';use English qw(-no_match_vars);use sigtrap qw(handler finish untrapped normal-signals);my %tasks;my $oktorun = 1;my $line;while ( $oktorun && (defined ($line = <>)) ) {   my ( $task, $pid, $activity, $where, $device );   ( $task, $pid, $activity, $where, $device )      = $line =~ m/(\S+)\((\d+)\): (READ|WRITE) block (\d+) on (\S+)/;   if ( !$task ) {      ( $task, $pid, $activity, $where, $device )         = $line =~ m/(\S+)\((\d+)\): (dirtied) inode \(.*?\) (\d+) on (\S+)/;   }   if ( $task ) {      my $s = $tasks{$pid} ||= { pid => $pid, task => $task };      ++$s->{lc $activity};      ++$s->{activity};      ++$s->{devices}->{$device};   }}printf("%-15s %10s %10s %10s %10s %10s %s\n",   qw(TASK PID TOTAL READ WRITE DIRTY DEVICES));foreach my $task (   reverse sort { $a->{activity} <=> $b->{activity} } values %tasks) {   printf("%-15s %10d %10d %10d %10d %10d %s\n",      $task->{task}, $task->{pid},      ($task->{'activity'}  || 0),      ($task->{'read'}      || 0),      ($task->{'write'}     || 0),      ($task->{'dirty'}     || 0),      join(', ', keys %{$task->{devices}}));}sub finish {   my ( $signal ) = @_;   if ( $oktorun ) {      print STDERR "# Caught SIG$signal.\n";      $oktorun = 0;   }   else {      print STDERR "# Exiting on SIG$signal.\n";      exit(1);   }}

以后有时间，得研究下iostat.c

http://code.google.com/p/tester-higkoo/source/browse/trunk/Tools/iostat/iostat.c

参考：

1. http://www.vpsee.com/2009/08/monitor-process-io-activity/
2. http://www.vpsee.com/2009/08/monitor-process-io-activity/