linux sar

命令均为以下环境：

[root@simpleit ~]# uname -aLinux simpleit.domain.cn 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:35 EDT 2010 i686 athlon i386 GNU/Linux[root@simpleit ~]# cat /etc/redhat-releaseCentOS release 5.5 (Final)

sar -u 检查cpu沉重程度
  %user     运行在用户模式下cpu使用百分比
  %system   运行在系统模式下cpu使用百分比
  %iowait   进程等待io时，空闲状态下cpu使用百分比
  %idle     空闲状态下cpu使用百分比

[oracle@simpleit ~]$ sar -u 10 8  --每10秒采样一次，共采样8次Linux 2.6.18-194.el5 (simpleit.domain.cn)       12/17/201111:50:44 PM       CPU     %user     %nice   %system   %iowait    %steal     %idle11:50:54 PM       all      0.00      0.00      0.14      0.00      0.00     99.8611:51:04 PM       all      0.00      0.00      0.14      0.00      0.00     99.8611:51:14 PM       all      0.00      0.00      0.00      0.14      0.00     99.8611:51:24 PM       all      0.00      0.00      0.44      0.15      0.00     99.4211:51:34 PM       all      0.14      0.00      0.14      0.00      0.00     99.7111:51:44 PM       all      0.00      0.00      0.00      0.00      0.00    100.0011:51:54 PM       all      0.00      0.00      0.00      0.00      0.00    100.0011:52:04 PM       all      0.00      0.00      0.14      0.14      0.00     99.71Average:          all      0.02      0.00      0.13      0.05      0.00     99.80

%iowait如果过高，说明磁盘的连接存在问题，可以使用iostat命令来确定磁盘连接问题;所以，较低的cpu空闲时间(%idle),可能是io的问题，而不是cpu的问题。
从以上输出结果可以检查出的问题：
低cpu空闲时间。
等待i/o或'%iowait > 10'所用时间的的高百分比。
%system > 15瓶颈，这可能说明交换，分页或备份会导致瓶颈。
%user异常高，这可能是由于没有正确调整应用程序或过度利用cpu。

[oracle@simpleit ~]$ sar -d 5 2  --查看磁盘块活动,每5秒采样一次，共采样2次Linux 2.6.18-194.el5 (simpleit.domain.cn)       12/18/201112:13:23 AM       DEV       tps  rd_sec/s  wr_sec/s  avgrq-sz  avgqu-sz     await     svctm     %util12:13:28 AM    dev8-0      2.31      0.00     59.94     26.00      0.00      0.75      0.25      0.0612:13:28 AM    dev8-1      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.0012:13:28 AM    dev8-2      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.0012:13:28 AM    dev8-3      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.0012:13:28 AM    dev8-4      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.0012:13:28 AM    dev8-5      0.86      0.00     20.75     24.00      0.00      0.67      0.67      0.0612:13:28 AM    dev8-6      0.29      0.00      9.22     32.00      0.00      1.00      1.00      0.0312:13:28 AM    dev8-7      1.15      0.00     29.97     26.00      0.00      0.75      0.25      0.0312:13:28 AM    dev8-8      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.0012:13:28 AM       DEV       tps  rd_sec/s  wr_sec/s  avgrq-sz  avgqu-sz     await     svctm     %util12:13:33 AM    dev8-0      2.91      0.00     81.40     28.00      0.00      0.70      0.30      0.0912:13:33 AM    dev8-1      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.0012:13:33 AM    dev8-2      0.58      0.00     20.93     36.00      0.00      0.50      0.50      0.0312:13:33 AM    dev8-3      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.0012:13:33 AM    dev8-4      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.0012:13:33 AM    dev8-5      0.87      0.00     20.93     24.00      0.00      0.67      0.67      0.0612:13:33 AM    dev8-6      0.29      0.00      9.30     32.00      0.00      1.00      1.00      0.0312:13:33 AM    dev8-7      1.16      0.00     30.23     26.00      0.00      0.75      0.25      0.0312:13:33 AM    dev8-8      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00Average:          DEV       tps  rd_sec/s  wr_sec/s  avgrq-sz  avgqu-sz     await     svctm     %utilAverage:       dev8-0      2.60      0.00     70.62     27.11      0.00      0.72      0.28      0.07Average:       dev8-1      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00Average:       dev8-2      0.29      0.00     10.42     36.00      0.00      0.50      0.50      0.01Average:       dev8-3      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00Average:       dev8-4      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00Average:       dev8-5      0.87      0.00     20.84     24.00      0.00      0.67      0.67      0.06Average:       dev8-6      0.29      0.00      9.26     32.00      0.00      1.00      1.00      0.03Average:       dev8-7      1.16      0.00     30.10     26.00      0.00      0.75      0.25      0.03Average:       dev8-8      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00-d     Report activity for each block device (kernels 2.4 and newer only).  When data is displayed, the device  specifi-  cation  dev m-n is generally used ( DEV column).  m is the major number of the device.  With recent kernels (post  2.5), n is the minor number of the device, but is only a sequence number with pre 2.5 kernels. Device  names  may  also  be  pretty-printed if option -p is used (see below). Values for fields avgqu-sz, await, svctm and %util may  be unavailable and displayed as 0.00 with some 2.4 kernels.  tps Indicate the number of transfers per second that were issued to the device.  Multiple logical requests can be combined into a single I/O request to the device. A transfer is of indeterminate size.  rd_sec/s Number of sectors read from the device. The size of a sector is 512 bytes.  wr_sec/s Number of sectors written to the device. The size of a sector is 512 bytes.  avgrq-sz The average size (in sectors) of the requests that were issued to the device.  avgqu-sz The average queue length of the requests that were issued to the device.  await The  average  time (in milliseconds) for I/O requests issued to the device to be served. This includes the time spent by the requests in queue and the time spent servicing them.  svctm The average service time (in milliseconds) for I/O requests that were issued to the device.  %util Percentage of CPU time during which I/O requests were issued to the device (bandwidth utilization for  the device). Device saturation occurs when this value is close to 100%.[root@simpleit ~]# sar -b  --查看io传输速率Linux 2.6.18-194.el5 (simpleit.domain.cn)       12/18/201112:00:01 AM       tps      rtps      wtps   bread/s   bwrtn/s12:10:01 AM     15.78      4.32     11.46    134.88    291.9912:20:01 AM     10.15      1.30      8.85     38.04    206.9612:30:01 AM     10.55      1.50      9.04     44.09    208.9912:40:01 AM     10.04      1.32      8.72     38.48    205.5412:50:01 AM      8.82      1.20      7.63     34.99    171.4101:00:01 AM     10.81      1.44      9.37     42.02    223.7201:10:01 AM     15.46      4.40     11.05    137.44    270.2701:20:01 AM      9.42      1.29      8.13     37.79    194.4801:30:01 AM     11.21      1.52      9.69     44.56    211.0201:40:01 AM     10.77      1.32      9.45     38.57    231.1801:50:01 AM     10.45      1.64      8.81     45.58    204.6302:00:01 AM      9.85      1.49      8.35     43.70    200.5802:10:01 AM     14.83      4.37     10.45    136.45    253.2102:20:01 AM     10.14      1.52      8.63     44.47    205.6402:30:02 AM      9.51      1.29      8.23     37.59    197.3702:40:01 AM     10.03      1.39      8.64     40.93    202.5602:50:01 AM     15.52      2.42     13.10     73.29    272.8303:00:01 AM     10.96      1.19      9.76     35.03    263.6403:10:02 AM     17.09      4.72     12.37    146.89    323.9703:20:01 AM     11.09      1.52      9.57     44.62    223.7003:30:01 AM     10.55      1.32      9.24     38.57    218.3303:40:52 AM     10.69      1.41      9.28     41.26    214.5103:50:01 AM     10.78      1.41      9.37     41.15    226.4904:00:01 AM     10.26      1.17      9.09     34.22    215.7304:10:01 AM     16.33      4.71     11.63    146.05    329.4104:20:01 AM     10.27      1.45      8.82     42.33    206.0004:30:01 AM     40.58     28.60     11.98    317.29    388.75Average:          tps      rtps      wtps   bread/s   bwrtn/sAverage:        12.83      3.16      9.67     71.88    236.68 -b     Report I/O and transfer rate statistics.  The following values are displayed:  tps Total number of transfers per second that were issued to physical devices.  A transfer is an I/O request to a physical device.  Multiple  logical  requests can  be  combined  into a single I/O request to the device.  A transfer is of indeterminate size.  rtps Total number of read requests per second issued to physical devices.  wtps Total number of write requests per second issued to physical devices.  bread/s Total amount of data read from the devices in blocks per second.  Blocks  are equivalent to sectors with 2.4 kernels and newer and therefore have a size of 512 bytes. With older kernels, a block is of indeterminate size.  bwrtn/s Total amount of data written to devices in blocks per second. [root@simpleit ~]# sar -q  --查看队列长度和load averages，队列越小越好Linux 2.6.18-194.el5 (simpleit.domain.cn)       12/18/201112:00:01 AM   runq-sz  plist-sz   ldavg-1   ldavg-5  ldavg-1512:10:01 AM         1       154      0.03      0.02      0.0012:20:01 AM         1       154      0.00      0.00      0.0012:30:01 AM         2       158      0.00      0.00      0.0012:40:01 AM         1       158      0.00      0.00      0.0012:50:01 AM         0       145      0.00      0.00      0.0001:00:01 AM         1       144      0.00      0.00      0.0001:10:01 AM         0       143      0.02      0.03      0.0001:20:01 AM         2       143      0.24      0.05      0.0201:30:01 AM         2       142      0.00      0.01      0.0001:40:01 AM         0       142      0.01      0.01      0.0001:50:01 AM         0       142      0.01      0.02      0.0002:00:01 AM         0       142      0.01      0.00      0.0002:10:01 AM         0       142      0.00      0.00      0.0002:20:01 AM         1       143      0.00      0.00      0.0002:30:02 AM         0       143      0.07      0.02      0.0002:40:01 AM         1       143      0.03      0.01      0.0002:50:01 AM         0       142      0.05      0.01      0.0003:00:01 AM         0       142      0.00      0.00      0.0003:10:02 AM         2       143      0.00      0.00      0.0003:20:01 AM         0       143      0.04      0.03      0.0003:30:01 AM         2       142      0.00      0.00      0.0003:40:52 AM         1       144      0.40      0.08      0.0303:50:01 AM         0       144      0.00      0.02      0.0004:00:01 AM         0       145      0.00      0.01      0.0004:10:01 AM         0       143      0.03      0.03      0.0004:20:01 AM         0       143      0.00      0.00      0.0004:30:01 AM         0       154      0.15      0.63      0.3604:30:01 AM   runq-sz  plist-sz   ldavg-1   ldavg-5  ldavg-1504:40:01 AM         1       160      0.00      0.15      0.2104:50:01 AM         1       152      0.03      0.04      0.1305:00:01 AM         1       155      0.00      0.00      0.07Average:            1       146      0.04      0.04      0.03 -q     Report queue length and load averages. The following values are displayed:  runq-sz Run queue length (number of processes waiting for run time).  plist-sz Number of processes and threads in the process list.  ldavg-1 System load average for the last minute.  ldavg-5 System load average for the past 5 minutes.  ldavg-15 System load average for the past 15 minutes.[root@simpleit ~]# sar -B  --查看分页信息Linux 2.6.18-194.el5 (simpleit.domain.cn)       12/18/201112:00:01 AM  pgpgin/s pgpgout/s   fault/s  majflt/s12:10:01 AM     33.72     73.00    148.99      0.0012:20:01 AM      9.51     51.74     28.18      0.0012:30:01 AM     11.02     52.25     38.95      0.0012:40:01 AM      9.62     51.38     23.46      0.0012:50:01 AM      8.75     42.85     36.53      0.0001:00:01 AM     10.51     55.93    113.14      0.0001:10:01 AM     34.36     67.57    133.89      0.0001:20:01 AM      9.45     48.62     13.46      0.0001:30:01 AM     11.14     52.76     33.40      0.0001:40:01 AM      9.64     57.80     53.93      0.0001:50:01 AM     11.39     51.16     26.90      0.0002:00:01 AM     10.92     50.15     15.03      0.0002:10:01 AM     34.11     63.30     71.48      0.0002:20:01 AM     11.12     51.41     30.74      0.0002:30:02 AM      9.40     49.34     13.41      0.0002:40:01 AM     10.23     50.64     24.54      0.0002:50:01 AM     18.32     68.21     75.52      0.0003:00:01 AM      8.76     65.91     15.04      0.0003:10:02 AM     36.72     81.01    174.94      0.0003:20:01 AM     11.16     55.90    102.84      0.0003:30:01 AM      9.64     54.58    103.05      0.0003:40:52 AM     10.31     53.63    113.14      0.0003:50:01 AM     10.29     56.62     84.43      0.0004:00:01 AM      8.55     53.93    121.55      0.0004:10:01 AM     36.51     82.35    497.64      0.0004:20:01 AM     10.58     51.69     23.54      0.0004:30:01 AM     79.32     97.02  22818.85      0.0004:30:01 AM  pgpgin/s pgpgout/s   fault/s  majflt/s04:40:01 AM     10.57     56.57    136.40      0.0004:50:01 AM      8.15     51.25     94.58      0.0005:00:01 AM      9.63     55.48     95.60      0.0005:10:01 AM     39.76     69.12    101.28      0.0305:20:01 AM      9.37     50.82     40.03      0.00Average:        17.58     58.77    899.74      0.00 -B     Report paging statistics. The following values are displayed:  pgpgin/s Total number of kilobytes the system paged in from disk  per  second.   Note: With old kernels (2.2.x) this value is a number of blocks per second (and not kilobytes).--每秒换进数(kilobytes)  pgpgout/s Total number of kilobytes the system paged out to  disk  per  second.   Note: With old kernels (2.2.x) this value is a number of blocks per second (and not kilobytes).--每秒换出数(kilobytes)  fault/s Number of page faults (major + minor) made by the system per second (post 2.5 kernels only).  This is not a count of page faults that generate I/O, because some page faults can be resolved without I/O.  majflt/s Number of major faults the system has  made  per  second,  those  which  have required loading a memory page from disk (post 2.5 kernels only). [root@simpleit ~]# sar -W  --查看交换信息Linux 2.6.18-194.el5 (simpleit.domain.cn)       12/18/201112:00:01 AM  pswpin/s pswpout/s12:10:01 AM      0.00      0.0012:20:01 AM      0.00      0.0012:30:01 AM      0.00      0.0012:40:01 AM      0.00      0.0012:50:01 AM      0.00      0.0001:00:01 AM      0.00      0.0001:10:01 AM      0.00      0.0001:20:01 AM      0.00      0.0001:30:01 AM      0.00      0.0001:40:01 AM      0.00      0.0001:50:01 AM      0.00      0.0002:00:01 AM      0.00      0.0002:10:01 AM      0.00      0.0002:20:01 AM      0.00      0.0002:30:02 AM      0.00      0.0002:40:01 AM      0.00      0.0002:50:01 AM      0.00      0.0003:00:01 AM      0.00      0.0003:10:02 AM      0.00      0.0003:20:01 AM      0.00      0.0003:30:01 AM      0.00      0.0003:40:52 AM      0.00      0.0003:50:01 AM      0.00      0.0004:00:01 AM      0.00      0.0004:10:01 AM      0.00      0.0004:20:01 AM      0.00      0.0004:30:01 AM      0.00      0.0004:40:01 AM      0.00      0.0004:50:01 AM      0.00      0.0005:00:01 AM      0.00      0.0005:10:01 AM      0.00      0.0005:20:01 AM      0.00      0.0005:30:01 AM      0.00      0.00Average:         0.00      0.00-W     Report swapping statistics. The following values are displayed:  pswpin/s Total number of swap pages the system brought in per second.  pswpout/s Total number of swap pages the system brought out per second.[root@simpleit ~]# sar -r  --查看空闲内存与空闲交换信息Linux 2.6.18-194.el5 (simpleit.domain.cn)       12/18/201112:00:01 AM kbmemfree kbmemused  %memused kbbuffers  kbcached kbswpfree kbswpused  %swpused  kbswpcad12:10:01 AM     40752    969972     95.97    120780    647636    714880         4      0.00         012:20:01 AM     40504    970220     95.99    120888    647664    714880         4      0.00         012:30:01 AM     39512    971212     96.09    120976    647688    714880         4      0.00         012:40:01 AM     39388    971336     96.10    121068    647692    714880         4      0.00         012:50:01 AM     41556    969168     95.89    121212    647716    714880         4      0.00         001:00:01 AM     42864    967860     95.76    121300    647740    714880         4      0.00         001:10:01 AM     44308    966416     95.62    121364    647764    714880         4      0.00         001:20:01 AM     44280    966444     95.62    121420    647764    714880         4      0.00         001:30:01 AM     45640    965084     95.48    121516    647800    714880         4      0.00         001:40:01 AM     45612    965112     95.49    121576    647808    714880         4      0.00         001:50:01 AM     45584    965140     95.49    121676    647832    714880         4      0.00         002:00:01 AM     45584    965140     95.49    121752    647844    714880         4      0.00         002:10:01 AM     45308    965416     95.52    121844    647856    714880         4      0.00         002:20:01 AM     43796    966928     95.67    121908    647880    714880         4      0.00         002:30:02 AM     43768    966956     95.67    121988    647892    714880         4      0.00         002:40:01 AM     43768    966956     95.67    122064    647908    714880         4      0.00         002:50:01 AM     44880    965844     95.56    122152    647928    714880         4      0.00         003:00:01 AM     44880    965844     95.56    122244    647928    714880         4      0.00         003:10:02 AM     43160    967564     95.73    122344    647960    714880         4      0.00         003:20:01 AM     42964    967760     95.75    122460    647976    714880         4      0.00         003:30:01 AM     44504    966220     95.60    122548    647988    714880         4      0.00         003:40:52 AM     39240    971484     96.12    122700    647984    714880         4      0.00         003:50:01 AM     41396    969328     95.90    122716    648020    714880         4      0.00         004:00:01 AM     39360    971364     96.11    122716    648044    714880         4      0.00         004:10:01 AM     42720    968004     95.77    122752    648116    714880         4      0.00         004:20:01 AM     42720    968004     95.77    122752    648132    714880         4      0.00         004:30:01 AM     15888    994836     98.43    122740    656040    714880         4      0.00         004:40:01 AM     15316    995408     98.48    122824    652124    714880         4      0.00         004:50:01 AM     25372    985352     97.49    122928    649576    714880         4      0.00         005:00:01 AM     26004    984720     97.43    123044    649596    714880         4      0.00         005:10:01 AM     23680    987044     97.66    123144    650828    714880         4      0.00         005:20:01 AM     27604    983120     97.27    123228    650852    714880         4      0.00         005:30:01 AM     22676    988048     97.76    123320    650872    714880         4      0.00         0Average:        38624    972100     96.18    122120    648620    714880         4      0.00         0-r     Report memory and swap space utilization statistics.  The following values are displayed:  kbmemfree Amount of free memory available in kilobytes. 空闲内存（kilobytes）  kbmemused Amount of used memory in kilobytes. This does not take into account memory used by the kernel itself.  %memused Percentage of used memory.  kbbuffers Amount of memory used as buffers by the kernel in kilobytes.  kbcached Amount of memory used to cache data by the kernel in kilobytes.  kbswpfree Amount of free swap space in kilobytes.空闲交换（kilobytes）  kbswpused Amount of used swap space in kilobytes.  %swpused Percentage of used swap space.  kbswpcad Amount  of  cached  swap  memory in kilobytes.  This is memory that once was swapped out, is swapped back in but still also is in the swap area (if memory is needed it doesn't need to be swapped out again because it is already in the swap area. This saves I/O).

当空闲内存低于一定值后，系统就开始分页（很高的换页数（sar -B）  ），说明内存不足），如果它继续下降，系统就开始将一些进程交换出去，这是系统性能快速恶化的信号，可以查找占用过多内存的进程，或者是否存在过多的进程，所以修正内存的不正常使用或增大内存才是正确的解决之道。