Cenos 6 下通过latencytop了解系统延迟

        LatencyTOP 是一个可对系统传输延迟时间进行检测的工具。

安装环境

Centos 6.4

Vmware esxi 5.5

mysql 5.7

latencytop 0.5

安装步骤

修改内核配置文件，重新编译内核

CONFIG_HAVE_LATENCYTOP_SUPPORT=yCONFIG_LATENCYTOP=y 

获取源码包

tar -zxvf latencytop-0.5.tar.gzcd latencytop-0.5vi Makefile# FIXME: Use autoconf ?#HAS_GTK_GUI = 1                     //注销改行DESTDIR =SBINDIR = /usr/sbinXCFLAGS = -W  -g `pkg-config --cflags glib-2.0` -D_FORTIFY_SOURCE=2 -Wno-sign-compareLDF = -Wl,--as-needed `pkg-config --libs glib-2.0`   -lncurseswOBJS= latencytop.o text_display.o translate.o fsync.oifdef HAS_GTK_GUI  XCFLAGS += `pkg-config --cflags gtk+-2.0` -DHAS_GTK_GUI  LDF += `pkg-config --libs gtk+-2.0`  OBJS += gtk_display.oendif## The w in -lncursesw is not a typo; it is the wide-character version# of the ncurses library, needed for multi-byte character languages# such as Japanese and Chinese etc.## On Debian/Ubuntu distros, this can be found in the# libncursesw5-dev package.## We write explicity this "implicit rule"%.o : %.c        gcc -c $(CFLAGS) $(XCFLAGS) $< -o $@latencytop:  $(OBJS) latencytop.h Makefile        gcc $(CFLAGS) $(OBJS) $(LDF) -o latencytopclean:        rm -f *~ latencytop DEADJOE *.oinstall: latencytop        mkdir -p $(DESTDIR)/usr/share/latencytop        install -m 0644 latencytop.trans $(DESTDIR)/usr/share/latencytop/latencytop.trans        install -m 0644 *.png $(DESTDIR)/usr/share/latencytop/        install -m 0755 latencytop $(DESTDIR)$(SBINDIR)/

安装latencytop

makemake install

运行latencytop

./latencytop --nogui   LatencyTOP version 0.5 (C) 2008 Intel Corporation  Cause                                                Maximum     Percentagefsync() on a file (type 'F' for details)           60.9 msec         15.5 %Reading from file                                  16.0 msec          6.1 %Waiting for data on unix socket                     5.0 msec         58.3 %Receiving TCP/IP data                               5.0 msec         10.0 %Userspace lock contention                           4.9 msec          9.5 %Waiting for a process to die                        4.5 msec          0.2 %Waiting for event (select)                          3.6 msec          0.4 %Marking inode dirty                                 0.4 msec          0.0 %synchronous write                                   0.2 msec          0.0 %Process mysqld (1519) Total: 677.4 msec  fsync() on a file (type 'F' for details)                                                   60.9 msec         67.9 %Reading from file                                  16.0 msec         26.7 %Userspace lock contention                           3.6 msec          3.8 %Waiting for data on unix socket                     0.9 msec          1.5 %

latencytop.trans告诉你，我们也可以自己修改这个文件，把新的延迟原因加上去

#1vfs_readReading from file1vfs_writeWriting to file1__mark_inode_dirtyMarking inode dirty1vfs_readdirReading directory content1vfs_unlinkUnlinking file1blocking_notifier_call_chainBlocking notifier1lock_superSuperblock lock contention1vfs_createCreating a file#2__breadSynchronous buffer read2do_generic_mapping_readReading file data2sock_sendmsgSending data over socket2do_sys_openOpening file2do_sys_pollWaiting for event (poll)2core_sys_selectWaiting for event (select)2proc_reg_readReading from /proc file2__pollwaitWaiting for event (poll)2sys_fcntlFCNTL system call2scsi_error_handlerSCSI error handler2proc_root_readdirReading /proc directory2ksoftirqdWaking ksoftirqd2worker_thread.2do_unlinkatUnlinking file2__wait_on_bufferWaiting for buffer IO to complete2pdflushpdflush() kernel thread2kjournaldkjournald() kernel thread2blkdev_ioctlblock device IOCTL2kauditd_threadkernel audit daemon2tty_ioctlTTY IOCTL2__filemap_fdatawrite_range fdatasync system call2do_sync_writesynchronous write2kthreaddkthreadd kernel thread2usb_port_resumeWaking up USB device2usb_autoresume_deviceWaking up USB device2kswapdkswapd() kernel thread2md_threadRaid resync kernel thread2i915_wait_requestWaiting for GPU command to complete2request_moduleLoading a kernel module#3tty_wait_until_sentWaiting for TTY to finish sending3pipe_readReading from a pipe3pipe_writeWriting to a pipe3pipe_waitWaiting for pipe data3read_block_bitmapReading EXT3 block bitmaps3scsi_execute_reqExecuting raw SCSI command3sys_wait4Waiting for a process to die3sr_media_changeChecking for media change3sr_do_ioctlSCSI cdrom ioctl3sd_ioctlSCSI disk ioctl3sr_cd_checkChecking CDROM media present3ext3_read_inodeReading EXT3 inode3htree_dirblock_to_treeReading EXT3 directory htree3ext3_readdirReading EXT3 directory3ext3_breadSynchronous EXT3 read3ext3_free_branchesUnlinking file on EXT33ext3_get_branchReading EXT3 indirect blocks3ext3_find_entryEXT3: Looking for file3__ext3_get_inode_locReading EXT3 inode3ext3_delete_inodeEXT3 deleting inode3sync_pageWriting a page to disk3tty_pollWaiting for TTY data3tty_readWaiting for TTY input3tty_writeWriting data to TTY3update_atimeUpdating inode atime3page_cache_sync_readaheadPagecache sync readahead3do_forkFork() system call3sys_mkdiratCreating directory3lookup_createCreating file3inet_sendmsgSending TCP/IP data3tcp_recvmsgReceiving TCP/IP data3link_path_walkFollowing symlink3path_walkWalking directory tree3sys_getdentsReading directory content3unix_stream_recvmsgWaiting for data on unix socket3ext3_mkdirEXT3: Creating a directory3journal_get_write_accessEXT3: Waiting for journal access3synchronize_rcuWaiting for RCU3input_close_deviceClosing input device3mousedev_close_deviceClosing mouse device3mousedev_releaseClosing mouse device3mousedev_openOpening mouse device3kmsg_readReading from dmesg3sys_futexUserspace lock contention3do_futexUserspace lock contention3vt_waitactivevt_waitactive IOCTL3acquire_console_semWaiting for console access3filp_closeClosing a file3sync_inode(f)syncing an inode to disk3ata_exec_internal_sgExecuting internal ATA command3writeback_inodesWriting back inodes3ext3_orphan_add EXT3 adding orphan3ext3_mark_inode_dirty EXT3 marking inode dirty3ext3_unlink EXT3 unlinking file3ext3_createEXT3 Creating a file3log_do_checkpointEXT3 journal checkpoint3generic_delete_inodeDeleting an inode3proc_delete_inodeRemoving /proc file3do_truncateTruncating file3sys_execveExecuting a program3journal_commit_transactionEXT3: committing transaction3__stop_machine_runFreezing the kernel (for module load)3sys_munmapunmapping memory3sys_mmapmmaping memory3sync_bufferWriting buffer to disk (synchronous)3inotify_inode_queue_eventInotify event3proc_lookupLooking up /proc file3generic_make_requestCreating block layer request3get_request_waitCreating block layer request3alloc_page_vmaAllocating a VMA#3__d_lookupLooking up a dentry3blkdev_direct_IODirect block device IO3sys_mprotectmprotect() system call3shrink_icache_memoryreducing inode cache memory footprint3vfs_stat_fdstat() operation3cdrom_openopening cdrom device3sys_epoll_waitWaiting for event (epoll)3sync_sb_inodesSyncing inodes3tcp_connectTCP/IP connect3ata_scsi_ioctlATA/SCSI disk ioctl3do_rmdirRemoving directory3vfs_rmdirRemoving directory3sys_flockflock() on a file3usbdev_openopening USB device3lock_kernelBig Kernel Lock contention3blk_execute_rqSubmitting block IO3scsi_cmd_ioctlSCSI ioctl command3acpi_ec_transactionACPI hardware access3journal_get_undo_accessWaiting for EXT3 journal undo operation3i915_irq_waitWaiting for GPU interrupt3i915_gem_throttle_ioctlThrottling GPU while waiting for commands##5do_page_faultPage fault5handle_mm_faultPage fault5filemap_faultPage fault5sync_filesystemsSyncing filesystem5sys_nanosleepApplication requested delay5sys_pauseApplication requested delay5evdev_readReading keyboard/mouse input5do_fsyncfsync() on a file (type 'F' for details)5__log_wait_for_spaceWaiting for EXT3 journal space