Sytem monitor commands and tools

sadc
/etc/init.d/boot.sysstat start
If you want to monitor your system about a longer period of time, use sadc to automatically
collect the data. You can read this data at any time using sar. To start sadc,
simply run /etc/init.d/boot.sysstat start. This will add a link to /etc/
cron.d/ that calls sadc with the following default configuration:
  All available data will be collected.
  Data is written to /var/log/sa/saDD, where DD stands for the current day. If
a file already exists, it will be archived.
  The summary report is written to /var/log/sa/sarDD, where DD stands for
the current day. Already existing files will be archived.
  Data is collected every ten minutes, a summary report is generated every 6 hours
(see /etc/sysstat/sysstat.cron).
  The data is collected by the /usr/lib64/sa/sa1 script (or /usr/lib/sa/
sa1 on 32bit systems)
  The summaries are generated by the script /usr/lib64/sa/sa2 (or /usr/
lib/sa/sa2 on 32bit systems)
If you need to customize the configuration, copy the sa1 and sa2 scripts and adjust
them according to your needs. Replace the link /etc/cron.d/sysstat with
a customized copy of /etc/sysstat/sysstat.cron calling your scripts.

sar
CPU Utilization Report: sar
Memory Usage Report: sar -r
Paging Statistics Report: sar -B
Block Device Statistics Report: sar -d
Network Statistics Reports: sar -n KEYWORD
The option -n lets you generate multiple network related reports. Specify one of the
following keywords along with the -n:
  DEV: Generates a statistic report for all network devices
  EDEV: Generates an error statistics report for all network devices
  NFS: Generates a statistic report for an NFS client
  NFSD: Generates a statistic report for an NFS server
  SOCK: Generates a statistic report on sockets
  ALL: Generates all network statistic reports
  
iostat -p sda 3 5 

mpstat
mpstat -P 1 2 5

pidstat
pidstat -C top 2 3

Kernel Ring Buffer: dmesg

lsof
lsof -i
lsof -p $$

ipcs

ps -p $(pidof sshd)
ps ax --format pid,rss,cmd --sort rss
ps aux --sort column
 Sort the output by column. Replace column with
 pmem for physical memory ratio
 pcpu for CPU ratio
 rss for resident set size (non-swapped physical memory)
ps axo pid,%cpu,rss,vsz,args,wchan
 Shows every process, their PID, CPU usage ratio, memory size (resident and virtual),
 name, and their syscall.
ps axfo pid,args
 Show a process tree.
 
Process Tree: pstree 
pstree -pa

nice and renice

free
/proc/meminfo
/proc/pid/smaps

netstat
netstat shows network connections, routing tables (-r), interfaces (-i), masquerade
connections (-M), multicast memberships (-g), and statistics (-s).
When displaying network connections or statistics, you can specify the socket type to
display: TCP (-t), UDP (-u), or raw (-r). The -p option shows the PID and name
of the program to which each socket belongs.

Interactive Network Monitor: iptraf

/proc/cpuinfo
/proc/interrupts
Allocation and use of interrupts
/proc/devices
Available devices
/proc/modules
Kernel modules loaded
/proc/cmdline
Kernel command line
/proc/meminfo
Detailed information about memory usage
/proc/config.gz
gzip-compressed configuration file of the kernel currently running

cat /proc/self/maps

procinfo -dn5

PCI Resources: lspci
lspci -v

USB Devices: lsusb
lsusb

file, mount, du, df, readelf, stat

stat filename

Explain for the Blocks, this is for the physical disk, 512 bytes/block, NOT for 4096 bytes, which is filesystem block(IO Block) size.
# stat test.sh
  File: `test.sh'
  Size: 96              Blocks: 8          IO Block: 4096   regular file
Device: 807h/2055d      Inode: 181908      Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2010-03-10 00:30:07.000000000 -0500
Modify: 2010-03-10 00:30:01.000000000 -0500
Change: 2010-03-10 00:30:01.000000000 -0500
# stat  firewall.sh
  File: `firewall.sh'
  Size: 3325            Blocks: 8          IO Block: 4096   regular file
Device: 807h/2055d      Inode: 181911      Links: 1
Access: (0755/-rwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2010-03-10 23:52:52.000000000 -0500
Modify: 2010-03-10 23:51:35.000000000 -0500
Change: 2010-03-10 23:51:35.000000000 -0500

fuser
fuser -v /mnt/*

w
Who Is Doing What

time find . > /dev/null

Graph Your Data: RRDtool