系统技术非业余研究

IOzone官网: http://www.iozone.org/
IOzone is a filesystem benchmark tool. The benchmark generates and measures a variety of file operations. Iozone has been ported to many machines and runs under many operating systems.

Benchmark Features:
* ANSII C source
* POSIX async I/O
* Mmap() file I/O
* Normal file I/O
* Single stream measurement
* Multiple stream measurement
* Distributed fileserver measurements (Cluster)
* POSIX pthreads
* Multi-process measurement
* Excel importable output for graph generation
* Latency plots
* 64bit compatible source
* Large file compatible
* Stonewalling in throughput tests to eliminate straggler effects
* Processor cache size configurable
* Selectable measurements with fsync, O_SYNC
* Builds for: AIX, BSDI, HP-UX, IRIX, FreeBSD, Linux, OpenBSD, NetBSD, OSFV3, OSFV4, OSFV5, SCO OpenServer, Solaris, MAC OS X, Windows (95/98/Me/NT/2K/XP)

他的定位非常明确是针对文件系统的性能测试的。和常用的IO性能测试工具sysbench, fio, iometer不同, 它主要是通过模拟用户访问文件模式的不同，典型的如下面的几种：
(0=write/rewrite, 1=read/re-read, 2=random-read/write
3=Read-backwards, 4=Re-write-record, 5=stride-read, 6=fwrite/re-fwrite
7=fread/Re-fread, 8=random_mix, 9=pwrite/Re-pwrite, 10=pread/Re-pread
11=pwritev/Re-pwritev, 12=preadv/Re-preadv)
来达到隔离访问文件系统的meta信息和data信息的不同的开销， 从而反应文件系统的性能。

Ubuntu下可以用 apt-get -y install iozone安装就好。

他有二种模式： 1. 测试吞吐量模式。 2. 测试文件系统对记录大小，文件大小不同组合的反应。

以下是我用过的测试吞吐量模式的参数:
iozone -t -l 1 -u 16 -L 64 -S 8192 -b fio.xls -R -M -s 10G -r 32k -I -T -C -j 32 -+p 60
参数解释
-t -> Throughput test
-s 10G -> File size set to 18874368 KB
-M ->Machine = Linux my174.cm4 2.6.18-164.el5 #1 SMP Tue Aug 18 15:51:48 EDT 2009
-r ->32k Record Size 32 KB
-I ->O_DIRECT feature enabled
-S 8192 ->Processor cache size set to 8192 Kbytes.
-L 64 ->Processor cache line size set to 64 bytes.
-j 32 ->File stride size set to 32 * record size.
-l 1 ->Min thread = 1
-u 16 ->Max thread = 16
-R ->Excel chart generation enabled
-b fio.xls ->产生的二进制格式execl文件名
-+p 60 ->Percent read in mix test is 60

测试文件系统对记录大小，文件大小不同组合的反应时候的参数：
TODO

玩的开心。

在集群的时候，通常需要登录到其他节点去执行命令。但是一般的机器都需要密码登录的，去做ssh信任又非常麻烦。sshpass来帮忙了。

sshpass -noninteractive ssh password provider.

在ubuntu下可以用 apt-get install sshpass来安装。

它支持密码从命令行，文件， 环境变量中读取。

有需要的同学实验下吧。 特别是erlang支持自定义的rsh, 参数是erl -rsh xxxxx

SysBench is a modular, cross-platform and multi-threaded benchmark tool for evaluating OS parameters that are important for a system running a database under intensive load.
Current features allow to test the following system parameters:
* cpu性能
* file I/O performance
* scheduler performance
* mutex的性能
* memory allocation and transfer speed, 支持hugepage
* POSIX threads implementation performance
* database server performance (OLTP benchmark) 支持mysql，pgsql, oracle

项目地址是：http://sysbench.sourceforge.net/

从源码来看这个项目做的非常的模块化。 sysbench提供了诸如读取配置， 创建线程， 日志， 计时和模块化框架，支持的测试模式都是通过插件方式加入到框架去的。

用户很容易扩展相应的模块， 通常模块只需要关注自己要实现的测试功能，其他的事情由框架来做，很大的方便用户自己编写特定的测试模块。其他的如多线程什么的都无需自己去考虑。

在数据库的驱动方面，目前提供了mysql，pgsql, oracle的驱动。在数据库抽象方面也是模块的，用户自己也能容易加入自己的数据库支持。

作为一个轻量级别的bench工具，在系统的系统测量方面，可以了解到对系统运行产生很大影响的性能，如内存，cpu，磁盘，锁，线程调度，数据库等方面的信息，是一个得心应手的工具。

在ubuntu下可以用apt-get install sysbench来安装，具体的使用参看 man sysbench。

玩的开心！

具体的可以参考这里

他的最大特点是快, 当你的系统有上万个tcp链接要了解的时候的时候, 你就知道我说什么了. netstat等常规工具变成废铁了, 这时候他的作用就非常明显了.

/proc interface is inadequate, unfortunately. When amount of sockets is enough large, netstat or even plain cat /proc/net/tcp/ cause nothing but pains and curses. In linux-2.4 the desease became worse: even if amount of sockets is small reading /proc/net/tcp/ is slow enough.

This utility presents a new approach, which is supposed to scale well. I am not going to describe technical details here and will concentrate on description of the command. The only important thing to say is that it is not so bad idea to load module tcp_diag, which can be found in directory Modules of iproute2. If you do not make this ss will work, but it falls back to /proc and becomes slow like netstat, well, a bit faster yet (see section “Some numbers”).

从技术上讲, 它主要是通过/proc来获取各种各样的统计信息. 特别是当处理tcp的时候用到了, tcp_diag功能. tcp_diag是tcp协议栈用于诊断和统计的一个模块, 用netfilter来获取第一手的信息的.  ss就用到了这个技术, 保证了信息的快捷获取.

这个软件是隶属于iproute包的,  如果你的系统没有的话可以这样安装:

apt-get  -y install iproute

用它可以了解下协议栈占用的内存, 每个链接的队列使用情况, 每个链接属于哪个进程.
而之前唯一能用的就是:

watch -n  1 'cat /proc/net/sockstat'

下面简单介绍下他的使用:

root@ubuntu:/usr/src/iproute-20090324/misc# ss --help
Usage: ss [ OPTIONS ]
ss [ OPTIONS ] [ FILTER ]
-h, --help           this message
-V, --version        output version information
-n, --numeric        don't resolve service names
-r, --resolve       resolve host names
-a, --all            display all sockets
-l, --listening      display listening sockets
-o, --options       show timer information
-e, --extended      show detailed socket information
-m, --memory        show socket memory usage
-p, --processes      show process using socket
-i, --info           show internal TCP information
-s, --summary        show socket usage summary

-4, --ipv4          display only IP version 4 sockets
-6, --ipv6          display only IP version 6 sockets
-0, --packet display PACKET sockets
-t, --tcp            display only TCP sockets
-u, --udp            display only UDP sockets
-d, --dccp           display only DCCP sockets
-w, --raw            display only RAW sockets
-x, --unix           display only Unix domain sockets
-f, --family=FAMILY display sockets of type FAMILY

-A, --query=QUERY
QUERY := {all|inet|tcp|udp|raw|unix|packet|netlink}[,QUERY]

-F, --filter=FILE   read filter information from FILE
FILTER := [ state TCP-STATE ] [ EXPRESSION ]

root@ubuntu:/usr/src/iproute-20090324/misc# ss -s
Total: 750 (kernel 761)
TCP:   21 (estab 7, closed 0, orphaned 0, synrecv 0, timewait 0/0), ports 0

Transport Total     IP        IPv6
*         761       -         -
RAW       0         0         0
UDP       3         3         0
TCP       21        18        3
INET      24        21        3
FRAG      0         0         0

root@ubuntu:/usr/src/iproute-20090324/misc# ss -t -m
State       Recv-Q Send-Q                                                                     Local Address:Port                                                                         Peer Address:Port
ESTAB       0      0                                                                        192.168.235.147:ssh                                                                       192.168.235.155:4251
mem:(r0,w0,f4096,t0)
CLOSE-WAIT  38     0                                                                        192.168.235.147:13910                                                                        174.36.30.67:https
mem:(r672,w0,f3424,t0)
ESTAB       0      0                                                                        192.168.235.147:ssh                                                                       192.168.235.155:4635
mem:(r0,w0,f4096,t0)
ESTAB       0      0                                                                              127.0.0.1:4369                                                                            127.0.0.1:54620
mem:(r0,w0,f0,t0)
ESTAB       0      0                                                                        192.168.235.147:ssh                                                                       192.168.235.155:1095
mem:(r0,w0,f4096,t0)
ESTAB       0      0                                                                              127.0.0.1:4369                                                                            127.0.0.1:44234
mem:(r0,w0,f0,t0)
ESTAB       0      0                                                                              127.0.0.1:54620                                                                           127.0.0.1:4369
mem:(r0,w0,f0,t0)
ESTAB       0      0                                                                              127.0.0.1:44234                                                                           127.0.0.1:4369
mem:(r0,w0,f0,t0)

root@ubuntu:/usr/src/iproute-20090324/misc# ss -p
State       Recv-Q Send-Q                                                                     Local Address:Port                                                                         Peer Address:Port
ESTAB       0      0                                                                        192.168.235.147:ssh                                                                       192.168.235.155:4251     users:(("sshd",8373,3))
CLOSE-WAIT  38     0                                                                        192.168.235.147:13910                                                                        174.36.30.67:https    users:(("dropbox",2674,13))
ESTAB       0      0                                                                        192.168.235.147:ssh                                                                       192.168.235.155:4635     users:(("sshd",350,3))
ESTAB       0      0                                                                              127.0.0.1:4369                                                                            127.0.0.1:54620    users:(("epmd",28338,5))
ESTAB       0      0                                                                        192.168.235.147:ssh                                                                       192.168.235.155:1095     users:(("sshd",29044,3))
ESTAB       0      0                                                                              127.0.0.1:4369                                                                            127.0.0.1:44234    users:(("epmd",28338,4))
ESTAB       0      0                                                                              127.0.0.1:54620                                                                           127.0.0.1:4369     users:(("beam.smp",25947,11))
ESTAB       0      0                                                                              127.0.0.1:44234                                                                           127.0.0.1:4369     users:(("inet_gethost",23783,8),("inet_gethost",23784,8))
root@ubuntu:/usr/src/iproute-20090324/misc# ss -i
State       Recv-Q Send-Q                                                                     Local Address:Port                                                                         Peer Address:Port
ESTAB       0      0                                                                        192.168.235.147:ssh                                                                       192.168.235.155:4251
cubic rto:396 rtt:124.5/66 ato:40 cwnd:4 send 375.3Kbps rcv_rtt:296456 rcv_space:66472
CLOSE-WAIT  38     0                                                                        192.168.235.147:13910                                                                        174.36.30.67:https
cubic wscale:7,9 rto:612 rtt:256/52 ato:40 cwnd:5 send 213.8Kbps rcv_rtt:259 rcv_space:5840
ESTAB       0      0                                                                        192.168.235.147:ssh                                                                       192.168.235.155:4635
cubic rto:356 rtt:144/33 ato:40 cwnd:4 send 324.4Kbps rcv_rtt:511258 rcv_space:46760
ESTAB       0      0                                                                              127.0.0.1:4369                                                                            127.0.0.1:54620
cubic wscale:9,9 rto:204 rtt:4/2 ato:40 cwnd:3 send 98.3Mbps rcv_space:32768
ESTAB       0      0                                                                        192.168.235.147:ssh                                                                       192.168.235.155:1095
cubic rto:232 rtt:21/18 ato:40 cwnd:5 ssthresh:4 send 2.8Mbps rcv_rtt:173516 rcv_space:118904
ESTAB       0      0                                                                              127.0.0.1:4369                                                                            127.0.0.1:44234
cubic wscale:9,9 rto:204 rtt:4/2 ato:40 cwnd:3 send 98.3Mbps rcv_space:32768
ESTAB       0      0                                                                              127.0.0.1:54620                                                                           127.0.0.1:4369
cubic wscale:9,9 rto:204 rtt:4/2 ato:40 cwnd:3 send 98.3Mbps rcv_space:32792
ESTAB       0      0                                                                              127.0.0.1:44234                                                                           127.0.0.1:4369
cubic wscale:9,9 rto:204 rtt:4/2 ato:40 cwnd:3 send 98.3Mbps rcv_space:32792

root@ubuntu:/usr/src/iproute-20090324/misc# ss -e
State       Recv-Q Send-Q                                                                     Local Address:Port                                                                         Peer Address:Port
ESTAB       0      0                                                                        192.168.235.147:ssh                                                                       192.168.235.155:4251     timer:(keepalive,50min,0) ino:1843366 sk:c2ea2600
CLOSE-WAIT  38     0                                                                        192.168.235.147:13910                                                                        174.36.30.67:https    ino:12200 sk:c2ea2140
ESTAB       0      0                                                                        192.168.235.147:ssh                                                                       192.168.235.155:4635     timer:(keepalive,2min59sec,0) ino:1101514 sk:c2ea4740
ESTAB       0      0                                                                              127.0.0.1:4369                                                                            127.0.0.1:54620    ino:2309430 sk:c2ea2ac0
ESTAB       0      0                                                                        192.168.235.147:ssh                                                                       192.168.235.155:1095     timer:(keepalive,6min59sec,0) ino:379842 sk:c2f4af80
ESTAB       0      0                                                                              127.0.0.1:4369                                                                            127.0.0.1:44234    ino:2044745 sk:c2ea3440
ESTAB       0      0                                                                              127.0.0.1:54620                                                                           127.0.0.1:4369     ino:2309429 sk:c2ea7200
ESTAB       0      0                                                                              127.0.0.1:44234                                                                           127.0.0.1:4369     ino:2044744 sk:c2ea1300

root@ubuntu:/usr/src/iproute-20090324/misc# ss -o
State       Recv-Q Send-Q                                                                     Local Address:Port                                                                         Peer Address:Port
ESTAB       0      0                                                                        192.168.235.147:ssh                                                                       192.168.235.155:4251     timer:(keepalive,50min,0)
CLOSE-WAIT  38     0                                                                        192.168.235.147:13910                                                                        174.36.30.67:https
ESTAB       0      0                                                                        192.168.235.147:ssh                                                                       192.168.235.155:4635     timer:(keepalive,2min50sec,0)
ESTAB       0      0                                                                              127.0.0.1:4369                                                                            127.0.0.1:54620
ESTAB       0      0                                                                        192.168.235.147:ssh                                                                       192.168.235.155:1095     timer:(keepalive,6min50sec,0)
ESTAB       0      0                                                                              127.0.0.1:4369                                                                            127.0.0.1:44234
ESTAB       0      0                                                                              127.0.0.1:54620                                                                           127.0.0.1:4369
ESTAB       0      0                                                                              127.0.0.1:44234                                                                           127.0.0.1:4369

Have fun!!!