fio test
目前主流的第三方IO测试工具有fio、iometer和Orion，这三种工具各有千秋。fio在Linux系统下使用比较方便，iometer在window系统下使用比较方便，Orion是oracle的IO测试软件，可在没有安装oracle数据库的情况下模拟oracle数据库场景的读写。
100%随机，100%读， 4K
fio -filename=/dev/emcpowerb -direct=1 -iodepth 1 -thread -rw=randread -ioengine=psync -bs=4k -size=1000G -numjobs=50 -runtime=180 -group_reporting -name=rand_100read_4k
100%随机，100%写， 4K
fio -filename=/dev/emcpowerb -direct=1 -iodepth 1 -thread -rw=randwrite -ioengine=psync -bs=4k -size=1000G -numjobs=50 -runtime=180 -group_reporting -name=rand_100write_4k
100%顺序，100%读 ，4K
fio -filename=/dev/emcpowerb -direct=1 -iodepth 1 -thread -rw=read -ioengine=psync -bs=4k -size=1000G -numjobs=50 -runtime=180 -group_reporting -name=sqe_100read_4k
100%顺序，100%写 ，4K
fio -filename=/dev/emcpowerb -direct=1 -iodepth 1 -thread -rw=write -ioengine=psync -bs=4k -size=1000G -numjobs=50 -runtime=180 -group_reporting -name=sqe_100write_4k
100%随机，70%读，30%写 4K
fio -filename=/dev/emcpowerb -direct=1 -iodepth 1 -thread -rw=randrw -rwmixread=70 -ioengine=psync -bs=4k -size=1000G -numjobs=50 -runtime=180 -group_reporting -name=randrw_70read_4k
参数说明：
filename=/dev/sdc 支持文件系统或者裸设备,压测多个磁盘 --filename=/dev/sda:/dev/sdb
direct=1 测试过程绕过机器自带的buffer，使测试结果更真实
rw=randwread 测试随机读的I/O
rw=randwrite 测试随机写的I/O
rw=randrw 测试随机混合写和读的I/O
rw=read 测试顺序读的I/O
rw=write 测试顺序写的I/O
rw=rw 测试顺序混合写和读的I/O
bs=4k 单次io的块文件大小为4k
bsrange=512-2048 同上，指定定数据块的大小范围
size=50g 本次的测试文件大小为50g，以每次4k的io进行测试
numjobs=30 本次的测试线程为30
runtime=1000 测试时间为1000秒，如果不写则一直将5g文件分4k每次写完为止
ioengine=psync io引擎使用pync方式，如果要使用libaio引擎，需要yum install libaio-devel包
rwmixwrite=30 在混合读写的模式下，写占30%
group_reporting 关于显示结果的，汇总每个进程的信息
此外其他参数
lockmem=1g 只使用1g内存进行测试
zero_buffers 用0初始化系统buffer
nrfiles=8 每个进程生成文件的数量
磁盘读写常用测试点：
1. Read=100% Ramdon=100% rw=randread (100%随机读)
2. Read=100% Sequence=100% rw=read （100%顺序读）
3. Write=100% Sequence=100% rw=write （100%顺序写）
4. Write=100% Ramdon=100% rw=randwrite （100%随机写）
5. Read=70% Sequence=100% rw=rw, rwmixread=70, rwmixwrite=30（70%顺序读，30%顺序写）
6. Read=70% Ramdon=100% rw=randrw, rwmixread=70, rwmixwrite=30 (70%随机读，30%随机写)
https://docs.oracle.com/en-us/iaas/Content/Block/References/samplefiocommandslinux.htm
1.安装fio
# yum install fio
Installed:
fio.x86_64 0:3.7-2.el7
Dependency Installed:
boost-iostreams.x86_64 0:1.53.0-28.el7 boost-random.x86_64 0:1.53.0-28.el7 boost-system.x86_64 0:1.53.0-28.el7 boost-thread.x86_64 0:1.53.0-28.el7
libibverbs.x86_64 0:22.4-6.el7_9 libpmem.x86_64 0:1.5.1-2.1.el7 libpmemblk.x86_64 0:1.5.1-2.1.el7 librados2.x86_64 1:10.2.5-4.el7
librbd1.x86_64 1:10.2.5-4.el7 librdmacm.x86_64 0:22.4-6.el7_9 pciutils.x86_64 0:3.5.1-3.el7 rdma-core.x86_64 0:22.4-6.el7_9
Complete!2.IOPS Performance Tests
2.1 随机读
# sudo fio --filename=/dev/sda --direct=1 --rw=randread --bs=4k --ioengine=libaio --iodepth=256 --runtime=120 --numjobs=4 --time_based --group_reporting --name=iops-test-job --eta-newline=1 --readonly
iops-test-job: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=256
...
fio-3.7
Starting 4 processes
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
iops-test-job: (groupid=0, jobs=4): err= 0: pid=10509: Tue Jul 26 11:18:23 2022
read: IOPS=39.0k, BW=153MiB/s (160MB/s)(17.9GiB/120003msec)
slat (nsec): min=1330, max=73194k, avg=93753.02, stdev=823434.15
clat (usec): min=8, max=211596, avg=26111.13, stdev=17562.51
lat (usec): min=418, max=211700, avg=26205.41, stdev=17613.60
clat percentiles (msec):
|1.00th=[ 7],5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 13],
| 30.00th=[ 15], 40.00th=[ 18], 50.00th=[ 22], 60.00th=[ 26],
| 70.00th=[ 32], 80.00th=[ 39], 90.00th=[ 49], 95.00th=[ 61],
| 99.00th=[ 87], 99.50th=[ 97], 99.90th=, 99.95th=,
| 99.99th=
bw (KiB/s): min=10083, max=116792, per=24.85%, avg=38803.98, stdev=16341.03, samples=956
iops : min= 2520, max=29198, avg=9700.67, stdev=4085.27, samples=956
lat (usec) : 10=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=15.23%, 20=30.60%, 50=44.87%
lat (msec) : 100=8.90%, 250=0.40%
cpu : usr=1.23%, sys=44.62%, ctx=40436, majf=0, minf=1169
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued rwts: total=4685150,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=256
Run status group 0 (all jobs):
READ: bw=153MiB/s (160MB/s), 153MiB/s-153MiB/s (160MB/s-160MB/s), io=17.9GiB (19.2GB), run=120003-120003msec
Disk stats (read/write):
sda: ios=4682667/157, merge=3/10, ticks=2734007/121, in_queue=2732223, util=100.00%2.2 文件随机读写
# sudo fio --filename=/app/fio --size=10GB --direct=1 --rw=randrw --bs=4k --ioengine=libaio --iodepth=256 --runtime=120 --numjobs=4 --time_based --group_reporting --name=iops-test-job --eta-newline=1
iops-test-job: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=256
...
fio-3.7
Starting 4 processes
iops-test-job: Laying out IO file (1 file / 10240MiB)
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
Jobs: 4 (f=4):
iops-test-job: (groupid=0, jobs=4): err= 0: pid=15955: Tue Jul 26 11:30:17 2022
read: IOPS=11.6k, BW=45.2MiB/s (47.4MB/s)(5428MiB/120005msec)
slat (nsec): min=1805, max=64962k, avg=135084.14, stdev=633735.09
clat (usec): min=266, max=201859, avg=43959.32, stdev=16345.61
lat (usec): min=345, max=201925, avg=44094.87, stdev=16391.37
clat percentiles (msec):
|1.00th=[ 22],5.00th=[ 26], 10.00th=[ 29], 20.00th=[ 32],
| 30.00th=[ 35], 40.00th=[ 37], 50.00th=[ 40], 60.00th=[ 43],
| 70.00th=[ 48], 80.00th=[ 55], 90.00th=[ 67], 95.00th=[ 79],
| 99.00th=[ 96], 99.50th=, 99.90th=, 99.95th=,
| 99.99th=
bw (KiB/s): min= 4134, max=18048, per=24.99%, avg=11576.08, stdev=3416.91, samples=959
iops : min= 1033, max= 4512, avg=2893.94, stdev=854.25, samples=959
write: IOPS=11.6k, BW=45.2MiB/s (47.4MB/s)(5429MiB/120005msec)
slat (usec): min=2, max=51335, avg=197.77, stdev=727.42
clat (usec): min=6, max=200588, avg=44095.02, stdev=16439.35
lat (usec): min=231, max=201897, avg=44293.40, stdev=16500.81
clat percentiles (msec):
|1.00th=[ 22],5.00th=[ 26], 10.00th=[ 29], 20.00th=[ 32],
| 30.00th=[ 35], 40.00th=[ 37], 50.00th=[ 40], 60.00th=[ 44],
| 70.00th=[ 48], 80.00th=[ 55], 90.00th=[ 67], 95.00th=[ 79],
| 99.00th=[ 96], 99.50th=, 99.90th=, 99.95th=,
| 99.99th=
bw (KiB/s): min= 4468, max=17776, per=24.99%, avg=11578.11, stdev=3430.21, samples=959
iops : min= 1117, max= 4444, avg=2894.44, stdev=857.57, samples=959
lat (usec) : 10=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.09%, 20=0.53%, 50=72.54%
lat (msec) : 100=26.19%, 250=0.64%
cpu : usr=1.69%, sys=28.76%, ctx=2496870, majf=0, minf=133
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued rwts: total=1389668,1389930,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=256
Run status group 0 (all jobs):
READ: bw=45.2MiB/s (47.4MB/s), 45.2MiB/s-45.2MiB/s (47.4MB/s-47.4MB/s), io=5428MiB (5692MB), run=120005-120005msec
WRITE: bw=45.2MiB/s (47.4MB/s), 45.2MiB/s-45.2MiB/s (47.4MB/s-47.4MB/s), io=5429MiB (5693MB), run=120005-120005msec
Disk stats (read/write):
dm-2: ios=1388779/1389063, merge=0/0, ticks=2476155/1879075, in_queue=4358558, util=98.85%, aggrios=1389737/1390083, aggrmerge=5/15, aggrticks=2436837/1823174, aggrin_queue=4259155, aggrutil=98.73%
sda: ios=1389737/1390083, merge=5/15, ticks=2436837/1823174, in_queue=4259155, util=98.73%
页:
[1]