鸟憨之家
0%

安装fio测速工具

发表于 分类于

安装fio测速工具

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
apt install fio
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  librbd1 librdmacm1
Suggested packages:
  gnuplot gfio python-scipy
The following NEW packages will be installed:
  fio librbd1 librdmacm1
0 upgraded, 3 newly installed, 0 to remove and 11 not upgraded.
Need to get 1,446 kB of archives.
After this operation, 5,276 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://192.168.88.11/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.10 [923 kB]
Get:2 http://192.168.88.11/ubuntu bionic-updates/main amd64 librdmacm1 amd64 17.1-1ubuntu0.2 [56.1 kB]
Get:3 http://192.168.88.11/ubuntu bionic/universe amd64 fio amd64 3.1-1 [467 kB]
Fetched 1,446 kB in 1s (2,113 kB/s)
Selecting previously unselected package librbd1.
(Reading database ... 71561 files and directories currently installed.)
Preparing to unpack .../librbd1_12.2.13-0ubuntu0.18.04.10_amd64.deb ...
Unpacking librbd1 (12.2.13-0ubuntu0.18.04.10) ...
Selecting previously unselected package librdmacm1:amd64.
Preparing to unpack .../librdmacm1_17.1-1ubuntu0.2_amd64.deb ...
Unpacking librdmacm1:amd64 (17.1-1ubuntu0.2) ...
Selecting previously unselected package fio.
Preparing to unpack .../archives/fio_3.1-1_amd64.deb ...
Unpacking fio (3.1-1) ...
Setting up librbd1 (12.2.13-0ubuntu0.18.04.10) ...
Setting up librdmacm1:amd64 (17.1-1ubuntu0.2) ...
Setting up fio (3.1-1) ...
Processing triggers for libc-bin (2.27-3ubuntu1.6) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...

用storcli等阵列卡工具设置阵列

目前为2*240g ssd作为系统盘，组raid1

目前为2*480g ssd作为缓存盘，组raid1

目前为9*8t hdd作为系统盘，组raid6

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
Scheduled Tasks :
===============
Consistency Check Reoccurrence = 168 hrs
Next Consistency check launch = 07/02/2022, 03:00:00
Patrol Read Reoccurrence = 168 hrs
Next Patrol Read launch = 07/02/2022, 03:00:00
Battery learn Reoccurrence = 670 hrs
Next Battery Learn = 07/28/2022, 04:00:00
OEMID = Broadcom

Drive Groups = 3

TOPOLOGY :
========

-----------------------------------------------------------------------------
DG Arr Row EID:Slot DID Type  State BT       Size PDC  PI SED DS3  FSpace TR
-----------------------------------------------------------------------------
 0 -   -   -        -   RAID1 Optl  N  223.000 GB dflt N  N   dflt N      N
 0 0   -   -        -   RAID1 Optl  N  223.000 GB dflt N  N   dflt N      N
 0 0   0   65:12    16  DRIVE Onln  N  223.062 GB dflt N  N   dflt -      N
 0 0   1   65:13    17  DRIVE Onln  N  223.062 GB dflt N  N   dflt -      N
 1 -   -   -        -   RAID6 Optl  Y   50.937 TB dflt N  N   dflt N      N
 1 0   -   -        -   RAID6 Optl  Y   50.937 TB dflt N  N   dflt N      N
 1 0   0   65:0     8   DRIVE Onln  N    7.276 TB dflt N  N   dflt -      N
 1 0   1   65:1     11  DRIVE Onln  N    7.276 TB dflt N  N   dflt -      N
 1 0   2   65:2     6   DRIVE Onln  N    7.276 TB dflt N  N   dflt -      N
 1 0   3   65:3     14  DRIVE Onln  N    7.276 TB dflt N  N   dflt -      N
 1 0   4   65:4     7   DRIVE Onln  N    7.276 TB dflt N  N   dflt -      N
 1 0   5   65:5     4   DRIVE Onln  N    7.276 TB dflt N  N   dflt -      N
 1 0   6   65:6     10  DRIVE Onln  N    7.276 TB dflt N  N   dflt -      N
 1 0   7   65:7     5   DRIVE Onln  N    7.276 TB dflt N  N   dflt -      N
 1 0   8   65:8     12  DRIVE Onln  N    7.276 TB dflt N  N   dflt -      N
 2 -   -   -        -   RAID1 Optl  N  446.625 GB dflt N  N   dflt N      N
 2 0   -   -        -   RAID1 Optl  N  446.625 GB dflt N  N   dflt N      N
 2 0   0   65:9     3   DRIVE Onln  N  446.625 GB dflt N  N   dflt -      N
 2 0   1   65:10    2   DRIVE Onln  N  446.625 GB dflt N  N   dflt -      N
-----------------------------------------------------------------------------

DG=Disk Group Index|Arr=Array Index|Row=Row Index|EID=Enclosure Device ID
DID=Device ID|Type=Drive Type|Onln=Online|Rbld=Rebuild|Optl=Optimal|Dgrd=Degraded
Pdgd=Partially degraded|Offln=Offline|BT=Background Task Active
PDC=PD Cache|PI=Protection Info|SED=Self Encrypting Drive|Frgn=Foreign
DS3=Dimmer Switch 3|dflt=Default|Msng=Missing|FSpace=Free Space Present
TR=Transport Ready

Virtual Drives = 3

VD LIST :
=======

----------------------------------------------------------------
DG/VD TYPE  State Access Consist Cache Cac sCC       Size Name
----------------------------------------------------------------
0/239 RAID1 Optl  RW     No      RWBD  -   ON  223.000 GB raid1
1/0   RAID6 Optl  RW     No      RWBD  -   ON   50.937 TB raid6
2/1   RAID1 Optl  RW     No      RWBD  -   ON  446.625 GB raid1
----------------------------------------------------------------

VD=Virtual Drive| DG=Drive Group|Rec=Recovery
Cac=CacheCade|OfLn=OffLine|Pdgd=Partially Degraded|Dgrd=Degraded
Optl=Optimal||dflt=Default|RO=Read Only|RW=Read Write|HD=Hidden|TRANS=TransportReady|B=Blocked|
Consist=Consistent|R=Read Ahead Always|NR=No Read Ahead|WB=WriteBack|
AWB=Always WriteBack|WT=WriteThrough|C=Cached IO|D=Direct IO|sCC=Scheduled
Check Consistency

Physical Drives = 13

PD LIST :
=======

----------------------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz Model                      Sp Type
----------------------------------------------------------------------------------------
65:0      8 Onln   1   7.276 TB SATA HDD N   N  512B HUS728T8TALE600            U  -
65:1     11 Onln   1   7.276 TB SATA HDD N   N  512B HUS728T8TALE600            U  -
65:2      6 Onln   1   7.276 TB SATA HDD N   N  512B HUS728T8TALE600            U  -
65:3     14 Onln   1   7.276 TB SATA HDD N   N  512B HUS728T8TALE600            U  -
65:4      7 Onln   1   7.276 TB SATA HDD N   N  512B HUS728T8TALE600            U  -
65:5      4 Onln   1   7.276 TB SATA HDD N   N  512B HUS728T8TALE600            U  -
65:6     10 Onln   1   7.276 TB SATA HDD N   N  512B HUS728T8TALE600            U  -
65:7      5 Onln   1   7.276 TB SATA HDD N   N  512B HUS728T8TALE600            U  -
65:8     12 Onln   1   7.276 TB SATA HDD N   N  512B HUS728T8TALE600            U  -
65:9      3 Onln   2 446.625 GB SATA SSD N   N  512B INTEL SSDSC2KB480G8        U  -
65:10     2 Onln   2 446.625 GB SATA SSD N   N  512B INTEL SSDSC2KB480G8        U  -
65:12    16 Onln   0 223.062 GB SATA SSD N   N  512B SAMSUNG MZ7LH240HAHQ-00005 U  -
65:13    17 Onln   0 223.062 GB SATA SSD N   N  512B SAMSUNG MZ7LH240HAHQ-00005 U  -
----------------------------------------------------------------------------------------

EID=Enclosure Device ID|Slt=Slot No.|DID=Device ID|DG=DriveGroup
DHS=Dedicated Hot Spare|UGood=Unconfigured Good|GHS=Global Hotspare
UBad=Unconfigured Bad|Sntze=Sanitize|Onln=Online|Offln=Offline|Intf=Interface
Med=Media Type|SED=Self Encryptive Drive|PI=Protection Info
SeSz=Sector Size|Sp=Spun|U=Up|D=Down|T=Transition|F=Foreign
UGUnsp=UGood Unsupported|UGShld=UGood shielded|HSPShld=Hotspare shielded
CFShld=Configured shielded|Cpybck=CopyBack|CBShld=Copyback Shielded
UBUnsp=UBad Unsupported|Rbld=Rebuild

初始化数据阵列和缓存阵列，

1
2
3
4
5
6
7
8
parted -s /dev/sdb mklabel gpt
parted -s /dev/sdb mkpart primary 0% 100%
parted -s /dev/sdb quit
pvcreate /dev/sdb1
parted -s /dev/sdc mklabel gpt
parted -s /dev/sdc mkpart primary 0% 100%
parted -s /dev/sdc quit
pvcreate /dev/sdc1

新建数据vg，通过命令查看vg大小，创建lv,格式化为xfs,设置开机自挂载,创建对应目录，挂载

1
2
3
4
5
6
7
vgcreate infokistvg /dev/sdc1
vgdisplay infokistvg | grep "Total PE" | awk '{print $3}'
lvcreate --name infokistlv -l $vgsize infokistvg
mkfs -t xfs /dev/infokistvg/infokistlv
echo "/dev/infokistvg/infokistlv  /infokist           xfs     noatime         0       2" >> /etc/fstab
mkdir /infokist
mount -a

未加速硬盘分区情况

1
2
3
4
5
6
7
8
9
10
11
12
13
lsblk
NAME                      MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                         8:0    0 222.6G  0 disk
├─sda1                      8:1    0   487M  0 part /boot
├─sda2                      8:2    0     1K  0 part
└─sda5                      8:5    0 222.1G  0 part
  ├─systemvg-swaplv       253:0    0  15.3G  0 lvm  [SWAP]
  └─systemvg-rootlv       253:1    0  30.5G  0 lvm  /
sdb                         8:16   0 446.1G  0 disk
└─sdb1                      8:17   0 446.1G  0 part
sdc                         8:32   0    51T  0 disk
└─sdc1                      8:33   0    51T  0 part
  └─infokistvg-infokistlv 253:2    0    51T  0 lvm  /infokist

顺序读取速度

1
2
3
4
5
hdparm -Tt /dev/infokistvg/infokistlv

/dev/infokistvg/infokistlv:
 Timing cached reads:   17052 MB in  2.00 seconds = 8535.06 MB/sec
 Timing buffered disk reads: 1228 MB in  3.01 seconds = 407.90 MB/sec

把缓存盘添加到vg中

1
2
vgextend infokistvg /dev/sdb1
  Volume group "infokistvg" successfully extended

查看剩余空间，最好取整数预留空间给meta。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
vgdisplay infokistvg | grep "Free  PE" | awk '{print $3}'
  --- Volume group ---
  VG Name               infokistvg
  System ID
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  6
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               0
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               51.37 TiB
  PE Size               4.00 MiB
  Total PE              13467309
  Alloc PE / Size       13353108 / <50.94 TiB
  Free  PE / Size       114201 / <446.10 GiB
  VG UUID               9aXPMO-LOIu-80aD-jeqI-pCt6-g93h-ufkXdY

取114000
lvcreate --name cachelv -l $vgsize infokistvg
  Logical volume "cachelv" created.

lvconvert --type cache --cachepool /dev/infokistvg/cachelv --cachemode writeback /dev/infokistvg/infokistlv
  Using 480.00 KiB chunk size instead of default 64.00 KiB, so cache pool has less then 1000000 chunks.
  WARNING: Converting infokistvg/cachelv to cache pool's data volume with metadata wiping.
  THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.)
Do you really want to convert infokistvg/cachelv? [y/n]: y
  Converted infokistvg/cachelv to cache pool.
  Logical volume infokistvg/infokistlv is now cached.

安装工具，否则会出现重启后无法挂载vg

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
apt install thin-provisioning-tools
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  thin-provisioning-tools
0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded.
Need to get 359 kB of archives.
After this operation, 1,352 kB of additional disk space will be used.
Get:1 http://192.168.88.11/ubuntu bionic/universe amd64 thin-provisioning-tools amd64 0.7.4-2ubuntu3 [359 kB]
Fetched 359 kB in 0s (1,372 kB/s)
Selecting previously unselected package thin-provisioning-tools.
(Reading database ... 71646 files and directories currently installed.)
Preparing to unpack .../thin-provisioning-tools_0.7.4-2ubuntu3_amd64.deb ...
Unpacking thin-provisioning-tools (0.7.4-2ubuntu3) ...
Setting up thin-provisioning-tools (0.7.4-2ubuntu3) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...

加速后硬盘分区情况

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
 lsblk
NAME                            MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                               8:0    0 222.6G  0 disk
├─sda1                            8:1    0   487M  0 part /boot
├─sda2                            8:2    0     1K  0 part
└─sda5                            8:5    0 222.1G  0 part
  ├─systemvg-swaplv             253:0    0  15.3G  0 lvm  [SWAP]
  └─systemvg-rootlv             253:1    0  30.5G  0 lvm  /
sdb                               8:16   0 446.1G  0 disk
└─sdb1                            8:17   0 446.1G  0 part
  ├─infokistvg-cachelv_cdata    253:3    0 445.3G  0 lvm
  │ └─infokistvg-infokistlv     253:2    0    51T  0 lvm  /infokist
  └─infokistvg-cachelv_cmeta    253:4    0    48M  0 lvm
    └─infokistvg-infokistlv     253:2    0    51T  0 lvm  /infokist
sdc                               8:32   0    51T  0 disk
└─sdc1                            8:33   0    51T  0 part
  └─infokistvg-infokistlv_corig 253:5    0    51T  0 lvm
    └─infokistvg-infokistlv     253:2    0    51T  0 lvm  /infokist

fio测试命令

1
fio -filename=/infokist/test -direct=1 -iodepth 1 -thread -rw=randrw -rwmixread=70 -ioengine=psync -bs=16k -size=2G -numjobs=30 -runtime=100 -group_reporting -name=mytest -ioscheduler=noop

加速前：写入为IOPS=1113, BW=17.4MiB/s (18.2MB/s)读取IOPS=2593, BW=40.5MiB/s (42.5MB/s)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
Jobs: 30 (f=30): [m(30)][100.0%][r=42.2MiB/s,w=17.5MiB/s][r=2697,w=1121 IOPS][eta 00m:00s]
mytest: (groupid=0, jobs=30): err= 0: pid=5706: Tue Jul 12 14:42:43 2022
   read: IOPS=2593, BW=40.5MiB/s (42.5MB/s)(4058MiB/100145msec)
    clat (usec): min=30, max=964422, avg=11172.19, stdev=30577.80
     lat (usec): min=30, max=964422, avg=11172.57, stdev=30577.80
    clat percentiles (usec):
     |  1.00th=[    44],  5.00th=[    48], 10.00th=[    51], 20.00th=[    64],
     | 30.00th=[    82], 40.00th=[   239], 50.00th=[  1582], 60.00th=[  5014],
     | 70.00th=[  8160], 80.00th=[ 13173], 90.00th=[ 25822], 95.00th=[ 46400],
     | 99.00th=[149947], 99.50th=[208667], 99.90th=[367002], 99.95th=[446694],
     | 99.99th=[583009]
   bw (  KiB/s): min=   32, max= 3520, per=3.35%, avg=1389.86, stdev=585.58, samples=5983
   iops        : min=    2, max=  220, avg=86.81, stdev=36.57, samples=5983
  write: IOPS=1113, BW=17.4MiB/s (18.2MB/s)(1743MiB/100145msec)
    clat (usec): min=31, max=820780, avg=877.49, stdev=11722.35
     lat (usec): min=31, max=820782, avg=878.57, stdev=11722.35
    clat percentiles (usec):
     |  1.00th=[    40],  5.00th=[    46], 10.00th=[    48], 20.00th=[    51],
     | 30.00th=[    55], 40.00th=[    60], 50.00th=[    67], 60.00th=[    78],
     | 70.00th=[    83], 80.00th=[    89], 90.00th=[    98], 95.00th=[   113],
     | 99.00th=[  4424], 99.50th=[ 50594], 99.90th=[189793], 99.95th=[242222],
     | 99.99th=[404751]
   bw (  KiB/s): min=   32, max= 2112, per=3.36%, avg=598.67, stdev=296.14, samples=5964
   iops        : min=    2, max=  132, avg=37.40, stdev=18.49, samples=5964
  lat (usec)   : 50=11.71%, 100=40.69%, 250=5.59%, 500=3.89%, 750=0.72%
  lat (usec)   : 1000=0.70%
  lat (msec)   : 2=2.16%, 4=4.19%, 10=12.87%, 20=7.95%, 50=6.21%
  lat (msec)   : 100=1.94%, 250=1.14%, 500=0.22%, 750=0.02%, 1000=0.01%
  cpu          : usr=0.10%, sys=0.41%, ctx=371339, majf=0, minf=1
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=259724,111528,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=40.5MiB/s (42.5MB/s), 40.5MiB/s-40.5MiB/s (42.5MB/s-42.5MB/s), io=4058MiB (4255MB), run=100145-100145msec
  WRITE: bw=17.4MiB/s (18.2MB/s), 17.4MiB/s-17.4MiB/s (18.2MB/s-18.2MB/s), io=1743MiB (1827MB), run=100145-100145msec

Disk stats (read/write):
    dm-2: ios=259677/111526, merge=0/0, ticks=2890872/95592, in_queue=2986464, util=99.96%, aggrios=259724/111539, aggrmerge=0/4, aggrticks=2895525/96105, aggrin_queue=2701848, aggrutil=99.95%
  sdc: ios=259724/111539, merge=0/4, ticks=2895525/96105, in_queue=2701848, util=99.95%

加速后：写入为IOPS=7190, BW=112MiB/s (118MB/s)读取IOPS=16.8k, BW=262MiB/s (275MB/s)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
Jobs: 30 (f=30): [m(30)][100.0%][r=268MiB/s,w=112MiB/s][r=17.2k,w=7146 IOPS][eta 00m:00s]
mytest: (groupid=0, jobs=30): err= 0: pid=3066: Tue Jul 12 16:12:15 2022
   read: IOPS=16.8k, BW=262MiB/s (275MB/s)(25.6GiB/100004msec)
    clat (usec): min=32, max=31820, avg=1582.67, stdev=2412.38
     lat (usec): min=32, max=31820, avg=1583.00, stdev=2412.40
    clat percentiles (usec):
     |  1.00th=[   69],  5.00th=[  139], 10.00th=[  223], 20.00th=[  343],
     | 30.00th=[  404], 40.00th=[  465], 50.00th=[  578], 60.00th=[  799],
     | 70.00th=[ 1123], 80.00th=[ 2180], 90.00th=[ 4621], 95.00th=[ 6915],
     | 99.00th=[11600], 99.50th=[13304], 99.90th=[17433], 99.95th=[19268],
     | 99.99th=[24249]
   bw (  KiB/s): min= 3808, max=18917, per=3.34%, avg=8965.01, stdev=1356.18, samples=6000
   iops        : min=  238, max= 1182, avg=560.18, stdev=84.78, samples=6000
  write: IOPS=7190, BW=112MiB/s (118MB/s)(10.0GiB/100004msec)
    clat (usec): min=32, max=254007, avg=464.10, stdev=934.21
     lat (usec): min=33, max=254008, avg=464.91, stdev=934.22
    clat percentiles (usec):
     |  1.00th=[   61],  5.00th=[   94], 10.00th=[  126], 20.00th=[  188],
     | 30.00th=[  245], 40.00th=[  289], 50.00th=[  322], 60.00th=[  355],
     | 70.00th=[  400], 80.00th=[  486], 90.00th=[  799], 95.00th=[ 1680],
     | 99.00th=[ 2835], 99.50th=[ 3195], 99.90th=[ 3785], 99.95th=[ 4113],
     | 99.99th=[27132]
   bw (  KiB/s): min= 1344, max= 9234, per=3.34%, avg=3839.05, stdev=682.06, samples=6000
   iops        : min=   84, max=  577, avg=239.70, stdev=42.59, samples=6000
  lat (usec)   : 50=0.24%, 100=3.48%, 250=13.81%, 500=37.75%, 750=11.74%
  lat (usec)   : 1000=7.76%
  lat (msec)   : 2=9.19%, 4=7.77%, 10=6.96%, 20=1.26%, 50=0.03%
  lat (msec)   : 100=0.01%, 250=0.01%, 500=0.01%
  cpu          : usr=0.55%, sys=3.20%, ctx=2401006, majf=0, minf=8723
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=1679208,719096,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=262MiB/s (275MB/s), 262MiB/s-262MiB/s (275MB/s-275MB/s), io=25.6GiB (27.5GB), run=100004-100004msec
  WRITE: bw=112MiB/s (118MB/s), 112MiB/s-112MiB/s (118MB/s-118MB/s), io=10.0GiB (11.8GB), run=100004-100004msec

Disk stats (read/write):
    dm-2: ios=1677579/718395, merge=0/0, ticks=2613348/313900, in_queue=2927248, util=99.98%, aggrios=577911/257879, aggrmerge=0/0, aggrticks=892072/133117, aggrin_queue=1025189, aggrutil=99.55%
    dm-4: ios=0/6, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=1788208/719129, aggrmerge=7/8, aggrticks=2730949/307852, aggrin_queue=1142204, aggrutil=99.56%
  sdb: ios=1788208/719129, merge=7/8, ticks=2730949/307852, in_queue=1142204, util=99.56%
    dm-5: ios=23/54515, merge=0/0, ticks=24/97408, in_queue=97432, util=84.33%, aggrios=37/109023, aggrmerge=0/0, aggrticks=41/173601, aggrin_queue=131284, aggrutil=84.35%
  sdc: ios=37/109023, merge=0/0, ticks=41/173601, in_queue=131284, util=84.35%
  dm-3: ios=1733711/719117, merge=0/0, ticks=2676192/301944, in_queue=2978136, util=99.55%

系统盘为纯固态，读取速度为IOPS=22.9k, BW=358MiB/s，写入速度为IOPS=9832, BW=154MiB/s (161MB/s)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
Jobs: 30 (f=30): [m(30)][100.0%][r=420MiB/s,w=180MiB/s][r=26.9k,w=11.5k IOPS][eta 00m:00s]
mytest: (groupid=0, jobs=30): err= 0: pid=2817: Tue Jul 12 16:53:23 2022
   read: IOPS=22.9k, BW=358MiB/s (376MB/s)(35.0GiB/100005msec)
    clat (usec): min=30, max=79391, avg=1185.06, stdev=1989.84
     lat (usec): min=30, max=79392, avg=1185.23, stdev=1989.86
    clat percentiles (usec):
     |  1.00th=[   57],  5.00th=[  101], 10.00th=[  141], 20.00th=[  202],
     | 30.00th=[  265], 40.00th=[  363], 50.00th=[  562], 60.00th=[  857],
     | 70.00th=[ 1270], 80.00th=[ 1991], 90.00th=[ 3228], 95.00th=[ 3949],
     | 99.00th=[ 5211], 99.50th=[ 5997], 99.90th=[28967], 99.95th=[44827],
     | 99.99th=[58459]
   bw (  KiB/s): min=  448, max=16929, per=3.34%, avg=12246.54, stdev=2686.80, samples=5995
   iops        : min=   28, max= 1058, avg=765.07, stdev=167.85, samples=5995
  write: IOPS=9832, BW=154MiB/s (161MB/s)(15.0GiB/100005msec)
    clat (usec): min=30, max=17388, avg=279.53, stdev=294.06
     lat (usec): min=31, max=17388, avg=279.95, stdev=294.12
    clat percentiles (usec):
     |  1.00th=[   50],  5.00th=[   73], 10.00th=[   90], 20.00th=[  118],
     | 30.00th=[  143], 40.00th=[  165], 50.00th=[  190], 60.00th=[  219],
     | 70.00th=[  258], 80.00th=[  330], 90.00th=[  537], 95.00th=[  996],
     | 99.00th=[ 1582], 99.50th=[ 1844], 99.90th=[ 2040], 99.95th=[ 2089],
     | 99.99th=[ 2245]
   bw (  KiB/s): min=   32, max= 8448, per=3.34%, avg=5249.73, stdev=1228.72, samples=5995
   iops        : min=    2, max=  528, avg=327.73, stdev=76.70, samples=5995
  lat (usec)   : 50=0.76%, 100=6.76%, 250=32.58%, 500=20.01%, 750=7.75%
  lat (usec)   : 1000=5.32%
  lat (msec)   : 2=12.84%, 4=10.75%, 10=3.09%, 20=0.04%, 50=0.08%
  lat (msec)   : 100=0.02%
  cpu          : usr=0.46%, sys=1.93%, ctx=3278659, majf=0, minf=4602
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=2293915,983331,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=358MiB/s (376MB/s), 358MiB/s-358MiB/s (376MB/s-376MB/s), io=35.0GiB (37.6GB), run=100005-100005msec
  WRITE: bw=154MiB/s (161MB/s), 154MiB/s-154MiB/s (161MB/s-161MB/s), io=15.0GiB (16.1GB), run=100005-100005msec

Disk stats (read/write):
    dm-1: ios=2290781/981975, merge=0/0, ticks=2682844/263744, in_queue=2946588, util=99.95%, aggrios=2293909/983345, aggrmerge=4/5, aggrticks=2688800/262927, aggrin_queue=540280, aggrutil=99.94%
  sda: ios=2293909/983345, merge=4/5, ticks=2688800/262927, in_queue=540280, util=99.94%