WebNX.com 服务器重新benchmark

和online.net 一样,前面的测试有点不太规范,这次有机会重新测试一下

机器配置和online.net 一样, E5 1650V3, 64GB RAM, 2 X 1TB SSD ( micron 5100 enterprise SSD)

测试使用的程序是:

wget -qO- bench.sh | bash

直接测试IO,使用的是下面的commands:

dd if=/dev/zero of=test_$$ bs=64k count=16k conv=fdatasync && rm -f test_$$

直接在服务器裸机安装Centos 7,下面是测试结果:

[root@webnx ~]# hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 1422 MB in 3.00 seconds = 473.59 MB/sec
[root@webnx ~]# hdparm -t /dev/sdb

/dev/sdb:
Timing buffered disk reads: 1452 MB in 3.00 seconds = 483.80 MB/sec

[root@sda ~]# dd if=/dev/zero of=test_$$ bs=64k count=16k conv=fdatasync && rm -f test_$$
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 2.75682 s, 389 MB/s

测试了几次,都在390 MB/s 左右

[root@sdb]# dd if=/dev/zero of=test_$$ bs=64k count=16k conv=fdatasync && rm -f test_$$
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 2.75827 s, 389 MB/s

两块SSD 都是micron 的全新 1 TB SSD, 看来SSD 的速度就在400 MB/s 左右

用bench.sh的程序测试下:

----------------------------------------------------------------------
CPU model : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz
Number of cores : 12
CPU frequency : 1199.920 MHz
Total size of Disk : 851.3 GB (1.5 GB Used)
Total amount of Mem : 64316 MB (564 MB Used)
Total amount of Swap : 30516 MB (0 MB Used)
System uptime : 0 days, 0 hour 8 min
Load average : 0.08, 0.05, 0.03
OS : CentOS 7.7.1908
Arch : x86_64 (64 Bit)
Kernel : 3.10.0-1062.4.1.el7.x86_64
----------------------------------------------------------------------
I/O speed(1st run) : 391 MB/s
I/O speed(2nd run) : 388 MB/s
I/O speed(3rd run) : 389 MB/s
Average I/O speed : 389.3 MB/s
----------------------------------------------------------------------

依然在390MB/s左右, micron 的SSD的表现要差于intel 和 samsung的SSD

下面看看在ESXi 6.7 上使用native drive 的表现:

Centos 7 with LVM

----------------------------------------------------------------------
CPU model : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz
Number of cores : 4
CPU frequency : 3499.997 MHz
Total size of Disk : 27.0 GB (1.3 GB Used)
Total amount of Mem : 3950 MB (165 MB Used)
Total amount of Swap : 3071 MB (0 MB Used)
System uptime : 0 days, 0 hour 6 min
Load average : 0.16, 0.10, 0.06
OS : CentOS 7.6.1810
Arch : x86_64 (64 Bit)
Kernel : 3.10.0-957.el7.x86_64
----------------------------------------------------------------------
I/O speed(1st run) : 469 MB/s
I/O speed(2nd run) : 466 MB/s
I/O speed(3rd run) : 468 MB/s
Average I/O speed : 467.7 MB/s
----------------------------------

可以看到这个这个数据非常的好,基本接近原生SSD 的表现了

下面是Centos 7 without LVM

----------------------------------------------------------------------
CPU model : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz
Number of cores : 4
CPU frequency : 3499.997 MHz
Total size of Disk : 26.3 GB (2.3 GB Used)
Total amount of Mem : 3950 MB (150 MB Used)
Total amount of Swap : 3813 MB (0 MB Used)
System uptime : 0 days, 0 hour 1 min
Load average : 0.58, 0.23, 0.08
OS : CentOS 7.6.1810
Arch : x86_64 (64 Bit)
Kernel : 3.10.0-957.el7.x86_64
----------------------------------------------------------------------
I/O speed(1st run) : 458 MB/s
I/O speed(2nd run) : 457 MB/s
I/O speed(3rd run) : 460 MB/s
Average I/O speed : 458.3 MB/s

Centos 7 with LVM and standard partition mounted on /sata,

----------------------------------------------------------------------
I/O speed(1st run) : 452 MB/s
I/O speed(2nd run) : 456 MB/s
I/O speed(3rd run) : 461 MB/s
Average I/O speed : 456.3 MB/s
----------------------------------------------------------------------

/data上, SSD 的IO 也会降速,需要很长一段时间才能恢复满速

在Debian上测试一下(无论LVM还是standard partition结果都一样):

----------------------------------------------------------------------
----------------------------------------------------------------------
CPU model            : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz
Number of cores      : 4
CPU frequency        : 3499.997 MHz
Total size of Disk   : 94.2 GB (13.1 GB Used)
Total amount of Mem  : 4044 MB (129 MB Used)
Total amount of Swap : 4191 MB (0 MB Used)
System uptime        : 0 days, 0 hour 9 min
----------------------------------------------------------------------
CPU model            : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz
Number of cores      : 4
CPU frequency        : 3499.997 MHz
Total size of Disk   : 94.2 GB (13.1 GB Used)
Total amount of Mem  : 4044 MB (129 MB Used)
Total amount of Swap : 4191 MB (0 MB Used)
System uptime        : 0 days, 0 hour 15 min
Load average         : 0.02, 0.13, 0.14
OS                   : Debian GNU/Linux 10
Arch                 : x86_64 (64 Bit)
Kernel               : 4.19.0-6-amd64
----------------------------------------------------------------------
I/O speed(1st run)   : 456 MB/s
I/O speed(2nd run)   : 457 MB/s
I/O speed(3rd run)   : 462 MB/s
Average I/O speed    : 458.3 MB/s
----------------------------------------------------------------------
Node Name                       IPv4 address            Download Speed
CacheFly                        205.234.175.175         109MB/s
Linode, Tokyo2, JP              139.162.65.37           22.3MB/s
Linode, Singapore, SG           139.162.23.4
Linode, London, UK              176.58.107.39           16.4MB/s
Linode, Frankfurt, DE           139.162.130.8           15.1MB/s
Linode, Fremont, CA             50.116.14.9             74.5MB/s
Softlayer, Dallas, TX           173.192.68.18           51.5MB/s
Softlayer, Seattle, WA          67.228.112.250          57.1MB/s
Softlayer, Frankfurt, DE        159.122.69.4            14.0MB/s
Softlayer, Singapore, SG        119.81.28.170           10.8MB/s
Softlayer, HongKong, CN         119.81.130.170          12.7MB/s
----------------------------------------------------------------------
----------------------------------------------------------------------

在Debian上,有的时候速度会降到200MB/s, 但是需要很长一段时间来恢复

下面来看看回退ssd 的driver 以后的表现:

回退driver以后,Debian 10 with LVM or without LVM

----------------------------------------------------------------------
CPU model            : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz
Number of cores      : 4
CPU frequency        : 3499.997 MHz
Total size of Disk   : 94.2 GB (13.1 GB Used)
Total amount of Mem  : 4044 MB (142 MB Used)
Total amount of Swap : 4191 MB (0 MB Used)
System uptime        : 0 days, 12 hour 13 min
Load average         : 0.00, 0.00, 0.00
OS                   : Debian GNU/Linux 10
Arch                 : x86_64 (64 Bit)
Kernel               : 4.19.0-6-amd64
----------------------------------------------------------------------
I/O speed(1st run)   : 456 MB/s
I/O speed(2nd run)   : 461 MB/s
I/O speed(3rd run)   : 459 MB/s
Average I/O speed    : 458.7 MB/s

在测试中发现,SSD在standard partition上会降速,降速到200MB/s以后,会花费很长一段时间恢复到满速460MB/s, 但是在centos 7下,这个恢复是很快的

Centos 7 Standard Partition:

----------------------------------------------------------------------
I/O speed(1st run) : 455 MB/s
I/O speed(2nd run) : 457 MB/s
I/O speed(3rd run) : 457 MB/s
Average I/O speed : 456.3 MB/s
----------------------------------------------------------------------

Centos 7 with LVM:

----------------------------------------------------------------------
I/O speed(1st run) : 464 MB/s
I/O speed(2nd run) : 466 MB/s
I/O speed(3rd run) : 462 MB/s
Average I/O speed : 464.0 MB/s
----------------------------------------------------------------------

Centos 7 LVM with standard partition mounted on /data

----------------------------------------------------------------------
I/O speed(1st run) : 456 MB/s
I/O speed(2nd run) : 461 MB/s
I/O speed(3rd run) : 458 MB/s
Average I/O speed : 458.3 MB/s
----------------------------------------------------------------------

在测试中发现,即使会有短暂的降速到200MB/s, 也会很快的恢复到460MB/s 的满速

通过以上的测试可以看到,vmware 全新的native driver,基本上解决了micron ssd 的low IO performance 的问题

micron 的ssd 偶尔会降速到200 MB/s,但是基本是可以恢复满速的,只是时间有长有短的问题。