和online.net 一样，前面的测试有点不太规范，这次有机会重新测试一下
机器配置和online.net 一样， E5 1650V3， 64GB RAM， 2 X 1TB SSD ( micron 5100 enterprise SSD)
测试使用的程序是:
wget -qO- bench.sh | bash
直接测试IO，使用的是下面的commands:
dd if=/dev/zero of=test_$$ bs=64k count=16k conv=fdatasync && rm -f test_$$
直接在服务器裸机安装Centos 7，下面是测试结果:
[root@webnx ~]# hdparm -t /dev/sda /dev/sda: Timing buffered disk reads: 1422 MB in 3.00 seconds = 473.59 MB/sec [root@webnx ~]# hdparm -t /dev/sdb /dev/sdb: Timing buffered disk reads: 1452 MB in 3.00 seconds = 483.80 MB/sec [root@sda ~]# dd if=/dev/zero of=test_$$ bs=64k count=16k conv=fdatasync && rm -f test_$$ 16384+0 records in 16384+0 records out 1073741824 bytes (1.1 GB) copied, 2.75682 s, 389 MB/s
测试了几次，都在390 MB/s 左右
[root@sdb]# dd if=/dev/zero of=test_$$ bs=64k count=16k conv=fdatasync && rm -f test_$$ 16384+0 records in 16384+0 records out 1073741824 bytes (1.1 GB) copied, 2.75827 s, 389 MB/s 两块SSD 都是micron 的全新 1 TB SSD, 看来SSD 的速度就在400 MB/s 左右
用bench.sh的程序测试下:
---------------------------------------------------------------------- CPU model : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz Number of cores : 12 CPU frequency : 1199.920 MHz Total size of Disk : 851.3 GB (1.5 GB Used) Total amount of Mem : 64316 MB (564 MB Used) Total amount of Swap : 30516 MB (0 MB Used) System uptime : 0 days, 0 hour 8 min Load average : 0.08, 0.05, 0.03 OS : CentOS 7.7.1908 Arch : x86_64 (64 Bit) Kernel : 3.10.0-1062.4.1.el7.x86_64 ---------------------------------------------------------------------- I/O speed(1st run) : 391 MB/s I/O speed(2nd run) : 388 MB/s I/O speed(3rd run) : 389 MB/s Average I/O speed : 389.3 MB/s ----------------------------------------------------------------------
依然在390MB/s左右， micron 的SSD的表现要差于intel 和 samsung的SSD
下面看看在ESXi 6.7 上使用native drive 的表现:
Centos 7 with LVM
---------------------------------------------------------------------- CPU model : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz Number of cores : 4 CPU frequency : 3499.997 MHz Total size of Disk : 27.0 GB (1.3 GB Used) Total amount of Mem : 3950 MB (165 MB Used) Total amount of Swap : 3071 MB (0 MB Used) System uptime : 0 days, 0 hour 6 min Load average : 0.16, 0.10, 0.06 OS : CentOS 7.6.1810 Arch : x86_64 (64 Bit) Kernel : 3.10.0-957.el7.x86_64 ---------------------------------------------------------------------- I/O speed(1st run) : 469 MB/s I/O speed(2nd run) : 466 MB/s I/O speed(3rd run) : 468 MB/s Average I/O speed : 467.7 MB/s ----------------------------------
可以看到这个这个数据非常的好，基本接近原生SSD 的表现了
下面是Centos 7 without LVM
---------------------------------------------------------------------- CPU model : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz Number of cores : 4 CPU frequency : 3499.997 MHz Total size of Disk : 26.3 GB (2.3 GB Used) Total amount of Mem : 3950 MB (150 MB Used) Total amount of Swap : 3813 MB (0 MB Used) System uptime : 0 days, 0 hour 1 min Load average : 0.58, 0.23, 0.08 OS : CentOS 7.6.1810 Arch : x86_64 (64 Bit) Kernel : 3.10.0-957.el7.x86_64 ---------------------------------------------------------------------- I/O speed(1st run) : 458 MB/s I/O speed(2nd run) : 457 MB/s I/O speed(3rd run) : 460 MB/s Average I/O speed : 458.3 MB/s
Centos 7 with LVM and standard partition mounted on /sata,
---------------------------------------------------------------------- I/O speed(1st run) : 452 MB/s I/O speed(2nd run) : 456 MB/s I/O speed(3rd run) : 461 MB/s Average I/O speed : 456.3 MB/s ----------------------------------------------------------------------
/data上， SSD 的IO 也会降速，需要很长一段时间才能恢复满速
在Debian上测试一下(无论LVM还是standard partition结果都一样)：
---------------------------------------------------------------------- ---------------------------------------------------------------------- CPU model : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz Number of cores : 4 CPU frequency : 3499.997 MHz Total size of Disk : 94.2 GB (13.1 GB Used) Total amount of Mem : 4044 MB (129 MB Used) Total amount of Swap : 4191 MB (0 MB Used) System uptime : 0 days, 0 hour 9 min ---------------------------------------------------------------------- CPU model : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz Number of cores : 4 CPU frequency : 3499.997 MHz Total size of Disk : 94.2 GB (13.1 GB Used) Total amount of Mem : 4044 MB (129 MB Used) Total amount of Swap : 4191 MB (0 MB Used) System uptime : 0 days, 0 hour 15 min Load average : 0.02, 0.13, 0.14 OS : Debian GNU/Linux 10 Arch : x86_64 (64 Bit) Kernel : 4.19.0-6-amd64 ---------------------------------------------------------------------- I/O speed(1st run) : 456 MB/s I/O speed(2nd run) : 457 MB/s I/O speed(3rd run) : 462 MB/s Average I/O speed : 458.3 MB/s ---------------------------------------------------------------------- Node Name IPv4 address Download Speed CacheFly 205.234.175.175 109MB/s Linode, Tokyo2, JP 139.162.65.37 22.3MB/s Linode, Singapore, SG 139.162.23.4 Linode, London, UK 176.58.107.39 16.4MB/s Linode, Frankfurt, DE 139.162.130.8 15.1MB/s Linode, Fremont, CA 50.116.14.9 74.5MB/s Softlayer, Dallas, TX 173.192.68.18 51.5MB/s Softlayer, Seattle, WA 67.228.112.250 57.1MB/s Softlayer, Frankfurt, DE 159.122.69.4 14.0MB/s Softlayer, Singapore, SG 119.81.28.170 10.8MB/s Softlayer, HongKong, CN 119.81.130.170 12.7MB/s ---------------------------------------------------------------------- ----------------------------------------------------------------------
在Debian上，有的时候速度会降到200MB/s, 但是需要很长一段时间来恢复
下面来看看回退ssd 的driver 以后的表现:
回退driver以后，Debian 10 with LVM or without LVM
---------------------------------------------------------------------- CPU model : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz Number of cores : 4 CPU frequency : 3499.997 MHz Total size of Disk : 94.2 GB (13.1 GB Used) Total amount of Mem : 4044 MB (142 MB Used) Total amount of Swap : 4191 MB (0 MB Used) System uptime : 0 days, 12 hour 13 min Load average : 0.00, 0.00, 0.00 OS : Debian GNU/Linux 10 Arch : x86_64 (64 Bit) Kernel : 4.19.0-6-amd64 ---------------------------------------------------------------------- I/O speed(1st run) : 456 MB/s I/O speed(2nd run) : 461 MB/s I/O speed(3rd run) : 459 MB/s Average I/O speed : 458.7 MB/s
在测试中发现，SSD在standard partition上会降速，降速到200MB/s以后，会花费很长一段时间恢复到满速460MB/s, 但是在centos 7下，这个恢复是很快的
Centos 7 Standard Partition:
---------------------------------------------------------------------- I/O speed(1st run) : 455 MB/s I/O speed(2nd run) : 457 MB/s I/O speed(3rd run) : 457 MB/s Average I/O speed : 456.3 MB/s ----------------------------------------------------------------------
Centos 7 with LVM:
---------------------------------------------------------------------- I/O speed(1st run) : 464 MB/s I/O speed(2nd run) : 466 MB/s I/O speed(3rd run) : 462 MB/s Average I/O speed : 464.0 MB/s ----------------------------------------------------------------------
Centos 7 LVM with standard partition mounted on /data
---------------------------------------------------------------------- I/O speed(1st run) : 456 MB/s I/O speed(2nd run) : 461 MB/s I/O speed(3rd run) : 458 MB/s Average I/O speed : 458.3 MB/s ----------------------------------------------------------------------
在测试中发现，即使会有短暂的降速到200MB/s, 也会很快的恢复到460MB/s 的满速
通过以上的测试可以看到，vmware 全新的native driver，基本上解决了micron ssd 的low IO performance 的问题
micron 的ssd 偶尔会降速到200 MB/s，但是基本是可以恢复满速的，只是时间有长有短的问题。