WebNX Utah机房E3-1275V5 1TB SSD服务器benchmark

在webhostingtalk上看到webnx 有一款机器在on sale,配置相当不错,而我又是webnx 的超级粉,立马拿下替换了原来旧的机器,新机器的配置如下:

Intel Xeon E3-1275v5 quadcore 3.6ghz
64gb ddr4 ECC ram
1TB Enterprise SSD
/29 5x usable ip's
30Tb over 1gbps
$65 ** limited time until we burn up the stock on hand!

实话实说,这个机器还是非常有新引力的,再加上webnx 100%的SLA 和 super connected network, 真的是非常爽的价格.

我准备用它来做ESXi 新的vCSA 主控. 

在deploy ESXi之前,先来测试一下这台机器的硬件和network. 

先检查硬盘信息:

[[email protected] ~]# smartctl -a /dev/sda
smartctl 7.0 2018-12-30 r4883 [x86_64-linux-3.10.0-1062.1.2.el7.x86_64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family: Samsung based SSDs
Device Model: Samsung SSD 860 EVO 1TB
Serial Number: S3Z6NB0M106674P
LU WWN Device Id: 5 002538 e40c2fc1b
Firmware Version: RVT02B6Q
User Capacity: 1,000,204,886,016 bytes [1.00 TB]
Sector Size: 512 bytes logical/physical
Rotation Rate: Solid State Device
Form Factor: 2.5 inches
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-4 T13/BSR INCITS 529 revision 5
SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Thu Oct 17 23:51:22 2019 EDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

1TB 的SSD 给的是samsung 的 860 EVO, 也算是enterprise 的SSD,通电时间只有815 hours,非常的爽

[[email protected] ~]# smartctl -a /dev/sdb
smartctl 7.0 2018-12-30 r4883 [x86_64-linux-3.10.0-1062.1.2.el7.x86_64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family: Seagate Barracuda 7200.14 (AF)
Device Model: ST1000DM003-1CH162
Serial Number: Z1D7JRQN
LU WWN Device Id: 5 000c50 05d0bd293
Firmware Version: CC49
User Capacity: 1,000,204,886,016 bytes [1.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 7200 rpm
Form Factor: 3.5 inches
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-2, ACS-3 T13/2161-D revision 3b
SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Thu Oct 17 23:54:01 2019 EDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

1TB 的备份盘给的是希捷的barracuda盘,通电时间是1435 hours,也是非常不错的

下面主要测试一下IO, 测试使用的程序是

wget -qO- bench.sh | bash

下面是测试结果:

----------------------------------------------------------------------
CPU model : Intel(R) Xeon(R) CPU E3-1275 v5 @ 3.60GHz
Number of cores : 8
CPU frequency : 877.368 MHz
Total size of Disk : 888.3 GB (1.5 GB Used)
Total amount of Mem : 64252 MB (521 MB Used)
Total amount of Swap : 30516 MB (0 MB Used)
System uptime : 0 days, 1 hour 4 min
Load average : 0.07, 0.03, 0.11
OS : CentOS 7.7.1908
Arch : x86_64 (64 Bit)
Kernel : 3.10.0-1062.1.2.el7.x86_64
----------------------------------------------------------------------
I/O speed(1st run) : 435 MB/s
I/O speed(2nd run) : 437 MB/s
I/O speed(3rd run) : 436 MB/s
Average I/O speed : 436.0 MB/s
----------------------------------------------------------------------
Node Name IPv4 address Download Speed
CacheFly 205.234.175.175 105MB/s
Linode, Tokyo2, JP 139.162.65.37 17.6MB/s
Linode, Singapore, SG 139.162.23.4 12.5MB/s
Linode, London, UK 176.58.107.39 18.6MB/s
Linode, Frankfurt, DE 139.162.130.8 16.8MB/s
Linode, Fremont, CA 50.116.14.9 89.1MB/s
Softlayer, Dallas, TX 173.192.68.18 8.54MB/s
Softlayer, Seattle, WA 67.228.112.250 28.9MB/s
Softlayer, Frankfurt, DE 159.122.69.4 7.12MB/s
Softlayer, Singapore, SG 119.81.28.170 6.79MB/s
Softlayer, HongKong, CN 119.81.130.170 7.89MB/s

测试了即便,860 evo 的IO 速度一直保持在436MB/s, 还是不错的

下面是测试sata 硬盘:

I/O speed(1st run) : 152 MB/s
I/O speed(2nd run) : 148 MB/s
I/O speed(3rd run) : 154 MB/s
Average I/O speed : 151.3 MB/s

测试了几次,发现IO speed基本都维持在153MB/s 左右, 这个IO 也确实不错的说

再用hdparm测试一下IO

[[email protected] ~]# hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 1532 MB in 3.00 seconds = 510.02 MB/sec
[[email protected] ~]# hdparm -t /dev/sdb

/dev/sdb:
Timing buffered disk reads: 430 MB in 3.01 seconds = 143.06 MB/sec

和用dd 测试IO 速度差别不大

下面开始ESXi VM IO测试:

  1. CentOS 7 使用新的ACHI Driver:  vmw-achi,使用LVM
----------------------------------------------------------------------
CPU model : Intel(R) Xeon(R) CPU E3-1275 v5 @ 3.60GHz
Number of cores : 4
CPU frequency : 3600.000 MHz
Total size of Disk : 46.0 GB (1.4 GB Used)
Total amount of Mem : 4048 MB (208 MB Used)
Total amount of Swap : 4095 MB (0 MB Used)
System uptime : 0 days, 0 hour 2 min
Load average : 0.35, 0.09, 0.03
OS : CentOS 7.7.1908
Arch : x86_64 (64 Bit)
Kernel : 3.10.0-1062.el7.x86_64
----------------------------------------------------------------------
I/O speed(1st run) : 499 MB/s
I/O speed(2nd run) : 499 MB/s
I/O speed(3rd run) : 498 MB/s
Average I/O speed : 498.7 MB/s
----------------------------------------------------------------------
Node Name IPv4 address Download Speed
CacheFly 205.234.175.175 105MB/s
Linode, Tokyo2, JP 139.162.65.37 18.5MB/s
Linode, Singapore, SG 139.162.23.4 12.8MB/s
Linode, London, UK 176.58.107.39 19.1MB/s
Linode, Frankfurt, DE 139.162.130.8 16.9MB/s
Linode, Fremont, CA 50.116.14.9 89.4MB/s
Softlayer, Dallas, TX 173.192.68.18 43.5MB/s
Softlayer, Seattle, WA 67.228.112.250 43.6MB/s
Softlayer, Frankfurt, DE 159.122.69.4 9.48MB/s
Softlayer, Singapore, SG 119.81.28.170 8.36MB/s
Softlayer, HongKong, CN 119.81.130.170 10.3MB/s

 

2. CentOS 7 使用新的ACHI Driver:  vmw-achi, 不使用LVM

在ssd上:

----------------------------------------------------------------------
CPU model : Intel(R) Xeon(R) CPU E3-1275 v5 @ 3.60GHz
Number of cores : 4
CPU frequency : 3600.000 MHz
Total size of Disk : 96.3 GB (1.5 GB Used)
Total amount of Mem : 4048 MB (207 MB Used)
Total amount of Swap : 3813 MB (0 MB Used)
System uptime : 0 days, 0 hour 4 min
Load average : 0.50, 0.18, 0.07
OS : CentOS 7.7.1908
Arch : x86_64 (64 Bit)
Kernel : 3.10.0-1062.el7.x86_64
----------------------------------------------------------------------
I/O speed(1st run) : 498 MB/s
I/O speed(2nd run) : 499 MB/s
I/O speed(3rd run) : 499 MB/s
Average I/O speed : 498.7 MB/s
----------------------------------------------------------------------
Node Name IPv4 address Download Speed
CacheFly 205.234.175.175 105MB/s
Linode, Tokyo2, JP 139.162.65.37 17.9MB/s
Linode, Singapore, SG 139.162.23.4 12.9MB/s
Linode, London, UK 176.58.107.39 17.9MB/s
Linode, Frankfurt, DE 139.162.130.8 16.5MB/s
Linode, Fremont, CA 50.116.14.9 89.7MB/s
Softlayer, Dallas, TX 173.192.68.18 42.6MB/s
Softlayer, Seattle, WA 67.228.112.250 48.8MB/s
Softlayer, Frankfurt, DE 159.122.69.4 9.74MB/s
Softlayer, Singapore, SG 119.81.28.170 8.83MB/s
Softlayer, HongKong, CN 119.81.130.170 10.5MB/s

在sata上, 使用LVM:

----------------------------------------------------------------------
I/O speed(1st run) : 148 MB/s
I/O speed(2nd run) : 149 MB/s
I/O speed(3rd run) : 148 MB/s
Average I/O speed : 148.3 MB/s
----------------------------------------------------------------------

在sata上不使用LVM:

----------------------------------------------------------------------
I/O speed(1st run) : 142 MB/s
I/O speed(2nd run) : 146 MB/s
I/O speed(3rd run) : 145 MB/s
Average I/O speed : 144.3 MB/s
----------------------------------------------------------------------

由此可见, ESXi 在6.7 u3中应该修复了新的driver 的问题,在VM 中的IO测试和在裸机上的IO测试基本差不多。

下面我们来看看回退在旧的driver 的情况下, IO 的表现如何.

Centos7 with LVM on SSD:

----------------------------------------------------------------------
I/O speed(1st run) : 498 MB/s
I/O speed(2nd run) : 499 MB/s
I/O speed(3rd run) : 501 MB/s
Average I/O speed : 499.3 MB/s
----------------------------------------------------------------------

Centos7 Without LVM on SSD:

----------------------------------------------------------------------
I/O speed(1st run) : 500 MB/s
I/O speed(2nd run) : 499 MB/s
I/O speed(3rd run) : 499 MB/s
Average I/O speed : 499.3 MB/s
----------------------------------------------------------------------

Centos7 without LVM on SSD mounted on /data

----------------------------------------------------------------------
I/O speed(1st run) : 488 MB/s
I/O speed(2nd run) : 489 MB/s
I/O speed(3rd run) : 489 MB/s
Average I/O speed : 488.7 MB/s
----------------------------------------------------------------------

Centos7 with LVM on SATA

----------------------------------------------------------------------
I/O speed(1st run) : 147 MB/s
I/O speed(2nd run) : 148 MB/s
I/O speed(3rd run) : 147 MB/s
Average I/O speed : 147.3 MB/s
----------------------------------------------------------------------

Centos7 witout LVM on SATA mounted on /data

----------------------------------------------------------------------
I/O speed(1st run) : 139 MB/s
I/O speed(2nd run) : 139 MB/s
I/O speed(3rd run) : 136 MB/s
Average I/O speed : 138.0 MB/s
----------------------------------------------------------------------

 

最后总结,在ESXi 6.7 u3这个版本上,新的sata 驱动vmw_ahci已经能够很好的驱动各种ssd 和 sata了,而且表现还很好,不需要再去回退到旧的驱动了

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

This site uses Akismet to reduce spam. Learn how your comment data is processed.