搬瓦工DC6机器的benchmark

刚买了一个DC6的机器准备放中文网站,首先来benchmark一下.

采用的程序是: 

wget -qO- bench.sh | bash

下面是结果:

----------------------------------------------------------------------
CPU model : QEMU Virtual CPU version (cpu64-rhel6)
Number of cores : 2
CPU frequency : 2699.998 MHz
Total size of Disk : 20.4 GB (3.0 GB Used)
Total amount of Mem : 1006 MB (84 MB Used)
Total amount of Swap : 1283 MB (0 MB Used)
System uptime : 0 days, 0 hour 1 min
Load average : 0.00, 0.00, 0.00
OS : CentOS 7.7.1908
Arch : x86_64 (64 Bit)
Kernel : 3.10.0-1062.12.1.el7.x86_64
----------------------------------------------------------------------
I/O speed(1st run) : 332 MB/s
I/O speed(2nd run) : 737 MB/s
I/O speed(3rd run) : 682 MB/s
Average I/O speed : 583.7 MB/s
----------------------------------------------------------------------
Node Name IPv4 address Download Speed
CacheFly 205.234.175.175 108MB/s
Linode, Tokyo2, JP 139.162.65.37 20.7MB/s
Linode, Singapore, SG 139.162.23.4 12.5MB/s
Linode, London, UK 176.58.107.39 16.6MB/s
Linode, Frankfurt, DE 139.162.130.8 14.7MB/s
Linode, Fremont, CA 50.116.14.9 139MB/s
Softlayer, Dallas, TX 173.192.68.18 39.4MB/s
Softlayer, Seattle, WA 67.228.112.250 51.0MB/s
Softlayer, Frankfurt, DE 159.122.69.4 6.50MB/s
Softlayer, Singapore, SG 119.81.28.170 9.70MB/s
Softlayer, HongKong, CN 119.81.130.170 11.0MB/s
----------------------------------------------------------------------

可以说整体网络上还是非常好的

OVH的小鸡benchmark

以前没有用过NVME,这次的OVH的服务器的SSD是NVME的,赶紧装个VM测试下IO

测试所使用的程序:

wget -qO- bench.sh | bash

下面是结果:

----------------------------------------------------------------------
CPU model : Intel(R) Xeon(R) E-2236 CPU @ 3.40GHz
Number of cores : 8
CPU frequency : 3407.998 MHz
Total size of Disk : 22.3 GB (1.8 GB Used)
Total amount of Mem : 7987 MB (183 MB Used)
Total amount of Swap : 7628 MB (0 MB Used)
System uptime : 0 days, 3 hour 40 min
Load average : 0.00, 0.00, 0.00
OS : Debian GNU/Linux 9
Arch : x86_64 (64 Bit)
Kernel : 4.9.0-12-amd64
----------------------------------------------------------------------
I/O speed(1st run) : 1.3 GB/s
I/O speed(2nd run) : 1.3 GB/s
I/O speed(3rd run) : 1.3 GB/s
Average I/O speed : 1331.2 MB/s
----------------------------------------------------------------------
Node Name IPv4 address Download Speed
CacheFly 205.234.175.175 114MB/s
Linode, Tokyo2, JP 139.162.65.37 9.46MB/s

可以看到,IO读写能够到1.3GB/s…比sata ssd 要快1倍,这是很爽的

Quadranet 服务器benchmark

看到hostloc, quadranet的官方发布了一个sale,最近正好缺多IP的服务器,于是就订了一台跑一些服务。下面先做一些基本的benchmark,然后就要安装ESXi了

使用的程序:

wget -qO- bench.sh | bash

下面是结果:

----------------------------------------------------------------------
CPU model : Intel(R) Xeon(R) CPU E3-1240 v6 @ 3.70GHz
Number of cores : 8
CPU frequency : 799.890 MHz
Total size of Disk : 278.0 GB (1.7 GB Used)
Total amount of Mem : 15834 MB (232 MB Used)
Total amount of Swap : 2047 MB (0 MB Used)
System uptime : 0 days, 6 hour 45 min
Load average : 0.04, 0.04, 0.05
OS : CentOS 7.7.1908
Arch : x86_64 (64 Bit)
Kernel : 3.10.0-1062.12.1.el7.x86_64
----------------------------------------------------------------------
I/O speed(1st run) : 316 MB/s
I/O speed(2nd run) : 317 MB/s
I/O speed(3rd run) : 318 MB/s
Average I/O speed : 317.0 MB/s

—————

用的是老款的intel s3500, 速度勉强可以吧,基本保持在321MB/s左右

检查一下硬盘信息:

[[email protected] ~]# smartctl -a /dev/sda
smartctl 7.0 2018-12-30 r4883 [x86_64-linux-3.10.0-1062.12.1.el7.x86_64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family: Intel 730 and DC S35x0/3610/3700 Series SSDs
Device Model: INTEL SSDSC2BB300G4
Serial Number: BTWL40210787300PGN
LU WWN Device Id: 5 5cd2e4 04b56befb
Firmware Version: D2010370
User Capacity: 300,069,052,416 bytes [300 GB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: Solid State Device
Form Factor: 2.5 inches
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-2 T13/2015-D revision 3
SATA Version is: SATA 2.6, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Mon Feb 24 19:41:04 2020 PST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 0) seconds.
Offline data collection
capabilities: (0x79) SMART execute Offline immediate.
No Auto Offline data collection support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine
recommended polling time: ( 2) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x003d) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
5 Reallocated_Sector_Ct 0x0032 100 100 000 Old_age Always - 0
9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 34840
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 37
170 Available_Reservd_Space 0x0033 100 100 010 Pre-fail Always - 0
171 Program_Fail_Count 0x0032 100 100 000 Old_age Always - 0
172 Erase_Fail_Count 0x0032 100 100 000 Old_age Always - 0
174 Unsafe_Shutdown_Count 0x0032 100 100 000 Old_age Always - 35
175 Power_Loss_Cap_Test 0x0033 100 100 010 Pre-fail Always - 637 (204 414)
183 SATA_Downshift_Count 0x0032 100 100 000 Old_age Always - 2
184 End-to-End_Error 0x0033 100 100 090 Pre-fail Always - 0
187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0
190 Temperature_Case 0x0022 081 081 000 Old_age Always - 19 (Min/Max 17/20)
192 Unsafe_Shutdown_Count 0x0032 100 100 000 Old_age Always - 35
194 Temperature_Internal 0x0022 100 100 000 Old_age Always - 28
197 Current_Pending_Sector 0x0032 100 100 000 Old_age Always - 0
199 CRC_Error_Count 0x003e 100 100 000 Old_age Always - 4
225 Host_Writes_32MiB 0x0032 100 100 000 Old_age Always - 61519
226 Workld_Media_Wear_Indic 0x0032 100 100 000 Old_age Always - 911
227 Workld_Host_Reads_Perc 0x0032 100 100 000 Old_age Always - 84
228 Workload_Minutes 0x0032 100 100 000 Old_age Always - 2090316
232 Available_Reservd_Space 0x0033 100 100 010 Pre-fail Always - 0
233 Media_Wearout_Indicator 0x0032 100 100 000 Old_age Always - 0
234 Thermal_Throttle 0x0032 100 100 000 Old_age Always - 0/0
241 Host_Writes_32MiB 0x0032 100 100 000 Old_age Always - 61519
242 Host_Reads_32MiB 0x0032 100 100 000 Old_age Always - 341415

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

再看看sdb:

[[email protected] ~]# smartctl -a /dev/sdb
smartctl 7.0 2018-12-30 r4883 [x86_64-linux-3.10.0-1062.12.1.el7.x86_64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family: Intel 730 and DC S35x0/3610/3700 Series SSDs
Device Model: INTEL SSDSC2BB300G4
Serial Number: BTWL347206YW300PGN
LU WWN Device Id: 5 5cd2e4 04b53fc0e
Firmware Version: D2010370
User Capacity: 300,069,052,416 bytes [300 GB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: Solid State Device
Form Factor: 2.5 inches
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-2 T13/2015-D revision 3
SATA Version is: SATA 2.6, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Mon Feb 24 19:43:51 2020 PST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 0) seconds.
Offline data collection
capabilities: (0x79) SMART execute Offline immediate.
No Auto Offline data collection support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine
recommended polling time: ( 2) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x003d) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
5 Reallocated_Sector_Ct 0x0032 100 100 000 Old_age Always - 0
9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 39582
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 59
170 Available_Reservd_Space 0x0033 100 100 010 Pre-fail Always - 0
171 Program_Fail_Count 0x0032 100 100 000 Old_age Always - 0
172 Erase_Fail_Count 0x0032 100 100 000 Old_age Always - 0
174 Unsafe_Shutdown_Count 0x0032 100 100 000 Old_age Always - 57
175 Power_Loss_Cap_Test 0x0033 100 100 010 Pre-fail Always - 629 (226 417)
183 SATA_Downshift_Count 0x0032 100 100 000 Old_age Always - 0
184 End-to-End_Error 0x0033 100 100 090 Pre-fail Always - 0
187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0
190 Temperature_Case 0x0022 083 083 000 Old_age Always - 17 (Min/Max 16/17)
192 Unsafe_Shutdown_Count 0x0032 100 100 000 Old_age Always - 57
194 Temperature_Internal 0x0022 100 100 000 Old_age Always - 26
197 Current_Pending_Sector 0x0032 100 100 000 Old_age Always - 0
199 CRC_Error_Count 0x003e 100 100 000 Old_age Always - 0
225 Host_Writes_32MiB 0x0032 100 100 000 Old_age Always - 4055029
226 Workld_Media_Wear_Indic 0x0032 100 100 000 Old_age Always - 16179
227 Workld_Host_Reads_Perc 0x0032 100 100 000 Old_age Always - 21
228 Workload_Minutes 0x0032 100 100 000 Old_age Always - 2374838
232 Available_Reservd_Space 0x0033 100 100 010 Pre-fail Always - 0
233 Media_Wearout_Indicator 0x0032 085 085 000 Old_age Always - 0
234 Thermal_Throttle 0x0032 100 100 000 Old_age Always - 0/0
241 Host_Writes_32MiB 0x0032 100 100 000 Old_age Always - 4055029
242 Host_Reads_32MiB 0x0032 100 100 000 Old_age Always - 1091089

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

/dev/sdb的media_wearout_indicator 是85,而非100,表明这个 ssd 硬盘已经开始downgraded

给quadranet 发了ticket 要求更换,等等看吧

经过好几天的communication,QN最后终于换了一个新的SSD给我,并且升级到了1TB SSD.

[[email protected] ~]# smartctl -a /dev/sda
smartctl 7.0 2018-12-30 r4883 [x86_64-linux-3.10.0-1062.12.1.el7.x86_64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family: Crucial/Micron BX/MX1/2/3/500, M5/600, 1100 SSDs
Device Model: CT1000MX500SSD1
Serial Number: 1949E22DBD7F
LU WWN Device Id: 5 00a075 1e22dbd7f
Firmware Version: M3CR023
User Capacity: 1,000,204,886,016 bytes [1.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: Solid State Device
Form Factor: 2.5 inches
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-3 T13/2161-D revision 5
SATA Version is: SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Thu Feb 27 17:48:28 2020 PST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status: (0x80) Offline data collection activity
was never started.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 0) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 30) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x0031) SCT Status supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 100 100 000 Pre-fail Always - 0
5 Reallocate_NAND_Blk_Cnt 0x0032 100 100 010 Old_age Always - 0
9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 8
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 3
171 Program_Fail_Count 0x0032 100 100 000 Old_age Always - 0
172 Erase_Fail_Count 0x0032 100 100 000 Old_age Always - 0
173 Ave_Block-Erase_Count 0x0032 100 100 000 Old_age Always - 0
174 Unexpect_Power_Loss_Ct 0x0032 100 100 000 Old_age Always - 1
180 Unused_Reserve_NAND_Blk 0x0033 000 000 000 Pre-fail Always - 47
183 SATA_Interfac_Downshift 0x0032 100 100 000 Old_age Always - 0
184 Error_Correction_Count 0x0032 100 100 000 Old_age Always - 0
187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0
194 Temperature_Celsius 0x0022 067 066 000 Old_age Always - 33 (Min/Max 0/34)
196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 100 100 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 100 100 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 100 100 000 Old_age Always - 0
202 Percent_Lifetime_Remain 0x0030 100 100 001 Old_age Offline - 0
206 Write_Error_Rate 0x000e 100 100 000 Old_age Always - 0
210 Success_RAIN_Recov_Cnt 0x0032 100 100 000 Old_age Always - 0
246 Total_Host_Sector_Write 0x0032 100 100 000 Old_age Always - 7136328
247 Host_Program_Page_Count 0x0032 100 100 000 Old_age Always - 128446
248 FTL_Program_Page_Count 0x0032 100 100 000 Old_age Always - 626

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Completed [00% left] (0-65535)
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

 

但是分区记录表很奇怪,我这里记录一下:

[[email protected] ~]# fdisk -l
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes, 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: gpt
Disk identifier: CC49A639-0ABB-4FC8-9CE4-9CB5D28B7ED7


# Start End Size Type Name
1 2048 4095 1M BIOS boot
2 4096 2101247 1G Microsoft basic
3 2101248 6295551 2G Microsoft basic
4 6295552 10489855 2G Linux swap
5 10489856 1953523711 926.5G Microsoft basic

Krypt iON Benchmark

从hostloc 坛友那里收的, 20.19刀那一款,双向CN2 GIA,还是很不错的,做一个benchmark记录一下.

使用的程序:

wget -qO- bench.sh | bash

下面是结果:

----------------------------------------------------------------------
CPU model : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
Number of cores : 1
CPU frequency : 2499.996 MHz
Total size of Disk : 39.0 GB (4.7 GB Used)
Total amount of Mem : 1837 MB (365 MB Used)
Total amount of Swap : 1023 MB (1 MB Used)
System uptime : 43 days, 20 hour 23 min
Load average : 0.30, 0.08, 0.07
OS : CentOS 7.7.1908
Arch : x86_64 (64 Bit)
Kernel : 3.10.0-1062.9.1.el7.x86_64
----------------------------------------------------------------------
I/O speed(1st run) : 632 MB/s
I/O speed(2nd run) : 669 MB/s
I/O speed(3rd run) : 553 MB/s
Average I/O speed : 618.0 MB/s
----------------------------------------------------------------------
Node Name IPv4 address Download Speed
CacheFly 205.234.175.175 66.1MB/s
Linode, Tokyo2, JP 139.162.65.37 21.6MB/s
Linode, Singapore, SG 139.162.23.4 13.3MB/s
Linode, London, UK 176.58.107.39 16.9MB/s
Linode, Frankfurt, DE 139.162.130.8 15.8MB/s
Linode, Fremont, CA 50.116.14.9 90.9MB/s
Softlayer, Dallas, TX 173.192.68.18 61.6MB/s
Softlayer, Seattle, WA 67.228.112.250 60.3MB/s
Softlayer, Frankfurt, DE 159.122.69.4 11.3MB/s
Softlayer, Singapore, SG 119.81.28.170 10.1MB/s
Softlayer, HongKong, CN 119.81.130.170 12.3MB/s
----------------------------------------------------------------------

 

AWS Lightsail Singapore Benchmark

AWS对于IO的限制,配置越高,限制越少,这里我们benchmark 一下80刀的这一款,4 cores,16GB RAM, 320GB SSD

测试使用的程序:

wget -qO- bench.sh | bash

下面是结果:

----------------------------------------------------------------------
CPU model : Intel(R) Xeon(R) CPU E5-2676 v3 @ 2.40GHz
Number of cores : 4
CPU frequency : 2399.858 MHz
Total size of Disk : 320.0 GB (1.1 GB Used)
Total amount of Mem : 15883 MB (171 MB Used)
Total amount of Swap : 0 MB (0 MB Used)
System uptime : 0 days, 0 hour 9 min
Load average : 0.15, 0.15, 0.09
OS : CentOS 7.6.1810
Arch : x86_64 (64 Bit)
Kernel : 3.10.0-957.1.3.el7.x86_64
----------------------------------------------------------------------
I/O speed(1st run) : 134 MB/s
I/O speed(2nd run) : 130 MB/s
I/O speed(3rd run) : 130 MB/s
Average I/O speed : 131.3 MB/s
----------------------------------------------------------------------

测试了几次,基本都是这个IO速度

 

搬瓦工DC9 CN2GIA 小鸡benchmark

用了很久的CN2GIA,今天没事正好看看网络质量,看看是不是合适做smb服务.

测试使用的程序:

wget -qO- bench.sh | bash

下面是结果:

----------------------------------------------------------------------
CPU model : QEMU Virtual CPU version (cpu64-rhel6)
Number of cores : 2
CPU frequency : 2599.998 MHz
Total size of Disk : 19.9 GB (2.5 GB Used)
Total amount of Mem : 1003 MB (89 MB Used)
Total amount of Swap : 259 MB (10 MB Used)
System uptime : 0 days, 0 hour 37 min
Load average : 0.33, 0.09, 0.03
OS : Debian GNU/Linux 10
Arch : x86_64 (64 Bit)
Kernel : 4.19.0-6-amd64
----------------------------------------------------------------------
I/O speed(1st run) : 444 MB/s
I/O speed(2nd run) : 431 MB/s
I/O speed(3rd run) : 406 MB/s
Average I/O speed : 427.0 MB/s
----------------------------------------------------------------------
Node Name IPv4 address Download Speed
CacheFly 205.234.175.175 84.2MB/s
Linode, Tokyo2, JP 139.162.65.37 2.32MB/s
Linode, Singapore, SG 139.162.23.4 1.09MB/s
Linode, London, UK 176.58.107.39 2.12MB/s
Linode, Frankfurt, DE 139.162.130.8 2.06MB/s
Linode, Fremont, CA 50.116.14.9 12.0MB/s
Softlayer, Dallas, TX 173.192.68.18 3.96MB/s
Softlayer, Seattle, WA 67.228.112.250 7.59MB/s
Softlayer, Frankfurt, DE 159.122.69.4 2.05MB/s
Softlayer, Singapore, SG 119.81.28.170 4.78MB/s
Softlayer, HongKong, CN 119.81.130.170 3.06MB/s

由此可见,DC9 的网络对US 的支持很一般,开启samba服务要当心

WebNX Utah机房E3-1275V5 1TB SSD服务器benchmark

在webhostingtalk上看到webnx 有一款机器在on sale,配置相当不错,而我又是webnx 的超级粉,立马拿下替换了原来旧的机器,新机器的配置如下:

Intel Xeon E3-1275v5 quadcore 3.6ghz
64gb ddr4 ECC ram
1TB Enterprise SSD
/29 5x usable ip's
30Tb over 1gbps
$65 ** limited time until we burn up the stock on hand!

实话实说,这个机器还是非常有新引力的,再加上webnx 100%的SLA 和 super connected network, 真的是非常爽的价格.

我准备用它来做ESXi 新的vCSA 主控. 

在deploy ESXi之前,先来测试一下这台机器的硬件和network. 

更多

Online.net Pro-4-M 独立服务器IO Benchmark

很是喜欢online.net 家的服务,以为他们家都是使用的Dell 的服务器。

今天准备再增加一台服务器专门来管理服务器集群的时候,才发现online只有在中高端服务器上采用Dell EMC 和 Dell 的服务器,在中低端市场上已online labs 自己的服务器以及QTS 服务器为主,中端市场也有一部分是Supermicro 服务器。

因为主要是拿这台服务器做管理以及存放一些静态ISO镜像,所以需求的配置不要求高,300 GB 加 500GB SSD 就可以了,最好还是SuperMicro 主板,选来选去就只有Pro-4-M 符合条件了。OVH 倒是也有一些符合条件的,就是经过验证的OVH账号实在太难获得了.

和Pro-4-L 不同,这台机器带的SSD是三星的,型号是PM871b,用国内的话来说,是850 evo 的马甲。看来online 的服务器用家用级别的SSD 多一点

测试使用的程序:

wget -qO- bench.sh | bash

首先是在裸机上安装CentOS 测试.

----------------------------------------------------------------------
CPU model : Intel(R) Xeon(R) CPU E3-1245 v5 @ 3.50GHz
Number of cores : 8
CPU frequency : 1099.945 MHz
Total size of Disk : 461.3 GB (1.5 GB Used)
Total amount of Mem : 64050 MB (485 MB Used)
Total amount of Swap : 8511 MB (0 MB Used)
System uptime : 0 days, 1 hour 24 min
Load average : 0.00, 0.01, 0.05
OS : CentOS 7.6.1810
Arch : x86_64 (64 Bit)
Kernel : 3.10.0-957.27.2.el7.x86_64
----------------------------------------------------------------------
I/O speed(1st run) : 427 MB/s
I/O speed(2nd run) : 427 MB/s
I/O speed(3rd run) : 427 MB/s
Average I/O speed : 427.0 MB/s

 

[[email protected] ~]# hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 1554 MB in 3.00 seconds = 517.37 MB/sec
[[email protected] ~]# hdparm -t /dev/sdb

/dev/sdb:
Timing buffered disk reads: 1356 MB in 3.00 seconds = 451.67 MB/sec
[[email protected] ~]# hdparm -t /dev/sdc

/dev/sdc:
Timing buffered disk reads: 1348 MB in 3.00 seconds = 449.08 MB/sec

WebNX.com 服务器重新benchmark

和online.net 一样,前面的测试有点不太规范,这次有机会重新测试一下

机器配置和online.net 一样, E5 1650V3, 64GB RAM, 2 X 1TB SSD ( micron 5100 enterprise SSD)

测试使用的程序是:

wget -qO- bench.sh | bash

直接测试IO,使用的是下面的commands:

dd if=/dev/zero of=test_$$ bs=64k count=16k conv=fdatasync && rm -f test_$$

直接在服务器裸机安装Centos 7,下面是测试结果:

[[email protected] ~]# hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 1422 MB in 3.00 seconds = 473.59 MB/sec
[[email protected] ~]# hdparm -t /dev/sdb

/dev/sdb:
Timing buffered disk reads: 1452 MB in 3.00 seconds = 483.80 MB/sec

[[email protected] ~]# dd if=/dev/zero of=test_$$ bs=64k count=16k conv=fdatasync && rm -f test_$$
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 2.75682 s, 389 MB/s

测试了几次,都在390 MB/s 左右

[[email protected]]# dd if=/dev/zero of=test_$$ bs=64k count=16k conv=fdatasync && rm -f test_$$
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 2.75827 s, 389 MB/s

两块SSD 都是micron 的全新 1 TB SSD, 看来SSD 的速度就在400 MB/s 左右

用bench.sh的程序测试下:

----------------------------------------------------------------------
CPU model : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz
Number of cores : 12
CPU frequency : 1199.920 MHz
Total size of Disk : 851.3 GB (1.5 GB Used)
Total amount of Mem : 64316 MB (564 MB Used)
Total amount of Swap : 30516 MB (0 MB Used)
System uptime : 0 days, 0 hour 8 min
Load average : 0.08, 0.05, 0.03
OS : CentOS 7.7.1908
Arch : x86_64 (64 Bit)
Kernel : 3.10.0-1062.4.1.el7.x86_64
----------------------------------------------------------------------
I/O speed(1st run) : 391 MB/s
I/O speed(2nd run) : 388 MB/s
I/O speed(3rd run) : 389 MB/s
Average I/O speed : 389.3 MB/s
----------------------------------------------------------------------

依然在390MB/s左右, micron 的SSD的表现要差于intel 和 samsung的SSD

下面看看在ESXi 6.7 上使用native drive 的表现:

Centos 7 with LVM

----------------------------------------------------------------------
CPU model : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz
Number of cores : 4
CPU frequency : 3499.997 MHz
Total size of Disk : 27.0 GB (1.3 GB Used)
Total amount of Mem : 3950 MB (165 MB Used)
Total amount of Swap : 3071 MB (0 MB Used)
System uptime : 0 days, 0 hour 6 min
Load average : 0.16, 0.10, 0.06
OS : CentOS 7.6.1810
Arch : x86_64 (64 Bit)
Kernel : 3.10.0-957.el7.x86_64
----------------------------------------------------------------------
I/O speed(1st run) : 469 MB/s
I/O speed(2nd run) : 466 MB/s
I/O speed(3rd run) : 468 MB/s
Average I/O speed : 467.7 MB/s
----------------------------------

可以看到这个这个数据非常的好,基本接近原生SSD 的表现了

下面是Centos 7 without LVM

----------------------------------------------------------------------
CPU model : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz
Number of cores : 4
CPU frequency : 3499.997 MHz
Total size of Disk : 26.3 GB (2.3 GB Used)
Total amount of Mem : 3950 MB (150 MB Used)
Total amount of Swap : 3813 MB (0 MB Used)
System uptime : 0 days, 0 hour 1 min
Load average : 0.58, 0.23, 0.08
OS : CentOS 7.6.1810
Arch : x86_64 (64 Bit)
Kernel : 3.10.0-957.el7.x86_64
----------------------------------------------------------------------
I/O speed(1st run) : 458 MB/s
I/O speed(2nd run) : 457 MB/s
I/O speed(3rd run) : 460 MB/s
Average I/O speed : 458.3 MB/s

Centos 7 with LVM and standard partition mounted on /sata,

----------------------------------------------------------------------
I/O speed(1st run) : 452 MB/s
I/O speed(2nd run) : 456 MB/s
I/O speed(3rd run) : 461 MB/s
Average I/O speed : 456.3 MB/s
----------------------------------------------------------------------

/data上, SSD 的IO 也会降速,需要很长一段时间才能恢复满速

在Debian上测试一下(无论LVM还是standard partition结果都一样):

----------------------------------------------------------------------
----------------------------------------------------------------------
CPU model            : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz
Number of cores      : 4
CPU frequency        : 3499.997 MHz
Total size of Disk   : 94.2 GB (13.1 GB Used)
Total amount of Mem  : 4044 MB (129 MB Used)
Total amount of Swap : 4191 MB (0 MB Used)
System uptime        : 0 days, 0 hour 9 min
----------------------------------------------------------------------
CPU model            : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz
Number of cores      : 4
CPU frequency        : 3499.997 MHz
Total size of Disk   : 94.2 GB (13.1 GB Used)
Total amount of Mem  : 4044 MB (129 MB Used)
Total amount of Swap : 4191 MB (0 MB Used)
System uptime        : 0 days, 0 hour 15 min
Load average         : 0.02, 0.13, 0.14
OS                   : Debian GNU/Linux 10
Arch                 : x86_64 (64 Bit)
Kernel               : 4.19.0-6-amd64
----------------------------------------------------------------------
I/O speed(1st run)   : 456 MB/s
I/O speed(2nd run)   : 457 MB/s
I/O speed(3rd run)   : 462 MB/s
Average I/O speed    : 458.3 MB/s
----------------------------------------------------------------------
Node Name                       IPv4 address            Download Speed
CacheFly                        205.234.175.175         109MB/s
Linode, Tokyo2, JP              139.162.65.37           22.3MB/s
Linode, Singapore, SG           139.162.23.4
Linode, London, UK              176.58.107.39           16.4MB/s
Linode, Frankfurt, DE           139.162.130.8           15.1MB/s
Linode, Fremont, CA             50.116.14.9             74.5MB/s
Softlayer, Dallas, TX           173.192.68.18           51.5MB/s
Softlayer, Seattle, WA          67.228.112.250          57.1MB/s
Softlayer, Frankfurt, DE        159.122.69.4            14.0MB/s
Softlayer, Singapore, SG        119.81.28.170           10.8MB/s
Softlayer, HongKong, CN         119.81.130.170          12.7MB/s
----------------------------------------------------------------------
----------------------------------------------------------------------

在Debian上,有的时候速度会降到200MB/s, 但是需要很长一段时间来恢复

下面来看看回退ssd 的driver 以后的表现:

回退driver以后,Debian 10 with LVM or without LVM

----------------------------------------------------------------------
CPU model            : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz
Number of cores      : 4
CPU frequency        : 3499.997 MHz
Total size of Disk   : 94.2 GB (13.1 GB Used)
Total amount of Mem  : 4044 MB (142 MB Used)
Total amount of Swap : 4191 MB (0 MB Used)
System uptime        : 0 days, 12 hour 13 min
Load average         : 0.00, 0.00, 0.00
OS                   : Debian GNU/Linux 10
Arch                 : x86_64 (64 Bit)
Kernel               : 4.19.0-6-amd64
----------------------------------------------------------------------
I/O speed(1st run)   : 456 MB/s
I/O speed(2nd run)   : 461 MB/s
I/O speed(3rd run)   : 459 MB/s
Average I/O speed    : 458.7 MB/s

在测试中发现,SSD在standard partition上会降速,降速到200MB/s以后,会花费很长一段时间恢复到满速460MB/s, 但是在centos 7下,这个恢复是很快的

Centos 7 Standard Partition:

----------------------------------------------------------------------
I/O speed(1st run) : 455 MB/s
I/O speed(2nd run) : 457 MB/s
I/O speed(3rd run) : 457 MB/s
Average I/O speed : 456.3 MB/s
----------------------------------------------------------------------

Centos 7 with LVM:

----------------------------------------------------------------------
I/O speed(1st run) : 464 MB/s
I/O speed(2nd run) : 466 MB/s
I/O speed(3rd run) : 462 MB/s
Average I/O speed : 464.0 MB/s
----------------------------------------------------------------------

Centos 7 LVM with standard partition mounted on /data

----------------------------------------------------------------------
I/O speed(1st run) : 456 MB/s
I/O speed(2nd run) : 461 MB/s
I/O speed(3rd run) : 458 MB/s
Average I/O speed : 458.3 MB/s
----------------------------------------------------------------------

在测试中发现,即使会有短暂的降速到200MB/s, 也会很快的恢复到460MB/s 的满速

通过以上的测试可以看到,vmware 全新的native driver,基本上解决了micron ssd 的low IO performance 的问题

micron 的ssd 偶尔会降速到200 MB/s,但是基本是可以恢复满速的,只是时间有长有短的问题。

Online.net Pro-4-L 重新benchmark

前面的测试有点不太规范,这次又机会重新测试一下。

机器配置是Xeon E5 1650V3, 96GB RAM, 3X500GB SSD

测试使用的程序是:

wget -qO- bench.sh | bash

直接测试IO,使用的是下面的commands:

dd if=/dev/zero of=test_$$ bs=64k count=16k conv=fdatasync && rm -f test_$$

直接在服务器裸机上安装Centos 7, 下面是测试结果:

[[email protected] ~]# hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 1454 MB in 3.00 seconds = 484.53 MB/sec

[[email protected] ~]# hdparm -t /dev/sdb

/dev/sdb:
Timing buffered disk reads: 1472 MB in 3.00 seconds = 490.04 MB/sec

[[email protected] ~]# hdparm -t /dev/sdc

/dev/sdc:
iming buffered disk reads: 1470 MB in 3.00 seconds = 489.96 MB/sec
[[email protected] sda~]# dd if=/dev/zero of=test_$$ bs=64k count=16k conv=fdatasync && rm -f test_$$
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 2.65959 s, 404 MB/s

测试了几次,都在400 MB/s 左右

[[email protected] sdb]# dd if=/dev/zero of=test_$$ bs=64k count=16k conv=fdatasync && rm -f test_$$
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 2.69217 s, 399 MB/s
[[email protected] sdc]# dd if=/dev/zero of=test_$$ bs=64k count=16k conv=fdatasync && rm -f test_$$
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 2.70218 s, 397 MB/s

可以看到裸机直接测试的话,SSD 的IO 一般就在400 MB/s 左右

在Host上安装上Esxi 6.7 以后,分别测试Debian 9 和 Debian 10

Debian 9 测试结果:

----------------------------------------------------------------------
CPU model : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz
Number of cores : 8
CPU frequency : 3495.142 MHz
Total size of Disk : 55.0 GB (0.9 GB Used)
Total amount of Mem : 3907 MB (92 MB Used)
Total amount of Swap : 4045 MB (0 MB Used)
System uptime : 0 days, 0 hour 11 min
Load average : 0.96, 0.28, 0.09
OS : Debian GNU/Linux 9
Arch : x86_64 (64 Bit)
Kernel : 4.9.0-9-amd64
----------------------------------------------------------------------
I/O speed(1st run) : 82.2 MB/s
I/O speed(2nd run) : 79.9 MB/s
I/O speed(3rd run) : 80.1 MB/s
Average I/O speed : 80.7 MB/s

Debian 10 测试结果:

----------------------------------------------------------------------
CPU model : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz
Number of cores : 8
CPU frequency : 3495.951 MHz
Total size of Disk : 55.0 GB (1.0 GB Used)
Total amount of Mem : 3897 MB (150 MB Used)
Total amount of Swap : 4045 MB (0 MB Used)
System uptime : 0 days, 0 hour 25 min
Load average : 1.14, 0.31, 0.11
OS : Debian GNU/Linux 10
Arch : x86_64 (64 Bit)
Kernel : 4.19.0-5-amd64
----------------------------------------------------------------------
I/O speed(1st run) : 78.0 MB/s
I/O speed(2nd run) : 77.0 MB/s
I/O speed(3rd run) : 76.9 MB/s
Average I/O speed : 77.3 MB/s

本来以为Debian 10 有什么特殊科技呢,IO 测试和Debian 9 差不多

下面是重头戏,回退一些Esxi 的 ACHI 的driver,再来测试一下:

----------------------------------------------------------------------
CPU model : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz
Number of cores : 8
CPU frequency : 3494.956 MHz
Total size of Disk : 55.0 GB (0.9 GB Used)
Total amount of Mem : 3907 MB (91 MB Used)
Total amount of Swap : 4045 MB (0 MB Used)
System uptime : 0 days, 0 hour 4 min
Load average : 0.22, 0.05, 0.02
OS : Debian GNU/Linux 9
Arch : x86_64 (64 Bit)
Kernel : 4.9.0-9-amd64
----------------------------------------------------------------------
I/O speed(1st run) : 301 MB/s
I/O speed(2nd run) : 299 MB/s
I/O speed(3rd run) : 303 MB/s
Average I/O speed : 301.0 MB/s

数据在350-370 之间乱跳

在Debian 10上,闲时IO 能到400 MB/s, 平时也基本就是300-350左右

----------------------------------------------------------------------
CPU model : Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz
Number of cores : 8
CPU frequency : 3495.951 MHz
Total size of Disk : 55.0 GB (2.0 GB Used)
Total amount of Mem : 3897 MB (148 MB Used)
Total amount of Swap : 4045 MB (0 MB Used)
System uptime : 0 days, 0 hour 21 min
Load average : 0.00, 0.01, 0.01
OS : Debian GNU/Linux 10
Arch : x86_64 (64 Bit)
Kernel : 4.19.0-5-amd64
----------------------------------------------------------------------
I/O speed(1st run) : 306 MB/s
I/O speed(2nd run) : 300 MB/s
I/O speed(3rd run) : 299 MB/s
Average I/O speed : 301.7 MB/s