Samsung SSD 840: Endurance Destruct Test

When you operate a Datacenter with many servers you also probably have big number of installed disks. In most cases even you have cluster, especialy in small companies, there is still SPOF (Single point of failure) somewhere.

In this SPOF it's critical to know condition of the system. When system using HDD it's important to know HDD condition to prevent failures, plan maintenance etc.

We will talk about Samsung SSD 840 PRO series. We realize this SSD has very good performance and lifetime. Before we use it in production we must know how to monitor condition. There is many articles and technical specification but we had a lot of questions without answer.

(Update: 10.10.2013)

(Samsung performance issues, read new post, 03.12.2014)

For example:

1. What happened when smart normalized value "Wearleveling count" (WLC)  drop to 0 ?


2. Will It effect disk performance ?


3. How temperature change during utilization of disk ?


4. How many data can we write to disk without error ?

Hardware

• Samsung SSD 840 PRO - 128 GB


• Dell Server R320


• 32 GB ECC Ram


• 1x CPU, Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz (6 Cores + HT)


• SAS Controller, Perc H710 NV


• OS Debian Squeeze


• Linux Kernel 3.2.2

Tests

Our test consist of two stages which are in a loop.

Stage 1 is performance test. This test writes 5 GB of data to disk for seq. read/write, rand. read/write, tests. During this stage we write something about 20 GB of data. We measure speed, latency and iops.

Stage 2 is fill test. This test measure all values from Stage 1 but fill whole disk.

All stages writes data with option "sync".

Test server is monitored separately for CPU, Memory, System Load Average, Network, Stats of active Sockets, Number of Processes, IO stats, Disks free space, number of connected users. We also monitor all Smart values from SSD disk.

You may ask, why monitored all these values. Simple answer is, to know what is going on the server. When we will processing the results of these tests, we should see some strange values in some time intervals. We must have additional information what happened to decide this results is false or it's real behavior of SSD disk.

I use FIO utility for all tests on SSD.

Overal I have 58 graphs from the server, most of it's smart values.

Wearleveling count

What happened when it drops to 0 ?

Simply nothing :). This value drops to 1 when it writes about 465 TB of data. This value I count from number of tests and crosscheck with Smart Value - "Total LBAs Written". What I realize this is only prefail value and there was no errors or sectors reallocation. To this days disk wrote another 235 TB without any errors or reallocations, and test still continue.

Wearleveling count vs performance

As you can see on graphs below, there is no performance decrease before or after we reach 1% WLC.



This graphs is only for comparison with indicatively values. You should see no performance decrease during test.

When we want exact values we must look into result logs from fio utility. Section Graphs from FIO utility of this article.

Temperature

In our datacenter temperature of SSD is somewhere around 26 °C. When SSD start writing and reading temperature grow up to 39.9 °C

Data written

As I mentioned above, test isn't done yet. When I wrote this article, Total LBAs Written was 1503673300498. This value show how many 512 bytes blocks ware written.

LBA_Value * 512 = Bytes written

So we written about 700 TB to this 128 GB disk. Our tests have 148 GB (128 GB fill disk and 20 GB test disk). We run about 5000 tests which I can see on counter. When we divide 700 TB with 148 GB we get around 4870 tests. It's really close, difference is around 3%.

Bandwidth and IOPS

First look on manufacturer site for some specification.

Manufacturer specification:

• Seq. read up to 530 MB/s


• Seq. write up to 390 MB/s


• Rand. read speed up to 97 000 IOPS


• Rand. write speed up to 90 000 IOPS

Let see my measured values:

• Seq. read ~ 275 MB/s


• Seq. write ~ 300MB/s


• Random read over 250 MB/s


• Random write ~ 100 MB/s


• Random read speed ~ 65 000 IOPS


• Random write speed ~ 28 000 IOPS


• Seq. read speed ~ 67 000 IOPS


• Seq. write speed ~ 75 000 IOPS

Quite interesting results.  Some values are similar to manufacturer specification and some are really different.

Graphs from FIO test utility

 

Conclusion

Test still continues, disk have WLC on 1% but no reserved blocks was used, no reallocation, no errors.

When we reach 1% of WLC, disk write about 465 TB of data.

It means if your server writes daily 20 GB of data, it will take 65 years. For example if you rewrite whole disk every day you reach 1% of WLC in about 10 years.

If you plan renewal HW every 5 years, you can be safe if you rewrite whole disk twice a day.

What I see, it's good to monitor these values:

• Normalized value of WLC


• Reallocated sector count


• Normalized value of Used Reserved Blocks Count

Dictionary

Wearleveling count - The maximum number of erase operations performed on a single flash memory block.

Reallocated sector count - When encountering a read/write/check error, a device remaps a bad sector to a "healthy" one taken from a special reserve pool. Normalized value of the attribute decreases as the number of available spares decreases.On a regular hard drive, Raw value indicates the number of remapped sectors, which should normally be zero. On an SSD, the Raw value indicates the number of failed flash memory blocks.

Used Reserved Blocks Count - On an SSD, this attribute describes the state of the reserve block pool. The value of the attribute shows the percentage of the pool remaining. The Raw value sometimes contains the actual number of used reserve blocks.

 

UPDATE 10.10.2013

Our SSD is finaly dead after almost 5 months of heavy write test.

Some numbers

• more then 3 PB written


• rewritten more then 24 400 times


• stable temperature about 37 C

and some nice graphs





 

UPDATE: (Samsung performance issues, read new post, 03.12.2014)