Saturday, November 17, 2012

AnandTech Article Channel

AnandTech Article Channel

AnandTech/Intel S3700 Roundtable Discussion & Webcast Videos Live

Posted: 16 Nov 2012 07:16 AM PST

Intel invited me to attend SC12 and participate in a webcast for the launch of its new DC S3700 SSD. I joined Roger Peene from Intel's SSD Solutions and we talked about the S3700 as well as answered your questions live. If you missed the webcast, you can find the pre-recorded video here. There's a great question about the future of NAND we discussed on the webcast that I'd highly recommend paying attention to.

Prior to the webcast, I had the chance to sit down with Arbin Kumar (responsible for Intel SSD reliability and validation), Allison Goodman (lead engineer on the S3700) and Roger Peene (Marketing Director for Intel's Datacenter SSD Solutions) once again to discuss the S3700 in greater detail. The discussion in the video below is from the first day I really learned about the S3700's architecture. The full discussion took several hours but the video below distills a lot of it down to 7 minutes. If you want to hear about the S3700 from the folks who actually had a hand in building the drive, I strongly suggest watching the video. Update: Looks like the video got pulled, trying to see what happened now.

Finally, at SC12 Intel rented a replica of the original series bridge from the starship Enterprise which we used as a backdrop for the webcast. Prior to the webcast airing, we had some fun on the bridge which you can check out in the gallery below.

At the end of the day it was a pretty fun experience. I learned quite a bit about Intel's NAND Solutions Group through this whole process. The SSD business is pretty unusual in that it's built around a replacement to a highly commoditized product (mechanical storage). It's surprising that we even have folks who typically play in high margin products even in this industry, but without them the market would be much worse off. I still remember what things were like with SSDs prior to the X25-M and even for the 12 - 18 months after its launch. The S3700 showed that there's still room for innovation even within the constraints of 6Gbps SATA, which should tide us over until SATA Express shows up.

Samsung SSD 840: Testing the Endurance of TLC NAND

Posted: 16 Nov 2012 06:18 AM PST

NAND endurance is something that always raises questions among those considering a move to solid state storage. Even though we have showed more than once that the endurance of today's MLC NAND based SSDs is more than enough for even enterprise workloads, the misconception of SSDs having a short lifespan still lives. Back in the day when we had 3Xnm MLC NAND with 5,000 P/E cycles, people were worried about wearing our their SSDs, although there was absolutely nothing to worry about. The move to ~20nm MLC NAND has reduced the available P/E cycles to 3,000, but that's still plenty.

We have tested MLC NAND endurance before but with the release of Samsung SSD 840, we had something new to test: TLC NAND. We have explained the architectural differences between SLC, MLC and TLC NAND several times by now, but I'll do a brief recap here (I strongly recommend reading the detailed explanation if you want to truly understand how TLC NAND works):

Bits per Cell 1 2 3
P/E Cycles (2Xnm) 100,000 3,000 1,000
Read Time 25us 50us ~75us
Program Time 200-300us 600-900us ~900-1350us
Erase Time 1.5-2ms 3ms ~4.5ms

The main difference is that MLC stores two bits per cell, whereas TLC stores three. This results in eight voltage states instead of four (also means that one TLC cell has eight possible data values). Voltages used to program the cell are usually between 15V and 18V, so there isn't exactly a lot room to play with when you need to fit twice as many voltage states within the same space. The problem is that when the cell gets cycled (i.e. programmed and erased), the room taken by one voltage state increases due to electron trapping and current leakage. TLC can't tolerate as much change in the voltage states as MLC can because there is less voltage headroom and you can't end up in a situation where two voltage states become one (the cell wouldn't give valid values because it doesn't know if it's programmed as "110" or "111" for example). Hence the endurance of TLC NAND is lower; it simply cannot be programmed and erased as many times as MLC NAND and thus you can't write as much to a TLC NAND based SSD.

No manufacturer has openly wanted to discuss the endurance of TLC, so the numbers we have seen before have been educated guesses. 1,000 - 1,500 P/E cycles is what I've heard for TLC NAND. The reality can also be different from what manufacturers claim as we discovered in the Intel SSD 335 (though there is a high probability that it's just a firmware bug), so actually testing the endruance is vital. 

There was one obstacle, though. Samsung does not report NAND writes like Intel does and without NAND writes we can't know for sure how much data is written to the NAND because of write amplification. Fortunately, there is a a workaround: I wrote incompressible 128KB sequential data (QD=1) to the drive and took down the duration of each run and the Wear Leveling Count (similar to Media Wear Indicator). If I know the average write speed and the duration, I can figure out how much I wrote to the drive. Sequential large block-size data should also result in write amplification near 1x because the data is sequential and thus doesn't fragment the drive. I then compared the amount of data I wrote to the WLC values I had recorded:

Samsung SSD 840 (250GB) Endurance Testing
Total Amount of Data Written 92,623 GiB
Total Amount of WLC Exhausted 34
Estimated Total Amount of P/E Cycles 1,064
Estimated Total Write Endurance 272,420 GiB

It seems that 1,000 P/E cycles is indeed accurate. The raw Wear Leveling Count seems to indicate the amount of exhausted P/E cycles as it's inversely proportional to the normalized WLC value and once it hits 1,000, the WLC will hit zero. 

Note that if Samsung's WLC is anything like Intel's Media Wear Indicator, when the normalized counter value drops to 0 there's still a good amount of endurance actually left on the NAND (it could be as high as another 20 - 30%). At least with Intel drives, the MWI hitting 0 is a suggestion that you may want to think about replacing the drive and not a warning of imminent failure.


1,000 P/E cycles may not sound much but when it's put into perspective, it's still plenty. Client workloads rarely exceed 10GiB of writes per day on average and write amplification should stay within reasonable magnitudes as well:

SSD Lifetime Estimation
NAND MLC—3K P/E Cycles TLC—1K P/E Cycles
NAND Capacity 128GiB 256GiB 128GiB 256GiB
Writes per Day 10GiB 10GiB 10GiB 10GiB
Write Amplification 3x 3x 3x 3x
Total Estimated Lifespan 35.0 years 70.1 years 11.7 years 23.4 years

Of course, if you write 20GiB a day, the estimated lifespan will be halved, although we are still looking at several years. Even with 30GiB of writes a day the 256GiB TLC drive should be sufficient in terms of endurance. Write amplification can also go over 10x if your workload is heavily random write centric, but that is more common in the enterprise side - client workloads are usually much lighter. 

Furthermore, it should be kept in mind that all SMART values that predict lifespan are conservative; it's highly unlikely that your drive will drop dead once the WLC or MWI hits zero. There is a great example at XtremeSystems where a 256GB Samsung SSD 830 is currently at nearly 5,000TiB of writes. Its WLC hit zero at 828TiB of writes, which means its endurance is over five times higher than what the SMART values predicted. That doesn't mean all drives are as durable but especially SSDs from NAND manufacturers (e.g. Intel, Crucial/Micron, Samsung etc.) seem to be more durable than what the SMART values and datasheets indicate, which isn't a surprise given that they can cherry-pick the highest quality NAND chips.

No comments:

Post a Comment