HPE Smart array storage controllers are available with 2 types of caches – FBWC and Smart-Cache. FBWC is a controller-based DDR cache with a flash module that preserves unsaved data, meanwhile, Smart-Cache uses SSDs.
Each HPE Performance RAID controller comes with a super-fast DDR module which acts as a primary cache. The main advantages of this cache are Write-back and Read-ahead caching technologies. Using a Write-back mechanism controller acknowledges the write as soon as it gets to the controller and notifies apps that the write has been accepted and that they can continue. This way application writes can be written even under milliseconds. After write has been written in cache it is offloaded to drive media.
Read-ahead uses different algorithms to guess sequential read data from drives and preloads that data ensuring faster reads.
As mentioned FBWC is included in each Performance series controller. The only requirement is that you need a Smart Cache battery. In case of a failure, a Smart storage battery will provide enough power to write all unsaved data to a non-energy-dependent flash module. FBWC is enabled by default as long as Smart Storage Battery is present.
In cases where only HDDs are present in the system, HDDs might be too slow for cache to offload the data depending on data. Also what could generally improve storage performance with minimal investment?
Smart-Cache comes into play. In comparison to FBWC, Smart-Cache is slower, however, it has massively increased capacity (HPE recommends ~10% of total spindle capacity). It caches writes to SSDs ensuring much faster future writes.
Hot data (cache) is being stored on SSD for accelerated data access, whereas “cold” data is being offloaded to HDDs, as the data isn’t accessed that frequently and when it is the performance hit is much more minor compared to the server being cacheless and data stored to HDDs.
Smart-cache only comes out of the box with P8xx series controllers, however, a license can be purchased to use this functionality on other P controllers, also no Smart Storage battery is needed as data is operated on persistent storage. In all SSD systems, this feature cannot be enabled as there are no slower drives to accelerate.
It can be concluded that FBWC is the primary cache as it is the fastest. It is also the first component that receives and processes data. Afterward when data is received and acknowledged to accelerate writes and reads to HDDs Smart-Cache is being utilized, this combo boosts storage performance quite a bit.
Let’s further evaluate how these caches work and what kind of performance we could expect. Following tests will be conducted:
- Best case scenario – HDD system with both FBWC and SSD Smart-Cache
- HDD system with only FBWC available
- HDD system with only SSD Smart-cache available
- Worst case scenario – HDD system without any cache
- P816i-a controller with 4GB FBWC + HPE Smart Storage Battery
- 2x 800GB HPE SAS 10K HDD as VM storage array
- 2x 120GB HPE SATA SSD RAID-1 as Smart-Cache for VM storage array
For tests we used Iometer with the following settings:
- Block size – 4K, 100% Random, 50-50 Read/Write
In the first test, we have both FBWC and SSD Smart-Cache. This really is the best-case scenario.
As we can see results are pretty impressive. All requests are processed and cached by DDR FBWC, afterward quickly offloaded to Smart-Cache when they finally arrive at HDDs.
As we touched this before – the capacity of FBWC is 4GB. Very high and long load of requests eventually will fill up the cache affecting the performance.
30 minutes after the start of the test FBWC cache is full, SSDs cannot keep up with the amount of data being loaded. However, these figures are much better than without this cache and it perfectly deals with IO bursts.
In the second scenario, we will look at how performance is affected when the controller’s FBWC cache is no longer available. To achieve this we just have to remove the Smart Storage battery from the server, as without the battery the cache is automatically disabled. When FBWC is no longer present all operations go straight to the drives.
The result is similar to the previous one where FBWC was full but with twice the latency. This kind of behavior could be explained by FBWC, being full, still managing to get some of the data to SSDs, therefore having lower latency when processing new data.
Most of the time where this happens when the Smart storage battery fails and FBWC is disabled. It’s important to replace the battery to ensure the best performance.
In the third test, FBWC is enabled back on and Smart-Cache is now removed. All operations go through FBWC and go to HDDs. So we have a super-fast 4GB DDR cache deployed with slow HDDs.
By not using Smart-Cache performance has decreased 10 times with exception of Write latency, this is due to FBWC still being present.
In our final test, we will take a look at an absolute worst-case scenario where absolutely no cache is present.
Performance has been degraded drastically, only about 780IOPS. This however shouldn’t come as surprise as the workload is being tested on only 2 spinning disk drives.
What should be taken into the consideration is high latency numbers. If we go at the beginning of the test # of Outstanding I/O was set to 16. Due to physical limitations, HDDs can only do 1 operation at the same time, anything more than that will just cause even higher latency with minor to no real gain. When Queue depth decreased to 1 we can observe a drop in IO but latency numbers come to the expected range.
When we compare the previous test (FBWC+HDD) we can see that by using FWBC with HDDs it is possible to achieve great Write times and about 3x more IOPS than by going cashless, however, results don’t come even close when all caches are present.
Utilizing FBWC + Smart-Cache really is a great and efficient way how to enhance overall storage performance just by adding a few SSDs. It is no longer needed to tier drives manually and have a small pool of fast storage and a large pool of slow when you can take advantage of both worlds.