Today, EMC announced general availability of the XtremIO (aka Project X) – all-flash storage array, redefining what’s possible with all-flash array storage.
Flash is supposed to be fast, but with XtremIO customers get much more than just speed – XtremIO sets the bar higher in the flash array market. Customers seem to agree, with already ~10PB of effective deduplicated capacity purchased as part of the Directed Availability program (which we announced in March 2013).
The XtremIO team has been working toward today’s announcement since founding the company in 2009. Listen to Ehud Rokach, XtremIO’s General Manager and one of the original co-founders of the company, and me discussing what it means for customers.
There are four architectural pillars that are totally unique to XtremIO.
1. Content-Based Data Placement: XtremIO is a scale-out, multi-controller design, capable of delivering linear performance increases as capacity in the array grows. One of our fundamental design tenants was to keep the array “inherently balanced” and optimized to within a fraction of a percent across all SSDs and array controllers. In order to balance across a multitude of controllers, and avoid hot spotting issues and back-end rebalancing processes, we developed a new way of determining where to place data by looking at the data itself with a sophisticated content fingerprinting process. This not only delivers better performance, better flash endurance, and makes the array easier to use; it has the secondary benefit of telling the array what it needs to know to deduplicate data inline in the data path at full speed. This type of algorithm would have been impossible to implement just a few years ago without today’s Remote Direct Memory Access cluster interconnect fabrics and it simply wouldn’t work with high performance on anything but an all-flash array – like XtremIO.
2. Dual-Stage Metadata Engine: Any modern storage array virtualizes its internal addresses from the external addresses used by hosts. XtremIO took this one step further: We built a dual-stage internal metadata engine to virtualize the inside of the array. This lets the array maintain the validity of the external host to content-based address, while maintaining flexibility of the content-based address to physical array address. This freedom of data placement is a key enabler for XDP, our next technology pillar.
3. XtremIO Data Protection (XDP): Other flash arrays in the market use standard disk-based RAID algorithms. XtremIO leverages both the random access nature of flash and the unique XtremIO dual-stage metadata engine to deliver much lower capacity overhead, better data protection and much better flash endurance and performance compared with RAID. Perhaps more importantly, XDP is optimized for long-term operating conditions where overwriting existing data becomes dominant in the array. XDP is designed to allow XtremIO arrays to maintain performance until they are completely full, thus driving better economics out of flash.
4. In-Memory Metadata: Storage systems have always maintained a portion of their metadata in controller memory. This can lead to inconsistent and unpredictable performance. XtremIO delivers consistent and predictable performance because all metadata is stored in-memory. XtremIO arrays are designed to be impervious to changes in workload. It doesn’t matter what LUN sizes are used, whether there are random or sequential access patterns, or if there is locality of reference or not. But perhaps best of all, metadata heavy operations (such as inline deduplication, thin provisioning allocations and internal array copy operations) are conducted entirely in-memory—at high speed and without impacting I/O to the SSD portion of the array.
And here is what one of our customers, CMA Consulting Services, had to say about their experience with XtremIO…
Needless to say, we are very excited about the possibilities XtremIO presents. In working with customers for over two years now, my favorite moments are watching their eyes get big as they realize what XtremIO can do. With XtremIO now available and at “EMC scale,” we anticipate those wide eyes and smiles to spread even faster.