Testing and Validation – The Secret Sauce of True Convergence

In early 2011, a colleague and I published a book called Enterprise Network Testing.  We were both working for the Cisco Performance and Validation Testing Services team.  We never believed the topic would be a gripping page-turner for readers, but we had several customers – working for some of the world’s largest companies – ask us for advice on how to do systems testing in their complex environments.

A few years have passed and now I work at VCE, but the importance of systems testing has not diminished. In fact, I believe that testing is a big part of the “secret sauce” that makes the Vblock System such a successful product. The VCE Release Certification Matrix (RCM) is published regularly to document software versions that have been fully tested and verified for Vblock Systems, and it’s our way of keeping track of all the updates VCE investor partners make to their various products that are included in Vblock Systems.  The RCM is the result of thousands of engineering man-hours and it saves VCE customers time and money while improving availability during the patch management process.

In order to understand why VCE pre-integrates, pre-tests and pre-validates every Vblock System, we need to compare component or software testing to full-system testing and certification.

Component testing makes sure that an individual component works as expected. For example, when the code on a single Cisco Nexus 5500 is upgraded, engineers may ask, “Can I still connect to it?” or “Does it still pass traffic?” These questions are important and valid for component testing.

However, let’s look at how that same validation test would work when we add a second Nexus 5500, a pair of Cisco UCS Fabric Interconnects, some upstream switches, a large number of VLANs and typical IP traffic loads. All of a sudden the testing is much more complex and also much more valid for a real-world use case, especially when you add in the EMC and VMware components that complete a Vblock System, which all need to work together seamlessly.

The VCE RCM validates dozens of different hardware and software components to work together.  This includes the software/firmware of Cisco UCS and all of its parts (CNAs, FIs, BIOS, UCSM, etc.) as well as the Cisco network software, the EMC storage firmware, backup software and hardware, the VMware virtualization software and many other integrations. There are numerous Vblock System configurations that can be manufactured for a customer.

Testing all of the components in this complex system together, and certifying that they will all work together seamlessly, is not an easy task.  We dedicate a lot of hardware, test tools, engineering time and expertise to this endeavor so that our customers don’t have to do it themselves – this is a key differentiator for VCE as it removes what otherwise would be a complex infrastructure maintenance process.

As part of the RCM process, VCE also does testing for integrating new hardware and software features from Cisco, EMC and VMware in order to make our customers more agile when deploying them into their Vblock Systems.

Along with validating that the Vblock System will perform as expected, the Release Certification Matrix can also be used as a troubleshooting tool to increase Vblock System uptime, as described by Greg Lyon in his recent blog post: Beat the Clock.

Today’s IT organizations spend a big part of their budgets and time on “keeping the lights on” in their data centers. Leveraging the work that goes into the VCE RCM can save VCE customers money and time so they can shift their focus away from basic maintenance toward innovation that drives business value for their organizations, and they don’t even need to buy a book about testing to do it.

About the Author: Andy Sholomon