We have certainly come a long way since the 12-Factors declared that backing services should be attachable or connectable and make no distinction between local and remote resources. Famously a great meme sprung up sometime later declaring that stateless apps were a hoax. Stateful apps are of course the most popular, and arguably the most valuable to one’s business, so I always find it odd when people talk about the power of containers without mentioning the footnote that stateful apps are difficult to operationalize. After all, the eco-system supporting these apps is currently splintered between Docker, Mesos and Kubernetes. Now, with the advent of the Container Storage Interface (CSI), we as a community are on the verge of solving this discrepancy.
The Container Storage Interface (CSI) – a Primer
The Container Storage Interface (CSI) is a universal storage interface (effectively an API) between container orchestrators and storage providers that will allow consistent interoperability between the two to help drive container adoption. Container orchestrators who adopt CSI can basically leverage any storage provider, cloud or otherwise. CSI enables the storage providers to provide storage services to any container orchestrator that implements the specification as well as a diverse range of storage services for any container orchestrator. The current fragmentation introduced by different interfaces maintained by different platforms is then eliminated. For the user, this means that one can count on a consistent and reliable user experience regardless of which storage provider is leveraged.
CSI – What’s New
Two new CSI drivers have been released for Dell EMC ScaleIO (csi-scaleio) and vSphere (csi-vsphere) bringing early support to a couple of key on-premise platforms. The ScaleIO driver facilitates container environments to leverage ScaleIO high performance software-defined storage services for bare metal environments, thus enabling container adoption and easing the path to leveraging containers for persistent applications. The vSphere driver implementation enables stateful cloud native applications on all vSphere supported storage vendor ecosystem including VMware vSAN – VMware’s HCI offering. In the near future, VMware’s Project Hatchway plans support for all leading Container Orchestrators by leveraging this CSI driver.
This is on top of the integration into the container platforms as CSI has already garnered support from two of the four major containerized platforms (Kubernetes and Mesosphere DC/OS) and work is also underway for the third (Cloud Foundry). Kubernetes, with contributions from {code} and others, has support for CSI in their 1.9 release. Mesosphere’s DC/OS is developing support for CSI with a planned early release at the end of 2017.
But, an end-to-end solution won’t be complete without storage platform drivers. Three more general CSI storage plugins (csi-nfs, csi-blockdevices, csi-vfs) were released in the DockerCon EU timeframe. This represented the first three native CSI storage drivers and opened the market for more storage platform support, while also functioning as a blueprint for contributors looking to create CSI drivers. With the newest CSI plugins, there are now 5 CSI plugins plus 12 supported storage plugins from REX-Ray.
There is still plenty more work to be done. For example, support for CSI in Go, called GoCSI, has provided the basis for development for some of the first native CSI drivers. These kinds of tools and frameworks make implementations much simpler and tend to help with adoption. Similar support for other languages like Python and even Java would further its adoption.
With supporting implementations on both the storage and container orchestrator side, the CSI specification is pre-1.0 and still a work in progress thus feedback from others implementing it on both sides is required if CSI is going to be a sustainable solution moving forward.
CSI Needs You
The easy part has been getting the COs on board since there are relatively few COs compared to the large number of various storage services. That being said, it is important that storage vendors take note of these developments and have a plan to build CSI drivers and support CSI with their storage products if they wish to be relevant to the cloud-native user of the future.
Where You Come In
As an individual your awareness and support of this project is important to its success. CSI is on the right track, but it needs everyone in the community and industry to dispel the myth that stateful applications in containers isn’t a valid architecture.
The community also needs people to deploy and test the CSI architecture in real environments to ensure it supports popular use cases. After all, if it doesn’t meet the needs of end users what’s the point?
What storage drivers will be important to you? What are you using in your environment? CSI activities are documented in a Google group, there are Community Sync calls, and information from the calls is published in the group for anyone to read. Join in these activities, this is your opportunity to make your voice heard.
CSI – In Summary
With shifts towards microservices and cloud native architectures, I do believe adopting and operating these new architectures being defined in CSI will be part of your strategy. This is something that is materially different, something that is truly an improvement over the past, and solves the stateful storage problem in an elegant way. This is not only an important step forward, but required as you rely on access to data while working in your container environment.
Reading Resources
- The Container Storage Interface – according to Josh by Josh Bernstein
https://blog.thecodeteam.com/2017/08/15/container-storage-interface-according-josh/
- Understanding the Container Storage Interface Project by Josh Bernstein
https://www.delltechnologies.com/en-us/blog/understanding-the-container-storage-interface-project/
- Analysis of the CSI Spec by Kendrick Coleman
https://blog.thecodeteam.com/2017/11/03/analysis-csi-spec/
- Docker Compatibility to Container Storage Interface (CSI) Plugins – New Features in REX-Ray 0.11.0 by Kendrick Coleman
- REX-Ray v0.10 Delivers CSI Validation, 3 Additional Drivers, and More Enhancements by Kendrick Coleman
- gRPC: A Framework for Efficient Service Architectures by Vladimir Vivien
https://blog.thecodeteam.com/2017/09/08/grpc-framework-efficient-service-architectures/
_______________________________________________________________________
The opinions of Dell and Dell authors are those of Dell only. Our linking to third-party content does not imply our endorsement or sponsorship of this content nor does it imply endorsement of sponsorship of Dell by the authors or hosts of this content. Dell’s Digital Millennium Copyright Act Notice can be found at DMCA.
