According to numerous studies, there is a rise in the adoption of containers for cloud-based service deployments and development, which can only mean that containers are the next big thing in infrastructure virtualization. A customer study by CNCF (Cloud Native Computing Foundation) stated that over 70% of customers ran containerized applications in production, while the rest said they planned to do so in the future. The rise and popularity of containers is due to the significant benefits containerization offers, which include efficient utilization of resources, infrastructure elasticity, improved scalability and productivity, easier deployment and configuration, continuous integration, and application portability.
Meanwhile, microservices architecture has evolved
While containers are on the rise, application architecture has evolved over the past couple of decades from service-oriented architecture to microservices architecture. Applications in microservices architecture are independently deployable and scalable, built as inter-communicating suites of services. An application that’s based on microservices architecture is designed with a number of key things in mind: decentralized governance and data management, infrastructure automation, and design for sustaining failure and extensibility.
The connection between microservices architecture and containers
The two are clearly linked and not by chance. Containerization creates clear boundaries between microservices without implying that an application has a microservices architecture. However, applications can be containerized even if they are monolithic, making the container a monolith as well. This approach does have a downside: since the changing cycles of an application’s components are linked, it means that any change made to a component requires everything else to be updated as well. Doing this once or twice in an application’s lifetime may be manageable, but in reality, it becomes difficult to maintain good modular structure over time, and makes scaling inefficient, if not downright impossible.
So what’s the connection between microservices architecture and containers all about? When it comes down to it, enterprises and developers can’t enjoy the full benefits of containerization without microservices architecture, which adds its own benefits to the mix. When combined, the two form an ideal and comprehensive approach to application modernization.
Managing data & storage
Initially, the adoption of containers was driven mostly by stateless applications. These containers had microservices operating as the front end, and a stateful, non-containerized back end. Moving to a fully container-based infrastructure requires applications (stateless and stateful) to be implemented as containers, which means that there are challenges involving the management of data and storage in these environments to overcome.
These days, stateful applications in containerized production deployments aren’t stored and managed exclusively in one way. Some rely on siloed storage devices that are external and not an integral part of the environment. Despite that, they are mature and provide extensive capabilities for data management, including live tiering, erasure coding, disaster recovery and data reduction, to name a few. In order to truly simplify management, reduce costs, improve resource utilization and benefit from what containerization has to offer, the compute side of containerized applications must coexist with the storage infrastructure in the same environment.
Making this a reality requires a new approach.
A new approach to storage architecture
When it comes to storage architecture, the Holy Grail is separating between the control and data planes. This allows independent scaling of metadata (control plane) and data (data plane) flows and data management operations, including data mobility, tiering, and snapshots. To date, storage implementations haven’t achieved effective separation of the two planes. Numerous things stand in the way, such as bolt-on incremental features, nonoptimal data[RA1] , and cumbersome features.
Now’s the time for things to change, as the emergence of containers and microservices brings with it the opportunity to start fresh in the storage world.
Microservices: the magic
The principles of microservices architecture can be applied naturally to the container-native storage system design. When the control and data planes are separated, a microservices-based, container-native solution has distinct entities of control and data services that scale and provide services in a distributed, scalable way. When a storage system is implemented using microservices, it doesn’t just enable the separation of the two planes – it actually forces it.
Here’s what happens when using a microservices-based container-native storage system that separates the two planes effectively:
- Performance & capacity scaling:
Scaling is provided on a number of axes, including capacity, bandwidth, and IOPs. This allows scaling up/down capacity and performance as needed, meaning resources can be scaled and shared efficiently across applications.
On their own, microservices can fail to restart. When using a combined solution with container orchestration, resilience improves.
- Data management:
A metadata microservice can carry out many data management operations on its own without affecting the data plane. When data does require manipulation, performance issues can be minimized by decoupling operations on the metadata and data – which also increases efficiency.
- Storage media support:
A microservices/container system could implement multiple flavors of the data-plane microservice, because microservices are independent and communicate with a well-defined protocol.
Tiering operations between media types can be controlled for optimized costs and optimal data layout.
- Data mobility:
Objects (e.g., files, volumes, etc.) can be virtualized as metadata-only objects, referring to a common data pool across numerous types of media and geographies. This results in new data mobility capabilities in hybrid and multi-cloud deployments.
- Storage protocol & application support:
When an application is in the front end as a microservice, it can also be implemented in a number of flavors. This results in greater flexibility, as it supports varying storage-access protocols and application-specific access.
While the benefits of microservices-based container-native storage system include portability, flexibility, and scalability, it’s important to also be aware of the issues that may arise. One example is maintaining strong consistency, which can be very difficult for distributed systems that need to deliver performance. Of course, this and other challenges are difficult but not impossible to solve, and shouldn’t stand in the way of pursuing a microservices-based architecture. The benefits are simply too significant to ignore.
Over the past two decades, containers and microservices have evolved, as more applications are implemented as cloud-native apps. Improved flexibility and extensibility help create solutions that today’s modern applications require, without the typical limitations of infrastructure. Embracing a microservices approach is needed to remove the limitations of storage solutions and realize the potential containers offer for application modernization.