Hybrid clouds: Get the best of private and public cloud infrastructures
As the Department of Defense (DoD) proceeds with the Joint Enterprise Defense Infrastructure (JEDI) contract, federal organizations seem to be considering the downsides of relying on one cloud service provider (CSP) for all of their computing and storage services. Potential drawbacks include reduced access to market innovations, increased costs, reduced operational performance, and loss of flexibility to third-party, cloud-based services. Most of these risks result from vendor lock-in to a single cloud infrastructure from which it is difficult to migrate, as well as from a “one-size-fits-all” business model that optimizes the CSP’s costs at scale.
Consequently, many industry professionals have begun advocating for “hybrid cloud” infrastructures. So what exactly is a hybrid cloud? This model can be difficult to conceptualize and challenging to execute. In this blog post, we will clarify what a hybrid cloud is; how it differs from common commercial cloud offerings; the benefits of choosing a hybrid cloud for your organization; and finally, some of the challenges you should be aware of if you decide to pursue a hybrid cloud model.
What do we mean by “hybrid cloud”?
To answer this question, we will rely on the National Institutes of Science and Technology (NIST) standards, particularly NIST Special Publication 800-145, The NIST Definition of Cloud Computing, and NIST Special Publication 500-323, The NIST Cloud Federation Reference Architecture. The NIST standards allow for clear and accepted definitions of these wide-ranging technologies, which can help alleviate confusion for those considering cloud computing options. Let’s start with a baseline understanding of cloud computing.
What is “Cloud Computing”?
Per NIST SP 800-145, “Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.” Cloud infrastructures have allowed IT professionals to provide computational and storage services to their organizations while reducing the capital needed to provide those capabilities. The most common and familiar model of cloud computing is the commercial cloud, such as those provided by Amazon, Microsoft, and Google. Instead of maintaining a large, on-premise IT infrastructure, IT departments can instead lease those capabilities from commercial cloud providers, reducing their capital costs and maintenance burden while increasing the speed at which computing resources can scale to meet changes in user requirements. Because these clouds are leased, not owned, by the customer, the cloud is considered a service. The result has been the proliferation of cloud-based offerings to the marketplace, each offering their own version of Infrastructure-as-a-Service (IaaS), Platform-as-a-Service (PaaS), and Software-as-a-Service (SaaS), along with an ever-expanding catalogue that makes good on the term, Anything-as-a-Service (XaaS). This represents a fundamental change to the way the government has traditionally approached the delivery of software. The question is not if, but how quickly government organizations adopt XaaS solutions in their ongoing modernization efforts..
What is a “Private Cloud”?
The NIST SP 800-145 does not indicate where a cloud actually resides. The dominant model most are familiar with is the commercial cloud model, but an organization can elect to build their own cloud infrastructure as a private cloud. A private cloud model still provides “ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources”, but also allows the organization to retain complete control over its computing and data storage services since those are still resident within the organization. Private clouds allow the organization to take advantage of automated network capabilities, CPU/GPU virtualization, container orchestration, and microservice infrastructure while keeping all data completely in-house. Private clouds are typically a more cost effective option for organizations compared to traditional IT infrastructures or purely leased services. However, private clouds can also be limited in scale to the capital investment by the organization and still require native maintenance of the servers on which they reside.
What is a “Hybrid Cloud”?
Hybrid clouds are a specific use case of a general federated, multi-cloud configuration where a private and public cloud are networked to allow an organization to operate on a private cloud while “bursting” to public clouds for specific operations. Bursting can include accessing additional computing capabilities to meet usage spikes, accessing temporary data storage for collaboration outside of the organization’s network, or accessing more specialized resources, like high performance computing (HPC) resources, that are not resident in-house. Consequently, a hybrid cloud delivers the advantages of cost, control, and optimization of a private cloud while also leveraging the scalability and collaborative promise of commercial clouds. They can also provide a measure of protection against vendor lock-in and its associated risks of cost escalation, tech stagnation, and dependence on a curated vendor pool of third-party services since the users are not completely dependent upon a single CSP.
Most importantly for federal clients, hybrid clouds can provide necessary data sequestration and security to ensure data ownership is protected among different stakeholders within an organization. Dr. Rion Dooley, Director of Platform Services at DMC, notes that a hybrid cloud “lets you physically isolate your data in a way standard DMZ cannot; lets you isolate your networks in a way standard SDN cannot; and lets you leverage the advanced compliance certifications needed to handle sensitive data, all while allowing you to keep running your sensitive operations on premise. When you compare that to the cost of rolling out comparable in-house solutions, it can save a ton of time and money. It’s one way small businesses can really punch above their weight.”
Dr. Dooley sums up the benefits of hybrid clouds, noting they are not simply a multi-cloud model. A hybrid cloud model is most valuable because of its bespoke specialization for the organization’s needs. “Hybrid clouds are an opportunity to build capability more so than just capacity. I enjoy working with hybrid cloud architectures, because I can strategically leverage their differences to accomplish the goals of our clients.” He points out that a large CSP is going to be optimized for up-time, elasticity, and raw capacity, but he relies on hybrid clouds for “bespoke, managed environments tuned to the unique needs of our clients with combinations of leading-edge, bleeding-edge, and even Edge-with-a-capital-E technologies in a way no enterprise would dream of attempting.”
Deploying a Hybrid Cloud
Given how promising the hybrid cloud model appears, what should decision-makers be concerned about if they choose to pursue this path? Here are some thoughts from DMC’s cadre of technologists on what IT professionals should be cognizant of when considering the hybrid cloud option:
Dr. Chris Monson, Chief Technology Officer for DMC, cautions that hybrid clouds, in practice, often provide users the “least common denominator of service”. IT professionals still need to engage and understand the computing requirements of their organizations and be able to access external cloud resources smartly. Having a hybrid cloud is often not enough. As he puts it, organizations still need “a group of folks that can constantly advise them on which cloud is best for particular tasks, and help them to ensure that using different clouds for different things (including on-prem) doesn’t give them massive maintenance headaches.” Often, hybrid clouds are not tooled to let their organizations take advantage of special capabilities and can end up creating unintended multi-cloud maintenance hassles. Identity Access Management (IAM) services are a good example. A poorly done hybrid cloud can proliferate IAM management (e.g. four IAM services to manage operation across four clouds).
Dr. Brian Dennis, Director of Data Engineering at DMC, is careful to consider the data that organization needs to use and in what manner. “Data gravity is a significant challenge. At a certain amount of data, let’s say 1Tb as a flyer, there has to be serious strategizing about where data lives, how to get it staged in all the right places, access control, performance tradeoffs, etc. It starts to have a big impact on agility and elasticity.”
Dr. Martial Michel, Chief Scientific Officer at DMC and co-author of the NIST SP500-332, cautions that data access and Identity Federation need to be assured when using a hybrid cloud solution reflecting the intricacies of Cloud Federation. DMC has contributed to the understanding and the determination of those in NIST SP500-332, "The NIST Cloud Federation Reference Architecture", and continues this work with the Institute of Electrical and Electronics Engineers (IEEE) 's P2302 effort.
Deploying a hybrid cloud is no small task, and there are various concerns to take into consideration before jumping in. Dr. Dooley points out a few of these issues. “Watch for infrastructure fragmentation, which can lead to vendor lock-in. Often organizations move a couple apps to the public cloud, leave the rest on-premise, and wind up not being able to easily roll back out. Is there an exit path defined and understood by all stakeholders once an application is deployed across the hybrid cloud? Is the technical capacity of current IT and developer staff up to the additional challenge of deploying this model? Sometimes, a hybrid cloud can be confused with multi-cloud models. Make sure you are getting what you want. Hybrid clouds can impose new requirements on existing application architecture and performance, so be sure that these are understood prior to implementation. Before deploying a hybrid cloud, teams typically must learn a new suite of tooling and establish a new security model to ensure things run appropriately. Invest in the necessary technical skill needed to make the transition, and maintain it over time either through training your existing workforce, or hiring the necessary talent.”
As the IT universe continues to expand with cloud-based XaaS delivery and consumption models, hybrid clouds will likely see increased deployment by organizations seeking the best performance and return on investment from their IT efforts. Specific use cases will continue to drive the model’s adoption, but there are growing pains to hybrid cloud deployment that practitioners need to consider carefully before execution. DMC is a leading provider of bespoke cloud infrastructures for advanced data science applications. We hope our experiences can help your organization’s cloud journey. Please contact us if we can be of further help.
