If you attended last year’s VMworld keynote by Steve Herrod or watched the online broadcasting, you may still recall the code2cloud.com website (see the top banner here). That was a very simple Web application meant for the keynote attendees to submit names and email addresses to win a chance to go to the backstage with the Foreigner band. The website was hosted at Terramark vCloud and continued to run for about one month afterward.
Provide a mechanism to fast provision virtual machines (VMs) and manage their lifecycles by maintaining a pool of virtual machines.
Virtual machines can be expensive to create. It takes several minutes to create a new virtual machine. Technologies like linked clone and storage offloading can help speed up the process, but it still takes time. And these alternative approaches, in some use cases, do not help when you need instant provisioning.
It’s generally a good practice to pool resources that are expensive to create. In programming, you pool threads and check them out on demand. When it’s done, you check them in back to the pool. This is what most Web Servers do for high performance.
You can leverage the same idea for VM provisioning. You create new virtual machines and put them into a pool. When there is a new request, you just check out one virtual machine from the pool. The following diagram shows how it works.
Several weeks ago, Christopher Wells sent me an email for permission for translating my blog into Japanese so that more people in Japan can benefit. Today he told me he had translated one of my posts into Japanese and posted it on his blog site. Here is the first paragraph in Japanese version:
オペレーティング･システム（OS）はソフトウェアの一部で、コンピュータのハードウェアを管理し、様々なアプリケーション用の一般サービスを提供しま す。クラウド・コンピューティングの台頭にともなって、OSがまだ関連するのか、また、将来のクラウドにおいてOSがどんな役割を果たすのか疑問に思う人 もいるでしょう。
Now a little quiz: which original article did Chris translate? I will leave it to you to guess out. Hint: here is the link to the full article.
In my last blog, I wrote about pattern idea from the famous books “Design Pattern” and “A Pattern Language” and how it can be applied to cloud architecture design. Below and in later posts in this series I shall follow the content outline used there to illustrate the cloud architecture patterns.
Provide a standard way to create new virtual machines based on user requirements.
There are enormous combinations of virtual machines with different operating systems, middleware, and applications. And then we have user data to the mix! We need a standard way to create new virtual machines.
General speaking, there are three basic ways to create new virtual machines:
Design patterns have been very popular among software developers since the book by Gang of Four (Enrich Gamma, Richard Helm, Ralph Johnson, John Vlissides) in 1995. If you go to an interview for a software engineering position today, the chances are you most likely get one or more questions on design patterns.
Like many concepts in software that from other disciplines, the “pattern” idea was “borrowed” from “A Pattern Language,” a book by Christopher Alexander on architectural patterns published 20 years before we started to talk about software design pattern. His book turns out to be a great way to summarize and document reusable design elements for different engineering works.
As Christopher points out, “Each pattern describes a problem which occurs over and over again in our environment, and then described the core of the solution to that problem, in such a way you can use the solution a million times over, without ever doing it the same way twice.”
Today I “borrow” the same idea and apply the concept to cloud computing on architecture designs. The main focus is on virtual machine-based architecture patterns so that you can best leverage virtual machines for your cloud computing.
What is a Cloud Architecture Pattern?
An architectural pattern extracts the common and re-usable design concepts and components. If we put it in the big picture, it should be somewhere below the overall system architecture and above software design as the middle layer.
An operating system (OS) is a piece of software. It manages the computer hardware and provides common services for various applications. With the rise of cloud computing, people may wonder whether the OS is still relevant and what role it will play in the future cloud.
Key Components of OS
There are different flavors of operating systems: from real-time OS, desktop OS, all the way to a mainframe OS. The most recent OS is the Cloud OS.
In general, every OS has these common components:
- The kernel, which manages memory, processes, etc.
- Device drivers, which drive different hardware from different vendors.
- User interfaces, including command line shell and Window system.
- File system, which provides a hierarchical way to persist data.
- Security, which authenticates users and protects information.
Depending on the type of OS, you may miss something here or have something extra. For example, an embedded OS may not have a user interface and everything is controlled remotely. For the desktop OS, you may have extra commonly used applications such as a calculator, a calendar, a browser, and so on.
With the gradual adoption of cloud computing, more companies are coming to the cloud service business. With so many new players in the game, how can you stand out in the competition as a service provider? In a marketplace already dominated by companies like Amazon, do you even have a chance?
Yes, you do! Differentiate.
I recently read a book, “Differentiate or Die: Survival in Our Era of Killer Competition” by Jack Trout and Steve Rivkin. Although it mainly talks about differentiation ideas in traditional businesses, it does offer good insights that can be applied in the cloud service business as well.
Before we get to cloud services, let us take a look at what Jack and Steve say about differentiation. They first discuss some commonly held misconceptions about differentiating ideas:
Thanks all for making this happen! The momentum will continue…
The “last mile” or “last kilometer” is a term in the networking industry describing “the close to end connectivity from a communication service provider to a customer”. Although your infrastructure like backbones is very powerful, your end user experience could suffer if your last mile is not there yet.
For cloud computing, we’ve talked a lot about the data centers, backend servers. What about the end users? Your cloud data center could be very powerful too, but does it mean your users will fully leverage that power? Not necessarily. It depends on how you deliver the service to them.
Because cloud service is delivered with traditional network, the traditional network “last mile” issue is there as well. You surely need a good, if not better, connection to the network.
Beyond the connectivity, you will need good interfaces for your users to interact with the cloud. Let me go over the “last inch” options here.
VMware released the long-awaited vCloud API at VMworld 2010. The API is based on REST with 75 URLs defined in the user related part as you would find in the vCloud API Specification and vCloud API Programming Guide.
I am an OO guy (I am sure many of you are as well), and find it difficult to go through the 75 URLs and numerous XML tags as either input or return. These URLs are like trees in a forest. But where is the forest?
So I decided to create and show you a UML diagram (shown below) so that you can easily capture the key concepts of the vCloud API. In fact, there was a similar diagram in the programming guide of version 0.8.
Technology can be a lot like fashion, with quickly shifting trends. Once we embraced big iron but after the mainframe age the industry went into the client/server age where we soon found too many servers to manage. So we consolidated them, not back to mainframe age, but onto hypervisors. With one physical server, you can run multiple virtual machines.
Server consolidation solved a big problem and resulted in big cost savings. From management’s point of view, however, it does not actually reduce the number of servers to manage in your enterprise. To some extent, it worsens the problem! In some circumstances it’s so easy and inexpensive to create a new virtual machine that you end up with many more servers than you really want or can effectively manage. This problem not only exists in private clouds, but also in the public cloud.
According to VMware CEO Paul Maritz in his keynote at VMworld 2010, the number of virtual machines exceeded physical machines in 2009, and will reach 10 million by the end of this year. This is definitely great news for the virtualization software industry but also a challenge moving forward.
So how should you try to solve the problem of virtual machine sprawl or even better, prevent it from happening? I discuss some solutions one by one here.
Many people already know the book “The Tipping Point: How Little Things Can Make a Big Difference.” According to the author Malcolm Gladwell, tipping points are “the levels at which the momentum for change becomes unstoppable.” He defines the term as sociological and uses it to explain sociological epidemics.
Three Rules of Epidemics
In his book, Gladwell laid out the “three rules of epidemics” as follows:
1) The Law of the Few.
“The success of any kind of social epidemic is heavily dependent on the involvement of people with a particular and rare set of social gifts.” The author categorized people into Connectors who link us up with the world; Mavens who are “people we rely upon to connect us with new information;” and Salesmen who are charismatic persuaders.
2) The Stickiness Factor
The specific content of a message that renders its impact memorable.
3) The Power of Context
Human behavior is sensitive to and strongly influenced by its environment.
Although the research comes from sociology, I think it applies to technology as well. After all, technology is social. Just think about social networks like Facebook, and the recent success of Apple’s iPad.
If you want your technology to be a huge success, you cannot ignore its social side story. In the end, it is human beings who make decisions regarding any technology adoption or product purchase.
As more and more clouds go live, it’s time to think about how they will need to interconnect and interact. InterCloud is a new terminology coined for cloud computing after Internet for networking.
Vint Cerf, the “father” of the Internet, said recently that the cloud is much like networking in 1973 when computer networks couldn’t connect or interact. He called for open standards for cloud computing so that InterCloud can become a reality.
It’s hard to design standards when people are still trying to reach a consensus on defining what a cloud is in the first place! The good news is that as an industry we went through a similar process for the Internet. So we can learn from that experience.
The idea is simple: look at basic building blocks we have for the Internet and think about their equivalent for the InterCloud. Believe it or not, InterCloud and Internet share many common characteristics. The following table summarizes some of these.
IBM recently announced its re-organization around its software and hardware business units. The previously separate business units were merged together as one – the Systems and Software Group led by the former software chief Steve Mills.
You may recall that IBM did not have a dedicated software group until Lou Gerstner created one 15 years ago to centralize all the software businesses into one business unit. This unit has been IBM’s most profitable business. Before that, IBM offered all the software as add-ons to the systems like 390 and AS/400.
Now can we expect IBM to offer hardware systems as add-ons to their software solutions?
Although companies constantly re-organize to streamline their business execution, this reorganization did indicate a big trend is happening in the IT industry. Computer vendors are striving to own vertically-complete stacks: from hardware all the way up to business applications.
When you think of portability in cloud computing, you think of how to move applications code, data, and workloads. These are mostly horizontal movements within the same level of software stacks – from one IaaS to another, and from one PaaS to another.
There is a more interesting and potentially very important movement that I would describe as “cross stack” portability. Today we don’t see cross stack portability unless we re-write the application, which is not what I cover here (although it could be a good business opportunity for companies to explore). Rather, I am talking about how to move your application built on PaaS to an IaaS vendor or even to a private cloud. The reason I call it cross stack is because the application is moved up or down to a different level in the software stack.
In this blog, I’ll focus on portability without code change. I’ll discuss three conversions: from PaaS to IaaS, SaaS to IaaS, and IaaS to PaaS. Mathematically we can have other forms of conversions – say from IaaS to SaaS – but those examples are either not that interesting or not that practical. So I won’t cover them here.
From PaaS to IaaS
In my last blog, I discussed how to optimize workloads across the cloud. This is based on the assumption that you already have an existing infrastructure. What if you don’t have an existing cloud infrastructure but would like to design one from scratch? Here is what you should be thinking about to get the most from your new cloud.
But first let’s take a look at other types of infrastructures – say a road. When you design a new road, you have to collect data such as population densities around the area, people’s working schedules, what types of vehicles will run on the road, and so on. With that information, you can decide how many lanes you want, what kind of road surface is required, and so on. You don’t just make up the design specification from scratch, and lay down an eight-lane freeway everywhere.
The same process applies in designing the cloud infrastructure as well. Unfortunately this is not what we see often today.
In my previous blog , I said infrastructure is a means and application is the end. We need to drive the design cloud architecture from the application perspective. This is what I call the top-down approach.
Cloud computing hasn’t changed the nature of computing – it just changed provisioning and management. That’s important to remember because workloads in the cloud are very much similar to what we see in traditional computing infrastructures. To get the most out of your investment in cloud services or in your own physical IT infrastructure, you need to understand how to optimize workloads.
Typical computing workloads involve four basic parts: computation, memory, networking, and storage. Almost all applications have these four parts but mostly not balanced.
Now let’s quickly review the essential categories of application workloads:
In my last post, I discussed when not to use cloud services. Basically you should avoid the cloud for your organization’s core competency IT systems. Remember, cloud computing is not a silver bullet for everything.
Today I want to share the stories from the other side: when you should use cloud services. As a rule of thumb, you use cloud services for your non-core competency IT systems. But, what are the typical non-core competency systems?
There could be many cases in which you can use cloud services. Let me go through some of them by sharing customer experiences:
Outsourcing projects. If something is outsourced, most likely you don’t think it’s a core competency to your business. You can then leverage the full benefit that public cloud services bring to you. You can easily have workspace that is accessible by both your employees and contractors, and it’s more secure than opening up your own infrastructure to your contractors.
During the July 4th long weekend, I got the chance to read the book “Delivering Happiness” by Tony Hsieh. It’s a great book with many great ideas and lessons he learned from LinkExchange and Zappos.
So, how does this relate to cloud computing?
Here’s what Tony wrote…
“It was a valuable lesson. We learned that we should never outsource our core competency. As an e-commerce company, we should have considered warehousing to be our core competency from the beginning. Outsourcing that to a third party and trusting that they would care about our customers as much as we would was one of our biggest mistakes. If we hadn’t reacted quickly, it would have eventually destroyed Zappos.”
In this paragraph Tony summarized the lesson from contracting eLogistics for inventory services in Kentucky, which turned out to be a mess and almost killed Zappos when cash flow became a big issue.
From a business perspective, cloud services are not much different from the inventory services. Both are all about outsourcing. The high tech nature of cloud doesn’t change the business nature of cloud services. What happened to Zappos could potentially happen to any cloud customers.
With the right system configuration in place, it’s time to install the applications. So why not use the same tools we used for the OS and middleware? Do we need yet another set of tools?”
It depends. You can use the same set of tools for middleware to install some applications. The middleware appears like an application to the OS as well. The difference is whether your application is stable enough and whether you need to customize per node. The tools like Puppet can be good for stable applications that can be deployed the same way across all nodes. If your application is still a work in progress and you need flexibility to tweak it, you need more specialized application provisioning tools.
The big technical difference between application and middleware provisioning tools is that application tools push the application to the nodes and remotely change anything as needed. The process is procedural.
The middleware provisioning tools normally have agents on the nodes to pull the software based on the prescribed configuration files. The process is declarative.
Beyond the “push” and “pull” difference, the application provisioning tools can also manage the lifecycles of applications (sometimes called services) distributed on different nodes with a single line of command or code. Given the nature of remote command dispatching framework, the application provisioning tool can do almost anything. If there has to be a limitation, it’s your imagination.
So if you develop applications by yourself, you most likely need application provisioning tools.
Let’s see what tools are there.