Cloud Architecture Patterns: VM Pool
Provide a mechanism to fast provision virtual machines (VMs) and manage their lifecycles by maintaining a pool of virtual machines.
Virtual machines can be expensive to create. It takes several minutes to create a new virtual machine. Technologies like linked clone and storage offloading can help speed up the process, but it still takes time. And these alternative approaches, in some use cases, do not help when you need instant provisioning.
It’s generally a good practice to pool resources that are expensive to create. In programming, you pool threads and check them out on demand. When it’s done, you check them in back to the pool. This is what most Web Servers do for high performance.
You can leverage the same idea for VM provisioning. You create new virtual machines and put them into a pool. When there is a new request, you just check out one virtual machine from the pool. The following diagram shows how it works.
Notice that there are two participants here:
1) VM pool manager which checks in/out VMs, and indexes the content, and searches the available VMs
2) VMs that are parked in the pool, ready to be checked out.
Depending on how you use the pool, you can keep VMs powered on and ready to work at any time. Alternatively, you can keep the VMs powered off and power them on after being checked out. The difference is that the latter approach saves energy but takes longer. It’s a tradeoff you have to make in your projects.
To create the checked out virtual machine for your purpose, you may have extra work to do. For example, you may still have to to change its IP address, register with a load balancer, etc.
There are several decision points for a VM pool in real projects. They are mostly trade-offs you have to make in the design process.
Fixed size or dynamic?
A VM pool keeps a certain amount of virtual machines. To check out, the algorithm can be either first-in-first-out (FIFO) or first-in-last-out (FILO). It doesn’t matter for the users as long as you keep a reasonable number of VMs available. In either case, the pool manager has to track the VMs parked there.
The number of VMs could be fixed or dynamic. If it’s fixed, you create a new VM and check it in upon one being checked out. Otherwise, you let it be, or optionally create a new one when the number drops to a predefined minimum.
New or Recycle?
For checking in new virtual machines, there could be two sources. One source is a VM factory where you create new ones. The other source is to “recycle” the virtual machines no longer needed. The latter may or may not be feasible depending on how “dirty” the virtual machine is after being used. You may need to run scripts or shut VMs for cleaning up before being checked in even if they are “recyclable. “
Single pool or Multi pools?
If you will need one type of virtual machine, you just need a single pool for easy management and efficiency. But if you need multiple types of virtual machines, you may need multiple VM pools, each of which holds one type of virtual machine.
For simplicity, you may want to stick with single pool approach. If your VMs have a good denominator, you can create a VM template based on that and pool it. The work then becomes installing additional software packages on the fly. It may or may not save you time depending on the complexity of the VMs.
When you want fast or instant provisioning, you end up using extra resources like disk spaces for the virtual machines parked in the pool. The amount of resources depends on the size of the pool. And if you want VMs hot standby or not, you may use extra power for the powered on VMs that are just waiting.
By understanding your priorities, you can tune the parameters and minimize these costs to a reasonable –but still good enough level for your projects.
In 2009, I got a cloud demo project for VMworld keynotes. It’s obvious that you cannot make your audience wait for three minutes after each command is issued.
To handle this, I designed a very simple VM pool which holds pre-created VMs. These VMs are on hot-standby in the pool. After the deployment command is issued, a new VM is checked out and injected with new code. In several seconds, you see the newly deployed application up and running.
At the same time, a backend process is kicked off to create a new virtual machine for future usage. It still takes about three minutes in total at backend, but no one cares because the application itself is up and running in seconds. The demos turned out to be a huge success.
- Factory VM: you can use a factory VM pattern to create new virtual machines before adding them to the pool.
- Stateless VM: If the VMs in the pool are stateless, they require much less management work. Basically you don’t need to track different VMs–just pick anyone from a single pool!
- Template VM: you can create different templates for significantly different VMs.