How to Create Linked Virtual Machines with vSphere API?

More often than not, you may have several virtual machines based on same software stacks running on the same host. Although they are very much the same, they take as much space as multitude of what one virtual machine takes.

Since vSphere 4.0, things are different. You can significantly reduce the storage usage by a new feature called linked virtual machines. The idea is simple: sharing a common virtual disk among the similar virtual machines. The shared virtual disk serves as a base. On top of that, each virtual machine has its own delta disk. When a guest operating system writes to disk, the data persists to the delta disk. When it reads from disk, the delta disk is checked first before trying the base disk.

Bothered by SLOW Web UI to manage vSphere? Want to manage ALL your VMware vCenters, AWS, Azure, Openstack, container behind a SINGLE pane of glass? Want to search, analyze, report, visualize VMs, hosts, networks, datastores, events as easily as Google the Web? Find out more about vSearch 3.0: the search engine for all your private and public clouds.

As a result, you only need to save one copy of the base disk no matter how many virtual machines you have (up to 8 virtual machines in a linked virtual machine group). One limitation is that you cannot use it with HA cluster.

How to create linked virtual machines? You have two approaches: clone a virtual machine either from a snapshot, or from its current running state.

You can automate this process by leveraging VMware vSphere API. Here is the tech note Linked Virtual Machines. Accompanied with this tech note, several samples are shipped in vSphere Web Service SDK. The following is a sample I rewrote using vSphere(VI) Java API.

package com.vmware.vim25.mo.samples.vm;

import java.net.URL;
import java.util.ArrayList;

import com.vmware.vim25.ManagedObjectReference;
import com.vmware.vim25.TaskInfoState;
import com.vmware.vim25.VirtualDevice;
import com.vmware.vim25.VirtualDeviceBackingInfo;
import com.vmware.vim25.VirtualDisk;
import com.vmware.vim25.VirtualDiskFlatVer1BackingInfo;
import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
import com.vmware.vim25.VirtualDiskRawDiskMappingVer1BackingInfo;
import com.vmware.vim25.VirtualDiskSparseVer1BackingInfo;
import com.vmware.vim25.VirtualDiskSparseVer2BackingInfo;
import com.vmware.vim25.VirtualMachineCloneSpec;
import com.vmware.vim25.VirtualMachineRelocateDiskMoveOptions;
import com.vmware.vim25.VirtualMachineRelocateSpec;
import com.vmware.vim25.VirtualMachineRelocateSpecDiskLocator;
import com.vmware.vim25.VirtualMachineSnapshotInfo;
import com.vmware.vim25.VirtualMachineSnapshotTree;
import com.vmware.vim25.mo.Datastore;
import com.vmware.vim25.mo.Folder;
import com.vmware.vim25.mo.InventoryNavigator;
import com.vmware.vim25.mo.ServiceInstance;
import com.vmware.vim25.mo.Task;
import com.vmware.vim25.mo.VirtualMachine;

/**
 * Author: Steve Jin (sjin@vmware.com)
 * This sample creates a linked clone from an existing snapshot. It works with vSphere 4 and not supported by the old VI SDK 2.* API
 * Each independent disk needs a DiskLocator with diskMoveType as moveAllDiskBackingsAndDisallowSharing.
 */

public class VMLinkedClone
{
 public static void main(String[] args) throws Exception
 {
 if (args.length != 6)
 {
 System.out.println("java VMLinkedClone <SdkUrl> <username> <password> <VmName> <SnapShotName> <CloneVmName>");
 System.out.println("java VMLinkedClone https://8.8.8.8/sdk root password myVM1 snapBaseLine myCloneVM2");
 return;
 }
 String vmName = args[3];
 String snapShotName = args[4];
 String cloneVmName = args[5];

 ServiceInstance si = new ServiceInstance(new URL(args[0]), args[1], args[2], true);
 VirtualMachine vm = (VirtualMachine) new InventoryNavigator(si.getRootFolder()).searchManagedEntity("VirtualMachine", vmName);

 createLinkedClone(vm, snapShotName, cloneVmName);

 si.getServerConnection().logout();
 }

 public static void createLinkedClone(VirtualMachine vm, String snapshotName, String cloneVmName) throws Exception
 {
 if(vm==null || snapshotName==null)
 return;

 ArrayList<Integer> diskKeys = getIndependenetVirtualDiskKeys(vm);

 VirtualMachineRelocateSpec rSpec = new VirtualMachineRelocateSpec();
 if(diskKeys.size() > 0)
 {
 Datastore[] dss = vm.getDatastores();

 VirtualMachineRelocateSpecDiskLocator [] diskLocator = new VirtualMachineRelocateSpecDiskLocator[diskKeys.size()];

 int count = 0;
 for(Integer key : diskKeys)
 {
 diskLocator[count] = new VirtualMachineRelocateSpecDiskLocator();
 diskLocator[count].setDatastore(dss[0].getMOR());
 diskLocator[count].setDiskMoveType(VirtualMachineRelocateDiskMoveOptions.moveAllDiskBackingsAndDisallowSharing.toString());
 diskLocator[count].setDiskId(key);
 count = count + 1;
 }
 rSpec.setDiskMoveType(VirtualMachineRelocateDiskMoveOptions.createNewChildDiskBacking.toString());
 rSpec.setDisk(diskLocator);
 }
 else
 {
 rSpec.setDiskMoveType(VirtualMachineRelocateDiskMoveOptions.createNewChildDiskBacking.toString());
 }

 VirtualMachineCloneSpec cloneSpec = new VirtualMachineCloneSpec();
 cloneSpec.setPowerOn(false);
 cloneSpec.setTemplate(false);
 cloneSpec.setLocation(rSpec);
 cloneSpec.setSnapshot(getSnapshotReference(vm, snapshotName));

 try
 {
 Folder parent = (Folder) vm.getParent();
 Task task = vm.cloneVM_Task(parent, cloneVmName, cloneSpec);

 task.waitForTask();
 if(task.getTaskInfo().getState() == TaskInfoState.error)
 {
 System.out.println("Failure: Virtual Machine cannot be cloned");
 }
 if(task.getTaskInfo().getState() == TaskInfoState.success)
 {
 System.out.println("Virtual Machine Cloned  successfully.");
 }
 }
 catch(Exception e)
 {
 System.out.println("Exception while cloning: " + e);
 }
 }

 private static ArrayList<Integer> getIndependenetVirtualDiskKeys(VirtualMachine vm) throws Exception
 {
 ArrayList<Integer> diskKeys= new ArrayList<Integer>();

 VirtualDevice[] devices = (VirtualDevice[]) vm.getPropertyByPath("config.hardware.device");

 for(int i=0; i<devices.length; i++)
 {
 if(devices[i] instanceof VirtualDisk)
 {
 VirtualDisk vDisk = (VirtualDisk) devices[i];
 String diskMode = "";
 VirtualDeviceBackingInfo vdbi = vDisk.getBacking();

 if(vdbi instanceof VirtualDiskFlatVer1BackingInfo)
 {
 diskMode = ((VirtualDiskFlatVer1BackingInfo) vdbi).getDiskMode();
 }
 else if(vdbi instanceof VirtualDiskFlatVer2BackingInfo)
 {
 diskMode = ((VirtualDiskFlatVer2BackingInfo)vdbi).getDiskMode();
 }
 else if(vdbi instanceof VirtualDiskRawDiskMappingVer1BackingInfo)
 {
 diskMode = ((VirtualDiskRawDiskMappingVer1BackingInfo)vdbi).getDiskMode();
 }
 else if(vdbi instanceof VirtualDiskSparseVer1BackingInfo)
 {
 diskMode = ((VirtualDiskSparseVer1BackingInfo)vdbi).getDiskMode();
 }
 else if(vdbi instanceof VirtualDiskSparseVer2BackingInfo)
 {
 diskMode = ((VirtualDiskSparseVer2BackingInfo)vdbi).getDiskMode();
 }

 if(diskMode.indexOf("independent") != -1)
 {
 diskKeys.add(vDisk.getKey());
 }
 }
 }
 return diskKeys;
 }

 private static ManagedObjectReference getSnapshotReference(VirtualMachine vm, String snapshotName) throws Exception
 {
 VirtualMachineSnapshotInfo snapInfo = vm.getSnapshot();

 ManagedObjectReference snapmor = null;

 if(snapInfo != null)
 {
 VirtualMachineSnapshotTree[] snapTree = snapInfo.getRootSnapshotList();
 snapmor = traverseSnapshotInTree(snapTree, snapshotName);
 }
 return snapmor;
 }

 private static ManagedObjectReference traverseSnapshotInTree(VirtualMachineSnapshotTree[] snapTree, String snapshotName)
 {
 if(snapTree == null || snapshotName == null)
 return null;

 ManagedObjectReference snapmor = null;

 for (int i=0; i < snapTree.length && snapmor == null; i++)
 {
 VirtualMachineSnapshotTree node = snapTree[i];
 if (node.getName().equals(snapshotName))
 {
 snapmor = node.getSnapshot();
 }
 else
 {
 VirtualMachineSnapshotTree[] childTree = snapTree[i].getChildSnapshotList();
 snapmor = traverseSnapshotInTree(childTree, snapshotName);
 }
 }
 return snapmor;
 }
} 

This entry was posted in vSphere API and tagged , , , . Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

15 Comments

  1. Terry lei
    Posted April 6, 2012 at 9:42 am | Permalink

    Hi Steve,

    Is this (vm.cloneVM_Task()) workable for ESXi 5? Or It can just be used by vCenter. My company has bought your Vi SDK book, from that book, it says, the clone is an advanced feature which is just supported by VirtualServer.

    Thanks,
    Terry

  2. Posted April 6, 2012 at 10:29 am | Permalink

    Hi Terry,
    Thanks for your support on my book! I don’t think it has been changed for the clone feature. You can easily test it out with ESXi. If you get exception like NotSupported, it’s not supported.
    Steve

  3. Terry lei
    Posted April 8, 2012 at 7:25 am | Permalink

    Hi Steve,

    Thanks for your feedback, I test it with ESXi env, it is still not supported.

    As there is no vCenter in our lab (we just have ESXi), is there other API we could use to clone a VM manged by ESXi?

    -Terry

  4. Posted April 8, 2012 at 9:26 am | Permalink

    Hi Terry,

    It’s a business decision to allow cloning only with vCenter even though technically it’s doable with ESXi. In fact, the vCenter has to rely on ESXi to clone VMs.

    Steve

  5. Terry Lei
    Posted April 8, 2012 at 10:28 pm | Permalink

    @Steve Jin

    Thanks a lot for you reply, Steve. I will check with my manager, if we can have vCenter or not .

    -Terry

  6. slimanihaythem
    Posted May 24, 2012 at 3:05 am | Permalink

    Hi Steve ,
    I tried this code but I got as answer “Failure: Virtual Machine cannot be cloned.” and in the vSphere client , the task view , it mentions :
    **********
    Cloner machine virtuelle
    Tomcat2
    Cette
    opération n’
    est pas prise
    en charge
    sur l’objet.
    root.
    ********
    this operation is not supported by vSphere?
    (I use Esxi5)

    thanks

  7. Posted May 24, 2012 at 10:59 am | Permalink

    Were you connecting to ESXi directly? The cloning operation only works on vCenter, not ESXi. So try it on vCenter.

    Steve

  8. Drum
    Posted July 12, 2012 at 10:42 am | Permalink

    Hi Steve,
    Is it possible to create a linked VM from a linked VM?

  9. Posted July 12, 2012 at 12:19 pm | Permalink

    Good question. I haven’t tried it before. You can give it a try.

    Steve

  10. smcg
    Posted July 30, 2012 at 3:25 pm | Permalink

    I have created chains of linked clones before and it works fine. However, you will eventually experience a performance degradation if the chain becomes too long (10+). Past a certain point, it becomes quicker to do a standard clone instead of a linked clone. I haven’t tested out what that point is, but unless you’re going hog-wild with snapshots and clones you probably won’t hit it.@Steve Jin

  11. Posted July 30, 2012 at 3:30 pm | Permalink

    Thanks for sharing real world experience with the community!
    Steve

  12. youssef
    Posted September 18, 2012 at 1:07 pm | Permalink

    thanks for sharing,
    i tried to clone an existing snapshot, but it always clone from the initial vm state, can you please advice

  13. Posted September 19, 2012 at 12:00 am | Permalink

    Hi Youssef, great question. I don’t have answer on top of my head. You may want to try vijava api forum to see others have done that.
    Steve

  14. Mark
    Posted April 3, 2013 at 9:55 pm | Permalink

    Hi Steve,
    I tried this code and it creates linked clones. The machine I am cloning only has one snapshot, which I call “base”. There were older snapshots, but those have been deleted.

    The linked clones sometimes are perfect, but other times are clones of an older snapshot of the parent! I even had a clone that had a mixture of files from an old snapshot and the “base” snapshot.

    How is this possible? Thanks.

  15. Janardan
    Posted July 21, 2014 at 5:12 am | Permalink

    How to create Host using Java? actually I am stuck at parent folder name.which folder name should be provide to create method of SDK(vsphere).
    please give me example.
    Thanks

One Trackback

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

  • NEED HELP?


    My company has created products like vSearch ("Super vCenter"), vijavaNG APIs, EAM APIs, ICE tool. We also help clients with virtualization and cloud computing on customized development, training. Should you, or someone you know, need these products and services, please feel free to contact me: steve __AT__ doublecloud.org.

    Me: Steve Jin, VMware vExpert who authored the VMware VI and vSphere SDK by Prentice Hall, and created the de factor open source vSphere Java API while working at VMware engineering. Companies like Cisco, EMC, NetApp, HP, Dell, VMware, are among the users of the API and other tools I developed for their products, internal IT orchestration, and test automation.