Archive

Archive for the ‘Virtualization’ Category

How to Import and Export OVF Packages

April 7th, 2010 103 comments

This article is based on a similar one at vSphere Java API home page. At that time, one of VMware community members sent me an email for samples of using OvfManager APIs. Then I went to office on a Saturday writing two samples, which have been validated by several folks as “working” samples.

The purpose of the samples are to illustrate the vSphere APIs. Let’s take a look at them one by one.

First, ExportOvfToLocal.java. This sample shows how to download either a VM or vApp to your local machine. The typical flow is:

  • Find the VM or vApp
  • Call their exportVm() or exportVApp() methods and get HttpNfcLease
  • Set lease time out
  • Wait for HttpNfcLease until it’s ready
  • From the HttpNfcLease.info property, find the all URLs from which you download the vmdk files
  • Call OvfManager.createDescriptor() API to create the content of ovf and save it to a file along with downloaded vmdk files.
  • Release the lease by calling httpNfcLeaseComplete() method
Categories: vSphere API Tags: ,

OVF vs. VMDK

April 6th, 2010 2 comments

OVF stands for Open Virtualization Format, a platform independent, extensible packaging and distribution format for virtual machines. It’s now a DMTF standard.

VMDK stands for Virtual Machine Disk, a format that encodes a single virtual disk for a virtual machine. It’s proprietary by VMware but whose format is publicly documented by the company. You can use VDDK to manipulate the VMDKs.

Categories: Virtualization Tags: , ,

Winners of the VMware Script-O-Mania Contest: Who, What, How and Why?

April 1st, 2010 No comments

The winners of the VMware Script-O-Mania contest was announced today by Pablo Roesch at VMware Developer Blog. Congratulations to Alan, William and Arnim who won the first three awards!

I just took some time to read these scripts even though PowerShell and Perl are not for me. Here I give you a brief introduction of the scripts, what vSphere APIs they used directly or indirectly, and why they topped the contest. Because vSphere API is based on Web Services, you can port the scripts to other languages like Java, .Net, whatever you feel comfortable with. If you want to port any of them using vSphere Java API, I am more than happy to include your contribution.

Note that the following comments are strictly my own opinions.

1. Who Created that VM ? – by Alan Renouf using PowerCLI

A script to add information back into the vSphere client, this script which is designed to run once a day (or more) as a scheduled task, will add a custom attribute to each VM with the creator and date created of that VM. A script to add information back into the vSphere client, this script which is designed to run once a day (or more) as a scheduled task, will add a custom attribute to each VM with the creator and date created of that VM.

Steve’s Comments:

Nice integration with the vSphere Client, making you almost doubt why it wasn’t there in the first place. Additional one liner scripts provide nice answers to the questions like who created the most VMs, how many VMs were created each month.

Introducing Security Model of VMware vSphere

April 1st, 2010 2 comments

This article introduces you the basic model and terminologies in vSphere security management, for example, privileges, permissions, roles, and how they are related to each other to secure vSphere. It helps you to better manage the vSphere and program the vSphere API. Much of the content is based on my book VMware VI and vSphere SDK by Prentice Hall.

In vSphere, the security model consists of three types of components: privileges, roles, and permissions.

Privileges

A privilege is the basic individual right required to perform an operation. It is statically defined and never changes in a single version of a product. Given the many operations in VI, there are many privileges (for example, the privilege to “power on a virtual machine”). These privileges are represented as strings separated by dots, such as VirtualMachine.Interact.PowerOn.

The operations and privileges are not one-to-one mapping. Many operations do share common privileges like System.View. Therefore, there are many fewer privileges defined than methods. In some exceptional cases, a method requires different privileges depending on the target it operates on and the nature of the operation. The CloneVM_Task() method, for example, requires VirtualMachine.Provisioning.Clone for cloning from one virtual machine to another, VirtualMachine.Provisioning.DeployTemplate for cloning from a template to a virtual machine, and so on.

Roles

The role groups privileges from a user’s perspective. A role is normally named and defined for a group of people who have common responsibilities in the system (for example, administrators). Each role can include zero to multiple privileges. The extreme cases are the predefined “Admin” roles, which by default, includes all the privileges and the NoAccess role, which includes no privileges.

Categories: vSphere API Tags: ,

Fundamentals of vSphere Performance Management

March 31st, 2010 9 comments

Performance monitoring is a critical aspect of vSphere administration. This article introduces you the basic concepts and terminologies in vSphere performance management, for example, performance counters, performance metrics, real time vs historical statistics, etc. Much of the content is based on my book VMware VI and vSphere SDK by Prentice Hall.

Once you understand these basics, the related tools and APIs should be relatively easy. If you are already familiar with vSphere Client performance monitoring or esxtop, they help as well.

Performance Counter

A performance counter is a unit of information that can be collected about a managed entity. PerfCounterInfo data object, shown in Figure 1, represents a performance counter. The property key is an integer that uniquely identifies a performance counter, like a primary key of a table in SQL database, and nothing more. There is no guarantee for a performance counter to have a fixed number. In fact, the same performance counter can have different values in ESX and VirtualCenter. Even for the same type of server, the number could change from version to version. Do not use it outside the context of the server you connect to.

Figure 1 PerfCounterInfo data object

The performance counter can be represented by the following dotted string notation:

3 Ways to Get Hold of Managed Objects in vSphere

March 30th, 2010 No comments

If you have ever used vSphere Web Service API, you must have known that there is no managed object but ManagedObjectReference object. Understanding it helps deepen your understanding of the vSphere API.

Honestly, the ManagedObjectReference is a little confusing by itself. It is in fact a data object but represents a managed object. You can think of a MOR as a pointer in some sense because it’s used to uniquely identify a managed object. Even better, you can think of the “type” and “value” defined in the MOR in the SQL way. The type is like a table name, and the value like the primary key which can uniquely identify a managed object in its type.

MOR is really intended to be used by program and should be carefully limited to the scope of where it comes from. That is why it’s hidden from application developers in vSphere Java API.

Anyway, let’s see how to get hold of MOR objects:

Categories: vSphere API Tags: ,

vSphere Inventory Structure Deep Dive

March 29th, 2010 4 comments

This blog digs into the vSphere inventory structure, and changes in vSphere 4 from VI. It explains the difference between ESX and vCenter inventory tree, why the inventory path you see from vSphere Client doesn’t work with API. Much of the content is adapted from my book VMware VI and vSphere SDK by Prentice Hall.

Inventory vs. Inheritance hierarchy

My previous blog introduced the object model of the vSphere API. The UML diagram there shows how different managed object types are structured in the inheritance hierarchy.

The inventory hierarchy is quite different. It shows how different managed object instances are associated. With this understood, you can easily navigate among different managed objects in inventory tree of both ESX and vCenter. It’s critical for programming vSphere API because navigating the inventory is the primary way to get hold of managed objects.

Categories: vSphere API Tags: ,

Demystifying 3 “View”s from VMware

March 23rd, 2010 6 comments

After I blogged the top 5 myths of vSphere API, William Lam suggested me to write a bit more on the views in his comments. If you haven’t followed him (@lamw) at Twitter yet, you may want to. His vGhetto Perl repository is one of the best resources for people who use VI Perl.

For sure, VMware loves the term “view”. As far as I know, there are 3 different ”views.” Two of them are for APIs; the last one is for the desktop product family. We are not going to talk about the product View in this blog. You can find more information at VMware web site.

Let’s instead focus on the two “Views” for developers: one is in VI Perl and .NET/PowerCLI; the other is part of the core vSphere API.

Categories: vSphere API Tags: , , , ,

Why Some vSphere Java API Methods don’t Work with Old Servers? A Story of Compatibility

March 15th, 2010 1 comment

Many of you already know there are some changes in the vSphere API from 2.5 to 4.0. The changes include 20+ new managed object types, additional properties (including sub-properties that embedded inside the first level properties), and several inheritance structure changes.

Several managed objects like Datastore became the subtype of ManagedEntity in vSphere 4, which is different from the hierarchy in 2.5 where it’s a subtype of ExtensibleManagedObject. The changes came for good reasons – we want permission control over these managed objects.

SimDK – A VMware vSphere Simulator

March 9th, 2010 5 comments

Just got the following email from Andrew Kutz (@sakutz) who wrote the famous VMware Infrastructure (VI) plug-ins whitepaper and created several other great projects like VMM.

David Marshall, Dave McCrory and I, as well as everyone else at Hyper9, are extraordinarily proud to announce SimDK – a VMware vSphere4 simulator which provides vSphere4 API-compatibility for official vSphere4 clients and other applications built using the vSphere4 SDK.. SimDK is an open source project available at http://simdk.sourceforge.net/. You can read more about this exciting announcement at http://akutz.wordpress.com/2010/03/09/simdk.

Categories: vSphere API Tags: ,

Vote for vSphere Java API at VMware Labs

March 9th, 2010 2 comments

Several smart bloggers (Eric Sloof, Justin EmersonIan Koenig, Alessandro Perilli) discovered the VMware Labs web site over the weekend. As many pointed out, it’s absolutely cool.

I actually knew it was going to be online this past weekend, but would like to get official announcement from the company before blogging it. The reason I knew the site to go live is because the vSphere Java API I created is one of the first 10 projects.

Categories: vSphere API Tags:

Lightweight Caching Framework in vSphere Java API 2.0

March 7th, 2010 5 comments

In vSphere Java API 2.0, I wrote a lightweight caching framework. It’s still experimental but has a great potential to greatly simplify your development work. Commercial companies already use it in their products.

The motivation behind this framework is simple – instead of keep polling the changes from the server side, you keep a local cache that is made as fresh as possible. The View in the vSphere Perl toolkit is one way to do. It caches all properties of a managed object despite the fact that you don’t need that many at all.

The caching framework in vSphere Java API takes another approach. You tells it what managed objects and what properties you want to be cached. After that, the caching framework does its best to read the properties and keep them as fresh as possible.

Architecturally the caching framework is totally separated from the core of the API. You can take it away without any impact on the rest of the API. This is quite different from other toolkit.

Have enough introduction? Let’s take a look at sample code:

Categories: vSphere API Tags: ,

Using vSphere Java API in Jython and Other JVM languages

March 5th, 2010 2 comments

As a by-product, the vSphere Java API makes Jython programming a lot easier. The following is a very simple sample written in Jython to print out the name of the first virtual machine in inventory.

    from java.net import *
    from com.vmware.vim25.mo import *

    si= ServiceInstance(URL(“https://sjin-dev1/sdk”),\
    “root”, “password”, True)
    rootFolder = si.getRootFolder()
    vms = InventoryNavigator(rootFolder) \
    .searchManagedEntities(“VirtualMachine”)
    print “Hello ” + vms[0].getName()
    si.getServerConnection().logout()

As you can see, it really brings in the benefit of VI Java API into Python community at almost no extra cost.

Categories: vSphere API Tags: ,

5 Easy Steps Using vSphere Java API

March 4th, 2010 2 comments

In my previous blogs, I have introduced the vSphere API object model, vSphere Java API architecture. I assume you’ve run through the 5 minute Getting Started Tutorials with HelloWorld sample.

Now, let’s take a look on how to use the API in general.

1. Always starts with a ServiceInstance with URL/username/password, or URL/sessionID. For example,

ServiceInstance si = new ServiceInstance(new URL(urlStr), username, password, true);

2. From the ServiceInstance object, you can:

Categories: vSphere API Tags: ,

vSphere Java API Architecture Deep Dive

March 3rd, 2010 2 comments

In my previous blog, I talked about the object model of the vSphere API. Many people like the UML diagram that illustrates how the managed objects are inherited from each other.

Following that blog, I will introduce the object model of the open source Java API that is built on top of the Web Services, as well as some key design decisions I made while designing the API.

The following UML diagram is extracted from the overall model but adds much more details with properties and methods. If you can understand this diagram, you can then easily understand all other managed object types.

Trying Self Paced Lab without Going to PEX 2010

March 2nd, 2010 No comments

I mentioned the vSphere API self paced lab at PEX in my previous blog. Not all the people who are interested in learning the API made it to PEX last month. A reader asked me when it can be online in his comment.

Here is the VI Java API part in the tutorial. We had the environment set up all together for you in the PEX lab including the Eclipse and all the related jar files. So it’s very easy to get started there. Without going to PEX, you need to do something extra by yourself. But that is not too hard at all. I promise it won’t take you much time at all. To get the basic one done, you probably need 5 to 30 minutes depending on your familiarity with Java.

Ready to learn?

Top 5 Myths about VMware vSphere API

February 28th, 2010 10 comments

If you have trouble to understand vSphere API when you first use it, you are definitely not alone. I had the same trouble when I first used it a while back.

Some of the troubles come from the disconnect between conventional programming model and that of vSphere API. In this blog, I summarize the top 5 myths about vSphere API based on my experience and the questions I see in the VMware community forum and vSphere Java API forum:

  1. Non-existing Managed Objects
  2. Pervasive PropertyCollector
  3. Short-Lived Task Object
  4. “Weak” HostSystem
  5. “Un-documented” View

Let’s examine each of them one by one.

Object Model of VMware vSphere API: A Big Picture in 2 Minutes

February 27th, 2010 22 comments

When I start to use a new API/SDK, I always look for the object model diagram before digging into the API Reference. With that, I can have a good overview of the API, from the concepts to the structure. This can save a lot of time.

Unfortunately, we don’t find such a object model diagram in any official document. The following is the UML diagram from my book VMware VI and vSphere SDK.

VQL – A Domain Specific Language for Virtualization

February 26th, 2010 No comments

Last summer, Reflex VP engineering Aaron Bawcom visited VMware campus. He is one of the authors of Virtualization For Security: Including Sandboxing, Disaster Recovery, High Availability, Forensic Analysis, And Honeypotting.

During our talk, Aaron told me a secret project called VQL. Because it’s a secret, I didn’t talk about it to anyone. Just before the PEX 2010, we exchanged emails about the VQL. Aaron told me it’s already shipped. So it’s time to broadcast it.

VQL is a DSL that looks like SQL, easy to understand and easy to use. Unlike SQL on data, it’s on virtual resources in vSphere environment. The following is a quick sample. It gets back the VMs installed with SQL servers with verions newer than 9.2 and they are running 10 minutes ago.

SQL Server and VersionMajor >= 9 and VersionMinor >= 2 project vm at 10 minutes ago

Categories: vSphere API Tags: , ,

3 Easy Ways Connecting to Your VM in Private Cloud

February 24th, 2010 30 comments

Several folks asked me about how to use vSphere(VI) Java API to connect to a VM running on vSphere. The quick answer is vSphere Java API is not designed for this. You will need VMware Remote Console, browser plug-in, remote desktop/VNC, SSH client etc. However, it can help you to get the information required by the console or plug-in. Tal Altman from CISCO suggested that it be a topic for doublecloud.org. Here it is.

There are 3 ways to connect to the VM from your client side outside the vSphere and Web Access which have built-in support for console access.

  1. Using VMware Remote Console which is a standalone application
  2. Using browser plug-in to either IE or Firefox (Note: this is NOT supported by VMware. Please don’t call the company tech support for this.)
  3. Using Remote Desktop, VNC or SSH

The first two connect to the ESX host, and work even there is no guest OS installed on the VM. The last one assumes you have guest OS installed, and have IP network and server components in place already.

Note that these 3 ways work for the VMs in the public cloud as well if the related ports are open in your firewall. It is, however, not the case for most enterprises, therefore I particularly say it’s for VMs in private cloud. If you don’t have firewall issue, feel free to give it a try with public cloud as well.

Let’s go over one by one in details and see how vSphere Java API helps.