Archive for the ‘Virtualization’ Category

Introducing PowerShell CLIs for VMware View

February 16th, 2011 2 comments

Two weeks ago when I twitted about VMware View APIs, several folks asked me if and where they could find View APIs. While VMware does not provide a full set of APIs, but does offer PowerShell based CLI as documented in the VMware View Integration Guide.

Beyond that, the guide also introduces the event database with tables and schemas so that you can retrieve various events from the database directly; how to import/export LDAP data to/from View; and how to “use Microsoft System Center Operations Manager (SCOM) to monitor the state of VMware View components, including View Connection Server instances and security servers and View services running on these hosts.”

While trying the PowerShell CLI,

Categories: Virtualization Tags: , ,

How to Enable or Disable Copy and Paste to Remote VM Console?

February 14th, 2011 No comments

In my previous post, I introduced how to change a virtual machine’s vmx file programmatically and promised to post a full sample in my presentation at VMware Parter Exchange 2011. Now that the conference is over, it’s time to post it.

The sample is based on guideline VMX03 in vSphere security hardening guide: disable copy/paste to remote console. To me, allowing copy and paste to remote console like vSphere Client is a nice feature which can save you a lot of time. When security is a concern, however, you may want to disable it.

I will not discuss when you should disable/enable it because it really depends on your requirements. In most cases, security and convenience contradict with each other. I leave it for you to decide the right balance, but show you how you can check the setting and change it here.

Like most samples I write,

Getting ESX and ESXi Memory Info

February 2nd, 2011 9 comments

With a vSphere Client, you can easily check the memory information of a host, either ESX or ESXi. To get that, you click on a host from the inventory tree, and then configuration tab. From the left side Hardware section of the configuration page, you click Memory and see a pane displaying the memory info as follows:

Note that if you have chosen a ESXi host, you won’t see the Service Console part because there is no console OS any more in ESXi. BTW, VMware wants you to migrate from ESX to ESXi and here is a link with helps.

This seemingly easy information is actually not easy to get. At first glance, it should be in the config property of HostSystem (managed object representing an ESX or ESXi). The config property has a sub property called systemResources, typed as HostSystemResourceInfo. But you will get null for the systemResources property most, if not all, of the time, as reported in VI Java API forum.

Interestingly enough, HostSystem has a systemResources property in peer to the config property as well. Luckily, it’s not null so you can dig down for something. Still, with 3 sub properties of complex types included, how to get the memory from the data object?

Here are the steps to collect and calculate the numbers:

Categories: vSphere API Tags: , , ,

How to Change VMX Programmatically?

January 31st, 2011 7 comments

As most of you may have known, the vmx file is the most important file at the back of a virtual machine. It defines the configurations of a virtual machine for example virtual hardware version, devices, disk files, etc. That is why it owns the virtual machine icon when you look at all the files behind a virtual machine with either datastore browser in vSphere Client or VMware Workstation/Fusion.

Given the importance of the vmx file, we don’t recommend anyone to manually modify it given that messing it up may corrupt a virtual machine. You can change the content indirectly from GUI and APIs. If you can change the virtual devices of a virtual machine, its vmx file is changed accordingly.

Beyond basic configuration entries, the vmx file can be extended to hold key/value pairs. A vmx file is really a text file with many lines of key/value pairs. By extending it, you can add extra lines of key/value pairs at the end of the file. Keep in mind that you should NOT add any key that is duplicated with predefined keys like “virtualHW.version.” As you already noticed, the key has the “.” delimiter as namespace pattern. You can start with your own namespace to avoid possible name clash.

How to Differentiate Folders in vSphere

January 28th, 2011 4 comments

As we know Folder is a container object in vSphere. It’s used to hold other managed objects in a similar way as directory holding files in an OS. There is an interesting question asking in VI Java API forum on how to tell whether a folder is vmFolder or hostFolder. BTW, you don’t see these two folders displayed in vSphere Client.

The questioner guessed correctly that you can find out by a folder’s childType property. Here is the documentation in API reference on the possible values of the childType property:

Free ESXi and API/CLI support

January 27th, 2011 8 comments

I think there is a pretty good understanding on this topic, but questions still come up once in a while in VI Java API community and my inbox. Some people got errors while trying to manage free ESXi servers via API. The API support of free ESXi is limited to ready only according to a VMware KB article:

vCLI, PowerCLI, and vSphere SDk for Perl are limited to read-only access for the free vSphere Hypervisor edition. To enable full functionality of vCLI on a VMware ESXi host, the host must be licensed with vSphere Essentials, vSphere Essential Plus, vSphere Standard, vSphere Advanced, vSphere Enterprise, or vSphere Enterprise Plus.

Categories: vSphere API Tags: , , , ,

How You Can Use VI Java API in Scala: A Quick Sample

January 25th, 2011 No comments

As I blogged before, you can use open source VI Java API in all JVM languages like Jython, Groovy, JRuby and frameworks like Grail. Today Dave Briccetti (@dcbriccetti) sent me a sample showing how to use it in Scala. Dave has been a happy user of the API since he started to work at VMware as an independent consultant more than a year ago.

Here is the code sample:

How to Delete Virtual Machine With vSphere API?

January 18th, 2011 3 comments

This question was asked at VI Java API forum recently and has been answered by the community.

There should be a “Destroy_Task()”-Method for each Managed Object, so also Virtual Machines…. Cheers, Joerg

As  Jörg Lew has correctly pointed out the solution, I would like to elaborate a bit more here.

If you are using VI Java API, the method name is destroy_Task(). The code is as simple as:

Difference of Two Common Privileges in vSphere API

January 17th, 2011 No comments

If you have used vSphere API and read its API Reference, you may have noticed two most commonly used privileges: System.View and System.Read. They are required in many methods. As their names suggest they are different, but what is the difference? It can be confusing for some people including me initially because it’s nowhere documented.

Here are some explanations after my talking to my colleague Jianping Yang who is the vCenter DB and Security guru.

Categories: vSphere API Tags:

HP Chooses Open Source vSphere Java API

January 12th, 2011 No comments

After NetApp, Brocade and many other companies showing up on the VI Java API poweredby page, we now have the world’s biggest technology company HP officially listed. HP started to use VI Java API about two years ago and has shipped products ever since. I have attended several meetings called by my colleague Lucas Nguyen to work with HP architect Zachary Speck and his team. Included in the following short paragraph is a link to the HP plugin product. Please feel free to check it out.

Categories: vSphere API Tags: ,

Join Me at Partner Exchange 2011

December 22nd, 2010 2 comments

VMware Partner Exchange takes place twice a year. One happens at the same time/location with VMworld US; the other in places like Las Vegas, Orlando. It’s a dedicated conference to educating and enabling partners for success with VMware. It has merged with Technology Exchange where you can find many technical presentations. I have been speaking at TechnologyExchange since I joined VMware in 2007. Here is the related articles I wrote earlier.

The coming ParterExchange will be in Orlando FL from Feb 7 to 11. Please join us to hear VMware’s plans for the coming year, learn of new technologies and partner programs, and understand the training roadmap. Here is the content catalog with all the sessions. Don’t forget the famous hands-on labs throughout the week. I will talk about securing vSphere infrastructure with vSphere API.

Open Source In Action: Open Source Projects from VMware

December 21st, 2010 2 comments

As a leading edge software company, VMware has a long history of support for open source software in its products. It also contributes back many patches and projects to the open source community including the vijava API that I created. With SpringSource and Zimbra acquisitions, more open source projects are associated with VMware brand.

Here is a list of 10 home grown open source projects from VMware. Please feel free to click links for more details and play with them.

1. Dr. Memory. It’s “a memory monitoring tool capable of identifying memory-related programming errors such as accesses of uninitialized memory, accesses to unaddressable memory (including outside of allocated heap units and heap underflow and overflow), accesses to freed memory, double frees, memory leaks, and (on Windows) access to un-reserved thread local storage slots.”

2. Virtual USB Analyzer. A “free and open source tool for visualizing logs of USB packets, from hardware or software USB sniffer tools. As far as we know, it’s the world’s first tool to provide a graphical visualization along with raw hex dumps and high-level protocol analysis.”

vSphere Performance Counters for Monitoring ESX and vCenter

December 3rd, 2010 11 comments

VMware vSphere provides comprehensive performance metrics for your needs on performance monitoring and diagnosis. These stats are available through not only vSphere Client but also vSphere APIs. To understand the overall performance management concepts, you want to read this article: Fundamentals of vSphere Performance Management.
Once having the basics, you may wonder what types of stats are exposed. The following table summaries all the 315 performance counters available in vSphere 4.1. As you might have guessed, the information is generated using open source Sphere Java API and then imported into WordPress using WP-Table Reloaded. You can easily sort and search the table.

Update: Carter Shanklin and Luc Dekens have articles on performance counters as well:

Wire Compatibility of Web Services

November 23rd, 2010 No comments

As a software professional, you may have heard about the source compatibility and binary compatibility. With the Web Services, a new type of compatibility came up. This is what I call wire compatibility. It’s not related to the programming but the XML messages passed on the wire. Since we don’t use XML directly but programming APIs, the wire compatibility surfaces and affects the source and binary compatibility.

Too abstract? You bet. Let’s pick up an example here. Because VMware vSphere API is defined in WSDL, I will use it in the following discussion.

In vSphere 4.1, the method PowerOnMultiVM_Task() gets an additional parameter called option typed as OptionValue array. The following are related parts in the WSDL:

<operation name="PowerOnMultiVM_Task">
  <input message="vim25:PowerOnMultiVM_TaskRequestMsg" />
  <output message="vim25:PowerOnMultiVM_TaskResponseMsg" />
  <fault name="RuntimeFault" message="vim25:RuntimeFaultFaultMsg"/>
<complexType name="PowerOnMultiVMRequestType">
    <element name="_this" type="vim25:ManagedObjectReference" />
    <element name="vm" type="vim25:ManagedObjectReference" maxOccurs="unbounded" />
    <element name="option" type="vim25:OptionValue" minOccurs="0" maxOccurs="unbounded" />

As you can see, the minOccurs of the option element is zero, meaning it’s optional. If you have an application built with 4.0 (no option parameter by then), the SOAP request still works. So it’s compatible on the wire.

Undocumented VI and vSphere API Methods: A Little History

October 25th, 2010 No comments

Most developers may have noticed the asynchronous methods in vSphere API like PowerOnVM_Task method, but not so many know their synchronous peers like PowerOnVM before 4.1. VMware vSphere API Reference doesn’t mention them at all. But you can find them in WSDL(check out the WSDL snippets at the end of this article).

There is an exception however. In VI Perl, these synchronous methods are exposed. There, you can choose which one to use. In vSphere Java API 2.0, these methods are exposed only in the stub layer but not the object layer. You don’t want to use stub methods directly when you can use objects, therefore I don’t talk much about it even in my book. Somehow I came across a question in the forum asking about this. So I think it may be good to share a little history and insight here.

The differences of these twin methods are minimal. They have exactly same parameters but different returns. The methods whose names include _Task suffix have Task returned. When you have the Task return, the operation may not yet be done at the server side. But with the Task object, you can track the progress, and even get the result data objects.

How You Can Use vSphere APIs to Collect vCenter and ESX Logs

October 20th, 2010 3 comments

If you manage a vSphere infrastructure, you may want to collect logs for troubleshooting, debugging, etc. You can get these logs from vSphere Client manually. You can also use vSphere API to collect them automatically.

The related managed object type in vSphere API is the DiagnosticManager. It helps to access logs from either a vCenter server or ESX server. It has no property but three methods:

1. queryDescriptions() provides a list of diagnostic files for a given system. It takes in an optional parameter host for specifying the HostSystem to extract information from. When you connect to the ESX server directly, the parameter isn’t needed. In vSphere Java API, you just pass in a null. When you connect to the vCenter server and the parameter isn’t specified, the method assumes you’re looking for vCenter logs. The return of this method is an array of DiagnosticManagerLogDescriptor data objects. The data object includes six properties: creator, fileName, format, info, key, and mimeType.

Categories: vSphere API Tags: , , , ,

Really Simple Tricks to Speed up Your CLIs 10 Times Using vSphere Java API

October 15th, 2010 4 comments

I recently had a short discussion with my colleague on implementing CLIs with vSphere Java API. One problem is that if you have multiple commands to run, each of them connects to the server and authenticate over and over. You’d better remember to logout the connection each time after you are done, or leave many un-used connections on the server that could significantly slow down your ESX or vCenter server (read this blog for details).

You can have two solutions to this problem. The first one is to have your own “interpreter” command. After you type the command, it shows you prompt for more sub-commands. It’s very much like the “ftp” command in that sense. You can have subcommands like “login” or “open” or “connect” for connecting to a server, and other commands. The “interpreter” command can then hold the ServiceInstance object until it’s closed in the end.

You can save about 0.3 to 0.5 second on creating new HTTPS connection and login for each command after the first one. It’s not a big deal given that vSphere Java API has hugely reduced that time from 3 to 4 seconds with Apache AXIS. So if you switch to vSphere Java API, you get instant 10 time performance gain. Still, if you have many commands to run, it could be a decent saving.

With this solution, you can also implement batch mode in which you can save all your commands into a file and then execute them all with one command. You can find many examples like PowerShell which support interactive mode and batch mode.

Another solution is just having normal commands. The problem becomes how to avoid the authentication for each command after the first. Luckily we have something for you in the API.

Categories: vSphere API Tags: , ,

vSphere Java API Exceeds 10,000 Downloads

October 7th, 2010 No comments

October 6, 2010, is a historical moment for VI Java API project – the total downloads exceeded 10,000. It’s two days earlier than I had expected. After yesterday’s blog on the NetApp and Brocade’s testimonials, the daily downloads suddenly doubled. When I found the stats approaching 10,000, I tweeted “vSphere Java API 9,999 downloads now. Who want to be No. 10,000?” I wish I could have been able to track who made the No. 10,000. :-)

Strictly speaking, the total had exceeded 10,000 a while back. Besides typical downloads, you can also directly sync up with the subversion. As I checked the number there, it had passed 1,000 reads early this year.

Thanks to you all, the vSphere Java API community!

10,000 downloads is not a big deal for an application especially when it’s for end users. It’s a big deal for an API, and even bigger for an enterprise API which requires vSphere environment which not every developer has access to.

Besides the download number, I would like to brag these numbers:

NetApp and Brocade Opt for Open Source vSphere Java API

October 6th, 2010 No comments

I am very pleased to welcome NetApp and Brocade to the vSphere(VI) Java API poweredby page. Many thanks to Patric Chang and Katie Colbert from Brocade, and George Costea and Eric Forgette from NetApp for making this happen.

NetApp and Brocade have been using open source vSphere(VI) Java API for quite some time and each has several products shipped with this open source API. As you may recall from my previous blog on VMworld 2010, I did not talk about NetApp and Brocade because I hadn’t got written permission even though they had great shows out there. Please feel free to check them out at VMworld in Copenhagen next week.

I think the key takeaway from this is that vSphere Java API has been stable enough to be used by companies like NetApp and Brocade that demand highest quality of products. For one thing, you can prabably afford not connecting to networks for a little while, but for sure cannot afford messing up your data storage. NetApp and Brocade’s confidence in this API is the best testimonial on the quality and readiness of the API. There are many other even bigger companies are using the API as well. I will talk more about them later.

A Workaround to Identify NFS based Datastore in vSphere

October 4th, 2010 2 comments

We just had the longest discussion in vSphere Java API forum regarding the “UUID of an NFS datastore.” The question is basically how to find the “UUID” via the vSphere API.

You can create datastores based on either VMFS or NFS. The VMFS can be backed up by local SCSI, or SAN (FC, iSCSI). It’s very easy to find UUID of a VMFS based datastore by calling getUuid() method from the corresponding data object VmfsDatstoreInfo.

For NFS based datastore, it’s a lot complicated. I am glad we digged to the bottom of the issue. Instead of going through the long discussion, I summarize the key takeways from the discussion.

Before jumping into details, let’s clarify one thing: NFS datastore does not have a UUID. (If you want to know more about UUID in vSphere, you should read this blog article.) You can check out the NasDatastoreInfo which does not have uuid property. It does, however, have an identifier like 73ca9790-6dbf88b0, which is not a UUID per se. We will call it simply an ID.


You may be wondering why you should care about the ID. It is pretty important in that it’s used in performance stats like the following: