After creating a light virtual appliance last year, Timo Sugliani continued with a full fledged version of virtual appliance with all you need for vSphere development with Java and Jython. This is what Timo called “my linux powershell toolkit.” The biggest advantage is that you are no longer limited by Windows as your development platform.
Examining logs is an important way for debugging and troubleshooting a system. There are about ten log files in the ESX server for the hostd agent, which listens API calls, with the same naming pattern as hostd-?.log under the /var/log/vmware directory. The hostd-index file has the number of currently used log files.
The log entry has a similar format to that of VC server logs. Following is a quick sample:
[2008-06-21 07:24:40.769 ‘SOAP’ 64834480 trivia] Received soap request from : checkForUpdates
The log level can be configured in the /etc/vmware/vpxa.cfg file. Just look for a section like the following. The possible levels are the same as those of VC logs: none, error, warning,info, verbose, or trivia, in an order from less to more detailed messages.
Task management is an important part of vSphere API. It enables asynchronous calls by returning a Task back before the work is actually done at the server side. If a task takes too long (for example, more than 15 minutes to create a virtualmachine snapshot from VirtualCenter), however, it can time out before it finishes. It means you lose tracking of the task.
You can easily fix this issue by changing the configurations. On the ESX server, add the following to the /etc/opt/vmware/vpxa/vpxa.cfg file:
<task> <timeout>7200</timeout> </task>
<vmomi> <soapStubAdapter> <blockingTimeoutSeconds>7200</blockingTimeoutSeconds> </soapStubAdapter> </vmomi>
Remember to restart the service:
Most people have the perception that vSphere Java API is for developers. It’s true but actually more than that. Administrators can take advantage of it as well.
Today, William Lam (@lamw) posted a Java version of his vdf tool which was originally written in Perl. For people don’t know William yet, he is a system administrator at Salesforce.com now and Yahoo before. He created and maintains the famous vGehtto script repository that almost every VI Perl developer knows about. William is also a vExpert and the No.1 contributor to the VMware developer community. Having not touched Java for 4+ years, William got his first HelloVM with VI Java API working in less than 5 minutes, and got the code converted in about one hour.
VMware PM Carter Shanklin (@cshanklin) once gave a great presentation on how to use Onyx with Java development at PEX 2010. I covered it briefly in a previous blog, and left out the “4 rules” hoping Carter would help.
As many of you have already known, Carter moved on to SpringSource division as the PM for tc server. So he has been pretty busy with his transition. With his coverage on both administrator and developer oriented products, he is right on the wave of devops movement. Make sure you follow him at Twitter.
In a previous post, I blogged about a tutorial on using the open source VI Java API in Groovy by Aaron Sweemer. Two weeks later, he wrote a new tutorial Easy vSphere Web Apps with Grails and the VI Java API.
In his new tutorial, Aaron introduced detailed steps in how to use the VI Java API for a simple web application with Grails framework. The web application displays a web page that lists all the virtual machines with the corresponding guest OS names and whether they support multiple snapshots.
Recently there were questions in the vSphere Java API forum on how to clone a new session from an existing one. Although vSphere Java API wraps around the basic Web Services API cloneSession() method, simply calling the method doesn’t get what you want.
Why? The signature of cloneSession() is as follows. It returns UserSession, which is embedded inside ServiceInstance managed object in vSphere Java API. While using the API, you always starts from the ServiceInstance.
public UserSession cloneSession(String cloneTicket) throws InvalidLogin, RuntimeFault, RemoteException;
In vSphere Java API, the ServiceInstance and UserSession have one to one mapping. Without a new ServiceInstance around the new UserSession, the UserSession is not much helpful.
Luckily, we got yet another contribution from Eric Forgette who works at NetApp. The contributed code includes a new overloaded method cloneSession() method to return a ServiceInstance object.
As I described before, we’ve been trying very hard to keep the vSphere Java API as close to the basic Web Services as possible. In this case, we decided to break the rule a little bit for good reasons, mainly for usability.
In a previous article Top 10 Best Practices Using VMware VI and vSphere SDK, I mentioned synchronous versus asynchronous calls in the second best practice “Choose Right APIs.” But no detail was provided there. In this article, which is based on my book VMware VI and vSphere SDK , I discuss all the details.
Some methods defined on managed objects in vSphere API are asynchronous, meaning they return right away whether the operations are done successfully or not. That makes sense for long-running operations; you don’t want to block your current thread by waiting for the return of the call, and you might want to cancel it before it’s done.
For these asynchronous methods, the VI SDK provides a way to track the progress and results after the invocation is returned. As a naming convention, a long-running asynchronous method has _Task as a suffix in the method name, and it returns MOR to a Task. With MOR pointing to the Task object, you can track the progress and even get the result of the operation. For example, the cloneVM_Task() method defined in VirtualMachine is a long-running method that returns MOR pointing to a Task managed object.
Every time I google for VI(vSphere) Java API, I get something new. Here is yet another one I just found. It’s a blog article Easy VMware Development with VI Java API and Groovy by Aaron Sweemer. By reading his blog site, I came to know Aaron is actually my colleague at VMware working as a Sr. System Engineer in Cincinnati Ohio. He is the principal blogger at Virtual Insanity.
With the growth of virtualization, a new term “virtual appliance” has been coined for a special type of virtual machines that are used like applications. What does it really mean?
First, a virtual appliance is still a virtual machine. When seen in vSphere Client, the virtual appliance does not look much different from other typical virtual machines. Secondly, the functionality of the virtual machine is limited to that of an application. More often than not, the virtual machine is installed with one application. Because of this, the underlying OS is stripped down only to the minimum required to support that application. This type of OS is also called Just Enough OS (JEOS). All the existence of the JEOS is to support the application in the virtual appliance.
Now, is it a VM or an application? It could be either, depending how you look at it. For ESX/vCenter, a virtual appliance is a virtual machine. You can manage it just like any other virtual machine. For application users, it’s an application, a special one that is different from a normal application.
Normally on the login dialog box, you enter a hostname or IP address. By default, the vSphere Client use HTTPS to communicate with the server. That means you cannot easily see what’s passed back and forth on the wire. As shown in the Onyx video, Carter showed how to use HTTP instead of the default HTTPS with the following in the IP Address / Name field:
So the vSphere Client does support HTTP. In Onyx case, it points to localhost on which the Onyx is installed. You can actually point to a real vCenter or ESX/ESXi server directly – just change the localhost to the IP address of the server and the port to the default port 80 or remove the port part.
Before connecting the server, you need to change the server a bit for it to support HTTP.
As I discussed extensively in my book, the PropertyCollector is very powerful yet not easy to use. There was a question posted at vSphere Java API forum related to the property collector which I think worths sharing here. Although it’s found using vSphere Java API, but it really goes beyond the API to the vSphere API itself.
Having answered many questions about IP addresses of virtual machines at different occasions, I still see more are coming. I think it’s time to write a blog about it. Hopefully people would search the Internet before raising the question.
First of all, there is a big confusion on the relationship of IP addresses and virtual machines. Many people tend to associate IP addresses with virtual machines, and want to retrieve/change the IP address of a virtual machine.
In fact, a virtual machine is very much like its physical counterpart. It does not have an IP address by itself. In other words, an IP address is NOT an intrinsic attribute of a machine, either virtual or physical. It might have one or more only after an OS is installed. In most cases, it does have one or more IP addresses, which gives the impression that every machine has an IP address.
A virtual machine does have intrinsic attributes such as MAC addresses if NIC cards are configured. Unlike its physical counterpart, a virtual machine’s MAC address can be re-configured. Some software vendors rely on MAC addresses to lock down their licensed software on particular machines. This mechanism can be, therefore, compromised in virtual environments.
While searching Twitter on “vSphere Java”, I found my presentation available today online at InfoQ (many thanks to @arm1433 and @toya256ForRSS). It has both video and slides for more than one hour. The voice was not quite clear in the first one or two minutes. After that it’s pretty good.
This presenation is a complete overview of the open source vSphere Java API. Because the audience then was new to virtualization, the first several minutes covered a little virtualization basics. You can scroll over if you know virtualization already.
With today’s global market, a software vendor has to consider the internationalization (I18N) issue to better serve users in different areas and maximize the return on the product investment. This article introduces the I18N basics of vSphere. Much of the content is based on my book VMware VI and vSphere SDK by Prentice Hall.
There are two basic meanings. First, you have to design your software so that it is localizable. In other words, you have to use the right APIs that can handle double byte characters. Sometimes people call this globalization (G11N).
Second, you should provide localized versions of your software so that users can read and use their native languages. Sometimes people call this localization.
In most cases, you externalize all the text strings that are visible to end users from the code to the resource files and translate them into different languages. Then localizing the software is as easy as combining the code and localized resource files. This is the way VirtualCenter server is localized. Depending on the programming language and platform, the resource files can be organized differently and might have another format. For example, Java uses properties files, yet C++ on Windows uses resource dlls.
That said, I18N is a broad topic that does much more than what is briefly covered here. Further discussion is beyond the scope of this book, but you can find more detailed information online.
As discussed, the VI SDK is essentially a set of Web Services interfaces. The WS-I18N summarizes four internationalization patterns that can be applied with Web Services when deployed.
This article is based on a similar one at vSphere Java API home page. At that time, one of VMware community members sent me an email for samples of using OvfManager APIs. Then I went to office on a Saturday writing two samples, which have been validated by several folks as “working” samples.
The purpose of the samples are to illustrate the vSphere APIs. Let’s take a look at them one by one.
First, ExportOvfToLocal.java. This sample shows how to download either a VM or vApp to your local machine. The typical flow is:
- Find the VM or vApp
- Call their exportVm() or exportVApp() methods and get HttpNfcLease
- Set lease time out
- Wait for HttpNfcLease until it’s ready
- From the HttpNfcLease.info property, find the all URLs from which you download the vmdk files
- Call OvfManager.createDescriptor() API to create the content of ovf and save it to a file along with downloaded vmdk files.
- Release the lease by calling httpNfcLeaseComplete() method
OVF stands for Open Virtualization Format, a platform independent, extensible packaging and distribution format for virtual machines. It’s now a DMTF standard.
VMDK stands for Virtual Machine Disk, a format that encodes a single virtual disk for a virtual machine. It’s proprietary by VMware but whose format is publicly documented by the company. You can use VDDK to manipulate the VMDKs.
I just took some time to read these scripts even though PowerShell and Perl are not for me. Here I give you a brief introduction of the scripts, what vSphere APIs they used directly or indirectly, and why they topped the contest. Because vSphere API is based on Web Services, you can port the scripts to other languages like Java, .Net, whatever you feel comfortable with. If you want to port any of them using vSphere Java API, I am more than happy to include your contribution.
Note that the following comments are strictly my own opinions.
1. Who Created that VM ? – by Alan Renouf using PowerCLI
A script to add information back into the vSphere client, this script which is designed to run once a day (or more) as a scheduled task, will add a custom attribute to each VM with the creator and date created of that VM. A script to add information back into the vSphere client, this script which is designed to run once a day (or more) as a scheduled task, will add a custom attribute to each VM with the creator and date created of that VM.
Nice integration with the vSphere Client, making you almost doubt why it wasn’t there in the first place. Additional one liner scripts provide nice answers to the questions like who created the most VMs, how many VMs were created each month.
This article introduces you the basic model and terminologies in vSphere security management, for example, privileges, permissions, roles, and how they are related to each other to secure vSphere. It helps you to better manage the vSphere and program the vSphere API. Much of the content is based on my book VMware VI and vSphere SDK by Prentice Hall.
In vSphere, the security model consists of three types of components: privileges, roles, and permissions.
A privilege is the basic individual right required to perform an operation. It is statically defined and never changes in a single version of a product. Given the many operations in VI, there are many privileges (for example, the privilege to “power on a virtual machine”). These privileges are represented as strings separated by dots, such as VirtualMachine.Interact.PowerOn.
The operations and privileges are not one-to-one mapping. Many operations do share common privileges like System.View. Therefore, there are many fewer privileges defined than methods. In some exceptional cases, a method requires different privileges depending on the target it operates on and the nature of the operation. The CloneVM_Task() method, for example, requires VirtualMachine.Provisioning.Clone for cloning from one virtual machine to another, VirtualMachine.Provisioning.DeployTemplate for cloning from a template to a virtual machine, and so on.
The role groups privileges from a user’s perspective. A role is normally named and defined for a group of people who have common responsibilities in the system (for example, administrators). Each role can include zero to multiple privileges. The extreme cases are the predefined “Admin” roles, which by default, includes all the privileges and the NoAccess role, which includes no privileges.
Performance monitoring is a critical aspect of vSphere administration. This article introduces you the basic concepts and terminologies in vSphere performance management, for example, performance counters, performance metrics, real time vs historical statistics, etc. Much of the content is based on my book VMware VI and vSphere SDK by Prentice Hall.
Once you understand these basics, the related tools and APIs should be relatively easy. If you are already familiar with vSphere Client performance monitoring or esxtop, they help as well.
A performance counter is a unit of information that can be collected about a managed entity. PerfCounterInfo data object, shown in Figure 1, represents a performance counter. The property key is an integer that uniquely identifies a performance counter, like a primary key of a table in SQL database, and nothing more. There is no guarantee for a performance counter to have a fixed number. In fact, the same performance counter can have different values in ESX and VirtualCenter. Even for the same type of server, the number could change from version to version. Do not use it outside the context of the server you connect to.
Figure 1 PerfCounterInfo data object
The performance counter can be represented by the following dotted string notation: