Comments (4)

VMware vSphere (as known as VI in earlier versions) SDK includes a comprehensive set of APIs managing the vSphere CloudOS. It can be used to build different types of applications: standalone GUI applications, vSphere Client plug-in, utility tools, Web applications, server applications. It is becoming increasingly important as more and more enterprises become 100% virtualized with vSphere.

This article intends to summarize the best practices for developing applications using VI and vSphere API. I only cover these specific to the applications built on top of VI and vSphere API. The generic best practices for application development are still critical to the success of your project, but not in the scope of this writing.

There are many best practices for each part of the APIs, for example networking, storage, event management, etc. We cannot cover them all here, but pick the top 10 that matter to most developers the most as follows.

#1 Connect to vCenter Server Whenever Possible

There are so many reasons for this to be #1 best practice all in all. This morning I got an email from my colleague who has a client with 10,000 VMs to be managed. They want to collect the performance data of VMs but concern the vMotion. How to track the VMs and get performance data when some of them move from one host to another? The easiest and best way to do it is this #1.
Besides the above, there are several advantages in general:

• Much more functionalities than ESX or ESXi.
• Less management with single connection than many sessions to each ESX/ESXi a vCenter manages.
• Easy tracking of virtual machines when they move around in your virtual data center.
• Aggregated information only available in vCenter.
• No worry about the versioning where different ESX/ESXi servers are present. The vCenter takes care of the older versions of hypervisors.
• There isn’t much performance advantage by connecting to individual hypervisors since 2.5.

#2 Choose the Right APIs

VI SDK 2.5 has 375 methods and 63 managed object types. vSphere addes additional 23 types. There could be more than one way to get things done. You have to think over these while choosing the right API:

• Synchronous vs Asynchronous
• Pull vs Push
• Single vs Batch processing
• Versioning

Also, you want to avoid Deprecated APIs. The VI and vSphere products evolve over time, so do the APIs. Some of the APIs get deprecated when new releases come out. There won’t be any immediate impact on your application, as the deprecated APIs are still usable. Over the time, these deprecated APIs will gradually phase out. So be proactive in preparing for the changes.

#3 Handle Faults Carefully

VI SDK 2.5 has more than 200 fault types. Each represents a type of failure, and holds information for further analyzing and debugging. Catching the faults makes your application more robust. Note that the intension of RuntimeFault is after the RuntimeException in Java meaning you don’t have to catch it. But at the client side, you do have to catch it as every fault type inherits from RemoteException directly or indirectly.

#4. Collect Only What You Really Need

Over collecting data hurts your performance and put burden on the server. This is especially true when you connect to vCenter which handles a lot of communication, data persistence and management work. Without your API clients, it’s mostly busy already. You don’t want to over burden it and slow down things.

Here is what you can do to restrict the collection:

• Retrieve only what’s must-to-have, not nice-to-have.
• Use property path to limit the nested data objects
• Be conservative with recursion with TravelSpec
• Collect only once the static data which doesn’t change at all or doesn’t change during a session lifecycle.

# 5 Use PropertyCollector and Task with Extra Caution

The PropertyCollector is a very unique managed object in the SDK. It’s super powerful but also very hard to use. A lot of questions in the community are related to it. The learning curve is pretty high. Even for the experienced developers, it’s fairly easy to be stuck with advanced features like recursive search.

Even so, there is no escape from this PropertyCollector because it’s almost the only way to get properties from any managed object. If you don’t use toolkits as recommended, you have to use the API directly.

Task is another important and commonly used managed object. Quite some (50+) methods return immediately a Task object, using which you can track the progress. A task object has very limited life time with about 10 minutes. After that, you cannot make any call to it directly or indirectly (for example, using the PropertyCollector to get its property). You have to use HistoryCollector for any historical tasks.

Stay tuend for next part with #6-10…

Author: Steve Jin is the author of VMware VI and vSphere SDK (Prentice Hall), creator of VMware vSphere Java API. For future articles, please subscribe to RSS or Email, and follow on Twitter.

Comments (2)

Besides the talks on VMware APIs, two self paced labs are there for you to try out the vSphere SDK and PowerShell at PEX. More details are quoted in later part.

As a bonus, you will get a chance to learn VI Java API in the SDK lab. Thanks to VMware TAM Alton Yu for making this happen! I will be there when I am not presenting my talk “Architecting Your Applications for VMware Cloud”, or helping on the BOF, genius bar, etc.

Note that pre-registration for Self-Paced Labs is not available. Lab attendance is on a first-come, first-served basis. Drop by to attend a lab in room Mandalay D. Lab hours are available in the onsite pocket guide.

SPLIBP1020 Intro to vSphere Web Services API (i.e. SDK)

This lab will provide training on how to integrate your products with VMware environment. Attendees will get an opportunity to try out the tools provided via vSphere Web Services SDK. These tools perform a variety of functions related to automation, integration, management and extension of VMware environments and virtual machines. Participants will have their choice of using Java or C#. You will walk away from the lab with a better understanding of vSphere, resources for using the SDK, and the essential skills for building the most common types of integrations. Exercises will cover a variety of scenarios such as examining VM attributes, performing the same task on many VMs at a time and triggering tasks based on policies. We will also touch upon performance enhancements and exporting data to enable this. This lab requires some basic programming experience, ideally in C# or Java.  

SPLIV0002 Managing the vSphere Infrastructure with PowerShell

This lab will provide hands on training for scripting a VMware environment. The goal of this lab is to provide all the tools needed to automate, integrate, and extend vCenter for your environment. Participants will use the Microsoft PowerShell based PowerCLI for scripting Virtual Center and ESX.

Author: Steve Jin is the author of VMware VI and vSphere SDK (Prentice Hall), creator of VMware vSphere Java API. For future articles, please subscribe to RSS or Email, and follow on Twitter.

Comments (0)

Search has been a hot topic since Google successfully monetized it with advertising business model. Besides general Web content search like Google does, there are many other types of searches needed for other Internet companies.

LinkedIn.com, with which most of us have created profiles, offers search capabilities based criteria like keywords, names, location, industry, companies currently with and before, school, etc. Because what Linkedin.com has is well structured data, you and I expect it to do a better job than Web searches. In fact, it does.

But how?

With the curiosity, I attended a SDForum SIG seminar this evening by John Wang (try your search with Linkedin to see if you can find him), who is the Search Architect in Linkedin.com. He introduced how they use open source Apache Lucene for their search project. Along the way, they created a real-time search and indexing system based on Lucene called Zoie , and an extension to Lucene for faceted search called Bobo-browse. Both of these two projects are open sourced and hosted at Google code.

Several interesting points from the seminar are:

• Linkedin search result is “no cheating.” It has exact results. Having tried Google or other search engines, you may notices that the number of hits is actually an estimate at best. When you click along the way after 30 pages, you will see significantly smaller number of hits than claimed on the first page. It doesn’t  mean it blows the numbers, but most likely does not expect you go that far.
• Their search system is distributed with 5 million profiles per partition. A broker dispatches the queries and aggregates the results. Each node has two partitions which are replicated to other servers in the cluster. They have 8G memory in each node, but don’t use that much at all, just for a peace of mind in case network traffic surges. This sounds like a good reason to get virtualized for dynamic resource allocation.  
• Their system gets about 15 M queries per day. Do you know who made the most queries? The paid recruiters. Some of them can use queries with 100 conditions to nail down the right candidates. The peak to trough ratio is 5. The weekly busiest time is on Monday morning, pacific time. Again, it’s when most recruiters start to find new candidates to work on for the week.
• A special search called reference search allows you to find people in your network who have overlaps with one person. You can use it searching references for a candidate as an employer. Or you can use it search connections to the hiring manager as a candidate. That is the good side. The bad side is it’s not free. To use it, you have to upgrade your account. Technically there are quite some challenges in implementing reference search to find out the time overlaps.
• The communications between their distributed nodes using RPC are slower than the searches. Kind of surprise to know. So they are going to move to Google protobuf, which is, as claimed at project home page, used by Google “for almost all of its internal RPC protocols and file formats.” From its face value, it’s very much like the Thrift project that was initially created by Facebook, and is now part of Apache.
• Last, but not least, has something to do with cloud computing. They did a little try with Amazon, but their data is too sensitive to pass back and forth in the public cloud. I think they are not alone on this with strong concerns on securities of public cloud. That is where private cloud can come in to help.

More information on their projects can be found at http://sna-projects.com/sna/. They are hiring, BTW.

Note: This blog is based on my note and memory solely.

Author: Steve Jin is the author of VMware VI and vSphere SDK (Prentice Hall), creator of VMware vSphere Java API. For future articles, please subscribe to RSS or Email, and follow on Twitter.