Book Review: 6 Musts of Intelligent System Automation

January 26th, 2011 1 comment

I recently came across a book which I got from LISA conference last November. It was written by Jake Sorofman who is VP, Product/Marketing at rPath. I thought it’s just another typical book from a vendor, therefore I didn’t read it until two weeks ago while sorting my bags. It’s not. After reading it, I found this 75 page booklet pretty easy to read and very insightful.

Here are 6 musts Jake discussed in his book:

  1. Get your platforms under control
  2. Get your applications under control
  3. Version control everything

How You Can Use VI Java API in Scala: A Quick Sample

January 25th, 2011 No comments

As I blogged before, you can use open source VI Java API in all JVM languages like Jython, Groovy, JRuby and frameworks like Grail. Today Dave Briccetti (@dcbriccetti) sent me a sample showing how to use it in Scala. Dave has been a happy user of the API since he started to work at VMware as an independent consultant more than a year ago.

Here is the code sample:

Cloud Architecture Patterns: VM Template

January 24th, 2011 No comments

Intent

Standardize new virtual machine provisioning with templates

Category:

Creational pattern

Problem

It’s been a pain to create new virtual machines with the right software installed and configured properly. You can always use tools like KickStart to automatically install the operating system and then install other software as needed. But configuring such an environment is not trivial, and it takes a long time from start to finish.

With the rise of virtualization, more virtual machines are provisioned (and decommissioned) than ever before. Installing each new virtual machine from scratch is not the ideal solution.

Solution

How to Delete Virtual Machine With vSphere API?

January 18th, 2011 3 comments

This question was asked at VI Java API forum recently and has been answered by the community.

There should be a “Destroy_Task()”-Method for each Managed Object, so also Virtual Machines…. Cheers, Joerg

As  Jörg Lew has correctly pointed out the solution, I would like to elaborate a bit more here.

If you are using VI Java API, the method name is destroy_Task(). The code is as simple as:

Difference of Two Common Privileges in vSphere API

January 17th, 2011 No comments

If you have used vSphere API and read its API Reference, you may have noticed two most commonly used privileges: System.View and System.Read. They are required in many methods. As their names suggest they are different, but what is the difference? It can be confusing for some people including me initially because it’s nowhere documented.

Here are some explanations after my talking to my colleague Jianping Yang who is the vCenter DB and Security guru.

Categories: vSphere API Tags:

HP Chooses Open Source vSphere Java API

January 12th, 2011 No comments

After NetApp, Brocade and many other companies showing up on the VI Java API poweredby page, we now have the world’s biggest technology company HP officially listed. HP started to use VI Java API about two years ago and has shipped products ever since. I have attended several meetings called by my colleague Lucas Nguyen to work with HP architect Zachary Speck and his team. Included in the following short paragraph is a link to the HP plugin product. Please feel free to check it out.

Categories: vSphere API Tags: ,

Quickest Way to Support SaaS Cloud for ISVs

January 10th, 2011 2 comments

One challenge almost every ISV faces today is how to quickly get “cloudy” with least investment. From business side, SaaS is a new model for most ISVs, meaning it’s still hard to project revenue. Even worse, the SaaS service may cut into the license revenue of existing product. Technically, it may require a full re-design and re-implementation of product  for maximum multi-tenancy. Putting together, it’s a big challenge for decision makers.

But it doesn’t have to be difficult with a right strategy. This post explains how to support SaaS without re-designing existing applications, which means a big saving for the ISVs.

How to Be a Smart IT Customer?

January 5th, 2011 No comments

This is the last note I took from LISA 2010 conference. It’s a great talk by Loren Jan Wilson drawing his experience with vendors while working at a super computer center.

The super computer, Intrepid, consists of 40,960 nodes on 40 racks. Each node has 4 core CPU. Of all the nodes, 640 nodes are dedicated for I/O. There is no local storage at each node. The super computer links to a very large tape library for archiving.

While operating the super computer, the speaker had some issues with high-speed network switches, e.g. 6% random port death, 15% quad port flaky but never fail 100%. To complicate the issue, there is no log and CLI for troubleshooting, but Web interface.

I believe the trouble the speaker faced before is not a single case in the industry, and never be. As long as you have to buy equipment/software from vendors, there will be issues one way or the other. A great thing the speaker did is to summarize and share the tips on how a customer should work with an IT vendor for a successful IT project.

I find these tips are very helpful, and think customers and vendors should all know about them as listed below:

Cloud Architecture Patterns: Cloud Broker

January 3rd, 2011 No comments

Intent

Provide a single point of contact and management for multiple cloud service providers and maximize the benefits of leveraging multiple external clouds.

Category

Structural

Motivation

When you are buying and selling stocks or other securities, you hire a broker to execute the trade on your behalf. One reason for that is convenience. You don’t need to take care of the details of placing orders and working with multiple stock exchanges, and whatever else is required to trade securities.

How about working with multiple cloud service providers? For sure, you can go online to any cloud provider as long as you have your credit card ready. But is the service provider the best fit for your requirements? Do you have a backup plan if you are not satisfied with your service provider? Can you easily switch among your service providers to minimize cost or maximize flexibility? If you are not sure, you may then need something like a cloud broker.

Solution

Top 5 Predictions on Cloud Computing for 2011

December 29th, 2010 No comments

While 2011 is coming soon, many technologists and medias are busy with predictions for 2011. I got an email from the chief of Cloud Computing Journal Jeremy Geelan (@jg21) for my predictions. Here are my thoughts on the cloud computing for 2011 and beyond:

  1. The focus of cloud computing will gradually shift from IaaS to PaaS which becomes key differentiator in competition. Developer enablement becomes more important than ever in ecosystem evangelism, full software lifecycle integration, IDE support, API and framework, and etc.
  2. Many more mergers and acquisitions (M&As) will take place in cloud space for companies to build stronger cloud portfolio. For big players, it should include dual vertically complete stacks both as services and products. Whoever gets there first will gain enormous advantages over its competitors.

Join Me at Partner Exchange 2011

December 22nd, 2010 2 comments

VMware Partner Exchange takes place twice a year. One happens at the same time/location with VMworld US; the other in places like Las Vegas, Orlando. It’s a dedicated conference to educating and enabling partners for success with VMware. It has merged with Technology Exchange where you can find many technical presentations. I have been speaking at TechnologyExchange since I joined VMware in 2007. Here is the related articles I wrote earlier.

The coming ParterExchange will be in Orlando FL from Feb 7 to 11. Please join us to hear VMware’s plans for the coming year, learn of new technologies and partner programs, and understand the training roadmap. Here is the content catalog with all the sessions. Don’t forget the famous hands-on labs throughout the week. I will talk about securing vSphere infrastructure with vSphere API.

Open Source In Action: Open Source Projects from VMware

December 21st, 2010 2 comments

As a leading edge software company, VMware has a long history of support for open source software in its products. It also contributes back many patches and projects to the open source community including the vijava API that I created. With SpringSource and Zimbra acquisitions, more open source projects are associated with VMware brand.

Here is a list of 10 home grown open source projects from VMware. Please feel free to click links for more details and play with them.

1. Dr. Memory. It’s “a memory monitoring tool capable of identifying memory-related programming errors such as accesses of uninitialized memory, accesses to unaddressable memory (including outside of allocated heap units and heap underflow and overflow), accesses to freed memory, double frees, memory leaks, and (on Windows) access to un-reserved thread local storage slots.”

2. Virtual USB Analyzer. A “free and open source tool for visualizing logs of USB packets, from hardware or software USB sniffer tools. As far as we know, it’s the world’s first tool to provide a graphical visualization along with raw hex dumps and high-level protocol analysis.”

Cloud Architecture Patterns: Service VM

December 20th, 2010 No comments

Intent

Provide an easy way to provision new infrastructure and application services for a computing cloud

Category

Behavioral

Problem

To run a large-scale computing infrastructure, you will need many different types of services, including compute, storage, and networking, among others. After virtualization has successfully detached compute from the physical hardware, it’s very easy to provision and scale compute. But compute requires storage and networking which are lagging behind. To maximize the benefits of virtualization and cloud computing, it’s natural to push the storage and networking in the same direction.

Looking beyond the infrastructure to consider applications, we need various types of services such as database, directory, messaging, and more. I’ve covered the App VM pattern that allows using IaaS for PaaS in a previous blog. While you can pack some of these services into an application VM, the problem is that it scales well but does not follow the aspectual centralization pattern.

Solution

Must Knows About Release Engineering: Lessons From Google

December 15th, 2010 No comments

This is yet another post based on my notes taken at LISA 2010 conference. The talk is The 10 commandments in release engineering by Dinah McNutt from Google. Dinah did a great job in summarizing the basics of release engineering therefore it’s worthwhile to compile my note and share it here.

Note that although typical release engineering does not produce virtual appliances, the basic principles are the same. You will find these basics helpful as well.

Release engineering is a critical part of software engineering and should be treated as products in their own rights. But often there is disconnect between development writing the code and the system administrator who installs it. Release process is usually an afterthought.

Typical Release Process

The following steps are executed during a release run:

From Developer to Devops: What System Administration Skills Should You Know?

December 14th, 2010 2 comments

With the rising trend of devops movement, I was curious about the system administration from a software developer’s perspective. That’s why I sat through Adam Moskowitz’s session “The Path to Senior Sysadmin.” Adam summarized the system administrator’s skills to three categories: hard tech skills, squishy tech sill, and software skills as detailed in following. Again, this is based on my note taken from LISA 2010 conference. For other posts related to the conference, check here.

Hard Tech Skills

  • All the commands for system administration;
  • System backup;
  • Some programming skills like Shell scripting, Perl/Python, C (read);
  • Software engineering knowledge like versioning, process;

Cloud Architecture Patterns: VM Pipeline

December 13th, 2010 No comments

Intent

Provide a configurable structure for modularized information processing

Category

Structural

Problem

Complicated data processing involves many distinctive and repetitive steps. Each of these steps can be mapped to a software module that is independently developed and assembled for particular cases of data processing.

Given the elastic nature of cloud computing, it’s a perfect platform for data processing. We need a solution that is flexible in two ways:

1.     Modularized components for data processing;

2.     Configurable so that different modules can be re-used easily in various cases.

Solution

Gadget Talk: What Are the Newest and Most Innovative Gadgets?

December 9th, 2010 No comments

I attended SDForum emerging technology SIG meeting on Wednesday night I. It’s about cool gadgets by award winning tech journalist and entrepreneur Fred Davis, who was part of Weird, CNET, Ask Jeeves, Ziff-Davis publishing on PC Magazine, PC Week, MacUser, A+. The following are gadgets Fred talked about in his presentation. It’s based on my note which may not be accurate. Hopefully it gives ideas what to buy for the holidays.

  1. iPod Touch. It’s like an iPhone but no phone support. You can use Skype with WIFI however. Price: $189 with 8G
  2. iPod Nano. Small music player with price about $139.
  3. iPod Shuffle. Smallest music player in the iPod family. Price: $49
  4. iPad. This is hottest consumer devices that created a record on fastest adoption. Price: $499
  5. iPad keyboard/wireless keyboard.
  6. Kindle Wifi 3G. It’s an e-book reader from Amazon. Compared with other readers or general purpose devices, it comes with no color but best selection of books and digital right management. $189
  7. MiFi 2200 mobile hotspot. It uses Verizon 3G to provide wireless connection to 5 WIFI enabled devices. Price: $269 and could be cheaper with iPad bundle.
Categories: News & Events Tags: , ,

How Twitter Operates Its IT infrastructure: From Process to Tools

December 6th, 2010 No comments

This post is based on my notes taken at the talk by John Adams at LISA 2010 conference. Any mistakes, if any, are all mine. Should you be interested in other sites, check out Google, Facebook, LinkedIn.

As one of the leading social Web site with 165M users, Twitter demands a huge infrastructure support its operation. There are 700M searches and 1,000 tweets per second and can go up to almost 4,000 at peak. The number of tweets is not that impressive, but these tweets need to be distributed to numerous followers which could be several millions after one account.

These days Twitter gets 75% traffic from API and 25% from the Web. The new twitter.com Web interface heavily uses AJAX and acts as API client to its backend.

As John put it, “nothing works the first time.” His recommendation is to use the best available technology for scaling. You will need to plan and build for more than one time to get it right.

Squares Aren’t Rectangles? A Common Misunderstanding of Object Oriented Design From MSDN Magazine

December 5th, 2010 3 comments

While reading the recent Dec 2010 issue of MSDN magazine, I found an article (Multiparadigmatic .Net, Part 4: Object Orientation) with misunderstandings on object oriented design. I was surprised that the author reached conclusions like, “squares aren’t rectangles,” and “no happy ending here.” The conclusions are based on misunderstandings of object oriented design.

Let me show you what the root problem is and how to get a happy ending. After reading this, you won’t be bothered by “squares aren’t rectangles.”

What’s the problem?

As most people already know, inheritance or generalization (I prefer the latter) is an important feature of OOD. Using it effectively can lead to a good object model and concise codebase. In an inheritance relationship, a subtype must maintain “IS-A” relationship with its super type, for example, a Student type IS-A Person. I think most people are just fine with this.

vSphere Performance Counters for Monitoring ESX and vCenter

December 3rd, 2010 11 comments

VMware vSphere provides comprehensive performance metrics for your needs on performance monitoring and diagnosis. These stats are available through not only vSphere Client but also vSphere APIs. To understand the overall performance management concepts, you want to read this article: Fundamentals of vSphere Performance Management.
Once having the basics, you may wonder what types of stats are exposed. The following table summaries all the 315 performance counters available in vSphere 4.1. As you might have guessed, the information is generated using open source Sphere Java API and then imported into WordPress using WP-Table Reloaded. You can easily sort and search the table.

Update: Carter Shanklin and Luc Dekens have articles on performance counters as well: