Category Archives: Software Development

Two Developers in VMware Community

Many folks talk about developer enablement today because it’s a key success factor for a platform company. If you haven’t watched this video by Microsoft CEO Steve Ballmer, you want to check it out. Also, my previous blog: CO2: The Formula For A Successful Developer Ecosystem.

To empower developers, we got to figure out who the developers are and what they want. It’s hard, if ever possible, to identify every developer in VMware community. But it’s normally easy to find out the types of the developers. In my observation, there are two types of developers (The title of this article is not that accurate, but

Also posted in Virtualization | Tagged , , , | Leave a comment

Use Java Technology to Fight Twitter SPAMs

I got quite a few SPAMs on my twitter last week. While it’s annoying to be spammed, I did pay a little attention to the spammers and noticed that their commonality: big number of tweets, 0 following, 0 follower.

These characteristics reminded me of the Java virtual machine technology to garbage collect unused Java objects. In Java, you don’t need to explicitly de-allocate memory as in C++. The Java garbage collector (GC) takes care of this for you. It looks at the objects to see if there is any reference to it. If not, it deletes the objects.

In the Twitter case, a follower is

Posted in Software Development | Tagged , | Leave a comment

A Gap In Object Oriented Methodology

There have been long debates on whether object oriented is the future of programming. Repeating it over here doesn’t make it any clearer. As you can tell from my blog, I am an OO bigot because it can significantly improve productivity. If you are not convinced about OO benefit, you can look around those top programming languages mostly support OO these days.

By reviewing the whole software lifecycle, however, you will find a gap between requirements and OO programming today. While describing application requirements, a business user almost always describe them in a number of steps (procedural). It’s not realistic to expect requirements described in an OO way. While developing, programmers write and see classes and objects.

How to bridge the gap between the procedural requirements and object oriented programming? It basically boils down how to

Posted in Software Development | Tagged , , | 2 Responses

Best Practice vs Never Practice

Today I read a commentary of latest MSDN magazine by David Platt who is famous for his book why software sucks. In his commentary titled Never, Never Land, he compared our software development to the medical industry on malpractices, and drew the following conclusion:

Certain occurrences are never, never, acceptable. We need to define these events, publicize them and educate developers about what they are and how to avoid them.

In IT world, we’ve heard many best practices. I have personally contributed many of them in using VMware vSphere APIscloud application development, and writing sample code. For consistency with best practices, I call these never land “never practices.” If you search the Internet with “never practice” today, you won’t get any but this post at the top. But we do need more never practices. In the article, David offered one general never: never lose users’ data, and he asked readers to share more.

Posted in Software Development | Tagged , , | Leave a comment

How I Customized My WordPress Blog

At the beginning of this year, I changed the theme of my WordPress based blog. Upon checking the site, several folks said it looked familiar and like In fact, when I searched for a new theme, I did use facebook as a keyword because I like the clean and simple look and feel. As you can check at the bottom of the blog, the new theme is called Crybook by Themetation and Crynobone, which you can hardly link the name with facebook except the word “book.”

The issues

After switching to Crybook, I really liked it but also found several minor issues with the theme. First of all, the “read more” link is not placed right in the overview of an article. Instead, it’s placed at its bottom. As a result, a reader may be confused where to read next. Secondly, it puts a line at the bottom at an article overview, and two links below the line. At a first impression, the two links seem more related to the next article than to the current one. Thirdly, it does not have the tags explicitly listed with the article as I expected. Tags are important to navigate through articles.

So I decided to customize the theme for my own preference which I hope leads to better reader experience.

Posted in Software Development | Tagged , , , | Leave a comment

Open Source In Action: Open Source Projects from VMware

As a leading edge software company, VMware has a long history of support for open source software in its products. It also contributes back many patches and projects to the open source community including the vijava API that I created. With SpringSource and Zimbra acquisitions, more open source projects are associated with VMware brand.

Here is a list of 10 home grown open source projects from VMware. Please feel free to click links for more details and play with them.

1. Dr. Memory. It’s “a memory monitoring tool capable of identifying memory-related programming errors such as accesses of uninitialized memory, accesses to unaddressable memory (including outside of allocated heap units and heap underflow and overflow), accesses to freed memory, double frees, memory leaks, and (on Windows) access to un-reserved thread local storage slots.”

2. Virtual USB Analyzer. A “free and open source tool for visualizing logs of USB packets, from hardware or software USB sniffer tools. As far as we know, it’s the world’s first tool to provide a graphical visualization along with raw hex dumps and high-level protocol analysis.”

Also posted in Virtualization | Tagged , | 2 Responses

Must Knows About Release Engineering: Lessons From Google

This is yet another post based on my notes taken at LISA 2010 conference. The talk is The 10 commandments in release engineering by Dinah McNutt from Google. Dinah did a great job in summarizing the basics of release engineering therefore it’s worthwhile to compile my note and share it here.

Note that although typical release engineering does not produce virtual appliances, the basic principles are the same. You will find these basics helpful as well.

Release engineering is a critical part of software engineering and should be treated as products in their own rights. But often there is disconnect between development writing the code and the system administrator who installs it. Release process is usually an afterthought.

Typical Release Process

The following steps are executed during a release run:

Posted in Software Development | Tagged , , , | Leave a comment

From Developer to Devops: What System Administration Skills Should You Know?

With the rising trend of devops movement, I was curious about the system administration from a software developer’s perspective. That’s why I sat through Adam Moskowitz’s session “The Path to Senior Sysadmin.” Adam summarized the system administrator’s skills to three categories: hard tech skills, squishy tech sill, and software skills as detailed in following. Again, this is based on my note taken from LISA 2010 conference. For other posts related to the conference, check here.

Hard Tech Skills

  • All the commands for system administration;
  • System backup;
  • Some programming skills like Shell scripting, Perl/Python, C (read);
  • Software engineering knowledge like versioning, process;
Posted in Software Development | Tagged , , , | 3 Responses

How Twitter Operates Its IT infrastructure: From Process to Tools

This post is based on my notes taken at the talk by John Adams at LISA 2010 conference. Any mistakes, if any, are all mine. Should you be interested in other sites, check out Google, Facebook, LinkedIn.

As one of the leading social Web site with 165M users, Twitter demands a huge infrastructure support its operation. There are 700M searches and 1,000 tweets per second and can go up to almost 4,000 at peak. The number of tweets is not that impressive, but these tweets need to be distributed to numerous followers which could be several millions after one account.

These days Twitter gets 75% traffic from API and 25% from the Web. The new Web interface heavily uses AJAX and acts as API client to its backend.

As John put it, “nothing works the first time.” His recommendation is to use the best available technology for scaling. You will need to plan and build for more than one time to get it right.

Posted in Software Development | Tagged , , , , | Leave a comment

Squares Aren’t Rectangles? A Common Misunderstanding of Object Oriented Design From MSDN Magazine

While reading the recent Dec 2010 issue of MSDN magazine, I found an article (Multiparadigmatic .Net, Part 4: Object Orientation) with misunderstandings on object oriented design. I was surprised that the author reached conclusions like, “squares aren’t rectangles,” and “no happy ending here.” The conclusions are based on misunderstandings of object oriented design.

Let me show you what the root problem is and how to get a happy ending. After reading this, you won’t be bothered by “squares aren’t rectangles.”

What’s the problem?

As most people already know, inheritance or generalization (I prefer the latter) is an important feature of OOD. Using it effectively can lead to a good object model and concise codebase. In an inheritance relationship, a subtype must maintain “IS-A” relationship with its super type, for example, a Student type IS-A Person. I think most people are just fine with this.

Posted in Software Development | Tagged , , , , , | 3 Responses

Really Simple Guidelines to Write Great Code Samples

When a developer learns a new programming language or API, the first thing is probably to try out a HelloWorld sample. As said, real programmers don’t read documents. Although I don’t fully agree on that, it has some truth in it.

In my own experiences, I normally continue with other samples after HelloWorld one. When something is not quite clear, I check out the API reference or read some tutorials. Anyway, I am not telling you how to learn a new language or API, but trying to make a point here on the importance of code samples for the developers. In my opinion, samples are the most effective way to empower your users.

I think you would agree with me, there are too many bad samples. Here are some typical symptoms:

  1. Too much boilerplate code to a point that the code illustrating the API usage got buried. Typical boilerplate code includes extensive exception handling, GUI, logging, etc. Some samples even have a common library that could confuse your users totally.
  2. Too many API calls in one sample. You may need several APIs for a use case, but don’t aim one sample for multiple use cases.
  3. Too much object oriented. Object oriented programming is a best practice for application development. But it could confuse your developers sometimes.
  4. Dependencies on other APIs. To run the sample, your users need to install other libraries which may or may not need extra configuration or tuning. To understand the sample, users need to understand additional APIs. Extra burden, really!
  5. Of course, typical bad smells of programming which are not unique for samples. For example, bad naming, unnecessary global variables, using object attributes for passing values between methods, etc.

Now, how you can develop great samples? Besides the best practices writing great applications, you want to follow the following guidelines:

Posted in Software Development | Tagged , , , , , , | 2 Responses

Wire Compatibility of Web Services

As a software professional, you may have heard about the source compatibility and binary compatibility. With the Web Services, a new type of compatibility came up. This is what I call wire compatibility. It’s not related to the programming but the XML messages passed on the wire. Since we don’t use XML directly but programming APIs, the wire compatibility surfaces and affects the source and binary compatibility.

Too abstract? You bet. Let’s pick up an example here. Because VMware vSphere API is defined in WSDL, I will use it in the following discussion.

In vSphere 4.1, the method PowerOnMultiVM_Task() gets an additional parameter called option typed as OptionValue array. The following are related parts in the WSDL:

<operation name="PowerOnMultiVM_Task">
  <input message="vim25:PowerOnMultiVM_TaskRequestMsg" />
  <output message="vim25:PowerOnMultiVM_TaskResponseMsg" />
  <fault name="RuntimeFault" message="vim25:RuntimeFaultFaultMsg"/>
<complexType name="PowerOnMultiVMRequestType">
    <element name="_this" type="vim25:ManagedObjectReference" />
    <element name="vm" type="vim25:ManagedObjectReference" maxOccurs="unbounded" />
    <element name="option" type="vim25:OptionValue" minOccurs="0" maxOccurs="unbounded" />

As you can see, the minOccurs of the option element is zero, meaning it’s optional. If you have an application built with 4.0 (no option parameter by then), the SOAP request still works. So it’s compatible on the wire.

Also posted in vSphere API | Tagged , , , , | Leave a comment

Critical Lessons Learned at Facebook on Scalability and Reliability is no doubt the biggest web site surpassing Google in terms of Web traffics in an article published half year ago. Given its scale, the lessons learned would be very helpful for others to build scalable IT infrastructures. This post is based on my notes taken at the talk by Robert Johnson and Sanjeev Kumar at LISA 2010 conference. Should there be any mistakes, they are all mine.

According to the speakers, the architecture of is relatively simple: Web servers in the front, databases at the back. In the middle is a caching layer with a lot of memcached servers. If you recall my previous post, they use PHP extensively.

Unlike other sites, like email sites, whose users are well mapped and isolated to different servers, social Websites like Facebook have unique challenges in that their users are linked together. Errors in one part of a system may cascade easily and bring down the whole site.

Here are several important lessons Facebook learned while building software and operating the site:

Posted in Software Development | Tagged , , , | 1 Response

What Lessons You Can Learn from Google on Building Infrastructure

Last week I attended a great talk by Google Fellow Jeffrey Dean at Stanford University. Jeff talked about his first hand experience on building software systems at Google since 1999 and lessons learned. The following summary is solely based on my notes, therefore may contain my misunderstandings.

A Brief History

During the past 10 years or so, the scale of the Google infrastructure has grown exponentially: # docs 1,000X; #query, 1,000X; per doc index, 3X; update rate from months to seconds, 50,000X; query latency, 5X; computer and computing powers, 1,000X. The underlying infrastructure has experienced 7 major revisions in the last 11 years.

At the concept level, the search infrastructure is simple. It has web servers upfront taking search queries. The queries are then passed on to two different types of servers: index servers and doc servers. For the index server, the input is the query string and the output is an array of doc-id and score pairs. For the doc servers, the input is the doc-id and query pair and the output is the title and snippet of the doc. Note that the snippet of the doc is query dependent so that you can find your keywords highlighted in the result pages. How to quickly and accurately calculate the output based on input involves a lot of advanced algorithms, and is not in the scope of Jeff’s talk.

Posted in Software Development | Tagged , , | 1 Response

Top 3 Trends Every IT Professional Should Care About

IBM DeveloperWorks recently published the result of a survey of 2000 IT professionals excluding IBM employees. The key findings are:

  1. Cloud Computing to overtake on-premise computing. For the question, “how do you rate the potential for cloud computing to overtake on-premise computing as the primary way organizations acquire IT by 2015?” 30.4% said likely, 21.6% most likely, and 13.6% definitely.
  2. Mobile application development to dominate. 55% of respondents see app development on mobile grows than other platforms in 5 years.
  3. IT professionals need, but often lack, industry-specific knowledge. 28.3% thought moderately important, 45.6% very important, and 15.9% extremely important. This is not an IT trend per se, but represents the demands for IT professionals.

The first two findings are mentioned and somewhat confirmed by another survey by Forrest and Dr. Dobbs, which is more developer oriented:

Also posted in Cloud Computing | Tagged , , , , , | Leave a comment

Code2Cloud Reborn With a Greater Purpose

If you attended last year’s VMworld keynote by Steve Herrod or watched the online broadcasting, you may still recall the website (see the top banner here). That was a very simple Web application meant for the keynote attendees to submit names and email addresses to win a chance to go to the backstage with the Foreigner band. The website was hosted at Terramark vCloud and continued to run for about one month afterward.

Also posted in Cloud Computing | Tagged , , , | Leave a comment

Google made Instantiations free

Following my prediction in a previous blog, Google just made the tools from its recently acquired Instantiations free to all developers/testers. Besides goodwills to the developer community, this move makes GWT a serious competitor to Adobe Flex, Microsoft Silverlight for Rich Internet Applications.

You can give these tools a try while developing and testing Web applications, or Java GUI applications:

  • GWT Designer
    “a powerful set of Eclipse-based development tools that enables Java developers to quickly create Ajax web applications using the Google Web Toolkit (GWT).”
Also posted in News & Events | Tagged , , , | 1 Response

Eclipse Day at GooglePlex: A Brief Trip Report

I attended Eclipse Day at Google headquarter today. It’s a great event packed with several great talks.

Mobile application is for sure a hot topic these days. There were several talks on mobile application development like Android Tools for Eclipse by Xavier Ducrohet from Google, Eclipse Sequoyah for Android App Developers by Eric Cloninger from Motorola, Tools for Mobile Web by Paul Beusterien from Symbian Foundation, and EMF for GWT by Ed Merks from Cloudsmith.

While sitting in the talk on Instantiations Eclipse Tools, I learned that Google bought the company 3 weeks ago. I expect Google will soon make WindowsBuilder, a famous WYSIWYG GUI authoring tool, free for GWT users, therefore further drive the adoption of GWT in competition with Flex, SilverLight.

Also posted in News & Events | Tagged , , , , , | 1 Response

Process or Principle: Which is More Important for Software Development?

While reading Who Says Elephants Can’t Dance by Lou Gerstner, the following paragraph caught my attention and made me think about it in the context of software development.

We started with a statement of principals. Why principals? Because I believe all high-performance companies are led and managed by principles, not by process. Decisions need to be made by leaders who understand the key drivers of success in the enterprise and then apply those principles to a given situation with practical wisdom, skill, and a sense of relevancy to the current environment.

Although managing a company and managing a software project are two different things, I believe above paragraph is perfectly true for software development as well. Let’s paraphrase it:

We started with a statement of principals. Why principals? Because I believe all high-performance software development teams are led and managed by principles, not by process. Decisions need to be made by leaders who understand the key drivers of success in the software development and then apply those principles to a given situation with practical wisdom, skill, and a sense of relevancy to the current project.

Sounds true? For your curiosity, the principles Mr. Gerstner wrote for IBM in 1993 were:

Posted in Software Development | Tagged , , | Leave a comment

Which Web Framework is the Best?

Web based applications become more and more popular. Not only normal Web sites but also enterprise management systems are adopting this to deliver the functionalities due to the benefit of zero installation on the user’s side.

There are a lot of Web Frameworks today, probably more than anyone can grasp. For each programming language, there is one or more Web frameworks that help you to create Web based applications. Choices are definitely good but may give you a hard time to decide which one is best for your project. More often than not, there is no single best one that suits all your need. In other words, you have to decide the best in the context of your problem.

What should be in the thought process to a decision?

  1. Programming language. You have to decide what programming language to use for your Web applications. With the preferred language in place, you can only use the frameworks supported by the language. This usually limits much fewer options to your list. The most popular programming languages for Web applications are PHP, Java, C#.
    The choice of your programming language is not a choice sometimes. The most rational choice is to use a programming language you and your team are already familiar with. If that happens to be C, you then want to go down your list. Learning a new programming language and a new framework can be daunting. Not long ago, I tried to learn the Lift framework based on Scala language, and found it’s not that easy at all.
    If it’s a team project, you have to find one language that all or most people are comfortable with and at least one person are good at.
Posted in Software Development | Tagged , , , , | 6 Responses

    My company has created products like vSearch ("Super vCenter"), vijavaNG APIs, EAM APIs, ICE tool. We also help clients with virtualization and cloud computing on customized development, training. Should you, or someone you know, need these products and services, please feel free to contact me: steve __AT__

    Me: Steve Jin, VMware vExpert who authored the VMware VI and vSphere SDK by Prentice Hall, and created the de factor open source vSphere Java API while working at VMware engineering. Companies like Cisco, EMC, NetApp, HP, Dell, VMware, are among the users of the API and other tools I developed for their products, internal IT orchestration, and test automation.