Tag Archives: scalability

Massive Scheduling with ScheduledExecutorService in Java

If you need to do certain things at certain points of time or intervals, you need scheduling capability. Don’t confuse the scheduling with multi-threading even though you can use multi-threading for scheduling but they are not equivalent. You can use single thread for many different tasks. Simply put, threads are executors, and tasks are jobs to be done by executors.

Posted in Software Development | Also tagged , | 3 Responses

What Is Missing in Current Software?

If we look closely at the software today, we will find some important pieces missing. For example, the software code defines logical behaviors of a system, but not the performance and scalability aspects. In other words, the operational aspects of the software are not clear even if you have a software product.

Posted in Software Development | Also tagged , | Leave a comment

Book: The Art of Scalability

Just finished reading the book The Art of Scalability: Scalable Web Architecture, Processes, and Organizations for the Modern Enterprise by Martin Abbott, Michael Fisher. The book draws on authors’ experiences working at PayPal/eBay and other Internet companies, and covers many aspects of scalability including people, organization, process, and technology. According to Yishan Wong, who used to work under the authors and is now an engineering director at Facebook, “the opportunity to directly absorb the lessons and experiences presented in this book is invaluable to me now working at Facebook.”

Posted in Cloud Computing | Also tagged , | 5 Responses

How Twitter Operates Its IT infrastructure: From Process to Tools

This post is based on my notes taken at the talk by John Adams at LISA 2010 conference. Any mistakes, if any, are all mine. Should you be interested in other sites, check out Google, Facebook, LinkedIn.

As one of the leading social Web site with 165M users, Twitter demands a huge infrastructure support its operation. There are 700M searches and 1,000 tweets per second and can go up to almost 4,000 at peak. The number of tweets is not that impressive, but these tweets need to be distributed to numerous followers which could be several millions after one account.

These days Twitter gets 75% traffic from API and 25% from the Web. The new twitter.com Web interface heavily uses AJAX and acts as API client to its backend.

As John put it, “nothing works the first time.” His recommendation is to use the best available technology for scaling. You will need to plan and build for more than one time to get it right.

Posted in Software Development | Also tagged , , , | Leave a comment

The Mythical Sessions in vSphere and VI

In my previous blogs, I talked about session management for scalability and best practices (#9). In this one, I am going to drill down to the bottom.

To your surprises, there are two types of sessions involved in vSphere SDK:

  • HTTP Session. It’s used to identify a client and tracked by the cookie in HTTP header. Once you login the server, all the successive requests have to carry the cookie header similar as follows


  • User Session. It’s used to identify a login session of a particular user. You can use SessionManager to find out more the details of the current user and other login users from the UserSession data object. The key in the UserSession is in the same format as the HTTP session, but you should never confuse them, or use them interchangeably.
Posted in vSphere API | Also tagged , , , , , , , , | 8 Responses

    My company has created products like vSearch ("Super vCenter"), vijavaNG APIs, EAM APIs, ICE tool. We also help clients with virtualization and cloud computing on customized development, training. Should you, or someone you know, need these products and services, please feel free to contact me: steve __AT__ doublecloud.org.

    Me: Steve Jin, VMware vExpert who authored the VMware VI and vSphere SDK by Prentice Hall, and created the de factor open source vSphere Java API while working at VMware engineering. Companies like Cisco, EMC, NetApp, HP, Dell, VMware, are among the users of the API and other tools I developed for their products, internal IT orchestration, and test automation.