Decomposition and Challenges in Parallel Programming: Is It Useful for Cloud Computing?

A recent article from Dr. Dobb’s introduced Fundamental Concepts of Parallel Programming. Richard Gerber and Andrew Binstock, authors of Programming with Hyper-Threading Technology, discussed three different forms of de-compositions for multi-threading:

  1. Functional decomposition. It’s one of the most common ways to achieve parallel execution. Using this approach, individual tasks are catalogued. If two of them can run concurrently, they are scheduled to do so by the developer.
  2. Producer/Consumer. It’s a form of functional decomposition in which one thread’s output is the input to a second. Can be hard to avoid, but frequently detrimental to performance.
  3. Data decomposition, a.k.a. “data level parallelism.” It breaks down tasks by the data they work on, rather than by nature of the task. Programs that are broken down via data decomposition generally have many threads performing the same work, just on different data items.

To make the three forms easy to understand, the authors used gardening as analogy where the threads map to gardners. For exmaple, the fuctional decomposition in gardening is to have one gardner to move the lawn and the other to weed. I find this analogy very intuitive and easy to follow. Even you don’t know multh-threading, you can guess it out from the gardening analogy.

Bothered by SLOW Web UI to manage vSphere? Want to manage ALL your VMware vCenters, AWS, Azure, Openstack, container behind a SINGLE pane of glass? Want to search, analyze, report, visualize VMs, hosts, networks, datastores, events as easily as Google the Web? Find out more about vSearch 3.0: the search engine for all your private and public clouds.

The challenges while working with multi-threading are:

  1. Synchronization is the process by which two or more threads coordinate their activities. For example, one thread waits for another to finish a task before continuing.
  2. Resource limitations refer to the inability of threads to work concurrently due to the constraints of a needed resource. For example, a hard drive can only read from one physical location at a time, which deprives threads of the ability to use the drive in parallel.
  3. Load balancing refers to the distribution of work across multiple threads so that they all perform roughly the same amount of work.
  4. Scalability is the challenge of making efficient use of a larger number of threads when software is run on more-capable systems. For example, if a program is written to make good use of four processors, will it scale properly when run a system with eight processors?

In cloud computing, we still need multi-threading and the above discussion about multi-threading still hold.

More often than not, we need to think about parallelism in a bigger scope – multiple processes, multiple applications, multiple machine instances, and multiple data centers. Although the scope and granularity is bigger, the above basic forms of decompositions and challenges are still helpful for us to architect applications for the cloud.

What do you think?

This entry was posted in Cloud Computing and tagged , , . Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

Post a Comment

Your email is never published nor shared. Required fields are marked *


You may use these HTML tags and attributes <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>


    My company has created products like vSearch ("Super vCenter"), vijavaNG APIs, EAM APIs, ICE tool. We also help clients with virtualization and cloud computing on customized development, training. Should you, or someone you know, need these products and services, please feel free to contact me: steve __AT__

    Me: Steve Jin, VMware vExpert who authored the VMware VI and vSphere SDK by Prentice Hall, and created the de factor open source vSphere Java API while working at VMware engineering. Companies like Cisco, EMC, NetApp, HP, Dell, VMware, are among the users of the API and other tools I developed for their products, internal IT orchestration, and test automation.