MultiSSH: Productivity Multiplier for Managing Multiple Servers like ESXi

January 28th, 2014 1 comment

As I develop software, I rarely need to manage several servers using SSH at same time. If I do, I just manually connect to each server and type same commands over and over. Of course, it takes much time for the repeated work. More importantly, it’s very hard to repeat the steps consistently across multiple servers especially when there are more than 4 servers.

Categories: Applications & Tools Tags:

User Authentication with Thrift Service: Comparing Different Approaches

January 27th, 2014 No comments

We’ve covered Apache Thrift in last few articles from simple HelloWorld sample, Python Thrift client, to the securing Thrift traffic. Here I am going to discuss more on user authentication, which is a must for protecting the services and user authorization. This is in general a weakness of Thrift, but could be solved with different approaches. Having said that, if you have chosen Thrift, you probably build internal system where user access control is not important.

Categories: Uncategorized Tags:

Securing Thrift Traffic: Uncommon But Important Use Case

January 22nd, 2014 No comments

Thrift is mostly used for distributed systems which run mostly in house. There is no strong demand for securing the traffic on the wire. There are however use cases in which the Thrift services are exposed as a public service. In these use cases, the Thrift traffic should be secured with SSL/TLS. It comes with a price which more work on client and server on encryption and decryption. This is not a big deal for light load server, but for heavy load server it could be a problem. It can be mitigated with hardware acceleration on load balance servers between which and the client can be SSL, but not after that to the Thrift server.

Categories: Software Development Tags: ,

Thrift Client in Python: Hello World Sample

January 20th, 2014 No comments

As mentioned in my last post, Thrift is a cross-language and cross platform RPC framework. We’ve seen how a Java based Thrift server and client work there. Let’s take a look how to write a quick python script that connects to the Java Thrift server. It’s all possibe to write a Python based Thrift server, but probably not what most people want to do due to performance and scalability.

Apache Thrift Hello World Sample

January 15th, 2014 No comments

Thrift is one of the RPC frameworks that are widely used nowadays. It’s originally developed at Facebook and then open sourced under Apache Foundataion. It’s supported by major programming or scripting languages like Java, C++, Python, Ruby, etc. The typical use case is for building distributed systems, mostly in house.

Categories: Applications & Tools Tags: ,

Tomcat Behind Proxy: How to Block Direct Access

January 14th, 2014 1 comment

As discussed in my last post, after installing and configuring Nginx as the reversed proxy server for Tomcat, it’s necessary to block remote access to the original port served by Tomcat. To achieve this, iptables should be a good solution. Simpler solution is to change one line in the Tomcat server configuration file so that Tomcat accepts only requests from local host.

With Tomcat 7 on Ubuntu, the configuration file is /var/lib/tomcat7/conf/server.xml. Just add address=”127.0.0.1″ into the related Connector section as follows:

Linux Firewall with iptables Command

January 12th, 2014 1 comment

After proxying a service with Nginx, it’s always a good idea to block the service from direct remote access. For example, you have a tomcat server running on port 8080, and you’ve configured Nginx to proxy requests from port 80 to port 8080. The port 8080 should then be blocked from any host except localhost.

To do this on Linux, one of the ways is to just install iptables. On Ubuntu, issue the following commands to install and add rules:

Announcing vijavaNG: Much Lighter and Faster with Commercial License and Support

January 8th, 2014 17 comments

Since I left VCE four months ago, I have been working intensively on a commercial version of the open source vijava API supporting all versions of vSphere APIs (5.5 is the latest). If you have used the open source API, you know the vijava is much faster than other alternatives. Since its debut, it has been used in many commercial products from companies like Cisco, EMC, HP, etc.

Nginx with PAM Authentication

January 7th, 2014 6 comments

As I introduced in last article, Nginx is a lightweight Web and reversed proxy server that is gaining momentum. If you have URLs to be accessed only by authenticated users, you can have many options. In this article, I just introduce a very easy way for the Nginx to leverage the PAM (Pluggable Authentication Module) for user authentication. We will use OS user for authentication (there are many more methods supported by PAM). If you have a valid user with the Linux on which Nginx runs, your request will pass through; otherwise, it would be blocked.

Categories: Applications & Tools Tags: , , ,

Configuring Nginx as Reversed Proxy Server for HTTPS

January 6th, 2014 No comments

Nginx (pronounced as ‘engine x’) is a light-weight HTTP/reverse proxy/mail proxy server written by Igor Sysoe. It is flexible, lightweight compared, and high-performant with Apache. The official nginx site is here. The beginner guide is a very good starting point. The following is based on my hands-on experience with Nginx. If you have similar requirement, you can copy over the scripts and configuration for your environment.

Installing and running Nginx

Categories: Applications & Tools Tags: , ,

Three Ways to Get Certificate and Thumbprint from ESXi

January 2nd, 2014 1 comment

Happy New Year 2014!

When adding a new ESXi host to vCenter server via vSphere API, you can supply the certificate thumbprint of the ESXi server expected to have. Before calling the vSphere API, you can get the thumbprint directly or indirectly from the ESXi server to be added. Here are three different ways to do that. The first two approaches retrieve SSL certificate with which you can generate thumbprint.

Categories: Uncategorized Tags: ,

VMware Open Sourced Python Binding for vSphere API: What Limit Does It Solve

December 24th, 2013 1 comment

As reported in the community, there were quite excitement about the open source of the pyVmomi, the Python equivalent of vijava API. It was heatedly debated whether to open source the API even when I was working at VMware years ago. One camp of people thought it should be open sourced and even supported as Web Service SDKs; while the other group didn’t think it’s mature and would cause a lot of trouble in so doing. So it didn’t go anywhere in the past few years.

Categories: vSphere API Tags: ,

Parsing XML in Python: A Quick Working Sample

December 22nd, 2013 No comments

In my recent consulting projects, I used Python on various projects including the VMware pyVmomi () for managing vSphere. Because XML is ubiqutous these days, I had to use XML for configuration, passing information, etc. Parsing XML is a very basic part of using XML. The following sample code can achieve the same thing (print out may be different) as the C# sample that parses XML I wrote before.

Categories: Software Development Tags: , ,

Why Renaming Datastore Not Working With Command Line and How to Fix it

December 16th, 2013 No comments

I just got into a very interesting problem recently – the vim-cmd does not work as expected when used for renaming a datastore in vSphere.

What is the problem exactly?

The following command, for example, should change the name of a datastore from datastore1 (which is the default datastore name) to doublecloudDS.

# vim-cmd hostsvc/datastore/rename datastore1 doubecloudDS

After the command is executed, there is no error message reported. But the datastore name remains the same as shown in either the vSphere Client or using the following command:

Categories: Virtualization Tags: , ,

Run esxcli Command in A Browser: Hidden But Probably Better Hack

December 12th, 2013 No comments

My article “Run esxcli Command in a Web Browser: Another ESXi Hack” got quite some interests from the community. Although it works, I am not quite satisfied with the fact that the real esxcfg-info.cgi is disabled to run the esxcli.cgi.

Categories: Virtualization Tags: ,

Setting Up vSphere Web Client SDK: A Few Mistakes to Avoid

December 10th, 2013 5 comments

With the vSphere Web Client, VMware has really made the system complicated and slower. The extension mechanism is more flexible, but forces developers to use more libraries/frameworks/languages, therefore represents a much deeper learning curve than before with the Web based plugins. Installing and configuring the development environment itself could be intimidating for some developers. That is why I wanted to avoid it as long as possible, until I got a consulting project that may involve developing plugin for the Web Client.

Run esxcli Command in a Web Browser: Another ESXi Hack

December 8th, 2013 2 comments

In my recent consulting projects, I really got into a lot of scripting either command lines or Python with ESXi management. As I mentioned the hidden HTML formatter in esxcli command, you may have speculated what could the usage. The answer is simple: Web. But it’s not quite clear how it can be used. That’s where my curiosity started.

Categories: Virtualization Tags: , ,

Lab Automation Made Easy for Training and Testing with VMware Tools for Nested ESXi

December 4th, 2013 No comments

For those who run ESXi on a virtual machine, it’s a great news that VMware has released VMware Tools for nested ESXi as a fling in VMware Labs. Why? With the VMware Tools, you can get guest OS (really the ESXi here) information, like the IP address directly. It may sounds trivial as you can see the IP address from the virtual machine console of a virtualized ESXi. But for automation, it’s pretty hacky to get it programatically. Some people may wonder, “why not run commands via SSH?” It’s true that it’s easy to get the IP by running esxcli command, but you have to get IP first before running the command. With the VMware Tools, you can easily get the IP from vSphere Java API as would with any other normal virtual machines. Even more, you can also run commands like vim-cmd/esxcli in the virtual ESXi via APIs.

Hidden esxcli Command Output Formats You Probably Don’t Know

December 3rd, 2013 No comments

Besides the vim-cmd command I covered earlier, there is another powerful set of commands in ESXi – esxcli. As you can find from the help of the command, it covers 10 namespaces and drills down several layers down. The typical operations with the namespaces are get, set, and list. If you are familiar with REST API, you can think of the bottom level namespaces are resources.

Categories: Virtualization Tags: , ,

Powerful Hacks With ESXi vim-cmd Command, Together With Shell Commands

December 1st, 2013 10 comments

If you have read my previous article on the vim-cmd, you may have realized how handy it is, especially when it comes to manage virtual machines. There is however a pretty challenging problem to use it – for most commands for a virtual machine, it requires vmid which is an integer that uniquely identifies the virtual machine in the context of an ESXi server. It’s like primary key in SQL database to locate a record (virtual machine instance) in a table (virtual machine type). For people who are familiar with vSphere APIs, the vmid is the same as the value of ManagedObjectReference value of a virtual machine in ESXi. Because most administrators who use commands are not necessarily familiar with vSphere API, it doesn’t help much.

Categories: Virtualization Tags: , , ,