Tag Archives: Linux

Install Squid as HTTP and HTTPS Proxy Server

While creating the vSearch 3.0, we needed a HTTP/HTTPS proxy server to test the new features like public cloud support. With a little search, I found the Squid server. It turned out to be pretty easy to install a new one and use it based on default settings.

Like most of the software package on CentOS, Squid can be installed using the yum command line. Here is what’s needed to install and configure it.

Posted in Applications & Tools | Also tagged , , | 2 Responses

Book Review: Docker Containers – Build and Deploy with Kubernetes, Flannel, Cockpit, and Atomic

After the virtualization buzz, the industry is actively looking for the next big thing. The container technology just came in at the right timing. Docker is the clear leader in the container technology, followed by CoreOS Rocket and RancherOS today.

Posted in Applications & Tools, Cloud Computing | Also tagged | 4 Responses

Harmful Big Data

In one of my recent projects, I got into a “big data” issue. One of the open source components emits so many logs that it quickly fills a hard disk. After isolating problem, I found huge number of log entries by the “find” command in a single log file whose size exceeds 50G – too big data for most system to handle.

The following is an example log entry in the log file:


Posted in Applications & Tools | Also tagged | Leave a comment

Tips and Tricks in Using Logrotate

In my last article, I talked about how to use logrotate to manage logs. As everything else, there are some tricks that are only learned when using it. Here are a few tricks and tips I learned recently. Hope it can save you some time.

Stickiness of Logrotate Rules

Posted in Applications & Tools | Also tagged , | 1 Response

How to Use Logrotate for Managing Log Files

Logging is an important for software development and operation. Over the time, the log files can grow fast to fill up the disk space. To avoid the problem, log files are rotated, compressed, and deprecated based on certain rules, for example, periodically, over certain size limit, and retention limit.

Most mordern logging frameworks can do log rotation and compression, but different applications may use different frameworks thus configure them differently. If you want to have a solution across different applications for consistent policies, the logrote (https://fedorahosted.org/logrotate/) is a good choice.

Posted in Applications & Tools, Software Development | Also tagged , | 2 Responses

How To Monitor Log File in Real Time and Save New Log To Another File

During software development, we often add lots of logs that help debug and trace the code. When the log files grow bigger, it gets harder to locate the right information of interest. Even we restart the application, the old log remains and new info appends the end of the file unless we delete log file. It’s OK to delete a log file but it’s better to keep it in case for information of previous runs. Here is a trick that I use to make it easier for me to find right log of interest, and it may help you as well.

Posted in Applications & Tools | Also tagged , | Leave a comment

Which Process Holds My File? Tools on Linux and Windows

Once in a while, I got into issues that my build failed because the IDE could not clean certain folder or file. The root cause was that they were opened by another application/process. If the application is known, just closed it and the build worked well. Some times I had no clue which application held the file. To find out quickly without guessing, right tools are needed.

On Linux, it’s quite easy with a command called lsof as follows:

Posted in Applications & Tools | Also tagged | 3 Responses

Linux Firewall with iptables Command

After proxying a service with Nginx, it’s always a good idea to block the service from direct remote access. For example, you have a tomcat server running on port 8080, and you’ve configured Nginx to proxy requests from port 80 to port 8080. The port 8080 should then be blocked from any host except localhost.

To do this on Linux, one of the ways is to just install iptables. On Ubuntu, issue the following commands to install and add rules:

Posted in Applications & Tools | Also tagged , | 1 Response

Building Linux Virtual Machine Templates: Tips and Checklist

While preparing for my home lab, I have created several virtual machine templates. Here are a few tips I found useful to smoothen the process and make your virtual machine templates easy to be deployed than otherwise.

Install VMware Tools
As you may have known, VMware Tools brings many features to the table, for example,

Significantly faster graphics performance and Windows Aero on operating systems that support Aero
Copying and pasting text, graphics, and files between the virtual machine and the host or client desktop
Improved mouse performance
Synchronization of the clock in the virtual machine with the clock on the host or client desktop
Scripting that helps automate guest operating system operations

Wait, it does not even mention APIs. For Guest APIs in vSphere 5.0 and later to work, you must have VMware Tools installed in your virtual machines.

Posted in Cloud Computing, Virtualization | Also tagged , , , | 2 Responses

Install and Configure VNC Server for Remote Access on Linux

Like RDP, VNC protocol allows for accessing computers remotely with full window GUI. It’s used in VMware vSphere and OpenStack for users to interact with virtual machines by connecting to hypervisors. Here is what I just did to set it up. The instruction should work on a host either it’s a physical machine or virtual machine as long as it’s installed with CentOS or equivalents.

Installing VNC Server

Posted in Cloud Computing | Also tagged , | 2 Responses

Sync up vCenter Server with NTP: Bug and Workaround

While playing with VMware Single Sign On (SSO) SDK, I got into an exception indicating that the request had expired.

Exception in thread "main" javax.xml.ws.soap.SOAPFaultException: Request has expired
	at com.sun.xml.internal.ws.fault.SOAP11Fault.getProtocolException(SOAP11Fault.java:178)
	at com.sun.xml.internal.ws.fault.SOAPFaultBuilder.createException(SOAPFaultBuilder.java:111)
	at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:108)
	at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:78)
	at com.sun.xml.internal.ws.client.sei.SEIStub.invoke(SEIStub.java:107)
	at $Proxy40.issue(Unknown Source)
	at com.vmware.sso.client.samples.AcquireHoKTokenByUserCredentialSample.getToken(AcquireHoKTokenByUserCredentialSample.java:233)
	at com.vmware.sso.client.samples.AcquireHoKTokenByUserCredentialSample.main(AcquireHoKTokenByUserCredentialSample.java:285)

Initially I thought it might be caused by timestamps in the arguments sent to SSO server. But further investigation showed that the time on my vCenter appliance server had run 3 hours faster than normal, so whatever request I had submitted from my desktop (whose time is up to the date) was “thought” to be submitted 3 hours ago. No wonder the request was rejected as expired. I think there is an allowance of a few minutes and 3 hours was just too big to ignore.

Posted in Virtualization | Also tagged , , | Leave a comment

Easy Terminal Server For Linux

A while back, I read an interesting article Enable multiuser logins with VNC – Help your users access a multiuser Linux system from anywhere on IBM developerWorks. I was thinking it could be used to implement something similar to the terminal service in Windows. There are many good use cases for terminal services. I think you probably know better than I do so I skip this part.

Posted in Cloud Computing | Also tagged , | 1 Response

KVM or QEMU: Which Runs My Virtual Machine?

While working with Openstack on both VMware virtual machines (with no virtualization instruction set exposed) and physical machines, I found virtual machine instances can be deployed seamlessly. On a machine that does not have virtualization instruction set exposed, KVM falls back to QEMU silently. That is why could I try out OpenStack on virtual machines before my hardware was ready. Because both KVM and QEMU support the same libvirt APIs, you would not notice any difference using command line like virsh, or Virtualization Manager. That is the beauty of standard APIs with different implementations, similar to the standard vSphere APIs that are implemented by both vCenter and ESXi.

Posted in Virtualization | Also tagged , , | 2 Responses

Tip For Tech Bloggers: Easy Way to Document Command History

While writing technical blogs, it’s always nice to include commands and scripts that readers can try on their own. I find it a bit challenging to accurately document these in steps while intensively testing or debugging something by myself.

There are actually commands that faithfully list all the commands you typed. You can then clean them up for your posts. In the following, I introduce how to do these on Linux, Windows.


Posted in Software Development | Also tagged | 1 Response

Configure Static IP Address on Linux VM in VMware Player

If you run VMware Player, you would have 3 networking options for virtual machines running there: Bridged, NAT, Host-Only. In the latest 5.0.1, I also found a new one: LAN Segment. This blog has a nice explanation on these three settings if you want to get more details.

In most of cases, I use NAT for networking because the virtual machine can have Internet access which allows me to install additional software as needed. By default, VMware Player uses DHCP to dynamically assign IP address while using NAT. So you cannot guarantee to get same IP address after each rebooting.

Posted in Virtualization | Also tagged , , | 22 Responses

Running Puppet in Client/Server Mode

In my last post, I introduced how to run a very simple HelloWorld script with Puppet 3.1.0 on CentOS 6.3. Although it shows how Puppet works, it’s not really how Puppet is used in real world. To get the most out of Puppet, you want to run the client/server mode where you have a master and many agents.

Part 1: Install Puppet server

Posted in Cloud Computing | Also tagged , , | 3 Responses

Running Puppet HelloWorld on CentOS 6.3

Puppet is a very well-known configuration management tool that has been adopted by many enterprises and service providers. VMware recently invested $30M in Puppet Labs, the company behind it. It’s clear that VMware needs such a tool if it wants to grow in data center space.

I actually got a book from last year’s PuppetConf in San Francisco and browsed through it on my flights. As with any other technology, reading it does not mean getting it. To get my hands dirty, I played with it in my home lab last week.

Posted in Cloud Computing | Also tagged , , | 7 Responses

Running DSL on VMware Player

DSL is an overloaded acronym standing for many different things. I first knew it as Digital Subscriber Line for Internet connection, and then Domain Specific Language. Recently I learnt a new one: Damn Small Linux. As you see the word small, you may think it’s for embedded system. It’s not.

Posted in Virtualization | Also tagged , | Leave a comment

VMware vSphere PowerCLI Alternative on Linux?

After creating a light virtual appliance last year, Timo Sugliani continued with a full fledged version of virtual appliance with all you need for vSphere development with Java and Jython. This is what Timo called “my linux powershell toolkit.” The biggest advantage is that you are no longer limited by Windows as your development platform.

Posted in vSphere API | Also tagged , , , , , | 4 Responses

    My company has created products like vSearch ("Super vCenter"), vijavaNG APIs, EAM APIs, ICE tool. We also help clients with virtualization and cloud computing on customized development, training. Should you, or someone you know, need these products and services, please feel free to contact me: steve __AT__ doublecloud.org.

    Me: Steve Jin, VMware vExpert who authored the VMware VI and vSphere SDK by Prentice Hall, and created the de factor open source vSphere Java API while working at VMware engineering. Companies like Cisco, EMC, NetApp, HP, Dell, VMware, are among the users of the API and other tools I developed for their products, internal IT orchestration, and test automation.