I came to know the Alpine Linux because of its small image size for Docker. Unlike almost 200MB in size of popular CentOS and Ubuntu, it just needs 5MB. That made me wonder if I can get much smaller Linux VM using the Alpine distro for special use cases like network tools.
While creating the vSearch 3.0, we needed a HTTP/HTTPS proxy server to test the new features like public cloud support. With a little search, I found the Squid server. It turned out to be pretty easy to install a new one and use it based on default settings.
Like most of the software package on CentOS, Squid can be installed using the yum command line. Here is what’s needed to install and configure it.
After the virtualization buzz, the industry is actively looking for the next big thing. The container technology just came in at the right timing. Docker is the clear leader in the container technology, followed by CoreOS Rocket and RancherOS today.
In one of my recent projects, I got into a “big data” issue. One of the open source components emits so many logs that it quickly fills a hard disk. After isolating problem, I found huge number of log entries by the “find” command in a single log file whose size exceeds 50G – too big data for most system to handle.
The following is an example log entry in the log file:
In my last article, I talked about how to use logrotate to manage logs. As everything else, there are some tricks that are only learned when using it. Here are a few tricks and tips I learned recently. Hope it can save you some time.
Stickiness of Logrotate Rules
Logging is an important for software development and operation. Over the time, the log files can grow fast to fill up the disk space. To avoid the problem, log files are rotated, compressed, and deprecated based on certain rules, for example, periodically, over certain size limit, and retention limit.
Most mordern logging frameworks can do log rotation and compression, but different applications may use different frameworks thus configure them differently. If you want to have a solution across different applications for consistent policies, the logrote (https://fedorahosted.org/logrotate/) is a good choice.
During software development, we often add lots of logs that help debug and trace the code. When the log files grow bigger, it gets harder to locate the right information of interest. Even we restart the application, the old log remains and new info appends the end of the file unless we delete log file. It’s OK to delete a log file but it’s better to keep it in case for information of previous runs. Here is a trick that I use to make it easier for me to find right log of interest, and it may help you as well.
Once in a while, I got into issues that my build failed because the IDE could not clean certain folder or file. The root cause was that they were opened by another application/process. If the application is known, just closed it and the build worked well. Some times I had no clue which application held the file. To find out quickly without guessing, right tools are needed.
On Linux, it’s quite easy with a command called lsof as follows:
After proxying a service with Nginx, it’s always a good idea to block the service from direct remote access. For example, you have a tomcat server running on port 8080, and you’ve configured Nginx to proxy requests from port 80 to port 8080. The port 8080 should then be blocked from any host except localhost.
To do this on Linux, one of the ways is to just install iptables. On Ubuntu, issue the following commands to install and add rules:
While preparing for my home lab, I have created several virtual machine templates. Here are a few tips I found useful to smoothen the process and make your virtual machine templates easy to be deployed than otherwise.
Install VMware Tools
As you may have known, VMware Tools brings many features to the table, for example,
Significantly faster graphics performance and Windows Aero on operating systems that support Aero Copying and pasting text, graphics, and files between the virtual machine and the host or client desktop Improved mouse performance Synchronization of the clock in the virtual machine with the clock on the host or client desktop Scripting that helps automate guest operating system operations
Wait, it does not even mention APIs. For Guest APIs in vSphere 5.0 and later to work, you must have VMware Tools installed in your virtual machines.
Like RDP, VNC protocol allows for accessing computers remotely with full window GUI. It’s used in VMware vSphere and OpenStack for users to interact with virtual machines by connecting to hypervisors. Here is what I just did to set it up. The instruction should work on a host either it’s a physical machine or virtual machine as long as it’s installed with CentOS or equivalents.
Installing VNC Server
While playing with VMware Single Sign On (SSO) SDK, I got into an exception indicating that the request had expired.
Exception in thread "main" javax.xml.ws.soap.SOAPFaultException: Request has expired at com.sun.xml.internal.ws.fault.SOAP11Fault.getProtocolException(SOAP11Fault.java:178) at com.sun.xml.internal.ws.fault.SOAPFaultBuilder.createException(SOAPFaultBuilder.java:111) at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:108) at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:78) at com.sun.xml.internal.ws.client.sei.SEIStub.invoke(SEIStub.java:107) at $Proxy40.issue(Unknown Source) at com.vmware.sso.client.samples.AcquireHoKTokenByUserCredentialSample.getToken(AcquireHoKTokenByUserCredentialSample.java:233) at com.vmware.sso.client.samples.AcquireHoKTokenByUserCredentialSample.main(AcquireHoKTokenByUserCredentialSample.java:285)
Initially I thought it might be caused by timestamps in the arguments sent to SSO server. But further investigation showed that the time on my vCenter appliance server had run 3 hours faster than normal, so whatever request I had submitted from my desktop (whose time is up to the date) was “thought” to be submitted 3 hours ago. No wonder the request was rejected as expired. I think there is an allowance of a few minutes and 3 hours was just too big to ignore.
A while back, I read an interesting article Enable multiuser logins with VNC – Help your users access a multiuser Linux system from anywhere on IBM developerWorks. I was thinking it could be used to implement something similar to the terminal service in Windows. There are many good use cases for terminal services. I think you probably know better than I do so I skip this part.
While working with Openstack on both VMware virtual machines (with no virtualization instruction set exposed) and physical machines, I found virtual machine instances can be deployed seamlessly. On a machine that does not have virtualization instruction set exposed, KVM falls back to QEMU silently. That is why could I try out OpenStack on virtual machines before my hardware was ready. Because both KVM and QEMU support the same libvirt APIs, you would not notice any difference using command line like virsh, or Virtualization Manager. That is the beauty of standard APIs with different implementations, similar to the standard vSphere APIs that are implemented by both vCenter and ESXi.
While writing technical blogs, it’s always nice to include commands and scripts that readers can try on their own. I find it a bit challenging to accurately document these in steps while intensively testing or debugging something by myself.
There are actually commands that faithfully list all the commands you typed. You can then clean them up for your posts. In the following, I introduce how to do these on Linux, Windows.
If you run VMware Player, you would have 3 networking options for virtual machines running there: Bridged, NAT, Host-Only. In the latest 5.0.1, I also found a new one: LAN Segment. This blog has a nice explanation on these three settings if you want to get more details.
In most of cases, I use NAT for networking because the virtual machine can have Internet access which allows me to install additional software as needed. By default, VMware Player uses DHCP to dynamically assign IP address while using NAT. So you cannot guarantee to get same IP address after each rebooting.
In my last post, I introduced how to run a very simple HelloWorld script with Puppet 3.1.0 on CentOS 6.3. Although it shows how Puppet works, it’s not really how Puppet is used in real world. To get the most out of Puppet, you want to run the client/server mode where you have a master and many agents.
Part 1: Install Puppet server
Puppet is a very well-known configuration management tool that has been adopted by many enterprises and service providers. VMware recently invested $30M in Puppet Labs, the company behind it. It’s clear that VMware needs such a tool if it wants to grow in data center space.
I actually got a book from last year’s PuppetConf in San Francisco and browsed through it on my flights. As with any other technology, reading it does not mean getting it. To get my hands dirty, I played with it in my home lab last week.
DSL is an overloaded acronym standing for many different things. I first knew it as Digital Subscriber Line for Internet connection, and then Domain Specific Language. Recently I learnt a new one: Damn Small Linux. As you see the word small, you may think it’s for embedded system. It’s not.
After creating a light virtual appliance last year, Timo Sugliani continued with a full fledged version of virtual appliance with all you need for vSphere development with Java and Jython. This is what Timo called “my linux powershell toolkit.” The biggest advantage is that you are no longer limited by Windows as your development platform.