Home > Virtualization > How To Get vCenter Database Records from VMware Support Bundle

How To Get vCenter Database Records from VMware Support Bundle

October 7th, 2014 Leave a comment Go to comments

If you want to find out the information in the vCenter database, the VMware support bundle comes handy. For example, if you want to analyze the event history, task history, you can dig out these information from the support bundle. By default, the vCenter support bundle is collected as part of the VMware support bundle on standalone client, but not on the vSphere Web Client. So make sure to mark the check box if you want the vCenter info.

The vCenter support bundle can be generated by running the vc-support.sh command on the vCenter appliance. I am sure it should be able to do so on Windows version of vCenter.

Time to learn how to "Google" and manage your VMware and clouds in a fast and secure

doublecloud-vc:~ # find / -name vc-support*

Once the command is run, a support bundle with .zip format is generated at /var/log/vmware/vpx/ directory with timestamp in its file name, i.e. /var/log/vmware/vpx/vcsupport-2014-10-01.12705.zip. You can use scp to copy it to anywhere you want.

doublecloud-vc:~ # /usr/lib/vmware-vpx/vc-support.sh
File: /var/log/vmware/vpx/vcsupport-2014-10-01.12705.zip
NOTE: vcsupport-2014-10-01.12705.zip is greater than 10 MB.
Please do not attach this file when submitting an incident report.
Please contact VMware support for an ftp site.
To file a support incident, go to http://www.vmware.com/support/sr/sr_login.jsp
doublecloud-vc:/var/log/vmware/vpx # ll /var/log/vmware/vpx/vcsupport-2014-10-01.12705.zip
-rw------- 1 root root 381131541 Oct  1 23:23 /var/log/vmware/vpx/vcsupport-2014-10-01.12705.zip

A Close Look at the vc-support Script

Within the vc-support.sh script, there is a fuction for dumping data from database as follows. As you can see, it uses Java JDBC API to retrieve data. Because of this, there could be issue that I will discuss in the end.

349 dbdump(){
351 #Adding common jars
352 export VPXD_LIB=/usr/lib/vmware-vpx
354 for JAR in $VPXD_LIB/common-jars/*; do
355    if [ -f "$JAR" ]; then
357    fi
358 done
360 export CLASSPATH=$CLASSPATH${CLASSPATH:+:}${JARDIR}/common.jar:${JARDIR}/ojdbc5.jar:${JARDIR}/commons-logging-1.1.1.jar:${JARDIR}/vimtool.jar:${JARDIR}/commons-codec-1.3.jar    :${JARDIR}/commons-cli-1.1.jar:${COMMON_JARS:+:}$COMMON_JARS
363 rm -rf "${DBDUMPDIR}"
364 mkdir -p "${DBDUMPDIR}"
366 ${JAVA} com/vmware/vim/vimtool/VimTool vcsupport -d "${DBDUMPDIR}"  >>"${DBDUMPDIR}vcsupport.log" 2>&1
367 if [[ $? -ne 0 ]]; then
368    banner "Error collecting database diagnostics..."
369 fi
370 # check if the DB dump filled up the partition (at least 100MB free)
371 if ! df -P -B1M "${DBDUMPDIR}" | tail -n 1 | awk '$4 < 100 {exit 1}'; then
372    banner "Low on disk space after DB dump: ${DBDUMPDIR}. Deleting incomplete bundle '$ZIPFILE' and DB dump."
373    rm -rf "${DBDUMPDIR}"
374    rm -f "${ZIPFILE}"
375    abort
376 fi
377 }

There are a few constants in the script that are defined above or below.

444 if ! mkdir "$BUNDLENAME"; then
445    banner "Could not create temporary directory '$working_dir/$BUNDLENAME'."
446    exit
447 fi
449 #Set this after bundle location has been created
450 DBDUMPDIR="${BUNDLENAME}/var/log/vmware/dbdump/"
451 mkdir -p ${DBDUMPDIR}
27 LOGDIR=/var/log/vmware/vpx
 29 BUNDLENAME=vcsupport-$(date -I).$$
 31 VER=0.03
 32 JARDIR=/usr/lib/vmware-vpx/scripts
 33 JAVA=/usr/java/jre-vmware/bin/java
 34 SLAPCAT=/usr/sbin/slapcat
 35 XMLCFG="/usr/lib/vmware-vpx/py/xmlcfg.py"
 36 VPXD_CONFIG_FILE="/etc/vmware-vpx/vpxd.cfg"
 37 MAXLOGFILES_XPATH="/config/log/maxFileNum"

What are Dumped from vCenter Database

After unzipping the support bundle, a new directory will be created. Under that directory, you will find var directory for copied content from original vCenter server. There are many logs, but since we are only interested in the database dump, we can jump right to the dbdump directory as follows:

root@steve-dev:/var/tmp/vc-sb/vcsupport-2014-10-01.12705/var/log/vmware/dbdump# ls
DB_STAT_INFO.xml     VMwareVCMSDS.ldif  VPX_DATACENTER.xml        VPX_ENTITY.xml            VPX_EXT.xml         VPX_HIST_STAT3.xml  VPX_RESOURCE_POOL.xml  VPX_TASK.xml      VPX_VM.xml
DB_TABLE_SIZE.xml    VPX_ACCESS.xml     VPX_DEVICE.xml            VPX_EVENT.xml             VPX_HIST_STAT1.xml  VPX_HIST_STAT4.xml  VPX_STAT_COUNTER.xml   VPX_UPTIME.xml

Now, let’s take a look at the event history in the VPX_EVENT.xml file:

root@sjin-dev:/var/tmp/vc-sb/vcsupport-2014-10-01.12705/var/log/vmware/dbdump# vi VPX_EVENT.xml
<VPX_EVENT.xml xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" ><Row>
<event_id>6925</event_id><chain_id>6925</chain_id><event_type>vim.event.BadUsernameSessionEvent</event_type><extended_class>0</extended_class><create_time>2014-10-01 14:28:16.27</create_time><username>storvisor</username><category>error</category></vm_id></vm_name><host_id>39</host_id><host_name>sysmgmt004-esx-1a.eng.storvisor.com</host_name><computeresource_id>36</computeresource_id><computeresource_type>3</computeresource_type><computeresource_name>mnode-cluster</computeresource_name><datacenter_id>31</datacenter_id><datacenter_name>mnode-dc</datacenter_name></datastore_id></datastore_name></network_id></network_name></network_type></dvs_id></dvs_name></storagepod_id></storagepod_name></change_tag_id>
<event_id>1</event_id><chain_id>1</chain_id><event_type>vim.event.ServerStartedSessionEvent</event_type><extended_class>0</extended_class><create_time>2014-07-29 10:24:26.035</create_time></username><category>info</category></vm_id></vm_name></host_id></host_name></computeresource_id></computeresource_type></computeresource_name></datacenter_id></datacenter_name></datastore_id></datastore_name></network_id></network_name></network_type></dvs_id></dvs_name></storagepod_id></storagepod_name></change_tag_id>
</VPX_EVENT.xml >


Software can fail anywhere not just the product itself – the vc-support script for troubleshooting can fail too. To figure out what is happening, just get the log for the support bundle generation. The following is a successful sample which does show what are dumped and how much time it took for each file.

root@steve-dev:/var/tmp/vc-sb/vcsupport-2014-10-01.12705/var/log/vmware/dbdump$ vi vcsupport.log
Picked up JAVA_TOOL_OPTIONS: -Xms16M -Xmx128M
[2014-10-01 23:19:42 INFO] Invoking vcsupport "-d" "vcsupport-2014-10-01.12705/var/log/vmware/dbdump/"
[2014-10-01 23:19:42 INFO] Starting Virtual Center Database Support
[2014-10-01 23:19:42 INFO] Config name=vcdb
[2014-10-01 23:19:42 INFO] Property file=/etc/vmware-vpx/vcdb.properties
[2014-10-01 23:19:42 INFO] Loaded url from props=jdbc:postgresql://
[2014-10-01 23:19:42 INFO] Overrides=
[2014-10-01 23:19:42 INFO] Dump of VPX_ACCESS took :204 ms
[2014-10-01 23:19:43 INFO] Dump of VPX_HOST took :276 ms
[2014-10-01 23:19:43 INFO] Dump of VPX_VM took :11 ms
[2014-10-01 23:19:43 INFO] Dump of VPX_RESOURCE_POOL took :9 ms
[2014-10-01 23:19:43 INFO] Dump of VPX_COMPUTE_RESOURCE took :19 ms
[2014-10-01 23:19:43 INFO] Dump of VPX_ENTITY took :14 ms
[2014-10-01 23:19:43 INFO] Dump of VPX_DISABLED_METHODS took :31 ms
[2014-10-01 23:19:43 INFO] Dump of VPX_EXT took :35 ms
[2014-10-01 23:19:43 INFO] Dump of VPX_EXT_SERVER took :8 ms
[2014-10-01 23:19:43 INFO] Dump of VPX_DATACENTER took :6 ms
[2014-10-01 23:19:44 INFO] Dump of VPX_EVENT took :847 ms
[2014-10-01 23:19:44 INFO] Dump of VPX_TASK took :524 ms
[2014-10-01 23:19:44 INFO] Dump of VPX_STAT_DEF took :14 ms
[2014-10-01 23:19:44 INFO] Dump of VPX_STAT_COUNTER took :7 ms
[2014-10-01 23:19:44 INFO] Dump of VPX_DEVICE took :6 ms
[2014-10-01 23:19:44 INFO] Dump of VPX_HIST_STAT1 took :382 ms
[2014-10-01 23:19:45 INFO] Dump of VPX_HIST_STAT2 took :174 ms
[2014-10-01 23:19:45 INFO] Dump of VPX_HIST_STAT3 took :119 ms
[2014-10-01 23:19:45 INFO] Dump of VPX_HIST_STAT4 took :136 ms
[2014-10-01 23:19:45 INFO] Dump of VPX_USAGE_STAT took :8 ms
[2014-10-01 23:19:45 INFO] Dump of VPX_UPTIME took :30 ms
[2014-10-01 23:19:45 INFO] Dump of DB_PRODUCT_INFO took :6 ms
[2014-10-01 23:19:45 INFO] Dump of DB_TABLE_SIZE took :20 ms
[2014-10-01 23:19:45 INFO] Dump of DB_STAT_INFO took :90 ms

In one case, I saw the VPX_EVENT.xml is zero byte, and the VPX_TASK.xml is totally missing. The log revealed that there was OutOfMemoryError from the Java code. It could be that the vCenter itself ran out of memory, or maybe the -Xmx128M is too low for working with a product vCenter server. It shouldn’t be the second case if the Java code is well designed. But if that is true, the JAVA_TOOL_OPTIONS should be updated before running the script.

doublecloud-vc:~ # echo $JAVA_TOOL_OPTIONS
-Xms16M -Xmx128M
doublecloud-vc:~ # export JAVA_TOOL_OPTIONS="-Xms16M -Xmx512M"
doublecloud-vc:~ # echo $JAVA_TOOL_OPTIONS
-Xms16M -Xmx512M
  1. No comments yet.
  1. No trackbacks yet.