Profile photo of Steve White

About Steve White

Software Engineer
Email Address
steve.white@netnow.co

Xen Virtual Machine Conversion HVM/PV

VMware to Xen Virtualised Image Conversion

We recently converted a RHEL 6.0 virtual image from VMware to Xen (Xen Cloud Platform 1.6) by using the XenServer Conversion Manager. After installing the XenTools on to the image XenCenter still showed them as “not installed”. We registered the VM with the RHN (via rhn_register) and for RH 6.1 updates and upgraded the kernel from vmlinuz-2.6.32-71.40.1.el6.x86_64 to 2.6.32-131.38.1.el6.x86_64 via the following command:

yum update kernel.x86_64

After this and a reboot the tools the Virtualization state appeared as “Optimized (version 6.1 installed)” and we could set the memory ballooning minimum and maximum in the XenCenter “memory” tab.

Hardware Virtualised Machine By Default

We later found out that it was running as a Hardware Virtualised Machine (HVM) instead of Paravirtualised (PV), which is less efficient. Apparently the actual performance difference varies depending on what hardware it’s running on but we wanted to aim for PV. From what I’ve read on various sites and posts I believe that HVM was used because we had converted the image instead of creating a new one from a Xen template.

To validate this from the XenServer console we got the uuid of our image via a “xe vm-list” and checked the HVM-boot-policy to see if it was set to “BIOS order”, e.g:

xe vm-param-get uuid=<VM uuid> param-name=HVM-boot-policy

If it is set to “BIOS order” (or other non-empty string?) then HVM is enabled.

Converting a Xen Virtual Machine from HVM to PV

We couldn’t find any formal documentation about converting VMs to PV but we came across this posting which pointed us in the right direction.

In order to convert the image to use PV:

1. Go in to your Linux VM and save the output from the “mount” command and /boot/grub/menu.1st contents.

2. Take a snapshot of your image in XenCenter to roll back to just in case!

3. From the XenServer console get the VM ‘uuid’ using the command:

xe vm-list

NB: You can list all VM params with: xe vm-param-list uuid=<VM uuid>

4. Set PV mode by running:

xe vm-param-set uuid=<VM uuid> HVM-boot-policy=""

5. Set the PV boot loader:

xe vm-param-set uuid=<VM uuid> PV-bootloader=pygrub

6. Set the kernel and ramdisk params from the grub menu.1st contents saved earlier:

xe vm-param-set uuid=<VM uuid> PV-bootloader-args="--kernel <kernel location> --ramdisk <initrd location>"

e.g.

xe vm-param-set uuid=c66eb11e-75a3-91a5-2f90-9aa3c03ec61b PV-bootloader-args="--kernel /vmlinuz-2.6.32-131.38.1.el6.x86_64 --ramdisk /initramfs-2.6.32-131.38.1.el6.x86_64.img"

7. List the disks in your VM:

xe vm-disk-list uuid=<VM uuid>

8. Get the VDI disk uuid (e.g. for Disk 0 VDI) and then:

xe vdi-param-get uuid=<VDI uuid> param-name=vbd-uuids

9. Get the vbd-uuids value and then set bootable=true:

xe vbd-param-set uuid=<vbd-uuids> bootable=true

10. Set the root device to point to that from the output of the ‘mount’ command from earlier:

xe vm-param-set uuid=<VM uuid> PV-args="root=<root device>"

e.g.

xe vm-param-set uuid=<VM uuid> PV-args="root=/dev/mapper/vg_vmrhel6-lv_root"

11. Reboot/Start the VM.

12. Unfortunately we could not get the framebuffer (/dev/fb0) device to work and so X wouldn’t start, but this wasn’t really an issue in our environment since we were only concerned with console access.

A few days later we came across a handy script in the following link which performs all of the above commands, which is probably much less error prone. Disclaimer – We have not tested this.

Reverting Back to HVM

To revert back to HVM either drop back to your previous Xen snapshot or run the following from the Xen server console:

xe vm-param-set uuid=<vm uuid> HVM-boot-policy="BIOS order"

By the way, Xen snapshots are great! The ability to branch and revert without losing changes is really handy.

– Steve –

Steve White

SVN Keywords and PropSet PowerShell Script

How we are using SVN Keywords and PropSet PowerShell Script

Some of our source code remains in an SVN repository and uses keyword substitution. By using SVN Keywords and PropSet PowerShell Script together we can provide up to date version and revision information. Keywords are generally defined in comments or variables in the format “$TagName: TagValue $:” in any source code. See the following link for more information.

A C source file may use keywords to insert version header information which can be pulled out by a version parameter (-v) passed to it, e.g:

 

const char *SVNVERSION={“@(#) $Revision: 1234 $:”};
const char *CODEAUTHOR={“@(#) $Author: steve.white $:”};
const char *DATEMOD={“@(#) $Date: 2013-01-01 12:00:00 +0100 (Tue, 01 Jan 2013) $:”};

 

Quite often, running a UNIX “strings” on the C generated binary will return the above information in its output which is useful for identifying binaries from an unknown source. So far I’ve noticed this works well with binaries produced by gcc on Linux, Solaris, Windows and AIX.

Eventually our code will all be migrated to a distributed version control system (DVCS) such as Git or Mercurial. However, keywords are not recommended so we will have to have a re-think as to how this is done.

Source Code with Missing Keywords

We recently discovered that TortoiseSVN on some of our developer machines had SVN keywords disabled. This meant that newly added source files hadn’t had them added and as a result keywords were not being updated upon a commit. Instead of running an SVN command manually against each source file I wrote a quick PowerShell script to do the legwork for us:

 

# Save as: svnPropSetSrc.ps1
# Change to source root and run: PowerShell .\path\to\svnPropSetSrc.ps1
#
# Recursively set SVN properties for common source files

$srcFiles=gci . -recurse -name -include @(“*.cpp”,”*.c”,”*.h”,”*.js”,”*.java”,”*.cs”)
Foreach ($Item in $srcFiles)
{
Write-Host $Item
svn propset “svn:eol-style” native $Item
svn propset “svn:keywords” “Author Date Id HeadURL Revision” $Item
}

 

This uses “gci” which is an alias for the Get-ChildItem PowerShell Cmdlet and outputs the file it’s modifying to standard output. It’s probably worth running this and piping output to a file so you can track what’s changed and see if there have been any errors reported.
 

Steve White

Samba 3.6.7 undefined symbol libintl_gettext, compile problem on solaris 10,

About the Samba Solaris 10 Problem – undefined symbol libintl_gettext

Recently I had to install the latest version of Samba on one of our Solaris 10 development boxes. Initially I tried to use the packages linked on http://www.sunfreeware.com but it appears that these had been compiled for a higher patch level of Solaris 10. Samba installed without issue and so did the dependencies listed on there, but none of the binaries would launch (nmbd, smbd, smbclient, etc…). Since our it’s critical this particular box stays up I didn’t want to patch the OS and have any downtime and so I downloaded the source from http://www.samba.org.

Compiling From Source on Solaris 10

After unpacking the tarball I changed to the ‘source3′ directory and tried to compile with the following:

./configure --without-ldap --without-ads --without-sys-quotas --without-manpages --with-libsmbclient --without-winbind --without-python --enable-shared --disable-cups
make

I disabled certain built in features during the configure to reduce the number of dependencies required and as a result the extra libraries which I may have had to install. However, no matter what I changed I kept coming across an undefined reference for libintl:

Undefined first referenced
 symbol in file
 libintl_gettext utils/net.o
 libintl_textdomain utils/net.o
 libintl_bindtextdomain utils/net.o

I’ve compiled this many times before without fault on various Linux distributions so I was surprised to have come across this problem. Now I know libintl.so was installed in /lib/ and I knew it worked fine as other applications which depended on it were fully functioning.

After searching through various google results I decided to edit the Samba Makefile. The first change I made was an edit to ./source3/Makefile in which I appended -lintl to the first instance of the LIBS= variable. To my surprise after running the make again everything compiled fine and a make install put it all in the right place. I wish every linking issue was that easy to fix.

Regards,

Steve

Steve White

Disabling Windows Installer Advertised Shortcuts in VS2010

About Windows Installer Advertised Shortcuts Problem

We recently migrated a product to Visual Studio 2010 and came across a problem whereby the programs shortcuts kept triggering re-installation of the application, even though the product had just been installed. Windows Installer Advertised shortcuts ensure that the shortcuts always have an icon, even if users move from machine to machine. No matter what, we could not find the root cause of this issue, so we looked at disabling this mechanism in the MSI.

Editing Generated MSI’s Using Orca

We used Microsofts ‘Orca’ tool to edit the MSI and scripted this as a post build event in VS2010. To do this we performed the following steps:

  1. Install Windows 7 SDK
  2. Install Orca.msi contained within the SDK: C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\Orca.Msi
  3. Set the following Visual Studio post build event for the installer project:

cscript “C:\Program Files\Microsoft SDKs\Windows\v7.0\Samples\sysmgmt\msi\scripts\WiRunSQL.vbs” “$(BuiltOuputPath)” “INSERT INTO Property(Property, Value) VALUES (‘DISABLEADVTSHORTCUTS’, ‘1’)”

This ensured every installer thereafter was generated with advertised shortcuts disabled. There are a plethora of options which can be set by the Orca tool, just look at the ‘scripts’ folder in the above post build event for examples of what you can do.

Regards,

Steve

Steve White

Disabling IPv6 in RHEL 6

How to Disable IPv6

I came across a piece of software running on RHEL Linux which had issues binding to IPv6 enabled network adapters. As a quick workaround I disabled IPv6 on the system with the following steps (as root):

1. Edit /etc/sysconfig/network and add a line:

NETWORKING_IPV6=off

2. Add a new file /etc/modprobe.d/disable_ipv6.conf and insert the following single line:

options ipv6 disable = 1

3. Edit /etc/modprobe.d/modprobe.conf.dist and add:

install ipv6 /bin/true

4. Run: service ip6tables stop
5. Run: chkconfig ip6tables off
6. Reboot the system

Not ideal but the software subsequently came up on the next reboot and bound to the correct network adapter.

Regards,

Steve

Steve White

Linux sockets in TIME_WAIT status

About Linux sockets in TIME_WAIT status

We recently came across an application used a significant number of open/close TCP connections under Redhat Enterprise Linux 5.4. As a result the system ran out of local ports to open new connections due a number of sockets that were stuck in TIME_WAIT status. Using the ‘netstat’ command from a shell we could see that it was this particular application causing the problem because the destination port numbers were all the same.

Sockets which are listed in CLOSE_WAIT status are different and many of these usually means that a particular application isn’t closing it’s socket file descriptors off correctly.

More about fixing Linux socket TIME_WAIT status

This problem was causing issues for all network related software on the system. To alleviate this problem we set the following kernel parameters as root and rebooted:

echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle
echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse
echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout

After this change the number of sockets in TIME_WAIT state were drastically reduced, problem solved!

Regards,

Steve