My Ubuntu Server Installation with VMware Server

Last updated $Date: 2008-09-17 11:23:09 $

Martti Kuparinen <martti.kuparinen@iki.fi>

http://www.iki.fi/kuparine/comp/ubuntu/en/server.html

Abstract

This is how I installed and configured few servers (both 32-bit and 64-bit) running Ubuntu 8.04 Server and VMware Server. With this setup I was able to reduce the number of physical computers in our lab, thus reducing electricity consumption and need for extra cooling.

IMPORTANT NOTICE: I no longer use VMware Server in our lab so this page (and the server.sh installation script) might not work correctly. I'm nowadays using VMware ESXi on all our servers as it gives me better performance than the Linux + VMware Server combination. Most likely I'm not going to update this page any more but I leave it here in case someone still finds it useful.


[ See also My Ubuntu Installation and My Ubuntu UML Installation ]


Table of Contents

1. Introduction
2. Initial install
3. Adding more software repositories
4. Installing applications
5. Installing and configuring SSH server
6. Installing VMware Server
7. Configuring VLAN interfaces
8. Install VMware Server Console
9. Optimizing virtual machines

1. Introduction

This document and the installation helper script applies to Ubuntu 8.04 Server.

2. Initial install

Start by fetching the installation CD image from the Ubuntu download page. I used the "Server install CD" image but it's possible to use the "Alternate install CD" and select the server installation from the initial menu. Make sure the MD5 checksum matches with the published value before burning the ISO file or the installation may fail. Reboot with the new installation CD and follow the instructions to install Ubuntu.

I prefer to use ext3 for the root filesystem, jfs for the /vmware filesystem and tmpfs for /tmp.


sudo aptitude install jfsutils
sudo umount /vmware
sudo mkfs.jfs /dev/sda3
sudo vi /etc/fstab

proc            /proc           proc    defaults        0       0
/dev/sda1       /               ext3    relatime,errors=remount-ro 0 1
/dev/sda2       none            swap    sw              0       0
/dev/sda3       /vmware         jfs     noatime         0       2
tmpfs           /tmp            tmpfs   defaults        0       0
/dev/scd0       /media/cdrom0   udf,iso9660 user,noauto,exec,utf8 0 0

Next I activated some tweaks to make the server perform faster.


sudo vi /boot/grub/menu.lst

# kopt=root=UUID=... ro elevator=deadline
...
kernel	/boot/vmlinuz-2.6.24 ... elevator=deadline

sudo vi /etc/rc.local

blockdev --setra 32768 /dev/sda 2> /dev/null
blockdev --setra 32768 /dev/sdb 2> /dev/null

sudo vi /etc/sysctl.conf

vm.swappiness=0
vm.overcommit_memory=1
vm.dirty_background_ratio=5
vm.dirty_ratio=10
vm.dirty_expire_centisecs=1000
dev.rtc.max-user-freq=1024

sudo reboot

3. Adding more software repositories

Login to the newly installed Ubuntu system, modify /etc/apt/sources.list and update the installed packages to the latest versions. With the following repositories you have more applications available for download and install. Replace fi with your country code (fi = Finland, se = Sweden, ...) in the following example to access the closest national mirror.


sudo vi /etc/apt/sources.list

## MAIN AND RESTRICTED REPOSITORIES
deb http://fi.archive.ubuntu.com/ubuntu/ hardy main restricted
deb-src http://fi.archive.ubuntu.com/ubuntu/ hardy main restricted

## UNIVERSE REPOSITORY
deb http://fi.archive.ubuntu.com/ubuntu/ hardy universe
deb http://fi.archive.ubuntu.com/ubuntu/ hardy-security universe
deb-src http://fi.archive.ubuntu.com/ubuntu/ hardy universe
deb-src http://fi.archive.ubuntu.com/ubuntu/ hardy-security universe

## MULTIVERSE REPOSITORY
deb http://fi.archive.ubuntu.com/ubuntu/ hardy multiverse
deb http://fi.archive.ubuntu.com/ubuntu/ hardy-security multiverse
deb-src http://fi.archive.ubuntu.com/ubuntu/ hardy multiverse
deb-src http://fi.archive.ubuntu.com/ubuntu/ hardy-security multiverse

## UBUNTU SECURITY UPDATES
deb http://fi.archive.ubuntu.com/ubuntu/ hardy-security main restricted
deb-src http://fi.archive.ubuntu.com/ubuntu/ hardy-security main restricted

## MAJOR BUG FIX UPDATES
deb http://fi.archive.ubuntu.com/ubuntu/ hardy-updates main restricted
deb-src http://fi.archive.ubuntu.com/ubuntu/ hardy-updates main restricted

## CANONICAL REPOSITORY
deb http://archive.canonical.com/ubuntu/ hardy partner

sudo aptitude update && sudo dist-upgrade

4. Installing applications

Next install the following programs.

## Some useful programs
sudo aptitude install openssh-client
sudo aptitude install wget
sudo aptitude install unattended-upgrades

## NIS
sudo aptitude install nis

## NFS
sudo aptitude install nfs-common

## NTP
sudo aptitude install ntp

## Rootkit detection
sudo aptitude install chkrootkit rkhunter
chkrootkit
rkhunter --checkall

5. Installing and configuring SSH server

Next install and configure SSH server to allow only your normal users to login. If your system has usernames jane and john make sure /etc/ssh/sshd_config has these lines. Modify the settings and restart the SSH server.


sudo aptitude install openssh-server
sudo vi /etc/ssh/sshd_config

Protocol 2
...
PermitRootLogin no
...
AllowUsers jane
AllowUsers john

sudo /etc/init.d/ssh restart

Sometimes you may need to limit which hosts can contact the SSH server. Restrictions can be implemented by using the /etc/hosts.allow and /etc/hosts.deny files. The next allows connections only from trusted networks.


sudo vi /etc/hosts.allow

# Allow SSH from my own networks
sshd : localhost
sshd : 10. 192.168.
sshd : .mydomain.com .mydomain.net

sudo vi /etc/hosts.deny

# Deny everything by default
ALL : ALL : severity auth.info

6. Installing VMware Server

VMware Server is a free virtualization software which enables you to run Windows, BSD and Linux on your Linux server. With VMware Server it's possible to access the virtual machines remotely as the VMware Server is always running in the background on the server. The server does not even need X server installed.

Start be fetching these three files.

Next install some packages needed during the installation.


sudo aptitude install xinetd
sudo aptitude install build-essential

## i686 users:
sudo aptitude install libxtst6
sudo aptitude install libxt6
sudo aptitude install libice6
sudo aptitude install libsm6
sudo aptitude install libxrender1
sudo aptitude install libxi6

## amd64 users:
sudo aptitude install ia32-libs

## If you are running the linux-server kernel
## (i.e. you installed from the Ubuntu Server Installation CD)
sudo aptitude install linux-headers-server

## If you are running the linux-generic kernel
## (i.e. you installed from Desktop or Alternate Installation CD)
sudo aptitude install linux-headers-generic

Next install the VMware Server.


tar xzf VMware-server-1.0.7-108231.tar.gz -C /tmp
cd /tmp/vmware-server-distrib
sudo ./vmware-install.pl

Finally note that you should have a firewall to protect the following VMware Server services. See My Ubuntu Installation for an example how to use iptables to protect these ports.

PortProtocolDescription
902TCPVMware remote access authentication daemon

7. Configuring VLAN interfaces

VMware Server (and VMware Workstation) has a nasty bug which makes it fail when connecting to a VLAN interface if the interface name is longer than 8 characters. The bug is visible if you use VLAN tags larger than 999. One solution is to rename the physical interface from ethN to vN so that the VLAN interfaces will be called vN.NNNN (in other words, the name is short enough). Here is an example where eth0 is used for the server maintenance and v1 is the interface connected to a VLAN capable switch.


sudo vi /etc/udev/rules.d/70-persistent-net.rules

# PCI device 0x14e4:0x1659 (tg3)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:15:c5:fc:28:74", ATTR{type}=="1", NAME="eth0"

# PCI device 0x14e4:0x1659 (tg3)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:15:c5:fc:28:75", ATTR{type}=="1", NAME="v1"

The v1 interface needs to be up in order for VLANs to work so I configured a fake address on it.


sudo vi /etc/network/interfaces

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
        address 192.168.0.10
        netmask 255.255.255.0
        gateway 192.168.0.1
        broadcast 192.168.0.255

auto v1
iface v1 inet static
        address 0.0.0.1
        netmask 255.255.255.255

Next install the tool needed to create the VLAN interfaces and load VLAN (802.1q) support during boot time.


sudo aptitude install vlan
sudo vi /etc/modules

8021q

We have a lot of VLANs in our switch so I created a simple script to create all those VLAN interfaces at boot time. That script reads its settings from /etc/network/vlan.conf so this file can be shared between all our servers without any modification.


cd /etc/network
sudo wget http://www.iki.fi/kuparine/comp/ubuntu/en/vlan.conf
sudo vi vlan.conf

cd /etc/init.d
sudo wget http://www.iki.fi/kuparine/comp/ubuntu/en/vlan
sudo chmod 755 vlan
cd /etc/rcS.d
sudo ln -s ../init.d/vlan S41vlan

After reboot you should see a lot of interfaces with the ifconfig command...


reboot

ifconfig -a
eth0      Link encap:Ethernet  HWaddr 00:15:c5:fc:28:74  
          inet addr:192.168.0.10    Bcast:192.168.0.255    Mask:255.255.255.0
          inet6 addr: fe80::215:c5ff:fefc:2874/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2712556 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4762970 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:266218687 (253.8 MB)  TX bytes:6719433076 (6.2 GB)
          Interrupt:16 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:43 errors:0 dropped:0 overruns:0 frame:0
          TX packets:43 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:3260 (3.1 KB)  TX bytes:3260 (3.1 KB)

v1        Link encap:Ethernet  HWaddr 00:15:c5:fc:28:75  
          inet addr:0.0.0.1  Bcast:255.255.255.255  Mask:255.255.255.255
          inet6 addr: fe80::215:c5ff:fefc:2875/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:422118 errors:0 dropped:0 overruns:0 frame:0
          TX packets:372 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:46801428 (44.6 MB)  TX bytes:32792 (32.0 KB)
          Interrupt:17 

v1.1000   Link encap:Ethernet  HWaddr 00:15:c5:fc:28:75  
          inet6 addr: fe80::215:c5ff:fefc:2875/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8144 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:468088 (457.1 KB)  TX bytes:406 (406.0 B)

...

v1.4009   Link encap:Ethernet  HWaddr 00:15:c5:fc:28:75  
          inet6 addr: fe80::215:c5ff:fefc:2875/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8144 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:468088 (457.1 KB)  TX bytes:406 (406.0 B)

Now it's possible to reconfigure VMware Server to provide virtual vmnet devices for all these VLAN interfaces, in other words it's possible to connect your virtual machines to different VLANs. However, please note that you can have max 4 ethernet interfaces for virtual machines. This is a limitation of VMware Server (and VMware Workstation). Consider using Xen if you need more interfaces for your virtual machines...


sudo vmware-config.pl
...
Would you like to skip networking setup and keep your old settings as they are?
(yes/no) [yes] no

Do you want networking for your virtual machines? (yes/no/help) [yes] yes

Would you prefer to modify your existing networking configuration using the
wizard or the editor? (wizard/editor/help) [wizard] editor

yes
10
bridged
v1.1000

yes
11
bridged
v1.1001

...

yes
49
bridged
v1.4009

no

8. Install VMware Server Console

Finally install VMware Server Console on your desktop PC. This is used to connect to your virtual machines running on the VMware Server.


unzip VMware-server-linux-client-1.0.7-108231.zip
tar xzf VMware-server-console-1.0.7-108231.tar.gz -C /tmp
cd /tmp/vmware-server-console-distrib
sudo ./vmware-install.pl
sudo cp /lib/libgcc_s.so.1 /usr/lib/vmware-server-console/lib/libgcc_s.so.1/

9. Optimizing virtual machines

9.1. Linux

Modify the .vmx file to include these settings.

MemTrimRate = "0"
sched.mem.pshare.enable = "FALSE"
MemAllowAutoScaleDown = "FALSE"

Within the virtual machine, consider using these kernel option in /boot/grub/menu.lst

noapic nolapic apci=off clocksource=acpi_pm elevator=noop

9.2. Microsoft Windows

Modify the .vmx file to include these settings.

MemTrimRate = "0"
sched.mem.pshare.enable = "FALSE"
MemAllowAutoScaleDown = "FALSE"