Create file-backed VBD:

The actual space of VBD will be the amount of disk the virtual machine used. And it will be convenient if the virtual machine will be duplicated since the work is just copying the VBD file and changing some configurations. But file-backed VBDs may not be appropriate for I/O-intensive domains because of the I/O handling cost to support file-backed VBDs in dom0.
Create a 20GB sparse file-backed VBD:

# dd if=/dev/zero of=/lhome/xen/f12install/vmdisk0 bs=1k seek=20480k count=1

Make a ext3 file system in the disk file:

# mkfs -t ext3 vmdisk0

Install Fedora 12 Linux via Internet:

First download the pxeboot kernel of Fedora 12 for installation via Internet. Download vmlinuz and initrd.img from here:

http://download.fedora.redhat.com/pub/fedora/linux/releases/12/Fedora/x86_64/os/images/pxeboot/

We assume these two files are stored in /lhome/xen/f12install/.

Create an installation profile f12.install:

name="F12INSTALL"
vcpus=2
memory=2048
disk = ['tap:aio:/lhome/xen/f12install/vmdisk0,xvda,w' ]
vif = [ 'bridge=eth0' ]
kernel = "/lhome/xen/f12install/vmlinuz"
ramdisk = "/lhome/xen/f12install/initrd.img"
on_reboot = 'restart'
on_crash = 'restart'

Here the blktap2 VBD driver is used for better performance than blkback backed VBD. If the blkback backed driver is used, the disk like should be changed to:

disk = ['file:/lhome/xen/f12install/vmdisk0,xvda,w' ]

The virtual machine’s name is “F12INSTALL”, memory is 2G, CPU number is 2, disk, kernel and ramdisk is prepared in the above steps.

Start this virtual machine and connect to this virtual machine’s console and complete the installation:

# xm create -c f12.install

The console can be released by “Ctrl+]”. And it can be reconnected by:

# xm console F12INSTALL

The installation of Fedora 12 will start. The gateway and DNS server should be set according to the network configuration.

The URL of installation source I used during installation is:

http://download.fedora.redhat.com/pub/fedora/linux/releases/12/Fedora/x86_64/os/

After successfully installation of this virtual machine. It can be shut down by:

# xm shutdown F12INSTALL

This virtual machine can be duplicated to get more VMs: How to duplicate Xen DomU virtual machines.

Start DomU:

Create a profile vm-10.0.0.123.run for loading the virtual machine:

name="10.0.0.123"
vcpus=2
memory=2048
disk = ['tap:aio:/lhome/xen/vm-10.0.0.123/vmdisk0,xvda,w' ]
vif = [ 'bridge=eth0' ]
bootloader = "/usr/bin/pygrub"
on_reboot = 'restart'
on_crash = 'restart'

Here we use the PyGrub (“/usr/bin/pygrub”) as the bootloader. PyGrub starts Linux DomUs with the kernels that lie in the filesystem of the DomU instead of the kernels that lie in the filesystem of the Dom0. That makes the kernel update and management easier.

Then the DomU can be started using this profile:

# xm create vm-10.0.0.123.run

The console of this DomU can be connected to:

# xm console vm-10.0.0.123

• Setting up Stable Xen Dom0 with Fedora: Xen 4.0.0 with Xenified Linux Kernel 2.6.32.13 in Fedora 12
• Setting up Xen pvops Dom0 on Fedora : Xen 3.4.2 + Kernel 2.6.31 with paravirt_ops in Fedora 12
• Setting up Xen Dom0 on Fedora : Xen 3.4.1 with Linux Kernel 2.6.29 on Fedora 12

Hardware:

Dom0′s hardware platform:

Motherboard: INTEL S5500BC S5500 Quad Core Xeon Server Board
CPU: 2 x Intel Quad Core Xeon E5520 2.26G (5.86GT/sec,8M,Socket 1366)
Memory: 8 x Kingston DDR-3 1333MHz 4GB ECC REG. CL9 DIMM w/Parity & Thermal Sensor
HD: 4 x WD WD10EARS 1 TB, SATA II 3Gb/s, 64 MB Cache

Linux system:

Fedora 12 x86_64
SELinux is disabled. Please refer here for detail: Disabled SELinux on Fedora.

ext3 is recommended for the file system of disk partition for /boot.

Update the system:

# yum update

The Xen and libvirt packages in Fedora should not be installed to avoid conflict.

# yum erase xen* libvirt

Build and install Xen hypervisor and tools

Download Xen 4.0.0

$ wget http://bits.xensource.com/oss-xen/release/4.0.0/xen-4.0.0.tar.gz
$ tar xf xen-4.0.0.tar.gz

Build Xen and tools

$ make xen
$ make tools

You may need to install packages depended by this.

Install Xen and tools

$ make install-xen
$ make install-tools

Build and install xenified Linux kernel

Download Linux kernel 2.6.32.13

$ wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.32.13.tar.bz2
$ tar xf linux-2.6.32.13.tar.bz2

Download 2.6.32 Xen patches v2

$ wget http://gentoo-xen-kernel.googlecode.com/files/xen-patches-2.6.32-2.tar.bz2
$ tar xf xen-patches-2.6.32-2.tar.bz2

Apply Xen patches

Apply all the patches downloaded above following the patch number. This patch.sh script can be used (we assume the patch and the kernel are in the same directory):

patch.sh:

#!/bin/bash
for P in `ls ../xen-patches-2.6.32-2/6*.patch1 | sort`
do
    patch -p1 -s -i $P
    if [ $? = 0 ]; then
        echo $P applied
    else
        echo "Error processing "$P
        exit 1
    fi
done

Put this script into Linux source directory and execute:

$ sh ./patch.sh

Configure Xenified Linux kernel

A working configuration file that I used can be downloaded directly from here:

config-for-xenified-linux-2.6.32.13

Just download this file, put it into the kernel source code file directory and rename it to .config .

Other than use my configuration file, you can also configure it by yourself by using “make menuconfig”.

Make sure you build the kernel with these components enabled:

Processor type and features  --->
 [*] Symmetric multi-processing support
 [*] Support sparse irq numbering
 [*] Enable Xen compatible kernel
 Preemption Model (No Forced Preemption (Server))  --->

Device Drivers  --->
 XEN  --->
 [*] Privileged Guest (domain 0)
 <*> Backend driver support (NEW)
 <*>   Block-device backend driver (NEW)
 <*>   Block-device tap backend driver (NEW)
 <*>   Block-device tap backend driver 2 (NEW)
 <*>   Network-device backend driver (NEW)
 (8)     Maximum simultaneous transmit requests (as a power of 2) (NEW)
 [ ]     Pipelined transmitter (DANGEROUS) (NEW)
 < >     Network-device loopback driver (NEW)
 <*>   PCI-device backend driver (NEW)
 PCI Backend Mode (Virtual PCI)  --->
 [ ]     PCI Backend Debugging (NEW)
 < >   TPM-device backend driver (NEW)
 <M>   SCSI backend driver (NEW)
 <M>   USB backend driver (NEW)
 <M> Block-device frontend driver
 <M> Network-device frontend driver
 <M>   Network-device frontend driver acceleration for Solarflare NICs (NEW)
 <M> SCSI frontend driver (NEW)
 <M> USB frontend driver (NEW)
 [*]   Taking the HCD statistics (for debug) (NEW)
 [ ]   HCD suspend/resume support (DO NOT USE) (NEW)
 <*> User-space granted page access driver (NEW)
 <*> Framebuffer-device frontend driver (NEW)
 <*>   Keyboard-device frontend driver (NEW)
 [*] Disable serial port drivers (NEW)
 <*> Export Xen attributes in sysfs (NEW)
 (256) Number of guest devices (NEW)
 Xen version compatibility (no compatibility code)  --->
 [*] Place shared vCPU info in per-CPU storage (NEW)

Build kernel

$ make -j16

Install modules and kernel

# make modules_install install

Configure grub

Add one entry for Xen in /boot/grub/grub.conf. This is an example entry:

title Xen 4.0.0 - Xenified Linux 2.6.32.13
  root (hd0,0)
  kernel /xen-4.0.0.gz console=vga vga=ask noreboot
  module /vmlinuz-2.6.32.13 ro root=/dev/mapper/VolGroup-LogVol_root noiswmd LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us
  module /initramfs-2.6.32.13.img

The root and other parameters may be different depending on the configuration.

Make Xend and Xendomains services automatically start when system boots

# cd /etc/init.d/
# chkconfig --add xend
# chkconfig --add xendomains

Check whether Xend and Xendomains services are automatically started in level 3-5:

# chkconfig --list | grep xend

It should be like this:

xend               0:off    1:off    2:off    3:on    4:on    5:on    6:off
xendomains         0:off    1:off    2:off    3:on    4:on    5:on    6:off

BIOS configuration

If xen stops at:

“I/O virtualization disabled.”

We may need to enable VT and I/O virtualization in BIOS.

These options can be enabled in BIOS:

Intel (R) Virtualization Technology
Intel (R) VT for Directed I/O
Interrupt Remapping
Coherency Support
ATS Support

Enjoy the fun now!

After booting the system, you can try to use xm to check xen info

# xm info

Then xm command can be used to start up DomUs.

This is one working configuration file for one DomU that I use:

name="10.0.1.201"
vcpus=2
memory=16384
disk = ['tap:aio:/lhome/xen/vm-10.0.1.201/vmdisk0,xvda,w' ]
# disk = ['file:/lhome/xen/vm-10.0.1.201/vmdisk0,xvda,w' ]
vif = ['bridge=eth0']
bootloader = "/usr/bin/pygrub"
on_reboot = 'restart'
on_crash = 'restart'

Here we use the blktap backed VBD device which has much better performance than Linux blkback backed VBD device.

Making the performance more stable

Allocating dedicated CPU core and memory for Dom0 may provide more stable performance for the Xen platform. Please refer to Managing Xen Dom0′s CPU and Memory/ for detailed instruction.

Problems

Here is a list of problem that may occur during the configuration.

Limited number of loop devices

The default number of loop device in this kernel is 8. When we are using blkback backed VBDs and we need to have more than 8 virtual machines, we should add more loop devices. You need to use the first method (pass parameter max_loop=32 to vmlinuz) if you use my kernel configuration file.

initramfs related problem

The initramfs image under /root generated by dracut doesn’t work on some servers. If you have the similiar problem, you can try to use image generated by mkinitrd:

1) Generate initrd-2.6.32.13.img using mkinitrd

mkinitrd /boot/initrd-2.6.32.13.img 2.6.32.13

2) Edit entry in /boot/grub/grub.conf

Change

module /initramfs-2.6.32.13.img

to

module /initrd-2.6.32.13.img

drm related problem

On one of our servers that uses radeon card we have experienced problem related to drm. The system crashes after the kernel printing out information about drm. We can add nomodeset option to kernel command line to bypass this problem.

The kernel command line in /boot/grub.conf will change to:

module /vmlinuz-2.6.32.13 ro root=/dev/mapper/VolGroup-LogVol_root nomodeset noiswmd LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us

Build kernel on 32bit platform

You first need to enable PAE support if you’re building 32 bit version of the kernel. Xen only supports 32 bit PAE kernels nowadays. Xen kernel build options won’t show up if you don’t enable PAE for 32 bit builds.
You can enable PAE in “Processor type and features -> High Memory Support (64GB) -> PAE (Physical Address Extension) Support”

I can find Xen options after enable PAE on my laptop. I have never run Xen Dom0 on a 32bit kernel. I can’t say whether it is stable. Please try it and I will appreciate it if you share your result with me ;)

Error message about ksm

If you get lots error messages generated by ksm, you can disable service ksm and ksmtuned to eliminate these error messages:

# chkconfig ksm off
# chkconfig ksmtuned off

• Setting up Stable Xen DomU with Fedora: Unmodified Fedora 12 on top of Xenified Fedora 12 Dom0 with Xen 4.0.0
• Setting up Xen Dom0 on Fedora : Xen 3.4.1 with Linux Kernel 2.6.29 on Fedora 12
• Setting up Xen pvops Dom0 on Fedora : Xen 3.4.2 + Kernel 2.6.31 with paravirt_ops in Fedora 12

Now let’s look at how to menage Xen Dom0′s CPU and memory in a better way.

Dedicate a CPU core for Dom0

Dom0 will have free CPU time to process the I/O requests from the DomUs if it has dedicated CPU core(s). Better performance may be achieved by this since there are less CPU context switches to do in Dom0.

We can dedicate CPU core for Dom0 by passing “dom0_max_vcpus=X dom0_vcpus_pin” options to Xen hypervisor (xen.gz) in /boot/grub/grub.conf. X is the number of vcpus dedicated to Dom0.

As hyperthreading technology is enabled in most modern CPUs, we need to specify two processors to dedicate one CPU core. So the “X” above should usually be 2 for one CPU core.

kernel /xen.gz console=vga vga=ask noreboot dom0_max_vcpus=2 dom0_vpus_pin

After booting the system, the VCPU list can be got on Dom0 by this command:

# xm vcpu-list

Even after booting the system, the VCPU number can be configured by xm command. We can set Domain-0 have two VCPUs and processor 0 and 1 to be dedicated to Dom0 by these commands:

# xm vcpu-set Domin-0 2
# xm vcpu-pin Domain-0 0
# xm vcpu-pin Domain-0 1

Dedicate memory for Dom0

We should always dedicate fixed amount of memory for Xen Dom0.

We can set the initial memory size of Dom0 by passing “dom0_mem=xxx” (in KB) option to Xen hypervisor (gen.gz) in /boot/grub/grub.conf. “xxx” is the amount of memory for Dom0 in KB.

If we set the initial memory size of Dom0 to 2GB, just change the entry in grub.conf to:

kernel /xen.gz console=vga vga=ask noreboot dom0_max_vcpus=2 dom0_vpus_pin dom0_mem=2097152

Set lowest permissible memory for Dom0

The option dom0-min-mem in Xend configuration file /etc/xen/xend-config.sxp is used to specify the lowest permissible memory for Dom0.

The value of dom0-min-mem (in MB) is the lowest permissible memory level for Dom0. The default value is 256. If we limit the memory size of Dom0 to 2G, just set:

(dom0-min-mem 2048)

Preventing dom0 memory ballooning

The “enable-dom0-ballooning” option in Xend configuration file is used to specify whether Dom0′s memory can be ballooned out. Setting “enable-dom0-ballooning” to “no” will make sure Xend never takes any memory away from Dom0:

(enable-dom0-ballooning no)

• A simple CPU and memory performance test of xen Dom0 and DomU
• Setting up Stable Xen Dom0 with Fedora: Xen 4.0.0 with Xenified Linux Kernel 2.6.32.13 in Fedora 12
• Setting up Xen pvops Dom0 on Fedora : Xen 3.4.2 + Kernel 2.6.31 with paravirt_ops in Fedora 12

Switching VESA modes of Linux kernel at boot time can be done by using the “vga=…“ kernel boot parameter. This parameter accept the decimal value of Linux video mode numbers instead of VESA video mode numbers. The Linux video mode number can be easily derived from the VESA number.

The video mode number of the Linux kernel is the VESA mode number plus 0×200.

Linux_kernel_mode_number = VESA_mode_number + 0x200

Here are some of the VESA mode numbers:

    | 640x480  800x600  1024x768 1280x1024
----+-------------------------------------
256 |  0x101    0x103    0x105    0x107
32k |  0x110    0x113    0x116    0x119
64k |  0x111    0x114    0x117    0x11A
16M |  0x112    0x115    0x118    0x11B

So the table for the Kernel mode numbers are:

    | 640x480  800x600  1024x768 1280x1024
----+-------------------------------------
256 |  0x301    0x303    0x305    0x307
32k |  0x310    0x313    0x316    0x319
64k |  0x311    0x314    0x317    0x31A
16M |  0x312    0x315    0x318    0x31B

The decimal value of the Linux kernel video mode number can be passed to the kernel in the form “vga=XXX“, where XXX is the decimal value.

Instead of the XXX decimal value, the “vga” parameter also accept “ask” which will list all the Linux kernel mode numbers and let the user select one. You can used it if you want to be asked every time booting Linux. It can also be used to find the best Linux video mode on your console.

The best way for configuring the “vga=XXX” parameter is following these steps.

First, add “vga=ask” parameter to the Linux kernel entry in grub configuration file /boot/grub/grub.conf. Like this:

kernel /vmlinuz-2.6.32.16-141.fc12.i686 ro root=/dev/mapper/VolGroup-LogVol00 vga=ask 

Second, reboot Linux and hit return when the kernel ask for the vga mode. Then select one mode from the list and remember the Linux mode number which is a hexdecimal value in the format YYY. You can also get more choice by entering “scan“.

Last, calculate the decimal value of the Linux video mode number. This simple python command can be used:

python -c "print 0xYYY"

YYY is the hexdecimal value you got.

Then change “ask” in grub configuration file to the decimal value calculated.

Here is a list of usually used Linux mode number and the decimal value if you like to choose one directly:

• Sending Email from mailx Command in Linux Using Gmail’s Smtp
• Setting up Stable Xen Dom0 with Fedora: Xen 4.0.0 with Xenified Linux Kernel 2.6.32.13 in Fedora 12
• Setting up Xen Dom0 on Fedora : Xen 3.4.1 with Linux Kernel 2.6.29 on Fedora 12

Please read [1] for requirement when using Spirit. Currently, the version of iTunes used should be iTune 9 earlier than 9.2. The firmware’s version should be 3.1.2, 3.1.3 or 3.2.

My iPod which I use in this tutorial is: iPod touch 8G 3.1.3(7E18) Model (MC086ZP). The Linux I used is Fedora 12.

Okay, let’s go!

1. Synchronise with iTunes

As what we usually do. This step is highly recommended by Spirit.

2. Dowload Spirit jaibreak for Linux and compile it

Get the source code

$ git clone http://github.com/posixninja/spirit-linux.git

Install depended development packages

$ sudo yum install libplist-devel libimobiledevice-devel

You may need to install other development packages such as make, gcc etc.

Build spirit

$ cd spirit-linux
$ make

3. Set iPod to not Auto-lock

In Settings->General->Auto-lock, select never.

4. Connect iPod touch to the Linux box with cable

5. jailbreak it

$ ./spirit

6. iPod will reboot

The jailbreak work will start. Wait for a few minutes.

7. Done

Congratulations! Now, look for the Cydia icon. My favourite applications are Terminal and OpenSSH. Remember to change the password of root and mobile users for security.

Reference:

[1] http://www.spiritjb.com/

• Setting up eCryptFS in Linux
• Speeding up Firefox on Linux
• Linux Setting Date, Time and Timezone

First, the gtk-qt-engine should have been installed to configure the theme for GTK application. If not, install it like this:

# yum install gtk-qt-engine

You can find the cursor icons which you are using in KDE under /usr/share/icons/. Let’s use “Oxygen_Blue” which is my favourite one as the example.

After installing gtk-qt-engine, there is one configuration file for it: ~/.gtkrc-2.0-kde4. It may be a little different depending on the package version.

To set the cursor icon theme for GTK application to “Oxygen_Blue”, add this like into ~/.gtkrc-2.0-kde4:

gtk-cursor-theme-name="Oxygen_BLUE"

It can be changed to any icon theme’s name that we like. We may need to reload the KDE desktop after changing it. After reloading the desktop environment, try to open a GTK application and see whether the icons are the same in both kind of applications.

• Beautiful Desktop – Gnome of OSX style on Linux
• Mac OSX-like Gnome Theme Style
• Gnome Style: Shiki Colors+Gnome Colors

The startx script in Fedora will read /etc/sysconfig/desktop for the “DESKTOP” variable and starts the desktop environment depending on it. While the default value for “DESKTOP” is set to “GNOME”, we will get to gnome if we don’t set the variable. When we want to startx to KDE we just need to add one line to /etc/sysconfig/desktop:

DESKTOP="KDE"

If there is already one line that defines “DESKTOP”, just change the value of it.

Then we will start KDE when we run startx from console.

Free additional gift

The “DISPLAYMANAGER” in the /etc/sysconfig/desktop file is used to specify the default display manager. To make KDM the default display, just add this line:

DISPLAYMANAGER="KDE"

• Sending Email from mailx Command in Linux Using Gmail’s Smtp
• Configuring Mouse Cursor Style for GTK Applications in KDE Desktop
• Making yum not Update Kernel

These are screen shoots of my desktop.

This theme is a mixture of Mac OS X Bundle 2.1[1], iRaveH20 3 Icon Theme (Full Gnome/XFCE) Lights 3.0 [2] and Oxygen-cursor-themes 4.4.4 which is originally designed for  KDE desktop[3].

The theme controls, the window border and the desktop background comes from Mac OS X Bundle. Download the package from the link in gnome-look.org and unpack it. You can get one picture and two tarballs. Then they can be easily installed by using the “install” button in Gnome’s “Appearance Preference” tool. For the background picture, just copy it somewhere.

The iRaveH20 Icon theme can also be installed by the “install” button. The Oxygen cursor theme is already in Fedora’s repository. If you use another Linux distribution that have KDE desktop environment which is the usual case, you can easily install the cursor theme which is a part of KDE. In my box, I can install it by:

# yum install oxygen-cursor-theme

After installing them, select any theme in the theme window and then click the “customize” button. The “Customize theme” window will appears. Select “mac-osx-controls” in the “Controls” tab, select “mac-osx-window” in “Window Border” tab, select “iRaveH20 3 Lights Edition” for icons and choose “Oxygen_Blue” for cursor them. The background can be added by clicking “add” button and choose the picture stored before.

This the style of mine. It’s stable and making me happy with my terminal. Of course, there are lots of other combination of these themes or others. Choose the one that you prefer ;) Linux is free and open for everyone.

Links:

[1] Mac OS X Bundle: http://gnome-look.org/content/show.php?content=28686
[2] iRaveH20 3 Icon Theme (Full Gnome/XFCE): http://gnome-look.org/content/show.php?content=119776
[3] KDE desktop: http://kde.org

• Gnome Style: Shiki Colors+Gnome Colors
• Beautiful Desktop – Gnome of OSX style on Linux
• Configuring Mouse Cursor Style for GTK Applications in KDE Desktop

Port forwarding

Port forwarding also called “port mapping” commonly refers to the network address translator gateway changing the destination address and/or port of the packet to reach a host within a masqueraded, typically private, network.

Port forwarding can be used to allow remote computers (e.g., public machines on the Internet) to connect to a specific computer within a private network such as local area network (LAN), sothat xternal hosts can communicate with services provided by hosts within a LAN. For example, running a public HTTP server (port 80) on a host within a private LAN, or permitting secure shell ssh (port 22) access to hosts within the private LAN from the Internet.

In Unix/Linux box where port numbers below 1024 can only be listened by software running as root, port forwarding is also used to redirect incoming traffic from a low numbered port to software listening on a higher port. This software can be running as a normal user, which avoids the security risk caused by running as the root user.

iptables

iptables is a very powerfull firewall which handles packets based on the type of packet activity and enqueues the packet in one of its builtin ‘tables’. In Linux box, iptables is implemented in Linux kernel as some kernel modules.

There are three tables in total: mangle, filter and nat. The mangle table is responsible for the alteration of service bits in the TCP header. The filter queue is responsible for packet filtering. The nat table performs Network Address Translation (NAT). Each tables may have some built-in chains in which firewall policy rules can be placed.

The filter table has three built-in chains:
* Forward chain: Filters packets destined for networks protected by the firewall.
* Input chain: Filters packets destined for the firewall.
* Output chain: Filters packets originating from the firewall.

The nat table has two built-in chains:
* Pre-routing chain: NATs packets when the destination address of the packet needs to be changed.
* Post-routing chain: NATs packets when the source address of the packet needs to be changed.

Below is a brief view of how packets are processed by the chains:

PACKET IN
    |
PREROUTING--[routing]-->--FORWARD-->--POSTROUTING-->--OUT
 - mangle      |           - mangle      - mangle
 - nat (dst)   |           - filter      - nat (src)
               |                            |
               |                            |
              INPUT                       OUTPUT
              - mangle                    - mangle
              - filter                    - nat (dst)
               |                          - filter
               |                            |
               `----->-----[app]----->------'

We only look into the packets that requires port forwarding which is the topic of this post.

The packet entering the firewall is inspected by the rules in the nat table’s PREROUTING chain to see whether it requires destination modification (DNAT). The packet is then routed by Linux router after leaving the PREROUTING chain. The packet which is destined for a “protected” network is filtered by the rules in the FORWARD chain of the filter table. The it will go through the packet undergoes SNAT in the POSTROUTING chain before arriving at the “protected” network. When the destination server decides to reply, the packet undergoes the same sequence of steps.

Port forwarding using iptables

A port-forwarded packet will pass the PREROUTING chain in nat table, FORWARD chain in filter table, POSTROUTING chain in nat table and other chains. We need to add rules to these chains.

Let’s use a senario to introduce how to configure iptables to do port forwarding. Suppose our gateway can connect to both the Internet (0.0.0.0/0) and the LAN (192.168.1.0/24). The gateway’s eth0 interface has a public IP 7.8.9.10 while the eth1 has a LAN IP 192.168.1.1. Now, suppose that we have set up a HTTP server on 192.168.1.2:8080 and we want to provides service to the Internet through the public IP. We need to configure iptables to forward packets coming to port 80 of 7.8.9.10 to 8080 of 192.168.1.2 in LAN.

Below is the network topology:

Internet---------[router/firewall]-------------LAN
0.0.0.0/0      7.8.9.10    192.168.1.1    192.168.1.0/24

Normally we deny all incoming connections to a gateway machine by default because opening up all services and ports could be a security risk. We will only open the ports for the services that we will use. In this example, we will open port 80 for HTTP service.

This is the rules to forward connections on port 80 of the gateway to the internal machine:

# iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 80 -j DNAT --to 192.168.1.2:8080
# iptables -A FORWARD -p tcp -d 192.168.1.2 --dport 8080 -j ACCEPT

These two rules are straight forward. The first one specifies that all incoming tcp connections to port 80 should be sent to port 8080 of the internal machine 192.168.1.2. This rule alone doesn’t complete the job as described above that we deny all incoming connections by default. Then we accept the incoming connection to port 80 from eth0 which connect to the Internet with the publich IP by the second rule. From the process path in the “iptables” part, the packet will also pass the FORWARD chains. We add the second rule in FORWARD chain to allow forwarding the packets to port 8080 of 192.168.1.2.

By now, we have set up the the iptables rules for forwarding the 80 port. For other service, the method is similiar with the HTTP service.

The conntrack entries

The “nf_conntrack_*” kernel modules enables iptables to examine the status of connections by caching the related information for these connections. A cat of /proc/net/nf_conntrack (in some old Linux kernels, the file is /proc/net/ip_conntrack) will give a list of all the current entries in the conntrack database.

A conntrack entry looks like this:

ipv4     2 tcp      6 431581 ESTABLISHED \
src=7.8.9.20 dst=7.8.9.10 sport=53867 dport=80 packets=22 bytes=13861 \
src=192.168.1.2 dst=7.8.9.20 sport=8080 dport=53867 packets=14 bytes=3535 \
[ASSURED] mark=0 secmark=0 use=2

This entry contains all the information that the conntrack module maintains to know the state of a specific connection. We can find the version of ip protocal version and the decimal coding, the protocol and the normal decimal coding. After this, we get how long this conntrack entry should live. Next is the actual state that this entry is in at this present point of time. Then, we get the source IP address, destination IP address, source port and destination port. After that, we get the IPs and ports of both source and destination we expect of return packets.

In this entry we can find that the arriving connection is:

7.8.9.20:53867 --> 7.8.9.10:80

while the returning connection is:

192.168.1.2:8080 --> 7.8.9.20:53867

which reflects the port forwarding which we have set.

Update history:
Jun. 30, 2010. Add the rule in FORWARD chain.

• Port forwarding using ssh tunnel
• Setting Up Gateway Using iptables and route on Linux
• Changing MAC Address in Linux aka. MAC Spoofing

We need to edit the Xorg configuration file /etc/X11/xorg.conf with root permission. Add a line ‘ Option “NoLogo” ‘ within the “Device” section which configuring the NVIDIA driver will disable the logo.

This is my xorg.conf file. I use two screen (a big LED and my laptop’s screen) and the screen of my laptop’s DPI is 96×96.

# Xorg configuration created by Zhiqiang Ma (http://pkill.info)
# Section "ServerFlags"
# 	Option	    "AIGLX" "on"
# EndSection

Section "ServerLayout"
    Identifier     "Default Layout"
    Screen      0  "Screen0" 0 0
    InputDevice    "Keyboard0" "CoreKeyboard"
    InputDevice    "Mouse0" "CorePointer"
    Option         "Xinerama" "0"
EndSection

Section "Files"
    ModulePath      "/usr/lib/xorg/modules/extensions/nvidia"
    ModulePath      "/usr/lib/xorg/modules"
EndSection

Section "InputDevice"
    # generated from data in "/etc/sysconfig/keyboard"
    Identifier     "Keyboard0"
    Driver         "kbd"
    Option         "XkbLayout" "us"
    Option         "XkbModel" "pc105"
EndSection

Section "InputDevice"
    # generated from default
    Identifier     "Mouse0"
    Driver         "mouse"
    Option         "Protocol" "auto"
    Option         "Device" "/dev/input/mice"
    Option         "Emulate3Buttons" "no"
    Option         "ZAxisMapping" "4 5"
EndSection

Section "Monitor"
    Identifier     "Monitor0"
    VendorName     "Unknown"
    ModelName      "Philips 170B"
    HorizSync       30.0 - 83.0
    VertRefresh     56.0 - 76.0
    Option         "UseEdidDpi" "false"
    Option         "DPI" "96 x 96"
EndSection

Section "Device"
    Identifier     "Device0"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "GeForce G 105M"
    Option         "NoLogo"
EndSection

Section "Screen"
    Identifier     "Screen0"
    Device         "Device0"
    Monitor        "Monitor0"
    DefaultDepth    24
    Option         "TwinView" "0"
    Option         "metamodes" "CRT: nvidia-auto-select +0+0"
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection

# Section "Extensions"
#     Option         "Composite" "Enable"
# EndSection

• Installing Nvidia Driver on Fedora
• Install nvidia driver on Fedora
• Setting up eCryptFS in Linux