Xen in WiFi networks

From: Xen in WiFi networks

Successfully running Xen on a machine with a single WiFi card can be tricky. In my situation the network indom0 worked fine but in domU I could only see incoming packets. Outgoing ones were sent out toxen-br0 but never transmitted over the wireless network.

Sample network configuration

There are two physical hosts on the network - Router and a Xen host runningdom0 and domU:

HostIP addressRouter192.168.157.20Xen dom0192.168.157.21Xen domU192.168.157.41

Xen dom0 network interface has the real MAC address of the WiFi card, Xen domU MAC address is set in /etc/xen/domU config:

~# cat /etc/xen/domU[...]nics = 1vif = ['eth0=00:60:00:00:00:01']

Problem and solution

After some research it turned out that the culprit blocking outgoing traffic wasn'tXen but the wireless card which refused to send out packets with a MAC address different from its own. Since Xen deploysethernet bridge to provide network for domU kernels the outgoing packets have a virtual source MAC address different from the real one of theWiFi card. You don't need Xen to observe this problem - simple ethernet bridge constructed with brctl(8) between wired and wireless segments will not work with many wifi cards either. See sectionIt doesn't work with my Wireless card at Linux Bridging FAQ.

The solution obviously is to change the source MAC address to the real one on all outgoing packets. At first I tried to set it in vif=['mac=...'] config option fordomU host, but it didn't work. Better solution turned out to be deployment ofebtables - netfilter infrastructure similar to iptables but working with MAC addresses instead of IP addresses. The simplest ebfilter rule to make it work is:

ebtables -t nat -A POSTROUTING -s 00:60:00:00:00:01 \              -j snat --to-source 00:60:aa:bb:cc:dd

This rule rewrites the source MAC address of all packets from 00:60:00:00:00:01 to 00:60:aa:bb:cc:dd.You may want to add some more checks, e.g. for source interface name or alike. This is the bare minimum. I have executed this command indom0 but in theory it could work in domU as well.

Details

Surprisingly enough there doesn't have to be a corresponding reversed dnat rule. Ping from outside todomU triggers this packet sequence:

01:56:03.972501 00:02:11:22:33:44 > ff:ff:ff:ff:ff:ff,ethertype ARP (0x0806), length 42:arp who-has 192.168.157.41 tell 192.168.157.2001:56:03.975205 00:60:aa:bb:cc:dd > 00:02:11:22:33:44,ethertype ARP (0x0806), length 60:arp reply 192.168.157.41 is-at 00:60:00:00:00:01

Interesting point is the ARP reply. Although it comes from WiFi real MAC address, it carries the domUvirtual MAC inside in the packet. Subsequently my workstation created an entry in its ARP table with thevirtual(!) address of domU and started sending all further communication to that MAC:

01:56:03.975231 00:02:11:22:33:44 > 00:60:00:00:00:01,ethertype IPv4 (0x0800), length 98:IP 192.168.157.20 > 192.168.157.41:icmp 64: echo request seq 101:56:03.982348 00:60:aa:bb:cc:dd > 00:02:11:22:33:44,ethertype IPv4 (0x0800), length 98:IP 192.168.157.41 > 192.168.157.20:icmp 64: echo reply seq 1

Fortunately the Linux kernel on Router only updates its ARP table from the body of ARP replies and not from packet headers, which I believe is a correct behaviour. However I'm not sure thatall operating systems do like this.

Notes

• To get the MAC address of your WiFi card use commandip link show:

ip link show# ip link show[...]2: eth0: <BROADCAST,MULTICAST,NOTRAILERS,UP> mtu 1500 qdisc pfifo_fast qlen 1000    link/ether 00:60:aa:bb:cc:dd brd ff:ff:ff:ff:ff:ff

• The chipset on my WiFi card that caused these troubles isIntersil Corporation Prism 2.5 Wavelan chipset with firmware 1.3.6. Other chipsets may work just fine without these kludges.
• Xen version in use here is 2.0.6 and both dom0 and domU are running SuSE Linux Professional 9.2 with kernel 2.6.11.4, but it is fairly irrelevant as the problem is in hardware not in software.