CentOS – Using NAT with KVM Guests
Wednesday, July 17th, 2019CentOS – Using NAT with KVM Guests
Please note that all commands in this guide must be run on the main HOST machine (the physical machine). They should not be run on KVM guests (virtual machines).
If your server has a limited number of IPv4 addresses, it might be best to setup and run some virtual machines that are configured to use the default NAT network interface that KVM provides. This will allow you to run multiple virtual machines that share the same IP address. Think of it as setting up a home network with multiple devices with certain ports forwarded to specific devices for incoming connections.
First, create the virtual machines that are going to use NAT using virt-manager as you normally would. In the virtual machine configuration wizard, assign the default NAT network interface named "virbr0". After the virtual machines have been created, shut down the virtual machines. Now, we'll assign these virtual machines static LAN IP addresses so that we can port forward certain ports and always have them reach the proper virtual machines.
The first thing we need to do is get the MAC address of each virtual machine. Write down the name of the virtual machine and its MAC address, as we'll need this information later on when we edit the NAT interface and assign static LAN IP addresses to our virtual machines. Run this command to retrieve the MAC address for a specific virtual machine.
virsh dumpxml VM_NAME | grep -i '<mac'
Using the MAC address information from the VMs we want to use NAT with, edit the default NAT interface by running the below command.
virsh net-edit default
If for some reason the NAT interface is not named default, you can find it by running the below command:
virsh net-list
After the <range /> entry, assign the static LAN IP addresses similar to the following:
<dhcp> <range start='192.168.122.2' end='192.168.122.254'/> <host mac='52:54:00:ff:4a:2a' name='vm1' ip='192.168.122.13'/> <host mac='52:54:00:bb:35:67' name='vm2' ip='192.168.122.14'/> <host mac='52:54:00:aa:d9:f2' name='vm3' ip='192.168.122.15'/> </dhcp>
Save the file with your desired values and quit the editor. Restart the NAT interface by running the below commands:
virsh net-destroy default virsh net-start default
Now, you'll need to setup your iptables port forwarding rules. Adjust the below rules as necessary (changing the port numbers to the ones you want to use) and then save them so that they persist:
iptables -I FORWARD -o virbr0 -d 192.168.122.13 -j ACCEPT iptables -t nat -I PREROUTING -p tcp --dport 39989 -j DNAT --to 192.168.122.13:39989 iptables -I FORWARD -o virbr0 -d 192.168.122.14 -j ACCEPT iptables -t nat -I PREROUTING -p tcp --dport 39990 -j DNAT --to 192.168.122.14:39990 iptables -I FORWARD -o virbr0 -d 192.168.122.15 -j ACCEPT iptables -t nat -I PREROUTING -p tcp --dport 39991 -j DNAT --to 192.168.122.15:39991 iptables -t nat -A POSTROUTING -s 192.168.122.0/24 -j MASQUERADE iptables -A FORWARD -o virbr0 -m state --state RELATED,ESTABLISHED -j ACCEPT iptables -A FORWARD -i virbr0 -o br0 -j ACCEPT iptables -A FORWARD -i virbr0 -o lo -j ACCEPT iptables-save service iptables save
Congrats, your virtual machines are now using NAT, have been assigned static LAN IP addresses, and iptables rules on the host server have been configured to port forward specific ports to each NAT VM.
Persistently Saving NAT Port Forward Rules
The only solution I found that would persistently save my NAT forwarding rules is to create a libvirt hook bash script as mentioned here.
service iptables stop iptables -F service iptables save service iptables start mkdir -p /etc/libvirt/hooks nano /etc/libvirt/hooks/qemu
The contents of the "/etc/libvirt/hooks/qemu" file should look similar to the following:
#!/bin/bash
# IMPORTANT: Change the "VM NAME" string to match your actual VM Name.
# In order to create rules to other VMs, just duplicate the below block and configure
# it accordingly.
if [ "${1}" = "vm1" ]; then # Update the following variables to fit your setup
GUEST_IP=192.168.122.13
GUEST_PORT=39989
HOST_PORT=39989
if [ "${2}" = "stopped" ] || [ "${2}" = "reconnect" ]; then
/sbin/iptables -D FORWARD -o virbr0 -d $GUEST_IP -j ACCEPT
/sbin/iptables -t nat -D PREROUTING -p tcp --dport $HOST_PORT -j DNAT --to $GUEST_IP:$GUEST_PORT
fi
if [ "${2}" = "start" ] || [ "${2}" = "reconnect" ]; then
/sbin/iptables -I FORWARD -o virbr0 -d $GUEST_IP -j ACCEPT
/sbin/iptables -t nat -I PREROUTING -p tcp --dport $HOST_PORT -j DNAT --to $GUEST_IP:$GUEST_PORT
fi
fi
if [ "${1}" = "vm2" ]; then # Update the following variables to fit your setup
GUEST_IP=192.168.122.14
GUEST_PORT=39990
HOST_PORT=39990
if [ "${2}" = "stopped" ] || [ "${2}" = "reconnect" ]; then
/sbin/iptables -D FORWARD -o virbr0 -d $GUEST_IP -j ACCEPT
/sbin/iptables -t nat -D PREROUTING -p tcp --dport $HOST_PORT -j DNAT --to $GUEST_IP:$GUEST_PORT
fi
if [ "${2}" = "start" ] || [ "${2}" = "reconnect" ]; then
/sbin/iptables -I FORWARD -o virbr0 -d $GUEST_IP -j ACCEPT
/sbin/iptables -t nat -I PREROUTING -p tcp --dport $HOST_PORT -j DNAT --to $GUEST_IP:$GUEST_PORT
fi
fi
if [ "${1}" = "vm3" ]; then # Update the following variables to fit your setup
GUEST_IP=192.168.122.15
GUEST_PORT=39991
HOST_PORT=39991
if [ "${2}" = "stopped" ] || [ "${2}" = "reconnect" ]; then
/sbin/iptables -D FORWARD -o virbr0 -d $GUEST_IP -j ACCEPT
/sbin/iptables -t nat -D PREROUTING -p tcp --dport $HOST_PORT -j DNAT --to $GUEST_IP:$GUEST_PORT
fi
if [ "${2}" = "start" ] || [ "${2}" = "reconnect" ]; then
/sbin/iptables -I FORWARD -o virbr0 -d $GUEST_IP -j ACCEPT
/sbin/iptables -t nat -I PREROUTING -p tcp --dport $HOST_PORT -j DNAT --to $GUEST_IP:$GUEST_PORT
fi
fi
Save and exit. Make the script executable.
Here is an example of forwarding multiple ports with one entry:
if [ "${1}" = "vm_name" ]; then
# Update the following variables to fit your setup
GUEST_IP=192.168.122.3
GUEST_PORTS=(15231 15232 15233)
if [ "${2}" = "stopped" ] || [ "${2}" = "reconnect" ]; then
for i in ${GUEST_PORTS[@]}; do
/sbin/iptables -D FORWARD -o virbr0 -d $GUEST_IP -j ACCEPT
#/sbin/iptables -t nat -D PREROUTING -p tcp --dport ${i} -j DNAT --to $GUEST_IP:${i}
/sbin/iptables -t nat -D PREROUTING -p tcp --dport ${i} -j DNAT --to $GUEST_IP:${i}
#/sbin/iptables -t nat -D PREROUTING -p udp --dport ${i} -j DNAT --to $GUEST_IP:${i}
#/sbin/iptables -t nat -D PREROUTING -p all -j DNAT --to $GUEST_IP:${i}
done
fi
if [ "${2}" = "start" ] || [ "${2}" = "reconnect" ]; then
for i in ${GUEST_PORTS[@]}; do
/sbin/iptables -I FORWARD -o virbr0 -d $GUEST_IP -j ACCEPT
#/sbin/iptables -t nat -I PREROUTING -p tcp --dport ${i} -j DNAT --to $GUEST_IP:${i}
/sbin/iptables -t nat -I PREROUTING -p tcp --dport ${i} -j DNAT --to $GUEST_IP:${i}
#/sbin/iptables -t nat -I PREROUTING -p udp --dport ${i} -j DNAT --to $GUEST_IP:${i}
#/sbin/iptables -t nat -I PREROUTING -p all -j DNAT --to $GUEST_IP:${i}
done
fi
fi
chmod +x /etc/libvirt/hooks/qemu
If you're attempting to forward incoming traffic (only traffic originating from the outside), be sure to exclude the main virbr0 interface so that requests originating from the DHCP LAN aren't forwarded to itself. This is helpful for port forwarding DNS (port 53). Incoming requests from outside of the network will be forwarded to the specified server, but outgoing DNS requests made from the server will NOT be forwarded back to itself. Basically, you want to forward all traffic on port 53 to that specific server except for traffic that originates from that server. So, be sure to add this line if that is the case:
/sbin/iptables -t nat -I PREROUTING \! -i virbr0 -p udp --dport 53 -j DNAT --to $GUEST_IP:$GUEST_PORT
More information can be found here: https://serverfault.com/questions/1042383/kvm-port-forwarding-port-53-to-guest-via-nat-temporary-failure-in-name-resolut/1042394#1042394
Reboot the host server.
Old Instructions for Persistent Saving (Non-Working)
If your iptables forwarding rules are not persisted after the host machine is rebooted or shutdown, run the following commands:
sudo -i yum install -y iptables-services systemctl stop firewalld systemctl disable firewalld systemctl enable iptables nano /etc/sysconfig/iptables-config
Change the below values to "yes":
IPTABLES_SAVE_ON_RESTART="yes" IPTABLES_SAVE_ON_STOP="yes"
Save and exit. Reboot the server.
If you're still having issues, try this (will clear your existing iptables rules):
iptables-save > iptables_bk service iptables stop iptables -F <run iptables NAT rules here> <run any other iptables rules you want> service iptables save service iptables start