Multiple IPs with DMZ and Internal Servers

Corporate Network

Notes:

This configuration is used on a corporate network that has a Linux (RedHat 8.0) server with three interfaces, running Shorewall 1.4.5 release,

Make sure you know what public IP addresses are currently being used and verify these before starting.

Verify you DNS settings before starting any Shorewall configuration especially if you have split DNS.

System names and Internet IP addresses have been changed to protect the innocent.

Warning: This configuration uses a combination of Static NAT and Proxy ARP. This is generally not relevant to a simple configuration with a single public IP address. If you have just a single public IP address, most of what you see here won't apply to your setup so beware of copying parts of this configuration and expecting them to work for you. What you copy may or may not work in your configuration.

I have a T1 with 64 static IP addresses (63.123.106.65-127/26). The internet is connected to eth0. The local network is connected via eth1 (10.10.0.0/22) and the DMZ is connected to eth2 (192.168.21.0/24). I have an IPSec tunnel connecting our offices in Germany to our offices in the US. I host two Microsoft Exchange servers for two different companies behind the firewall hence, the two Exchange servers in the diagram below.

Summary:

SNAT for all systems connected to the LAN - Internal addresses 10.10.x.x to external address 63.123.106.127.

Static NAT for Polaris (Exchange Server #2). Internal address 10.10.1.8 and external address 63.123.106.70.

Static NAT for Sims (Inventory Management server). Internal address 10.10.1.56 and external address 63.123.106.75.

Static NAT for Project (Project Web Server). Internal address 10.10.1.55 and external address 63.123.106.84.

Static NAT for Fortress (Exchange Server). Internal address 10.10.1.252 and external address 63.123.106.93.

Static NAT for BBSRV (Blackberry Server). Internal address 10.10.1.230 and external address 63.123.106.97.

Static NAT for Intweb (Intranet Web Server). Internal address 10.10.1.60 and external address 63.123.106.115.

The firewall runs on a 2Gb, Dual PIV/2.8GHz, Intel motherboard with RH8.0.

The Firewall is also a proxy server running Privoxy 3.0.

The single system in the DMZ (address 63.123.106.80) runs sendmail, imap, pop3, DNS, a Web server (Apache) and an FTP server (vsFTPd 1.1.0). That server is managed through Proxy ARP.

All administration and publishing is done using ssh/scp. I have X installed on the firewall and the system in the DMZ. X applications tunnel through SSH to Hummingbird Exceed running on a PC located in the LAN. Access to the firewall using SSH is restricted to systems in the LAN, DMZ or the system Kaos which is on the Internet and managed by me.

The Ethernet 0 interface in the Server is configured with IP address 63.123.106.68, netmask 255.255.255.192. The server's default gateway is 63.123.106.65, the Router connected to my network and the ISP. This is the same default gateway used by the firewall itself. On the firewall, Shorewall automatically adds a host route to 63.123.106.80 through Ethernet 2 (192.168.21.1) because of the entry in /etc/shorewall/proxyarp (see below). I modified the start, stop and init scripts to include the fixes suggested when having an IPSec tunnel.

Some Mistakes I Made:

Yes, believe it or not, I made some really basic mistakes when building this firewall. Firstly, I had the new firewall setup in parallel with the old firewall so that there was no interruption of service to my users. During my out-bound testing, I set up systems on the LAN to utilize the firewall which worked fine. When testing my NAT connections, from the outside, these would fail and I could not understand why. Eventually, I changed the default route on the internal system I was trying to access, to point to the new firewall and "bingo", everything worked as expected. This oversight delayed my deployment by a couple of days not to mention level of frustration it produced.

Another problem that I encountered was in setting up the Proxyarp system in the DMZ. Initially I forgot to remove the entry for the eth2 from the /etc/shorewall/masq file. Once my file settings were correct, I started verifying that the ARP caches on the firewall, as well as the outside system "kaos", were showing the correct Ethernet MAC address. However, in testing remote access, I could access the system in the DMZ only from the firewall and LAN but not from the Internet. The message I received was "connection denied" on all protocols. What I did not realize was that a "helpful" administrator that had turned on an old system and assigned the same address as the one I was using for Proxyarp without notifying me. How did I work this out. I shutdown the system in the DMZ, rebooted the router and flushed the ARP cache on the firewall and kaos. Then, from kaos, I started pinging that IP address and checked the updated ARP cache and lo-and-behold a different MAC address showed up. High levels of frustration etc., etc. The administrator will not be doing that again! :-)

Lessons Learned:

Read the documentation.

Draw your network topology before starting.

Understand what services you are going to allow in and out of the firewall, whether they are TCP or UDP packets and make a note of these port numbers.

Try to get quiet time to build the firewall - you need to focus on the job at hand.

When asking for assistance, be honest and include as much detail as requested. Don't try and hide IP addresses etc., you will probably screw up the logs and make receiving assistance harder.

Read the documentation.

Futures:

This is by no means the final configuration. In the near future, I will be moving more systems from the LAN to the DMZ. I will also be watching the logs for port scan programs etc. but, this should be standard security maintenance.

Here are copies of my files. I have removed most of the internal documentation for the purpose of this space however, my system still has the original files with all the comments and I highly recommend you do the same.

Shorewall.conf

##############################################################################
# /etc/shorewall/shorewall.conf V1.4 - Change the following variables to
# match your setup
#
# This program is under GPL [http://www.gnu.org/copyleft/gpl.htm]
#
# This file should be placed in /etc/shorewall
#
# (c) 1999,2000,2001,2002,2003 - Tom Eastep (teastep@shorewall.net)
##############################################################################
# L O G G I N G
##############################################################################
LOGFILE=/var/log/messages
LOGFORMAT="Shorewall:%s:%s:"
LOGRATE=
LOGBURST=
LOGUNCLEAN=info
BLACKLIST_LOGLEVEL=
LOGNEWNOTSYN=
MACLIST_LOG_LEVEL=info
TCP_FLAGS_LOG_LEVEL=debug
RFC1918_LOG_LEVEL=debug
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin
SUBSYSLOCK=/var/lock/subsys/shorewall
STATEDIR=/var/lib/shorewall
MODULESDIR=
FW=fw
NAT_ENABLED=Yes
MANGLE_ENABLED=Yes
IP_FORWARDING=On
ADD_IP_ALIASES=Yes
ADD_SNAT_ALIASES=Yes
TC_ENABLED=Yes
CLEAR_TC=No
MARK_IN_FORWARD_CHAIN=No
CLAMPMSS=No
ROUTE_FILTER=Yes
NAT_BEFORE_RULES=No
MULTIPORT=Yes
DETECT_DNAT_IPADDRS=Yes
MUTEX_TIMEOUT=60
NEWNOTSYN=Yes
BLACKLIST_DISPOSITION=DROP
MACLIST_DISPOSITION=REJECT
TCP_FLAGS_DISPOSITION=DROP
#LAST LINE -- DO NOT REMOVE

Zones File

#
# Shorewall 1.4 -- Sample Zone File For Two Interfaces
# /etc/shorewall/zones
#
# This file determines your network zones. Columns are:
#
# ZONE Short name of the zone
# DISPLAY Display name of the zone
# COMMENTS Comments about the zone
#
#ZONE DISPLAY COMMENTS
net Net Internet
loc Local Local Networks
dmz DMZ Demilitarized Zone
vpn1 VPN1 VPN to Germany
#LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE

Interfaces File:

##############################################################################
#ZONE INTERFACE BROADCAST OPTIONS
net eth0 62.123.106.127 routefilter,norfc1918,blacklist,tcpflags
loc eth1 detect dhcp,routefilter
dmz eth2 detect
vpn1 ipsec0
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE

Routestopped File:

#INTERFACE HOST(S)
eth1 -
eth2 -
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE

Policy File:

###############################################################################
#SOURCE DEST POLICY LOG LEVEL LIMIT:BURST
loc net ACCEPT
loc fw ACCEPT
loc dmz ACCEPT
# If you want open access to the Internet from your Firewall 
# remove the comment from the following line.
fw net ACCEPT
fw loc ACCEPT
fw dmz ACCEPT
dmz fw ACCEPT
dmz loc ACCEPT
dmz net ACCEPT
# 
# Adding VPN Access
loc vpn1 ACCEPT
dmz vpn1 ACCEPT
fw vpn1 ACCEPT
vpn1 loc ACCEPT
vpn1 dmz ACCEPT
vpn1 fw ACCEPT
#
net all DROP info
all all REJECT info
#LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE

Masq File:

#INTERFACE SUBNET ADDRESS
eth0 eth1 163.123.106.126
#
#LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE

NAT File:

#EXTERNAL INTERFACE INTERNAL ALL INTERFACES LOCAL
#
# Intranet Web Server
63.123.106.115 eth0:0 10.10.1.60 No No
#
# Project Web Server
63.123.106.84 eth0:1 10.10.1.55 No No
#
# Blackberry Server
63.123.106.97 eth0:2 10.10.1.55 No No
#
# Corporate Mail Server
63.123.106.93 eth0:3 10.10.1.252 No No
#
# Second Corp Mail Server
63.123.106.70 eth0:4 10.10.1.8 No No
#
# Sims Server
63.123.106.75 eth0:5 10.10.1.56 No No
#
#LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE

Proxy ARP File:

#ADDRESS INTERFACE EXTERNAL HAVEROUTE
#
# The Corporate email server in the DMZ
63.123.106.80 eth2 eth0 No
#
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE

Tunnels File:

# TYPE ZONE GATEWAY GATEWAY ZONE PORT
ipsec net 134.147.129.82
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE

Rules File (The shell variables are set in /etc/shorewall/params):

##############################################################################
#ACTION SOURCE DEST PROTO DEST SOURCE ORIGINAL
# PORT PORT(S) DEST
#
# Accept DNS connections from the firewall to the network
#
ACCEPT fw net tcp 53
ACCEPT fw net udp 53
#
# Accept SSH from internet interface from kaos only
#
ACCEPT net:63.123.106.98 fw tcp 22
#
# Accept connections from the local network for administration 
#
ACCEPT loc fw tcp 20:22
ACCEPT loc net tcp 22
ACCEPT loc fw tcp 53
ACCEPT loc fw udp 53
ACCEPT loc net tcp 53
ACCEPT loc net udp 53
#
# Allow Ping To And From Firewall
#
ACCEPT loc fw icmp 8
ACCEPT loc dmz icmp 8
ACCEPT loc net icmp 8
ACCEPT dmz fw icmp 8
ACCEPT dmz loc icmp 8
ACCEPT dmz net icmp 8
DROP net fw icmp 8
DROP net loc icmp 8
DROP net dmz icmp 8
ACCEPT fw loc icmp 8
ACCEPT fw dmz icmp 8
DROP fw net icmp 8
#
# Accept proxy web connections from the inside
#
ACCEPT loc fw tcp 8118
#
# Forward PcAnywhere, Oracle and Web traffic from outside to the Demo systems
# From a specific IP Address on the Internet.
# 
# ACCEPT net:207.65.110.10 loc:10.10.3.151 tcp 1521,http
# ACCEPT net:207.65.110.10 loc:10.10.2.32 tcp 5631:5632
#
# Intranet web server
ACCEPT net loc:10.10.1.60 tcp 443
ACCEPT dmz loc:10.10.1.60 tcp 443
#
# Projects web server
ACCEPT net loc:10.10.1.55 tcp 80
ACCEPT dmz loc:10.10.1.55 tcp 80
# 
# Blackberry Server
ACCEPT net loc:10.10.1.230 tcp 3101
#
# Corporate Email Server
ACCEPT net loc:10.10.1.252 tcp 25,53,110,143,443
#
# Corporate #2 Email Server
ACCEPT net loc:10.10.1.8 tcp 25,80,110,443
#
# Sims Server
ACCEPT net loc:10.10.1.56 tcp 80,443
ACCEPT net loc:10.10.1.56 tcp 7001:7002
ACCEPT net:63.83.198.0/24 loc:10.10.1.56 tcp 5631:5632
#
# Access to DMZ
ACCEPT loc dmz udp 53,177
ACCEPT loc dmz tcp 80,25,53,22,143,443,993,20,110 -
ACCEPT net dmz udp 53
ACCEPT net dmz tcp 25,53,22,21,123
ACCEPT dmz net tcp 25,53,80,123,443,21,22
ACCEPT dmz net udp 53
#
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE

Start File:

############################################################################
# Shorewall 1.4 -- /etc/shorewall/start
#
# Add commands below that you want to be executed after shorewall has
# been started or restarted.
#
qt service ipsec start

Stop File:

############################################################################
# Shorewall 1.4 -- /etc/shorewall/stop
#
# Add commands below that you want to be executed at the beginning of a
# "shorewall stop" command.
#
qt service ipsec stop

Init File:

############################################################################
# Shorewall 1.4 -- /etc/shorewall/init
#
# Add commands below that you want to be executed at the beginning of
# a "shorewall start" or "shorewall restart" command.
#
qt service ipsec stop

Last updated 7/16/2003