Cloud Security (Part 1): Passive Security Monitoring in RackSpace
This is Part 1 of our “Cloud Security” series, with a goal of setting up a simple passive security monitoring capability in the cloud. Future posts will show how to enable out-of-band management, as well as setting up the monitoring services themselves. Enjoy!
Over the past few years, Rackspace has been making its way into the enterprise Cloud Provider market as a competitor to Amazon EC2. While the power and efficiencies of “the cloud” can be enticing, the security is often overlooked by new-comers that build simple servers directly connected to the internet. In this post, I’m going to show how to quickly setup an inexpensive gateway, firewall, and passive network security monitoring capability within a Rackspace hosted cloud using a simple CentOS Linux system acting as the gateway to your internal, DMZ (passive), and management networks. The diagram below shows the final outcome of this setup. This is a very simple setup to show the concept of clone & forwarding traffic to monitoring devices. An enterprise network would likely have a much more complex setup (adding users, securing the systems using a well-documented benchmark (e.g. DISA STIGs or Red Hat Security Guide).
Example Virtual Machines
Rackspace allows users to create internal private networks that systems can be joined to. In our example, we use 3 internal networks (this is in addition to the pre-built “ServiceNet” and “PublicNet”).
- MGMT: Used for managing system
- DMZ: Monitor network
- Internal: Houses the Application servers
In Rackspace, we created 3 virtual machines (NOTE: the VM Types can be changed based on needs):
**NOTE: CentOS/RHEL 6.5 kernel does not include the “-j TEE” options that allows the gateway to clone and forward packets. CentOS/RHEL 7 does.
In this example, only the Gateway is accessible from the internet. To configure the other systems, you will need to first configure the gateway to allow access. This can be done using Putty, or whatever you’re preferred SSH client is. Note that Rackspace automatically assigns parts to networks. Your ports might vary from what I have in this example. Please adjust your commands and methodology accordingly.
The following steps will setup the Gateway to route traffic through the firewall to the other systems, as well as perform port forwarding, and “clone & forward” of all traffic to a monitoring system.
For security reasons, I always make updating the first step.
yum –y update
Enable IP Forwarding on the system, allowing the system to route packets between interfaces (OFF by default)
Insert the following lines at the end of the file, and exit
#Enable forwarding of IPv4 Packets
net.ipv4.ip_forward = 1
Reload the kernel with updated parameters
CentOS/RHEL 7 comes with the new firewall userland application known as “firewalld”. There are benefits to using firewalld over iptables (e.g. no traffic interruption when reloading), so we will use the updated command sets to perform the tasks. However the same effect can be gotten from using iptables commands. First thing to do is enable MASQUERADING on your external zone (it may already be there by default…)
firewall-cmd –permanent –zone=external –add-masquerade
Next, place your interfaces into separate “zones” (we’ll use the default zones that come with CentOS/RHEL)
firewall-cmd –zone=external –change-interface=eth0
firewall-cmd –zone=internal –change-interface=eth3
firewall-cmd –zone=dmz –change-interface=eth2
firewall-cmd –zone=trusted –change-interface=eth4
Because we have a Windows Application Server, we will forward all inbound Remote Desktop traffic through the gateway to the Windows server. This can be done with the following command[
firewall-cmd –permanent –zone=external –add-forward-port=port=3389:proto=tcp:toport=3389:toaddr=192.168.5.3
**You’ll want to forward any additional services that you’re application will be providing (e.g. HTTPS, FTP, SQL, etc.). DON’T forward SSH (port 22) or you’ll need to connected via console to fix it!!
For passive monitoring, we want to “clone & forward” all inbound and outbound traffic to/from the internal network. The next command will take care of the outbound traffic
firewall-cmd –permanent –direct –add-rule ipv4 mangle PREROUTING 0 -i eth3 -j TEE –gateway 192.168.4.2
**The “–direct” means we are passing in advanced options, in this case to the mangle table. This command instructs netfilter to act on all traffic inbound from eth3 (-i eth3), before routing it (PREROUTING), and clone and forward (-j TEE) it to our monitoring device (–gateway 192.168.4.2)
The last command only handles half the story (outbound traffic). We want to see traffic coming from the internet as well (inbound traffic). The following rule will handle that.
firewall-cmd –permanent –direct –add-rule ipv4 mangle FORWARD 0 -d 192.168.3.0/24 -j TEE –gateway 192.168.4.2
**If you have more than one network to monitor, you’ll need to have a rule for each network (modify the -d x.x.x.x/24 accordingly).
Now your gateway is setup as a network firewall and to route internal applications to the internet, as well as passively forward traffic to monitoring devices.
Setup Internal Application Server
The only setup that you’ll need to perform on your internal servers/clients is to point the default route (gateway) at the gateway server address (192.168.3.1). Traffic will NAT through it just like any other firewall. Don’t forget to update your system with the latest patches!!
Setup Monitoring System
You’ll need want to configure the monitoring system with a default gateway on the MGMT network (192.168.5.1). At this point, all traffic going to/from your internal/application network is being forwarded to your monitoring system on eth2. Be sure to access this system via the MGMT network (192.168.5.2) so that you don’t interfere with the monitor traffic.
To validate everything is working correctly, login to your monitoring system from the gateway (ssh 192.168.5.2), install tcpdump, and start monitoring eth1:
yum -y update
yum -y install tcpdump
tcpdump -i eth1 -v
Now login to your Application server and ping an external IP address (e.g. 126.96.36.199). If everything was setup correctly, you should see something like this:
03:49:35.540763 IP 192.168.3.2 > google-public-dns-a.google.com: ICMP echo request, id 1, seq 9, length 40 03:49:35.542349 IP google-public-dns-a.google.com > 192.168.3.2: ICMP echo reply, id 1, seq 9, length 40
You can see that you are getting both the outbound ping as well as the inbound ping. IDS, NetFlow, and other security devices that you install on this server will be able to monitor that traffic effectively, without causing a bottleneck to your applications.
From a functional standpoint, you’re all setup. Since security systems are generally sensitive in nature, you’ll likely want to add additional firewall rules in your gateway to prevent routing of traffic between the DMZ network and the Internal network. It’s also a good idea to have your MGMT network be completely out of band and place rules to prevent traffic from routing to/from the MGMT network at all. In a future post, we’ll dive into how to setup a VPN on your gateway to get into your out-of-band network (MGMT) so that you can always access and administrate your systems.