Configuring firewall rules in operational environments

Reconfiguring firewalls in operational environments are rife with concerns. This article helps to develop a plan for implementing change.

Operational enterprise environments are tempermental. Touch one thing, break another. Replace a server, break the interfaces to that server. Increase the security posture of the organization by changing an operational firewall? Well, we don’t want to think about that!

Wait. Actually, we do want to think about increasing the organization’s security posture.

This article focuses on protecting enterprises with outbound firewall rules. The same plan can also be used to protect inbound firewall rules.

Introducing firewalls

Firewalls are security devices that protect enterprises from uncontrolled network flows, in much the same way as dams protect towns from uncontrolled water flows. Most enterprises recognize firewalls as “inbound protection devices”. But firewalls are much more than inbound protection devices. Configured correctly, firewalls protect against unauthorized inbound traffic AND unauthorized outbound traffic.

What does this mean? Consider an adversary (possibly an insider) has landed on your network. This is already a bad situation — something has happened that allowed the adversary to wind up on the network.

This is where your outbound firewall configuration comes in. The firewall can make it more difficult for the adversary to exfiltrate data from your network. Even though the adversary is on the network, getting sensitive data out of the network can be made more difficult with the use of firewalls.

Firewalls in newly deployed enterprises

Greenfield firewall deployments require configuration, but are less likely to impact operational users

Configuring firewalls in new environments is a much simpler task than configuring firewalls in operational environments. In a new environment, the firewall can start life with outbound connections set to Block All. Each new device, each new service, can be assessed for traffic requirements. For example, you know your employees need to access web sites? Open outbound TCP 80 and 443 for the endpoint IPs. You know a server engineer needs to sftp to a remote server? Open outbound TCP 22 for that server IP.

Firewalls in operational environments

Operational environments require a bit more planning and diligence. The problem is that blocking all ports is going to break everything — suddenly, nothing will work.

Complex environments are tempermental. Changes need to be structured and planned.

The basis of this recommendation is: Make a plan! Whatever you are going to do, make sure you’ve developed a plan, and make sure the plan includes adjoining backout steps.

Here is an operational plan for changing firewall rules that will work in every environment.

1. Monitor netflows

Understanding basic network metrics is the best place to start in protecting an existing environment with firewalls. Users are not impacted since traffic shaping does not occur during the monitor phase.

The monitor phase should continue for at least a month, more reasonably at least a quarter. The reason for this duration is that vendor software updates are normally scheduled to occur at least quarterly, and the monitor phase must include capturing netflows during these update cycles. For example, Microsoft and other vendors initiate the the infamous “Patch Tuesday“. The update traffic must be reflected in the Monitor phase.

The monitor phase metrics produce two primary report artifacts.

• First, ports that are not used during the normalization phase can be considered for blocking (explained in the next phase).
• Second, the netflows can be used during threat hunts — the threat hunters can understand “normal” traffic, and thereby can also recognize “not normal” traffic.

Network Threat hunts require data flows

Recognizing “not normal” traffic is a key to threat hunting. During a threat hunt, the team is looking for anomalies, for traffic that just doesn’t belong. If netflow has been collected for a reasonable amount of time, a netflow shows up that doesn’t belong might be a key sign of trouble that needs to be investigated by the threat hunt team.

Ports that wind up on the “open” list continue to be monitored for excess traffic. The threat hunters need to investigate if a normally low traffic port suddenly exhibits a large traffic pool.

2. Identify “unused” ports

The second phase of turning up the outbound firewall rules is to identify the known “unused” ports.

These ports are easily identified in the Netflow since the unused ports simply will not show up. For example, if Port 3389 (a port associated with Remote Desktop Connection) doesn’t show up during the monitor phase, and the team does not know of any reasonable work related outbound remote desktop connections, then it is likely reasonable to block outbound Port 3389.

A reasonable warning is to be cautious about hastening to Port Blocks. Blocking “expected unused” ports can introduce problems for an operational environment. The problems will surface when ports that appear unused during the Monitor phase are in fact required ports, but the Monitor phase was of insufficient duration to identify the port as operationally required.

For some operational environments, it may be more reasonable to “alert on unused ports” instead of “block unused ports” during the early parts of the transition. However at some point this phase must conclude with “block” unused ports.

3. Refactor “used” ports

Once the “known unused” ports have been handled successfully, it is time to move on to refactoring the “used” ports.

The simplest refactoring will identify “all endpoints” allowed outbound traffic to “all destinations” over “listed ports”. However, this is just the beginning of tightening down the firewall.

In operational environments, refactoring operational ports is likely a multi-phased approach; one phase covering workstation endpoints; another phase covering servers; and several phases covering “other endpoints” like phones, cameras, and keypads/door entry systems. Eventually the firewall will have a collection of rules for many different types of endpoints.

For example, say that Ports 25, 465, and 587 show up in the “operational port” report. These ports are associated with SMTP (or Simple Mail Transport Protocol). While it is reasonable for a mail relay such as an Exchange server to communicate over these ports, it is less reasonable that a workstation/user endpoint relay their own mail.

Another example exists for web traffic over 80 and 443. While it may be reasonable to open web traffic for all endpoints, an adversary can use those open ports to exfiltrate traffic. One might consider, is it reasonable for a server to contact web sites over 80 & 443, or only endpoints configured for user traffic? Even moreso, is it appropriate for the endpoints to communicate out directly, or is there a web proxy protecting the end users from visiting known malicious web sites?

During the refactor phase, it is reasonable to lock down these three ports to only Exchange servers or other mail relay servers, and disallow endpoint communication over these ports.

4. Continue monitoring netflows

Threat hunters are in a constant battle with threat actors. The more data available for the hunt, the more likely the hunt will succeed.

Threat hunters need data. Continue monitoring netflows even after the firewalls have been normalized. This provides data that is useful for threat hunters. Identifying anomalies is a bases for alert generation, and identifying anomalous traffic volumes is an event that should trigger an alert.

Conclusion and after thoughts

Managing operational environments is a task in balancing many parts of a complex puzzle, from satisfying user demands, to enforcing security, to addressing Cxx level board room concerns for the future. Managing underused firewalls in these operational environments can be an undoubtedly perilous concern, and equally necessary for properly protecting the environment.

As always, Prior planning prevents poor performance, and this adage holds true for deploying Firewall changes in operational environments. Make a plan, and stick to it. If the need arises to deviate from the plan, change the plan itself and restart instead of deviating from the plan.