Category: Cisco

Some important Cisco ASA firewall details I and others have learned and shared over the years:

• Don’t use “security-level” as your method of security. In the long term at best “security-level” will cause you to block traffic you didn’t expect, at worst, it will allow traffic you didn’t want. Why? Well…
• If you add an ACL on the “in” side of any interface (that is “into the ASA”), once it’s in the ASA, the security level doesn’t matter anymore. It’s very easy to forget this. However you can protect yourself by…
• Always add “out” rules. Any “in” rules should be matched by “out” rules on the final destination interface. This is insurance in case you missed or were overly broad on your “in” rules.
• Configure all of the interfaces to the same “security-level”. If you enable “same-security-traffic permit inter-interface” be careful as it allows traffic to flow to other same security levels without ACLs. You don’t want traffic to flow when you haven’t allowed it explicitly. The only exception to using different security levels might be the “outside” interface, which you may want to set to “security-level 0”. However, assuming “outside” is the Internet, ideally you want to be explicit there too. Otherwise you’re potentially setting yourself up for easy, unlogged, data exfiltration (among other things).
• Remember that the ASA is a stateful firewall. If you establish some sort of connection out of an interface, the firewall should see that the return traffic belongs to the conversation and allow it through regardless. For the most part you don’t need to explicitly create return rules (or use the old IOS “established” trick).
• If you’re trying to turn up a firewall on a network that existed, but was never firewalled before and you are having difficult categorizing the existing traffic, place the rules that you know are correct into the ASA, then add a “permit ip any any log” entry at the end. This will send logging of what fell to the wildcard rule to your syslog server, which you can then evaluate later. Once analyzed and missing rules in place, turn it to a “deny ip any any” and you’re done. Remember you can also do packet capture on the ASA as well.
• Never trust a 3rd party. If they are coming into your network and saying they are properly filtering traffic toward you, filter them again anyway. First, their error could be your exploit, second you can’t assume their firewalls aren’t going to get hacked. Protect your network like it was your own child.
• Beware of mixing ASA “access-list”s and ASA VPNs on the same firewall. Unless you want to enter “filter” hell, which generally you can only apply usefully in one direction, turn off VPN bypass with “no sysopt connection permit-vpn”. If you don’t do this YOUR VPN TRAFFIC BYPASSES ALL “access-list” RULES! Note that once you disable “VPN bypass”, your VPN traffic will appear to come from the “in” of the interface it initially arrived at. Since that’s usually “outside” and the Internet, you can have a seemingly less-than pretty mix of private addressing and public addressing to deal with on your Internet interface. This can make it cleaner to get a dedicated ASA for VPN and hang it off an arm of your firewall ASA.

The most critical thing with firewalls is don’t be lazy. Take the time to do the configuration and rules needed. It takes extra effort up front, but a failure is far more expensive.

If you get something like this in your Cisco’s IOS firewall log:

Mar 12 15:05:33 192.168.1.1 3129: 003121: *Mar 12 15:03:03.195 EST: %FW-4-TCP_OoO_SEG: Dropping TCP Segment: seq:525214740 1415 bytes is out-of-order; expected seq:525170856. Reason: TCP reassembly queue overflow – session 192.168.1.5:53022 to 208.79.250.63:80 on zone-pair ccp-zp-in-out class ccp-protocol-http

sometimes accompanied by hangs in downloads, then what is happening is you are blowing out the buffers used to reassemble TCP segments when the segments have arrived “out-of-order” (also abbreviated “OoO”).

The problem for a stateful firewall or IDS/IPS is it often needs to see more of the packet stream than just the initial segment to make a forwarding/block decision. Thus it has to collect these segments together, however sometimes the segments don’t arrive “in order”. This can particularly happen when VPN is used.

In order to get around this, it essentially collects the streaming segments going by in a queue until it can find the missing segment (assumed to be “out-of-order”). It queues those packets in memory, but for obvious reasons it cannot have infinitely sized queues – it would run out of resources. In fact if it did, this would offer a very effective DoS (Denial of Service) attack.

Thus, there are defined limits set to the TCP reassembly queue. Those limits are actually fairly small to start (16 entries and 1 mb), thus you may want to adjust them if you are regularly seeing messages like above.

Using the old CBAC method of inspection, you could insert the following command:

ip inspect tcp reassembly {[queue length packet-number] [timeout seconds] [memory limit size-in-kb] [alarm {on | off}]}

However with the newer Zone Firewall inspection methods don’t use the same settings. Instead the new command format is:

parameter-map type ooo global
tcp reassembly alarm {on | off}
tcp reassembly memory limit memory-limit-kb
tcp reassembly queue length queue-length
tcp reassembly timeout time-limit-secs

To note the defaults are as follows:

parameter-map type ooo global
tcp reassembly alarm off
tcp reassembly memory limit 1024
tcp reassembly queue length 16
tcp reassembly timeout 5

So, if say you wanted to quadruple the default queue/memory lengths:

parameter-map type ooo global
tcp reassembly memory limit 4096
tcp reassembly queue length 64

Note it’s not clear if a dropped segment appears the same as an “out-of-order” segment to the router – that is with a dropped/lost segment the router keeps expecting it to arrive, just out of order. Thus the error could be telling you more that you’re dropping packets than you’re blowing out your “out-of-order” queues. Unfortunately I cannot find documentation one way or another on this.

Also to note if you’re increasing the queue length, you might want to increase the timeout (“tcp reassembly timeout time-limit-secs“), however 5 seconds is an awful long time for a segment that might be out-of-order not to arrive. As bandwidth increases, while it is likely that more packets/bytes might come in to blow out the queue, it’s unlikely they would take more time to do so (quite the opposite – an out-of-order packet at higher bandwidth is if anything likely to show up sooner, not later), thus I wouldn’t expect this to need adjustment.

Completely different from Cisco IOS, so hard to remember:

Log into the PIX/FWSM and go to “enable” mode. Do a “who”:

Choose the IP of the session you want to kill and grab the number. In this case I want to kill the “192.168.100.5” session, so I want “2”. Then kill it:

The target session will then drop.

Note if you’re coming from the same IP it may make it harder because the sessions will reference the same IP. In that case, just assume the later session has a higher number (or conversely, the earlier session has a lower number).

Be careful. I have no idea what this does is you’re in mid-access-list update.