Category: Cisco

  • Fix Apple Bonjour with Cisco autonomous APs

    I purchased some used Cisco C1140 autonomous access points for my home network (autonomous meaning not lightweight or requiring a WLC). While everything seemed to be fine at first, later we noticed that printouts to our Canon laser printer were no longer working from our Macs. After some research I realized that the Macs were failing to locate the printer due to Apple Bonjour protocol issues. Google searches led to partial solutions, but most required a downgrade of the AP IOS – a no, no as a security professional.

    I kept looking and it turns out my savior was actually a Chromecast user with the same issue. Two configuration changes on the APs to disable IGMP snooping had to be executed, not one:

    All the prior advice was just to disable the former, which didn’t work (at least without an AP downgrade!). Adding the second line did the full trick.

    You may need to disconnect and reconnect to the wireless for full effect. Since multicast IGMP has other uses, I can’t guarantee the impact in a larger environment.

    UPDATE:

    Well, this may or may not work. In the end it seemed not to for me, but it’s still worth a try in your network.


  • ASA Firewall Rules of Thumb

    Some important Cisco ASA firewall details I and others have learned and shared over the years:

    • Don’t use “security-level” as your method of security. In the long term at best “security-level” will cause you to block traffic you didn’t expect, at worst, it will allow traffic you didn’t want. Why? Well…
    • If you add an ACL on the “in” side of any interface (that is “into the ASA”), once it’s in the ASA, the security level doesn’t matter anymore. It’s very easy to forget this. However you can protect yourself by…
    • Always add “out” rules. Any “in” rules should be matched by “out” rules on the final destination interface. This is insurance in case you missed or were overly broad on your “in” rules.
    • Configure all of the interfaces to the same “security-level”. If you enable “same-security-traffic permit inter-interface” be careful as it allows traffic to flow to other same security levels without ACLs. You don’t want traffic to flow when you haven’t allowed it explicitly. The only exception to using different security levels might be the “outside” interface, which you may want to set to “security-level 0”. However, assuming “outside” is the Internet, ideally you want to be explicit there too. Otherwise you’re potentially setting yourself up for easy, unlogged, data exfiltration (among other things).
    • Remember that the ASA is a stateful firewall. If you establish some sort of connection out of an interface, the firewall should see that the return traffic belongs to the conversation and allow it through regardless. For the most part you don’t need to explicitly create return rules (or use the old IOS “established” trick).
    • If you’re trying to turn up a firewall on a network that existed, but was never firewalled before and you are having difficult categorizing the existing traffic, place the rules that you know are correct into the ASA, then add a “permit ip any any log” entry at the end. This will send logging of what fell to the wildcard rule to your syslog server, which you can then evaluate later. Once analyzed and missing rules in place, turn it to a “deny ip any any” and you’re done. Remember you can also do packet capture on the ASA as well.
    • Never trust a 3rd party. If they are coming into your network and saying they are properly filtering traffic toward you, filter them again anyway. First, their error could be your exploit, second you can’t assume their firewalls aren’t going to get hacked. Protect your network like it was your own child.
    • Beware of mixing ASA “access-list”s and ASA VPNs on the same firewall. Unless you want to enter “filter” hell, which generally you can only apply usefully in one direction, turn off VPN bypass with “no sysopt connection permit-vpn”. If you don’t do this YOUR VPN TRAFFIC BYPASSES ALL “access-list” RULES! Note that once you disable “VPN bypass”, your VPN traffic will appear to come from the “in” of the interface it initially arrived at. Since that’s usually “outside” and the Internet, you can have a seemingly less-than pretty mix of private addressing and public addressing to deal with on your Internet interface. This can make it cleaner to get a dedicated ASA for VPN and hang it off an arm of your firewall ASA.

    The most critical thing with firewalls is don’t be lazy. Take the time to do the configuration and rules needed. It takes extra effort up front, but a failure is far more expensive.


  • Zone Firewall TCP reassembly size

    If you get something like this in your Cisco’s IOS firewall log:

    Mar 12 15:05:33 192.168.1.1 3129: 003121: *Mar 12 15:03:03.195 EST: %FW-4-TCP_OoO_SEG: Dropping TCP Segment: seq:525214740 1415 bytes is out-of-order; expected seq:525170856. Reason: TCP reassembly queue overflow – session 192.168.1.5:53022 to 208.79.250.63:80 on zone-pair ccp-zp-in-out class ccp-protocol-http

    sometimes accompanied by hangs in downloads, then what is happening is you are blowing out the buffers used to reassemble TCP segments when the segments have arrived “out-of-order” (also abbreviated “OoO”).

    The problem for a stateful firewall or IDS/IPS is it often needs to see more of the packet stream than just the initial segment to make a forwarding/block decision. Thus it has to collect these segments together, however sometimes the segments don’t arrive “in order”. This can particularly happen when VPN is used.

    In order to get around this, it essentially collects the streaming segments going by in a queue until it can find the missing segment (assumed to be “out-of-order”). It queues those packets in memory, but for obvious reasons it cannot have infinitely sized queues – it would run out of resources. In fact if it did, this would offer a very effective DoS (Denial of Service) attack.

    Thus, there are defined limits set to the TCP reassembly queue. Those limits are actually fairly small to start (16 entries and 1 mb), thus you may want to adjust them if you are regularly seeing messages like above.

    Using the old CBAC method of inspection, you could insert the following command:

    ip inspect tcp reassembly {[queue length packet-number] [timeout seconds] [memory limit size-in-kb] [alarm {on | off}]}

    However with the newer Zone Firewall inspection methods don’t use the same settings. Instead the new command format is:

    parameter-map type ooo global
    tcp reassembly alarm {on | off}
    tcp reassembly memory limit
    memory-limit-kb
    tcp reassembly queue length queue-length
    tcp reassembly timeout
    time-limit-secs

    To note the defaults are as follows:

    parameter-map type ooo global
    tcp reassembly alarm off
    tcp reassembly memory limit 1024
    tcp reassembly queue length 16
    tcp reassembly timeout 5

    So, if say you wanted to quadruple the default queue/memory lengths:

    parameter-map type ooo global
    tcp reassembly memory limit 4096
    tcp reassembly queue length 64

    Note it’s not clear if a dropped segment appears the same as an “out-of-order” segment to the router – that is with a dropped/lost segment the router keeps expecting it to arrive, just out of order. Thus the error could be telling you more that you’re dropping packets than you’re blowing out your “out-of-order” queues. Unfortunately I cannot find documentation one way or another on this.

    Also to note if you’re increasing the queue length, you might want to increase the timeout (“tcp reassembly timeout time-limit-secs“), however 5 seconds is an awful long time for a segment that might be out-of-order not to arrive. As bandwidth increases, while it is likely that more packets/bytes might come in to blow out the queue, it’s unlikely they would take more time to do so (quite the opposite – an out-of-order packet at higher bandwidth is if anything likely to show up sooner, not later), thus I wouldn’t expect this to need adjustment.


  • How to kill a session on a Cisco PIX/FWSM

    Completely different from Cisco IOS, so hard to remember:

    Log into the PIX/FWSM and go to “enable” mode. Do a “who”:

    Choose the IP of the session you want to kill and grab the number. In this case I want to kill the “192.168.100.5” session, so I want “2”. Then kill it:

    The target session will then drop.

    Note if you’re coming from the same IP it may make it harder because the sessions will reference the same IP. In that case, just assume the later session has a higher number (or conversely, the earlier session has a lower number).

    Be careful. I have no idea what this does is you’re in mid-access-list update.


  • Fixing that stupid Cisco IOS telnet thing…

    One of the things that most drives me crazy about Ciscos is the default setting that makes when you’re at a Cisco IOS “exec” prompt that if you type something that isn’t a command, it interprets it as an attempt to “telnet” to a host. This is a real pain in the backside as all typos become unwanted telnet attempts. It’s just a dumb default really.

    Fortunately the fix is simple. Just go into the VTY settings and add a “transport preferred none”, eg:

    and that P.I.A. is fixed.


  • Unlocking a Cisco IP phone

    One of those things I can never remember! It’s:

    **#

    Lets you change the network configs among other things.

    Also can factory reset with:

    Settings> Phone settings> Press **2

    Works on older 7921 at least.


  • Defaulting a Cisco interface…

    One pain with Cisco IOS is trying to get a configured interface back to defaults. Half the time you don’t even remember what those were.

    If it’s a sub-interface you can “no” it, but you will still have configuration left behind:

    cisco(config)#no interface ATM1/0.1
    Not all config may be removed and may reappear after reactivating the sub-interface

    with physical top level interfaces you can’t “no” them at all anyway.

    The answer is to use the “default” command:

    cisco(config)#default interface ATM1/0.1
    Building configuration…

    Interface ATM1/0.1 set to default configuration

    Curiously this doesn’t seem to clear PVC definitions!

    NOTE: If you do this on the primary physical interface, all sub-interfaces will be defaulted and deleted (which may or may not be what you want). So use carefully!

    UPDATE: Well the docs say the sub-interfaces will be deleted, but they’re not in my experience. Also it’s not even clear if this works on sub-interfaces. The combination of a “no” on the sub-interface first and then a “default” after the fact seemed to maybe work, but no promises.


  • How to remove a VLAN from a port in CatOS…

    I can never seem to remember how to “remove” a VLAN on a switch (eg: Cisco 6500) running the older CatOS. The new IOS based switches are much easier.

    Anyway, it’s actually quite simple, just force the port to VLAN 1 (assuming that is your default/native VLAN). For example if port 6/5 was set to a VLAN and you wanted to remove it, just type:

    set vlan 1 6/5

    and bingo it’s removed from the current VLAN.

    Often this is necessary when reusing a port that had a VLAN assignment but you want to use as a trunk.