Just own it…

“Even robust defenses and prosecutors aren’t sufficient to protect against the state-sponsored attack, especially when they’re extremely sophisticated and persistent,” Marissa Mayer testified.

Just own it. This “the Russians did it” is becoming a tired trope. There is now way to even conclusively know if a state sponsored entity did do it. From what I understand Yahoo! still doesn’t even know the definitive source of the breach. Moreover, Google hasn’t lost 3 billion IDs, so clearly it’s possible to defend against such attacks.

That said, there is some truth in the statement – a concerted hacker, and it doesn’t need to be a state sponsored hacker, will take most organizations down if they want to. There are very few organizations that can withstand a concerted attack of any sort. I know many people who are capable of getting into just about any company if they put their minds to it. As a defender you have to protect everything, they just have to find one hole.

Still, it happened and when it happens you are responsible as the (once) leader of your company and ultimately it’s security program. Coming from Google you are clearly highly technical so you can’t feign ignorance.

Being at fault and taking that blame isn’t a terrible thing – you are one of thousands, if not millions, of organizations that have been compromised. People get that security is hard. Still, no matter what, if you are the leader, the buck stops at you, not the Russians.

PS: “and prosecutors”? Huh?

Equifax

Being in the industry, I understand how difficult it is to secure an organization, so I have some sympathy for Equifax. As an ex-NSA colleague noted (paraphrasing), “A defender has to protect everything, an attacker only has to find one hole.” That said, their business is PII, so there is a higher standard there.

In the end my concern is less that the hack happened, than the difficulty in navigating their site and ultimately receiving the credit protection. First of all, the initial page they are telling “customers” isn’t intuitive:

https://www.equifaxsecurity2017.com

It is mostly PR material. You ultimately need to go to the “POTENTIAL IMPACT” button on the bottom:

https://www.equifaxsecurity2017.com/potential-impact

Then when you do sign up, they tell you you’ll have to wait for roughly a week then sign up at a different URL. You had better write down the URL because they say, “you will not receive additional reminders”. The URL, if you made the mistake of not writing it down is:

https://faq.trustedidpremier.com

Then “click through the link to continue through the enrollment process”. What link that is, god knows.

In fact if you click the above “faq.trustedidpremier.com” today, it goes back to, well, “www.equifaxsecurity2017.com”, which I assume then you are supposed to click the “ENROLL” button on the bottom???:

https://www.equifaxsecurity2017.com/enroll

Just mildly confusing.

VMware virtual or not?

Depending on naming, in a VMware environment you may not actually be sure if a Linux system is VMware or not. Here’s a quick command to find out:

which will generally output “VMware, Inc.” if VMware.

On older systems you may find “-s” doesn’t work, in which case just pipe “dmidecode” through grep looking for “VMware”.

UPDATE:

If “dmidecode” oddly isn’t available, you can also run:

If you see “VMware” in the output, it’s a safe bet that it’s a VMware virtual.

Installing Plixer’s “Scrutinizer” NPMD

Plixer makes a good “Network Performance Monitoring and Diagnostics” (NPMD) application called “Scrutinizer“. NPMD, as Gartner calls it, mostly omeans, collecting, aggregating, and reporting on Netflow data.

Plixer provides a VMware OVF for installation of a virtual appliance. I, however, ran into a few issues with the installation:

  • I couldn’t get the install to work OVF through vCenter successfully, or at least vCenter 6.5. It would install, but when I booted it would come up to a PXE boot, rather than CentOS which the appliance runs on. The answer was to install it through the Windows vSphere ESXi client or through the web vSphere ESXi client.
  • Setting up SSL (HTTPS) during the initial install prompts wouldn’t work. Everything seemed fine, but on final boot of the Scrutinizer appliance, the HTTP/HTTPS wouldn’t come up at all. It turned out it hadn’t actually generated the certificates and files were missing. The answer is to select “no” to SSL in the initial dialog, then when fully up, log in using the “plixer” login and use the “set ssl on” option after the fact. SSL then works correctly afterwards.
  • By default it will bind to IPv6 ports and not to IPv4 ports (!) to listen for Netflow data. The solution is to log into the Scrutinizer server/guest as root and disable IPv6 per this document. Specifically, I recommend the “/etc/sysctl.conf” change as it is relatively simple to execute.
  • When logged in as “root”, doing a “yum update” is useful, though I would do the following bullet after.
  • When logged in as “plixer”, it’s useful to run the “set tuning” as well as “update packages”, though oddly it seems to run back one of the kernel updates from the last bullet.

Now I just need to figure why I’m still not seeing the packets from the ASA…

I was but the learner, now I am the “Radar Technician”…

Matt Radar Technician

Ubuntu package commands

Because Ubuntu has a mix of utilities to manage packages I constantly seem to be forgetting the options I need when I go to do basic package management. Mostly for my sake are the ones I use most regularly:

List installed packages:

List names of available packages (including those not installed):

Tell what package owns what file:

List files in a given package:

Force a package reinstall:

Show general package information:

Show package dependency information:

Good basic email advice

Professor Alan Woodward from the Department of Computer Science at the University of Surrey via The Register:

“Educate users not to open files that they are not expecting. Practice your ABCs – Assume nothing. Believe no one, and Check everything should be drummed into users – personally I preach ABCD – if in any doubt Delete.”

Incidentally internal simulated phishing is extremely effective in my experience.

Fix Apple Bonjour with Cisco autonomous APs

I purchased some used Cisco C1140 autonomous access points for my home network (autonomous meaning not lightweight or requiring a WLC). While everything seemed to be fine at first, later we noticed that printouts to our Canon laser printer were no longer working from our Macs. After some research I realized that the Macs were failing to locate the printer due to Apple Bonjour protocol issues. Google searches led to partial solutions, but most required a downgrade of the AP IOS – a no, no as a security professional.

I kept looking and it turns out my savior was actually a Chromecast user with the same issue. Two configuration changes on the APs to disable IGMP snooping had to be executed, not one:

All the prior advice was just to disable the former, which didn’t work (at least without an AP downgrade!). Adding the second line did the full trick.

You may need to disconnect and reconnect to the wireless for full effect. Since multicast IGMP has other uses, I can’t guarantee the impact in a larger environment.

UPDATE:

Well, this may or may not work. In the end it seemed not to for me, but it’s still worth a try in your network.

Dell PERC 5i/R or H200 VMware Performance Fix

I had an old Dell PERC 5i/R RAID card laying around and wanted to use it for a home lab ESXi box (note: also works on Dell PERC H200). The card isn’t amazingly high performance, but it it’s good enough for simple RAID. Well, that is, it’s good enough performance if you change the settings. By default “write caching” is disabled – that unfortunately includes even “write caching” on the drives themselves (5i/R doesn’t have cache so it’s always “Write Through”, the H200 has cache, but is disabled by default). Therefore by default write performance is downright painful.

Fortunately it’s not too difficult to fix if you can pull together the right tools. I was lucky enough to find a post by “tonyd88” on this Dell support forum which explains the process. Below, I attempt to summarize the steps for posterity.

WARNING: If you enable write caching on the 5i/R or H200, because of the lack of battery backup (BBU) there is a risk that if you lose power mid-write, you will corrupt your disk, OS, etc. Not only use at your own risk, but ideally at least have a UPS on your system.

Steps:

  • Locate a copy of “LSIUtil.exe”. The Dell RAID 5i/R was made by LSI. LSI was sold to Avago Technologies and a copy of it appears to be here, but you may need to look around in Avago’s legacy driver downloads. The latest version I have found is 1.62.
  • You’ll need to create a DOS boot disk or thumb drive with the LSIUtil.exe on it. Unfortunately explaining how to do that is a bit beyond the scope of this article. Google is your friend.
  • It is likely you will need “DOS4GW.EXE” also on the boot disk. You’ll have to find a reputable download or buy it here. This may be a potential alternative.
  • Install the Dell 5i/R RAID card in the system and boot to your newly created DOS boot disk/drive.
  • Run the LSIUtil.exe binary.
  • Select your controller. Hopefully there will only be one, but otherwise you’ll have to figure it out. Choose the number that matches and hit <enter>.
  • Select option “21”, “RAID actions” (type 21 and <enter>).
  • Select option “32”, “Change Volume Setting” (type 32 and <enter>). This submenu’s first item is the critical change:
    • On the first prompt for “write caching” type “yes” and <enter>.
    • For “Offline on SMART Data” just hit <enter> for default or change to whatever you want.
    • For “Auto configuration” just hit <enter> for default or change to whatever you want.
    • For “Priority resynch” just hit <enter> for default or change to whatever you want.
    • For “Hot Spare Pools” just hit <enter> for default or change to whatever you want.
  • At the next prompt type 0 <enter> to quit. Just continue to hit <enter> until it drops all the way out to DOS.

You’re done. Pull the boot drive and reboot to whatever OS you’re going to use on it. VMware happily uses the cards.

Fix VMware Web Client Integration Plugin for Chrome on Mac OS X El Capitan (10.11)

The latest vCenter Server 6.0 VMware Web Client Integration Plugin does not work on OS X El Capitan. The installer finishes, but silently fails due to missing libraries, libraries that probably existed in earlier OS X versions.

Because the libraries don’t exist, necessary certificates don’t get generated, and even re-running the installer from the application directory won’t solve it (including with the below hack). What you need to do is ensure the libraries will be there when the installer gets to the “Running package scripts…” section on initial install.

There are a number of possible solutions, but the below seems the cleanest and doesn’t require multiple installs.

Before installing the application, do the following:

Then run the full installer.

This will create a hack to allow the packaged libraries to be used when the package scripts get run. If it’s working correctly the “Running package scripts…” will take many minutes to run as it executes “openssl” to generate the following:

/Applications/VMware Client Integration Plug-in.app/Contents/Library/data/ssl/dh512.pem

If it instead installs very quickly, you can be fairly certain it didn’t install correctly and probably VMware has changed something yet again. If it works, you can both upload files and deploy OVF files.

Hopefully VMware will create a permanent fix. More on why this plugin is required can be found here. How to install/upgrade the plugin itself can be found here.

UPDATE:

Jonathon McTaggart (thank you Jonathon!) gave the following update for the latest plugin:

UPDATE 2:

It appears VMware has essentially documented the same fix here, rather than fixing the installer:

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2144550

The problem is, they are also suggesting you disable a fundamental OS protection temporarily as well. That is a major PIA and sadly doesn’t seem to work on macOS Sierra. I can use OVFs, but I can’t do file uploads. Apparently there is a integrated ESXi HTTP client that some are working on here (via here) that seems to offer some options. This has been a problem for over a year now…