GEEKNOTE: Translation: They really MAY be out to get you.
As those of you who read the comments realized, last week's Geeknote was slightly off the mark. (See: http://newportrichey.patch.com/blog_posts/geeknote-another-saturday-bites-the-dust )
Based on the vintage of one of my servers, I had initially jumped to the conclusion that I needed to replace an aging drive system on that server. While that may have been true, the problem persisted until I powered down both of the effected servers and simultaneously power cycled the Brighthouse cable modem. Things got better. This pointed to the possibility that our new cable modem had gone bad.
Things then ran fine until early Monday morning when the symptoms returned. The folks at Brighthouse assured me that they saw nothing unusual on our circuit. They were certain the problem was on our end. They were mistaken.
Because things invariably seemed to go south when we connected to the Internet, the next step in testing was to set up a filter that blocked and logged everything. This proved to be the key to the puzzle. Both servers were being buried under a near constant barrage of DNS requests, hundreds a second. The servers were buckling under the load. This is commonly referred to as a "DOS" or Denial of Service" attack.
The requests were coming from two address blocks belonging to a California host that provides "anonymous" web services. People can use their servers to hide their true location and identities while using the Internet.
Why just two of my servers? It turns out that we filter spam using an internally managed DNS Blacklist. For whatever reason, when I had initially set up the blacklist several years ago, I had chosen these two servers to host DNS for the blacklist. Working on a hunch, I reconfigured the blacklist so that it ONLY had a single DNS server and I pointed that at a test machine. Sure enough, the attacks on our two production servers trailed off and the test server came under attack.
What appears to have happened is that somebody got upset because we blocked their spam and decided to attack us to get even.
The permanent solution to the DOS attack was fairly simple: I created two filters to block everything from the two address blocks that were used in the attack and let everything else through. I created a "dummy" DNS Blacklist using the original blacklist name on the test machine and then created a well hidden DNS Blacklist for my servers to continue to use in filtering spam. The bad guys can hammer at the test machine all day long and we don't really care.
The filters look like this:
FILTER PROVIDER DENY IN * xxx.194.232.0/21 *
FILTER PROVIDER DENY IN * xx.215.92.0/22 *
I x'd out the first octet to mask the identity of the California bozos that let their servers be used to attack others. The filters are saying "If anything comes in from these two blocks addressed to any of our addresses, block it."
I had initially spun up the test server so that I could be ready to assist a company in New York after Sandy took down their Internet connection. They got their Internet back about the same time as I got the server running, so it was available for other use.
About 11 hours after the test server went live, the Chinese found it and started a dictionary attack. Dictionary attacks have a structure something like this:
[I] Nov 17 11:41:00 [22.214.171.124:53201]SMTP Server: AUTH failed, username administrator, password 654321
[I] Nov 17 11:41:00 [126.96.36.199:53201]SMTP Server: AUTH failed, username administrator, password 54321
[I] Nov 17 11:41:01 [188.8.131.52:53201]SMTP Server: AUTH failed, username administrator, password 88888888
[I] Nov 17 11:41:01 [184.108.40.206:53201]SMTP Server: AUTH failed, username administrator, password admin
[I] Nov 17 11:41:03 [220.127.116.11:53201]SMTP Server: AUTH failed, username administrator, password good
[I] Nov 17 11:41:03 [18.104.22.168:53201]SMTP Server: AUTH failed, username administrator, password Boy
The attack shown above originated in Guangzhou, China....
They try common user names and then go through a list of common passwords, looking to see if they can log in.
We also noted the Chinese attempting a port scan, where they attempt to connect sequentially on every one of the 65000+ possible ports that a computer might use.
Over the years, we've seen these type of probes on a regular basis, often coming from countries who have political reasons for wanting to compromise computers in the US. I see dictionary attacks and port scans so often that they don't excite me much anymore.
The solution for both businesses and individuals to protect against dictionary attacks and port scans is to use a firewall / router and to change the default login /password combinations for the router and anything that can be touched from the Internet to something complex. (eg. Don't use "password" for your password.)
Our Internet servers have some very sophisticated logging options, but most business class routers can also log these attacks. Most residential class routers do not have the logging functions of the more expensive routers.
It is not as important to log all this stuff as it is to know that the bad guys are indeed out there and, probably sooner rather than later, they will be rattling the electronic door knob on your home or business to see if you left it unlocked. Do yourself a favor and make sure you've locked the door.
Feel free to drop me a note or give me a call if you have any questions about your computer.
Rob Marlowe, Senior Geek, Gulfcoast Networking, Inc.
(Rob also serves as deputy mayor of the City of New Port Richey. Opinions expressed here are his own and do not necessarily represent the position of the city.)