It’s been a watershed moment for our startup over the weekend. I got the components for the server from TigerDirect.ca and put it all together. It’s a sexy system, i7 CPU, 8cores, 12 GB DDR3 ram, lots of cooling, RAID5 configuration with terabytes reliable drive space. It literally floats an inch in the air through it own power, and emits a wicked blue glow. This is more powerful than a dozen of the servers that Google started out with. It only set me back around $1,700, and is a performance machine architected to withstand loads of queries.
I’ll get this machine collocated at a managed site later (once we get some revenue coming in), but for now, I have it in my basement, and accessible via dynamic DNS.
I was super-excited to get started and created three accounts for my collaborators and mailed the info out. Within 10 minutes, I noticed using the ‘w’ command that one of my collaborators had logged in. I tried to talk to him via the console ‘talk’ command and my attempt to talk was refused. I figured that it was a configuration issue and fired up Skype to talk to him. The conversation went as below:
[10:25:56 AM] Shahzad Khan says: Hi AL
[10:26:04 AM] Shahzad Khan says: How is the connection to the server?
[10:26:11 AM] Shahzad Khan says: Is it at a workable speed ?
[10:26:49 AM] AL says: Oh, I had not even seen that e. mail.
[10:26:53 AM] AL says: Let me check.
Ok, at this point, I’m perplexed. What’s going on? He’s already logged in!
I double-checked, and another of my collaborators was logged in. I now fired up the ‘who’ command to see where they were coming in from. Well, they were apparently no longer in Ontario, and were coming in from Spain!
In a blinding flash, I realized that 10 minutes after setting up the servers and accounts, we’d already been hacked!
I booted the ‘unwanted guests’ off, and changed the passwords to the AOL style pw, rather than the throwaway ones that I shared with my friends. Paranoai is my new watchword now ! I haven’t been hacked in 14 years, since the core wars that used to take place between the IRC junkies… and that was all good fun among friends. These hackers who tried to hijack my previous server are professionals. They’re either scanning the block of IP’s that my ISP uses for their DSL customers, or the Dynamic DNS server that I employ to keep my server’s name updated for the world. There is a standard going rate for ‘bots’ and ‘smurfs’ that are harvested this way, and my poor server was about to be kidnapped and sold into slavery.
My mistake was to believe in ‘security by obscurity’ and not worry about the strength of the credential. It’s only my instincts on ‘normal’ server operation (and noticing something amiss) that saved us this time. Next time could be messier. I noticed that only the two accounts with usernames that are common were compromised. The other accounts were not broken into, as the username was different. This leads me to suspect that this incident was the outcome of a plain vanilla dictionary attack on my citadel. Oh the shame of it all!
How’s that for baptism by fire? 10 minutes! Real life can be so brutal.