NKCSS Logo

LeaseWeb Server #1 Crash

Over two years ago, I leased my first server at LeaseWeb to host Nickkusters.com and my console shop BuyInBulk.nl

Due to my bussy job, I havn't had much time to write articles for NickKusters.com, but the SpeedTouch (Thompson) WPA2 Key Lookup tool has been so popular, the machine was always bussy as hell.

The old machine just had a single HDD, 1 CPU core & 1GB ram, running Windows Server 2003 Standard. It was struggling to server 2000+ people a day, who were looking to do SSID lookups for their Thompson modems.

Midway thru the 22nd of April 2011, the machine seems to have died all to geather. Only +/- 50 requests a day were able to be served, and that's about it.

It took me 12 days not notice this (bad, very bad, mea culpa), but as I've said, my job keeps me pretty busy.

I decided to give the machine a remote power cycle to see if a reboot would help. After 25 minutes, the machine still didn't come back up, so I dropped an e-mail to leaseweb, letting them know their server probably died, asking for help.

The e-mail was sent around midnight, 10 minutes after the standard auto-reply e-mail, a support agent let me know that he requested an engineer to look at the server.

At 7:31 AM the next morning I recieved the following e-mail:

Dear mr. Kusters,

I would like to inform you with the following.

Unfortunately the server is unable to boot. Halfway through loading Windows, it spontaneously reboots. I can not find the cause of the error as before anything can be read on the blue screen, the server reboots.

Currently there is no way to start your installation of Windows and the server needs to be reinstalled to be able to make it functional again.

Therefore I've sent you a rescue environment request per e-mail. Within this environment you can create back-ups. After you have created your back-ups we will reinstall the server for you.

Once you are done with the back-up, please let us know and we will issue a reinstallation on the server.

I hope to have informed you sufficiently and await your reply.

Kind regards,

Jurriën Kol
LeaseWeb - Technical Support

After 10 minutes, I received the following e-mail:

Dear Nick Kusters,
Your server has been requested for rebooting in rescue mode. 
To confirm this request please run:
https://secure.leaseweb.com/confirmation/rescueRequestConfirmation/key//answ/true 
To reject this request please run:
https://secure.leaseweb.com/confirmation/rescueRequestConfirmation/key//answ/false 
Your server details: 
•	Server name: 
•	IP address: 85.17.155.78
If you have any questions, please do not hesitate to contact us at support@leaseweb.com.
With kind regards,
Your Leaseweb Team

I clicked the Link, accepting the 'rescue mode', the website let me know I've accepted, but nother else happened, so I've sent them an e-mail asking how this would allow me to make backups.

Dear mr. Kusters,

I would like to inform you with the following. 

In the rescue environment that will be loaded after you have clicked the acceptation link in the e-mail I sent you about it, will load a Linux environment. In this environment you can mount your harddrives and transfer the databases to your back-ups location.

I hope to have informed you sufficiently.

Kind regards,

Jurriën Kol
LeaseWeb - Technical Support

After more time passend & no follow up response happened, I decided to call LeaseWeb around 9:20 AM to see what's what.

The woman I spoke to told me that I had not clicked the accept link, and that they were waiting for me to do so. After assuring her I did, I did it a second time & created a screenshot of the result page which I've e-mailed her, aparently their new user panel still has some bugs in it, so it never registered.

Last, she asked for my phone number, promissing me that she'd look into it & have an engineer call me to sort this out.

At 3:30 PM, I sill hadn't recieved a call; I assumed that this shouldn't take 5 hours, so I called back yet again. Someone from the sales department answered because tech support wasn't answering, noted down my number again and promissed me that someone would call me back. Not long after (it couldn't have been more then 15 minutes), an engineer called me back.

After asking me if I knew anything about Linux, I had to admit that it had been a long time since I've used a Linux distro (the last distro I used was Red Hat 5.3, so you can imagine...), but as always, I'll manage. He let me know he'll set up a Linux Live CD to let me rescue my data. I hoped that there was like a remote X server I could use (Windows Remote Desktop Spoiled me) or something, but he said I'd have to make due with SSH.

After ls-ing around a bit, I quickly found both Harddisk partions in /dev/sda1 and /dev/sda2. I quickly checked if ntfs was supported (cat /proc/filesystems), which it did, so I mounted both disks & started googling for a way to transfer whole directories over SSH. I quickly found SCP, and, from my Mac Mini, it was a piece of cake to transfer both disks recursively to my Mac. I must say that it's handy to have a Mac or a Linux machine to do this, but Cygwin should work as well.

I've set up two simultanious SCP transfers (one for each disk) since the network connection wasn't the overhead, 'threading' improved the overal speed of the task. I was getting between 2.4 and 3.7 MiB/sec on large file transfers. While that's not great, it's not that slow that it becomes a painstaking job.

I'll be sure to update this article with a step-by-step guide on how to do all the above to help others in the same situation.

More Articles

Popular in the last 7 days

Article   Last 7 Days Week Before Change Total
NumberLink Statistics 857 views 732 views +125 views (+17%) 1,985
@LulzSec dumped Sony BMG data via SQL Injection Attack Statistics 109 views 161 views -52 views (-32%) 11,578
LeaseWeb Server #1 Crash Statistics 104 views 106 views -2 views (-2%) 3,823
Sak Noel - Loca People Statistics 71 views 72 views -1 views (-1%) 2,674
Fable: The Lost Chapters 4 CD to DVD Statistics 69 views 81 views -12 views (-15%) 2,826
Detailed Information
Article ID 20
Written By Nick Kusters
Created 05-05-2011 (293 days ago)
Last Edited By Nick Kusters
Last Modified 05-05-2011
Revision 2
Page Views 3,823 (≈13 pageviews a day)
Back to the main News page
Sponsored Links
Want to share your opinion?
Leave a message! Comment
Ugh (Reply) Created on 06-05-2011 at 18:42 by SjaakSpaak
Man, that sucks!
iPhone test (Reply) Created on 06-05-2011 at 21:15 by Nick Kusters
Works fine