How did we Recreate the complete Blog without a Damn Backup
17th September was a tragic day for Me. The whole site was down and we lost everything including articles, related images , themes and design. The VPS node was corrupt and the the hard disk crashed on the server. It was 2 Am in morning and I was in state of shock. Me and Manav had no Idea what we gonna do next. If you had been thinking that why the host never took a backup, well they did but the program used for it never worked.
Call it dramatic but all of a sudden the newsgator feed reader which I use to read feeds started popping up the feed updates. It triggered my mind and I rushed to check my blog feed there. 80 Posts I had there which made me little happy and then All of a sudden I got the idea of Google Reader. On Immediate check I found some posts there and then I kept scrolling and Guess what I had all 1080 posts there. Wow…. that was one major luck.
Google Reader containing Posts and their dat
Google Feed reader had stored all my posts from the day I started. That was so surprising. I could have never expected. Now we were sure that the articles were there. The next was to plan out and which was :-
Our Strategy:
- To post back all the post with exact url.
- To bring most of the comments ( If possible )
- Find the changes I would have made with time as Google Reader doesnt keep the changed things.
- Find when the articles was written so we can maintain the chronology.
Hurdles :
- To maintain the traffic.
- Find all the popular post.
- Changes made with time in the posts.
- Recovering lost images.
Tools we used to get the posts back:
- Google Reader : For Approximate Dates, Post Content
- Google Cache : To get the change done on the articles.
- Google Analytics and Wordpress Stats to find the popular posts.
Step: 1 Pulling out Popular and Posts which changed with time
Then I started pulling all the popular posts one by one. The best was to use Google Search with the exact title I had for the titles as seen in the Google Reader. This gets my posts in the first result. Next thing I did was to hit the cache link of the article to get the latest updates if any.
Using Google Cache to get latest udpates
We had to be quick on this as Google Cache changes every week. I found which ever posts I knew I had changed and started posting updates. These steps I did for 250 + posts of mine which started bring good amount of hits slowly.
The hurdle of getting the exact url was getting over we were able to pull the correct url both from feedburner or the Google Cache. We were not able to find the exact date but we used the dates which we saw in Google Reader as approximate dates.
Step 2: Checking for Not Found Pages
its really impossible to find all the post which would gets hit specially the smaller ones . So i decided to spend 2-3 hrs on tracking the not found pages.
- I used Alexa King 404 Plugin which gives out email or feed for every not found thing on a wordpress blog.
- I use Woopra for live tracking. This came out to be very handy as I can find in real time which articles were giving me 404. I was immedatiely recording everthing down so I can pull it up later on.
Woopra for Live tracking for 404
This gave me around 200 articles which use to get hits. Remember I was watching the traffic when its high traffic time for.
Step 3 : Pulling Articles by Month:
By this time I had already more then 400 articles up and running. Ths traffic was stil on down side now. Since Google did not find articles i guess the traffic was going slowly down.
Now the 404 errors were also reducing and moreover the cahce had already begun to change. So we had no other choice left but start pulling from the Google reader and putting it one by one. Trust me it was so painful i used to hate my self for not taking the backup.
Step 4 : Image Recovery ( This was Major Luck )
Well there is no means we could have recovered images but when I started blogging I did not used to have my images here. I had many of my images hosted at photo bucket. This was little relievf. I decided to start re-making the images article by article until tomorrow.
I was in office and it alarmed me that I had my backups of images to an extent in my old laptop. I got back home and found it. Wow I am so happy I did not reformat it after i got my new laptop.
So now most of the problems got solved. I had the images, article and dates. Problem was 80% solved.
Image Recovery
Stage 5: Check , Re check everything .
This is what I am doing now as I write this post. Since I copied everything I had to keep a keen check if there was something missing, loss of formatting, and links. In this phase I am putting back the images also. This should be over by tonight or max in the morning.
We have recovered by 85% and should be ok in a week or so as the traffic starts flowing back.So this is how Technospot.Net is back now. We have a new theme which we are still working on to tweak it and add utilities every where.
As a summary :
- We found all our posts in the Google Reader. Since I used to give full feeds we had everything, Thanks to Darren Rowse whose one of the post inspired me to switch from partial feed to complete feed
- Google Cache helped us in getting the modified articles.
- Woopra and Alexa Kings 404 Plugin helped me to find quick not found pages.
- Hosting Images on other server is a good idea but depends.
What did I learn from this :
The major mistake I made was to rely on the host backup. I had never any issue with my hosting except this.
- Even though I pay 10$ for the back up , they have given me the direct access to those backups
- I should always take backup of my Images, Database and Theme on my side alone which I have started doing it again and will continue to do it.
- One of the major thing which I gained was that I got a chance to re visit all my posts and fixed a lot of things. Even the images which I am adding have their alt tags availble. It would be interesting to see if this affects.
The 2 weeks of painful time is coming to end now. I had been through a lot but it was a learning experience. I wonder if this would have happened next year and I would have never check my backup. I though of sharing this with you all in case if somebody looses his data like I did, this is one good way.
At the end I would also like to thanks some of the people specially people on Twitter who had been always asking me how its going. My Wife who had seen me just sitting on the computer all the time since last week and had no time talking to her either. Thanks for Sampat who helped in finding things at places and Of course Manav who had been so morally supportive to me.
Thats the story peeps and We are back with a BANG!!!!!!!!!!!!
Posted on 27th September 2008 by Ashish Mohta , A Professional Tech blogger, Editor and Writer who talks about solving day to day problems of people who use computer. He also writes on How to use the applications like Office, PC tips, Online tools,Browsers and more. All posts by Ashish Mohta | Connect with me @ Twitter | Linkedin | Facebook | Stumble
That would have been a really challenging work! But still, good that everything is back now.
Backing up is really essential, and i seem to be not taking much care about that.
2 Weeks working on just the recreation process is just hard.
Pretty tough to do all this tasks in few days! Kudos to you man.. Thanks for alerting us to make frequent backups..
Excellent narration.
Ashish sir, It was very tough to do this without a backup but you managed it very well i must say. I hope you will take backups from now on.
one thing you never need to do is updating from the cache…you know what? the feedbot would have fetched the updates as well…
As an example you make a post check the feed, change the content(you do some updates) and then check the feed again..it would have changed…
you could have confirmed me via twitter that you were doing restores from google reader and i would have helped you there…
anyway you are done now and congrats for getting back on track…
Hey.. Glad you got everything back dude
Now only the party is due 
Rajesh, It would have updated only when there was a feed availble. By the time I discovered my site was gone and there was no feed. If I would have fetched again I would have lost everything right there.
Iam still confused how were you able to get back all the comments if you lost all the data, ie commentors IP, other comment stats etc
Hard work paid it all and Technospot is back with a bang, congrats both of you to bring it back.
BTW whats your host doing now for this damage? I think most of the VPS have RAID protection for hard drives, then how come they dont have it?
I’m glad that you were able to pull it off in the end. It’s understandable what kind of effort it must have required and certainly a lesson learned for you and for everyone who was following this – backup regularly at your end.
Thank goodness for Google its Cache and its other web services. I think it would next to impossible to recover without all of it.
Btw, will your host do something about their failed backup systems? I mean like a refund for that or something?
@Jhay: Yes my host refunded around 160$ which includes mone for downtime of this month and money which I had been giving them for backup.
@Abhijeet : Yes people should learn from this. Its very necessary to take backup.
@Nirmal : I will ask my host about it. Not sure about it.
@Amit : We did it manually like the post but very few.
@Naryanan: he he we will plan something
@rajesh,rockstar, chetan and amdhur. Thanks guys
Google reader had all the content right..Google reader stores all your feed data in their database.If any updates had happened, i guess google reader would also have updated the post/s in their database.
Comments is something amazing.How did you recover the comments?
Only when it gets pinged and when the source feed was alive. My source feed was dead. And it never updated even before when it was alive.
Excellent article buddy, and I can understand and see the pain you have gone through to get your blog back on its legs. But that said kudos to you on such a huge feat.
Also please remember to backup your blog database regularly now
.
Congrats man. This is one of the biggest post recoveries I have seen till now. Its kinda cool though it was sad and tiring. And as Narayanan said we want the party.
Great post. When in need, you thought quickly and restored your blog. Thanks for sharing it with all of us. As a blogger, I can understand the pain of losing all that you have put together over the years.
This also highlights the downside to this. Internet call have a long memory of things you say, so, make sure you say the right things!
wow! good work, y’all!
Good to see you got all the posts back…
Nice Post…
This is Really Great Idea…
- Anurag R.C.
I have been through this when my hosting deleted all my data in mid 2007, at that time i also don’t have the backup.
The only thing i was left with that time was some days html image of my blog on my local system, which helped me to reproduce some post, even then i lost most of my most of my data.
I can understand the pain, and really commendable job on your part for making everything up.
Great job ashish
, feel good to see technospot.net coming back with a bang
None of your readers will become victim for the same pain which you faced… Anyway, a really good way to recover and you did everything in time… Its now better that you are back completely….
Ashish, hats off to you for taking so much pain and recovering all of your posts.the grit and determination with which you pulled this off is amazing and inspiring. I remember reading your tweets early morning and late nights on you starting your work on recovering the data. And each time you posted a tweet on x articles updated, I would get all very happy. Also remember the way you explained how you were recovering it during the tweet up. Awesome stuff dude. Old Monk party soon!!
PS: I recommed WP-DB backup plugin: it creates your database backup on demand, as well as emails you the backup at specific times:every hour, every day, everyweek etc. Must have plugin IMO.
Cheers mate!
That was a great recovery! Congrats. Its really nice to see how much effort you put into this!
And very thanx for sharing what you did, will come in handy for sure!
You seriously have a lot of patience and kudoes to you for not giving up!
Good to know. Even I learnt it from hard way..
WOW… this is some hard work. Keep the damn good work up and good to see you back!
Hats of to You guys, if some thing like this happen to my blog i am sure i will quit blogging .
Amount of work required to recover 1000+ posts manually is unimaginable but you guys done it.
Hope you guys are not suffering from blog crash nightmare any more.
Can any one have wordpress plugin which can automatically backup wordpress db and upload it to multiple private FTP server (Keith !!!) ??
@Keith @Karthiksn @Jawahar Thanks guys. Been hard time but trust me it was really worth it.
Thanks Joel. Dont see you a lot these times. How are you doing ?
@Sandip : haha the nightmare is over. Its all good though we will keep on ifixing things here and there. There are still back office work we are doing
Thanks Thejesh!!!!
@Pallab: Hehe when things are so close to you , you can do anything for it.
Thanks a lot to everybody. Really aoppriciate you people read us and gave so much of encouragement.
hey, great post, this was pretty interesting
Leave your response!
Subscription
Follow Us
Monthly Archives
Topics
What others are Saying ?
An impressive and easy to understand website which brings you productive applications, online tools, Windows and helps you better understand how you can use your computer, mobile and internet in a right way.