GFS ot Google File System is one the most innovative feature of Google which forms the backbone of all the searches and storage of Google Technology. GFS is a scalable distributed file system which is based on Linux, built with the just one aim, to handle BIG files. Scalable means “the ability of a product or network to accommodate growth” where as Distributed means ” Thing which can be spread around, here its aim is to decentralize to handle multiple request for data.
What is GFS?
GFS as I described above is storage system which allows main functionality of storage, processing search and retrieval and in case of failure should be able to self correct it in order to retrieve huge amount of data which Google’s paper say is in terms of Multi-GB.
How GFS Work?
According to Google Lab’s Jeffery Dean, The idea behind GFS is to store data reliably even in the presence of unreliable machines. The GFS system works on master slave module. There is one machine which acts as server and several other which are slaves or node.
The master is responsible to keep track of “which data is stored on which machine” , called as meta data ( data about data). The GFS is said to maintain 3 copies of any data or file including executable. The meta data resides on the main memory of Master , i.e on ram, thus allowing faster access.GFS was designed to store huge amount of data, till current date,
The largest Google Cluster ( Cluster is group of computers together as network) stores hundreds of terrabytes of memory across thousands of disks.
How Data Failure and tolerance works in GFS ?
In GFS, The master server handles all the requests , which in turn transfer the request to exact location of data to one or more nodes or slaves. In case the request takes more time than alloted the system switches to a backup-copy ( the reason of maintaining 3 copies!!).
In case of one of the slaves or nodes fail, its Master which is responsible to maintain the count of nodes either by reallocating ro some other machine or creating a duplicate copy. Although there is only single master “active” at any given point, the state of master ( log of what the master has been doing) is present on other machines too.
Thus in case of failure of master , another machine which knows what the failed master was doing takes the position and keeps the work on move.This is an overview of how GFS works. For more technical details look at GFS PapersNext post I will talk about MapReduce Tech used by Google which forms another heart of Google’s success.
