MapReduce algorithm is used bye Google’s web servers for to process large amount of data and index it so it can be referred in web searches. GFS technology alone is not enough. Its a file system on which MapReduce algorithm works to give what Google wants to achieve, reliable data and faster searches.MapReduce Programming model is Google tool developed in c++.
What is MapReduce ?
MapReduce is an algorithm which is used on GFS to make searches among clusters of machines which Google use to fetch data for web searches.Its main aim is processing and generating large data sets
How MapReduce Algorithm Works ?
Map and Reduce, thats the main logic. It separates operation being done on large files from applications. They application only need to call correctly rest of things are separated and it gets only result of it. Thus its mapped and reduced processing on applications.
The primary job of MapReduce is to select the node on cluster, where the jobs need to run and how balancing is to be done.Thus applications being separated they need not to think how big is the system, because there is already sub system which is taking care of that.
They just need to ask for data and maintain other factors like load balancing and tolerance in case of failure.The beauty of MapReduce system is it stores intermediate result as a separate list. Thus while processing complete result this intermediate result is used which reduces the complexity and saves time.
Conclusion:
GFS and MapReduce together had give a big success of Google in the world of internet as the biggest search engines. GFS being a base , thousands of MapReduce jobs is carried out. They are improving them everyday to keep up in the market and We all can hope they keep on doing the good job.You can read more about GFS and MapReduce on googl’s lab
