5.2 Map Reduce Programming In Java Case Study
Chapter 5
IMPLEMENTATION
Implementation is that the stage of the project wherever the theoretical design is changed into a working system. The implementation stage needs Careful designing, Investigation of system and constraints, design of ways to realize the transformation, analysis of the transformation technique, Correct decisions relating to selection of the platform and applicable choice of the language for application development.
5.1 General Implementation Discussions
Implementation part should perfectly map the design document in a suitable programming language so as to realize the required final and correct product.
5.1.1 Java
In this project, for implementation purpose Java is chosen because the programming language. We have used …show more content…
This makes Java programs easier to write down down and fewer vulnerable to memory errors.
Swing support: Swing was developed to supply a lot of sophisticated set of user interface components than the sooner Abstract Window Toolkit. Swing provides a native look and feel.
5.2 Map Reduce programming in java
Map reduce may be a special style of programming framework that is usually used for data processing of big quantity of data. The framework is employed to process large amount of data in distributed nodes. The scalability and extensibility is that the main advantage of MapReduce programming model. and therefore the alternative individual is that the process is on the location wherever data resides and thus it is fast and economical.
5.2.1Implement the MapReduce classes
MapReduce framework may be a light weight framework used for process large quantity of knowledge therefore we need to know that the model is efficient given that we have multiple artifact servers and therefore the process is functioning during a distributed manner on all the servers. The framework is scalable, fault tolerant extendable long. The subsequent functions …show more content…
Therefore when mapping is complete, the reduce () function operates on the intermediate data set by retrieving them from disk/memory or the other place. The ultimate result from reduce () function is consolidating the information from all processes. When the mapper and before the reducer, the shuffler and combining phases take place. The shuffler phase assures that each key value combine with the same key goes to the same reducer, the combining part converts all the key value pairs of the same key to the grouping form key,list(values).
5. 2 Project Implementation
5.2.1 The algorithm of dynamic slot allocation under PI-DHSA
1: When a heartbeat is received from a compute node
2: Compute demand for Map slots and Reduce slots of current MapReduce workload.
3: Determine dynamically the need to borrow map or reduce slots for map or reduce tasks based on their demands. Check for following cases
4: Case 1: If both map slots and reduce slots are sufficient
5: Then No borrow operation is needed.
6: End if
7: Case 2: If both map slots and reduce slots are not enough.
8: Then No borrow operation is
IMPLEMENTATION
Implementation is that the stage of the project wherever the theoretical design is changed into a working system. The implementation stage needs Careful designing, Investigation of system and constraints, design of ways to realize the transformation, analysis of the transformation technique, Correct decisions relating to selection of the platform and applicable choice of the language for application development.
5.1 General Implementation Discussions
Implementation part should perfectly map the design document in a suitable programming language so as to realize the required final and correct product.
5.1.1 Java
In this project, for implementation purpose Java is chosen because the programming language. We have used …show more content…
This makes Java programs easier to write down down and fewer vulnerable to memory errors.
Swing support: Swing was developed to supply a lot of sophisticated set of user interface components than the sooner Abstract Window Toolkit. Swing provides a native look and feel.
5.2 Map Reduce programming in java
Map reduce may be a special style of programming framework that is usually used for data processing of big quantity of data. The framework is employed to process large amount of data in distributed nodes. The scalability and extensibility is that the main advantage of MapReduce programming model. and therefore the alternative individual is that the process is on the location wherever data resides and thus it is fast and economical.
5.2.1Implement the MapReduce classes
MapReduce framework may be a light weight framework used for process large quantity of knowledge therefore we need to know that the model is efficient given that we have multiple artifact servers and therefore the process is functioning during a distributed manner on all the servers. The framework is scalable, fault tolerant extendable long. The subsequent functions …show more content…
Therefore when mapping is complete, the reduce () function operates on the intermediate data set by retrieving them from disk/memory or the other place. The ultimate result from reduce () function is consolidating the information from all processes. When the mapper and before the reducer, the shuffler and combining phases take place. The shuffler phase assures that each key value combine with the same key goes to the same reducer, the combining part converts all the key value pairs of the same key to the grouping form key,list(values).
5. 2 Project Implementation
5.2.1 The algorithm of dynamic slot allocation under PI-DHSA
1: When a heartbeat is received from a compute node
2: Compute demand for Map slots and Reduce slots of current MapReduce workload.
3: Determine dynamically the need to borrow map or reduce slots for map or reduce tasks based on their demands. Check for following cases
4: Case 1: If both map slots and reduce slots are sufficient
5: Then No borrow operation is needed.
6: End if
7: Case 2: If both map slots and reduce slots are not enough.
8: Then No borrow operation is