HDFS Setup
Configuration:
1 NameNode
3 DataNodes
1 SecondaryNameNode
Requirements: Assuming your team has three students: Tom (999900012), Jerry (999900034), Mike (999900056)
Configure the hostname of each DataNode:
Hostname of DataNode 1: Tom
Hostname of DataNode 2: Jerry
Hostname of DataNode 3: Mike
Set the last two digits of the IP address of each DataNode:
IP address of DataNode 1: [login to view URL]
IP address of DataNode 2: [login to view URL]
IP address of DataNode 3: [login to view URL]
Submission Requirements:
Submit the following screenshots:
Use commands to create three directories on HDFS, named after the first name of each team member.
Use commands to upload the Hadoop package to HDFS.
Use commands to show the IP addresses of all DataNodes.
Provide detailed information (ls -l) of the blocks on each DataNode.
Provide detailed information (ls -l) of the fsimage file and edit log file.
Include screenshots of the Overview module, Startup Process module, DataNodes module, and Browse Directory module on the Web UI of HDFS.
MapReduce Temperature Analysis
You are given a collection of text documents containing temperature data. Your task is to implement a MapReduce program to find the maximum and minimum temperatures for each year.
Data Format:
Year: Second item in each line
Minimum temperature: Fourth item in each line
Maximum temperature: Fifth item in each line
Submission Requirements:
Submit the source code of your MapReduce program along with instructions for running it. Also, include a short document explaining your design choices and how you tested your solution.
Mapreduce [login to view URL]
I understand that you are looking for someone to help you configure HDFS on your three DataNodes. Specifically, you need to set the hostname of each DataNode, last two digits of its IP address, and submittal requirements for three directories on HDFS to be created, named after the first name of each team member.
I am confident that I am the best fit for this job because my extensive experience in Linux and other related technologies makes me an ideal candidate for this project. As part of my work at Bedant Consultancy, we have developed a variety of web and Android applications that can be used to demonstrate my skillset.