Create Virtual Machine Template
Build the initial VM by following the my blob post: Installing Apache Hadoop on a Single Node .
Follow the steps in the below Windows Azure article:
How to Capture a Linux Virtual Machine to Use as a Template
Create Five Node Cluster on Windows Azure
Virtual Machine 1 : Name Node and Job Tracker
Virtual Machine 2 : Secondary Name Node
Virtual Machine 3 : Data Node 1
Virtual Machine 4 : Data Node 2
Virtual Machine 5 : Data Node 3
Enable Localhost SSH
ssh-keygen -f "/home/mahesh/.ssh/known_hosts" -R localhost
Change the yellow highlighted text with your user name.
Copy SSH Key to each machine from the Name Node
ssh-copy-id -i $HOME/.ssh/id_rsa.pub mahesh@hd-name2
ssh-copy-id -i $HOME/.ssh/id_rsa.pub mahesh@hd-data1
ssh-copy-id -i $HOME/.ssh/id_rsa.pub mahesh@hd-data2
ssh-copy-id -i $HOME/.ssh/id_rsa.pub mahesh@hd-data3
If you get the error "/usr/bin/ssh-copy-id: ERROR: ssh: Could not resolve hostname hd-name2: Name or service not known". Change host name to the IP.
And if you happened to use the IP, change host namer of "/usr/local/hadoop/conf/core-site.xml" and "/usr/local/hadoop/conf/mapred-site.xml" to IP of the Name Node.
Update "Masters" configuration file with Secondary Name node IP
sudo vi /usr/local//hadoop/conf/masters
Update "Slaves" configuration file with Data node IPs (Each IP should be entered as a new line)
sudo vi /usr/local//hadoop/conf/slaves
Format the name node
hadoop namenode -format
Start Name, Secondary Name Node and Data Nodes
start-dfs.sh
start-mapred.sh
No comments:
Post a Comment