Micheal Phelps vs Cavic from a under water Camera

Peer to Peer Lookup using Consistent Hashing and Virtual Nodes

It took a long time for me to understand the relationship between Consistent Hashing and Virtual Nodes.

Consitent Hashing and Virtual Nodes. Paper
Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications

Consitent Hashing Paper
Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web

Consistent Hashing and Virtual Nodes

Amazon’s Dynamo DB uses consistent hashing with virtual nodes to locate nodes and keys in the dynamo ring.
Dynamo: Amazon’s Highly Available Key-value Store

Debug Hadoop Unit Test which runs on Maven

I am just running a single unit test

mvn -Dmaven.surefire.debug test -Dtest=org.apache.hadoop.hdfs.server.namenode.TestAddBlock


The tests will automatically pause and await a remote debugger on port 5005. You can then attach to the running tests using Eclipse.

Run > Debug Configarations

If you need to configure a different port, you may pass a more detailed value. For example, the command below will use port 8000 instead of port 5005.

mvn -Dmaven.surefire.debug="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Xnoagent -Djava.compiler=NONE" test

CAP Theorem

In 2000, Eric Brewer published a paper stating that a Distributed system cannot simultaneously provide all three of the following properties:

• Consistency: A read sees all previously completed writes.
• Availability: Reads (Actual Data) and writes always succeed.
• Partition tolerance: Guaranteed properties are maintained even when network failures prevent some machines from communicating with others.
   

The CAP Theorem says that the Distributed system cannot have all 3 of the above, it can only have any two of the above properties (CA, AP, CP).

Run Haddop Unit Tests

Rull all the tests
mvn test

Run a single test
mvn test -Dtest=org.apache.hadoop.fs.TestSymlinkLocalFSFileContext

Run all the test even if there are errors
mvn -Dmaven.test.failure.ignore=true test

Run a single test even if there are errors
mvn -Dmaven.test.failure.ignore=true test -Dtest=org.apache.hadoop.fs.TestSymlinkLocalFSFileContext

Basic Git Commands

Get a copy of specific repository branch to the local computer
git clone -b [branch name] https://github.com/[namespace]/[project].git


Initializes and push a new project to GitHub
git init
git add .
git commit -m "first commit"
git remote add origin https://github.com/[namespace]/[project].git
git push -u origin master


Reset all the local repository changes to Head.
git reset --hard

Show the working tree status
git status

Hide current working directory in Ubuntu Terminal

Just type below command in the Terminal.
export PS1='\u@\h:~$ '

Add above command to the bashrc file, so you do not have to type in the command again every time you load a new terminal.
sudo vi $HOME/.bashrc

Go to the bottom of the file and add the following using a editor such as VI.
export PS1='\u@\h:~$ '