MapReduce algorithm and Mumbai’s Dabbawalas!!!!

It’s astonishing to see the effectiveness and the operation scale of Mumbai dubbawala delivery system.Mumbai dubbawala’s are basically your personalized DoorDash guys. They source freshly cooked lunch from your home and deliver right to your office so that you can enjoy home food.

Mumbai Dabbawala code
Mumbai Dabbawala code

Few things you should know about Dabbawala association:

  1. It started in British rule way back in 1880 when many Indian people worked under British dislike the food and they want it to be served to them from home.
  2. It got Six sigma certification(CMMi 6 level) which means 99.9999% successful delivery rate.This means there is a chance of 1 error in 60 million transaction
  3. Every day they doing 4,00,000 transaction of boxes in the radios of 60 KM in 3 hour.

Now what is MapReduce:
It was introduced by Google in 2004 and Hadoop is the famous open source tool that implements MapReduce Algorithm and it is the backbone of many larger data computations now.MapReduce basically a divide and con-quire algorithm which breaks down the problem in to small components and processing it in parallel to achieve effective computation on a larger data set. It is having two steps.
1. Map
2. Reduce
Map:

Map Reduce skeleton
Map Reduce skeleton

In the Map step the master node takes the input, partitions it up into smaller sub-problems, and distributes them to worker nodes. A worker node may do this again in turn, leading to a multi-level tree structure. The worker node processes the smaller problem, and passes the answer back to its master node.
Reduce:
In the Reduce step the master node then collects the answers to all the sub-problems and combines them in some way to form the output – the answer to the problem it was originally trying to solve.

Now do you wonder how dabbawalla functioning similar with MapReduce Algorithm?
Dabbawalla Map step :
Dabbawalla collects the boxes from individual homes of particular area ( Distributed Master node) and sort it out and partition in to groups based on the location and the buildings. Then they distribute in to nearest delivery point (distributed to workers node). Now at the delivery point (the workers point), if it is the destination they switch in to reduce step ( see below of Dabbawalla Reduce Step) else they perform same operation like partitioning and distributing to another delivery point ( next level of workers), which typically form the multilevel tree mode.

Dabbawalla Reduce Step:
Now if it is the destination point, the dabbawala’s collect all the boxes based on the partition and start to delivering it. If a particular building or street needs more boxes to be delivered, they allocate more workers ( dynamically scaling nodes based on the workload).

To read more about MapReduce, read here

Now you could say,

Dear Google, We implemented MapReduce 100 years back itself!!!!

Advertisements

An introduction to video steaming

The streaming media is hot now as broadband speed increases and more people are watching video’s and programs over internet. The media, television in particular moving out of prime time concept. The traditional prime time concept is like the publisher chooses the time and broadcast their prime content like news or drama. Now the industry moving towards like instead of publishers picks what to see when, the end users or the viewers can choose what to see and when.
The technology improvement on streaming technology over internet to deliver digital content leads this shift. Already Hulu and Netflix well on their way to utilize this.
What is Streaming?
Media streaming is a process to deliver continuous media such as video or audio to a receiver as a continuous stream of packets. Due to this stream of packets, the receiver does not need to download the entire file before starting to play (or render) the media
Types of Streaming:
There are three types of streaming available now,
1. Traditional Streaming
2. Progressive Download
3. HTTP adaptive Streaming.

1. Traditional Streaming:
Using a traditional streaming protocol media is delivered to the client as a series of packets. Clients can issue commands to the media server to play (i.e., to send a media stream, to temporarily suspend this stream (i.e., to pause the media), or to terminate the media stream (i.e., to teardown the media stream). One of the standard protocols for issuing these commands is the Real Time Streaming Protocol (RTSP)
2. Progressive Download:
One of the most widely used methods of media delivery on the web today is progressive download. This is basically a download of a file from the web server, but with the client starting to play the media contents of this file before the file is completely downloaded. Unless the media stream is terminated, eventually the
entire file will be downloaded. In progressive download, downloading continues even if the user pauses the player.
The Internet’s most popular video sharing website – YouTube – uses progressive download
3. HTTP Adaptive Streaming
Adaptive streaming is based upon progressive download of small fragments of the media, but the particular fragments that are downloaded are chosen based upon an estimate of the current network conditions. Each of these fragments is called a chunk. Thus “adaptive streaming” is not actually streaming the media content, but instead it is an adaptive version of HTTP progressive download!

    Advantages of HTTP Adaptive Streaming over other approaches:

  1. It provides fast start-up and seek times with in a given item of content by initiating the video at the lowest video rate and later switching to a higher bit rate.
  2. There is no disconnection, buffering, or playback stutter problem.
  3. It provides seamless bit rate switching based on network conditions.
  4. It also provides the user with seamless video playback.

java singleton pattern using enum

There are two ways we can create singleton pattern in java.
1. create a private constructor and static method to return one instance
2. use enum.
Though the first approach is most common in use, there is a problem with it. what will happen if more than one thread trying to access the instance create code at the same time. That will eventually create more than one instances. Ofcourse we can make it thread safe but that will make even cumbersome to maintain in multi threading environment.

So this is the code for using enum to create a singleton. (Consider we need to create a singleton class called “Employee” and need to access a method called “getEmpDetails”

public enum Employee{
   INSTANCE;

   public List getEmployees() {
      //  method implementation goes here
   }
}

To access the method all you need to do is,

Employee.INSTANCE.getEmployees();

This is easy to implement and to maintain. Though some people argue the enum type is not meant for this, I leave the argument open. 🙂

Java Implementation of Haversine Formula for distance calculation between two points


package com.ananth.haversine;

/**
 * This is the implementation Haversine Distance Algorithm between two places
 * @author ananth
 * 	R = earth’s radius (mean radius = 6,371km)
	Δlat = lat2− lat1
	Δlong = long2− long1
	a = sin²(Δlat/2) + cos(lat1).cos(lat2).sin²(Δlong/2)
	c = 2.atan2(√a, √(1−a))
	d = R.c
 *
 */

public class HaversineDistance {

	/**
	 * @param args
	 * arg 1- latitude 1
	 * arg 2 - latitude 2
	 * arg 3 - longitude 1
	 * arg 4 - longitude 2
	 */
	public static void main(String[] args) {
		// TODO Auto-generated method stub
		final int R = 6371; // Radious of the earth
		Double lat1 = Double.parseDouble(args[0]);
		Double lon1 = Double.parseDouble(args[1]);
		Double lat2 = Double.parseDouble(args[2]);
		Double lon2 = Double.parseDouble(args[3]);
		Double latDistance = toRad(lat2-lat1);
		Double lonDistance = toRad(lon2-lon1);
		Double a = Math.sin(latDistance / 2) * Math.sin(latDistance / 2) + 
				   Math.cos(toRad(lat1)) * Math.cos(toRad(lat2)) * 
				   Math.sin(lonDistance / 2) * Math.sin(lonDistance / 2);
		Double c = 2 * Math.atan2(Math.sqrt(a), Math.sqrt(1-a));
		Double distance = R * c;
		
		System.out.println("The distance between two lat and long is::" + distance);

	}
	
	private static Double toRad(Double value) {
		return value * Math.PI / 180;
	}

}