Hadoop MapReduce HTTP Notification

Normally MapReduce programs tend to run for long time and you might want to setup a way to get notification when the job is done finishing to find out if the job was executed successfully or not. Hadoop provides a mechanism in which you can get notification on the progress of your MapReduce job. This is two step process
  1. First set up job.end.notification.url property with value equal to a web application URL that should get invoked with the progress of job job.getConfiguration().set("job.end.notification.url", "http://localhost:8080/mrnotification/MRNotification/$jobId?status=$jobStatus");
  2. Next create a web application that will receive the notification and printout the job status and job name like this

2 comments: