Loading...

Messages

Proposals

Stuck in your homework and missing deadline? Get urgent help in $10/Page with 24 hours deadline

Get Urgent Writing Help In Your Essays, Assignments, Homeworks, Dissertation, Thesis Or Coursework & Achieve A+ Grades.

Privacy Guaranteed - 100% Plagiarism Free Writing - Free Turnitin Report - Professional And Experienced Writers - 24/7 Online Support

Hadoop streaming python word count example

06/12/2021 Client: muhammad11 Deadline: 2 Day

1. In class we wrote a MapReduce program in Java to compute the word counts for any given input. In this assignment, you will repeat solving the same problem but using Hadoop streaming.

2. Create two scripts in Python namely wordcount_map.py and wordcount_reduce.py to be used by the mappers and reducers of the streaming job.

3. Your script files must be executable (consider chmod command), and must include the necessary shebang (like in the attached script files).

4. Attached are the script files we used in class to demonstrate Hadoop streaming, namely: maxtemp_map.py and maxtemp_reduce.py. They can help you to get started.

5. Recall the streaming command:

$ mapred streaming \

-files , \

-mapper \

-reducer \

-input \

-output

(extra options: -combiner, -numReduceTasks, etc.)

MaxTemperature Example file is the program file discussed in Class.

Mapper for the Maximum temperature Example:

import java.io.IOException;

import org.apache.hadoop.io.IntWritable;

import org.apache.hadoop.io.LongWritable;

import org.apache.hadoop.io.Text;

import org.apache.hadoop.mapreduce.Mapper;

public class MaxTemperatureMapper

extends Mapper {

private static final int MISSING = 9999;

@Override

public void map(LongWritable key, Text value, Context context)

throws IOException, InterruptedException {

String line = value.toString();

String year = line.substring(15, 19);

int airTemperature;

if (line.charAt(87) == '+') { // parseInt doesn't like leading plus signs

airTemperature = Integer.parseInt(line.substring(88, 92));

} else {

airTemperature = Integer.parseInt(line.substring(87, 92));

}

String quality = line.substring(92, 93);

if (airTemperature != MISSING && quality.matches("[01459]")) {

context.write(new Text(year), new IntWritable(airTemperature));

}

}

}

Reducer for the max temperature example

import java.io.IOException;
import org.apache.hadoop.io.IntWritable;

import org.apache.hadoop.io.Text;

import org.apache.hadoop.mapreduce.Reducer;

public class MaxTemperatureReducer

extends Reducer {

@Override

public void reduce(Text key, Iterable values, Context context)

throws IOException, InterruptedException {

int maxValue = Integer.MIN_VALUE;

for (IntWritable value : values) {

maxValue = Math.max(maxValue, value.get());

}

context.write(key, new IntWritable(maxValue));

}

}

Application to find the maximum temperature in the weather dataset

import org.apache.hadoop.fs.Path;

import org.apache.hadoop.io.IntWritable;

import org.apache.hadoop.io.Text;

import org.apache.hadoop.mapreduce.Job;

import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;

import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;

public class MaxTemperature {

public static void main(String[] args) throws Exception {

if (args.length != 2) {

System.err.println("Usage: MaxTemperature ");

System.exit(-1);

}

Job job = new Job();

job.setJarByClass(MaxTemperature.class);

job.setJobName("Max temperature");

FileInputFormat.addInputPath(job, new Path(args[0]));

FileOutputFormat.setOutputPath(job, new Path(args[1]));

job.setMapperClass(MaxTemperatureMapper.class);

job.setReducerClass(MaxTemperatureReducer.class);

job.setOutputKeyClass(Text.class);

job.setOutputValueClass(IntWritable.class);

System.exit(job.waitForCompletion(true) ? 0 : 1);

Homework is Completed By:

Writer Writer Name Amount Client Comments & Rating
Instant Homework Helper

ONLINE

Instant Homework Helper

$36

She helped me in last minute in a very reasonable price. She is a lifesaver, I got A+ grade in my homework, I will surely hire her again for my next assignments, Thumbs Up!

Order & Get This Solution Within 3 Hours in $25/Page

Custom Original Solution And Get A+ Grades

  • 100% Plagiarism Free
  • Proper APA/MLA/Harvard Referencing
  • Delivery in 3 Hours After Placing Order
  • Free Turnitin Report
  • Unlimited Revisions
  • Privacy Guaranteed

Order & Get This Solution Within 6 Hours in $20/Page

Custom Original Solution And Get A+ Grades

  • 100% Plagiarism Free
  • Proper APA/MLA/Harvard Referencing
  • Delivery in 6 Hours After Placing Order
  • Free Turnitin Report
  • Unlimited Revisions
  • Privacy Guaranteed

Order & Get This Solution Within 12 Hours in $15/Page

Custom Original Solution And Get A+ Grades

  • 100% Plagiarism Free
  • Proper APA/MLA/Harvard Referencing
  • Delivery in 12 Hours After Placing Order
  • Free Turnitin Report
  • Unlimited Revisions
  • Privacy Guaranteed

6 writers have sent their proposals to do this homework:

Assignment Guru
Online Assignment Help
Pro Writer
Instant Assignment Writer
Engineering Exam Guru
Helping Hand
Writer Writer Name Offer Chat
Assignment Guru

ONLINE

Assignment Guru

I am a professional and experienced writer and I have written research reports, proposals, essays, thesis and dissertations on a variety of topics.

$35 Chat With Writer
Online Assignment Help

ONLINE

Online Assignment Help

I am an academic and research writer with having an MBA degree in business and finance. I have written many business reports on several topics and am well aware of all academic referencing styles.

$47 Chat With Writer
Pro Writer

ONLINE

Pro Writer

I have done dissertations, thesis, reports related to these topics, and I cover all the CHAPTERS accordingly and provide proper updates on the project.

$39 Chat With Writer
Instant Assignment Writer

ONLINE

Instant Assignment Writer

I am an academic and research writer with having an MBA degree in business and finance. I have written many business reports on several topics and am well aware of all academic referencing styles.

$23 Chat With Writer
Engineering Exam Guru

ONLINE

Engineering Exam Guru

I have read your project description carefully and you will get plagiarism free writing according to your requirements. Thank You

$24 Chat With Writer
Helping Hand

ONLINE

Helping Hand

I am a professional and experienced writer and I have written research reports, proposals, essays, thesis and dissertations on a variety of topics.

$40 Chat With Writer

Let our expert academic writers to help you in achieving a+ grades in your homework, assignment, quiz or exam.

Similar Homework Questions

Below is a unit cell for a hypothetical metal - Business process management the third wave - Argument for and Against Slavery - Does the shoe company sell toms - Test chapter 5 legal and ethical responsibilities answers - Mr barton maths m1 - The wright group sunshine books guided reading level - Lab 4 cell structure and function - Describe a difficult interaction you had with a customer - John maddocks jazz band - International 5 - I kuan tao rituals - Dq response - Discussion (Chapter 6): - Capital budgeting excel assignment - What is the ring of fire made of - Question ppt - School sport unit nsw - A project that provides annual cash flows of - Basics of information technology ppt - Disadvantages of tightly coupled systems - Cloud computing as a resource for knowledge management - 8 1 mymarketinglab simulation activity market research answers - 123 passport photo review - Avon swot analysis 2017 - Bargaining power of suppliers in restaurant industry - CAREER GOALS - Help in writing letter and user manual - 4 indicators of a chemical reaction - Comprehensive problem 2 palisade creek co answers - Https miic health state mn us - Needs and wants of ancient communities pdf - Youth soccer injuries have skyrocketed - Diversity and culture - Persuasive written communication - Managerial economics markets and the firm 2nd edition pdf - Is macconkey agar a defined or undefined medium - Cognitive psychology and its implications 9th edition - Post a comparison of at least two APRN board of nursing regulations in your state/region with those of at least one other state/region. Describe how they may differ. Be specific and provide examples. - Morganton company makes one product - Wireless plc communication with gsm module - Tort Law - Event tree analysis fire - Matlab state space simulation - Kelly's personal construct theory emphasizes the notion that - Transformational Leadership - Meaning of the word economic - Amazon succession plan - Edinburgh napier university library - Discussion Board 4.1: Global Advocacy in Action - How to use galvanic spa system ii - Punching the clock lyrics j cole - Are toxins involved in set point theory - Key supply chain metrics - Ni usb 6501 datasheet - Mkt - Racq vehicle insurance certificate - Properties of aldehydes and ketones experiment - Order 2288566: Small Group Budget at a school - Www turtlebeach com warranty - Eastspring investments global target income fund - Graduate engineer trainee program - Compute the total overhead cost applied to job 127 - Adler v george 1964 - PS490 Assignment 8 Research Proposal - Blackboard valpo edu - Change selected cell to 20pt in excel - Warning letter for taking frequent leaves - Ffdc logs in websphere application server - Questionnaire based on maslow's hierarchy of needs - Applying Industrial Relations Principles - Perform the gram-schmidt process on the following sequence of vectors - How do you calculate grain bin capacity - Accommodation near mt baw baw - Game show network cox - 1- Identify themes of crime causation as seen in Taxi Driver and Joker. 2- Compare Travis and Arthur and their respective environments - Act 1 scene 3 macbeth analysis - Shannon weaver transactional model of communication - Rise and fall of toms shoes - How to calculate nursing care hours - Rsa authentication manager 8.1 - Writing a case study - System positioning device nessus - Si njay njay njay lyrics - Red rooster line sydney - Louis Vuitton Clearance - Exsys corvid download - Network logical layout - Accepted value of copper - A list of accounts and their balances - ** WRITERVICTOR ** DISCUSSION QUESTION (250 WORDS, 1 CURRENT SOURCE) APA - Ag144ut 001 a4 compressor - From net earnings of $740 per month - Partnership - Discussion (course - Bussiness Continuation Plan & Disaster Recovery Plan ) - Section 12.4 early single celled organisms study guide answers - Cosmos episode 8 worksheet answers - Essay type - Case Study Outline - Aqa english literature a level grade boundaries