Loading...

Messages

Proposals

Stuck in your homework and missing deadline? Get urgent help in $10/Page with 24 hours deadline

Get Urgent Writing Help In Your Essays, Assignments, Homeworks, Dissertation, Thesis Or Coursework & Achieve A+ Grades.

Privacy Guaranteed - 100% Plagiarism Free Writing - Free Turnitin Report - Professional And Experienced Writers - 24/7 Online Support

Hadoop streaming python word count example

06/12/2021 Client: muhammad11 Deadline: 2 Day

1. In class we wrote a MapReduce program in Java to compute the word counts for any given input. In this assignment, you will repeat solving the same problem but using Hadoop streaming.

2. Create two scripts in Python namely wordcount_map.py and wordcount_reduce.py to be used by the mappers and reducers of the streaming job.

3. Your script files must be executable (consider chmod command), and must include the necessary shebang (like in the attached script files).

4. Attached are the script files we used in class to demonstrate Hadoop streaming, namely: maxtemp_map.py and maxtemp_reduce.py. They can help you to get started.

5. Recall the streaming command:

$ mapred streaming \

-files , \

-mapper \

-reducer \

-input \

-output

(extra options: -combiner, -numReduceTasks, etc.)

MaxTemperature Example file is the program file discussed in Class.

Mapper for the Maximum temperature Example:

import java.io.IOException;

import org.apache.hadoop.io.IntWritable;

import org.apache.hadoop.io.LongWritable;

import org.apache.hadoop.io.Text;

import org.apache.hadoop.mapreduce.Mapper;

public class MaxTemperatureMapper

extends Mapper {

private static final int MISSING = 9999;

@Override

public void map(LongWritable key, Text value, Context context)

throws IOException, InterruptedException {

String line = value.toString();

String year = line.substring(15, 19);

int airTemperature;

if (line.charAt(87) == '+') { // parseInt doesn't like leading plus signs

airTemperature = Integer.parseInt(line.substring(88, 92));

} else {

airTemperature = Integer.parseInt(line.substring(87, 92));

}

String quality = line.substring(92, 93);

if (airTemperature != MISSING && quality.matches("[01459]")) {

context.write(new Text(year), new IntWritable(airTemperature));

}

}

}

Reducer for the max temperature example

import java.io.IOException;
import org.apache.hadoop.io.IntWritable;

import org.apache.hadoop.io.Text;

import org.apache.hadoop.mapreduce.Reducer;

public class MaxTemperatureReducer

extends Reducer {

@Override

public void reduce(Text key, Iterable values, Context context)

throws IOException, InterruptedException {

int maxValue = Integer.MIN_VALUE;

for (IntWritable value : values) {

maxValue = Math.max(maxValue, value.get());

}

context.write(key, new IntWritable(maxValue));

}

}

Application to find the maximum temperature in the weather dataset

import org.apache.hadoop.fs.Path;

import org.apache.hadoop.io.IntWritable;

import org.apache.hadoop.io.Text;

import org.apache.hadoop.mapreduce.Job;

import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;

import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;

public class MaxTemperature {

public static void main(String[] args) throws Exception {

if (args.length != 2) {

System.err.println("Usage: MaxTemperature ");

System.exit(-1);

}

Job job = new Job();

job.setJarByClass(MaxTemperature.class);

job.setJobName("Max temperature");

FileInputFormat.addInputPath(job, new Path(args[0]));

FileOutputFormat.setOutputPath(job, new Path(args[1]));

job.setMapperClass(MaxTemperatureMapper.class);

job.setReducerClass(MaxTemperatureReducer.class);

job.setOutputKeyClass(Text.class);

job.setOutputValueClass(IntWritable.class);

System.exit(job.waitForCompletion(true) ? 0 : 1);

Homework is Completed By:

Writer Writer Name Amount Client Comments & Rating
Instant Homework Helper

ONLINE

Instant Homework Helper

$36

She helped me in last minute in a very reasonable price. She is a lifesaver, I got A+ grade in my homework, I will surely hire her again for my next assignments, Thumbs Up!

Order & Get This Solution Within 3 Hours in $25/Page

Custom Original Solution And Get A+ Grades

  • 100% Plagiarism Free
  • Proper APA/MLA/Harvard Referencing
  • Delivery in 3 Hours After Placing Order
  • Free Turnitin Report
  • Unlimited Revisions
  • Privacy Guaranteed

Order & Get This Solution Within 6 Hours in $20/Page

Custom Original Solution And Get A+ Grades

  • 100% Plagiarism Free
  • Proper APA/MLA/Harvard Referencing
  • Delivery in 6 Hours After Placing Order
  • Free Turnitin Report
  • Unlimited Revisions
  • Privacy Guaranteed

Order & Get This Solution Within 12 Hours in $15/Page

Custom Original Solution And Get A+ Grades

  • 100% Plagiarism Free
  • Proper APA/MLA/Harvard Referencing
  • Delivery in 12 Hours After Placing Order
  • Free Turnitin Report
  • Unlimited Revisions
  • Privacy Guaranteed

6 writers have sent their proposals to do this homework:

Assignment Guru
Online Assignment Help
Pro Writer
Instant Assignment Writer
Engineering Exam Guru
Helping Hand
Writer Writer Name Offer Chat
Assignment Guru

ONLINE

Assignment Guru

I am a professional and experienced writer and I have written research reports, proposals, essays, thesis and dissertations on a variety of topics.

$35 Chat With Writer
Online Assignment Help

ONLINE

Online Assignment Help

I am an academic and research writer with having an MBA degree in business and finance. I have written many business reports on several topics and am well aware of all academic referencing styles.

$47 Chat With Writer
Pro Writer

ONLINE

Pro Writer

I have done dissertations, thesis, reports related to these topics, and I cover all the CHAPTERS accordingly and provide proper updates on the project.

$39 Chat With Writer
Instant Assignment Writer

ONLINE

Instant Assignment Writer

I am an academic and research writer with having an MBA degree in business and finance. I have written many business reports on several topics and am well aware of all academic referencing styles.

$23 Chat With Writer
Engineering Exam Guru

ONLINE

Engineering Exam Guru

I have read your project description carefully and you will get plagiarism free writing according to your requirements. Thank You

$24 Chat With Writer
Helping Hand

ONLINE

Helping Hand

I am a professional and experienced writer and I have written research reports, proposals, essays, thesis and dissertations on a variety of topics.

$40 Chat With Writer

Let our expert academic writers to help you in achieving a+ grades in your homework, assignment, quiz or exam.

Similar Homework Questions

Common core math learning progressions - Primary consumer marine food chain - HW - John maxwell 5 levels of leadership summary - Hx heart murmur icd 10 - John agard half caste - Benefits realisation plan template - Capella project management - Jotun balloxy hb light - Harvey norman moore park - No1 ccytotec pills +27835179056 NEW HOPE WOMEN ABORTION CLINIC in St Michael's-on-Sea Umkomaas Umtentweni Umzinto - Http www quickmba com marketing plan - Subway footlong sub of the month - 2 page, 4 questions. Human resources management. - Rias gifted test sample questions - Security Architecture and Design - Expressive techniques for piano - Journalize the four entries required to close the accounts - Statistics chapter 1 questions and answers - Entertainer who sang about deeds of knights - Which one of the following is not one of the major drivers of unethical managerial behavior? - Financial accounting chapter 8 solutions - Chocolate rain lyrics meaning - Theatre production planning worksheet - Finding ksp from molar solubility - Pros & Con of Bacteria - Sweat by hurston summary - Monica diaz and michael naranjo - Physiology mcq with answers - Vcaa study design biology - A light flexible rope is wrapped several times - Paxton net2 entry default engineer code - Safety and Quality Improvement in Professional Nursing Practice - Designing team and team identity articles - CRIMINAL JUSTICE - How to clean a walbro carburetor - Pruning currants and gooseberries - Alitalia verona to delhi - Cpl food safety answers - Peer Evaluation - Vi characteristics of germanium diode - Gee's "Literacy, Discourse, and Linguistics." - Westjet organizational structure - Operational Excellence - Autoways prestige car sales - Aastra 6755i firmware download - Beam light fixture projector - Description of a walk robert gray annotations - Cloud Computing Week 7 Assignment - Socw 6456 - Coca cola brotherly love commercial - Accounting identifying recording communicating - Organizational Profile & Executive Summary - Human Resource Managemnt - Brain case studies assignment answers - Reaction paper - Solomon four group design statistical analysis - Institute of interim management - Www stress org holmes rahe stress inventory - Chastity and shane are fraternal twins born 17 minutes apart - Radius of gyration of flywheel - Discussion 1 and 2 - Competitor analysis of itc ltd - Elementary differential equations 11th edition boyce pdf - Barani university rawalpindi jobs - Policy - Formula for manganese 4 oxide - Yo / llevar / este / reloj / la semana pasada - Malcolm x coming to an awareness of language summary - Multi wave oscillator for sale - Famous ancient greek writers - Post-mortem - Automotive standards for electronics - Define roundness in metrology - Caloric content of food labpaq answers - Collaboration diagram for student registration system - Similarities between npv and irr - Business Assignment - Australian standards tiling floors - Rear view camera installation diagram - African American Studies - IB315 Discussion Post 2 - Questions to ask your upline - St augustines st helens - Ainslie wood primary school - How to pronounce chinese names in english - Saudis expand regional power as others falter summary - Authoritarian personality theory states that extreme prejudice is - Bus 377 discussion question - Intermediate algebra with applications and visualization 3rd edition pdf - Intentional Torts, Negligence, and Strict Liability - Just a discussion question not a paper 100 words. - Philosophy and Vision for Nursing - 66a callows road bulli - Sikhism cosmogony origin of the universe - One nibble everyone knows the laws - 2 page paper - Bmc knowledge check answers - He have his goodness now - Woodside cattery dean lane