How to make a codebook for content analysis

BACKGROUND. Imagine that you work for the United Nations, who issued (through their Intergovernmental Panel on Climate Change) a scientific consensus statement indicating that humans almost certainly cause global warming. At the same time, you are concerned that newspapers who report on global warming (also called climate change) may be presenting the issue with a “false balance” – presenting arguments for and against humans causing climate change, when there is a scientific consensus on one side of the issue. A previous study found evidence for this false balance (Boykoff & Boykoff, 2004); your job is to assess whether patterns of coverage in the last 6 months reflect a similar tendency to report with a false balance.

YOUR TASKS. Your job is to develop a sampling strategy and codebook for a content analysis of New York Times, Washington Post, Los Angeles Times, and Wall Street Journal news stories related to global warming and/or climate change published in the last 6 months. In order to do so, you will need to complete the 6 steps involved in doing a content analysis.

The population of interest for this assignment includes all news stories related to global warming and/or climate change published in the last 6 months. To obtain copies of the text of these news stories, you should use Nexis Uni and enter the following search term for stories that have appeared in the New York Times, Washington Post, Los Angeles Times, & Wall Street Journal abstracts (you may choose whether to focus on print stories only, blogs, or both - see Nexis’ list of sources for these papers):

“global warming” OR “climate change”

AND DATE is within the last 6 months

You may not be able to analyze all of these news stories (e.g., conduct a census), depending on how many newspaper articles are retrieved with this search term, so you may need to figure out a sampling strategy that will allow you to test whether or not these newspapers cover global warming and/or climate change in a falsely balanced way using a sample of news stories.

Task 1. In approximately two single-spaced, typed pages (12-pt font), answer each of the following questions: (4.5 points in total)

1) DEVELOP YOUR MEASURES (step 2 from lecture)

a. Provide a one-sentence conceptual definition of each variable involved in the research question. You’ll need to provide a conceptual definition for (a) news stories related to global warming and/or climate change, and (b) false balance. You will find the following articles helpful to look at as a starting point. You may start from the authors’ conceptualizations of related terms like “information bias,” “balanced reporting,” and “false balance”, but you should come up with your own definitions of what would constitute a “false balance.” (1 point)

Boykoff, M. T., & Boykoff, J. M. (2004). Balance as bias: Global warming and the US prestige press. Global Environmental Change, 14, 125-136. doi: 10.1016/j.gloenvcha.2003.10.001. Available online at:

http://www.eci.ox.ac.uk/publications/downloads/boykoff04-gec.pdf

Dixon, G., & Clarke, C. (2013). The effect of falsely balanced reporting of the autism-vaccine controversy on vaccine safety perceptions and behavioral intentions. Health Education Research, 28, 352-359. doi:10.1177/1075547012458290. Available online at: http://her.oxfordjournals.org/content/28/2/352.long

Grading Criteria:

o Provide a conceptual definition for news stories related to global warming and/or climate change, being sure to (1) address what you mean by news stories and (2) what counts as being related to these topics (.5 pts)

o Provide a conceptual definition for “false balance” including key dimensions or elements. The definition should explain clearly (1) what do you mean by “balance” and (2) why such balance is considered biased or false (.5 pts)

o Note: These definitions should (1) be written in your own words (not copying the authors’ terms but citing the authors if they influence your definition), (2) represent a conceptual, not operational definition (-.25 for each error).

2) DEVELOP A SAMPLING STRATEGY (step 3 from lecture)

Specify how many news stories you’ll need to analyze to test your research question. (1 point)

In other words, calculate your sample size(s). Describe why you chose this particular sample size. To make these decisions, you’ll want to specify the level of confidence and error you are willing to tolerate for this study. You will want to make use of a sample size calculator in making your decisions.

Grading Criteria: [Fulfill all three for full credit; deduct .5 for each error up to 1]

o Specify the number of news stories in your population

o Specify level of confidence

o Specify error willing to tolerate

o Include sample size (Note: Given your level of confidence and standard error, the number of your sample size should look reasonable to us)

Describe your sampling strategy using terms and concepts we’ve learned about in class and/or in the textbook. (1.5 points)

In this description, be sure that you answer each of the following questions: (i) Among all types of news stories (e.g., news stories, editorials, blog posts, opinion pieces, etc.), which ones will you choose as news stories related to global warming and/or climate change? (HINT: this decision should match your conceptual definition of what constitutes a news story related to the topic).

You must start by using the Nexis Uni search terms listed above to search for relevant stories to include in your sample. The set of articles that are returned from your search term will serve as the population for this assignment (HINT: depending on what issues of climate change you are focusing on, your population may be smaller or larger).

You will then need to describe the process of selecting a sample from this population by answering the following questions: (ii) How will you select specific news stories, from the overall population of news stories, to include in your sample? Will this process involve non- random sampling (convenience or snowball) or random sampling (simple or multi-stage cluster)? You can propose any strategy that you consider appropriate, but be sure to justify your decision and identify tradeoffs associated with that decision.