Paribus amazon checking in progress

Research Methods in Education

2

For my wife Angela, my loving supporter and most honest advisor, always.

—Joseph Check

To my wife, Elizabeth.

—Russell Schutt

3

Research Methods in Education

Joseph CHECK Russell K. SCHUTT

University of Massachusetts Boston

4

FOR INFORMATION:

SAGE Publications, Inc.

2455 Teller Road

Thousand Oaks, California 91320

E-mail: order@sagepub.com

SAGE Publications Ltd.

1 Oliver's Yard

55 City Road

London EC1Y 1SP

United Kingdom

SAGE Publications India Pvt. Ltd.

B 1/I 1 Mohan Cooperative Industrial Area

Mathura Road, New Delhi 110 044

India

SAGE Publications Asia-Pacific Pte. Ltd.

33 Pekin Street #02-01

Far East Square

Singapore 048763

Copyright © 2012 by SAGE Publications, Inc.

All rights reserved. No part of this book may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission in writing from the publisher. Printed in the United States of America

Library of Congress Cataloging-in-Publication Data

Check, Joseph W., 1947-

Research methods in education / Joseph Check, Russell K. Schutt.

p. cm.

Includes bibliographical references and index.

ISBN 978-1-4129-4009-2 (pbk.)

1. Education—Research—Methodology. I. Schutt, Russell K. II. Title.

LB1028.C523 2012

370.72—dc232011032014

This book is printed on acid-free paper.

11 12 13 14 15 10 9 8 7 6 5 4 3 2 1

Acquisitions Editor: Jerry Westby

Editorial Assistant: Erim Sarbuland

Production Editor: Catherine M. Chilton

5

Copy Editor: Gillian Dickens

Typesetter: C&M Digitals (P) Ltd.

Proofreader: Annette R. Van Deusen

Indexer: Sheila Bodell

Cover Designer: Candice Harman

Marketing Manager: Katherine Winter

Permissions: Karen Ehrmann

6

Detailed Contents

Preface Acknowledgments Part I. Foundations of Research

1. Science, Schooling, and Educational Research 2. The Process and Problems of Educational Research 3. Ethics in Research 4. Conceptualization and Measurement 5. Sampling

Part II. Research Design and Data Collection 6. Causation and Research Design 7. Evaluation Research 8. Survey Research 9. Qualitative Methods: Observing, Participating, Listening 10. Single-Subject Design 11. Mixing and Comparing Methods and Studies 12. Teacher Research and Action Research

Part III. Analyzing and Reporting Data 13. Quantitative Data Analysis 14. Qualitative Data Analysis 15. Proposing and Reporting Research

Appendices Glossary References Author Index Subject Index About the Authors

7

Preface

This is an exciting time to begin learning about educational research. For many years, the study of education was slow to change and regularly criticized—by practitioners for not being practical enough, by researchers from other disciplines for not being theoretical enough (Lagemann, 2000). Research that met rigorous “scientific” standards was accused of staying on library shelves and being of little use to teachers. Research that documented the daily life of students and teachers was derided as mere storytelling and “not scientific enough” (Walters & Lareau, 2009, pp. 1–2). Overall, education research was characterized by “continuing contests among different groups, especially scholars of education, scholars in other fields and disciplines, school administrators, and teachers” (Lagemann, 1997, p. 5).

Despite criticisms and contention, in the early years of the new century—the 2000s—it became clear that things were changing, that we were seeing “an explosion of new methodologies and approaches to inquiry” (Biesta & Burbules, 2003, p. 1) that could make educational research an evolving and exciting field, both practical and theoretical, both rigorous and context sensitive.

Educational research today is marked by three qualities. It is eclectic, dynamic, and essential. By eclectic, we mean that many disciplines and schools of thought contribute to educational research. By dynamic, we mean rapidly evolving, with new philosophies and methods emerging on a regular basis. By essential, we mean necessary, and recognized as necessary, for the optimal development of schooling.

8

Electic

Eclectic means coming from many sources. Because learning is affected both by schooling and by factors outside schooling, educational research must be interdisciplinary, meaning that it involves more than one academic discipline or field of study. An “educational researcher” might be a professor of education, a sociologist, an anthropologist, an economist, a psychologist, or a medical doctor. In some forms of educational research, teachers are themselves researchers of their own classrooms.

Many common techniques in educational research originated outside the field of education. Anthropology gave us ethnography, the study of culture and cultural processes, now used to study classrooms and the culture of schools. Sociology gave us survey research; survey techniques are now used throughout education to study large populations of teachers and students. Psychology gave us tools for understanding the intellectual and emotional development of children. Quantitative disciplines such as mathematics and economics contribute multiple tools based on statistical analysis, including recent work in areas such as effect sizes, which measure the strength of relationships between variables. Many academic disciplines have scholarly journals that deal specifically with the intersection between their discipline and education, including Anthropology and Education, Education Economics, Sociology of Education, and the Journal of Educational Psychology.

9

Dynamic

Educational research is a continually developing field. New theoretical approaches and data-gathering techniques emerge regularly. This is due in part to education's dynamic relationship with other fields of study, mentioned above. Examples of techniques that have been developed or refined in recent years include multilevel analysis and narrative inquiry. Multilevel analysis is an advanced statistical technique that enables a researcher to accurately analyze “nested” or “hierarchical” data sets at different levels, such as an individual student, a classroom of students, and a whole school (Hox, 2010). Narrative inquiry concentrates on collecting and telling the life stories of a single individual (Clandinin, Pushor, & Orr, 2007). A second kind of dynamic tension is created by the current federal push for scientifically based research and the simultaneous questioning of what that means in a naturalistic setting such as a classroom, particularly in classrooms with culturally diverse and disadvantaged populations.

Education researchers also have their own energetic and up-to-date professional organization, the American Educational Research Association (AERA), which provides a platform for discussion and new inquiry techniques and is open to participation from anyone involved in educational inquiry, whatever their philosophical or methodological approach.

10

Essential

Accountability-based school reform efforts, particularly federal and state legislation, have made test scores, dropout rates, and other measurements an essential component of life in schools. Regular data collection, one of the key elements of research, is now the norm. New professional standards in fields such as school counseling and special needs teaching stress the importance of evidence-based practice. In addition, the growing popularity of formats such as teacher research and action research means that teachers are no longer just subjects of research conducted by others. Classroom teachers now have the tools to become researchers of their own practice and change agents in their own buildings. Taken together, these developments make research an essential dimension of life in schools today, both for the students, teachers, and administrators inside them and the parents, legislators, and policy makers outside.

At its heart, research starts with questions; every chapter of this book begins with a research question. Examples of published research studies related to the opening question are then used as examples throughout the chapter. By the end of the book, you will have encountered more than 30 published research studies that take different theoretical approaches and use many different methods to gather and interpret data.

11

Acknowledgments

Our thanks to Jerry Westby, Acquisitions Editor, SAGE Publications. Jerry has encouraged and supported this project continuously from the idea stage through final production. His patience, confidence, and human touch have smoothed the rough spots and helped us always keep the final goal in view. No author could ask for a better publisher.

We also thank the personable and highly creative SAGE professionals who turned a typescript into a scrupulously edited, highly attractive book: editor Denise Simon, production manager Catherine Chilton, copy editor Gillian Dickens, marketing manager Katharine Winter, associate editor Megan Krattli, editorial assistant Erim Sarbuland, and assistant editor Rachael Leblond. Kathryn Stoeckert Sabella did a superb job of developing the online exercises, and Gina Cook and Michelle Turner Mangan developed the fine set of ancillary materials.

We feel gratitude and friendship for the unique, cross-disciplinary support group of SAGE authors of related methods texts who have extended their wisdom and expertise: Ronet Bachman (University of Delaware), Dan Chambliss (Hamilton College), Ray Engel (University of Pittsburgh), and Paul Nestor (University of Massachusetts, Boston).

We are also indebted to the first-rate group of reviewers whose thoughtful suggestions and cogent insights have helped improve every chapter. They are Kimberly Alkins, Queens College; Maureen Angell, Illinois State University; Scott Bauer, George Mason University; Calvin Brown, Tennessee State University; Elaine Bukowiecki, Bridgewater State College; Laurie Carlson, Colorado State University; Dick M. Carpenter II, University of Colorado; Stephen E. Cramer, University of Georgia; Patrick Dilley, Southern Illinois University; Paul Erickson, Eastern Kentucky University; Chester Fuller, Central Michigan University; Gerry Giordano, University of North Florida; Carol Ann Gittens, Santa Clara University; Kristine Hogarty, University of South Florida; John Huss, Northern Kentucky University; Richard M. Jacobs, Villanova University; Lori Kim, California State University, Los Angeles; Lydia Kyei-Blankson, Illinois State University; Marie Kraska, Auburn University; Maria K. E. Lahman, University of Northern Colorado; Brian Lawler, California State University San Marcos; Alar Lipping, Northern Kentucky University; Jeffrey Lorentz, University of Houston, Clear Lake; Crystal Machado, Indiana University of Pennsylvania; Jerry R. McMurtry, University of Idaho; Steven Miller, Loyola University Chicago; Ralph Mueller, George Washington University; Victor Nolet, Western Washington University; Jeffrey Oescher, Southeastern Louisiana University; Gary W. Ritter, University of Arkansas; Charles Thomas, George Mason University; Jacqueline Waggoner, University of Portland; Scott Walter, Washington State University; Carol Wickstrom, University of North Texas; Diane Wilcox, James Madison University; and Nadia Zabtcheva, Montclair State University.

Joe thanks Russ greatly for the opportunity to co-create this book and for his constant concern not just for high-quality content but for straightforward writing to make the content accessible to students. Joe also

12

thanks Karen Daniels and Margo Moore, the two hard-working graduate assistants who helped with this project; departmental administrative staffer Molly Pedriali, who was always needed and always there; his

faculty colleagues, for their invaluable expertise not just as researchers but as teachers; and his students, on whom many of the approaches in this book were tried out. Finally, Joe thanks his wife Angela for her constant and unstinting encouragement and support, both emotional and editorial, and his children, Joe and Pietra, for their suggestions and support, and for enriching his life just by being who they are.

Russ thanks Joe for his skill, dedication, and good cheer throughout this project, and he thanks his wife Elizabeth for her love and support, and his daughter Julia, for the joy she brings to his life and for all that she contributes to the social world.

13

Part I Foundations of Research

14

Chapter 1 Science, Schooling and Educational Research

Research Question: How Do Early Childhood Experiences Affect Schooling?

15

Chapter Contents

Learning About the Educational World The Educational Research Approach Educational Research Philosophies

Can watching too much television hurt toddlers, putting some children at a disadvantage before they enter preschool? Are preschool children whose mothers work outside the home academically behind when they go to school? Understanding early childhood influences is a key task for educational researchers and forms the research focus for this chapter. Dimitri A. Christakis, Frederick J. Zimmerman, David L. DiGiuseppe, and Carolyn A. McCarty (2004, pp. 708–713) studied more than 1,200 children and found that 1- and 3-year- olds who watched more than 3 hours of television daily had a substantially increased risk of developing attention problems by age 7. This was one of the first large-scale, systematic investigations that linked difficulty concentrating, restlessness, and impulsive behavior to watching television as a young child.

The Christakis et al. (2004) study appeared in the medical journal Pediatrics, yet it deals with an educational question that parents and early childhood teachers face every day. The study took a scientific, research-based approach to a complex, emotional question: Does too much TV harm very young children?

Have you thought about TV's effects on your own learning or that of children you have contact with? No one wants to feel that children are threatened by television, yet TV is widely used as an “electronic babysitter.” In this chapter, you will learn how the Christakis et al. (2004) study and other, more recent investigations are helping to answer important questions about early childhood learning.

16

Learning about the Educational World

Just one research question about learning raises so many more questions about education. Take a few minutes to read each of the following questions and jot down your answers. Don't worry about your responses: This is not a test; there are no “wrong” answers.

1. Do you think you watched too much television as a child? 2. How has television watching affected your learning? 3. How many hours of television does the average American child watch every day? 4. Does the content of what young children watch make a difference? Is educational programming such as

Sesame Street better for a 3-year-old than Saturday cartoons? 5. How much does quality of early child care in general affect learning? 6. Most television viewing takes place outside school, but its effects show up in the classroom. How do you

think other forces outside school affect how children will perform in school?

We'll bet you didn't have too much trouble answering the first two questions, about your own experiences, but what about the others? These four questions concern “the educational world”—the educational experiences and orientations of people in addition to ourselves. Usually, when we answer such questions, we combine information from many different sources. We may also recognize that our answers to the last four questions will be shaped by our answers to the first two—that is, what we think about the educational world will be shaped by our own experiences and by the ways we have learned about the experiences of others. Of course, this means that different people will often come up with different answers to the same questions. Studying research methods will help you learn what criteria to apply when evaluating these different answers and what methods to use when seeking to develop your own answers.

Take a bit of time in class and share your answers to the six questions. Why do some of your answers differ from those of your classmates? Now, let's compare your answers to Questions 3 through 6 to the findings of researchers using educational research methods.

Question 3: Preschool children in general watch anywhere from 0 to 30 hours of television a week, but the average is more than 4 hours per day. Four-year-olds average 50 to 70 minutes of viewing daily, most of which is cartoons (Huston, Wright, Rice, Kerkman, & St. Peters, 1990).

Exhibit 1.1 shows the amount of television watched by 18-month-olds in the Christakis et al. (2004) study mentioned earlier. Notice that the highest bar is at the zero level—no TV watching—and that more than half of the children studied watched 0 to 2 hours per day of television.

Question 4: Television viewing at an early age affects learning in school at a later age, but the connection is not a simple one. Important considerations include the type of programs viewed (commercial or educational), whether parents talk with children about what they're seeing, and how well children understand the difference between cartoon-type fantasies and real life (Peters & Blumberg, 2002).

17

Question 5: Conditions of child care, particularly the amount and quality of adult attention, can make a large difference in how television viewing affects later learning. Researchers at the National Institute of Child Health and Human Development (2002) studied the effects of early child care on more than 1,000 children. The study found that “children's development was predicted by early child-care experience” (p. 133). Educational development was connected to quantity, quality, and type of child care just prior to the child going to school. Higher quality care was associated with better language skills for 4½-year-olds, and after a certain point, the more hours children spent in care, the more behavior problems they showed.

Exhibit 1.1 Hours of Television Viewing for 18-Month-Old Children

Source: Christakis et al. (2004, p. 711).

Question 6: Many factors outside school affect learning in school. Physical and cognitive disabilities, for example, affect in-school learning for many children. Later in this chapter, you will learn of research on a legally blind preschool child in a classroom with 13 physically normal students. You will also learn of research that looks at cognitive effects on children when mothers work during the first year of the child's life. Economic and social factors play a large role in educational success or failure, and the federal government has created “compensatory” programs to level the educational playing field. One such program, Early Head Start, seems to be succeeding, and you will also learn of this research.

How do these answers compare with the opinions you recorded earlier? Do you think your personal experiences have led you to different answers than others might have given? Do you see how different people can come to such different conclusions about educational issues?

We cannot avoid asking questions about our complex educational world or trying to make sense of our position in it. In fact, the more that you begin to “think like an educational researcher,” the more such questions will come to mind. But as you've just seen, our own prior experiences and orientations, particularly

18

our own experiences as learners and teachers, can have a major influence on what we perceive and how we interpret these perceptions. As a result, one person may see television as a way to extend learning to millions of children, another person may think television for preschoolers should be completely banned, and others may think the entire issue is overblown. In this chapter, you will begin to look at research results in an analytic way, asking what questions have been researched, what the results of these studies are and what they mean, and how much confidence we can place in them.

19

Errors to Avoid

Educational research relies on analytic thinking, and one important element of analytic thinking is avoiding errors in logic. As readers and consumers of educational research, we have a right to expect rigorous thinking in research articles. Errors in thinking can occur in the way a research question is constructed, the methods used to carry it out, or the conclusions the researcher draws. Becoming aware of some of the most common errors in thinking will give you a head start on becoming a smart reader of educational research.

Four common errors in reasoning occur in the nonscientific, unreflective talk and writing about education that we encounter daily. Our favorite examples of these “everyday errors in reasoning” come from a letter to Ann Landers. The letter was written by someone who had just moved with her two cats from the city to a house in the country. In the city, she had not let her cats outside and felt guilty about confining them. When they arrived in the country, she threw her back door open. Her two cats cautiously went to the door and looked outside for a while, then returned to the living room and lay down. Her conclusion was that people shouldn't feel guilty about keeping their cats indoors—even when they have the chance, cats don't really want to play outside.

Do you see this person's errors in reasoning? She was guilty of the following:

Selective observation. She observed the cats at the outside door only once. Overgeneralizing She observed only two cats, both of which previously were confined indoors. Yet she drew a conclusion about cats in general. Illogical reasoning. She assumed that others feel guilty about keeping their cats indoors and that cats are motivated by feelings about opportunities to play. Resistance to change. She was quick to conclude that she had no need to change her approach to the cats.

If you recognize these errors for what they are and make a conscious effort to avoid them, you can improve your own reasoning. You will guard against stereotyping people, avoid jumping to conclusions, and look at the big picture. These are errors in observing, generalizing, reasoning, and reevaluating that the methods of educational research help us avoid.

Observing

One common observing mistake is selective observation—choosing to look only at things that are in line with our preferences or beliefs. When we are inclined to criticize individuals or institutions, it is all too easy to notice their every failing. For example, if we are convinced in advance that all television viewing by children is harmful, we can find many confirming instances. But what about educational programs such as Sesame Street and educational software games that use television to teach basic concepts in language and arithmetic? If we acknowledge only the instances that confirm our predispositions, we are victims of our own selective observation.

Our observations can also simply be inaccurate. If a child says she is “hungry” and we think she said she is

20

“hunted,” we have made an inaccurate observation. If we think five students are standing in a hallway when seven actually are, we have made an inaccurate observation.

Such errors occur often in casual conversation and in everyday observation of the world around us. In fact, our perceptions do not provide a direct window onto the world around us, for what we think we have sensed is not necessarily what we have seen (or heard, smelled, felt, or tasted). Even when our senses are functioning fully, our minds have to interpret what we have sensed (Humphrey, 1992). The optical illusion in Exhibit 1.2, which can be viewed as either two faces or a vase, should help you realize that perceptions involve interpretations. Different observers may perceive the same situation differently because they interpret it differently.

Exhibit 1.2 An Optical Illusion

Generalizing

Overgeneralization occurs when we conclude that what we have observed or what we know to be true for some cases is true for most or all cases. We are always drawing conclusions from our experience, but sometimes we forget that our experiences are limited. The educational world is, after all, a complex place. We have the ability to interact with just a small fraction of the individuals who inhabit the educational world, especially in a limited span of time. Some people feel that television watching can't hurt young children because it exposes them to ideas and information they would never have otherwise encountered. Would their experience generalize to yours? To others?

Exhibit 1.3 depicts the difference between selective observation, which we have already discussed, and

21

overgeneralization.

Exhibit 1.3 The Difference between Selective Observation and Overgeneralization

Reasoning

When we prematurely jump to conclusions or argue on the basis of invalid assumptions, we are using illogical reasoning. For example, it is not reasonable to propose that children who watch no television will have no attention problems in school because there are sources of attention problems other than television. On the other hand, an unquestioned assumption that every child who watches television will have some attention problems overlooks some important considerations, such the type of programs that are being watched and the amount of parent interaction around the child's TV viewing. Logic that seems impeccable to one person can seem twisted to another—the problem usually is reasoning from different assumptions rather than just failing to “think straight.”

22

Reevaluating

Resistance to change, the reluctance to reevaluate our ideas in light of new information, may occur for several reasons:

Ego-based Commitments

We all learn to greet with some skepticism the claims by leaders of companies, schools, agencies, and so on that people in their organization are happy, that revenues are growing, and that services are being delivered in the best possible way. We know how tempting it is to make statements about education that conform to our own needs rather than to the observable facts. It can also be difficult to admit that we were wrong once we have staked out a position on an issue.

Excessive Devotion to Tradition

Some degree of devotion to tradition is necessary. Learning both in and out of school can be traditional in many ways, both open and hidden. Some skepticism about the effects of television on children can be a healthy antidote to panic about media's effects, which neither parents nor teachers can control. But too much devotion to tradition can stifle adaptation to changing circumstances. When we distort our observations or alter our reasoning so that we can maintain beliefs that “were good enough for my grandfather, so they're good enough for me,” we hinder our ability to accept new findings and develop new knowledge.

Uncritical Agreement with Authority

If we do not have the courage to evaluate critically the ideas of those in positions of authority, we will have little basis for complaint if they exercise their authority over us in ways we don't like. And if we do not allow new discoveries to call our beliefs into question, our understanding of the educational world will remain limited.

Now take just a minute to reexamine the opinions about learning that you recorded earlier. Did you grasp at a simple explanation even though reality is far more complex? Were your beliefs influenced by your own ego and a tendency to overgeneralize your own learning experiences? Did you weigh carefully both the positive and negative effects that television has on learning? Do you see some of the challenges faced by educational researchers?

23

The Educational Research Approach

Educational research is designed to reduce potential sources of error in reasoning about the educational world. Educational research builds on the methods of science, so it relies on logical and systematic methods to answer questions, and it does so in a way that allows others to inspect and evaluate its methods. In the realm of educational research, these methods are not so unusual. After all, they involve asking questions, observing social groups, and counting people, which we often do in our everyday lives. However, educational researchers develop, refine, apply, and report their understanding of the educational world more systematically, or “scientifically,” than the general public does.

24

Science and Educational Research

What is “scientific” about educational research methods? Consider this:

Science: A set of logical, systematic, documented methods for investigating nature and natural processes; the knowledge produced by these investigations.

Educational research: The use of scientific methods to investigate teaching and learning, both inside and outside school—the educational world; the knowledge produced by these investigations.

Educational research methods reduce the likelihood of overgeneralization by using systematic procedures for selecting individuals or groups to study that are representative of the individuals or groups to which we wish to generalize. Educational researchers use explicit criteria for identifying causes and for determining whether these criteria are met in a particular instance to avoid illogical reasoning. Educational research methods reduce the risk of selective or inaccurate observation by requiring that we measure and sample phenomena systematically. Because they require that we base our beliefs on evidence that can be examined and critiqued by others, educational research methods lessen the tendency to develop answers about the educational world from ego-based commitments, excessive devotion to tradition, and/or unquestioning respect for authority.

25

Motives for Educational Research

Research begins with questions—in this chapter, questions about television viewing and early childhood development. What motivates selection of a research question or focus? Usually it's one or more of the following reasons:

Personal Motivations

Some educational researchers who explore early childhood learning feel that by doing so, they can help to improve the lives of children, the effectiveness of schooling, or the conditions of disadvantaged groups in their communities or countries. Educational researchers may become interested in early childhood as a result of watching their own children or grandchildren or after teaching or working with young children. A teacher- researcher who teaches second graders might want to find out how much television the children in her class watch and what kinds of programs they spend the most time on. Many research questions spring from the researcher's own life, experiences, and values.

Academic Motivations

Academic questions about influences on educational processes have stimulated much educational research. Early childhood researchers want to understand the strength and meaning of family and other outside influences on the school performance of young students. Do lower income, family disintegration, or other factors that often lead to increased day care mean that some children enter school at a large disadvantage? Can public interventions in the early years help to equalize this disadvantage? The desire to gain a better understanding of questions such as these is motivation enough for many educational researchers.

Policy Motivations

Many government agencies, elected officials, and private organizations seek better descriptions of the effects of television so they can advocate for policies that use television for the public good. School officials may need information for planning curriculum, teaching approaches, and media awareness programs. Parent groups may want to inform their members to limit television viewing in the best interests of their children. These policy and program management needs can stimulate numerous research projects.

Quantitative methods: Methods such as surveys and experiments that record variation in educational life in terms of categories that vary in amount. Data that are treated as quantitative are either numbers or attributes that can be ordered in terms of magnitude.

Qualitative methods: Methods such as participant observation, intensive interviewing, and focus groups that are designed to capture educational reality as participants experience it, rather than in categories predetermined by the researcher. Data that are treated as qualitative are mostly written or spoken words or observations that do not have a direct numerical interpretation.

26

Quantitative and Qualitative Orientations

One of the most common divisions in educational research is the distinction between quantitative research and qualitative research. Both quantitative and qualitative researchers collect data and then use them to tell a meaningful story—data analysis and findings, in research terms—but the data they collect and the methods they use to analyze them differ substantially. Here's a very basic way of understanding the difference. Quantitative researchers collect numbers and quantities as basic data and employ a whole array of statistical procedures to analyze those data. Qualitative researchers record words, pictures, or video as data and identify patterns and themes in those data that result in narrative interpretations that create meaning.

The distinction between quantitative and qualitative data is not always sharp. Qualitative data can be converted to quantitative data, when we count the frequency of particular words or phrases in a text or measure the time elapsed between different observed behaviors. Surveys that collect primarily quantitative data may also include questions asking for written responses, and these responses may be used in a qualitative, textual analysis. Qualitative researchers may test explicit explanations of educational phenomena using textual or observational data. We'll examine such “mixed method” possibilities in Chapter 11.

Triangulation: The use of multiple methods to study one research question.

Educational researchers often combine these methods to enrich their research. The use of multiple methods to study one research question is called triangulation. The term suggests that a researcher can get a clearer picture of the educational situation being studied by viewing it from several different perspectives. Each will have some liabilities in a specific research application, and all can benefit from combination with one or more other methods (Brewer & Hunter, 1989; Sechrest & Sidani, 1995).

The distinction between quantitative and qualitative methods involves more than just the type of data collected. Quantitative methods are most often used when the motives for research are explanation, description, or evaluation. Exploration is most often the motive for using qualitative methods, although researchers also use these methods for descriptive, explanatory, and evaluative purposes. Chapters 9 and 14 present qualitative methods in much more detail, and most other chapters include some comparison of quantitative and qualitative approaches. The next section discusses four primary types of educational research and their relationship to qualitative and quantitative orientations.

27

Types of Educational Research

There are four types of educational research projects. This section illustrates each type with examples from educational research about early childhood:

Descriptive Research

Defining and describing education-related phenomena is a part of almost any research investigation, but descriptive research is often the primary focus of the first research about some issue. One of the early descriptive questions researchers asked about the extent of television viewing by young children was, “What patterns of viewing do children have in early childhood and how do they differ from the viewing habits of older children?” (Huston et al., 1990). Measurement (the topic of Chapter 4) and sampling (Chapter 5) are central concerns in descriptive research. Survey research (Chapter 8) is often used for descriptive purposes.

Exploratory Research

Exploratory research seeks to find out how people get along in the setting under question, what meanings they give to their actions, and what issues concern them. The goal is to learn “What is going on here?” and to investigate educational phenomena without explicit expectations. This purpose is associated with the use of methods that capture large amounts of relatively unstructured information. For example, researchers investigating young children's learning have had to look closely at the educational effects of social context and peer interaction. Exploratory research frequently involves qualitative methods.

Explanatory Research

Many consider explanation the premier goal of any science. Explanatory research seeks to identify causes and effects of educational phenomena, to predict how one phenomenon will change or vary in response to variation in some other phenomenon. Early childhood researchers adopted explanation as a goal when they began to focus on factors that influence children's development and behavior. Their explanatory questions have included “Is maternal employment in the first year of life associated with negative child outcomes in the first three years of life?” and “Are these effects … mediated by the quality of child care or the home environment?” (Brooks-Gunn, Han, & Waldfogel, 2002, p. 1052). We focus on ways of identifying causal effects in Chapter 6. Explanatory research often involves experiments or surveys (see Chapter 8), both of which are most likely to use quantitative methods.

Evaluation Research

Seeking to determine the effects of an educational program or other type of intervention is a type of explanatory research because it deals with cause and effect (see Chapter 7). However, evaluation research differs from other forms of explanatory research because evaluation research considers the implementation and effects of educational policies and programs. These issues may not be relevant in other types of explanatory research. For example, concern over the impact of Early Head Start, an extensive federal early childhood program, provided the impetus for a major federal evaluation study that involved more than 3,000 families

28

(U.S. Department of Health and Human Services, 2002).

Certain types of research are often used with certain research methods because those methods yield a kind of data that are helpful for answering that type of question. Exhibit 1.4 shows some common correspondences between type of research, the goal of the research, and methods and techniques that match. The correspondence between types and methods should not be regarded as a hard-and-fast rule—it's possible to find exploratory research that uses some quantitative techniques, for example—but the chart gives a handy way of thinking about relationships between types and methods.

We'll now summarize four actual early childhood research projects that exemplify these four types of research.

Description: What Types of TV Programs do Young Children Watch?

In the 1980s, when this research was conducted, researchers knew that children watched a lot of television but knew little about what types of programs they were actually watching. A team of researchers followed 326 children for 2 years to determine how much television and what kind of programming the children were watching. The children were in two age groups, 3 to 5 and 5 to 7 years old.

The researchers found that the children watched, on average, 2 to 3 hours of television per day. Children's viewing changed with age, and boys tended to watch more cartoon and action programs than girls. Boys were also more interested in adult informational programs. As they grew older, children of both sexes began watching more comedies meant for general audiences. As the children matured, they moved from informative programs aimed at children to entertainment programs for a general audience (Huston et al., 1990).

Exploration: How does Classroom Social Context Affect a Disabled Preschool Child?

Elizabeth Erwin, Elizabeth Alimaras, and Nikki Price (1999) knew that research going back to the 1930s shows the importance of social context and peer interaction to learning. They also knew that there was little

29

research on peer interactions in early childhood settings that included children with disabilities. They designed a small pilot study of socialization experiences in a preschool class that included Ryan, a 3-year-old with identified disabilities (Ryan had detached retinas in both eyes with no light perception in his right eye and a visual acuity of 20/600 in his left eye, no cognitive impairments, but other medical complications, including kidney disease). To conduct the study,

qualitative methods (i.e. participant observations and semi-structured personal interviews) were used … in an effort to provide a rich portrait of events, experiences, and perspectives. Data were gathered across daily classroom routines and natural settings within the school such as the music room, playground, hallways, and classrooms. (p. 57)

Observations began in September and were conducted approximately once a month until school closed in June. Observational data were supplemented by 1-hour personal interviews with the classroom teacher, the teacher assistant, and Ryan's mother and father. Because this was an exploratory study, the findings included narrative descriptions of the types of interactions that occurred between Ryan and other children. Overall, the authors felt that their exploratory study, rich in description, resulted in more questions than answers, and they urged other researchers to further explore the skills and knowledge practitioners needed to successfully support disabled students. This type of conclusion, which points the way for further research, is not uncommon in exploratory studies, which are typically undertaken when a question or research area is just starting to be investigated.

Explanation: When Mothers Work during the Child's First Year, Do Children Suffer Cognitively?

One of the large changes in our society over the past 40 years has been the increasing number of mothers who work outside the home. Questions have been raised about whether this helps or harms young children as they prepare to begin formal schooling. Researchers at the U.S. National Institute of Child Health and Human Development (Brooks-Gunn et al., 2002) tried to find out if maternal employment during a child's first year had positive or negative cognitive effects later, when the child was almost ready to go to school. They used quantitative methods to analyze a large body of information (called a “data set”) that the agency had already collected.

The researchers studied 900 European American children. The number of hours that mothers worked was compared to their children's scores on a standard test of cognitive skills. They concluded that

maternal employment by the ninth month was found to be linked to lower Bracken School Readiness Scores at 36 months, with the effects more pronounced when mothers were working 30 hours or more per week and with effects more pronounced for certain subgroups (i.e., children whose mothers were not sensitive, boys, and children with married parents). (Brooks-Gunn et al., 2002, p. 1052)

30

The researchers were careful to note limitations of their study. These included the fact that they had studied only one population, European Americans, and the idea that factors other than working also made a difference. These factors included “quality of child care, home environment, and maternal sensitivity.” However, even when these additional factors were taken into account, “the negative effects of working 30 or more hours per week in the first 9 months were still found” (Brooks-Gunn et al., 2002, p. 1052).

The researchers also made policy recommendations based on their findings, which is not unusual in explanatory research in education. They recommended, for instance, that “it would be prudent for policy makers to go slow on measures … that would require mothers to enter the labor force (full-time) early in the first year of life and to consider measures (such as the proposed Family and Maternal Leave Act extensions) that would allow more mothers to choose to delay their return to the labor force and/or to work part-time until late in the first year of life” (p. 1068).

Evaluation: Is the Early Head Start Program Working?

Because large- and small-scale programs are a common feature of schools, many educational research studies try to measure program effectiveness. In 1995, Congress created Early Head Start, which was the only federal program specifically designed to improve the early education experiences of low-income babies and toddlers. In August 2002, the federal government published the first full evaluation study of the program, which aims to improve infants’ and toddlers’ later school success by supporting prenatal health and improving children's cognitive, social, and emotional development (U.S. Department of Health and Human Services, 2002).

The researchers used a large, randomly assigned sample—more than 3,000 families—in 17 sites across the country and a wide range of evaluation measures of cognitive, language, and social-emotional development. Evaluation results showed that the program was working across the whole range of indicators. These included cognitive functioning, interaction with parents, movement toward self-sufficiency, child-father interactions, and greater school readiness (U.S. Department of Health and Human Services, 2002).

Exhibit 1.5 shows some of the indicators used in the Early Head Start study to measure parents’ knowledge of and involvement with their children. The last three items refer to a hypothetical scenario presented to parents in which their child's behavior required parental intervention. Unfortunately, one of the other findings of the evaluation was that the program currently reached only 3% of those eligible to receive services (Stark, 2003).

Source: U.S. Department of Health and Human Services (2002, p. 16).

31

32

Educational Research Philosophies

Different educational researchers are guided by different research philosophies. A philosophy, in this case, means a viewpoint on what constitutes educational reality. Naturally, how you think about reality has implications for what methods you use to investigate that reality. In this section, we will describe and explain two alternative research philosophies that are prevalent in educational research, positivism and interpretivism. Positivism—and its descendent postpositivism—is more closely linked to quantitative research approaches. Interpretivism—and its descendent constructivism—is more closely linked to qualitative approaches.

33

Positivism and Postpositivism

Researchers with a positivist philosophy believe that an objective reality exists apart from the perceptions of those who observe it and that the goal of research is to better understand this reality. This is the philosophy traditionally associated with natural science (e.g., biology, chemistry, physics), with the expectation that there are universal laws of human behavior and with the belief that scientists must be objective and unbiased to see reality clearly (Weber, 1949, p. 72). Positivists believe that a well-designed test of a theoretically based prediction can move us closer to understanding actual educational processes.

Positivism: The belief, shared by most scientists, that there is a reality that exists quite apart from our own perception of it, that it can be understood through observation, and that it follows general laws.

Postpositivism: The belief that there is an empirical reality, but that our understanding of it is limited by its complexity and by the biases and other limitations of researchers.

Intersubjective agreement: An agreement by different observers on what is happening in the natural or educational world.

The philosophy of postpositivism is closely related to positivism. Postpositivists believe that there is an external, objective reality, but they are very sensitive to the complexity of this reality and to the limitations and biases of the scientists who study it (Guba & Lincoln, 1994, pp. 109–111). As a result, they do not think we can ever be sure that scientific methods allow us to perceive objective reality. Instead, postpositivists believe that the goal of science is to achieve intersubjective agreement among scientists about the nature of reality (Wallace, 1983, p. 461). For example, postpositivists may worry that researchers’ predispositions bias them in favor of a certain theory. Therefore, they remain skeptical of research results that support that theory until a number of researchers report such evidence. A postpositivist has much more confidence in the community of researchers than in any individual researcher (Campbell & Russo, 1999, p. 144).

34

Interpretivism and Constructivism

Qualitative educational research is often guided by a different, interpretivist philosophy. Interpretivist approaches have become increasingly influential in educational research, and their growing prevalence since the 1990s, sometimes referred to as the “interpretive turn,” has changed the educational research landscape (Howe, 1998). Interpretivist researchers believe that educational reality is socially constructed and that the goal of educational research is to understand what meanings people give to reality, not to determine how reality works apart from these interpretations. This philosophy rejects the positivist belief that there is a concrete, objective reality that scientific methods help us to understand (M. Lynch & Bogen, 1997). Instead, interpretivists believe that people construct an image of reality based on their own preferences and prejudices and their interactions with others and that this is as true of scientists as it is of everyone else. This means that we can never be sure that we have understood reality properly, that “objects and events are understood by different people differently, and those perceptions are the reality—or realities—that social science should focus on” (Rubin & Rubin, 1995, p. 35).

Interpretivism: The belief that reality is socially constructed and that the goal of social scientists is to understand what meanings people give to that reality.

Constructivist paradigm: A perspective that emphasizes how different stakeholders in educational settings construct their beliefs.

The constructivist paradigm extends interpretivist philosophy by emphasizing the importance of exploring how different stakeholders in a social setting construct their beliefs (Guba & Lincoln, 1989, pp. 44–45). It gives particular attention to the different goals of researchers and other participants in a research setting and seeks to develop a consensus among participants about how to understand the focus of inquiry. From this standpoint, “Truth is a matter of the best-informed and most sophisticated construction on which there is consensus at a given time” (Schwandt, 1994, p. 128).

35

Taking a Balanced Approach

It is tempting to think of positivism as representing an opposing research philosophy to interpretivism and constructivism. Then it seems that we should choose the one philosophy that seems closest to our own preferences and condemn the other as “unscientific,” “uncaring,” or perhaps just “unrealistic.” But there are good reasons to prefer a research philosophy that integrates some of the differences between these philosophies (J. Smith, 1991). Researchers influenced by a positivist philosophy should be careful to consider how their research approaches and interpretations are shaped by their own social and educational background —just as we are cautioned to do by interpretivist researchers. Researchers influenced more by an interpretivist philosophy should be careful to ensure that they use rigorous procedures to check the trustworthiness of their interpretations of data (Reissman, 2008, pp. 185–199). If we are not willing to ask “hard questions” about our projects and the evidence we collect, we are not ready to investigate the educational world (Reissman, 2008, p. 200). The educational phenomena we study are often complex, and we must take this complexity into account when we choose our methods and interpret our results.

But even in areas of research that are fraught with controversy, the quest for new and more sophisticated research has value. What is most important for improving understanding of the educational world is not the result of any particular study but the accumulation of evidence from different studies of related issues. By designing new studies that focus on the weak points or controversial conclusions of prior research, educational researchers contribute to a body of findings that gradually expands our knowledge about the educational world and resolves some of the disagreements about it.

36

Conclusions

We began this chapter with the question, “How do early childhood experiences affect schooling?” Throughout the chapter, you saw examples of ways in which researchers approached this topic, including looking at the effects of television on attention span and aggressive behavior; the effects of social dynamics in a preschool classroom with 14 students, one of whom was severely disabled; the possible cognitive effects on very young children when mothers work; and the evaluation of Early Head Start, a federal preschool program. Each study examined the question from a different perspective, used different methods, and reached different conclusions. Taken together, though, they begin to build a research-based answer to the original question. They show, in a small way, how the aggregation of individual studies can eventually help us to attain big answers.

We hope this first chapter has given you an idea of what to expect in the rest of the book. Our aim is to introduce you to educational research methods by describing what educational researchers have learned about the educational world as well as how they have learned it. The substance of educational research inevitably is more interesting than its methods, but the methods become more interesting when they're linked to substantive investigations.

This book's first six chapters deal with Foundations of Research. We have focused attention on early childhood research in this chapter; in Chapter 2, we use studies of reading instruction to illustrate the research process. Chapter 2 outlines the research process as a whole and also presents specific techniques for becoming a savvy reader and user of educational research. We also introduce the process of writing research proposals, which is then continued in special end-of-chapter exercises throughout the book. Chapter 3 explains research ethics, particularly the importance of the ethical treatment of human subjects.

To complete our overview of the foundations of research, in Chapters 4, 5, and 6, we introduce a variety of measurement approaches, different ways of sampling larger populations, and specific techniques to maximize the validity of causal assertions. Methods of designing research and collecting data are the focus of the book's second section (Chapters 7–12). Evaluation research and educational assessment, the subjects of Chapter 7, are conducted to identify the impact of educational programs or to clarify educational processes involving such programs. Experimental methods are often part of evaluation research, and so they are discussed in this chapter. Survey research (Chapter 8) can be used to collect data from a large population of students, teachers, or community members.

Chapter 9 shows how qualitative methods can uncover aspects of the educational world that we are likely to miss in experiments and surveys and sometimes result in a different perspective on educational processes.

Chapters 10, 11, and 12 introduce data collection approaches that can involve several methods. Chapter 10 focuses on single subject designs, which can be extremely useful in investigations in the classroom and in counseling, where a single student can be studied. Chapter 11 covers mixed methods. As you might suspect, mixed-method approaches combine two or more of the other methods. Chapter 12 gives special attention to methods of inquiry—teacher research and action research—that emphasize the concerns and insights of

37

practitioners—classroom teachers and others directly involved in education.

The third section of this book, comprising the final three chapters, deals with Analyzing and Reporting Data. Chapter 13 is not a substitute for an entire course in statistics, but it gives you a basic idea of how to use statistics when analyzing research data and reporting or reviewing research results. In Chapter 14, we examine in some detail the logic and procedures of qualitative data analysis. You will be struck by the differences between qualitative data analysis techniques and the quantitative data analysis techniques of Chapter 13. Chapter 15 deals with reporting research; our research efforts are really only as good as the attention we give to our research reports. In Chapter 15, we finish the discussion of research proposals started in Chapter 2.

Each chapter ends with several helpful learning tools. Lists of key terms and chapter highlights will help you to review. Discussion questions and practice exercises will help you to apply and deepen your knowledge. Special exercises guide you in developing your first research proposal and finding information on the World Wide Web.

38

Key Terms

Constructivism 15 Descriptive research 11 Evaluation research 12 Explanatory research 11 Exploratory research 11 Illogical reasoning 8 Inaccurate observations 6 Measurement 11 Qualitative methods 10 Quantitative methods 10 Positivism 15 Postpositivism 15 Resistance to change 8 Science 9 Selective observation 6 Triangulation 11

39

Highlights

Educational research cannot resolve value questions or provide permanent, universally accepted answers. Empirical data are obtained in educational research investigations from either direct experience or others’ statements. Four common errors in reasoning are overgeneralization, selective or inaccurate observation, illogical reasoning, and resistance to change. These errors result from the complexity of the educational world, subjective processes that affect the reasoning of researchers and those they study, researchers’ self- interestedness, and unquestioning acceptance of tradition or of those in positions of authority. Educational research is the use of logical, systematic, documented methods to investigate individuals, processes, contents, and educational systems, as well as the knowledge produced by these investigations. Educational research can be motivated by personal preferences, academic issues, and policy concerns. Educational research can be descriptive, exploratory, explanatory, or evaluative—or some combination of these. Quantitative and qualitative methods structure research in different ways and are differentially appropriate for diverse research situations. It is possible to mix qualitative and quantitative methods to gain accurate knowledge of particular questions.

40

Student Study Site

To assist in completing the web exercises, please access the study site at www.sagepub.com/check, where you will find the web exercise with accompanying links. You'll find other useful study materials such as self- quizzes and e-flashcards for each chapter, along with a group of carefully selected articles from research journals that illustrate the major concepts and techniques.

41

http://www.sagepub.com/check
Discussion Questions

1. Select an educational issue that interests you, such as television watching or charter schools. List at least four of your beliefs about this phenomenon. Try to identify the sources of each of these beliefs.

2. Find a report of an educational research in an article in a daily newspaper. What were the major findings? How much evidence is given about the methods the researcher used? What additional design features might have helped to improve the study's validity?

42