Meaning of evaluation in science

Scientific Merit Evaluation Paper

INTRODUCTION

Throughout this course, you have learned about several research approaches in both qualitative and

quantitative methodology. Even though there are many ways in which researchers can solve research problems

with different research approaches and designs, there is one thing that all good scientific research has in

common: scientific merit.

At the beginning of the course, you were introduced to the concept of scientific merit. You learned that in order

for research to have scientific merit, it must achieve three things:

• It must contribute something new to the scientific knowledge base in the field. In order for research to

do this, a researcher must conduct a thorough literature review to identify a research problem. A

research problem is something that has not been addressed by previous research, and would be

meaningful to know. Often, the research problem is referred to as a gap in the literature.

• It must contribute to theory. Theories are the primary tools by which science organizes its

knowledge—the tools that allow the field to explain previous and current findings and predict future

ones. Research that has scientific merit can advance theories in several ways, including confirming,

extending, refuting, or proposing theories.

• It must meet the hallmarks of good research. This is the component that has been most emphasized in

this course as you have learned about the defining characteristics of the two methodologies and the

various approaches of both. But a researcher who has conducted high-quality research has only met this

criterion of scientific merit to the extent that the research design and procedures allowed the research

problem to be solved and the research questions to be answered. It is crucial to remember that the

purpose of research is to solve a research problem by investigating something that has not been

previously researched; that is, to fill the gap in the literature. Research methods are simply tools that

researchers use to do this. When researchers effectively use research methods to solve a research

problem, then we can say that the research has met the hallmarks of good research.

As a professional, you will be called upon to use research findings in the psychological literature. In order to do

this, you will be responsible for evaluating the scientific merit of the research that produced those findings.

While most research published in peer-reviewed journals can be trusted to have good scientific merit, it can be

risky not to evaluate research for yourself, especially research findings most pertinent to your work and career.

This course has given you the opportunity to learn how to do this for yourself, should you ever plan to do your

own research in the future. It has provided you with the necessary background regarding how to plan research

that has good scientific merit.

OBJECTIVES

To successfully complete this learning unit, you will be expected to:

1. Discuss hallmarks of good research.

2. Distinguish three dimensions of scientific merit.

3. Apply principles of scientific merit to chosen research study.

Scientific Merit - STUDIES

Readings

Leedy and Ormrod text to complete the following:

• Review the "What Is Research" section of Chapter 1, "The Nature and Tools of Research," pages 3–7.

This review will put the details of research methods you have learned in the class into a larger

Perspectives

Transcript

• Review the Scientific Merit Presentation transcript piece. This will refresh your knowledge on

scientific merit to help you address this unit's assignment.

SCIENTIFIC MERIT PAPER

Resources

Scientific Merit Paper Scoring Guide.

• Evaluates the scientific merit of the article by analyzing how the research advances the scientific knowledge base.

• Evaluates the scientific merit of the article by analyzing how the research contributes to research theory and the

field of study.

• Evaluates the scientific merit of the article by analyzing the scientific methods.

• Analyzes the validity and reliability of a selected research study and elaborates on how it contributes to scientific

merit.

• Analyzes the effectiveness of strategies selected for addressing ethical concerns in the design of a research study.

• Communicates in a manner that is completely scholarly, professional, and consistent with expectations for

members of the identified field of study, and uses APA style and formatting with few or no errors.

APA Style and Format.

Research Library.

Persistent Links and DOIs.

Using the research study you selected and attached, along with the course assignments for which

you deconstructed your research study, write a scientific merit paper. In your paper, address the following:

• Evaluate how a research study advances the scientific knowledge base of an identified field of study.

• Evaluate how a specific research study contributes to research theory and the field of study.

• Evaluate how the scientific methods of inquiry are applied to a specific research study.

• Evaluate the validity and reliability of a selected research study.

• Assess strategies for addressing ethical concerns in the design of a research study.

• Apply the terminology of research.

• Demonstrate both the depth and the breadth of your understanding of scientific research and

scientific merit by including relevant examples and supporting evidence.

Requirements

scientific merit by including relevant examples and supporting evidence.

Requirements

• The paper should be 10–12 pages long, not counting references, block quotes, or title page. Neither

an abstract nor table of content is required, and would not count in the paper length.

• Minimum of seven scholarly resources. It is expected that one of these resources will be the study you

selected for the Unit 2 assignment, and another will be the required textbook for the course, by

Leedy and Ormrod.

• Use APA style and formatting, including correct in-text citations, proper punctuation, double-spacing

throughout, proper headings and subheadings, no skipped lines before headings and subheadings,

proper paragraph and block indentation, no bolding, and no bullets.

• Include the permalink to your article on the title page of your paper.

Refer to the project description and the assignment scoring guide to make sure you meet the requirements

of this assignment.

SCIENTIFIC MERIT

INTRODUCTION Scientific merit is the degree of quality of a research study. When the scientific merit of a research study is high, this means that the research has contributed valuable, meaningful, and valid information to a scientific discipline. This presentation will explain the basic components of scientific merit so that you so that you can evaluate the work of others as well as your own.

WHAT IS SCIENTIFIC MERIT There are three dimensions to scientific merit, and each will be discussed in turn. The three dimensions include advancing the current knowledge base, contributing to theory, and meeting the hallmarks of good research.

ADVANCE KNOWLEDGE BASE The first requirement for research to have scientific merit, is that is must advance the knowledge base in the field of study. That is to say that for us to value the research, it must contribute new knowledge to one's discipline, such as psychology. In order to determine whether research advances the knowledge base, we should ask, "Does the study address something that is not known or has not been studied before?" If the study only replicates previous findings, then it does not advance the knowledge base. In order to know whether or not research addresses something that has been studied before, you need to read and have an understanding of the literature, meaning the primary research articles on the topic. Knowledge represented in a literature review tells us what IS known. Once we understand what IS known, then we are in a position to see what is NOT known. The gap in the literature is also called the research problem. Research that is published in journal articles will present a pertinent literature review in the Introduction section of the article so that the reader can understand the current knowledge in the field.

We should also ask, "How does the research study make new contributions to the empirical knowledge base about the specific topic?" The manner in which information is gained in order for it to contribute to the knowledge base is important. Research in psychology must be empirical, which means the data collected is based upon observations that can be made and measured by anyone under the same conditions. We should also ask, "How is the research study extending the information to what is known in the literature already?" That is, we need to know how the research study connects to, relates to, or fits in, with other research findings. By extending information, we increase our depth of understanding about a phenomenon.

To evaluate how well the research advances the knowledge base, we might ask questions in layman's terms such as: So what? Why is this research important? Who cares? Seriously, who is interested in the results of the study? Who is the intended audience, and how might they use the results? What change or new understanding do the findings bring? What does this study offer that is new and fresh? As an analogy, think of research findings as individual pieces of a jigsaw puzzle, how and where will this research finding—this piece—fit in? Will it help bridge a large gap, a small one? Is it a piece that will pull together several areas of the

puzzle? Is it a piece that might go in a certain area of the puzzle, but not really connect to anything that's already there? As the French scientific philosopher Henri Poincare noted, "Science is facts; just as houses are made of stones, so is science made of facts; but a pile of stones is not a house and a collection of facts is not necessarily science." Therefore, part of understanding how well a research finding advances knowledge in the field depends on how it fits in with other findings.

CONTRIBUTION TO THEORY Theories are the foundation of a researcher's toolkit. Theories provide the theoretical framework that underlies research and organize the observations that have been made regarding a particular phenomenon. We all have theories. For example, you might have a theory about effective parenting. Whether you are a parent or not, you have made observations of people parenting their children, seen movies or TV shows of the same, heard about others' experience, and read books. If you are a parent, you have direct experience with parenting. As we make observations, we think about what is being observed. We organize our thoughts in such a way that beliefs, attitudes, and opinions are formed about phenomenon. These beliefs, attitudes and opinions that are formed from our thoughts about observations can be used to formulate theories about how and why things occur. Researchers use theories as the basis of their research studies; testing, advancing, extending, and even creating new theories that provide insight into a particular phenomenon.

The second dimension used to evaluate scientific merit is the degree to which the research contributes to theory. There are various ways in which research can do this. We might ask if the research provides evidence that generates a brand new theory. We might also ask, does the research finding refine or add to an existing theory? Theories are never perfect, and they are always open to verification. So we might also ask, does the research test to confirm or refute current theory? And last, we could ask, "Does the research expand theory by telling us something new about how it can be applies?" Some theories lend themselves well to practical applications, such as therapy or education. Research might contribute to theory by showing how the theory might be applied in a certain context.

HALLMARKS OF GOOD RESEARCH Like most other human endeavors, research can be well done, or poorly done. Obviously, research that is done well has more scientific merit than research that is done poorly. But how can we evaluate whether research is good? Simply put, research is good when it is based upon sound methodology. Specifically, we can look at whether the research meets certain hallmarks of good research. The first question is, "Can the study answer the research questions with the design and method proposed?" That is to say, the research Questions must address the research problem (the gap in the literature). Furthermore, the research questions must help solve the research problem. The methodology should logically follow the research questions. You might think of the research questions as a direct bridge between the research problem, the gap in the literature, and the actual research study.

The second question is, "Are the research questions aligned with the selection of the design, method, instruments/measures, and data analysis?" In other words, does everything match up? Do the research questions logically correspond to the design and method? Are the instruments used to collect the data appropriate to the methodology and design, and do the instruments generate data that can be used to answer the research questions? Is the data analysis appropriate? Will the data analysis give the researcher an answer to the research question? Notice that what makes research good is not the results that were generated by the study but how the study was conducted, more specifically the methodology, and how well the methodology answers the research questions, which in turn helps solve the research problem.

The third question asks, "Is the study ethically sound?" All research must meet the ethical standards put in place by a researcher's Institutional Review Board, or IRB, guidelines and regulation required by the American Psychological Association, and federal law. As part of your training as a Capella graduate student, you are required to undergo special training through the CITI program to help ensure you understand the ethical standards required when performing research. How is scientific merit related to ethics?

SCIENTIFIC MERIT AND ETHICS Besides the requirements regarding the ethical treatment of research participants, such as giving informed consent, there is the additional requirement that the research be scientifically valid. If research does not advance the knowledge base, does not contribute to theory, and does not meet the hallmark of utilizing sound scientific methodology, then it is unethical to do the research. Stated another way, it is unethical to do bad or unnecessary research. As Pilkington wrote in 2002, "If a study does not hold substantial promise of answering a significant question, thereby generating valuable knowledge, then there is no justification for exposing persons to the actual or potential risks and inconvenience of participation."

SUMMARY To summarize, there are the three dimensions of scientific merit. First, the research must advance the knowledge base of the field by contributing new, meaningful information that connects to other findings in the field. Second, the research must make a contribution to theory by providing research evidence that forms the basis of a new theory, or adds to, extends, tests, or demonstrates the applicability of current theory. Third, research must meet certain hallmarks of good research. The hallmarks include the degree to which the research methods are appropriate for answering the research question, and whether the research is ethically sound.

Whether you plan to do your own research or not in the future, it is imperative that as a professional in psychology, you understand as much as possible about the research process. As a professional, you will make use of research findings as part of your work. In order to use them correctly, it is your responsibility to learn about the processes involved in conducting research and how to evaluate scientific merit. You are encouraged to visit these Capella sites to enhance your understanding of research.

• Learn about doing research at Capella: https://campus.capella.edu/web/library/library-research-skills

• Learn about doing research in your school: https://campus.capella.edu/web/dissertation-research/research-in-your-school

• Learn about the Capella Research Center and research ethics: https://campus.capella.edu/web/library/more-services-and-tools

CREDITS Subject Matter Expert: Interactive Designer:

Instructional Designer:

Project Manager:

Tsuey-Hwa Chen, Carlos Contreras, Rosanne Roberts

Jay Austin, Marc Ashmore

Laura Badaracco Amend

Alan Campbell