Lincoln and guba 1985 trustworthiness of the study

Linda Amankwaa, PhD, RN, FAAN

Abstract: Experienced and novice researchers, plan qualitative proposals where evidence o f rigor m ust be provided within the document. One option is the creation o f a trustworthiness protocol with details noting the characteristic o f rigor, the process used to document the rigor, and then a timeline directing the planned time for conducting trustworthiness activities. After reviewing several documents, an actual plan o f conducting trustworthiness as not found. Thus, these authors set out to create a trustworthiness protocol designed not only for the dissertation, but a framework for others who m ust create similar trustworthiness protocols for their research. The purpose o f this article is to provide a reference for the trustworthiness plan, a dissertation example and showcase a trustworthiness protocol that may be used as an example to other qualitative researchers embarking on the creation o f a trustworthiness protocol that is concrete and clear.

K ey Words: Trustworthiness, Research Protocols, Qualitative Research

C reating P rotocols for T rustworthiness in Q ualitative

R esearch

Anything perceived as being of low or no value is also perceived as being worthless, unreliable, or invalid. Research that is perceived as worthless is said to lack rigor. This means findings are not worth noting or paying attention to, because they are unreliable. To avoid this argument, proof of reliability and validity in qualitative research methods is required. However, some researchers have suggested that reliability and validity are not terms to be used to explain the usefulness of qualitative research. They believe that those terms are to be used to validate quantitative research (Altheide & Johnson, 1998; Leininger, 1994). Morse (1999) expressed concern about qualitative research losing value by em­ phasizing when qualitative researchers fail to recognize crucial importance of reliability and validity in qualita­ tive methods, they are also mistakenly supporting the idea that qualitative research is defective and worthless, lacking in thoroughness, and of unempirical value. Guba and Lincoln (1981) stated that, "All research must have 'tru th value', 'applicability', 'consistency', and 'neutrality' in order to be considered worthwhile. They concluded that the end result of establishing rigor or "trustworthiness," (the analogous for rigor in qualitative research), for each method of research requires a differ­ ent approach. It was noted by Guba and Lincoln (1981),

Linda A m ankw aa, PhD , RN, FAAN, is an Associate Professor in the Department o f Nursing at Albany State Uni­ versity in Albany, GA31705. Dr. Amankwaa may be reached at: 229-430-4731 or at: Linda.Amankwaa@asurams.edu.

within the rationalistic paradigm, criteria to reach the goal of rigor are internal validity, external validity, reli­ ability, and objectivity. They proposed use of terms such as credibility, fittingness, auditability, and confirmability in qualitative research to ensure "trustworthiness" (Guba & Lincoln, 1981). Later, these criteria were changed to credibility, transferability, dependability, and confirm­ ability (Lincoln & Guba, 1985). Lincoln and Guba (1985) suggested that the value of a research study is strengthened by its trustworthiness. As established by Lincoln and Guba in the 1980s, trustwor­ thiness involves establishing:

• Credibility - confidence in the 'tru th ' of the finding

• Transferability - show ing that the findings have applicability in other contexts

• D ependability - show ing that the findings are consistent and could be repeated

• Confirmability - a degree of neutrality or the ex­ tent to w hich the findings of a stu d y are shaped by the respondents and not researcher bias, motivation, or interest.

For purposes of this discussion, this classic work is used to frame trustworthiness actions and activities to create a protocol for qualitative studies. Nursing faculty and doctoral nursing students who conduct qualitative research will find this reference useful.

Journal of C ultural Diversity • Vol. 23, No. 3 Fall 2016

mailto:Linda.Amankwaa@asurams.edu
Credibility Activities Lincoln and Guba (1985) described a series of techniques

that can be used to conduct qualitative research that at­ tains the criteria they outlined. Techniques for establishing credibility as identified by Lincoln and Guba (1985) are: prolonged engagem ent, persistent observation, triangula­ tion, peer debriefing, negative case analysis, referential adequacy, and member-checking. Typically member check­ ing is view ed as a technique for establishing the validity of an account. Lincoln and Guba posit that this is the most crucial technique for establishing credibility.

Transferability Activities One strategy that can be em ployed to facilitate transfer-

ability is thick description (Creswell & Miller, 2000; Lincoln & Guba, 1985). Thick description is described by Lincoln and Guba as a w ay of achieving a type of external valid­ ity. By describing a phenom enon in sufficient detail one can begin to evaluate the extent to which the conclusions d raw n are transferable to other times, settings, situations, and people. Since, as stated by M erriam (1995) it is the responsibility of the consum er of research to determ ine or decide if and how research results m ight be applied to other settings, the original researcher m u st provide detailed inform ation about the phenom enon of study to assist the consum er in m aking the decision. This requires the provision of copious am ounts of inform ation regard­ ing every aspect of the research. The investigator will include such details as the location setting, atm osphere, climate, participants present, attitudes of the participants involved, reactions observed that m ay not be captured on audio recording, bonds established betw een participants, and feelings of the investigator. One w ord descriptors will not suffice in the developm ent of thick description. The investigator in essence is telling a story w ith enough detail that the c o n su m er/read er obtains a vivid picture of the events of the research. This can be accom plished through journaling and m aintaining records w h eth er digital or handw ritten for review by the consum er/reader.

Confirmability Activities To establish confirmability Lincoln and Guba (1985)

suggested confirmability audit, audit trail, triangulation, and reflexivity. An audit trail is a transparent description of the research steps taken from the start of a research project to the developm ent and reporting of findings (Lincoln & Guba). These are records that are kept regarding w hat was done in an investigation. Lincoln and Guba cite H alpern's (1983) categories for reporting inform ation w hen develop­ ing an audit trail:

"1) Raw data - including all raw data, written field notes, unobstrusive measures (documents); 2) Data reduction and analysis products - including sum ­ maries such as condensed notes, unitized information and quantitative summaries and theoretical notes; 3) Data reconstruction and synthesis products - includ­ ing structure o f categories (themes, definitions, and relationships), findings and conclusions and a final report including connections to existing literatures and an integration o f concepts, relationships, and interpretations; 4) Process notes - including method­ ological notes (procedures, designs, strategies, ratio­ nales), trustworthiness notes (relating to credibility, dependability and confirmability) and audit trail notes; 5) Materials relating to intentions and dispositions -

including inquiry proposal, personal notes (reflexive notes and. motivations) and expectations (predictions and intentions); 6) Instrum ent development informa­ tion - including pilot forms, preliminary schedules, observation form ats" (page#).

Using m ultiple data sources w ithin an investigation to enhance understanding is called triangulation. Researchers see triangulation as a m ethod for corroborating findings and as a test for validity (Lincoln & Guba, 1985). Rather than seeing triangulation as a m ethod for validation or veri­ fication, qualitative researchers generally use this technique to ensure that an account is rich, robust, comprehensive and well-developed (Lincoln & Guba, 1985).

D enzin (1978) and Patton (1999) identify four types of triangulation: m ethods triangulation, source triangulation; analyst triangulation; th eo ry /p ersp ectiv e triangulation. They suggested that m ethods triangulation involves check­ ing out the consistency of finding generated by different data collection m ethods. Triangulation of sources is an exam ination of the consistency of different data sources from w ithin the same m ethod (i.e. at different points in time; in public vs. private settings; com paring people w ith different viewpoints).

A nother one of the four m ethods identified by Denzin and Patton includes analyst triangulation. This is the use of m ultiple analysts to review findings or using m ultiple observers and analysts. This provides a check on selective perception and illum inate blind spots in an interpretive analysis. The goal is to understand m ultiple ways of see­ ing the data. Finally, they described th eory/perspective triangulation as the use of m ultiple theoretical perspectives to examine and interpret the data.

According to Lincoln and Guba (1985) reflexivity is, "A n attitude of attending system atically to the context of know ledge construction, especially to the effect of the researcher, at every step of the research process." They suggested the following steps to develop reflexivity: 1) Designing research that includes m ultiple investigators. This fosters dialogue, leads to the developm ent of comple­ m entary and divergent understandings of a study situation and provides a context in w hich researchers' (often h id ­ den) - beliefs, values, perspectives and assum ptions can be revealed and contested; 2) Develop a reflexive journal. This is a type of diary w here a researcher m akes regular entries during the research process. In these entries, the researcher records methodological decisions and the reasons for them, the logistics of the study and reflection u p o n w hat is h a p ­ pening in term s of one's ow n values and interests. Diary keeping of this type is often very private and cathartic; 3) Report research perspectives, positions, values and beliefs in m anuscripts and other publications. Many believe that it is valuable and essential to briefly report in m anuscripts, as best as possible, how one's preconceptions, beliefs, values, assum ptions and position m ay have come into play during the research process.

Dependability Activities To establish dependability, Lincoln and Guba (1985) sug­

gested a technique know n as inquiry audit. Inquiry audits are conducted by having a researcher that is not involved in the research process examine both the process and product of the research study (Lincoln & Guba, 1985). The purpose is to evaluate the accuracy and evaluate w hether or not the findings, interpretations and conclusions are supported by the data (Lincoln & Guba, 1985).

Journal of Cultural Diversity • Vol. 23, No. 3 Fall 2016

Creating a Protocol for Qualitative Researchers The creation of a protocol for establishing trustwor­

thiness within qualitative research is essential to rigor. Further, we note that researchers rarely document how or w hat their trustworthiness plan or protocol consisted of within research documents. Thus, we posit here that creating such a protocol prior to initiation of the research study is essential to revealing trustworthiness within the research process. By creating this plan a priori, the rigor of qualitative research is apparent.

This history and purposed need for this article heralds from a doctoral dissertation search to find examples of trustworthiness protocols for direction to complete trust­ worthiness within doctoral qualitative research. Since none could be found, discussions lead the researcher to create a table that could used by those who are planning qualita­ tive studies. Another interesting point is that qualitative researchers, unlike quantitative researchers, rarely create protocol guidelines.

The establishment of trustworthiness protocols in quali­ tative research requires the use of several techniques. This protocol will be detail specific so those researchers have a guideline for trustworthiness activities. Such a protocol guides prospective qualitative researchers in their quest for rigor. Several tables are presented here. The first table outlines the main topics within the trustworthiness proto­ col. The remaining tables outline the suggested activities within trustworthiness protocol and for those creating a trustworthiness protocol.

Table one is the basic criteria for a trustworthiness pro­ tocol using Lincoln and Guba (1985). However, researchers may use other models of rigor. Creating a table aligned with the planned model of rigor is the recommendation. The following five table are examples of a "created" protocol w ith examples of very specific activities related to each trustworthiness criteria. Summary

In summary, trustw orthiness is a vital com ponent within the research process. Attending to the language of trustworthiness and the important activities of reliabil­ ity, add to the comprehensiveness and the quality of the research product. This discussion heralds the new idea that trustworthiness must be planned ahead of time with a protocol. This protocol must include dates and times trustworthiness actions. We contend that researchers can use the protocol by adding two columns which specify the date of the planned trustworthiness action and the date the action was actually completed. This information can then be included in the audit trail thus authenticating the work qualitative researcher and the rigor of the research.

REFERENCES Altheide, D., & Johnson, J. (1998). Criteria for assessing interpre­

tive validity in qualitative research. In N. K. Denzin, & Y. S. Lincoln (Eds.), Collecting and interpreting materials, 283- 312.

Creswell, J. & Miller, D. (2000). D eterm ining validity and qualita­ tive inquiry. Theory Into Practice, 39(3), 125-130.

Denzin, N. (1978). Sociological Methods. N ew York: M cGraw-Hill. Guba, E. & Lincoln, Y. (1981). Effective evaluation: improving the

usefidness of evaluation results through responsive and naturalistic approaches. San Francisco, CA: Jossey-Bass.

Leininger, M. (1994). Evaluation criteria and critique of qualitative and interpretive research. Qualitative Inquiry, 1, 275-279.

Lincoln, Y. S. & Guba, E. G. (1985). Naturalistic Inquiry. N ew bury Park, CA: Sage Publications.

Morse, J. (1999). Myth #3: Reliability and validity are n ot relevant to qualitative mquiry.Qualitative Heath Research, 9, 717.

Patton, M. Q. (1999). "Enhancing the quality and credibility of qualitative analysis." HSR: Health Services Research. 34(5), Part II, 1189-1208.

BIBLIOGRAPHY Bitsch, V. (2005). Qualitative research: A grounded theory example

and evaluation criteria. Journal of Argibusiness, 23 (1), 75-91. Carpenter, R. (1995). G rounded theory research approach. In H.

J. Streubert & R. D. Carpenter(Eds-), Qualitative research and in nursing: Advancing the humanistic imperative, 145-161.

C ohen D., Crabtree, B. (2006). Q ualitative Research Guidelines Project. July 2006. http://w w w .qualres.org/H om eRefl-3703. htm l

Giacomini, M. & Cook, D. (2000). A u s e r's guide to qualitative research in health care. In Users' guides to evidence-based medicine. Journal o f the American Medical Association, 284(4), 478-482.

Morse, J. Barrett, M., Mayan, M., Olson, K., & Spiers, J. (2002). Verification strategies for establishing reliability and valid­ ity in qualitative research. International Journal o f Q u a lita ­ tive Methods, 1, 2, Article 2. Retrieved April 30, 2010 from http: / / w w w .u alb erta.ca/-ijq rn /

Neuman, L. (2003). Qualitative and quantitative measurements. In Social research methods: Qualitative and quantitative approaches, fifth edition, 169-209.

Plack, M. (2005). H um an nature and research paradigm s: Theory meets physical therapy practice. The Qualitative Report, 10(2), 223-245.

Polit, D. & Hungler, B. (1999). Research control in quantitative research. In N ursing research: P rin c ip le s a n d m e th o d s , sixth edition, 219-238. Lippincott.

Rubin, H. & Rubin, I. (1995). Qualitative interviewing: The art of hearing data. Thousand Oaks, CA: Sage Publications.

Siegle, D. (2002). Principles and m ethods in educational research: A web-based course from the University of Connecticut. Re­ trieved April 30, 2010 from http: / / w w w .gifted.uconn.edu/ siegle / research/qualitative / qualitativeInstructorNotes.html

Tobin, G. & Begley, C. (2004). M ethodological rigour w ithin a qualitative fram ew ork. Journal of Advanced Nursing, 48(4), 388-396.

Table 1. Basic Trustworthiness Criteria (Lincoln & Guba, 1985)

Criteria Technique

Credibility Peer debriefing, m em ber checks, journaling

Transferability Thick description, journaling

Dependability Inquiry audit with audit trail

Confirm ability Triangulation, journaling

Journal of Cultural Diversity Fall 2016

http://www.qualres.org/HomeRefl-3703
http://www.ualberta.ca/-ijqrn/
http://www.gifted.uconn.edu/
Table 2. Credibility

Credibility R ecom m ended activities/plan

Peer 1. W rite plan within proposal. debriefing/debriefer

2. Com m ission a peer to w ork with researcher during the tim e of interviews and data collection.

3. This person must com plete an attestation form to work with researcher. Plan to meet with this person after each interview.

4. During visits with the peer debriefer, research and peer discuss interviews, feelings, actions o f subjects, thoughts, and ideas that present during this time. Discuss blocking, clouding and other feelings of researcher. Discuss dates and tim es needed fo r these activities. W ill meet once a w eek fo r 30 minutes to an hour.

5. Journal these meetings. W rite about thoughts that surfaced and keep these dated for research and evaluation during data analysis.

6. Need to be com puter files so that you may use this inform ation within data analysis.

M em ber Checks 1. Outline different tim es and reasons you plan to conduct m em ber checks or collect feedback from m em bers about any step in the research process.

2. M em ber checks will consist o f com m unication with mem bers after significant activities.

3. These activities may include interviews, data analysis, and other activities.

4. W ithin two weeks o f the interview, send mem bers a copy o f their interview so that they can read it and edit for accuracy.

5. W ithin two weeks o f data analysis com pletion, m em ber will review a copy o f the final them es.

6. M em bers are asked the question, “ Does the interview transcript reflect your words during the interview?”

7. C hoose negative cases and cases that follow pattern.

8. Be sure these check are recorded and are com puter files so that you may use this inform ation in data analysis.

Journaling plans 1. Journaling will begin with the writing o f the proposal.

2. Journaling will be conducted after each significant activity. These include each interview, w eekly during analysis, after peer debriefing visits, and them e production.

3. Journals will be audited by research auditor.

4. Journals will include dates, times, places and persons on the research team.

5. Journals need to be com puter files so that you may use them in data analysis.

Protocol Create a tim eline with planned dates fo r each activity related to credibility before com m encing the study. This protocol with dates and activities should appear in the appendix.

Journal of Cultural Diversity • Vol. 23, No. 3 Fall 2016

Table 3. Transferability

Thick Description Actions for this activity include:

1. Reviewing crafted questions with Peer reviewer for clarity.

2. Planning questions that call for extended answers.

3. Asking open ended questions that solicit detailed answers.

4. Interviewing in such a way as to obtain a detailed, thick and robust response.

5. The object is to reproduce the phenomenon of research as clearly and as detailed as possible.

6. This action is replicated with each participant and with each question (sub-question) or statement.

7. This continues until all questions and sub-questions are discussed.

8. The peer reviewer along with the researcher review responses for thickness and robustness.

9. There are two issues related to thick description here. The first is receiving thick responses (not one sentence paragraphs). The second is writing up the responses of multiple participants in such a way as to describe the phenomena as a thick response.

Journaling Actions for this activity include:

1. Planning journal work in advance is an option. Such that the researcher could decide what dates and how often the journal will occur.

2. Journaling after interview is common.

3. Journaling after peer-review sessions.

4. Journaling after a major event during the study.

5. Journal entries should be discussed with peer reviewer such that expression of thoughts and ideas gleaned during research activities can be connected to participants’ experiences.

6. Journals can be maintained in various formats. Information for the journal can be received in the form of emails, documents, recordings, note cards/note pads. We recommend that the researcher decide on one of the options.

7. Journaling includes dates of actions related to significant and insignificant activities of the research.

8. Journal may start on the first date a decision is made to conduct the research.

9. Journaling ends when the research is completed and all participants have been interviewed.

10. As with each of the concepts here, create a timeline with a date-line protocol for each activity before commencing the study.

Protocol Create a timeline with planned dates for each activity related to transferability before commencing the study. This protocol with dates and activities should appear in the appendix.

Journal of Cultural D iversity • Vol. 23, No. 3 Fall 2016

Table 4. Dependability

A udit Trail Com ponents o f the audit trail include:

1. Make the list of docum ents planned for audit during the research work.

2. Com m ission the auditor based on plan for study.

3. Decide audit trail review dates and times.

4. See auditor inform ation below

5. W rite up audit trail results in the journal.

Journaling A ctions for this activity include:

I . Planning journal w ork in advance is an option. Such that the researcher could decide what dates and how often the journal will occur.

I I . Journaling after interview is common.

12. Journaling after peer-review sessions.

13. Journaling after a m ajor event during the study.

14. Journal entries should be discussed with peer reviewer such that expression of thoughts and ideas gleaned during research activities can be connected to participants’ experiences.

15. Journals can be maintained in various form ats. Information fo r the journal can be received in the form of emails, docum ents, recordings, note cards/note pads. W e recom m end that the researcher decide on one o f the options.

16. Journaling includes dates o f actions related to significant and insignificant activities o f the research.

17. Journal may start on the first date a decision is made to conduct the research.

18. Journaling ends w hen the research is com pleted and all participants have been interviewed.

Auditor 1. The auditor is reviewing the docum ents fo r authenticity and consistency.

2. The auditor may be a colleague or som eone unfam iliar with the research such that activities can be questioned fo r clarity.

3. The auditor should have som e com prehension o f the research process.

4. Planning in advance fo r the tim e com m itm ent as an auditor is crucial.

5. Should provide constructive feedback on processes in an honest fashion.

6. Auditor, researcher, and participants should speak the same language.

7. Must be able to create and maintain audit trail documents.

Protocol Create a tim eline with planned dates fo r each activity related dependability before com m encing the study. This protocol with dates and activities should appear in the appendix.

Journal of Cultural Diversity • Vol. 23, No. 3 Fall 2016

Table 5. Confirmability

Triangulation 1. Determine triangulation methods

2. Document triangulation plans within journal.

3. Discuss triangulation results peer-reviewer

4. Decide if further triangulation is needed

5. Write up the triangulation results.

Journaling Actions for this activity include:

2. Planning journal work in advance is an option. Such that the researcher could decide what dates and how often the journal will occur.

19. Journaling after interview is common.

20. Journaling after peer-review sessions.

21. Journaling after a major event during the study.

22. Journal entries should be discussed with peer reviewer such that expression of thoughts and ideas gleaned during research activities can be connected to participants experiences.

23. Journals can be maintained in various formats. Information for the journal can be received in the form of emails, documents, recordings, note cards/note pads. We recommend that the researcher decide on one of the options.

24. Journaling includes dates of actions related to significant and insignificant activities of the research.

25. Journal may start on the first date a decision is made to conduct the research.

Journaling ends when the research is completed and all participants have been interviewed.

Protocol Create a timeline with planned dates for each activity related confirmability before commencing the study. This protocol with dates and activities should appear in the appendix.

• Vol. 23, No. 3 ( E 9Journal of C ultural Diversity Fall 2016

Copyright of Journal of Cultural Diversity is the property of Tucker Publications, Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.