Basic guide to program evaluation

Discussion 1: Use of Group Designs in Program Evaluation

Group programs are common in social work. Just as with other types of programs, social workers must understand the options available to them and know how to select the appropriate research design.

For this Discussion, you evaluate group research design methods that can be used for an outcome evaluation of a foster parent training program. You also generate criteria to be measured in the program.

To prepare for this Discussion, review the “Social Work Research: Planning a Program Evaluation” case study in this week’s resources, Plummer, S.-B., Makris, S., & Brocksen S. (Eds.). (2014b). Social work case studies: Concentration year. Retrieved from http://www.vitalsource.com , and the section of “Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations with Very Limited Resources”, titled “Overview of Methods to Collect Information."

· Post your explanation of which group research design and data collection method from those outlined in the Resources you selected as appropriate for the “Social Work Research: "Planning a Program Evaluation" case study and why.

· Then, generate criteria to be measured using the research design by identifying a specific outcome and a method for measuring that outcome. Specify who will collect the data and how the data will be collected.

References (use 3 or more)

Dudley, J. R. (2014). Social work evaluation: Enhancing what we do. (2nd ed.) Chicago, IL: Lyceum Books.

· Chapters 9, “Is the Intervention Effective?” (pp. 213–250)

· Chapter 10, “Analyzing Evaluation Data” (pp. 255–275)

McNamara, C. (2006a). Contents of an evaluation plan. In Basic guide to program evaluation (including outcomes evaluation). Retrieved from http://managementhelp.org/evaluation/program-evaluation-guide.htm#anchor1586742

McNamara, C. (2006b). Reasons for priority on implementing outcomes-based evaluation.In Basic guide to outcomes-based evaluation for nonprofit organizations with very limited resources. Retrieved from http://managementhelp.org/evaluation/outcomes-evaluation-guide.htm#anchor30249

Plummer, S.-B., Makris, S., & Brocksen S. (Eds.). (2014b). Social work case studies: Concentration year. Baltimore, MD: Laureate International Universities Publishing. [Vital Source e-reader].

Read the following section:

“Social Work Research: Planning a Program Evaluation”

Social Work Research: Planning a Program Evaluation

Joan is a social worker who is currently enrolled in a social work PhD program. She is planning to conduct her dissertation research project with a large nonprofit child welfare organization where she has worked as a site coordinator for many years. She has already approached the agency director with her interest, and the leadership team of the agency stated that they would like to collaborate on the research project.

The child welfare organization at the center of the planned study has seven regional centers that operate fairly independently. The primary focus of work is on foster care; that is, recruiting and training foster parents and running a regular foster care program with an emphasis on family foster care. The agency has a residential program as well, but it will not participate in the study. Each of the regional centers services about 45–50 foster parents and approximately 100 foster children. On average, five to six new foster families are recruited at each center on a quarterly basis. This number has been consistent over the past 2 years.

Recently it was decided that a new training program for incoming foster parents would be used by the organization. The primary goals of this new training program include reducing foster placement disruptions, improving the quality of services delivered, and increasing child well-being through better trained and skilled foster families. Each of the regional centers will participate and implement the new training program. Three of the sites will start the program immediately, while the other four centers will not start until 12 months from now. The new training program consists of six separate 3-hour training sessions that are typically conducted in a biweekly format. It is a fairly proceduralized training program; that is, a very detailed set of manuals and training materials exists. All trainings will be conducted by the same two instructors. The current training program that it will replace differs considerably in its focus, but it also uses a 6-week, 3-hour format. It will be used by those sites not immediately participating until the new program is implemented.

Joan has done a thorough review of the foster care literature and has found that there has been no research on the training program to date, even though it is being used by a growing number of agencies. She also found that there are some standardized instruments that she could use for her study. In addition, she would need to create a set of Likert-type scales for the study. She will be able to use a group design because all seven regional centers are interested in participating and they are starting the training at different times.

Discussion 2: Assessing Outcomes

How do you determine the “success” of a human services program? Part of your role as an administrator is to collaborate with your staff to determine how a particular program’s effectiveness will be measured. The outcomes must be clear, realistic, and feasible, and how the outcomes will be assessed must be clear also.

For this Discussion, you will address the “Social Work Research: Program Evaluation” case study in Social Work Case Studies: Foundation Year. Assume the role of an administrator in the case study to evaluate what has occurred in the program and how you might improve it.

· Post an evaluation of the success of the CALWORKS program based on the information presented in the case study.

· Be sure to define what success would be for the program and how you, as an administrator of the program, might evaluate whether success has been achieved.

· Finally, make one recommendation for improving the program’s effectiveness.

Reference (use 3 or more)

Benton, A. D., & Austin, M. J. (2010). Managing nonprofit mergers: The challenges facing human service organizations. Administration in Social Work, 34(5), 458–479.

King, D., & Hodges, K. (2013). Outcomes-driven clinical management and supervisory practices with youth with severe emotional disturbance. Administration in Social Work, 37(3), 312–324.

Lawrence, C., Strolin-Goltzman, J., Caringi, J., Claiborne, N., McCarthy, M., Butts, E., & O’Connell, K. (2013). Designing evaluations in child welfare organizations: An approach for administrators. Administration in Social Work, 37(1), 3–13.

Lynch-Cerullo, K., & Cooney, K. (2011). Moving from outputs to outcomes: A review of the evolution of performance measurement in the human service nonprofit sector. Administration in Social Work, 35(4), 364–388.

Plummer, S.-B., Makris, S., & Brocksen, S. M. (Eds.). (2014c). Social work case studies: Foundation year. Baltimore, MD: Laureate International Universities Publishing [Vital Source e-reader].

· “Social Work Research: Program Evaluation” (pp. 66–68)

Social Work Research: Planning a Program Evaluation

Joan is a social worker who is currently enrolled in a social work PhD program. She is planning to conduct her dissertation research project with a large nonprofit child welfare organization where she has worked as a site coordinator for many years. She has already approached the agency director with her interest, and the leadership team of the agency stated that they would like to collaborate on the research project.

The child welfare organization at the center of the planned study has seven regional centers that operate fairly independently. The primary focus of work is on foster care; that is, recruiting and training foster parents and running a regular foster care program with an emphasis on family foster care. The agency has a residential program as well, but it will not participate in the study. Each of the regional centers services about 45–50 foster parents and approximately 100 foster children. On average, five to six new foster families are recruited at each center on a quarterly basis. This number has been consistent over the past 2 years.

Recently it was decided that a new training program for incoming foster parents would be used by the organization. The primary goals of this new training program include reducing foster placement disruptions, improving the quality of services delivered, and increasing child well-being through better trained and skilled foster families. Each of the regional centers will participate and implement the new training program. Three of the sites will start the program immediately, while the other four centers will not start until 12 months from now. The new training program consists of six separate 3-hour training sessions that are typically conducted in a biweekly format. It is a fairly proceduralized training program; that is, a very detailed set of manuals and training materials exists. All trainings will be conducted by the same two instructors. The current training program that it will replace differs considerably in its focus, but it also uses a 6-week, 3-hour format. It will be used by those sites not immediately participating until the new program is implemented.

Joan has done a thorough review of the foster care literature and has found that there has been no research on the training program to date, even though it is being used by a growing number of agencies. She also found that there are some standardized instruments that she could use for her study. In addition, she would need to create a set of Likert-type scales for the study. She will be able to use a group design because all seven regional centers are interested in participating and they are starting the training at different times.