Why is it desirable to pretest survey instruments

Chapter 12 Discussion Questions

Terms in Review

1

Discuss the relative merits of and problems with:

1. Rating and ranking scales.

2. Likert and differential scales.

3. Unidimensional and multidimensional scales.

Making Research Decisions

2

Assume you are Menu Foods and you planned a major research study just prior to the largest pet food recall in our history. You plan to proceed with the study and feel you must add one or more questions to measure the consumer’s confidence that your firm will be able to recover. Draft a scale for each of the following types that will measure that confidence.

1. Constant-sum scale.

2. Likert-type summated scale.

3. Semantic differential scale.

4. Stapel scale.

5. Forced ranking scale.

3

An investigative question in your employee satisfaction study seeks to assess employee “job involvement.” Create a measurement question that uses the following scales:

1. A graphic rating scale.

2. A multiple rating list.

Which scale do you recommend and why?

4

You receive the results of a paired-comparison preference test of four soft drinks from a sample of 200 persons. The results are as follows:

Koak

Zip

Pabze

Mr. Peepers

Koak

—

50 *

115

35

Zip

150

—

160

70

Pabze

85

40

—

45

Mr. Peepers

165

130

155

—

*Read as 50 persons preferred Zip to Koak.

1. How do these brands rank in overall preference in this sample?

2. Develop an interval scale for these four brands.

5

One of the problems in developing rating scales is the choice of response terms to use. Below are samples of some widely used scaling codes. Do you see any problems with them?

1. Yes—Depends—No

2. Excellent—Good—Fair—Poor

3. Excellent—Good—Average—Fair—Poor

4. Strongly Approve—Approve—Uncertain—Disapprove— Strongly Disapprove

6

You are working on a consumer perception study of four brands of bicycles. You will need to develop measurement questions and scales to accomplish the tasks listed below. Be sure to explain which data levels (nominal, ordinal, interval, ratio) are appropriate and which quantitative techniques you will use.

1. Prepare an overall assessment of all the brands.

2. Provide a comparison of the brands for each of the following dimensions:

1. Styling

2. Durability

3. Gear quality

4. Brand image

7

Below is a Likert-type scale that might be used to evaluate your opinion of the educational degree program in which you are enrolled. There are five response categories: Strongly Agree, Agree, Neither Agree nor Disagree, Disagree, and Strongly Disagree. If Strongly Agree (SA) represents the most positive attitude, how would you value the items below? Record your answers to the items.

1. This program is not very challenging. SA A N D SD

2. The general level of teaching is good. SA A N D SD

3. I really think I am learning a lot from this program. SA A N D SD

4. Students’ suggestions are given little attention here. SA A N D SD

5. This program does a good job of preparing one for a career. SA A N D SD

6. This program is below my expectations. SA A N D SD

In what two different ways could such responses be used? What would be the purpose of each?

Bringing Research to Life

8

What is the basis of Jason and Myra’s argument for the need of an arbitrary scale to address customer expectations?

From Concept to Practice

9

Using the response strategies within Exhibit 12-1 or 12-10, which would be appropriate and add insight to understanding the various indicants of student demand for the academic program in which the students are enrolled?

From the Headlines

10

According to BusinessWeek, the U.S. workforce is becoming a temporary workforce. Even full-time workers are often contract employees without health, vacation, or other benefits. What measurement scale(s) would be appropriate to measure this trend?

Chapter 13 Discussion Questions

Terms in Review

1

Distinguish between:

1. Direct and indirect questions.

2. Open-ended and closed questions.

3. Research, investigative, and measurement questions.

4. Alternative response strategies.

2

Why is the survey technique so popular? When is it not appropriate?

3

What special problems do open-ended questions have? How can these be minimized? In what situations are open-ended questions most useful?

4

Why might a researcher wish to disguise the objective of a study?

5

One of the major reasons why survey research may not be effective is that the survey instruments are less useful than they should be. What would you say are the four possible major faults of survey instrument design?

6

Why is it desirable to pretest survey instruments? What information can you secure from such a pretest? How can you find the best wording for a question on a questionnaire?

7

One design problem in the development of survey instruments concerns the sequence of questions. What suggestions would you give to researchers designing their first questionnaire?

8

One of the major problems facing the designer of a survey instrument concerns the assumptions made. What are the major “problem assumptions”?

Making Research Decisions

9

Following are six questions that might be found on questionnaires. Comment on each as to whether or not it is a good question. If it is not, explain why. (Assume that no lead-in or screening questions are required. Judge each question on its own merits.)

1. Do you read National Geographic magazine regularly?

2. What percentage of your time is spent asking for information from others in your organization?

3. When did you first start chewing gum?

4. How much discretionary buying power do you have each year?

5. Why did you decide to attend Big State University?

6. Do you think the president is doing a good job now?

10

In a class project, students developed a brief self-administered questionnaire by which they might quickly evaluate a professor. One student submitted the following instrument. Evaluate the questions asked and the format of the instrument.

Professor Evaluation Form

1. Overall, how would you rate this professor?

· Good

· Fair

· Poor

2. Does this professor

0. Have good class delivery? _____

1. Know the subject? _____

2. Have a positive attitude toward the subject? _____

3. Grade fairly? _____

4. Have a sense of humor? _____

5. Use audiovisuals, case examples, or other classroom aids? _____

6. Return exams promptly? _____

1. What is the professor’s strongest point? _____

1. What is the professor’s weakest point? _____

1. What kind of class does the professor teach? _____

1. Is this course required? _____

1. Would you take another course from this professor? _____

11

Assume the American Society of Training Directors is studying its membership in order to enhance member benefits and attract new members. Below is a copy of a cover letter and mail questionnaire received by a member of the society. Please evaluate the usefulness and tone of the letter and the questions and format of the instrument.

Dear ASTD Member:

The ASTD is evaluating the perception of value of membership among its members. Enclosed is a short questionnaire and a return envelope. I hope you will take a few minutes and fill out the questionnaire as soon as possible, because the sooner the information is returned to me, the better.

Sincerely,

Director of Membership

Questionnaire

Directions: Please answer as briefly as possible.

1. With what company did you enter the field of training?

__________

2. How long have you been in the field of training?

__________

3. How long have you been in the training department of the company with which you are presently employed?

__________

4. How long has the training department in your company been in existence? __________

5. Is the training department a subset of another department? If so, what department? __________

__________

6. For what functions (other than training) is your department responsible? __________

__________

7. How many people, including yourself, are in the training department of your company (local plant or establishment)?

__________

8. What degrees do you hold and from what institutions?

__________

Major _____ Minor _____

9. Why were you chosen for training? What special qualifications prompted your entry into training?

__________

__________

10. What experience would you consider necessary for an individual to enter into the field of training with your company? Include both educational requirements and actual experience.

__________

__________

__________

Bringing Research to Life

12

Design the introduction of the Albany Outpatient Laser Clinic survey, assuming it will continue to be a self-administered questionnaire.

13

To evaluate whether presurgery patient attitudes affect recovery and ultimate patient satisfaction with the Albany Outpatient Laser Clinic, design a question for the self-administered survey. (You may wish to review the opening vignettes in this chapter and Chapter 9 .)

From Concept to Practice

14

Using Exhibits 13-1 , 13-4 , and 13-9 , develop the flowchart for the Albany Outpatient Laser Clinic study in the opening vignette.

From the Headlines

15

One of Kraft’s hallmark brands is Jell-O. During May 2009, the Jell-O Sugar Free Pudding cups line was getting a makeover with the introduction of two new flavors: Boston Cream Pie and Cinnamon Roll.

1. What survey research would you have done to determine whether to introduce these new flavors?

2. Design the questionnaire for the survey research you would have done.

Discussion Questions Chapter 14

Terms in Review

1

Distinguish between:

1. Statistic and parameter.

2. Sample frame and population.

3. Restricted and unrestricted sampling.

4. Simple random and complex random sampling.

5. Convenience and purposive sampling.

6. Sample precision and sample accuracy.

7. Systematic and error variance.

8. Variable and attribute parameters.

9. Proportionate and disproportionate samples.

2

Under what kind of conditions would you recommend:

1. A probability sample? a nonprobability sample?

2. A simple random sample? a cluster sample? a stratified sample?

3. A disproportionate stratified probability sample?

3

You plan to conduct a survey using unrestricted sampling. What subjective decisions must you make?

4

Describe the differences between a probability sample and a nonprobability sample.

5

Why would a researcher use a quota purposive sample?

Making Research Decisions

6

Your task is to interview a representative sample of attendees for the large concert venue where you work. The new season schedule includes 200 live concerts featuring all types of musicians and musical groups. Since neither the number of attendees nor their descriptive characteristics are known in advance, you decide on nonprobability sampling. Based on past seating configurations, you can calculate the number of tickets that will be available for each of the 200 concerts. Thus, collectively, you will know the number of possible attendees for each type of music. From attendance research conducted at concerts held by the Glacier Symphony during the previous two years, you can obtain gender data on attendees by type of music. How would you conduct a reasonably reliable nonprobability sample?

7

Your large firm is about to change to a customer-centered organization structure, in which employees who have rarely had customer contact will now likely significantly influence customer satisfaction and retention. As part of the transition, your superior wants an accurate evaluation of the morale of the firm’s large number of computer technicians. What type of sample would you draw if it was to be an unrestricted sample?

Bringing Research to Life

8

Design an alternative nonprobability sample that will be more representative of infrequent and potential riders for the CityBus project.

9

How would you draw a cluster sample for the CityBus project?

From Concept to Practice

10

Using Exhibit 14-8 as your guide, for each sampling technique describe the sample frame for a study of employers’ skill needs in new hires using the industry in which you are currently working or wish to work.

From the Headlines

11

During the 2010 Super Bowl, the Snicker’s candy bar campaign was voted the best ad by women 18 or older, men 18 or older, and youth 17 or younger. This ad features a neighborhood football game in which one player, who has flagging energy, is repeatedly tackled. His teammates tell him, “You play like Betty White!” The octogenarian actress and comedian appears in the ad being tackled. If you were AdBowl.com, how would you draw the sample to measure the best Super Bowl ad among the three groups of interest?