Comparative and noncomparative scaling techniques

MARK977: Research for Marketing Decisions

Dr. Thomas Lee Trimester 1, 2018

2

Measures and Questionnaire Design

WEEK 5 READING: CHAPTERS 7 & 8

2

Learning objectives

Measurement and Scaling

1. Introduce the concepts of measurement and scaling and show how scaling may be considered an extension of measurement.

2. Discuss the primary scales of measurement and differentiate nominal, ordinal, interval, and ratio scales.

3. Classify scaling techniques as comparative and non-comparative, and describe the comparative techniques of paired comparison, rank order, and constant sum.

4. Describe the non-comparative scaling techniques, distinguish between continuous and itemized rating scales, and explain Likert, semantic differential, and Stapel scales.

MARK977 Research for Marketing Decisions T1 20183

Measurement and scaling • Measurement means assigning numbers or other symbols to characteristics of

objects according to certain set rules.

 What we measure is not the object, but some characteristic of it (e.g., weight, height, etc.)

• Scaling involves creating a continuum upon which measured objects are located according to the amount of the measured characteristic possessed.

• Consider an attitude scale from 1 to 100. Each respondent is assigned a number from 1 to 100, with 1 = Extremely Unfavorable, and 100 = Extremely Favorable.

• Measurement is the actual assignment of a number from 1 to 100 to each respondent. Scaling is the process of placing the respondents on a continuum with respect to their attitude toward department stores.

MARK977 Research for Marketing Decisions T1 20184

3

Primary scales of measurement

MARK977 Research for Marketing Decisions T1 20185

Nominal scale

• In a nominal scale, the numbers serve only as labels or tags for identifying and classifying objects (e.g., respondent ID in a dataset).

• When used for classification purposes, the nominally scaled numbers serve as labels for classes or categories (e.g., 1 = male and 2 = female)

• See Fortune Ranking example (p. 210) where companies are assigned to a unique ID

MARK977 Research for Marketing Decisions T1 20186

4

Ordinal scale

• An ordinal scale is a ranking scale in which numbers are assigned to objects to indicate the relative extent to which the objects possess some characteristic.

• Can determine whether an object has more or less of a characteristic than some other object, but not how much more or less.

• For example, respondents rank five fast-food restaurants in order of preference by assigning 1 to most preferred restaurant, 2 to second most preferred, and so on (see Table 7.2 on p. 209)

• See also Fortune Ranking example (p. 210) to see how companies are ranked on the basis of what characteristics (e.g., innovativeness, quality)

MARK977 Research for Marketing Decisions T1 20187

Interval scale • In an interval scale, the numbers are used to rate objects. • The numerically equal distances on an interval scale represent

equal values in the characteristic being measured – Thus, it permits comparison of the differences between objects

• For example, asking respondents to express their degree of agreement with the statement “I like Starbucks coffee” on a 5- point scale (where 1=strongly disagree and 5=strongly agree) – This measures their attitude towards Starbucks coffee

• The location of the zero point is not fixed. Both the zero point and the units of measurement are arbitrary. Thus, not meaningful to take ratios of scale values. For example: – Preference score: 2 vs 4 on 5-point scale (2x) – If +2 and measured on 7-point scale, then 24 and 46 (not 2x)

MARK977 Research for Marketing Decisions T1 20188

5

Ratio scale • A ratio scale possesses all the properties of the nominal, ordinal,

and interval scales

• It has an absolute zero point and is measured as exact numbers

– E.g., firm size as measured by number of full-time employees or firm age as measured by years in operation

• Ratio scales can

– Identify or classify objects (e.g., < 50 employees = 1 = small firms, 50 or more employees = 2 = large firms)

– Rank the objects (e.g., in order of size – small, medium, large)

– Compare intervals or differences (e.g., 500 employees vs. 100 employees (5x more), 2 years vs. 20 years), thus meaningful to compare ratios of scale values

MARK977 Research for Marketing Decisions T1 20189

Measurement scales illustrated

MARK977 Research for Marketing Decisions T1 201810

6

Primary scales of measurement

MARK977 Research for Marketing Decisions T1 201811

Primary Scale Basic Characteristics Common Examples Marketing

Examples

Nominal Numbers

Identify and classify objects

Social Security numbers;

numbering of football

players

Brand numbers;

store types; gender

classification

Ordinal Numbers indicate the relative

positions of the objects but not

the magnitude of differences

between them

Quality rankings;

rankings of teams in a

tournament

Preference rankings;

market position

Interval Differences between objects can

be compared

Temperature

(Fahrenheit, Celsius)

Attitudes; opinions

Ratio Exact numbers; ratios of scale

values can be computed

Length; weight Age; income; costs;

sales; market shares

Primary scales of measurement illustrated

MARK977 Research for Marketing Decisions T1 201812

NOMINAL

SCALE ORDINAL SCALE INTERVAL SCALE RATIO SCALE

No. Store Preference Ranking

Preference Ratings $ spent

in last 2

months

¢ spent

in last 2

months1 - 7 11 - 17

1. Burger King 4 79 4 14 10 1000

2. McDonald’s 2 25 7 17 50 5000

3. Sub Way 5 82 3 13 0 0

4. KFC 3 30 6 16 25 2500

5. Pizza Hut 1 10 7 17 75 7500

7

Classification of scaling techniques

MARK977 Research for Marketing Decisions T1 201813

A comparison of scaling techniques • Comparative scales involve the direct comparison of stimulus

objects. Comparative scale data must be interpreted in relative terms and have only ordinal or rank order properties. – E.g., respondents might be asked whether they prefer Coke

or Pepsi. • In noncomparative scales, each object is scaled

independently of the others in the stimulus set. The resulting data are generally assumed to be interval scaled. – E.g., respondents might be asked to evaluate Coke (or Pepsi)

on a 1 to 7 preference scale (1=not at all preferred, 7=highly preferred).

MARK977 Research for Marketing Decisions T1 201814

Comparative scaling is also referred to as nonmetric scaling. Noncomparative scaling is also called metric scaling.

8

Comparative: Paired Comparison Scaling

• In paired comparison scaling, a respondent is presented with two objects and asked to select one according to some criterion (e.g., preference).

• The data obtained are ordinal in nature.

• Paired comparison scaling is useful when the number of brands is limited, but with a large number of brands, the number of comparisons becomes unwieldy.

MARK977 Research for Marketing Decisions T1 201815

Paired Comparison Scaling Illustrated Instructions We are going to present you with ten pairs of shampoo brands. For each pair, please indicate which one of the two brands of shampoo in the pair you would prefer for personal use.

Recording Form

MARK977 Research for Marketing Decisions T1 201816

Jhirmack Finesse Vidal Sassoon Head & Shoulders Pert

Jhirmack 0 0 1 0

Finesse 1A 0 1 0

Vidal Sassoon 1 1 1 1

Head & Shoulders 0 0 0 0

Pert 1 1 0 1

Number of times preferred 3B 2 0 4 1

A A 1 in a particular box means that the brand in that column was preferred over the brand in the corresponding row. A 0 means that the row brand was preferred over the column brand.

B The number of times a brand was preferred is obtained by summing the 1s in each column.

9

Comparative: Rank Order Scaling • In rank order scaling, respondents are presented with several

objects simultaneously and asked to order or rank them according to some criterion.

– E.g., respondents are asked to rank brands of toothpaste according to overall preference

– Assign 1 to most preferred brand, 2 to second most preferred brand, n to least preferred brand

• See Table 7.2 for illustration where fast-food restaurants are ranked in order of preference

MARK977 Research for Marketing Decisions T1 201817

Distances or differences between ranks are not defined, i.e., not meaningful in a rank order scale.

Rank Order Scaling Illustrated Instructions Rank the various brands of toothpaste in order of preference. Begin by picking out the one brand that you like most and assign it a number 1.

Then find the second most preferred-brand and assign it a number 2.

Continue this procedure until you have ranked all the brands of toothpaste in order of preference.

The least preferred brand should be assigned a rank of 10. No two brands should receive the same rank number. The criteria of preference is entirely up to you.

There is no right or wrong answer— Just try to be consistent.

MARK977 Research for Marketing Decisions T1 201818

Brand Rank Order

1. Crest

2. Colgate

3. Aim

4. Mentadent

5. Macleans

6. Ultra Brite

7. Close Up

8. Pepsodent

9. Plus White

10. Stripe

10

Comparative: Constant Sum Scaling

• In constant sum scaling, respondents allocate a constant sum of units, such as 100 points, to attributes of a product to reflect their importance.

• If an attribute is unimportant, the respondent assigns it zero points.

• If an attribute is twice as important as some other attribute, it receives twice as many points.

• The sum of all the points is 100. Hence, the name of the scale.

MARK977 Research for Marketing Decisions T1 201819

Constant Sum Scaling Illustrated Instructions Below are eight attributes of bathing soaps. Please allocate 100 points among the attributes so that your allocation reflects the relative importance you attach to each attribute. The more points an attribute receives, the more important the attribute is. If an attribute is not at all important, assign it zero points. If an attribute is twice as important as some other attribute, it should receive twice as many points.

Form

MARK977 Research for Marketing Decisions T1 201820

11

Noncomparative Scaling Techniques

• In noncomparative scaling respondents evaluate only one object at a time, and for this reason noncomparative scales are often referred to as monadic scales.

• Noncomparative techniques consist of continuous and itemized rating scales.

• In noncomparative scaling, respondents employ whatever rating standards seem appropriate to them.

MARK977 Research for Marketing Decisions T1 201821

Classification of noncomparative rating scales

MARK977 Research for Marketing Decisions T1 201822

12

Continuous Rating Scale In a continuous rating scale, respondents rate the objects by placing a mark at the appropriate position on a line that runs from one extreme of the

criterion variable to the other. The form of the continuous scale may vary

considerably.

MARK977 Research for Marketing Decisions T1 201823

Itemized Rating Scales

• In itemized rating scales, the respondents are provided with a scale that has a number or brief description associated with each category.

• The categories are ordered in terms of scale position, and the respondents are required to select the specified category that best describes the object being rated.

• Itemized rating scales are widely used in marketing research and form the basic components of multi-item scales.

• The commonly used itemized rating scales are the Likert, semantic differential, and Stapel scales.

MARK977 Research for Marketing Decisions T1 201824

13

Likert Scale  The Likert scale requires respondents to indicate a degree of agreement or

disagreement with each of a series of statements about the stimulus objects.

 The analysis can be conducted on an item-by-item basis (profile analysis), or a total (summated) score can be calculated.

 When arriving at a total score, the categories assigned to the negative statements should be scored by reversing the scale.

MARK977 Research for Marketing Decisions T1 201825

Semantic Differential Scale • The semantic differential is a seven-point rating scale with end points associated with

bipolar labels that have semantic meaning (e.g., cold and warm).

• The negative adjective or phrase sometimes appears at the left side of the scale and sometimes at the right.

• This controls the tendency of some respondents, particularly those with very positive or very negative attitudes, to mark the right- or left-hand sides without reading the labels.

• Individual items on a semantic differential scale may be scored on either a -3 to +3 or a 1 to 7 scale.

MARK977 Research for Marketing Decisions T1 201826

14

Stapel Scale

• The Stapel scale is a unipolar

rating scale with ten categories

numbered from -5 to +5,

without a neutral point (zero).

This scale is usually presented

vertically.

• The data obtained by using a

Stapel scale can be analysed in

the same way as semantic

differential data.

MARK977 Research for Marketing Decisions T1 201827

Basic noncomparative scales

MARK977 Research for Marketing Decisions T1 201828

Scale Basic Characteristics Examples Advantages Disadvantages

Continuous

Rating Scale

Place a mark on a

continuous line

Reaction to TV

commercials

Easy to construct Scoring can be

cumbersome

unless

computerized

Itemized Rating Scales

Likert Scale Degree of agreement on

a 1 (strongly disagree) to

5 (strongly agree) scale

Measurement of

attitudes

Easy to construct,

administer, and

understand

More time

consuming

Semantic

Differential

Seven-point scale with

bipolar labels

Brand, product,

and company

images

Versatile Difficult to cons-

truct bipolar

adjectives

Stapel Scale Unipolar ten-point scale,

-5 to +5, without a neutral

point (zero)

Measurement of

attitudes and

images

Easy to construct

and administer

over telephone

Confusing and

difficult to apply

15

Variations in rating scales

• Noncomparative itemized rating scales need not be used as originally proposed but can take many different forms.

• For example, the number of categories in a Likert scale need not be 5, but could be 7.

• The number of categories could be odd or even.

• Categories can be expressed by boxes, discrete lines, or units, with or without numbers.

– See Figure 7.6 (p. 224) for example

MARK977 Research for Marketing Decisions T1 201829

Balanced and unbalanced scales

MARK977 Research for Marketing Decisions T1 201830

16

MARK977 Research for Marketing Decisions T1 201831

MARK977 Research for Marketing Decisions T1 201832

17

Learning objectives

Questionnaire and Form Design

1. Explain the purpose of a questionnaire and its objectives. 2. Describe the process of designing a questionnaire, the

steps involved, and guidelines that must be followed at each step.

MARK977 Research for Marketing Decisions T1 201833

Definition and objectives

Definition • A questionnaire is a formalized set of questions for obtaining

information from respondents. • It is a structured technique that consists of a series of questions,

written or verbal, that a respondent answers. Objectives • It must translate the information needed into a set of specific

questions that the respondents can and will answer. • A questionnaire must uplift, motivate, and encourage the

respondent to become involved in the interview, to cooperate, and to complete the interview.

• A questionnaire should minimize response error.

MARK977 Research for Marketing Decisions T1 201834

18

Questionnaire design process

MARK977 Research for Marketing Decisions T1 201835

Questionnaire design process (cont.)

MARK977 Research for Marketing Decisions T1 201836

19

Information needed and interviewing method

• Specify the information needed – Remember the problem – Remember the solution – This guides the information needed

• Specify the interviewing method – Interviewing method influences questionnaire design – For example:

• Personal-in-home and mall-intercept interviews permit face to face interaction, hence complex questions can be asked

• In telephone interviews, no face to face interaction and respondents do not see questionnaire, hence questions have to be short and simple

• Mail questionnaires are self-administered, so questions must be simple and detailed instructions provided.

MARK977 Research for Marketing Decisions T1 201837

Individual question content: Is the question

necessary?

• If there is no satisfactory use for the data resulting from a question, that question should be eliminated.

• However, in some situations, questions not directly related to the information needed may be asked, for example: – Neutral questions at the beginning to establish

involvement and rapport – Filler questions to disguise purpose of project – Questions about competing brands (rather than the brand

of interest) to disguise sponsorship of the project

MARK977 Research for Marketing Decisions T1 201838

20

Individual Question Content: Are Several

Questions Needed Instead of One?

• Sometimes, several questions are needed to obtain the required information in an unambiguous manner. Consider the question,

“Do you think Sprite is a tasty and refreshing soft drink?” (Incorrect)

• Such a question is called a double-barreled question, because two or more questions are combined into one. To obtain the required information, two distinct questions should be asked:

“Do you think Sprite is a tasty soft drink?” and “Do you think Sprite is a refreshing soft drink?”

(Correct)

MARK977 Research for Marketing Decisions T1 201839

Overcoming inability to answer

MARK977 Research for Marketing Decisions T1 201840

21

Overcoming unwillingness to answer

MARK977 Research for Marketing Decisions T1 201841

Effort required of respondents Please list all the departments from which you purchased merchandise on your most recent shopping trip to a department store. (Incorrect)

In the list that follows, please check all the departments from which you purchased merchandise on your most recent shopping trip to a department store.

1. Women's dresses ____ 2. Men's apparel ____ 3. Children's apparel ____ 4. Cosmetics ____ . . . 16. Jewelry ____ 17. Other (please specify) ____ (Correct)

MARK977 Research for Marketing Decisions T1 201842

22

Sensitive information

• Respondents are unwilling to disclose, at least accurately, sensitive information because this may cause embarrassment or threaten the respondent’s prestige or self-image.

• Place sensitive topics at the end of the questionnaire. • Preface the question with a statement that the behaviour of

interest is common. • Ask the question using the third-person technique (see Chapter

4): Phrase the question as if it referred to other people. • Provide response categories rather than asking for specific

figures (e.g., age, income, profit).

MARK977 Research for Marketing Decisions T1 201843

Types of questions

MARK977 Research for Marketing Decisions T1 201844

23

Unstructured questions

• Unstructured questions are open-ended questions that respondents answer in their own words.

• Unstructured questions have a much less biasing influence on response than structured questions.

• A disadvantage is high potential for interviewer bias (e.g., some important information may be overlooked).

• Also, the coding of responses is costly and time consuming.

MARK977 Research for Marketing Decisions T1 201845

Structured questions

• Structured questions specify the set of response alternatives and the response format. A structured question may be multiple-choice, dichotomous, or a scale.

MARK977 Research for Marketing Decisions T1 201846

24

Multiple-choice questions • In multiple-choice questions, the researcher provides a

choice of answers and respondents are asked to select one or more of the alternatives given.

Do you intend to buy a new car within the next six months? ____ Definitely will not buy ____ Probably will not buy ____ Undecided ____ Probably will buy ____ Definitely will buy ____ Other (please specify)

MARK977 Research for Marketing Decisions T1 201847

Dichotomous questions

• A dichotomous question has only two response alternatives: yes or no, agree or disagree, and so on.

• Often, the two alternatives of interest are supplemented by a neutral alternative, such as “no opinion,” “don't know,” “both,” or “none.”

Do you intend to buy a new car within the next six months? _____ Yes _____ No

MARK977 Research for Marketing Decisions T1 201848

25

Choosing Question Structure Scales

• Scales were discussed in detail in Chapter 7: Do you intend to buy a new car within the next six months?

MARK977 Research for Marketing Decisions T1 201849

Definitely will not buy

Probably will not buy Undecided

Probably will buy

Definitely will buy

1 2 3 4 5

Choosing question wording: Define the issue

Define the issue in terms of who, what, when, where, why, and way (the six Ws). Who, what, when, and where are particularly important.

Which brand of shampoo do you use? (Incorrect)

Which brand or brands of shampoo have you personally used at home during the last month? In case of more than one brand, please list all the brands that apply.

(Correct)

MARK977 Research for Marketing Decisions T1 201850

26

Use ordinary words

“Do you think the distribution of soft drinks is adequate?” (Incorrect)

“Do you think soft drinks are readily available when you want to buy them?” (Correct)

MARK977 Research for Marketing Decisions T1 201851

Use unambiguous words

In a typical month, how often do you shop in department stores? _____ Never _____ Occasionally _____ Sometimes _____ Often _____ Regularly (Incorrect)

In a typical month, how often do you shop in department stores? _____ Less than once _____ 1 or 2 times _____ 3 or 4 times _____ More than 4 times (Correct)

MARK977 Research for Marketing Decisions T1 201852

27

Avoid leading or biasing questions • A leading question is one that clues the respondent to what the

answer should be, as in the following:

Do you think that patriotic Americans should buy imported automobiles when that would put American labor out of work? _____ Yes _____ No _____ Don't know (Incorrect)

Do you think that Americans should buy imported automobiles? _____ Yes _____ No _____ Don't know (Correct)

MARK977 Research for Marketing Decisions T1 201853

Dual statements – positive and negative

• Questions that are in the form of statements should be worded both positively and negatively.

MARK977 Research for Marketing Decisions T1 201854

28

Determining order of questions Opening Questions • The opening questions should be interesting, simple, and non-

threatening. Could be a screening/filter question.

Type of Information • As a general guideline, basic information should be obtained first

(e.g., questions capturing consumers’ lifestyle and attitude towards Apple), followed by classification (e.g., consumers’ age, income, gender), and finally, identification (e.g., business card information) information.

Difficult Questions • Difficult questions or questions which are sensitive, embarrassing,

complex, or dull, should be placed late in the sequence. Build rapport and set the scene first, then ask difficult questions (e.g., information about credit card debt, age, income) at the end.

MARK977 Research for Marketing Decisions T1 201855

Determining order of questions

Effect on Subsequent Questions • General questions should precede the specific questions

(funnel approach).

Q1: “What considerations are important to you in selecting a department store?” (e.g., characteristics important in a soft drink)

Q2: “In selecting a department store, how important is convenience of location?” (e.g., characteristics of Coca Cola)

(Correct)

MARK977 Research for Marketing Decisions T1 201856

29

Determining order of questions

Logical Order

• General to specific

• Questions that deal with a particular topic (e.g., Honda cars) should be asked before beginning a new topic (e.g., Samsung electronics)

• Transition should be smooth and respondents should be properly guided to switch their train of thought

• If branching questions (e.g., if “no” please go to Q23) are in place then ensure skip patterns and instructions for respondents are clear

MARK977 Research for Marketing Decisions T1 201857

General ordering of questions

MARK977 Research for Marketing Decisions T1 201858

Question Type Nature Function Example

Qualifying/

Screening

Questions

Focus on

respondent

inclusion criteria

To determine if a

respondent is eligible

to participate in the

survey

Who in your household

does most of the

shopping for groceries?

Introductory

Questions/ warm-

ups

Broad, easy

questions

To break the ice and

put the respondent at

ease

How often do you shop

for groceries?

Main Questions:

Easy

Related to the

information

needed but easy

to answer

To focus on the

survey topic and

reassure the

respondent that

survey is easy

How important Is each of

the following factors in

selecting a supermarket?

30

General ordering of questions (cont.)

MARK977 Research for Marketing Decisions T1 201859

Question Type Nature Function Example

Main questions:

More difficult

Related to the

information needed but

may be difficult to

answer

To obtain the rest of the

information needed

How would you rank order

the following eight

supermarkets in terms of

your preference to shop?

Psychographics/

Lifestyles

Not relevant in all

surveys

To obtain personality

related information

Please indicate your degree

of dis/agreement with the

following statements

Demographics Personal information To classify the

respondents What is your occupation?

Identification

Information

Name, address,

telephone

To identify the

respondent Name:

Form and layout

• Divide a questionnaire into several parts (e.g., Section A for opening questions, Section B for basic information, Section C for difficult questions, Section D for classification information).

For ease of coding and analysis:

• The questions in each part should be numbered, particularly when branching questions are used (also to guide respondents through skip patterns).

• The questionnaires should preferably be pre-coded (e.g., 1=under 18, 2=18 and 24, 3=25 and 30, etc.).

• The questionnaires themselves should be numbered serially (e.g., Q1.6, Q2.3, etc.).

MARK977 Research for Marketing Decisions T1 201860

31

Pretesting Pretesting refers to the testing of the questionnaire on a small sample of respondents to identify and eliminate potential problems.

• A questionnaire should not be used in the field survey without adequate pretesting.

• All aspects of the questionnaire should be tested, including question content, wording, sequence, form, layout, difficulty, and instructions.

• The respondents for the pretest and for the actual survey should be drawn from the same population.

• Pretests are best done initially by personal interviews followed by the actual survey method if that is different.

• The pretest sample size varies from 15 to 30 respondents for each wave (until no further changes are needed).

MARK977 Research for Marketing Decisions T1 201861