Astronomical point in run chart

Institute for Healthcare Improvement ● ihi.org Page 1

Patient Experience Data Self-Assessment

1. What is a run chart?

A plot that shows patient experience values (e.g., willingness to recommend) in time order, with a median

reference line, is an example of a run chart. You can interpret the run chart with a set of rules. The rules

help you to detect patterns that may be signals that your system has improved or deteriorated.

 True

 False

 Not sure

2. The median of the HCAHPS Nursing Communications “top box” scores in the

accompanying chart…

a. Is 78

b. Is 77

c. Can not be determined from the graph

72

73

74

75

76

77

78

79

80

81

82

Sep-10 Oct-10 Nov-10 Dec-10 Jan-11 Feb-11 Mar-11 Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11

Nursing Communications

Good

Institute for Healthcare Improvement ● ihi.org Page 2

3. Using the standard run chart rules for shift, trend, and “astronomical point,” examine

each chart for signs of improvement or degradation. The dashed line is the median

value to be used for reference. The first chart uses a median based on the first 12

months, also known as a “baseline reference” median.

60

65

70

75

80

85

90

95

100

Jan-10 Mar-10 May-10 Jul-10 Sep-10 Nov-10 Jan-11 Mar-11 May-11 Jul-11 Sep-11

p e

r c e

n t

Friendliness of nurses/ambulatory

40

45

50

55

60

65

70

75

80

Jan-10 Mar-10 May-10 Jul-10 Sep-10 Nov-10 Jan-11 Mar-11 May-11 Jul-11 Sep-11

p e

r c e

n t

Overall Quality of Care

60

65

70

75

80

85

90

95

100

Jan-10 Mar-10 May-10 Jul-10 Sep-10 Nov-10 Jan-11 Mar-11 May-11 Jul-11 Sep-11

p e

r c e

n t

Did everything to help your pain

B

A

C

Institute for Healthcare Improvement ● ihi.org Page 3

4. Interpreting a New Month’s Value

At your hospital, suppose the HCAHPS overall “top box” willingness to recommend* data shows an

average monthly performance of 80% for the past 12 months; the median monthly value is also 80%.

Now you get the latest month’s data — the value is 76%.

Circle “Yes” if the action is always justified as part of your reaction to this new month’s value of 76%.

a. Plot the 76% value on a run chart of “willingness to recommend” to see if

there is any signal over time according to the run chart rules.

Yes No Not

sure

b. Look at the “willingness to recommend” data in the context of the other

elements of the HCAHPS survey.

Yes No Not

sure

c. Re-iterate to staff the importance of continuous improvement and the

financial implications of a relatively poor score for “willingness to

recommend.”

Yes No Not

sure

d. Initiate an investigation to find out what changed in the most recent month

that caused the decrease as this month is below average.

Yes No Not

sure

e. Interpret the 76% value in terms of the number of patients surveyed. Yes No Not

sure

*On average over the past 12 months, 80% of respondents have answered “Definitely yes” to the

HCAHPS question, “Would you recommend this hospital to your friends and family?”

Institute for Healthcare Improvement ● ihi.org Page 4

5. Interpreting HCAHPS Percentile Tables

Use the table below to answer the questions on the following page.

(source: http://www.hcahpsonline.org/files/January%202012%20HCAHPS%20Percentiles%20Table.pdf)

http://www.hcahpsonline.org/files/January%202012%20HCAHPS%20Percentiles%20Table.pdf
Institute for Healthcare Improvement ● ihi.org Page 5

Circle “T” for true, “F” for false, or “?” for Can’t tell from the Information provided. The questions pertain

to the hospitals described in the note marked by the red arrow.

a. If your hospital averaged 83 from April 2010 to March 2011 on the top box

Communications with Nurses composite, your hospital had a score lower

than more than 150 hospitals nationally.

T F ?

b. If your hospital averaged 83 from April 2010 to March 2011 on the top box

Communications with Nurses composite, on average 17% of your hospital’s

patients who responded to the survey feel they did not experience “top

box” communications.

T F ?

c. On the most recent HCAHPS monthly data report, your hospital scored 78

on the top box Recommend the Hospital. That means your hospital

outperformed more than 75% of the surveyed hospitals.

T F ?

d. If Hospital A averaged a score of 6 points higher than Hospital B on top box

Overall Hospital Rating from April 2010 to March 2011, then in percentile

terms, Hospital A is at least 15% higher than Hospital B.

T F ?

e. The national HCAHPS survey response rate for the period April 2010-March

2011 was 32%. This means that about 2 out of 3 patients contacted did

not complete the survey questions.

T F ?

f. It always makes sense to compare ratings and percentiles of specific

departments or services (e.g. OB) rather than rely only on whole-hospital

summary measures.

T F ?

Institute for Healthcare Improvement ● ihi.org Page 6

6. Correlations: Friend or Foe? Use this table of national survey data for overall rating of inpatient experience to answer the questions

below. Circle “T” for true, “F” for false, or “?” for Can’t tell from the Information provided. The two digit

number in each row is the correlation between the overall rating and the rating of the item in the row,

e.g. the correlation between “Skill of physician” and “overall rating of inpatient experience” is 0.63.

Source: Press Ganey National database – through June 30, 2011

Questions

a. The correlation value in each row can’t be any larger than 1 but could be

zero or even negative.

T F ?

b. If your aim is to improve the overall rating of in-patient experience at your

hospital, you should concentrate improvement on staff sensitivity to

inconvenience and staff actions to address emotional needs.

T F ?

c. If the national correlations align well with survey data at your hospital,

then reduction of noise level in and around room is not as likely to

improve your overall rating compared to staff including patients in

decisions re: treatment.

T F ?

d. The table implies that in terms of national summary data, room

cleanliness doesn’t really matter in terms of overall rating.