Stroke rehabilitation assessment of movement stream

A Simplified Stroke Rehabilitation Assessment of Movement Instrument

Background and Purpose. An efficient, reliable, and valid instrument for assessing motor function in patients with stroke is needed by both clinicians and researchers. To improve administration efficiency, we applied the multi- dimensional Rasch model to the 30-item, 3-subscale Stroke Rehabilitation Assessment of Movement (STREAM) instrument to produce a concise, reliable, and valid instrument (simplified STREAM [S-STREAM]) for measur- ing motor function in patients with stroke. Subjects and Methods. The STREAM (consisting of 3 subscales: upper-limb movements, lower-limb move- ments, and mobility) was administered to 351 subjects with first stroke occurrence and a median time after stroke of 12.5 months. The unidimen- sionality of each subscale of the STREAM first was verified with unidimen- sional Rasch analysis. Each subscale of the STREAM then was simplified by deleting redundant items on the basis of expert opinion and the results of the Rasch analysis. The Rasch reliability of the S-STREAM and the concurrent validity of the S-STREAM with the STREAM were examined with multidimen- sional Rasch analysis and the intraclass correlation coefficient (ICC), respec- tively. Results. After deleting the items that did not fit the Rasch model, we found that the 8-item upper-limb movement subscale, the 9-item lower-limb movement subscale, and the 10-item mobility subscale assessed single, uni- dimensional upper-limb movements, lower-limb movements, and mobility, respectively. We selected 5 items from each subscale to construct the S-STREAM and found that the reliability of each subscale of the resulting simplified instrument was high (Rasch reliability coefficients of �.91). The agreement between the subscale scores (Rasch estimates) of the S-STREAM and those of the STREAM was excellent (ICC of �.99, with a lower limit for the 95% confidence interval of �.985), indicating good concurrent validity of the S-STREAM with the STREAM. Discussion and Conclusion. The S-STREAM demonstrates high Rasch reliability, unidimensionality, and concurrent valid- ity with the STREAM in patients with stroke. Furthermore, the S-STREAM is efficient to administer, as it consists of only half the number of items in the original STREAM. Additional studies to examine other psychometric proper- ties (eg, predictive validity and responsiveness) of the S-STREAM or its psychometric properties in various recovery stages after stroke are needed to further establish its utility in both clinical and research settings. [Hsueh IP, Wang WC, Wang CH, et al. A simplified stroke rehabilitation assessment of movement instrument. Phys Ther. 2006;86:936 –943.]

Key Words: Motor function, Psychometrics, Rasch model, Stroke.

I-Ping Hsueh, Wen-Chung Wang, Chun-Hou Wang, Ching-Fan Sheu, Sing-Kai Lo, Jau-Hong Lin, Ching-Lin Hsieh

936 Physical Therapy . Volume 86 . Number 7 . July 2006

Re se

ar ch

Re po

rt �

����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������

��� ���

��� ��

D ow

nloaded from https://academ

ic.oup.com /ptj/article/86/7/936/2805160 by guest on 10 M

arch 2021

M otor and mobility problems are very com- mon after stroke.1 For the purposes of treatment planning and outcome assess- ment, it is important to reliably and accu-

rately assess motor function in patients with stroke.2 Although a number of assessment tools are available to measure the recovery of movement after stroke, they have rarely been used in clinical practice, partly because of the lengthy administration time and complexity of scoring.3 A reliable, valid, and efficient instrument for the assessment of motor function in patients with stroke is needed by both clinicians and researchers.

The Stroke Rehabilitation Assessment of Movement (STREAM) instrument was designed to provide a com- prehensive and quantitative evaluation of voluntary movements (ie, an impairment measurement) and basic mobility (ie, a disability measurement) in patients with stroke.4 The STREAM consists of three 10-item sub- scales: upper-limb movements, lower-limb movements, and mobility. The psychometric characteristics of the STREAM have been shown by classical test theory to be satisfactory.2– 6 The STREAM is preferred over other related impairment or disability measures (eg, the Box and Block Test, the Berg Balance Scale, gait speed testing, the Timed “Up & Go” Test, and the Barthel Index) for monitoring recovery from a stroke at the acute stage, as those measures appeared not to focus on the goals of immediate therapy during this period.5 Furthermore, those measures had limited abilities to discriminate or evaluate patients with stroke because the

Box and Block Test, the Berg Balance Scale, gait speed testing, and the Timed “Up & Go” Test showed floor effects in patients with severe stroke, whereas the Barthel Index showed ceiling effects in patients with mild stroke.5,7–9

However, the 3 subscales of the STREAM have never been tested for unidimensionality (one type of construct validity); such testing is required to justify the summa- tion of scores to quantify motor function in patients with stroke. Only items measuring the same, unique dimen- sion (construct) should be retained in a measure. Fur- thermore, the extremely high internal consistency of the STREAM (ie, the Cronbach alpha value was found to be as high as .98 for each of the subscales)3 indicates possible redundancy among the items. These observa- tions suggest the potential for shortening the STREAM.

Standard Rasch analysis enables the examination of whether items from a scale constitute a unidimensional construct10,11 so as to construct a concise scale.12 How- ever, when an instrument consisting of more than one subscale (eg, the STREAM) is to be calibrated, it is inefficient to apply the standard unidimensional Rasch model separately to each subscale. The unidimensional approach ignores correlations between latent traits (ie, the constructs of the subscales) and thus may yield imprecise measurements of the construct (or character- istic) to be measured, especially when the subscales are short. On the other hand, the multidimensional Rasch model simultaneously calibrates all subscales and there-

IP Hsueh, OT, MA, is Assistant Professor, School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan.

WC Wang, PhD, is Professor, Department of Psychology, Chung-Cheng University, Chiayi, Taiwan.

CH Wang, PT, BS, is Professor, School of Physical Therapy, College of Medical Technology, Chung-Shan Medical University, and Department of Physical Therapy, Chung-Shan Medical University Rehabilitation Hospital, Taichung, Taiwan.

CF Sheu, PhD, is Professor, Institute of Cognitive Science, National Cheng Kung University, Tainan, Taiwan.

SK Lo, PhD, is Professor, Faculty of Health and Behavioral Sciences, Deakin University, Melbourne, Australia.

JH Lin, PT, PhD, is Professor, Faculty of Physical Therapy, Kaohsiung Medical University, Kaohsiung, Taiwan.

CL Hsieh, OT, PhD, is Professor and Chair, School of Occupational Therapy, College of Medicine, National Taiwan University, and Department of Physical Medicine and Rehabilitation, National Taiwan University Hospital, 4F, 17 Shiujou Rd, Taipei 100, Taiwan (mike26@ha.mc.ntu.edu.tw). Address all correspondence to Dr Hsieh.

Ms Hsueh and Dr Hsieh provided concept/ideas/research design, writing, and fund procurement. Dr WC Wang provided data analysis and facilities/equipment. Dr Hsieh provided project management. Professor CH Wang provided institutional liaisons. Dr WC Wang, Professor CH Wang, Dr Sheu, Dr Lo, and Dr Lin provided consultation (including review of manuscript before submission).

The research protocol was approved by local institutional review boards.

This study was supported by research grants from the National Science Council (NSC 93-2314-B-002-033 and NSC 94-2314-B-002-078) and the National Health Research Institute (NHRI-EX94-9204PP).

This article was received April 16, 2005, and was accepted January 30, 2006.

Physical Therapy . Volume 86 . Number 7 . July 2006 Hsueh et al . 937

D ow

nloaded from https://academ

ic.oup.com /ptj/article/86/7/936/2805160 by guest on 10 M

arch 2021

fore uses the correlations to increase measurement precision.13,14 Theoretically, it may be difficult to con- ceive of constructs that are independent in the movement domains after stroke. Therefore, the multidimensional Rasch model takes into account the between-subscale correlations to increase measurement precision: the higher the correlations, the greater the measurement precision.15,16 In other words, short subscales, if moder- ately correlated, still can yield precise measurements with the multidimensional approach. Because the 3 subscales of the STREAM are highly correlated with each other,2 the multidimensional approach can be useful in simplifying the STREAM.

To improve administration efficiency, we aimed to shorten the 30-item, 3-subscale STREAM to produce a simplified STREAM (S-STREAM) by using the multi- dimensional Rasch model. We examined the psychomet- ric properties of the S-STREAM (including Rasch reli- ability, unidimensionality, and concurrent validity with the STREAM) in subjects with stroke.

Method

Subjects Subjects with a broad range of motor deficits were recruited from the rehabilitation departments of 5 hos- pitals in northern, central, southern, and eastern Taiwan between October 2003 and January 2004. These rehabil- itation departments provide both inpatient and out- patient services (including physical therapy, occupa- tional therapy, and speech therapy). Subjects were included in the study if they met the following criteria: diagnosis (International Classification of Diseases, 9th Revi- sion, Clinical Modification [ICD-9] codes)17 of cerebral hemorrhage (ICD-9 code 431) or cerebral infarction (ICD-9 code 434), absence of other major diseases (eg, tumors, arthritis) or impairments (eg, amputations, frac- tures) that would reduce or limit a subject’s ability to perform movements, and ability to follow 2-step instruc- tions. Only subjects who had had their first stroke and were able to give informed consent personally or by proxy were included in the study.

Procedure The STREAM, with instructions in Chinese, was admin- istered by the same physical therapist to all of the participants in the 5 rehabilitation departments. The intrarater reliability of data obtained by the physical therapist was satisfactory (intraclass correlation coeffi- cient [ICC] of .94). Demographic characteristics and comorbidity data for the participants were collected from medical records.

Instrument Items of the STREAM3 for voluntary movements of the limbs are scored on a 3-point scale (0�unable to per- form the test movement, 1�able to perform the test movement only partially, and 2�able to complete the test movement). Mobility items are scored on a 4-point scale (0�unable to perform the test movement, 1�able to perform the test movement only partially, 2�able to complete the test movement with a mobility aid, and 3�able to complete the test movement without an aid). Thus, each of the 10-item limb movement subscales was scored out of 20 points, and the 10-item mobility sub- scale was scored out of 30 points.

Data Analysis The unidimensionality of the 3 subscales of the STREAM was examined with WINSTEPS.18 The variance- covariance matrix (and the correlation matrix) for the 3 latent traits (ie, the constructs of the 3 subscales of the STREAM) was computed with ConQuest,19 which was developed for the multidimensional random-coefficients multinomial logit model (MRCMLM).13 A brief descrip- tion of the MRCMLM is given in the Appendix.

To examine the unidimensionality of each subscale, infit and outfit statistics were used to examine whether the data fit the expectation of the Rasch rating scale model (RSM). The infit mean square (MNSQ) is sensitive to unexpected behavior affecting responses to items near the subject’s proficiency measure (eg, motor status); the outfit MNSQ is sensitive to unexpected behavior on items far from the subject’s motor status. Items with infit or outfit MNSQ values of greater than 1.4 indicate potential misfits.20 The MNSQ can be transformed to a standardized z value (ZSTD) which, for large samples, follows approximately the standard normal distribution when the items fit the expectation of the model. Items with both infit and outfit ZSTD values beyond �2.58 (twice the tailed area of the normal curve above or below �2.58 is 0.01) were considered to have poor fit.

In addition, when items fit the expectation of the model, the residuals (observed scores minus expected scores) should be distributed randomly. A principal compo- nents analysis was conducted to determine whether any dominant component existed among the residuals. If dominant components were found, then the unidimen- sionality assumption was violated.

Rasch reliability, which can be viewed as the counterpart of classical test reliability (eg, the Cronbach alpha), was calculated.10,20 Reliability coefficients of greater than .7 were considered good for group comparisons, whereas those greater than .9 were considered good for individ- ual comparisons.21

938 . Hsueh et al Physical Therapy . Volume 86 . Number 7 . July 2006

��� ���

��� ���

��� ���

��� ��

D ow

nloaded from https://academ

ic.oup.com /ptj/article/86/7/936/2805160 by guest on 10 M

arch 2021

The appropriateness of the scoring levels in each item of the STREAM was investigated with the RSM. The RSM is useful for polytomous items in a scale that share the same rating scale structure (eg, all items are rated 0, 1, or 2). Estimates of the threshold difficulty between the adjacent scoring levels can be used to examine the appropriateness of the scoring points of a test.20 If disorderings of the step difficulty (ie, the difficulty of a higher step was lower than that of its adjacent lower step) between any 2 adjacent levels were found, then the levels of scaling of the items might be reorganized to achieve suitable scaling.

After the unidimensionality and appropriate scoring levels in each item of the STREAM were established, we attempted to reduce the length of the test further while maintaining its psychometric properties. Each of the 3 subscales of the STREAM was shortened to produce the S-STREAM on the basis of 2 criteria: content represen- tativeness, assessed by a panel of therapists (2 physical therapists and 2 occupational therapists who each had more than 10 years of experience in stroke rehabilita- tion); and difficulty diversity, that is, even scattering of the difficulties of the selected items over the range of the difficulty continuum.

For each subject, the multidimensional form of the RSM can provide estimates for the 3 subscale scores for both the S-STREAM and the STREAM. We used the RSM estimates for each subscale of the STREAM as the gold standard in this study. Because the Rasch estimates for each subscale have different score ranges, all estimates were linearly transformed to a range of 0 to 100 to facilitate comparisons. The relationship and agreement among corresponding Rasch estimates for subscale scores (ie, the concurrent validity of the S-STREAM with the STREAM) were examined with the Pearson correla- tion coefficient (r) and the ICC(3,1),22 respectively. Correlation coefficients of greater than .6 indicate acceptable concurrent validity.23

Results A total of 351 subjects with a median time after stroke of 12.5 months met the selection criteria and agreed to participate in the study. The participants had a wide range of motor deficits, and their scores were found to be scattered throughout the entire ranges of the STREAM subscales. The clinical characteristics of the study participants are shown in Table 1.

Unidimensional Rasch analyses of the 3 subscales, sepa- rately, revealed that 2 items (scapular elevation and opposition) in the upper-limb movement subscale and 1 item (hip abduction) in the lower-limb movement sub- scale did not fit the expectation of the model (both infit and outfit ZSTD values of beyond �2.58). These 3 items

were removed from the instrument in subsequent anal- yses. Thereafter, the 8-item upper-limb movement sub- scale and the 9-item lower-limb movement subscale fit the expectation of the model (infit and outfit ZSTD values within the range of �2.58). In addition, none of these items had infit and outfit MNSQs of greater than 1.4. Principal components analysis revealed that no dominant component existed among the residuals of the Rasch-transformed scores for the 8-item upper-limb movement subscale, the 9-item lower-limb movement subscale, or the 10-item mobility subscale. These results indicate that the 8-item upper-limb movement subscale, the 9-item lower-limb movement subscale, and the 10-item mobility subscale assess single, unidimensional upper-limb movements, lower-limb movements, and mobility, respectively.

A multidimensional analysis with ConQuest was per- formed for the remaining 27 items (ie, 8 items from the upper-limb movement subscale, 9 items from the lower- limb movement subscale, and 10 items from the mobility subscale). Table 2 shows the correlation matrix for the

Table 1. Clinical Characteristics of Subjects With Stroke (N�351)

Characteristica Value

Sex (no. of men/women) 222/129

Age, y, median (25th–75th percentiles) 63 (53–71)

Month after onset, median (25th–75th percentiles)

12.5 (4–30)

Diagnosis, no. (%) of subjects Cerebral hemorrhage 113 (32) Cerebral infarction 238 (68)

Side of paresis, no. (%) of subjects Right 175 (50) Left 176 (50)

STREAM score, median (25th–75th percentiles) Upper-limb movement raw score 9 (0–17) Lower-limb movement raw score 7 (3–14) Mobility raw score 15 (8–21)

S-STREAM score,b mean (SD) Upper-limb movement score 48.8 (26.4) Lower-limb movement score 48.3 (24.0) Mobility score 49.2 (22.9)

a STREAM�Stroke Rehabilitation Assessment of Movement, S-STREAM� simplified STREAM. b Rasch-transformed score ranging from 0 to 100.

Table 2. Correlation Matrix for the Stroke Rehabilitation Assessment of Movement With the Multidimensional Approach

Subscale Upper-Limb Movement

Lower-Limb Movement

Lower-limb movement .90 Mobility .78 .84

Physical Therapy . Volume 86 . Number 7 . July 2006 Hsueh et al . 939

D ow

nloaded from https://academ

ic.oup.com /ptj/article/86/7/936/2805160 by guest on 10 M

arch 2021

STREAM, which revealed that the underlying latent traits of the subscales of the STREAM were highly correlated, with Pearson coefficients of between .78 and .90. Table 3 shows that the Rasch reliability for the 3 subscales was good (reliability coefficients of �.86).20 Moreover, the 3 subscales of the STREAM showed better reliability when the multidimensional approach was used (reliability coefficients of �.93) than when the uni- dimensional approach was used (reliability coefficients of �.86).

We selected items that fit the RSM to construct the S-STREAM. To avoid possible floor or ceiling effects, we generally retained the most difficult and easiest items for each subscale. The only exception was that the most difficult item of the mobility subscale, 3 steps backward, was not selected because it apparently cannot be classi- fied as a daily mobility activity. The second most difficult item, walking down 3 stairs, was selected instead. These 2 items were similar in difficulty (Tab. 4). Furthermore, 3 items with an intermediate degree of difficulty were selected for each subscale on the basis of expert opinion and the results of Rasch analysis. We first selected items that were scattered evenly over the range of the difficulty continuum. Some items of similar difficulty (eg, “making a fist” and “moving hand to sacrum” in the upper-limb movement subscale, “knee flexion while sitting” and “knee flexion while standing” in the lower-limb move- ment subscale, and “10-m walk” and “3 steps to the affected side” in the mobility subscale) were selected by the panel of therapists. “Making a fist” was used to provide 2 items, in total, measuring hand function for the upper-limb movement subscale. “Knee flexion while standing” was not used because some subjects tended to flex their hips simultaneously, making the rating diffi- cult. “Three steps to the affected side” was not used because it is obviously not a daily activity, compared with “10-m walk.” The final 5 items used for each subscale of the S-STREAM are shown in Table 4.

Unidimensional and multidimensional analyses were conducted on the 15-item version of the S-STREAM. Table 4 shows the 2 kinds of item parameter estimates (ie, difficulty logit and standard error) for the 3 sub- scales of the 27-item STREAM and the S-STREAM with

the multidimensional approach. These 2 versions of the instrument had similar estimates for corresponding items.

The threshold difficulty estimates within each subscale were rather far apart (�2.18 logits). In addition, the ordering of the threshold difficulty estimates was not reversed.

Table 3 shows that the use of the multidimensional approach with the S-STREAM resulted in high test reliability (Rasch reliability coefficients of �.91) for the 3 subscales. These results indicate that the 3 subscales of the S-STREAM can yield very precise estimates for indi- vidual subjects. When the unidimensional approach was used, the test reliability values were .85, .88, and .94 for the upper-limb movement, lower-limb movement, and mobility subscales of the S-STREAM, respectively.

The agreement between each pair of subscales was excellent (transformed scores of 0 –100), with ICCs (95% confidence intervals) of .99 (.993–.995), .99 (.989 – .993), and .99 (.985–.990), for the upper-limb move- ment, lower-limb movement, and mobility subscales, respectively. Furthermore, the Pearson correlation co- efficients for the multidimensional Rasch estimates for the STREAM and the S-STREAM were all .99 for the 3 subscales. These results indicate that each subscale of the S-STREAM demonstrates high concurrent validity with the corresponding subscale of the STREAM.

Discussion To the best of our knowledge, this study is the first to use the multidimensional approach to produce a concise measure of motor function for people with stroke. The 15-item S-STREAM was constructed on the basis of the original STREAM, expert opinion, and results of Rasch analysis. The S-STREAM contains only half the number of items in the original STREAM and shows sound reliability and validity.

There are 2 major advantages of using the S-STREAM. First, it is simple and quick to use for patients with stroke compared with the original STREAM. As the S-STREAM contains only half the number of items in the original

Table 3. Rasch Reliability for the 3 Subscales of the Stroke Rehabilitation Assessment of Movement (STREAM) and the Simplified STREAM (S-STREAM) With the Unidimensional and Multidimensional Approaches

Subscale

27-Item STREAM S-STREAM

No. of Items

Reliability No. of Items

Reliability

Unidimensional Multidimensional Unidimensional Multidimensional

Upper-limb movement 8 .86 .93 5 .85 .91 Lower-limb movement 9 .91 .96 5 .88 .93 Mobility 10 .97 .98 5 .94 .95

940 . Hsueh et al Physical Therapy . Volume 86 . Number 7 . July 2006

��� ���

��� ���

��� ���

��� ��

D ow

nloaded from https://academ

ic.oup.com /ptj/article/86/7/936/2805160 by guest on 10 M

arch 2021

STREAM, the 15-item S-STREAM can be administered within 10 minutes, that is, half the time required to administer the original STREAM. Rapid assessment is a clinically important feature of this simplified version of the STREAM, as long tests can take a substantial amount of time to complete and may place unreasonable demands upon the respondents, especially in instances in which they may be seriously unwell, as in the case of stroke. Rapid and accurate assessment of functional outcomes in patients with stroke therefore will provide benefits to both clinicians and patients.

A second advantage of using the S-STREAM is that the Rasch estimates for the 3 subscales can be viewed as interval-level measurements.10 In contrast, most mea-

sures currently used in the assessment of patients with stroke use ordinal-level measurements. For an ordinal scale, a given difference in scores at one point on the scale does not necessarily represent the same amount of functional change as an identical difference at another point on the scale.24 Interval scores, rather than ordinal scores, can provide a more precise reflection and better resolution of disease impact, differences between indi- viduals and groups, and treatment effects.25 Further- more, an ordinal scale precludes the use of standard parametric statistical inferences. Because most statistical techniques assume that the data are at least on an interval scale, the Rasch estimates for the S-STREAM are recommended for future applications.

Table 4. Item Parameter Estimates for the Stroke Rehabilitation Assessment of Movement (STREAM) and the Simplified STREAM (S-STREAM) With the Multidimensional Approach

Itema 27-Item STREAM 15-Item S-STREAM