Evaluating the Effectiveness of Treatment
Presented by Jennifer Chase and Johnetta Chavis
"Individuals shall evaluate the effectiveness of services rendered and of products dispensed..."
The American Speech-Language Hearing Association
The World Health Organization (WHO) views an individual's circumstances along the dimensions of body function and structure, activity, and participation. Thus, because of a problem or impairment in body function or structure (e.g., hearing loss), an individual may or may not have difficulty in the performance of activities (e.g., communicating, listening, speaking). Participation refers to an individual's involvement in life situations and society's response or reaction to the individual's level of functioning. For example, for someone with a hearing loss we would ask "How does the person's difficulty with the activities of hearing restrict participation in life situations"?
Looking at disorders from the perspective of the World Health Organization helps us understand that each person's disability is different and affects each one differently in terms of what the person can and cannot do.
Treatment effectiveness involves the extent to which services are shown to be beneficial under typical (or real world) conditions.
It is important to evaluate a client's therapy throughout the process and at the end of treatment. In order to accurately measure effectiveness over time, a baseline measure should be taken before any intervention. A baseline is a line serving as a basis for measurement used for comparison of future progress. A midterm measurement should be taken, and then a final evaluation should be completed at the end of treatment and before discharging the client.
In evaluating progress over the course of treatment we must study the outcome. An outcome is simply a result of an intervention. It may also be defined as the likelihood of observing a change in performance between two points in time.
Measuring Outcomes of a Progressive Motor Speech Disorder
Parkinson's disease is a progressive disorder that affects the Central Nervous System. There is no cure for the disease, but it is caused by a combination of genetic predisposition and environmental factors. Parkinson's disease is directly related to a loss of cells in the substania nigra, located in the brain. This causes a decrease in dopamine, which is responsible for transmitting signals to the brain. Communication disorders become more severe as the disease progresses. With a disease such as Parkinson's, which is treated with medication and incurable, the outcome is focused on the pathology level of the disease. When assessing pathology levels, chemical levels and physiological test of tremor, strength, and rate of movement are involved. There are four major symptoms of Parkinson's disease: rigidity of limbs, tremor of limbs (hands), postural instability (impaired refluxes) and bradykinesia (slow movement). Levodopa (L-dopa) is used to help a person function with Parkinson's disease. L-dopa replaces the dopamine as the disease progresses. It allows improved mobility and speech functioning but does not prevent the disease from progressing. L-dopa will cause an increase in norepinephrine, which is associated with adrenalin (fight or flight response). L-dopa is needed for individuals to function on a daily basis, but there are severe side effects. Extended use of L-dopa leads to Parkinson's disease paranoia including hallucinations and can trigger aggression. Speech performance should be measured on the progression and severity of the disease depending on communication deficits, which vary extensively from person to person. The table below shows the typical pattern of digression of a Parkinson’s patient. As the disease progresses, speech decreases until no vocalization is evident. For more information on Parkinson's disease and its effect on communication, go to www.apdaparkinson.com.
Summary of Speech Severity Scales for Amytrophia Lateral Sclerosis & Parkinson's Disease:
| Scale Score |
ALS Amytrophic Lateral Sclerosis |
PD Parkinson's Disease |
| 10 | Normal Speech | Normal Speech |
| 9 | Nominal speech abnormality | Speech entirely adequate; minor voice disturbances present |
| 8 | Perceived speech changes | Speech easily understood, but voice rhythm may be disturbed |
| 7 | Obvious speech abnormalities | Communication accomplished with ease, although speech impairment detracts from content |
| 6 | Repeats messages on occasion | Speech can always be understood if listener pays close attention and; both voice and articulation and voice may be defective |
| 5 | Frequent repetition required | Speech always employed for communication, but articulation is very poor; usually uses complete sentences |
| 4 | Speech plus augmentative communication | Uses speech for most communication, but articulation is highly unintelligible; may have occasional difficulty in initiating speech; usually speaks in single words or short phases |
| 3 | Limits speech to one-word response | Attempts to use speech for communication, but has difficulty in intitinating vocalization; may stop speaking in middle of phrases and be unable to continue. |
| 2 | Vocalizes for emotional expression | Vocalizes to call attention to self |
| 1 | Nonvocal |
Vocalizes, but rarely for communicative purposes
|
| 0 | Does not vocalize at all |
Adapted from Yorkston, Miller, & Strand, 1995.
There are multiple methods for measuring the results of intervention.
Below are some examples of methods one might use when working with children:
On a daily basis, direct observation data are helpful to determine the appropriateness of treatment elements. This data can also help "drive" relatively short term changes.
The percentage of objectives mastered, and their corresponding developmental equivalent (the age at which typically developing children first demonstrate these skills) reveal the rate at which each child is progressing.
Videotapes of the child in naturalistic settings provide yet another measure (though changes are often difficult to quantify). Structured Interviews with the child and family members also offer another perspective on treatment effectiveness.
Another method involves looking at the results from testing (standardized or non-standardized) in the areas of cognitive abilities, speech and language, and adaptive functioning. Once again, the client should participate in testing prior to the start of treatment (baseline) and as needed throughout the treatment process, and at the completion of intervention.
Outcomes can be...
Clinically derived (ability to sustain phonation, integrity of the swallowing mechanism)
Functional (ability to communicate basic needs, telephone use)
Social (employability, ability to learn, community reentry)
Client defined (satisfaction with services, quality of life)
Why monitor progress?
1. Determines the effectiveness of the intervention
-progress related to curriculum & intervention strategies
-signals need for modifications
-reduces time spent stagnating
2. Provides continuous feedback
-patients and families receive specific information about progress
3. Program accountability
-law requires goals must be specified in IEP/IFSP
-helps explain progress or lack thereof
-keeps program staff aware of effectiveness and quality of the program
What to monitor?
Rate - how many times the patient performs a skill or behavior
Length - how long the patient engages in a behavior
Latency - the time it takes the patient to perform a skill after an antecedent has been provided
Topography - what a particular skill or behavior looks like
Force - the intensity of the behavior
Measurement will never replace our keen observations or our sharpened professional sense of both the needs and gains of those whom we serve. In the end, to lose the art of behavioral observation is to lose the human sense of connectedness with our clients, our students, and ultimately ourselves. -Carol M. Frattali, Author, Measuring Outcomes in Speech Language Pathology
References
The American Speech-Language Hearing Association
Measuring Outcomes in Speech-Language Pathology, Frattali, Carol M. (New York 1998.)
American Parkinson Disease Association