Outcome Measures in Audiology: Knowing We've Made a Difference

Introduction:

The greatest thrill I get as a clinician is having patients tell me what a difference I've made in their lives. I'm often told how my efforts have made it possible for them to hear family members, enjoy TV and movies, go out with friends and participate in the conversation and laughter, hear the birds, the insects, and the rain. Although these comments are what you and I call professional validation and personal gratification, it's what the insurers, health maintenance organizations (HMOs), and accrediting agencies call 'not good enough'.

Throughout healthcare today, there is an increasing emphasis on objectively measuring, demonstrating, and documenting the benefits, or outcomes, achieved through intervention. The instruments used to assess these benefits are known as outcome measures. As more third party payers add hearing aid coverage to their plans and as new technology drives up the cost of hearing instruments, the need to demonstrate benefit becomes increasingly important. Other reasons for measuring outcomes include:

1. Providing documentation to insurers, HMOs, state and federal government entities, and other third party payers that their money is well spent;
2. Demonstrating to accrediting agencies such as the Joint Commission for the Accreditation of Healthcare Organizations (JCAHO) and CARF - The Rehabilitation Accreditation Commission, that our services are effective as viewed by the patient and family members;
3. Demonstrating to patients and families that they are better off after our professional intervention thanthey were beforethey came to see us;
4. Validating the clinical decisions we make regarding our hearing aid selection and fitting practices. Measuring clinical outcomes allows us to determine what we are doing right and how we can continuously and effectively improve our care;
5. Marketing our practices to the community and managed care organizations; and
6. Establishing and maintaining 'best practices' for our profession.

The Foundation:

Fortunately, audiologists have a wide variety of outcome measures from which to choose; so many, in fact, that the we need to determine precisely which benefit we really want to measure to ensure we're using the appropriate tool! It is the purpose of this article to explore the various categories of outcome and to discuss the appropriate measuring instruments available to the dispensing audiologist. Although measuring outcome is equally important for other audiologic interventions such as tinnitus management and vestibular rehabilitation, this article will focus specifically on measuring the benefits of amplification.

Most of us are very familiar with the International Classification of Diseases (ICD) coding system. In fact, this is not the only international coding system available. In 1980, the World Health Organization (WHO) developed the International Classification of Impairment, Disability, and Handicap (ICIDH; WHO, 1980). Unlike the ICD that codes specific diseases, the ICIDH was designed to classify the impact of disease in the impairment, disability, and handicap domains. This classification system is presently undergoing revision and worldwide field trials. It is now known as the ICIDH-2 (WHO, 1997). The significant differences between ICIDH and ICIDH-2 are the inclusion of mental health disorders, the inclusion of contextual factors (architectural, political, social) that influence the impact of an individual's impairment, and the elimination of 'disability' and 'handicap'. In their place, the WHO suggests the terminology of 'activity limitation' and 'participation restriction'. I would like to begin this discussion by framing audiology outcome measures within the ICIDH-2 structure.

The Impairment Domain:

The WHO defines impairment as a loss or abnormality of body structure or of a physiological or psychological function. Obviously, a conductive or sensory impairment of the auditory system results in abnormal hearing. The outcome goals in the impairment domain are to improve our patient's ability to detect sounds, improve loudness perception, improve temporal resolution, and improve frequency resolution. We typically measure the impact of the impairment and the outcome of our intervention in the impairment domain through audiometry. For example, a patient with otosclerosis is assessed pre- and post-surgically. The difference in audiometric thresholds is a measure of surgical outcome. Pre- vs post- hearing aid fitting sound field threshold measures (functional gain) is another measure of outcome in the impairment domain. Real ear insertion gain indirectly measures the impact of amplification on audibility.

The Articulation Index/Audibility Index (AI) is a less frequently used but effective measure of improvement in the impairment domain. The AI is a measure of the amount of speech information available to the listener scored on a scale from 0 (meaning no information is available) to 1.0 (meaning all of the speech energy is available). The AI can be calculated from insertion gain measures or derived from aided versus unaided soundfield measures. The unaided AI is subtracted from the aided AI to determine the measure of outcome (improved audibility) in the impairment domain. The AI is directly associated with speech perception. As sensorineural hearing loss exceeds 50 dB and more inner hair cells are involved, the relationship between AI and speech recognition is less predictable.

Aided audiometry, AI, and real ear measures reveal the extent to which audibility has improved but tells us nothing about communication performance or resolution of our patient's complaints. Such information is better obtained through measures targeting the effect of hearing loss on activity limitations and participation restrictions.

Activity Limitations and Participation Restrictions:

We are particularly interested in how our patients' hearing loss affects their ability to perform activities and participate in social interchange that are considered normal for each individual. While the impairment domain views the hearing impairment at the level of the organ or structure (the auditory system), activity limitations and participation restrictions are viewed from the perspectives of the person and society. The outcome goals in the activity domain are to improve speech understanding and communication functioning. The outcome goal in the participation domain is to improve communication functioning in specific situations and with specific partners.

To examine the outcome of hearing aid intervention in these domains, the clinician needs to determine the problems the patient is experiencing prior to and following the hearing aid fitting. There are a number of standardized questionnaires available for this purpose including the Hearing Handicap Inventory for the Elderly (HHIE; Ventry & Weinstein, 1982), the Hearing Handicap Inventory for Adults (HHIA, Newman, Weinstein, Jacobson & Hug, 1991), and the Abbreviated Profile of Hearing Aid Benefit (APHAB; Cox & Alexander, 1995).

Unlike impairment, the impact of a hearing loss on the individual and society is greatly influenced by the contextual factors in which the patient functions. For example, what is the local political climate concerning disability? What type of access is available to the hearing impaired? What is the local availability of t-coil compatible public telephones, text telephones (TDDs), large area infrared systems, and open captioning? Are the social service agencies knowledgeable about the needs and resources for individuals with hearing loss? Are employers sensitive to the needs of employees with hearing impairment? These and other contextual factors will significantly influence the impact of the hearing impairment on an individual's ability to actively engage and participate in society.

Returning to the issue of questionnaires like the APHAB and HHIE, one of the disadvantages of standardized measuring instruments is that some of the questions on the scale may not be particularly important or relevant to a specific patient. The consequence of irrelevant items is that the outcome measure may not be sensitive to the benefits of your intervention.

The Client Oriented Scale of Improvement (COSI; Dillon, James & Ginis, 1997) addresses this problem by using an open-ended approach to problem identification. The patient and clinician work together to identify up to five situations the patient feels are most impacted by the hearing loss. These situations are prioritized and become the outcome goals of the intervention strategy. For example, your patient may identify being able to hear and understand TV at a volume setting that is comfortable for other family members as her number one priority. Following the hearing aid fitting, the patient is asked how her ability to hear the TV at a comfortable volume has improved (this is the relative performance outcome measure) and she is also asked how often she can achieve this goal (this the absolute performance measure). If the TV issue was her highest priority and she feels the hearing instruments have not sufficiently resolved her problem, there is a good likelihood that the outcome will be poor and the fitting will be a failure. It is particularly important to state the COSI goals in very specific terms. For example, 'hearing in noise' is too vague a goal and is likely to set the two of you up for failure. The audiologist needs to probe and determine what kind of noise ('several people talking at once'), in what type of situation ('playing cards'), and who is it that the patient wants to be able to understand ('the people at my table'). Getting this specific information helps to drive the selection and fitting decisions as well as suggesting additional communication and technology strategies.

Satisfaction:

Let's assume you've successfully addressed the problems identified by your patient and your APHAB scores indicate your patient is better off as a result of those new digital hearing aids he's purchased from you. He's certainly going to be satisfied, right? Well, not necessarily. Satisfaction is a construct that involves more than just your clinical results. Satisfaction includes elements of your service and product only peripherally related to clinical outcomes. For example, how difficult was it to find your office? How was the parking? How courteous was your reception staff? How clean was your waiting area and your restroom? How long did your patient have to wait before being seen? Did your patient feel he got his money's worth? Were the hearing aids cosmetically acceptable? Is your patient experiencing occlusion, feedback, discomfort, etc? The outcome goal in the satisfaction domain is to meet or exceed your patient's expectations with your product and service

So, how do audiologists measure satisfaction? Many practices use questionnaires similar to those employed by retail businesses to assess product and service satisfaction. In fact, the audiology community has its own standardized satisfaction questionnaire, the Satisfaction with Amplification in Daily Life (SADL; Cox & Alexander, 1999). The SADL consists of 15 questions categorized in 4 subscales: positive effect, service and cost, negative features, and personal image. At 15 questions, the SADL is easily administered and completed and the information can be very meaningful for you and your practice.

Health-Related Quality of Life:

Up to this point, I've discussed outcomes in the impairment, activities, participation and satisfaction domains. But ultimately, we want to know how our intervention has impacted our patient's health-related quality of life (HQL). By health-related quality of life we mean the impact of an illness or disease on an individual's overall physical, mental, and social well being. Measuring HQL outcome is particularly important because it permits us to compare the effects of hearing loss and alternative treatment strategies within our own discipline and across other disorders and treatments. For example, is an individual's HQL impacted more by improved hearing and communication or by arthritis pain relief, or by allergy relief? Insurers, managed care entities and health care policy makers want answers to this, and similar questions.

The outcome goal in the HQL domain is to improve your patient's overall perception of well being. There are several ways to measure HQL. One approach is to use disease specific outcome measures such as the HHIE or APHAB. These instruments compare the outcomes of several treatment options within the same health condition (multi-microphone vs. single microphone technology). The other type is known as generic HQL measures which are designed to measure the impact of diseases and their intervention across different health conditions (hearing loss vs. arthritis, for example). Within the generic measure option are two styles of measures - health profiles and utility measures. Health profiles are questionnaires that usually contain several subscales. An example is the MOS SF-36 Health Survey (Ware & Sherbourne, 1992) which measures eight subscales of general health categorized into physical and mental components of the individual's perceived health status. Unfortunately, most generic health profiles are insensitive to the effects of hearing loss. The questions asked in many of these instruments address issues such as independence, pain, ambulation, depression, and activities of daily living. The other generic instrument, utility measures, is used to determine an individual's health state preference by comparing different health care conditions.

The simplest form of a utility measure is a feeling thermometer. The patient is asked to place a pointer along the thermometer between 0 (least desirable health state) and 100 (perfect health) to indicate how a particular disorder (arthritis, lung cancer, hypertension, hearing loss) is impacting their health-related quality of life at that moment in time. The thermometer can be used to compare the patient's perceived health state across various disorders or can be used to measure the effect of intervention on any one or more of their health conditions. The extent to which the pointer has moved up (we hope) on the scale following the use of amplification, for example, can be quantified and is a direct measure of the patient's improved health-related quality of life. There are other ways to measure health utility such as the Time Trade-Off and Standard Gamble techniques but a discussion of these methods is beyond the scope of this article.

How Much Does a Good Outcome Cost?

Unfortunately, it's no longer enough to demonstrate that we've made a difference. Now we need to document how much that difference cost. The following is a brief description of the types of analyses that can answer 'how much does it cost'?

Straight cost analysis - an analysis of all costs (labor, equipment, supplies, administrative support, space, maintenance, etc.) that go into delivering the intervention. The output of a cost analysis is expressed in dollars without regard for any benefits achieved.
Cost benefit analysis - compares the dollars spent against the dollars earned or saved. For example, how much do we save in special education costs and how much does society gain in additional income tax revenue when we provide hearing aids to children with hearing impairment identified at birth?
Cost effectiveness analysis - compares the dollars spent against a measure of clinical benefit. For example, how much does it cost for a 1-% improvement in speech recognition in noise? Such an analysis allows us to compare the cost vs. outcome for alternative interventions (e.g., digital vs. non-digital, directional vs. omnidirectional).
Cost utility analysis - A cost-utility analysis allows us to determine how much it costs to improve a patient's quality of life over the lifetime of the patient. The resulting value of this analysis is expressed in cost per quality adjusted life-years gained (QALY). Let's look at the following example:

Example 1
Age of patient -70
Cost of hearing aids-$2,000/pair
Benefit obtained - 25%
Life expectancy - 5 years

The cost per QALY gained = $2,000 ÷ .25 X 5 = $1,600

Example 2
Age of patient - 70
Cost of hearing aids- $4,000/pair
Benefit obtained- 60%
Life expectancy - 5 years

The cost per QALY gained = $4,000 ÷ .60 X 5 = $1,333

Sometimes, things that cost more initially cost less in the long term.

Conclusions:

I stated at the beginning of this article that my purpose was to explore the various categories of outcome and discuss the appropriate measuring instruments available to the dispensing audiologist. It's clear that there are a wide variety of instruments we can use. The selection of any particular outcome measure will depend on what domain (impairment, satisfaction, HQL, etc.) is of interest and on the nature of the practice. A small but busy practice needs to balance the time available for administering and analyzing outcome results against the value of the information obtained from those measures. Larger practices, particularly those competing for managed care contracts, will likely use several measures to help identify trends and differentiate the successful clinicians and practices from those who may require additional training. Which outcome measure is best? The answer is the same for the question, 'which hearing aid is best?' It's the one that's used.

References
Cox R, Alexander G. (1999). Measuring satisfaction with amplification in daily life: The SADL scale. Ear & Hearing 20:306-320.

Cox R. Alexander G. (1995). The Abbreviated Profile of Hearing Aid Benefit. Ear & Hearing 16:176-186.

Dillon H, James A, Ginis, J. (1997). Client Oriented Scale of Improvement (COSI) and its relationship to several other measures of benefit and satisfaction provided by hearing aids. Journal of the American Academy of Audiology 8:27-43.

Newman CW, Weinstein B, Jacobson GP, Hug GA. (1991). Test-retest reliability of the Hearing Handicap Inventory for Adults. Ear & Hearing 12:355-357.

Ventry I, Weinstein B. (1982). The hearing handicap inventory for the elderly: A new tool. Ear and Hearing 3:128-134.

Ware JE, Sherbourne CD. (1992). The MOS 36-item short-form health survey (SF-36): I. Conceptual framework and item selection. Medical Care 30:473-481.

WHO (1980). International Classification of Impairments, Disabilities and Handicaps - A Manual of Classification Relating to the Consequences of Disease. Geneva: World Health Organization.

WHO (1997). Towards a common language for functioning and disablement: ICIDH-2 Beta-draft for Field Trials. Geneva: World Health Organization.