Are hearing tests reliable?

I am not an audi, however am familiar from grad school with test construction (primarily psychological) yet many of the concepts still apply. Is the test Valid? Reliable?

Validity is a determination of whether the test measures what it purports to measure. Part of reliability has to do with whether repeated measurements when the expectation is that results should remain constant is whether they actual do remain the same. (Usually referred to as test-retest reliability and often measured by correlation co-efficient.)

Op is wondering whether the test is actually valid as it might be conflating actual hearing with a strategy of anticipating when sound should be present and “guessing” even if not actually heard. The description of standard test procedures (https://www.asha.org/policy/gl2005-00014/#sec1.42) among other things indicates that the tones should be presented for varying lengths of time. While this would not eliminate a strategy of guessing based on learning what the parameters are for the test procedure it should tend to make the results more valid by making accurate guessing more difficult.

While I expect that there is some improvement in test scores due to familiarization with the test procedure which also can lead to relaxing more about it and doing better due to a relaxed state; overall I expect the results are fairly valid and fairly consistent. While the test I expect is not given if there is excessive ear wax (or ear infection), I believe/hypothesize that amount and location of ear wax has more impact on test results than guessing; and perhaps also outweigh the impact of excessive nervousness which might be the case for those who have never had HAs and are being evaluated for whether they are to be recommended. I believe that test validity and reliability is probably higher for experienced HA people.

If I were to design test procedures, I would probably do the following [and then compare these procedures with the current protocol to determine whether there is a cost-benefit for changing current procedures}:

1. Present the tones (not) in the typical ascending order of frequency starting with 1K, but do so randomly. Present the intensity of the tone not in an ascending sequence of intensity starting below the intensity that is expected to NOT be heard and increasing intensity in 5DB increments, but vary the intensity is a somewhat more varied manner.

2. I would also test so that the presentation is randomly presented to target (left ear, or right, ear, or both ears).

3. Test procedure should ask for pressing button immediately when tone is heard, AND to release it immediately when tone is turned off.

4. Scoring of subject responses should look for accurate answers at the presumed threshold AND subtract “points” for incorrect (or delayed responses) for both the ON situation and for the OFF situation.

5. I’d note that the current test procedures have face validity for measuring/answering the ?S: What tones can be heard at all? and at what intensity do they need to be in order to be heard?
   My suggested procedure is related but possibly more a test of and answer to the ?:
   When a frequency-intensity pair is present in the sound environment is the subject consistent in actively responding (processing it as “relevant” information) to it? This is a more nuanced question and might(???) have greater relevance to a subjects real world ability process speech.

Both of these suggestions I believe would arrive at a more accurate threshold (validity) than current procedures and are a little bit more like the real world in which one can not always anticipate from where a sound will be coming, when it will come, and whether the voice will be a male, female, or child.

However, the suggested modifications would take significantly longer to administer and fitters or audis are sensitive to how much time they spend with someone. My hunch is that a test procedure that might be more valid and/or more reliable while it might have incremental improvement might not be worth the additional time.

From std test procedures:
" Threshold determination . The method described, an ascending technique beginning with an inaudible signal, is recommended as a standard procedure for manual pure-tone threshold audiometry.

1. Tone duration . Pure-tone stimuli of 1 to 2 seconds’ duration.
2. Interval between tones . The interval between successive tone presentations shall be varied but not shorter than the test tone."

========
**My bottom lines are:
A) I don’t expect procedures to change; I think they are probably good enough. Client who is motivated and understands the importance of the audiogram to the fitting procedure is not likely to want to game the system (in most situations) in any significant way.

B) My bigger concern/issue is that of ear wax and I have thought that since I am soon to go for new testing and new HAs that I will make sure my ears are clean, but that they have an “average for me” amount of ear wax. My hunch is that most of us might tend (if we do any kind of routine cleaning) that we do it too close to the timing of a pure tone hearing test and that the results might thus be slightly skewed to measuring toward our best possible hearing rather than what is typical for us.** Related is the fact that most aids have a wax guard and that while we might clean/brush it daily that it at least gets partially clogged during the interval of a month. (I actually look at wax guard with smart phone magnifying glass when replacing it to see if and how much it might have clogged. Basically if our technology for hearing changes/degrades over a period of weeks - assuming that the placement of the receiver is identical from day-to-day - then there are a lot of moving parts in the equation of well we hear (sensation to ear drum) over a period of time. Quite frankly I am grateful and in awe that often it works quite well and consistently!

Note also that as @molarface points out in comment above this, how much vigilance/attention we are giving our awareness to hearing (as opposed to any other thoughts or sensations) has impact not so much on whether the ear drum and nervous system receives the sensation, but on whether we process it. I expect that for many the test procedure is like waking up in the middle of the night and hearing a strange sound and then being still and intently listening to determine what it is that was heard, where the sound was coming from, and what the meaning is (an animal, a human intruder, a window that has blown open). Except this middle of the night experience; or truly focusing on just enjoying listening to music; that most of the time even when we say we are listening to someone it is only a portion of our awareness. Pure tone testing is interesting/valuable but is only partially correlated to real world situations.

It has been fun for me to speculate about the ? raised by OP; thanks for indulging my long comment.

Thanks Ed, I found your post interesting.

I don’t know if there is a standard or not but all three tests I have taken were random

I have been getting yearly hearing test for about 18 years and I have found it depends on me mostly. If my allergies or sinus are acting up then the test will show worse hearing loss. If I feel stressed the test will show worse hearing loss. Also the time of day the test is done will cause my test results to vary. The later in the day of the test the more I am tired the test is worse hearing loss. All of this changes my tinnitus which changes test results. And also, my blood pressure will also change my tinnitus which changes my test results.

Good (and to me interesting points) about what for you contributes to inconsistent test results. However a loose working definition of whether a test is reliable might be that when administered in the same manner and under the same circumstances that the test results from test to test to are highly similar.

As a simple example if you use a calibrated ruler to measure a steel rod when the temperature is near freezing versus when it is summer hot, this would tell you nothing about whether the test (“calibrated ruler”) is reliable. OP asked about “reliability” and while somewhat related consistency of results under different circumstances (condition of sinus, rested-fatigued, time of day, months between tests, etc.) is a different matter.

Why would anyone want to fake the test?

It seems if you do, and you need hearing aids, you will either fake it so well you won’t get aids, and if you don’t fake it that well, your aids won’t be adjusted properly.

It seems to me to be in your best interest to do your best and don’t try to cheat.

I think when my tinnitus is acting up, I don’t hear some high tones as soon as I would otherwise, as they are being masked. So before the test, I don’t drink coffee or do anything else that may increase the noise in my head.

The VA does see it happen so the person can hopefully get a larger disability rating which means more monthly money

@BobbyBoomer said:

Why would anyone want to fake the test?

It’s not that anyone wants to “fake” a test.

Through 12+++ years of schooling, we’re conditioned to always do our best. To compound the problem, the instructions tell us to hit the button if we hear a beep OR THINK we hear a beep. With the ability to anticipate the beep, I’ve found that I’ve practically heard the beep before it actually happened.

You fake the automatic hearing test because you have a hearing disability but want to still serve in the armed forces. You also fake the hearing test because you ve been drafted and don’t want to go to Nam. Which led to officers screaming in everyone face that no one goes home till you pass. These tests aren’t to see if you need hearing aids bobby. But the point is an automated hearing test can be manipulated

What are you talking about automatic test never heard of such a thing or even seen one

Were you in the military CV. I took a hearing test for submarine duty. I was out in a sound proof booth. The instructor hit the button and walked away. The best part was the test graph was placed in front of me so I could watch the needle trace my results. I ended up with the hearing of a dog. Just because you never hear of it doesn’t mean it doesn’t exist. The beeps come in automatically along with the different volume in a pretty set manner. When I took the test for induction into the service I got the timing down and then just just held the button every few seconds until the test was done. I love people who question what I experienced first hand on numerous occasions. I failed my first test and went back to my recruiter and he showed me how to beat the test. Here’s an added note. Since I had two tests with different results they sent me to see a navy Dr. He ran tests and said to me. What do you want in or out . I chose in. Thank you for your service. So please don’t tell me you never heard of it. It doesn’t matter

I was in the Navy and never had such a thing. I only had the same type as I get now from the VA. I boot camp I failed the Morse code and was given the same as I get nowadays and was told my hearing was normal. But I couldn’t tell the difference between a dot and dash. After a flight from Andrews Air Force base is to Indonesia I had several tinnitus, and the flight back made it even worse. I was given a hearing test and was told a had a slight hearing loss. Then a year later I mustered out and the test showed a mild hearing loss. That was 1977 it wasn’t until 2005 that I got my first set of hearing aids and by then I had a moderate hearing loss. But I haven’t heard of the test you are talking about.