Employing deep learning model to evaluate speech information in acoustic simulations of Cochlear implants

Radford, A. K., J.W.; Xu, T.; Brockman, G.; McLeavey, C.; Sutskever, I. Robust Speech Recognition via Large-ScaleWeak Supervision. arXiv, https://doi.org/10.48550/arXiv.2212.04356 (2022).Weerts, L. R. S., Clopath C.; Goodman D. F. M. . The Psychometrics of Automatic Speech Recognition. bioRxiv, https://doi.org/10.1101/2021.04.19.440438 (2021).Rossbach, J., Kollmeier, B. & Meyer, B. T. A model of speech recognition for hearing-impaired listeners based on deep learning. J. Acoust. Soc. Am. 151, 1417. https://doi.org/10.1121/10.0009411 (2022).Article 
ADS 
PubMed 

Google Scholar 
Wouters, J., McDermott, H. J. & Francart, T. Sound Coding in Cochlear Implants. Ieee Signal Proc Mag 32, 67–80. https://doi.org/10.1109/Msp.2014.2371671 (2015).Article 
ADS 

Google Scholar 
Shannon, R. V., Zeng, F. G., Kamath, V., Wygonski, J. & Ekelid, M. Speech recognition with primarily temporal cues. Science 270, 303–304 (1995).Article 
ADS 
CAS 
PubMed 

Google Scholar 
Dorman, M. F., Loizou, P. C. & Rainey, D. Speech intelligibility as a function of the number of channels of stimulation for signal processors using sine-wave and noise-band outputs. J. Acoust. Soc. Am. 102, 2403–2411 (1997).Article 
ADS 
CAS 
PubMed 

Google Scholar 
Shannon, R. V., Fu, Q. J. & Galvin, J., 3rd. The number of spectral channels required for speech recognition depends on the difficulty of the listening situation. Acta Otolaryngol Suppl, 50–54, https://doi.org/10.1080/03655230410017562 (2004).Xu, L., Thompson, C. S. & Pfingst, B. E. Relative contributions of spectral and temporal cues for phoneme recognition. J. Acoust. Soc. Am. 117, 3255–3267. https://doi.org/10.1121/1.1886405 (2005).Article 
ADS 
PubMed 

Google Scholar 
Souza, P. & Rosen, S. Effects of envelope bandwidth on the intelligibility of sine- and noise-vocoded speech. J. Acoust. Soc. Am. 126, 792–805. https://doi.org/10.1121/1.3158835 (2009).Article 
ADS 
PubMed 
PubMed Central 

Google Scholar 
Fitzgerald, M. B., Prosolovich, K., Tan, C. T., Glassman, E. K. & Svirsky, M. A. Self-Selection of Frequency Tables with Bilateral Mismatches in an Acoustic Simulation of a Cochlear Implant. J. Am. Acad. Audiol. 28, 385–394. https://doi.org/10.3766/jaaa.15077 (2017).Article 
PubMed 
PubMed Central 

Google Scholar 
Jethanamest, D., Azadpour, M., Zeman, A. M., Sagi, E. & Svirsky, M. A. A Smartphone Application for Customized Frequency Table Selection in Cochlear Implants. Otol Neurotol 38, e253–e261. https://doi.org/10.1097/MAO.0000000000001409 (2017).Article 
PubMed 
PubMed Central 

Google Scholar 
Bingabr, M., Espinoza-Varas, B. & Loizou, P. C. Simulating the effect of spread of excitation in cochlear implants. Hear Res 241, 73–79. https://doi.org/10.1016/j.heares.2008.04.012 (2008).Article 
PubMed 
PubMed Central 

Google Scholar 
Stafford, R. C., Stafford, J. W., Wells, J. D., Loizou, P. C. & Keller, M. D. Vocoder simulations of highly focused cochlear stimulation with limited dynamic range and discriminable steps. Ear Hear. 35, 262–270. https://doi.org/10.1097/AUD.0b013e3182a768e8 (2014).Article 
PubMed 

Google Scholar 
Loizou, P. C., Dorman, M. & Fitzke, J. The effect of reduced dynamic range on speech understanding: implications for patients with cochlear implants. Ear Hear. 21, 25–31. https://doi.org/10.1097/00003446-200002000-00006 (2000).Article 
CAS 
PubMed 

Google Scholar 
Friesen, L. M., Shannon, R. V., Baskent, D. & Wang, X. Speech recognition in noise as a function of the number of spectral channels: comparison of acoustic hearing and cochlear implants. J. Acoust. Soc. Am. 110, 1150–1163 (2001).Article 
ADS 
CAS 
PubMed 

Google Scholar 
Hervais-Adelman, A., Davis, M. H., Johnsrude, I. S. & Carlyon, R. P. Perceptual learning of noise vocoded words: effects of feedback and lexicality. J. Exp. Psychol. Hum. Percept. Perform. 34, 460–474. https://doi.org/10.1037/0096-1523.34.2.460 (2008).Article 
PubMed 

Google Scholar 
Davis, M. H., Johnsrude, I. S., Hervais-Adelman, A., Taylor, K. & McGettigan, C. Lexical information drives perceptual learning of distorted speech: evidence from the comprehension of noise-vocoded sentences. J Exp Psychol Gen 134, 222–241. https://doi.org/10.1037/0096-3445.134.2.222 (2005).Article 
PubMed 

Google Scholar 
Loebach, J. L. & Pisoni, D. B. Perceptual learning of spectrally degraded speech and environmental sounds. J. Acoust. Soc. Am. 123, 1126–1139. https://doi.org/10.1121/1.2823453 (2008).Article 
ADS 
PubMed 

Google Scholar 
Spahr, A. J. et al. Development and validation of the AzBio sentence lists. Ear Hear. 33, 112–117. https://doi.org/10.1097/AUD.0b013e31822c2549 (2012).Article 
PubMed 
PubMed Central 

Google Scholar 
Grange, J. A., Culling, J. F., Harris, N. S. L. & Bergfeld, S. Cochlear implant simulator with independent representation of the full spiral ganglion. J. Acoust. Soc. Am. 142, EL484, https://doi.org/10.1121/1.5009602 (2017).Goupell, M. J., Draves, G. T. & Litovsky, R. Y. Recognition of vocoded words and sentences in quiet and multi-talker babble with children and adults. PLoS ONE 15, e0244632. https://doi.org/10.1371/journal.pone.0244632 (2020).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Oxenham, A. J. & Kreft, H. A. Speech Perception in Tones and Noise via Cochlear Implants Reveals Influence of Spectral Resolution on Temporal Processing. Trends in hearing 18, https://doi.org/10.1177/2331216514553783 (2014).Bierer, J. A., Spindler, E., Bierer, S. M. & Wright, R. An Examination of Sources of Variability Across the Consonant-Nucleus-Consonant Test in Cochlear Implant Listeners. Trends in hearing 20, 1–8. https://doi.org/10.1177/2331216516646556 (2016).Article 

Google Scholar 
Faulkner, A., Rosen, S. & Norman, C. The right information may matter more than frequency-place alignment: simulations of frequency-aligned and upward shifting cochlear implant processors for a shallow electrode array insertion. Ear Hear. 27, 139–152. https://doi.org/10.1097/01.aud.0000202357.40662.8500003446-200604000-00005[pii] (2006).Article 
PubMed 

Google Scholar 
Landsberger, D. M., Svrakic, M., Roland, J. T. Jr. & Svirsky, M. The Relationship Between Insertion Angles, Default Frequency Allocations, and Spiral Ganglion Place Pitch in Cochlear Implants. Ear Hear. 36, e207-213. https://doi.org/10.1097/AUD.0000000000000163 (2015).Article 
PubMed 
PubMed Central 

Google Scholar 
Gifford, R. H., Sunderhaus, L. W., Dawant, B. M., Labadie, R. F. & Noble, J. H. Cochlear implant spectral bandwidth for optimizing electric and acoustic stimulation (EAS). Hear Res 426, 108584. https://doi.org/10.1016/j.heares.2022.108584 (2022).Article 
PubMed 
PubMed Central 

Google Scholar 
Sagi, E., Azadpour, M., Neukam, J., Capach, N. H. & Svirsky, M. A. Reducing interaural tonotopic mismatch preserves binaural unmasking in cochlear implant simulations of single-sided deafness. J. Acoust. Soc. Am. 150, 2316. https://doi.org/10.1121/10.0006446 (2021).Article 
ADS 
PubMed 
PubMed Central 

Google Scholar 
Fu, Q. J. Temporal processing and speech recognition in cochlear implant users. Neuroreport 13, 1635–1639 (2002).Article 
PubMed 

Google Scholar 
Won, J. H., Drennan, W. R., Nie, K., Jameyson, E. M. & Rubinstein, J. T. Acoustic temporal modulation detection and speech perception in cochlear implant listeners. J. Acoust. Soc. Am. 130, 376–388. https://doi.org/10.1121/1.3592521 (2011).Article 
ADS 
PubMed 
PubMed Central 

Google Scholar 
Kohlrausch, A., Fassel, R. & Dau, T. The influence of carrier level and frequency on modulation and beat-detection thresholds for sinusoidal carriers. J. Acoust. Soc. Am. 108, 723–734. https://doi.org/10.1121/1.429605 (2000).Article 
ADS 
CAS 
PubMed 

Google Scholar 
Zeng, F. G. et al. Speech dynamic range and its effect on cochlear implant performance. J. Acoust. Soc. Am. 111, 377–386 (2002).Article 
ADS 
PubMed 

Google Scholar 
Fraser, M. & McKay, C. M. Temporal modulation transfer functions in cochlear implantees using a method that limits overall loudness cues. Hear Res 283, 59–69. https://doi.org/10.1016/j.heares.2011.11.009 (2012).Article 
PubMed 
PubMed Central 

Google Scholar 
Monaghan, J. J. M., Carlyon, R. P. & Deeks, J. M. Modulation Depth Discrimination by Cochlear Implant Users. J. Assoc. Res. Otolaryngol. 23, 285–299. https://doi.org/10.1007/s10162-022-00834-6 (2022).Article 
PubMed 
PubMed Central 

Google Scholar 
Loizou, P. C., Dorman, M., Poroy, O. & Spahr, T. Speech recognition by normal-hearing and cochlear implant listeners as a function of intensity resolution. J. Acoust. Soc. Am. 108, 2377–2387 (2000).Article 
ADS 
CAS 
PubMed 

Google Scholar 
Azadpour, M., McKay, C. M. & Svirsky, M. A. Effect of Pulse Rate on Loudness Discrimination in Cochlear Implant Users. J. Assoc. Res. Otolaryngol. 19, 287–299. https://doi.org/10.1007/s10162-018-0658-8 (2018).Article 
PubMed 
PubMed Central 

Google Scholar 
Nelson, D. A., Schmitz, J. L., Donaldson, G. S., Viemeister, N. F. & Javel, E. Intensity discrimination as a function of stimulus level with electric stimulation. J. Acoust. Soc. Am. 100, 2393–2414 (1996).Article 
ADS 
CAS 
PubMed 

Google Scholar 
Brochier, T. et al. From Microphone to Phoneme: An End-to-End Computational Neural Model for Predicting Speech Perception with Cochlear Implants. IEEE Trans Biomed Eng PP, https://doi.org/10.1109/TBME.2022.3167113 (2022).Bruce, I. C. et al. A stochastic model of the electrically stimulated auditory nerve: pulse-train response. IEEE Trans Biomed Eng 46, 630–637 (1999).Article 
CAS 
PubMed 

Google Scholar 
Takanen, M., Bruce, I. C. & Seeber, B. U. Phenomenological modelling of electrically stimulated auditory nerve fibers: A review. Network 27, 157–185. https://doi.org/10.1080/0954898X.2016.1219412 (2016).Article 
PubMed 

Google Scholar 
Vaswani A., S. N., Parmar N., Uszkoreit J., Jones L., Gomez A.N., Kaiser L., Polosukhin, I. in Neural Information Processing Systems. 5998–6008.Kreft, H. A., Donaldson, G. S. & Nelson, D. A. Effects of pulse rate and electrode array design on intensity discrimination in cochlear implant users. J. Acoust. Soc. Am. 116, 2258–2268 (2004).Article 
ADS 
PubMed 

Google Scholar