Voice Perception
Abstract
While the perception of faces from static portraits has been investigated in many studies, little research has been devoted to processes mediating auditory recognition of people via their voices. This is despite the fact that the voice is by far the most important auditory stimulus that supports person identification, and that it carries a wealth of further social information including emotion, gender, or age. This project is intended to fill a major gap in the research on auditory person perception, by addressing three key aspects of voice perception. First, using a design that incorporates both recognition memory and priming approaches, we explore the role of attention for explicit and implicit voice memory. Second, using novel voice morphing technology, we recently presented the first behavioural evidence that adaptation to non-linguistic information in voices elicits systematic auditory aftereffects in the perception of gender (Schweinberger et al., 2008). Here we will build on this new line of research, and will study behavioural and neurocognitive correlates of auditory adaptation to two other important social signals conveyed by voices: person identity and age. The studies on voice identity adaptation can be expected to have far-reaching theoretical implications with respect to the question of whether individual voices are represented in a prototype-referenced manner, similar to what has been suggested for the representation of facial identity. Building on findings from the visual modality that different visual adaptation effects depend on attention and conscious perception to very different degrees, we study the combined effects of attention and voice adaptation.
With increasing progress in developing basic research paradigms, we have now also begun a number of more applied projects. In one, we look at individual differences in voice recognition abilities and their potential links with autistic behavioral tendencies. In another, we target voice recognition in prosopagnosic individuals. In a clinical project, we have begun to use parameter-specific voice morphing technology to investigate the perception of social signals in the voice by cochlear implant users.
People
Selected Relevant Publications
Dogdu, C., Kessler, T., Schneider, D., Shadaydeh, M., & Schweinberger, S.R. (2022). A Comparison of Machine Learning Algorithms and Feature Sets for Automatic Vocal Emotion Recognition in Speech. Sensors, 22, 7561. (Link to PDF).
Humble, D., Schweinberger, S.R., Mayer, A., Jesgarzewsky, T.L., Dobel, C., & Zäske, R. (2022). The Jena Voice Learning and Memory Test (JVLMT): A standardized tool for assessing the ability to learn and recognize voices. Behavior Research Methods, Advance Online. (Link to PDF).
Schweinberger, S.R., & von Eiff, C.I. (2022). Enhancing Socio-emotional Communication and QoL in Young CI Recipients: Perspectives from Parameter-specific Morphing and Caricaturing. Frontiers in Neuroscience, 16:956917. (Link to PDF). Article is part of the Special Topic Quality of Life in Young Cochlear Implant Recipients: Are there Controlling Factors and Regional Differences? (Guest Editors: M. Huber, H.-J. Lee, M. Langereis, A. Vermeulen).
Tsuchiya, A., & Schweinberger, S.R. (2022). Erkennen von Verwandtschaft zu sich Selbst und bei Anderen. In-Mind Themenausgabe: Wie verstehen wir andere besser? Teil 2 Heft 1/2022. Gasteditoren: P. Burgmer, K. Mushold, D.Schneider. (Link to PDF)
Nussbaum, C., Schirmer, A., & Schweinberger, S.R. (2022). Contributions of Fundamental Frequency and Timbre to Vocal Emotion Perception and their Electrophysiological Correlates. Social, Cognitive and Affective Neuroscience. Advance Online. (Link to PDF).
Nussbaum, C., von Eiff, C.I., Skuk, V.G., & Schweinberger, S.R. (2022). Vocal emotion adaptation aftereffects within and across speaker genders: Role of timbre and fundamental frequency. Cognition, 219, 104967. (Link to PDF)
Von Eiff, C.I., Skuk, V.G., Zäske, R., Nussbaum, C., Frühholz, S., Feuer, U., Guntinas-Lichius, O., & Schweinberger, S.R. (2022). Parameter-specific morphing reveals contributions of timbre to the perception of vocal emotions in cochlear implant users. Ear and Hearing, Advance Online. (Link to PDF)
Frühholz, S., & Schweinberger, S.R. (2021). Nonverbal auditory communication – Evidence for Integrated Neural Systems for Voice Signal Production and Perception. Progress in Neurobiology, 199, 101948. (Link to PDF)
Nussbaum, C., & Schweinberger, S.R. (2021). Links between musicality and vocal emotion perception. Emotion Review. (Link to PDF)
Garrido, L., Eisner, F., McGettigan, C., Stewart, L., Sauter, D., Hanley, J.R., Schweinberger, S.R., Warren, J., & Duchaine, B. (2009). Developmental phonagnosia: a selective deficit to vocal identity recognition. Neuropsychologia, 47, 123-131.
Humble, D., Schweinberger, S.R., Dobel, C., & Zäske, R. (2019). Voices to remember: Comparing neural signatures of intentional and non-intentional voice learning and recognition. Brain Research, 1711, 214-225. (Link to PDF)
Kawahara, H., & Skuk, V.G. (2019). Voice Morphing. In: S. Frühholz and P. Belin (Eds.): The Oxford Handbook of Voice Perception (Chapter 31, pp. 685-706).
Kirchen, L., Skuk, V.G., Schmidt, T., Guntinas-Lichius, O., Feuer, U., Voigt, P., & Schweinberger, S.R. (2015). Perception of social information in voices by cochlear-implant users. Poster presented at the 12th Workshop of the DFG Research Unit Person Perception, Jena, October 15-16, 2015.
Latinus, M. & Zäske, R. (2019). Perceptual Correlates and Cerebral Representation of Voices – Identity, Gender and Age. In: S. Frühholz and P. Belin (Eds.): The Oxford Handbook of Voice Perception (Chapter 25, pp. 561-583).
Neuner, F., & Schweinberger, S.R. (2000). Neuropsychological impairments in the recognition of faces, names, and voices. Brain and Cognition, 44, 342-366.
Nussbaum, C., & Schweinberger, S.R. (in press). Links between musicality and vocal emotion perception. Emotion Review. (Link to PDF)
Schweinberger, S.R. (2001). Human brain potential correlates of voice priming and voice recognition. Neuropsychologia, 39, 921-936.
Schweinberger, S.R., Casper, C., Hauthal, N., Kaufmann, J.M., Kawahara, H., Kloth, N., Robertson, D.M.C., Simpson, A.P., & Zäske, R. (2008). Auditory adaptation in voice perception of voice gender. Current Biology, 18, 684-688. (Link to PDF)
Schweinberger, S.R., Herholz, A., & Sommer, W. (1997). Recognizing famous voices: Influence of stimulus duration and different types of retrieval cues. Journal of Speech, Language, and Hearing Research, 40, 453-463. (Link to PDF)
Schweinberger, S.R., Herholz. A., & Stief, V. (1997). Auditory long-term memory: Repetition priming of voice recognition. The Quarterly Journal of Experimental Psychology, 50A, 498-517.
Schweinberger, S.R., Kawahara, H., Simpson, A.P., Skuk, V.G., & Zaeske, R. (2014). Speaker Perception. Wiley Interdisciplinary Reviews: Cognitive Science, 5, 15-25. (Link to PDF) (Link to Supplementary Material)
Schweinberger, S.R., von Eiff, C.I., Kirchen, L., Oberhoffner, T., Guntinas-Lichius, O., Dobel, C., Nussbaum, C., Zäske, R., &; Skuk, V.G. (2020). The Role of Stimulus Type and Social Signal for Voice Perception in Cochlear Implant Users: Response to the Letter by Meister H et al. Journal of Speech, Language, and Hearing Research, 63(12), 4327-4328. (Link to PDF)
Schweinberger, S.R., Walther, C., Zäske, R., & Kovács, G. (2011). Neural correlates of adaptation to voice identity. British Journal of Psychology, 102(4), 748-764.
Schweinberger, S.R. & Zäske, R. (2019). Perceiving Speaker Identity from the Voice. In: S. Frühholz and P. Belin (Eds.): The Oxford Handbook of Voice Perception (Chapter 24, pp. 539-560).
Skuk, V.G., Dammann, L.M., & Schweinberger, S.R. (2015). Role of timbre and fundamental frequency in voice gender adaptation. Journal of the Acoustical Society of America, 138(2), 1180-1193. doi: 10.1121/1.4927696. (Link to PDF) Supplemental Material (.pdf) ca. 4.4. MB).
Skuk, V.G., Kirchen, L., Oberhoffner, T., Guntinas-Lichius, O., Dobel, C., & Schweinberger, S.R. (2020). Parameter-specific morphing reveals contributions of timbre and F0 cues to the perception of voice gender and age in cochlear implant users. Journal of Speech, Language, and Hearing Research, 63(9), 3155-3175. (Link to PDF)
Skuk, V.G., Palermo, R., Broemer, L., & Schweinberger, S.R. (2019). Autistic Traits are Linked to Individual Differences in Familiar Voice Identification. Journal of Autism and Developmental Disorders, 49(7), 2747-2767. (Link to PDF)
Skuk, V.G., & Schweinberger, S.R. (2014). Influences of fundamental frequency, formant frequencies, aperiodicity and spectral level information on the perception of voice gender. Journal of Speech, Language, and Hearing Research, 57(1), 285-296. (Link to PDF)
Skuk, V.G., & Schweinberger, S.R. (2013). Gender differences in familiar voice identification. Hearing Research, 296, 131-140. (Link to PDF)
Young, A.W., Frühholz, S., & Schweinberger, S.R. (in press). Face and voice perception: Understanding commonalities and differences. Trends in Cognitive Sciences.
Zäske, R., Frisius, N., Ivansic, D., Schweinberger, S.R., Guntinas-Lichius, O., & Dobel, C. (2021). Phonetic perception but not perception of speaker gender is impaired in chronic tinnitus. Progress in Brain Research, 260, 397-422.
Zäske, R., Fritz, C., & Schweinberger, S.R. (2013). Spatial inattention abolishes voice adaptation. Attention, Perception & Psychophysics, 75(3), 603-613.
Zäske, R., Muehl, C., & Schweinberger, S.R. (2015). Benefits for voice learning caused by concurrent faces develop over time. PloS One, 10(11): e0143151. doi:10.1371/journal.pone.0143151. (Link to PDF)
Zäske, R., Perlich, M.-C., & Schweinberger, S.R. (2016). To hear or not to hear: Voice processing under visual load. Attention, Perception, & Psychophysics, 78, 1488-1495. (Link to PDF)
Zäske, R., & Schweinberger, S.R. (2011). You are only as old as you sound: Auditory aftereffects in vocal age perception. Hearing Research, 282(1-2), 283-288.
Zäske, R., Schweinberger, S.R., Kaufmann, J.M., & Kawahara, H. (2009). In the ear of the beholder: neural correlates of adaptation to voice gender. European Journal of Neuroscience, 30, 527-534.
Zäske, R., Schweinberger, S.R., & Kawahara, H. (2010). Voice Aftereffects of Adaptation to Speaker Identity. Hearing Research, 268, 38-45.
Zäske, R., Skuk, V.G., Golle, J., & Schweinberger, S.R. (in press). The Jena Speaker Set (JESS) – A database of voice stimuli from unfamiliar young and old adult speakers. Behavior Research Methods.
Zäske, R., Skuk, V.G., Kaufmann, J.M., & Schweinberger, S.R. (2013). Perceiving vocal age and gender: An adaptation approach. Acta Psychologica, 144(3), 583-593.
Zäske, R., Skuk, V.G., & Schweinberger, S.R. (2020). Attractiveness and distinctiveness between speakers ́ voices in naturalistic speech and their faces are uncorrelated. Royal Society Open Science, 7, 201244. (Link to PDF)
Zäske, R., Volberg, G., Kovács, G., & Schweinberger, S.R. (2014). Electrophysiological correlates of voice learning and recognition. Journal of Neuroscience, 34(33), 10821-10831. (Link to PDF)
Zäske, R., Hasan, B.A.S., & Belin, P. (2017). It doesn’t matter what you say: FMRI correlates of voice learning and recognition independent of speech content. Cortex, 94, 100-112. (Link to PDF)
Zäske, R., Limbach, K., Schneider, D., Skuk, V.G., Dobel, C., Guntinas-Lichius, O., & Schweinberger, S.R. (2018). Electrophysiological correlates of voice memory for young and old speakers in young and old listeners. Neuropsychologia, 116, 215-227. (Link to PDF)
