ENFORMATİK ENSTİTÜSÜOYUN TEKNOLOJİLERİ Programı


Audio for Games and Virtual Environments

Instructor: Assist. Prof. Dr. Hüseyin HACIHABİBOĞLU

 

The course will first aim to bring all students to the same level on fundamental concepts of digital audio, including, sampling, frequency-domain representations, filtering, and time-frequency analysis. Human auditory perception, including concepts such as critical bands, perception of pitch, timbre, loudness, rhythm, and texture; as well as masking will be covered. Spatial auditory perception will be covered specifically including its aspects relevant to game audio such as binaural hearing, localization of sound sources in 3D space, and distance perception. Concepts related to room acoustics modelling, simulation and auralization will be covered from game audio perspective.  Artificial reverberators which are strongly related to perception-based simulation of room acoustics will be covered in detail. Most important existing and emerging spatial audio reproduction techniques will be covered. Subjective evaluation of spatial audio systems and algorithms including experiment design, ITU recommendations, and statistical analysis of results will be discussed. Coding of spatial audio for use in computer games and virtual reality applications will be given with existing MPEG standards. Discussion of sound synthesis will include a brief review of existing speech, musical instrument, and environmental sound synthesis methods. Special emphasis will be given to expressive synthesis and morphing techniques that allow using the same sound synthesis method to generate perceptually similar but not identical sounds. Finally, existing game audio APIs will be reviewed.

By successful completion of this course the students will:
  • Learn the basic concepts of digital audio
  • Learn the basics of human auditory perception and spatial hearing
  • Learn how acoustics of enclosed spaces can be modelled, simulated, and auralized and how these models can be incorporated to computer games and virtual reality applications
  • Gain knowledge on different 3D audio systems and usage scenarios
  • Learn about subjective evaluation of game audio algorithms
  • Have a working knowledge of different sound synthesis methods
  • Gain familiarity with different audio APIs
  • Develop audio algorithm development skills for games and virtual reality applications
  • Improve their technical writing skills

 

Outline

  • Introduction
  • Fundamentals of digital audio
  • Human auditory system and psychoacoustics        
  • Spatial hearing
  • Room acoustics modelling, simulation, and perception
  • Artificial reverberators
  • Reproduction of spatial audio for games
  • Subjective evaluation of spatial audio
  • Audio coding for computer games
  • Sound synthesis algorithms        
  • Existing game audio tools and APIs


Audio for Games and Virtual Environments