Haptic Music - A Music Staff Reading Interface for Visually Impaired
Malta, Carolina Sofia Gouveia
MSc Thesis · 2024 · University of Coimbra
Supervisors: Fernando Amílcar Bandeira Cardoso, Tiago José dos Santos Martins da Cruz
Abstract
Musical practice should be accessible to everyone. Playing an instrument while looking at a music sheet is a feeling of freedom, which frees our memory and allows us to focus on interpretation. This project aims to give that same freedom to all those who are visually impaired and therefore conditioned to practise an instrument from memory. In concrete, we aim to design and develop a device that can allow a visually impaired person to read a music staff without the need to mobilise the hands, freeing them up for instrument practice. With this challenge in our hands, and given the complexity of the task, we realised we could only strive to consider a small part of the music symbols of a staff, so we decided to constrain the input to the notes of simple scales. We had plans to make something people could wear while playing an instrument. We thought that micro motors programmed using Arduino could be a viable option. However, we found a wearable already finished at our disposal: a TactSuit x40 from bHaptics, designed for virtual reality games. It is made of EMR motors distributed in the front and back within a grid of four to five actuators. The method we followed to assemble this project was Design Science Research. Due to its iterative nature, we could easily navigate the phases based on whether we needed to step back or forward. To pass the information on the sheet music to the vest, we created various shapes and patterns of stimuli that were later tested and implemented. Furthermore, we also created a method to replicate the Braille writing system to make the work more accessible based on their background knowledge. The test results showed us that the vest could respond to what was needed for this first phase. Nevertheless, we also concluded that the vest was unbalanced in the superior part because of the hardware box on top of the first line of actuators. In the future, we will continue writing an article and demonstrate the work we have accomplished to attract possible investors for the project. We will produce a wearable focused on the stakeholders' needs that can respond fully to the translation of a music staff.