Accessible music sheet renderer and player for blind people (ABC-notation for screenreader usage)
Music Notes Reading App - Concept ABC playback
MusicXML is a universal digital music note exchange format. Which can then be used to create rendered music notes in pdf (svg), to create MIDI or mp3 sound files or to create ABC notation.
music sheet ABC notation
ABC notation for music sheets/scores is common in the Irish community to describe music notes with letters and textual codes. This is very handy for blind people because it can be read by a screenreader or displayed on a braille display.
Tools for music notes recognition
Audiveris is a tool (OpenSource, GUI and commandline) for optical music sheet recognition in order to digitize a music sheet. The recognition sometimes needs visual corrections which can be done within the GUI.
See: https://github.com/Audiveris/audiveris
xml2abc
xml2abc is python CLI tool to convert MusicXML to ABC notation.
MusicXML Samples
Sample files from musicxml.com
Demo websites
Example Website where a given MusicXML file is shown as ABC notation and as visual notation (rendered in SVG). Try out the file BrahWiMeSample.musicxml of the attached musicxml examples.
see https://wim.vree.org/js/xml2abc-js_index.html and a demo with ABC notation on the left side https://wim.vree.org/js/xml2abc-js.html
There is also a nice javascript library (xmlplay) and a xmlplay demo page.
Concept for Project Task
Adapt or create a new webpage that is accessible for a screenreader/braille display user and enables him/her to navigate measure by measure through the music notes. As a braille display has only one line, there must be a field for each of the voices and the text showing the synchronized contents of the current measure. The user should be able to easily jump to a voice or text directly by shortcuts. Have a look a the UI mockup:
Navigation
First of all, the user would like to manually navigate through the measures. When he/she presses Alt+1 the focus jumps to the measure field of voice 1 (The same for voice 2,3,...). If he/she presses Alt+Right, the measure fields change the contents to the respective contents of the next measure. The contents of the other voices is changed for the respective voice at the same time. So she can easily jump between the voices and read the relevant contents for the measure.
Another idea is to show all measures at once and display the ABC notes in a table where each cell represents a measure.
Additional features (Optional)
There are some other tools and webdemos for MIDI, pdf synced playback or playback with youtube videos. You can take them as an example.
Automatic playback (Optional)
Additionally, it would be great to have an automatic playback which can be triggered by pressing Space. In this mode the contents of the measures is time-triggered and changes as soon as the timeline gets into a new measure.
ABC TTS synthetization (Optional)
Additionally, it would be great if the current contents of the measure of the selected voice could be read out loud.
MIDI playback (Optional)
Additionally, it would be gread if the score could be played back by MIDI instruments.