Supporting illiterate people, helping those who are uncomfortable ill at ease with digital technologies, giving them useful tools to better handle with written human computer interfaces, these are the main goals aimed by the sonification team of Orange Labs
Reaching an average rate of 38%, illiteracy is high in Africa. Moreover, many intertwined linguistic communities coexist there. Both factors imply complex and costly text to speech strategies. Yet in Africa, services of mobile telephony are exploding. In 2016, the continent should count up to 600 million of subscribers. In addition, the number of smartphones is foreseen to double from now to 2017 and is expected to rise to 350 million of devices.
In very different contexts, sonification could achieve great possibilities. Whenever some information is visually difficult to convey, or if in peculiar circumstances one cannot simply glance one’s interface, sonification should improve comfort and fun.
What sonification is…
Sonifying a Human Computer interface means translating information into non-verbal audio codes. Our hearing is in this way especially appealed. Till today, the Orange research team’s choice has consisted in giving priority to auditory icons –sort of “audio pictograms”- to acoustically transpose elements of a user interface. When a user encounters some problems with reading- and we all do in front of foreign idioms or different alphabets…- this way is supposed to provide an efficient guidance.
What is an auditory icon?
Any type of icon invites its user to link a figured item [as buttons, pop-up, keys, etc.…] with its meaning. Let us consider visual icons: occidental users will consider the association of a picture of an “envelope” with an e-mail box as natural and obvious. In the same way, the image of a “paper clip” easily conveys the concept of the attached file. Auditory icons are expected to achieve similar goals. They are produced with sounds of everyday life [involving sounds produced by water, earth, fire, thunder, manufactured objects…], and they are conceived to “whisper” to anyone the meaning of the functions they are associated with. This is the main reason they are easy to understand: a hearer has only to recall everyday life to hear the targeted meaning. So, making a strong analogy with a past situation may allow anyone him to understand the meaning of the figured function.
How do auditory icons challenge us?
To produce an auditory icon, a clever association has to be made between one interface element and the most relevant and widespread audio metaphor.
In that purpose, if audio metaphors are expressed as verbal ones, the questions having to be answered are for example: “what “jumping to the next step” sounds like?”… or:
“what “confirming my choice” sounds like?”
These questions sometimes remain hardly solved. In our case, finding relevant acoustic metaphors requires the cognitive most common processes to be reversed. The issue becomes to start from the meaning of a concept and to go back to the source after having identified a fitting sound. Usually we are -most of the time- used to perceiving a sound and interpreting it, after going through the identification of its components.
It is undeniable that auditory icons also express a wide part of dreams, symbols, and daily life experiences of the targeted user’s. When Europeans immediately associate to emergency the firemen wee woo, will the Dogon inhabitants do the same? Let us wonder if the abrupt stop of birdsong is interpreted the same by both hearers in a Londoner’s garden and inside the jungle, where wild animals are close to you. Probably not.
It is easy to figure out: the diversity of possible sources of sounds [environment, nature, fauna, musical instruments…], the variable contexts of production [countries and their configurations], and wide ranges of symbolic codes [through myths and collective unconsciousness]…. all imply that auditory icons reflect a multiplicity of cultural questions.
How to sonify…
To look for the right metaphors and to achieve a good work, a tight association with members or experts of the targeted community is needed. Co construction is essential; otherwise the relevant auditory icons cannot be produced.
The mobile banking “Orange Money” app is already widely implemented in West Africa countries. A sonified prototype of OM has been realized. The achievement of the best metaphors has been led with the collaboration of persons coming from several countries of West Africa.
All along successive workshops, three transversal goals are carried on: the first of all is the necessity to identify some referential relevant cultural audio areas [within musical worlds, or daily life experiences, or natural environments] ; then the wish to maintain a maximized coherence within these universes, to finally manage to select the most intuitive “sounds-functions” associations.
At last, the best relevant metaphors are sonorously produced by an audio designer. Their implementation is then transferred to one or several human computer interfaces.
Auditory icons: examples
As an illustration, let us consider two suitable examples. They transpose the concept of an on-going process. It is to say that a message is conveyed by the system to encourage the user to calmly wait the completion of the process before trying anything further.
The concept is easily figured by several widely understood visual pictograms: like an hourglass, a progressing bar or a turning wheel. Nevertheless, none of those icons provides or suggests any sound to human ears.
Three examples for visual icons figuring “waiting”
Three examples for audotory icons figuring “waiting”
For the focused item, two acoustic metaphors are available and explained as following:
Listen to sound 2 :
Within the research project framework, Orange Money auditory icons have been submitted to several members of the West African community. It is undeniable that the terminal devices acoustic characteristics have a strong influence on the auditory icons interpretation. It is why these factors do play a role in the present achieved measures and tests.
In addition, all cultural features, all diversity of contexts of use [alone or not, at home or outdoor, in the bus, walking in the street…], and the specificities of the local mental representations are essential factors for a good comprehension. That is why we keep on tightly studying them. Next tests will very soon take place in the African targeted countries, with various people and suitable populations. Thanks to many observations and to measures of precise indicators, they will confirm the rich possibilities of the principles and means of this type of sonification.
Considering the diversity of all digital services, apps, situations.., the power of auditory icons appears as both durable and strong. However, to improve the understanding of digital devices by auditory icons in order to lower the users’ remaining difficulties, some questions still remain.
Beyond the African west countries borders, we wonder if some of these auditory icons are still understandable by other populations or cultures. May we so consider the emergence of a really efficient “language of sounds”, with its syntax, grammar and vocabulary?