Paolo Ciuccarelli and I have just published for a series of reflections on using sound to communicate socially relevant issues, the responsibility that comes with it, and how design as an intentional approach to creating experiences and artifacts for people, could help. The contribute
Here’s the blog post we wrote for Big Data & Society blog. You find the full article, "‘Intentionality and Design in Data Sonification of Social Issues’, in Open Access, for Big Data & Society DEMOS. Demos are are annual multi-media demonstrations focused on new methods, visualizations, experiments and approaches to the analysis of Big Data and curated by the Editorial Team of Big Data and Society
————
Data sonification is the practice of representing data through sound. From the sonar to the Geiger counter, from the beep of an electric car approaching or the buzz of a forgotten seatbelt, the use of sound to convey information is not new. An official, academic definition of data sonification dates back to the 1990s, when the International Community for Auditory Display was founded. From that point forward sonification has been applied and debated as a possible means of representing numerical data in support of scientific analysis. Early fields of application include the auditory translation of seismic waves into audible frequencies, the use of sound to complement visualization of astronomical data, sonification of stock markets historical data to predict markets’ behaviour, and the representation of ECG data as sound. As data has dramatically expanded into everyday life questions about how data are represented and communicated has become a key issue in particular, in the case of data dealing with complex social issues.
Traditional visual representational methods struggle to support publics in gathering insights and engaging with complex, abstract, multi-dimensional sets of data so that awareness and knowledge is increased. In this context, the use of sound - a sensory modality often described as engaging listeners at a visceral, intuitive level - to represent data has, in recent years, gained momentum. But how does a practice born to support experts in performing task-specific, data-driven scientific analysis transition to a means of mass communication addressing relevant social issues for a non-expert audience? One might look to other sound-related disciplines that have proven extremely successful in complementing audiences’ visual experiences. For instance, sound design for film and video gaming, both well- established design practices with a solid theoretical framework and structured educational curricula, focus attention both on the aesthetic component of the auditory experience and the efficiency and efficacy of the informative content. As a design discipline, sound design intentionally addresses the needs and the goals of the listener to build a purposeful, contextualised experience that engages as well as informs a non-specialised audience. The data sonification research community is explicitly advocating for the inclusion of such design practices into the process of creating sonifications for non-expert listeners. Such an approach will inevitably lead authors/designers to make intentional, deliberate decisions to address the specific needs of a given context and with a purpose, when transforming data into sound.
In this article we introduce intentionality as a framing condition for sonification as a design process. Through the lenses of intentionality, we analyse five recent projects of sonifications which aim to engage publics with social issues. Taking into consideration explicit statements of the authors/designers of these sonifications, we distribute the works on a scale from a higher to a lower degree of intentionality.
At the highest degree of intentionality sits Egypt Building Collapses by the activist group Tactical Technology Collective. In this work, data on one year of accidents involving the sudden collapse of residential buildings in Egypt are visualized on a dedicated website to raise awareness of a serious issue affecting Egyptian society, that resulted in 192 casualties and more than 800 homeless families in one year. The sonification takes the form of a 2.35 minute-long soundscape built on the occurrences of such accidents over one year and uses real sounds of collapsing buildings, an explicit choice made by the authors to use sound as a ‘connecting element’ between the real experience of the phenomenon (a building collapsing) and the abstract figures (the data) representing it. At the opposite pole of our intentionality scale sits Two Trains, by the sound artist Brian Foo, which uses music samples scoured from the Internet to represent income inequality in the areas crossed by Line Two of the New York City’s subway. In the words of the author, the selection of ‘agnostic sound traits’ will allow data to ‘speak for themselves’.
But can data really speak from themselves? What is the role of the author in shaping the message conveyed to the listener through sound? Through the discussion of these cases and others sitting at different points in the scale of intentionality we recognize the inevitability of a communicative relationship in every translation process - and the need to design it intentionally and responsibly for data sonification to become an important medium of communication for a wider audience.