OK! Let’s charge right in to our next modality up for consideration: audio. Having recently worked on an audio project now being groomed for possible publication, this topic has really been at the forefront of my mind, so I’m excited to dig in. Let’s go!
Digital texts are often very visual, so accessibility for those with hearing disabilities may sometimes go unconsidered, or dismissed with the simple addition of subtitles to videos. However, as those of us who have composed exclusively or primarily audio works can attest, there is an immense amount of information that can be conveyed through subtleties of sound, far beyond the words being spoken. The key questions for us to ask ourselves here are 1) what information is being conveyed through audio? and 2) how can we bring that information into another mode?
Subtitles & Closed Captioning
First, let’s address that most obvious first step in audio accessibility: subtitles. More specifically, the difference between subtitles and closed captioning. In many film and video productions, a subtitle track is added which presents all spoken dialogue in text–sometimes in varying languages–at the bottom of the screen. While this is helpful for hearing viewers who don’t understand the language used in the film, hearing impaired viewers are losing out on a lot more than just the dialogue. Closed captioning adds contextual information to the text track, allowing hearing impaired viewers to experience the fullness of a scene. Includification offers this example to help make clear the difference between subtitles and closed captioning and to demonstrate the importance of true closed captioning for the hearing impaired:
Sally: Good morning, John
[Sally knocks on the door.]
Sally: Good morning, John.
[The floorboard creaks underfoot.]
In addition to adding a closed captioning track to video, these considerations can be incorporated into text transcripts that accompany audio works. Podcasts have seen a rise in popularity in recent years, and many podcasts provide text transcripts of their content. However, these transcripts do not always contain contextual information. Consider the wildly popular and genre-shaking podcast Serial. While the official website for Serial does provide a lot of supporting media in text and image form (scans of documents, charts, timelines, maps), there actually are no official transcripts of the podcast itself. In a great display of the widespread desire for accessibility options, the Serial discussion group on Reddit has collectively produced a full set of text transcripts for the podcast, as well as a FAQ to help navigate the series. The website Genius, which allows users to share and collectively annotate texts, also features user-created text transcripts of Serial. While these transcripts are essentially the same as the Reddit transcripts–dialogue only–the annotations take a step in the right direction by allowing users to add more contextual information. Still, the official Serial site devotes a whole section to crediting those who have composed music specifically for this work; what might be lost to a reader who doesn’t hear that music? What is lost in the absence of human voices, their emotive qualities beyond the words they use? A dialogue-only subtitle track or text transcript may increase accessibility for some of the more explicit ideas in an audio work, but a closed-captioning-style approach would come much closer to providing a genuinely accessible experience.
But wait! Aren’t we multimodal composers? Can’t we do more than just turn everything into a traditional alphabetic text? Sure we can! Visual cues are a great way to reinforce some of the contextual information contained in your audio track. Includification considers this option with regard to video game design, describing games that use flashing on-screen warnings or an increasingly obscured field of vision to indicate a character’s decreasing health. These kinds of visual cues are often used alongside audio cues, which not only increases accessibility but also deepens the player’s immersion in the game experience. In dealing with an audio text like Serial, visual cues could take a variety of forms. Text transcripts could be accompanied by images of characters and settings (photographs or even artists’ interpretations), to fill in contextual information otherwise provided by voice and ambient sound. The layout, background, and font choices made for a digital text can all be chosen with considerations of mood and tone, elements often provided by music in an audio work. Even within the text itself, font size, style, and color might be adjusted to reflect shifts in voice. A particularly ambitious and interesting project might even be to attempt to translate an audio text into a silent video text, if nothing else than as an experiment in how audio information can be conveyed visually. This is an area where I think there are so many questions, but just as many possibilities, and all are worth exploring and experimenting.
While it may not be a widespread option yet, there is another way in which video games are approaching audio accessibility that is just starting to reach other formats. Tactile modalities, engaging the sense of touch, are generally overlooked and underutilized in digital media. This is largely because not all media formats have the technical capabilities for tactile engagement, but video games have long used vibration–or haptic feedback–to communicate information and increase player immersion. From Sega’s Moto-Cross arcade game in 1976, which vibrated its handlebars on collision, to Sony’s DualShock 4 controller for their latest PlayStation console, which features a capacitive touch pad and motion detection as well as vibration, video games provide some of the most interesting and diverse applications for communicating a digital text experience through touch. Outside of video game technologies, however, options become much more limited, though hopefully that is beginning to change. Mobile devices (such as cell phones) use haptic feedback technologies as well, but have primarily used it in fairly limited ways, such as a simple buzz for a notification or soft feedback to imitate a button press when using a touch screen. Apple’s new smartwatch seems to be trying to take haptic feedback a little further. With new “Force Touch” and “Taptic Engine” technology, Apple is playing with new ways for an electronic device to interact with a user. Apple Watch wearers can even send one another their own heartbeat, read from the sender’s pulse and communicated to the receiver through gentle haptic feedback. While the practical use of these technologies may still be undetermined, developing new ways to communicate through underutilized modes may have incredible possibilities for composition, as a way of increasing accessibility and as a whole new mode of expression.
It is worth noting too that, while digital technologies that engage tactile modes aren’t yet available to everyone, multimodality does NOT have to be digital! Tactile and kinesthetic engagement are great reasons to play with physical artifacts as a mode for composition. One particularly interesting use of physical artifacts as a supplement to a traditional text is artist Miranda July’s novel The First Bad Man, which was accompanied by a series of online auctions allowing readers to buy physical objects referenced in the book. The objects ranged from a brush with blonde hair in it (sold for $93) to a small gold crucifix (sold for $117.50) to a post-it note (sold for $355)–all proceeds, of course, went to charity. Like Apple’s smartwatch heartbeat, this is certainly more art than practical accommodation, but this kind of experimentation and play can be viewed as inspirational, as a jumping off point for thinking about how we might bring our texts into the physical world.
Accessibility is for Everyone
Audio accessibility and accommodation for the hearing impaired is one area where, to me, it becomes more clear than ever just how much accessibility is not about disability. Providing audio-free accessibility can benefit many people in many different ways. Includification uses the example of Baby-Friendly Settings, which they describe as “the idea that those who are parents trying to play video games should be able to do so at 3 in the morning with the sound disabled and the baby sleeping right beside them”. This is just one example of why audio might be inconvenient, undesirable, or inaccessible for a hearing person. Many offices and other professional environments may be inappropriate for listening to an audio track, which means including audio-free accessibility can increase your chance of being heard in these situations. In fact, I think this kind of accessibility stands out to me the most because I work in a library, where quiet is an absolute must! I often find myself simply skipping video and audio content I encounter online while I’m at work, no matter how interesting I find it. Even when I plan to come back to it later, I rarely do; the moment is lost. Those who can hear the audio may still need additional help understanding it. Someone unfamiliar with the language used in audio work can much more easily seek translation assistance with an alphabetic text, or gain understanding through recognizable symbols and images. The point is this: the more accessibility options you can provide for your text, the more people you will reach–any person, any time, any place. And who doesn’t want that?
My next and (probably?) final post in this series will be about a kind of disability that is far too often overlooked despite having a significant impact on a huge number of people–including myself. Cognitive disabilities: what are they? how do they cause problems with accessing texts? what can we do to increase cognitive accessibility? Stay tuned to help me figure it out.