New(s) Readers: Multimodal Meaning-Making in AJ+ Captioned Video

IntroductionIn 2013, Facebook introduced autoplay video into its newsfeed. In order not to produce sound disruptive to hearing users, videos were muted until a user clicked on them to enable audio. This move, recognised as a competitive response to the popularity of video-sharing sites like YouTube, has generated significant changes to the aesthetics, form, and modalities of online video. Many video producers have incorporated captions into their videos as a means of attracting and maintaining user attention. Of course, captions are not simply a replacement or translation of sound, but have instead added new layers of meaning and changed the way stories are told through video.In this paper, I ask how the use of captions has altered the communication of messages conveyed through online video. In particular, I consider the role captions have played in news reporting, as online platforms like Facebook become increasingly significant sites for the consumption of news. One of the most successful producers of online news video has been Al Jazeera Plus (AJ+). I examine two recent AJ+ news videos to consider how meaning is generated when captions are integrated into the already multimodal form of the video—their online reporting of Australian versus US healthcare systems, and the history of the Black Panther movement. I analyse interactions amongst image, sound, language, and typography and consider the role of captions in audience engagement, branding, and profit-making. Sean Zdenek notes that captions have yet to be recognised “as a significant variable in multimodal analysis, on par with image, sound and video” (xiii). Here, I attempt to pay close attention to the representational, cultural and economic shifts that occur when captions become a central component of online news reporting. I end by briefly enquiring into the implications of captions for our understanding of literacy in an age of constantly shifting media.Multimodality in Digital MediaJeff Bezemer and Gunther Kress define a mode as a “socially and culturally shaped resource for meaning making” (171). Modes include meaning communicated through writing, sound, image, gesture, oral language, and the use of space. Of course, all meanings are conveyed through multiple modes. A page of written text, for example, requires us to make sense through the simultaneous interpretation of words, space, colour, and font. Media such as television and film have long been understood as multimodal; however, with the appearance of digital technologies, media’s multimodality has become increasingly complex. Video games, for example, demonstrate an extraordinary interplay between image, sound, oral language, written text, and interactive gestures, while technologies such as the mobile phone combine the capacity to produce meaning through speaking, writing, and image creation.These multiple modes are not simply layered one on top of the other, but are instead “enmeshed through the complexity of interaction, representation and communication” (Jewitt 1). The rise of multimodal media—as well as the increasing interest in understanding multimodality—occurs against the backdrop of rapid technological, cultural, political, and economic change. These shifts include media convergence, political polarisation, and increased youth activism across the globe (Herrera), developments that are deeply intertwined with uses of digital media and technology. Indeed, theorists of multimodality like Jay Lemke challenge us to go beyond formalist readings of how multiple modes work together to create meaning, and to consider multimodality “within a political economy and a cultural ecology of identities, markets and values” (140).Video’s long history as an inexpensive and portable way to produce media has made it an especially dynamic form of multimodal media. In 1974, avant-garde video artist Nam June Paik predicted that “new forms of video … will stimulate the whole society to find more imaginative ways of telecommunication” (45). Fast forward more than 40 years, and we find that video has indeed become an imaginative and accessible form of communication. The cultural influence of video is evident in the proliferation of video genres, including remix videos, fan videos, Let’s Play videos, video blogs, live stream video, short form video, and video documentary, many of which combine semiotic resources in novel ways. The economic power of video is evident in the profitability of video sharing sites—YouTube in particular—as well as the recent appearance of video on other social media platforms such as Instagram and Facebook.These platforms constitute significant “sites of display.” As Rodney Jones notes, sites of display are not merely the material media through which information is displayed. Rather, they are complex spaces that organise social interactions—for example, between producers and users—and shape how meaning is made. Certainly we can see the influence of sites of display by considering Facebook’s 2013 introduction of autoplay into its newsfeed, a move that forced video producers to respond with new formats. As Edson Tandoc and Julian Maitra write, news organisations have had been forced to “play by Facebook’s frequently modified rules and change accordingly when the algorithms governing the social platform change” (2). AJ+ has been considered one of the media companies that has most successfully adapted to these changes, an adaptation I examine below. I begin by taking up Lemke’s challenge to consider multimodality contextually, reading AJ+ videos through the conceptual lens of the “attention economy,” a lens that highlights the profitability of attention within digital cultures. I then follow with analyses of two short AJ+ videos to show captions’ central role, not only in conveying meaning, but also in creating markets, and communicating branded identities and ideologies.AJ+, Facebook and the New Economies of AttentionThe Al Jazeera news network was founded in 1996 to cover news of the Arab world, with a declared commitment to give “voice to the voiceless.” Since that time, the network has gained global influence, yet many of its attempts to break into the American market have been unsuccessful (Youmans). In 2013, the network acquired Current TV in an effort to move into cable television. While that effort ultimately failed, Al Jazeera’s purchase of the youth-oriented Current TV nonetheless led to another, surprisingly fruitful enterprise, the development of the digital media channel Al Jazeera Plus (AJ+). AJ+ content, which is made up almost entirely of video, is directed at 18 to 35-year-olds. As William Youmans notes, AJ+ videos are informal and opinionated, and, while staying consistent with Al Jazeera’s mission to “give voice to the voiceless,” they also take an openly activist stance (114). Another distinctive feature of AJ+ videos is the way they are tailored for specific platforms. From the beginning, AJ+ has had particular success on Facebook, a success that has been recognised in popular and trade publications. A 2015 profile on AJ+ videos in Variety (Roettgers) noted that AJ+ was the ninth biggest video publisher on the social network, while a story on Journalism.co (Reid, “How AJ+ Reaches”) that same year commented on the remarkable extent to which Facebook audiences shared and interacted with AJ+ videos. These stories also note the distinctive video style that has become associated with the AJ+ brand—short, bold captions; striking images that include photos, maps, infographics, and animations; an effective opening hook; and a closing call to share the video.AJ+ video producers were developing this unique style just as Facebook’s autoplay was being introduced into newsfeeds. Autoplay—a mechanism through which videos are played automatically, without action from a user—predates Facebook’s introduction of the feature. However, autoplay on Internet sites had already begun to raise the ire of many users before its appearance on Facebook (Oremus, “In Defense of Autoplay”). By playing video automatically, autoplay wrests control away from users, and causes particular problems for users using assistive technologies. Reporting on Facebook’s decision to introduce autoplay, Josh Constine notes that the company was looking for a way to increase advertising revenues without increasing the number of actual ads. Encouraging users to upload and share video normalises the presence of video on Facebook, and opens up the door to the eventual addition of profitable video ads. Ensuring that video plays automatically gives video producers an opportunity to capture the attention of users without the need for them to actively click to start a video. Further, ensuring that the videos can be understood when played silently means that both deaf users and users who are situationally unable to hear the audio can also consume its content in any kind of setting.While Facebook has promoted its introduction of autoplay as a benefit to users (Oremus, “Facebook”), it is perhaps more clearly an illustration of the carefully-crafted production strategies used by digital platforms to capture, maintain, and control attention. Within digital capitalism, attention is a highly prized and scarce resource. Michael Goldhaber argues that once attention is given, it builds the potential for further attention in the future. He writes that “obtaining attention is obtaining a kind of enduring wealth, a form of wealth that puts you in a preferred position to get anything this new economy offers” (n.p.). In the case of Facebook, this offers video producers the opportunity to capture users’ attention quickly—in the time it takes them to scroll through their newsfeed. While this may equate to only a few seconds, those few seconds hold, as Goldhaber predicted, the potential to create further value and profit when videos are viewed, liked, shared, and commented on.Interviews with AJ+ producers reveal that an understandin

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.