Category Archives: Information

Text in images, a reflection of our visuality

We’ve fallen in love with putting text in image formats on the Internet.

On some level, this is nothing new. For as long as humans have been making art, text and imagery have intermingled. To speak only of the computer age: Photoshop has a text tool, after all, and wherefore a tool if not to be used? One of the earliest viral images in my memory, from 2005, is the O RLY? owl, seen below. And then of course, we’ve long had Someecards and Lolcats (both ca. 2007).


Traditional usability heuristics condemn embedding text within images, because text in images can have problems with readability, and it’s obviously not searchable. And, of course, HTML canonically separates image tags from text tags. Moreover, if we close our eyes and are asked to think of an image, whatever comes to mind will likely not include any text. It seems, at least prototypically, that images and text are meant to be separate.

But, if anything, our images are only getting more text-laden. Wikipedia recently enhanced their Android app in order to allow users to share text extracts from articles as images. An example appears below.


On the surface, this seems absurd. What’s going on?

I think it has to do with social media. Platforms like Tumblr have seemed to work best with images, and users on this site have long preferred text embedded within images. For a while I ran a Top Chef–inspired blog, where I published screenshots from recent episodes along with their captions, written as text; a friend urged me to incorporate the text into the image so she could share the images on her own tumblog more easily.

This trend seems to have crept into other social media platforms, too: Facebook’s algorithm seems to favor images over text in constructing your newsfeed. Since 2013, Twitter has automatically shown images within its stream, whereas previously the service was text-only. Thus sharing text in images on Facebook makes them more visible, and sharing images on Twitter allows users to bypass the 140-character message limit.

Whether these things are causes of our text-in-image frenzy or co-conspirators is an open question.

Perhaps it reflects our society’s orientation toward the visual rather than the oral, which seems to be getting stronger and stronger. (Think about how “no one reads anymore” but how “300 hours of video are uploaded to YouTube every minute.”)

In a recent interview, social researcher danah boyd discussed that this orientation has had ramifications in other areas of our lives, such as how we now navigate conflict. In the workplace, she says, if there is a problem between two employees, they won’t deal with it verbally, as employees might have done in the past. Instead, they will use a written medium, such as email, to express their arguments—and perhaps protect themselves. The bottom line is that, as our lives are structured, we have more exposure to and comfort with textual, visual environments.

And yet, orality is primary, whereas literacy (and visuality) is secondary. What does this mean for the future?

Visual bias in the characterization of knowledge

We live in a world that privileges written information over oral information.

It’s a bias of the eyes over the ears, and it seems to include our entire conceptualization of knowledge. Just look at the metaphors we use in our everyday language: A bright person is a smart one, as is a brilliant person. (Brilliance has all but lost its meaning except that denoting intelligence.) We talk of the Enlightenment as the birth of modern science and enlightened people as the smartest among us. We say, “Do you see what I mean?” and “Can you picture it?” even when there’s nothing to be really seen or pictured.

Why is this? Of course, it’s an apt metaphor: In the dark, we can’t see, and we don’t have knowledge of what might be lurking. In the light, we can see, and we know. But why privilege the eyes over the ears? Human language is, after all, primarily manifested as sound.

Perhaps this bias has roots in the traditional permanence of records. Things that were chiseled in stone, brushed in ink or set in type were meant to last a long time. The letters couldn’t fritter off into oblivion, get easily erased and be rewritten, etc. But nowadays, more and more of our written information is not recorded in the same way. Electronic information is malleable. Sure it’s redundant over any number of Google-sized caches (and, if you put stock in the 2014 novel Whisky Tango Foxtrot by David Shafer, perhaps being stored at the bottom of the ocean for future nefarious purposes), but in everyday cases, words can be reappropriated, edited and republished with supreme ease. If I change my mind about a blog post, I can revise or even unpublish it. If I am angry with a commenter, I can change their words. On Facebook, if I am embarrassed by something I posted earlier, I can delete or change it. Moreover, we have just so much information streaming by in blogs and social media, that what’s here today can be very difficult to find tomorrow.

Maybe today’s written information isn’t much more permanent than oral information after all. (And that isn’t even to speak of sound and video recordings.) I think that means it’s time to put more stock in other modalities of information transfer, don’t you?

Reading as transaction

It’s probably happened to you before: There was some article, poem or story that you really loved, and you told your friend to read it. They did, and maybe they said, “Yeah, it was good,” but they just didn’t seem affected by it like you were. How do we account for two people having vastly different experiences with the same piece of text?

Louise Rosenblatt was a researcher who wanted to understand how humans experienced literature. She proposed a model (based on the work of philosopher John Dewey in his famed Art As Experience) wherein both the reader and the text are active participants in the encounter we call reading. In her words:

Instead of two fixed entities acting on one another, the reader and the text are two aspects of a total dynamic situation. The “meaning” does not reside ready-made “in” the text or “in” the reader but happens or comes into being during the transaction between reader and text… “Meaning” is what happens during the transaction.

The act of reading, then, is the fusion of a person, along with their lifetime experiences, thought associations, current state of mind and interests, with the text.

According to Rosenblatt, these transactions result in experiences along the efferent–aesthetic continuum. Efferent experiences are characterized by intellectual analysis, focusing on information to be extracted from the encounter for later conscious, public use. Aesthetic experiences, on the other hand, are private. They are artistic and of-the-moment, incorporating the background, emotional state and thoughts of the beholder. Aesthetic experiences are moments of epiphany—almost spiritual. They are when a piece of writing, as we often say, strikes a chord with us.

Efferent–Aesthetic Continuum

Rosenblatt highlighted that, as we read, we are constantly navigating this continuum. Some parts of a text might strike us more aesthetically, and others more efferently. Even reading the driest of academic treatises might spark moments of aesthetic, just as reading the most beautiful poem might trigger efference.

Now, Rosenblatt focused on the experience with literary texts, but there is so much more to consider. In a recent article in the Journal of DocumentationKiersten Latham extended this model to describe experiences with museum objects, demonstrating its versatility in describing information behavior. But even in the world of text, there is so much more than just the text itself: There is the typography, the kind of paper (or screen!), the print quality… All these factors contribute to the reading experience, even though we don’t often think about it. (Granted, the growing pervasiveness of e-readers in recent years has actually caused many more of us to consider this than otherwise would have.) Latham notes, for example, that “many people talk about the handling of the paper as part of their reading experience.”

What is the point of theorizing here? I think questions of person–document experience are important in today’s archive environment. As we rush to digitize everything, are there aspects of the physical experience that we are depriving future generations of? What can we learn from the study of information experience to hedge these losses? Where should we direct our efforts as we develop new technologies (e.g., high-density displays, haptic response)?