The reading experience

What’s a book? We have a number of ways of answering that question. On one hand, a book is a physical object, a rectangular thing with pages held together with binding. My copy of Candide is a book. On the other hand, we also think of a book as the words that inhabit that object. When someone sees you reading Candide and says, “I’ve read that book,” they probably don’t mean they read the same copy you’re reading, but rather a different copy—perhaps from the library. So there’s a sense that all the different copies of Candide in the universe are somehow the same book. Nowadays, these copies aren’t all in print—we have web-based text versions and audiobook versions, for instance.

Thinking about books in this way can serve us well enough, but we have to be careful. Different technologies have different possibilities for use—for instance, a tablet is multifunctional, whereas a printed book only does one thing—and exist in different social climates—for instance, a person may not feel comfortable using their smartphone in a certain part of a city. Both these things present affordances which constrain and enable how we use technology. And since books are presented through technology (we don’t live in the world of The Matrix yet), each book has a set of affordances through which a reader’s interaction with that book unfolds.

In many cases, a text presented in different technologies might be close enough that we’d be justified calling them all the “same.” For instance, the book White Noise by Don DeLillo presents a narrative that seems to be conveyed similarly whether it’s read in print or listened to in audio form.

But some authors specifically take advantage of the possibilities of a technology in making their book. I call this “entanglement,” because in books like this, the form and the content become entangled to a degree that can’t reasonably be ignored. An example of this is the book The Neverending Story, in which different colors are used in order to present the narrative—and the narrative is diminished in editions that print it in only black ink.

At iConference this year, I presented a study I did on people’s experiences reading two books I felt were entangled in a similar way—Ulysses, by James Joyce, and Infinite Jest, by David Foster Wallace—to see how people experienced these books in print, audiobook and Kindle formats. What I found was that, though people in all formats “got” what the books were about in terms of content (plot, themes), only the people who read the print editions talked about having deep, transformative, personal experiences with the books.

Now, this isn’t to say that print is better and that’s the end of the story. Rather, what I conclude from this study is that a book is likely best in the form its author intended, particularly when it is entangled. When a book is adapted, it is wrested from that entanglement and becomes, as a result, impoverished. While this cannot be seen if we think of a book as just the verbal content of its text, it is quite visible when the reader’s experience is taken into account. What’s important to note in all this is that audiobooks can be entangled just as can movies and webcomics.

What results is an important lesson for publishers and designers who are adapting books from one format to another: Put the reader’s experience first.

If you’re interested in reading more, I’d invite you to read my paper “Novel Experiences: On Page, In Ear, On Screen,” which is available in the open-access iConference 2016 proceedings. You may also be interested in the book Scrolling Forward: Making Sense of Documents in the Digital Age, by David Levy, and others that I have listed on my Reading List page.

In Document we trust

How do we decide what information can be trusted? There are, of course, innumerable factors that come to play in that decision: the stakes involved, the subject matter, the situation, time constraints…

But if we had to draw out some generalizations when it comes to trusting information, it seems that we inherently trust some kinds of information more than others. Broadly, we can talk about this in terms of formal and informal information. Formal information is information that is organized, structured and traceable by others, and examples include a printed book, an agendaed meeting and a computer database. Informal information, on the other hand, is disorganized, unstrctured and untraceable, and examples include a meandering conversation over coffee, a handwritten memo and smalltalk. If you’re like me, these two categories may immediately strike you as fuzzy; indeed, I think we can best think of formal and informal as poles on a continuum rather than distinct binary categories.

Given this distinction, I’d posit that we inherently trust formal information more than informal information. Broadly speaking, that is. In academia, it’s de rigueur to reference a written source but generally unacceptable to reference a hallway conversation—even if the conversants are established experts. In everyday life, we may not believe what people say unless they can back it up with “sources.” Of course, there are some people we would trust just based on who they are—a loved one, perhaps, or a tour guide—which is an interesting phenomenon to explore further. For now, though, I’ve been thinking about why it is that we—again, broadly and in general—trust formal information more than informal information.

I broached the subject in a recent paper in the Journal of Documentation called “How the Document Got Its Authority.” In that paper, I suggest that human trust is based on persistence (that is, unchangingness), and we trust formal information (that is, documents) because they are persistent.

This spray-painted "No Parking" sign doesn't seem to have much authority
This spray-painted “No Parking” sign doesn’t seem to have much authority

And the more formal a document is, the more trustworthy it seems to become. For instance, near my apartment in Philadelphia I noticed a “sign” spray-painted in the parking lane that said NO PARKING. People didn’t seem keen to obey, as you can see in the photo to the right. To me, it seems that this is because a spray-painted “No Parking” sign is not as authoritative as the metal, permanent “Yes Parking” sign on the post nearby.

The feature of human cognition that seems to encourage us to trust persistent things more than less-persistent things can make us vulnerable. Not everything permanent is to be trusted. In the article, I give the example of The Onion, a satirical newspaper from my home state (Wisconsin), which was sometimes mistaken as “real” news because of the way it mimicked the newspaper format.

cato-phony-not-sciBut less innocuously, we can sometimes fall prey to those with pernicious intentions. I recently saw the documentary Merchants of Doubt, for instance, which details how big companies manipulate the public. The film talks about the cigarette industry, for example, and how cigarette companies used manipulative tactics to make the science behind the unhealthiness of smoking seem less clear-cut than it actually was. And the same thing is happening regarding climate change in recent years: Virtually all scientists agree that human-exacerbated climate change is really happening, but a few corporations have managed to put out information that is formal and seemingly authoritative enough to garner respect, making the scientific community seem like it doesn’t have the consensus it does. In the image (source) above and to the right, you can see the legitimate scientific report on the left, and the corporate-sponsored “rejoinder” to the right. The rejoinder features fabricated information that closely mimics the format of the legitimate report.

In my view, education around what I call document literacy is vital in today’s society. When fake reports and real reports share a common format, how can the average person know what to trust? Those of us who are trained as information professionals are better at this than the average person, and I think it’s our duty to teach our strategies to the public.

Visual, verbal

Aztec Codex
Aztec codex from the National Museum of Anthropology in Mexico City

An insight underlying my view of written language is that written language necessarily takes a visual form. Well, that seems obvious, doesn’t it? But what it means is there’s not a clean line between what is “visual” and what is “verbal.” Even so, common parlance and academic discourse alike consider “visual” to be implicitly non-linguistic and “verbal” to be linguistic.

Text is readily considered “visual” only when it verges on graphic design, as in company logos and poster headlines. However, all text should be considered visual: We undeniably perceive text using our visual faculties; moreover, all text is formatted, and this formatting can contribute to a text’s meaning. Some work has explored the non-verbal aspects of text. Edward A. Levenston, for example, argues that literary meaning is derived not only from the cognitive/linguistic meaning of the words on the page, but also from how those words are presented. He surveys how spelling, punctuation, typography and layout can modulate a text’s meaning, using examples ranging from Aristophanes and Chaucer to Shakespeare and e. e. cummings.

Along these lines, W. J. T. Mitchell discusses the notion of “imagetext,” which recognizes that every medium is a mixed medium—that images and text are inextricable. Vivid descriptions conjure images, just as images conjure descriptions, and all text can be seen as interpreted pictorially or visually. Years later, in a 2005 paper, Mitchell concluded that “there are no purely visual media because there is no such thing as pure visual perception in the first place.” Indeed, all media are mixed media.


More recently, sociolinguists like Mark Sebba have begun to focus on how orthography can be a site for social action quite apart from anything inherent in the meaning of the words themselves.

This also comes to the fore in the notion of “document,” which I’ve been thinking about a lot recently as I’ve been preparing my prospectus for PhD candidacy. In past lives, I studied advertising and linguistics. I thought I loved linguistics, but somehow it was never viscerally satisfying to me. I realized why only recently, upon reflecting on the “document.” Linguists talk about texts, manifestations of language. But this ignores everything we just talked about—the fundamental and inescapable entanglement between text and page, content and form. The document acknowledges this, whereas other concepts don’t seem to.

This is just a very brief post, but there will be more to come.