recommendations from members of their communities of practice

with data mining, we will be informed about the title that fits to us (decided by personalised algorithms) and it will be pushed to us as we finish a title, or even, dealing with something else. Recommandation will be crucial.

Since late 2012, one of my favorite infographics on publishing came out of Bowker, showing the percentage of book sales by major distribution channel:

In this graph, “eCommerce” represents book sales (both print and digital) happening through online retailers such as Amazon, Barnes & Noble, Wal-Mart, etc. In roughly a two-year period, the percentage of books sold online jumped from 25.1 percent to 43.8 percent. Meanwhile, large chain bookstores, such as Barnes & Noble and Borders, fell from 31.5 percent of book sales to 18.7 percent of book sales.

The large decline from 2011–2012 in bricks-and-mortar bookstore sales is attributable to the Borders bankruptcy. Barnes & Noble’s future is far from certain; they plan to close about 10 percent of their stores in the coming years, and one wonders if it will end up being more.

All this to say: the bookstore has long been the primary means of book discovery, but soon it will be a minor player in how books get marketed and sold. As sales increasingly move online, very different dynamics take hold, such as search optimization, algorithm-driven recommendations and social conversations. Probably the biggest buzzword these days in publishing insider circles is “metadata,” particularly ever since Nielsen released a study showing a dramatic increase in sales for books that satisfied the industry’s core and enhanced metadata requirements.

Core metadata includes: ISBN, title, author, category, price and publisher. Enhanced metadata includes: cover, blurb, author biography, sample chapters, quotes and reviews.

Metadata has different purposes depending on the context. For now, I want to primarily focus on the importance of metadata for a population of readers who are more likely to be discovering books online rather than in a store. In the online shopping environment, a reader has no personal guidance, but there is an unlimited selection of books. Results are based on recommendation algorithms, search algorithms driven by keywords and the book’s metadata.

In a talk by Ronald Schild at Frankfurt Book Fair’s CONTEC 2013 conference, called “The Future of Metadata,” he emphasized the need for semantic analysis, which relates to identifying the “core concept” of a book. Without semantic analysis, recommendations are less valuable; people do not search for books by ISBNs, but by themes (e.g., gay coming-of-age story set in Communist Czechoslovakia) and emotional topics.

What’s most fascinating about the metadata discussion thus far is how much it can affect the sales of fiction; conventional wisdom might’ve led us to believe that it would be most important for information-driven books, but Bowker’s data indicates just the opposite. In a talk at 2013 BookExpo America, Phil Madans from Hachette said, “If you don’t want your books to be found ever, use the Fiction: General category as your BISAC code.” Perhaps historically publishers have been less detailed with fiction metadata, thinking it doesn’t matter, but they have now changed course, delivering dramatic lifts in sales. The same discussion has also been happening in the self-publishing community, where authors have discovered that being very careful and intentional with their categories, keywords and summary descriptions have resulted in better visibility and thus sales.

The metadata discussion doesn’t just stop with filling out the fields appropriately when cataloguing a book. Veteran book marketer Peter McCarthy has argued that there are far more potential readers for each book than is ever reached, and that if publishers are to keep their value to authors, they need to be the best at connecting authors and titles to the most right readers. When he develops a marketing campaign, he uses a subset of 100 tools to triangulate, plan and execute, including a range of social analytics, search-engine optimization and other support tools, to help him understand how “ordinary” readers (not publishing insiders) go about searching for things – and to make sure those people find the right book. A good part of what McCarthy suggests amounts to uncovering and analyzing how online conversations represent potential markets for a book.

This falls in line with a keynote talk that journalist Sascha Lobo recently gave on how the Internet will change the book. His argument is that selling books has always been social, and – in fact – the social element has always been most important. People buy books that are talked about, and his contention is that the bestselling tool for books, on the Internet, is buzz. And buzz is exactly what McCarthy is attempting to quantify with his subset of 100 tools, and what metadata experts want to see captured, analyzed and displayed with book search results.

But the one question that often bothers more astute industry observers: Do readers really have trouble finding books, or is discoverability a problem of the publisher (and/or author)? If you take a look at the magazine/periodical world (or other media surveys), you often find that people’s biggest problem has nothing to do with finding stories, information or entertainment, but with having time to consume everything they find. One strategy in the self-publishing community, which has been a double-edged sword for authors, is keeping their prices very low (even free), and posing a low risk, to encourage a large volume of readers to buy. However, this can have the unintended effect of encouraging readers to download or buy many more books than they could ever read, with no or reduced consequences for not consuming what is bought.

In the future, readers will not go in search of books to read. Feral books will stalk readers, sneak into their e-book libraries and leap out to ambush them. Readers will have to beat books off with a baseball bat; hold them at bay with a flaming torch; refuse to interact; and in extreme cases, feign dyslexia, blindness or locked-in syndrome to avoid being subjected to literature.

You think I’m exaggerating for effect, don’t you?

Today, roughly 40-50,000 books are published commercially each year in the English language. But the number is rapidly rising, as traditional barriers to entry are fading away. Meanwhile, the audience for these works remains stubbornly static. The limits to reading are imposed by its time-rivalrous nature, in conjunction with the size of the English-reading population and the number of hours in the day. Tools that make writing and publishing easier work to increase the volume of work because the creation of books is to some extent an exercise of ego: we are all convinced that we have something of value to communicate, after all. It therefore seems inevitable that in future, there will be more books – and with them, more authors who are convinced that the existence of their literary baby entitles them to prosper from the largesse of their readers.

A burgeoning supply of books and a finite number of reader-hours is a predictor of disaster, insofar as the average number of readers per book will dwindle. The competition for eyeballs will intensify by and by. Many writers will stick to the orthodox tools of their profession, to attractive covers and cozening cover copy. Some will engage in advertising, and others in search engine optimization strategies to improve their sales ranking. But some will take a road less well-trodden.

Historically, publishers attempted to use cheap paperback novels as advertising sales vehicles. Books incorporated ads, as magazines and websites do today: they even experienced outbreaks of product placement, car chases interrupted so that the protagonists could settle down for half an hour to enjoy a warming dish of canned tomato soup. Authors and their agents put an end to this practice, for the most part, with a series of fierce lawsuits waged between the 1920s and 1940s that added boilerplate to standard publisher contracts forbidding such practices: for authors viewed their work as art, not raw material to deliver eyeballs to advertisements.

But we have been gulled into accepting advertising-funded television, and by extension an advertising-funded web. And as the traditional verities of publishing erode beneath the fire-hose force of the book as fungible data, it is only a matter of time before advertising creeps into books, and then books become a vehicle for advertising. And by advertising, I mean spam.

The first onset of bookspam went unnoticed, for it did not occur within the pages of the books themselves. Spam squirted its pink and fleshy presence into the discussion forums of Goodreads and the other community collaborative book reading and reviewing websites almost from the first. And we shrugged and took it for granted because, well, it’s spam. It’s pervasive, annoying and it slithers in wherever there’s space for feedback or a discussion.

But that isn’t where it’s going to end. An EPUB e-book file is essentially an HTML5 file, encapsulated with descriptive metadata and an optional DRM layer. The latest draft standard includes support for all aspects of HTML5 including JavaScript. Code implodes into text, and it is only a matter of time before we see books that incorporate software for collaborative reading. Not only will your e-book save your bookmarks and annotations; it’ll let you share bookmarks and annotations with other readers. It’s only logical, no? And the next step is to let readers start discussions with one another, with some sort of tagging mechanism to link the discussions to books, or chapters, or individual scenes, or a named character or footnote.

Once there is code there will be parasites, viral, battening on the code. It’s how life works: around 75% of known species are parasitic organisms. A large chunk of the human genome consists of endogenous retroviruses, viruses that have learned to propagate themselves by splicing themselves into our chromosomes and lazily allowing the host cells to replicate themselves whenever they divide. Spammers will discover book-to-book discussion threads just as flies flock to shit.

But then it gets worse. Much worse.

Authors, expecting a better reaction from the reading public than is perhaps justifiable in this age of plenty for all (and nothing for many) will eventually succumb to the urge to add malware to their e-books in return for payment. The malware will target the readers’ e-book libraries. The act of reading an infected text will spread the payload, which will use its access to spread advertising extracts and favorable reviews throughout the reader communities. You may find your good reputation taken in vain by a second-rate pulp novel that posts stilted hagiographies of its author’s other books on the discussion sites of every book you have ever commented on (and a few you haven’t). Worse, the infested novels will invite free samples of all their friends to the party, downloading the complete works of their author just in case you feel like reading them. Works which will be replete with product placement and flashing animated banner ads, just in case you didn’t get the message.

Finally, in extremis, feral spambooks will deploy probabilistic text generators seeded with the contents of your own e-book library to write a thousand vacuous and superficially attractive nuisance texts that at a distance resemble your preferred reading. They’ll slide them into your e-book library disguised as free samples, with titles and author names that are random permutations of legitimate works, then sell advertising slots in these false texts to offshore spam marketplaces. And misanthropic failed authors in search of their due reward will buy the ad marquees from these exchanges, then use them to sell you books that explain how to become a bestselling author in only 72 hours.

Books are going to be like cockroaches, hiding and breeding in dark corners and keeping you awake at night with their chittering. There’s no need for you to go in search of them: rather, the problem will be how to keep them from overwhelming you.

In the legacy publishing world, an oligopoly of gatekeepers decided what books would be available. Publishers chose which authors deserved attention. Reviewers, librarians and bookstores winnowed the field further. (If you could get Oprah Winfrey to recommend a book, its future was golden.) The system assured a certain level of quality at the top of the ladder. But discovery, apart from recommendations from friends and colleagues, was largely a top-down method.

Reputation was integral to that system. Publishers put their own reputations on the line by choosing their authors. Similarly, we learned to trust reviewers and their organizations, or not. And when our local bookstore owner recommended a book we hated, we were much less likely to take his word in the future.

The digital revolution hasn’t done away with the top-down recommendation model, even though news organizations have dumped book reviews, traditional bookstores are disappearing and the big publishing companies focus as much as possible on books they already know will sell. The most important recommender today may be Amazon*, which makes some corporate editorial judgments but mostly suggests books based on what “people like you” buy according to complex and proprietary algorithms.

Those highly customized online recommendations, in a variety of media formats such as video (Netflix) and audio (Spotify), suffer from their own imprecision. Sometimes the results are utterly laughable. They can often be amazingly right. They are based on deep dives into data, and over time the recommendations become more refined as we use them. But they rely much more on correlation than reputation.

In a system where readers’ choices are part of the formula, their own reputations can and should carry more weight. Some of those readers are our social media contacts. Others are bloggers whose work we’ve come to admire. They are part of an edge-in rather than top-down recommendation engine where readers make more or less explicit choices about who to trust. This is how I find much of the news I read (listen to/watch/etc.), but much less so when it comes to books.

That will change in coming years as we combine human and machine intelligence in more sophisticated ways. Here’s an extremely simple example: Suppose I could designate three people whose work I trust in a specific arena to tell me what they’re reading – as well as any three people each of them recommends in that arena. That would aggregate expertise and recommendations in ways I can’t easily do today. Someone will build a big business by creating better reputation-based tools for discovery.

How can we avoid finding out mostly (or only) about books we’re predisposed to liking, and thereby missing out on books we didn’t know we’d enjoy? I worry about the fact that Amazon tailors recommendations based on what it thinks I want. One of the joys of traditional bookstores is serendipity: the discovery of a nearby volume that I browse through and then decide to buy. This isn’t entirely random; the bookstore manager decided what books to put on the shelves, and a clever jacket design can entice me to check out a book I wouldn’t otherwise notice.

At some level we’ll need to create our own serendipity in the e-book era. This won’t be difficult, but we’ll probably need to do it more consciously, by going outside our zones of comfort and the recommendations of people we trust. Discovery can’t be a passive act.

Is there a future for the bookstore in a digital age?

Despite the death of independent bookstores, despite the failure of major booksellers like Borders and Barnes & Noble, I think the answer is yes. Bookstores may well survive, if we’re open to the possibility that they may not, strictly speaking, be stores or physically house books. It might be better to say that the function of the bookstore will persist, albeit in a new material form.

Bookstores had an important mission: They physically distributed books to readers. They curated the books that they stocked. They guided individual readers to new books. They were (and still are) community centers, hosting readings, effectively serving as reading rooms, at their best creating not only readers but also reading publics. In what follows, I will assume the continued value of print books (see my previous essay, in the chapter “How will people read in the future?”).

Beyond existing modes of distribution — indies, big booksellers, and mega-retailers like Costco — how will we find new p-books in the future? How should we? Here are a few suggestions.

AMAZON STORE FOR BOOKS. Just as Apple has an Apple Store where it displays its sleek wares, Amazon might consider creating a bricks-and-mortar establishment meant to showcase its papery products. It’s possible, just possible, that customers will come into such stores, browse through physical books, and then decide to, you know, buy them. It’s a crazy idea, but if any innovative forward-looking technology company can make it work, Amazon can.

BOOK POP-UP. As physical bookstores increasingly go out of business, we might imagine a version of pop-up retail for the book sector. Such pop-up stores would by necessity be small, but they could colonize existing retail spaces, either legally or (what would be neater) extra-legally. With the aid of social media we might organize flash bookstores, which feature curated collections of the very coolest books, past and present, all handpicked by what we might call Book DJs (let’s all agree not to call them Book Jockeys, for obvious reasons), whose reputations will depend on their meticulous taste. No self-respecting hipster should buy his or her book from any other sort of store.

POD MACHINE. Some independent booksellers, like McNally Jackson in New York, have brought Espresso Book Machines into their store, allowing the printing of public domain books on demand. Such machines could populate many different retail locations, or even in time be part of every home. There’s also no technical reason that every book, both public domain and private, shouldn’t be available via POD Machine. Until technologies like 3D printing make it possible to print a high-quality book on demand in the home, let’s install a fast POD machine in every café in the land (Starbucks: I’m looking at you), set them up among vending machines wherever fine sugar drinks and fatty snacks are sold, and incorporate them into every airplane, where airline carriers can take their predatory cut from text-hungry frequent fliers. The whole human library should be available on demand, as a beautiful physical print-off, at any time.

PUBLIC LIBRARY. A radically socialist scheme, the public library is a place where stuffy government bureaucrats purchase books using tax dollars, store these books and then make them available to the general public. In the future, public libraries may become a key resource for preserving literary culture, if rapacious capitalists don’t kill them off first.

These are all ideas that could be pursued now, with a little bit of will, either on the part of private or public organizations. The future of the book is in our hands. We should make sure that readers can find the books they want, and that our institutions of book discovery work in their (that is, our) interests.