recommendations from members of their communities of practice

with data mining, we will be informed about the title that fits to us (decided by personalised algorithms) and it will be pushed to us as we finish a title, or even, dealing with something else. Recommandation will be crucial.

Since late 2012, one of my favorite infographics on publishing came out of Bowker, showing the percentage of book sales by major distribution channel:

In this graph, “eCommerce” represents book sales (both print and digital) happening through online retailers such as Amazon, Barnes & Noble, Wal-Mart, etc. In roughly a two-year period, the percentage of books sold online jumped from 25.1 percent to 43.8 percent. Meanwhile, large chain bookstores, such as Barnes & Noble and Borders, fell from 31.5 percent of book sales to 18.7 percent of book sales.

The large decline from 2011–2012 in bricks-and-mortar bookstore sales is attributable to the Borders bankruptcy. Barnes & Noble’s future is far from certain; they plan to close about 10 percent of their stores in the coming years, and one wonders if it will end up being more.

All this to say: the bookstore has long been the primary means of book discovery, but soon it will be a minor player in how books get marketed and sold. As sales increasingly move online, very different dynamics take hold, such as search optimization, algorithm-driven recommendations and social conversations. Probably the biggest buzzword these days in publishing insider circles is “metadata,” particularly ever since Nielsen released a study showing a dramatic increase in sales for books that satisfied the industry’s core and enhanced metadata requirements.

Core metadata includes: ISBN, title, author, category, price and publisher. Enhanced metadata includes: cover, blurb, author biography, sample chapters, quotes and reviews.

Metadata has different purposes depending on the context. For now, I want to primarily focus on the importance of metadata for a population of readers who are more likely to be discovering books online rather than in a store. In the online shopping environment, a reader has no personal guidance, but there is an unlimited selection of books. Results are based on recommendation algorithms, search algorithms driven by keywords and the book’s metadata.

In a talk by Ronald Schild at Frankfurt Book Fair’s CONTEC 2013 conference, called “The Future of Metadata,” he emphasized the need for semantic analysis, which relates to identifying the “core concept” of a book. Without semantic analysis, recommendations are less valuable; people do not search for books by ISBNs, but by themes (e.g., gay coming-of-age story set in Communist Czechoslovakia) and emotional topics.

What’s most fascinating about the metadata discussion thus far is how much it can affect the sales of fiction; conventional wisdom might’ve led us to believe that it would be most important for information-driven books, but Bowker’s data indicates just the opposite. In a talk at 2013 BookExpo America, Phil Madans from Hachette said, “If you don’t want your books to be found ever, use the Fiction: General category as your BISAC code.” Perhaps historically publishers have been less detailed with fiction metadata, thinking it doesn’t matter, but they have now changed course, delivering dramatic lifts in sales. The same discussion has also been happening in the self-publishing community, where authors have discovered that being very careful and intentional with their categories, keywords and summary descriptions have resulted in better visibility and thus sales.

The metadata discussion doesn’t just stop with filling out the fields appropriately when cataloguing a book. Veteran book marketer Peter McCarthy has argued that there are far more potential readers for each book than is ever reached, and that if publishers are to keep their value to authors, they need to be the best at connecting authors and titles to the most right readers. When he develops a marketing campaign, he uses a subset of 100 tools to triangulate, plan and execute, including a range of social analytics, search-engine optimization and other support tools, to help him understand how “ordinary” readers (not publishing insiders) go about searching for things – and to make sure those people find the right book. A good part of what McCarthy suggests amounts to uncovering and analyzing how online conversations represent potential markets for a book.

This falls in line with a keynote talk that journalist Sascha Lobo recently gave on how the Internet will change the book. His argument is that selling books has always been social, and – in fact – the social element has always been most important. People buy books that are talked about, and his contention is that the bestselling tool for books, on the Internet, is buzz. And buzz is exactly what McCarthy is attempting to quantify with his subset of 100 tools, and what metadata experts want to see captured, analyzed and displayed with book search results.

But the one question that often bothers more astute industry observers: Do readers really have trouble finding books, or is discoverability a problem of the publisher (and/or author)? If you take a look at the magazine/periodical world (or other media surveys), you often find that people’s biggest problem has nothing to do with finding stories, information or entertainment, but with having time to consume everything they find. One strategy in the self-publishing community, which has been a double-edged sword for authors, is keeping their prices very low (even free), and posing a low risk, to encourage a large volume of readers to buy. However, this can have the unintended effect of encouraging readers to download or buy many more books than they could ever read, with no or reduced consequences for not consuming what is bought.

In the future, readers will not go in search of books to read. Feral books will stalk readers, sneak into their e-book libraries and leap out to ambush them. Readers will have to beat books off with a baseball bat; hold them at bay with a flaming torch; refuse to interact; and in extreme cases, feign dyslexia, blindness or locked-in syndrome to avoid being subjected to literature.

You think I’m exaggerating for effect, don’t you?

Today, roughly 40-50,000 books are published commercially each year in the English language. But the number is rapidly rising, as traditional barriers to entry are fading away. Meanwhile, the audience for these works remains stubbornly static. The limits to reading are imposed by its time-rivalrous nature, in conjunction with the size of the English-reading population and the number of hours in the day. Tools that make writing and publishing easier work to increase the volume of work because the creation of books is to some extent an exercise of ego: we are all convinced that we have something of value to communicate, after all. It therefore seems inevitable that in future, there will be more books – and with them, more authors who are convinced that the existence of their literary baby entitles them to prosper from the largesse of their readers.

A burgeoning supply of books and a finite number of reader-hours is a predictor of disaster, insofar as the average number of readers per book will dwindle. The competition for eyeballs will intensify by and by. Many writers will stick to the orthodox tools of their profession, to attractive covers and cozening cover copy. Some will engage in advertising, and others in search engine optimization strategies to improve their sales ranking. But some will take a road less well-trodden.

Historically, publishers attempted to use cheap paperback novels as advertising sales vehicles. Books incorporated ads, as magazines and websites do today: they even experienced outbreaks of product placement, car chases interrupted so that the protagonists could settle down for half an hour to enjoy a warming dish of canned tomato soup. Authors and their agents put an end to this practice, for the most part, with a series of fierce lawsuits waged between the 1920s and 1940s that added boilerplate to standard publisher contracts forbidding such practices: for authors viewed their work as art, not raw material to deliver eyeballs to advertisements.

But we have been gulled into accepting advertising-funded television, and by extension an advertising-funded web. And as the traditional verities of publishing erode beneath the fire-hose force of the book as fungible data, it is only a matter of time before advertising creeps into books, and then books become a vehicle for advertising. And by advertising, I mean spam.

The first onset of bookspam went unnoticed, for it did not occur within the pages of the books themselves. Spam squirted its pink and fleshy presence into the discussion forums of Goodreads and the other community collaborative book reading and reviewing websites almost from the first. And we shrugged and took it for granted because, well, it’s spam. It’s pervasive, annoying and it slithers in wherever there’s space for feedback or a discussion.

But that isn’t where it’s going to end. An EPUB e-book file is essentially an HTML5 file, encapsulated with descriptive metadata and an optional DRM layer. The latest draft standard includes support for all aspects of HTML5 including JavaScript. Code implodes into text, and it is only a matter of time before we see books that incorporate software for collaborative reading. Not only will your e-book save your bookmarks and annotations; it’ll let you share bookmarks and annotations with other readers. It’s only logical, no? And the next step is to let readers start discussions with one another, with some sort of tagging mechanism to link the discussions to books, or chapters, or individual scenes, or a named character or footnote.

Once there is code there will be parasites, viral, battening on the code. It’s how life works: around 75% of known species are parasitic organisms. A large chunk of the human genome consists of endogenous retroviruses, viruses that have learned to propagate themselves by splicing themselves into our chromosomes and lazily allowing the host cells to replicate themselves whenever they divide. Spammers will discover book-to-book discussion threads just as flies flock to shit.

But then it gets worse. Much worse.

Authors, expecting a better reaction from the reading public than is perhaps justifiable in this age of plenty for all (and nothing for many) will eventually succumb to the urge to add malware to their e-books in return for payment. The malware will target the readers’ e-book libraries. The act of reading an infected text will spread the payload, which will use its access to spread advertising extracts and favorable reviews throughout the reader communities. You may find your good reputation taken in vain by a second-rate pulp novel that posts stilted hagiographies of its author’s other books on the discussion sites of every book you have ever commented on (and a few you haven’t). Worse, the infested novels will invite free samples of all their friends to the party, downloading the complete works of their author just in case you feel like reading them. Works which will be replete with product placement and flashing animated banner ads, just in case you didn’t get the message.

Finally, in extremis, feral spambooks will deploy probabilistic text generators seeded with the contents of your own e-book library to write a thousand vacuous and superficially attractive nuisance texts that at a distance resemble your preferred reading. They’ll slide them into your e-book library disguised as free samples, with titles and author names that are random permutations of legitimate works, then sell advertising slots in these false texts to offshore spam marketplaces. And misanthropic failed authors in search of their due reward will buy the ad marquees from these exchanges, then use them to sell you books that explain how to become a bestselling author in only 72 hours.

Books are going to be like cockroaches, hiding and breeding in dark corners and keeping you awake at night with their chittering. There’s no need for you to go in search of them: rather, the problem will be how to keep them from overwhelming you.