Category: Uncategorized

[3D-printed zoetrope from The National Science and Media Museum’s Wonderlab, via Sheryl Jenkins]

Modeling Humane Features

If it wasn’t a famous catchphrase coined by a young social media billionaire, and etched onto the walls of Facebook’s headquarters, the most likely place for “Move fast and break things” to appear would be on a mall-store t-shirt for faux skate punks. But even though Facebook has purportedly moved on from Mark Zuckerberg’s easily mocked motto, it is an ideology that remains widely in use, and that also engenders easy counter-takes, with some variation on “Move slowly and fix things.”

In a newsletter that seeks the contours of humane technology, moving slowly and fixing things seems like an obvious starting point, but it’s not enough and has the potential to be its own vapid motto unless we do a better job spelling out what it actually means. We need some examples, some modeling of how one goes about creating digital media that is Not Facebook. And for that we should revisit my emphasis in HI2on considering the psychology of humans before creating digital technology, and then again, constantly, during its implementation and growth.

In the signature block of this newsletter, you may have noticed that my preferred social media platform is Micro.blog, which I decamped to from Twitter last year in an attempt to cleanse my mind and streamline my online presence. (I am, however, still unclean; I have chosen to syndicate my Micro.blog posts to Twitter, since I still have many friends over there who wish to hear from me.) A big part of what I like about Micro.blog is that moving deliberately and considering things is deeply valued by Manton Reece, the architect of the platform, as well as his colleagues Jean MacDonald and Jonathan Hays.

Manton is admirably transparent and thoughtful about how he is designing and iterating on the platform (so much so that he’s writing a book about it, which surely makes him the most academic of app-makers). Even before Micro.blog’s launch, Manton thought through, and just as important, stuck to, key features—or non-features—such as no follower or like counts, anywhere, ever. (On Micro.blog you can see who people follow, to find others to follow yourself, but not how many people follow you or anyone else.) As Micro.blog has developed, Manton has also consulted with the burgeoning community about new features, to ensure that what he’s doing won’t disrupt the considerate, but still engaging, vibe of the place.

I have much more to say about Micro.blog, including some thoughts about its size, sustainability plan, and approach to hosting and personal data, but for this issue of HI I want to focus on a good case study of ethical technological implementation—a happy accident that turned into a valued (non-)feature. Last year an unintended update to the Micro.blog code suddenly made it impossible for users to see if someone was following them by clicking on their follows list. This seemed odd and in need of a fix, until everyone swiftly realized what a relaxing plus this is in today’s social media. The long thread of this revelation reads like a morality play; here’s a brief excerpt:

@jack: Either no one follows me here or the following list doesn’t include the user looking at the list. This has the effect of hiding whether or not someone follows me. If that’s the case, it’s genius.

@macgenie: @jack I had not realized that we don’t show whether someone is following you when you look at their Following list! That makes sense. / @manton

@jack: @macgenie I think it’s terrific. Avoids the awkwardness around any implied obligation for mutual follows :).

@manton: @macgenie @jack So… this is actually a bug that I introduced last night. But now I’m wondering if it’s a good thing! It’s confusing right now but maybe an opportunity to rethink this feature.

@jamesgowans: @manton I like it. It relieves the worry about whether someone follows you or doesn’t (a useless popularity metric), and places more value in replies/conversations. It’s sort of “all-in” on the no follower count philosophy.

@ayjay: @manton As the theologians say, O felix insectum!

Two weeks ago Rob Walker had a piece revisiting “Move fast and break things” and urging Silicon Valley to have an overriding focus on designing apps with bad actors in mind, for the worst of humanity who might seek to exploit an app for their own gain and to disrupt society. This should undoubtedly be an important aspect of digital design, especially so given what has happened online in the last few years.

But while architects should surely design an office building to thwart arsonists, they need to spend even more time designing spaces for those who are trying to work peacefully and productively within the building. Ultimately, we have to design platforms to withstand not only the worst of us, but the worst aspects of the rest of us. In thinking about seemingly small elements like whether you can see if someone is following you, Micro.blog’s Manton, Jean, and Jonathan get this critical point.

Teaching a Robot to Crack a Whip

In Robin Sloan’s novel Sourdough, the protagonist tries to teach a robot arm how to delicately and effectively crack an egg for cooking. This week on the What’s New podcast from the Northeastern University Library, I spoke to Dagmar Sternad from the Action Lab, who is, among other things, teaching a robot how to crack a whip. (She said that’s a very hard problem, especially to hit a particular point in space with the tip of the whip, but that egg cracking is a very, very hard problem.)

The Action Lab studies the complete range and technique of human motion very closely using the same technology Hollywood uses to create CGI characters—my favorite Action Lab case study looks at how ballet dancers in a duet transmit information through their fingertips to their partners—and then they encode that knowledge into digital and then robotic surrogates through biomimetics.

[Dagmar Sternad with two ballet dancers and a robot arm]

What I also found interesting about Dagmar’s work, and very germane to what I’m trying to do in this newsletter, is the bidirectionality of learning between humans and machines. You will be glad to hear that the Action Lab is not seeking to create whip-cracking robots of doom. But by observing humans doing intricate tasks, and then replicating those actions in the computer and with machines, they can better understand what is going on—and can even alter and simplify the motions so that they remain effective but require less expertise and motion. In turn, they can transfer these lessons back to physical human behaviors. (This is a variation on Sloan’s “flip-flop”: “the process of pushing a work of art or craft from the physical world to the digital world and back.”) In short, they study ballet dancers and engineer robots so that they can find ways to help the elderly walk better and avoid falls.

This was a fun conversation—give it a listen or subscribe to the podcast through the What’s New site.

Follow-up on AI/ML in Libraries, Archives, and Museums

My thanks to HI readers who sent me recent discussions about the use of artificial intelligence/machine learning in libraries, archives, and museums, in response to HI3:

• Museums + AI, New York workshop notes, from Mia Ridge.
  • Includes the slide deck from Mia’s own talk, “Operationalising AI at a national library.”
• The latest volume of Research Library Issues from the Association of Research Libraries is on the “Ethics of Artificial Intelligence”—good timing!“After decades of worries that the popularity of science and technology paradigms threaten humanistic learning and scholarship, it is now becoming evident that unique opportunities are emerging to demonstrate why humanistic expertise and informed considerations of the human condition are essential to the very future of humanity in a technological age.” —Sylvester Johnson

Welcome back to Humane Ingenuity. It’s been gratifying to see this newsletter quickly pick up an audience, and to get some initial feedback from readers like you that can help shape forthcoming issues. Just hit reply to this message if you want to respond or throw a link or article or book my way. (Today’s issue, in fact, includes a nod to a forthcoming essay from one of HI’s early subscribers.)

OK, onto today’s theme: what can artificial intelligence do in a helpful way in archives and special collections? And what does this case study tell us more generally about an ethical and culturally useful interaction between AI and human beings?

Crowdsourcing to Cloudsourcing?

Over a decade ago, during the heady days of “Web 2.0,” with its emphasis on a more interactive, dynamic web through user sharing and tags, a cohort of American museums developed a platform for the general public to describe art collections using more common, vernacular words than the terms used by art historians and curators. The hope was that this folksonomy, that riffy portmanteau on the official taxonomy of the museum, would provide new pathways into art collections and better search tools for those who were not museum professionals.

The project created a platform, Steve, that had some intriguing results when museums like the Indianapolis Museum of Art and the Metropolitan Museum of Art pushed web surfers to add their own tags to thousands of artworks.

Some paintings received dozens of descriptive tags, with many of them straying far from the rigorous controlled vocabularies and formal metadata schema we normally see in the library and museum world. (If this is your first time hearing about the Steve.museum initiative, I recommend starting with Jennifer Trant’s 2006 concept paper in New Review of Hypermedia and Multimedia, “Exploring the Potential for Social Tagging and Folksonomy in Art Museums: Proof of Concept” (PDF))

(Official museum description, top; most popular public tags added through Steve, bottom)

Want to find all of the paintings with sharks or black dresses or ice or wheat? You could do that with Steve, but not with the conventional museum search tools. Descriptors like “Post-Impressionism” and “genre painting” were nowhere to be seen. As Susan Chun of the Met shrewdly noted about what the project revealed: “There’s a huge semantic gap between museums and the public.”

I’ve been thinking about this semantic gap recently with respect to new AI tools that have the potential to automatically generate descriptions and tags—at least in a rough way, like the visitors to the Met—for the collections in museums, libraries, and archives. Cloud services from Google, Microsoft, Amazon, and others can do large-scale and fine-grained image analysis now. But how good are these services, and are the descriptions and tags they provide closer to crowdsourced words and phrases or professional metadata? Will they simply help us find all of the paintings with sharks—not necessarily a bad thing, as the public has clearly shown an interest in such things—or can they—should they—supplement or even replace the activity of librarians, archivists, and curators? Or is the semantic gap too great and the labor issues too problematic?

Bringing the Morgue to Life

Pilots using machine vision to interpret the contents of historical photography collections are already happening. Perhaps most notably, the New York Times and Google inked a deal last year to have Google’s AI and machine learning technology provide better search tools for their gigantic photo morgue. The Times has uploaded digital scans of their photos to Google’s cloud storage—crucially, the fronts andbacks of the photos, since the backs have considerable additional contextual data—and then Google uses their Cloud Vision API and Cloud Natural Language API to extract information about each photo.

Unfortunately, we don’t have access to the data being produced from the Times/Google collaboration. Currently it is being used behind the scenes to locate photos for some nice slide shows on the history of New York City. But we can assume that the Times gets considerable benefit from the computational processing that is happening. As Google’s marketing team emphasizes, “the digital text transcription isn’t perfect, but it’s faster and more cost effective than alternatives for processing millions of images.”

Unspoken here are the “alternatives,” by which they clearly mean processes involving humans and human labor. These new AI/ML techniques may not be perfect (yet? no, likely ever, see “Post-Impressionism”), but they have a strong claim to the “perfect is the enemy of the good” school of cultural heritage processing. There’s a not-so-subtle nudge from Google: Hey archivists, you wanna process millions of photos by hand, with dozens of people you have to hire writing descriptions of what’s in the photos and transcribing the backs of them too? Or do you want it done tomorrow by our giant computers?

As Clifford Lynch writes in a forthcoming essay (which I will link to once it’s published), “[Machine learning applications] will substantially improve the ability to process and provide access to digital collections, which has historically been greatly limited by a shortage of expert human labor. But this will be at the cost of accepting quality and consistency that will often fall short of what human experts have been able to offer when available.”

This problem is very much in the forefront of my mind, as the Northeastern University Library recently acquired the morgue of the Boston Globe, which contains the original prints of over one million photographs that were used in the paper, as well as over five million negatives, most of which have never been seen beyond the photo editing room at the Globe. It’s incredibly exciting to think about digitizing, making searchable, and presenting this material in new ways—we have the potential to be a publicly accessible version of the NYT/Google collaboration.

But we also face the difficult issue of enormous scale. It’s a vast collection. Along with the rest of the morgue, which includes thousands of boxes of clippings, topical files, and much else, we have filled a significant portion of the archives and special collections storage in Snell Library.

Fortunately the negative sleeves have some helpful descriptive metadata, which could be transcribed by humans fairly readily and applied to the 20-40 photos in each envelope. But what’s really going on in each negative, in detail? Who or what appears? That’s a hard and expensive problem. (Insert GIF of Larry Page slowly turning toward the camera and laughing like a Bond villain.)

I have started to test several machine vision APIs, and it’s interesting to note the different approaches and strengths of each service. Here’s a scan of a negative from our Globe archive, of a protest during the upheaval of Boston’s school desegregation and busing era, uploaded to Google’s Cloud Vision API (top) and Microsoft’s Computer Vision API (bottom).

I’ll return to these tests in a future issue of HI, as I am still experimenting, but some impressive things are happening here, with multiple tabs showing different analyzes of the photograph. Both services are able to identify common entities and objects and any text that appears. They also do a good job analyzing the photo as a set of pixels—its tones and composition, which can be helpful if, say, we don’t want to spend time examining a washed out or blurry shot.

There are also creepy things happening here, as each service has a special set of tools around faces that appear. As Lynch notes, “The appropriateness of applying facial recognition [to library materials] will be a subject of major debate in coming years; this is already a very real issue in the social media context, and it will spread to archives and special collections.”

Google, in a way that is especially eyebrow-raising, also connects its Cloud Vision API to its web database, and so was able to figure out the historical context of this test photograph rather precisely (shown in the screen shot, above). Microsoft synthesizes all of the objects it can identify into a pretty good stab at a caption: “A man holding a sign walking in a street.” For those who want to make their collections roughly searchable (and just as important, provide accessibility for those with visual impairments through good-enough textual equivalents for images), a quick caption service like this is attractive. And it assigns that caption a probability: a very confident score, in this case, of 96.9%.

In the spirit of Humane Ingenuity, we should recognize that this is not an either/or situation, a binary choice between human specialists and vision bots. We can easily imagine, for instance, a “human in the loop” scenario in which the automata provide educated guesses and a professionals in libraries, archives, and museums provide knowledgeable assessment, and make the final choices of descriptions to use and display to the public. Humans can also choose to strip the data of facial recognition or other forms of identity and personal information, based on ethics, not machine logic.

In short, if we are going to let AI loose in the archive, we need to start working on processes and technological/human integrations that are the most advantageous combinations of bots and brains. And this holds true outside of the archive as well.

Neural Neighbors

The numerical scores produced by the computer can also relate objects in vast collections in ways that help humans do visual rather than textual searches. Doug Duhaime, Monica Ong Reed, and Peter Leonard of Yale’s Digital Humanities Lab ran 27,000 images from the nineteenth-century Meserve-Kunhardt Collection (one of the largest collections of 19th-century photography) at the Beinecke Rare Book and Manuscript Library through a neural network, which then clustered the images into categories through 2,048 ways of “seeing.” The math and database could then be used to produce a map of the collection as a set of continents and archipelagos of similar and distant photographs.

Through this new interface, you can zoom into one section (at about 100x, below) to find all of the photographs of boxers in a cluster.

The Neural Neighbors project reveals the similarity scores in another prototype search tool. As David Leonard noted in a presentation (PDF) of this work at a meeting of the Coalition for Networked Information, much of the action is happening in the penultimate stage of the process, when those scores emerge. And that is also where an archivist can step in and take the reins back from the computer, to transform the mathematics into more rigorous categories of description, or to dream up a new user interface for accessing the collection.

(See also: John Resig’s pioneering work in this area: “Ukiyo-e.org: Aggregating and Analyzing Digitized Japanese Woodblock Prints.”)

Some Fun with Image Pathways

Liz Gallagher, a data scientist at the Wellcome Trust, recently used similar methods to the Neural Neighbors project on the Welcome’s large collection of digitized works related to health and medical history.

Then, using Google’s X Degrees of Separation code, she created pathways of hidden connections between dissimilar images, tracing a set of hops between more closely related images to get there one step at a time, from left to right in each row.

Each adjacent image is fairly similar according to the computer, but the ends of the rows are not. And some of the hops, especially in the middle, are fairly amusing and even a bit poetic?

Today Apple will release new iPhones and other gizmos and services, and as they do every year, the tech pundits will ask: “Does this live up to the expectations and vision of Steve Jobs?” I, on the other hand, will ask: “Does this live up to the expectations and vision of Jef Raskin?” Apple likes to imagine itself as the humane tech company, with its emphasis on privacy and a superior user experience, but the origins of that humaneness—if it still exists beyond marketing—can be traced less to the ruthless Jobs than to the gentler Raskin. Jobs may have famously compared a computer to a “bicycle for the mind,” but Raskin articulated more genuinely a desire that computers be humane and helpful instruments.

To be clear, Raskin, like Jobs, wanted to sell millions of personal computers, but only Raskin worried aloud about what would happen if that seemingly ridiculous goal was achieved: “Will the average person’s circle of acquaintances grow? Will we be better informed? Will a use of these computers as an entertainment medium become their primary value? Will they foster self-education? Is the designer of a personal computer system doing good or evil?” It is remarkable to read these words in an internal computer design document from 1980, but such reflections were common in Raskin’s writing, and clearly more heartfelt than Google’s public, thin, and short-lived “Don’t be evil” motto.

Jobs may have dabbled in calligraphy and obsessed over design, but Raskin was the polymath who truly lived at the intersection of the liberal arts and technology. In addition to physics, math, and computer science, Raskin studied philosophy, music (which he also composed and performed at a professional level), and visual arts (he was also an accomplished artist). He clearly read a lot, which was reflected in his clear and often mirthful writing style, flecked with nerdy guffaws. (The end of one of his long Apple memos: “Summery: That means fair, warm weather, just after spring.”) He wrote a book on user interface design called The Humane Interface and sought to build a new computing system called The Humane Environment. For the purposes of this newsletter, and for some ongoing conversations I would like to have with you about the ethical dimension of technological creation, he is one important touchstone.

(Jef Raskin with a model of the Canon Cat, photo by Aza Raskin)

Raskin was one of the earliest Apple employees, hired to direct their publications and documentation, and is widely known for leading the early Macintosh project, before Jobs swooped in and recast it in his (and Xerox PARC’s) image. But before that happened, Raskin, as the consummate documenter, got to lay out the founding principles of the Mac. This set of documents became known—in a quasi-religious way—as the Book of Macintosh.

That “book” (really, a collection of documents) is now in the Special Collections at Stanford University, and they have made some of it available online if you would like to read them at the next Apple high holiday. Within these pages, you can witness Raskin pondering computational devices and user experiences that would become gospel within Apple. “This should be a completely self-teaching system.” “If this is to be truly a product for the home, shouldn’t we offer it in various colors?” “Telecommunications will become a key part of every computer market segment.” “The computer must be in one lump.” (Note to Apple: Raskin would have hated the proliferation of dongles.)

In one especially cogent document, Raskin summarized the philosophy of the Mac: “Design Considerations for an Anthropophilic Computer,” an oddly latinate title considering that Raskin dropped the second F in his first name because he considered it superfluous. Raskin imagines what the computing of the future should look like, once it moves beyond the hobbyists of the 1970s and into the mainstream:

This is an outline for a computer designed for the Person In The Street (or, to abbreviate: the PITS); one that will be truly pleasant to use, that will require the user to do nothing that will threaten his or her perverse delight in being able to say: “I don’t know the first thing about computers,” and one which will be profitable to sell, service and provide software for.

You might think that any number of computers have been designed with these criteria in mind, but not so. Any system which requires a user to ever see the interior, for any reason, does not meet these specifications. There must not be additional ROMS, RAMS, boards or accessories except those that can be understood by the PITS as a separate appliance. For example, an auxiliary printer can be sold, but a parallel interface cannot. As a rule of thumb, if an item does not stand on a table by itself, and if it does not have its own case, or if it does not look like a complete consumer item in and of itself, then it is taboo.

If the computer must be opened for any reason other than repair (for which our prospective user must be assumed incompetent) even at the dealer’s, then it does not meet our requirements.

Seeing the guts is taboo. Things in sockets is taboo (unless to make servicing cheaper without imposing too large an initial cost). Billions of keys on the keyboard is taboo. Computerese is taboo. Large manuals, or many of them (large manuals are a sure sign of bad design) is taboo. Self-instructional programs are NOT taboo.

There must not be a plethora of configurations. It is better to offer a variety of case colors than to have variable amounts of memory. It is better to manufacture versions in Early American, Contemporary, and Louis XIV than to have any external wires beyond a power cord.

And you get ten points if you can eliminate the power cord.

As I’ve argued elsewhere, I think the iPad, not the Mac, came closest to what Raskin was dreaming of here, although I suspect that as a text-lover, and given his other writing on user interfaces, he would have preferred an iPad that was oriented more toward writing and communication than consumption. But Raskin’s deep sense of how most people don’t have time to fidget with software or hardware—who just want the damn computer to work, in an understandable and consistent way—was ahead of its time. Most people are busy and tired and don’t want to be hobbyists with their digital devices.

Unfortunately, latent in Raskin’s understanding is a dark upside down world, the flip side to designing computer environments for the PITS. When you grasp that people don’t have time to fiddle with bits, when you start focusing of the software of the mind—the psychology of the user—rather than the hardware of the computer, the temptation emerges to design platforms where ease of use acts to lock people in, or perform social experiments on them. Those who are busy and tired and don’t have time to tinker—that is, most of us—may also prefer using Facebook to maintaining a personal blog or website. And that’s one way our computers became more misanthropic than anthropophilic.

What can we do when these platforms turn against us after drawing us in? On the opening podcast of the third season of What’s New, I talk to Christo Wilson, who is part of a team at Northeastern University that “audits” the algorithms within the black boxes of Facebook, Google, Amazon, and other monolithic internet services that dominate our world. There has been some very good writing recently about how these algorithms have gone awry—I recommend Cathy O’Neil’s Weapons of Math Destruction and Safiya Noble’s Algorithms of Oppression—and Christo and his colleagues have established rigorous methods for testing these services from the outside to identify their attributes and flaws. They also are able to provide you, the user of Facebook, Google, and Amazon, an understanding of exactly which of your personal attributes these services are using to customize your online environment (and track you). Scary but important work. Tune in.

An increasing array of cutting-edge, often computationally intensive methods can now reveal formerly hidden texts, images, and material culture from centuries ago, and make those documents available for search, discovery, and analysis. Note how in the following four case studies, the emphasis is on the human; the futuristic technology is remarkable, but it is squarely focused on helping us understand human culture better.

Gothic Lasers

If you look very closely, you can see that the stone ribs in these two vaults in Wells Cathedral are slightly different, even though they were supposed to be identical. Alexandrina Buchanan and Nicholas Webb noticed this too and wanted to know what it said about the creativity and input of the craftsmen into the design: how much latitude did they have to vary elements from the architectural plans, when were those decisions made, and by whom? Before construction or during it, or even on the spur of the moment, as the ribs were carved and converged on the ceiling? How can we recapture a decent sense of how people worked and thought from inert physical objects? What was the balance between the pursuit of idealized forms, and practical, seat-of-the-pants tinkering?

In “Creativity in Three Dimensions: An Investigation of the Presbytery Aisles of Wells Cathedral,” they decided to find out by measuring each piece of stone much more carefully than can be done with the human eye. Prior scholarship on the cathedral—and the question of the creative latitude and ability of medieval stone craftsmen—had used 2-D drawings, which were not granular enough to reveal how each piece of the cathedral was shaped by hand to fit, or to slightly shape-shift, into the final pattern. High-resolution 3-D scans using a laser revealed so much more about the cathedral—and those who constructed it, because individual decisions and their sequence became far clearer.

Although the article gets technical at moments (both with respect to the 3-D laser and computer modeling process, and with respect to medieval philosophy and architectural terms), it’s worth reading to see how Buchanan and Webb reach their affirming, humanistic conclusion:

The geometrical experimentation involved was largely contingent on measurements derived from the existing structure and the Wells vaults show no interest in ideal forms (except, perhaps in the five-point arches). We have so far found no evidence of so-called “Platonic” geometry, nor use of proportional formulae such as the ad quadratum and ad triangulatum principles. Use of the “four known elements” rule evidenced masons’ “cunning”, but did not involve anything more than manipulation and measurement using dividers rather than a calibrated ruler and none of the processes used required even the simplest mathematics. The designs and plans are based on practical ingenuity rather than theoretical knowledge.

Hard OCR

Last year at the Northeastern University Library we hosted a meeting on “hard OCR”—that is, physical texts that are currently very difficult to convert into digital texts using optical character recognition (OCR), a process that involves rapidly improving techniques like computer vision and machine learning. Representatives from libraries and archives, technology companies that have emerging AI tech (such as Google), and scholars with deep subject and language expertise all gathered to talk about how we could make progress in this area. (This meeting and the overall project by Ryan Cordell and David Smith of Northeastern’s NULab for Texts, Maps, and Networks, “A Research Agenda for Historical and Multilingual Optical Character Recognition,” was generously funded by the Andrew W. Mellon Foundation.)

OCRing modern printed books has become if not a solved problem at least incredibly good—the best OCR software gets a character right in these textual conversions 99% of the time. But older printed books, ancient and medieval written works, writing outside of the Romance languages (e.g., in Arabic, Sanskrit, or Chinese), rare languages (such as Cherokee, with its unique 85-character alphabet, which I covered on the What’s New podcast), and handwritten documents of any kind, remain extremely challenging, with success rates often below 80%, and in some cases as low as 40%. That means 1-3 characters are mistakenly translated by the computer in a five-character word. Not good at all.

The meeting began to imagine a promising union of language expertise from scholars in the humanities and the most advanced technology for “reading” digital images. If the computer (which in the modern case, really means an immensely powerful cloud of thousands of computers) has some ground-truth texts to work from—say, a few thousand documents in their original form and a parallel machine-readable version of those same texts, painstakingly created by a subject/language expert—then a machine-learning algorithm can be created to interpret with much greater accuracy new texts in that language or from that era. In other words, if you have 10,000 medieval manuscript pages perfectly rendered in XML, you can train a computer to give you a reasonably effective OCR tool for the next 1,000,000 pages.

Transkribus is one of the tools that works in just this fashion, and it has been used to transcribe 1,000 years of highly variant written works, in many languages, into machine-readable text. Thanks to the monks of the Hilandar Monastery, who kindly shared their medieval manuscripts, Quinn Dombrowski, a digital humanities scholar with a specialty in medieval Slavic texts, trained Transkribus in handwritten Cyrillic manuscripts, and calls the latest results from the tool “truly nothing short of miraculous.”

X-Manuscripts

Lisa Davis Fagin is the Executive Director of the Medieval Academy of America and her excellent blog, Manuscript Road Trip, is highly recommended. In a recent post, she explores the helpful use of X-ray florescence on an unusual Book of Hours.

It’s interesting to see the interplay between the intuition of scholars—this looks off in some way—and the data generated by the scientific instruments. Of course, things are not what they appear.

Really Hard OCR

Imagine trying to read an ancient text that was written in black ink on a scroll that was then roasted to a uniform, black crisp, and made so brittle it can never be unrolled. That’s what happened when Mount Vesuvius dumped twenty meters of lava on Herculaneum and flash-charred their libraries. These scrolls contain huge amounts of text that scholars are eager to read and that would undoubtedly greatly expand our understanding of ancient Rome and the Mediterranean region. But again: these texts look like the worst burnt burrito you’ve ever seen.

Enter the Digital Restoration Initiative, which has been developing a way to scan and virtually unwrap these blackened scrolls, and then extract the text from them so we can read what people wrote two thousand years ago. They pioneered this technique on the En-Gedi scroll (shown to the right of the penny, below) to computationally produce a flattened, readable text (to the left of the penny).

DRI is now working on the Herculaneum scrolls, and you can watch their techniques and tools, and the sheer complexity of the process, in this recent video:

These ancient texts, encased in a protective layer of hardened lava, look not unlike femurs, and their sectional scans really look like a CAT scan of a human bone. And that’s kind of beautiful, no?

Adam Glanzman/Northeastern University

I’m delighted that the news is now out about the Andrew W. Mellon Foundation‘s grant to Northeastern University Library to launch the Boston Research Center. The BRC will seek to unify major archival collections related to Boston, hundreds of data sets about the city, digital modes of scholarship, and a wide array of researchers and visualization specialists to offer a seamless environment for studying and displaying Boston’s history and culture. It will be great to work with my colleagues at Northeastern and regional partners to develop this center over the coming years. Having grown up in Boston, and now having returned as an adult, it has a personal significance for me as well.

I’m also excited that the BRC will build upon, and combine, some of the signature strengths of Northeastern that drew me to the university last year. For decades, the library has been assembling and working with local communities to preserve materials and stories related to the city. We now have the archives of a number of local and regional newspapers, and the library has been active in the gathering of oral and documentary histories of nearby communities such as the Lower Roxbury Black History Project. We also have strong connections with other important regional collections and institutions, such the Boston Public Library, the Boston Library Consortium, and data sets produced by Boston’s municipal government and other sources, through our campus’s leadership in BARI.

My friends in digital humanities will know that Northeastern has a world-class array of faculty and researchers doing cutting-edge, interdisciplinary computational analysis. We have the NULab for Texts, Maps, and Networks, the Network Science Institute, numerous faculty in our College of Arts, Media, and Design who work on digital storytelling and information design, and the library has its own terrific Digital Scholarship Group and dedicated specialists in GIS and data visualization. We will all be working together, and with many others from beyond the university, to imagine and develop large-scale projects that examine major trends and elements of Boston, such as immigration, neighborhood transformations, economic growth, and environmental changes. There will also be an opportunity for smaller-scale stories to be documented, and of course the BRC itself will be open to anyone who would like to research the city or specific communities. As a place with a long and richly documented history, with a coastal location and educational, scientific, and commercial institutions that have long involved global relationships, the study of Boston also means the study of themes that are broadly important and applicable.

My thanks to the Mellon Foundation for their generous support. It should be fascinating to watch all of this come together—stay tuned.

I am extremely fortunate to work in a library, an institution that is designed to help others and to share knowledge, resources, and expertise. Snell Library is a very busy library. Every year, we have two million visits. On some weekdays we receive well over 10,000 visitors, with thousands of them in the building at one time. It’s great to see a library so fully used and appreciated.

Just as important, Snell Library fosters projects that help others in our Boston community and well beyond. Our staff has worked alongside members of the Lower Roxbury community to record, preserve, and curate oral histories of their neighborhood; with other libraries and archives to aggregate and make accessible thousands of documents related to school desegregation in Boston; and with other institutions and people to save the personal stories and images of the Boston Marathon bombing and its aftermath.

Our library is the home of the archives of a number of Boston newspapers, including the The Boston Phoenix, the Gay Community News, and the East Boston Community News, with more to come. The Digital Scholarship Group housed in the library supports many innovative projects, including the Women Writers Project and the Early Caribbean Digital Archive. We have a podcast that explores new ideas and discoveries, and tries to help our audience understand the past, present, and future of our world better.

It’s National Library Week, and today is Northeastern’s Giving Day. So I have a small request of those who read my blog and might appreciate the activities of such a library as Snell: please consider a modest donation to my library to help us help others. And if at least 50 students, parents, or friends donate today—and I’d really love that to be 100, even at $10—I’ll match that with $1,000 of my own. Thank you. 

>> NU Giving Day – Give to the Library <<

On the latest What’s New Podcast from Northeastern University Library, I interview Woody Hartzog, who has a new book just out this week from Harvard University Press entitled Privacy’s Blueprint: The Battle to Control the Design of New Technologies. We had a wide-ranging discussion over a half-hour, including whether (and if so, how) Facebook should be regulated by the government, how new listening devices like the Amazon Echo should be designed (and regulated), and how new European laws that go into effect in May 2018 may (or may not) affect the online landscape and privacy in the U.S.

Woody provides a plainspoken introduction to all of these complicated issues, with some truly helpful parallels to ethical and legal frameworks in other fields (such as accounting, medicine, and legal practice), and so I strongly recommend a listen to the episode if you would like to get up to speed on this important aspect of our contemporary digital lives. Given Mark Zuckerberg’s testimony today in front of Congress, it’s especially timely.

[Subscribe to What’s New on iTunes or Google Play]

When I was in sixth grade our class got an Apple ][ and I fell in love for the first time. The green phosphorescence of the screen and the way text commands would lead to other text instantly appearing was magical. The true occult realm could be evoked by moving beyond the command line and into assembly language, with mysterious hexidecimal pairs producing swirling lines and shapes on the screen. It was enthralling, and led to my interest in programming at an early age. I now have an almost identical Apple ][ in the corner of my office as a totem from that time.

Of course, very few people learn assembly language anymore, and for good reason. The history of computing is the history of successive generations of coders moving up the technical stack, from low-level languages like assembly to higher languages that put all of the rudimentary calculations behind a curtain.

I’ve been thinking about this coding escalator recently because of my kids and the still-vibrant “learn to code” movement. My kids are in their early teens and I can say as a proud parent that they are very good at all of the skills needed to be great programmers. They also go to a public school that was the archrival of the public school I went to—in the Boston-area math league. The school is filled with similar kids, sons and daughters of highly educated people, many of whom work in technical and scientific fields, or at one of Boston’s many universities.

Yet I would characterize the general interest of my kids’ generation in coding as being lukewarm. They get it, they see the power of programming, and yet they are much more interested in the creativity that can occur on top of the technical stack. I suppose we should not be surprised. They are the first generation whose interactions with computers were with devices that do not have a command line—that is, with smartphones and tablets. So naturally they are drawn to the higher-level aspects of computing, which doesn’t seem like computing at all to my generation. While some may roll their eyes at Apple adding an “Everyone Can Create” initiative this week as a counterpart to “Everyone Can Code,” my kids thought this was a truly interesting development.

To be sure, those who know how to code, and code well, will always be able to shape computer platforms and apps in powerful ways, just as those who understand what’s under the hood of their car can maximize its performance. The skills one learns in programming are broadly applicable, and under the right circumstances coding can stir the imagination about what is possible in the digital realm. But most of us just want to drive, even in a suboptimal automobile, and get somewhere for some other reason, and many “learn to code” programs are frankly not especially imaginative.

In Digital History, Roy Rosenzweig and I wrote that although they are both noble professions, “historians planning a digital project should think like architects, not like plumbers.” I suspect my kids’ generation may see coding as plumbing, and would prefer to work on the design of the overall house. I’m not sure that we have fully accounted for this next generation’s shift yet, or have even come to realize that at some point the coding escalator would reach the top, and those on it would step off.

If timing is everything, history professor Mills Kelly didn’t have such great timing for his infamous course “Lying About the Past.” Taught at George Mason University for the first time in 2008, and then again in 2012—both, notably, election years, although now seemingly from a distant era of democracy—the course stirred enormous controversy and then was never taught again in the face of institutional and external objections. Some of those objections understandably remain, but “Lying About the Past” now seems incredibly prescient and relevant.

Unlike other history courses, “Lying About the Past” did not focus on truths about the past, but on historical hoaxes. As a historian of Eastern Europe, Kelly knew a thing or two about how governments and other organizations can shape public opinion through the careful crafting of false, but quite believable, information. Also a digital historian, Kelly understood how modern tools like Photoshop could give even a college student the ability to create historical fakes, and then to disseminate those fakes widely online.

In 2008, students in the course collaborated on a fabricated pirate, Edward Owens, who supposedly roamed the high (or low) seas of the Chesapeake Bay in the 1870s. (In a bit of genius marketing, they called him “The Last American Pirate.”) In 2012, the class made a previously unknown New York City serial killer materialize out of “recently found” newspaper articles and other documents.

It was less the intellectual focus of the course, which was really about the nature of historical truth and the importance of careful research, than the dissemination of the hoaxes themselves that got Kelly and his classes in trouble. In perhaps an impolitic move, the students ended up adding and modifying articles on Wikipedia, and as YouTube recently discovered, you don’t mess with Wikipedia. Although much of the course was dedicated to the ethics of historical fakes, for many who looked at “Lying About the Past,” the public activities of the students crossed an ethical line.

But as we have learned over the last two years, the mechanisms of dissemination are just as important as the fake information being disseminated. A decade ago, Kelly’s students were exploring what became the dark arts of Russian trolls, putting their hoaxes on Twitter and Reddit and seeing the reactive behaviors of gullible forums. They learned a great deal about the circulation of information, especially when bits of fake history and forged documents align with political and cultural communities.

As Yoni Appelbaum, a fellow historian, assessed the outcome of “Lying About the Past” more generously than the pundits who piled on once the course circulated on cable TV:

If there’s a simple lesson in all of this, it’s that hoaxes tend to thrive in communities which exhibit high levels of trust. But on the Internet, where identities are malleable and uncertain, we all might be well advised to err on the side of skepticism.

History unfortunately shows that erring on the side of skepticism has not exactly been a widespread human trait. Indeed, “Lying About the Past” showed the opposite: that those who know just enough history to make plausible, but false, variations in its record, and then know how to push those fakes to the right circles, have the chance to alter history itself.

Maybe it’s a good time to teach some version of “Lying About the Past” again.

Cory Doctorow’s 2008 novel Little Brother traces the fight between hacker teens and an overactive surveillance state emboldened by a terrorist attack in San Francisco. The novel details in great depth the digital tools of the hackers, especially the asymmetry of contemporary cryptography. Simply put, today’s encryption is based on mathematical functions that are really easy in one direction—multiplying two prime numbers to get a large number—and really hard in the opposite direction—figuring out the two prime numbers that were multiplied together to get that large number.

Doctorow’s speculative future also contains asymmetries that are more familiar to us. Terrorist attacks are, alas, all too easy to perpetrate and hard to prevent. On the internet, it is easy to be loud and to troll and to disseminate hate, and hard to counteract those forces and to more quietly forge bonds.

The mathematics of cryptography are immutable. There will always be an asymmetry between that which is easy and that which is hard. It is how we address the addressable asymmetries of our age, how we rebalance the unbalanced, that will determine what our future actually looks like.