Peer Review

The Other Academy Awards

Two related problems have been bedeviling the current discussion of new modes of academic work. First, it remains unclear to many academics how we can effectively assess digital scholarship, given its many shapes and sizes and often complex, collaborative production. This problem is receiving so much attention right now that we devoted our entire last issue of the Journal of Digital Humanities to it.

Second, given that many of these digital genres—multimedia scholarly sites, sophisticated digital collections, long-form academic blogs, and the like—are published directly to the web, a need has arisen for post-publication, rather than traditional pre-publication, peer review. Last week I was on a panel at the 2013 American Historical Association annual meeting on the future of history journals and peer review, and many in the audience were journal editors who were skeptical about the notion of post-publication peer review. Indeed, for many history and humanities scholars, “post-publication peer review” is an oxymoron; the only true form of peer review is the one that occurs before publication, and that helps to determine in a binary way whether an article or book is published in the first place.

Yet there is an obvious form of post-publication peer review already in wide use—awards—and I would like to suggest that we use them even more widely to help solve the problem of how to assess digital scholarship. As they currently stand, academic awards are icing on the cake. The prizes that the AHA gives every year—mostly books, with a few additional categories for articles, films, reference resources, and lifetime work—are wonderful signifiers of highly distinguished work. To receive one of these prizes is a major career achievement. One of the goals of the Roy Rosenzweig Prize for Innovation in Digital History, awarded by the AHA since 2009, was to validate the very top work in this new field.

One award isn’t nearly enough for our field, however, or for others that increasingly involve digital work. We need to recognize a broad swath of scholarship and innovation using awards, since it’s an effective way to signal creative, good work. Awards can be a clear form of professional validation for digital scholarship that is understandable to everyone in academia (including those outside of digital humanities), and that doesn’t rely on more controversial forms of post-publication peer review such as open review or crowdsourcing. How do we know that a professor’s blog is worthy of significant credit, that it is more than just musings and is having an impact in a field? A certifying scholarly organization or review panel has deemed it so, from a crowded field.

Furthermore, like the Grammys, we need to give awards not just for Record of the Year but for Best Jazz Instrumental and Achievement in Sound Engineering. Knowledgeable review panels with a deep understanding of the composition of digital work should be able to give out both general awards for digital projects and distinct awards, for instance, for outstanding work in user interfaces for digital archives.

Thankfully, there are already initiatives on this front. A new slate of annual Digital Humanities Awards has just launched, with an international review committee. (Nominations due today!) Now we need scholarly societies to back both interdisciplinary and disciplinary awards with their imprimatur. From conversations I’ve had recently, I sense that is likely to happen in the near future.

Of course, we need to be aware of the rather valid objection that awards can be overdone. We should avoid the digital equivalent of an award for Best Zydeco/Metal Duet. (Actually, that sounds incredible.) Last year the Recording Academy sensibly lowered the number of Grammy categories from 109 to 78. Furthermore, it’s important for awards to have some minimum number of entries or nominations every year, and as with book awards, peer review panels must retain the option of giving no award in a year if none of the options are deemed worthy. Awards must be meaningful.

Right now, however, we should think more broadly, rather than narrowly, about giving awards for digital work. There are precious few opportunities for digital projects to receive external validation. I continue to believe that other forms of post-publication peer review are needed as well (especially for developmental editing rather than vetting), but let’s at least start with a larger slate of rigorously determined awards and some (virtual) gold statues.


5 thoughts on “The Other Academy Awards

  1. Pingback: The Other Academy Awards « LTSG

  2. Pingback: Open Access: revolutionary moment, disparate visions | Tim McCormick

  3. Pingback: Open Access: Revolutionary Disagreements and the Global Library | Tim McCormick

  4. As an academic, it appears you are using the term ‘peer’ to invoke other academics. Awards will certainly accomplish this kind of review. Meanwhile ‘peers’, as in the legal court sense of “a jury of his peers”, are voting with their ‘hits’, ‘downloads’, ‘links’ and ‘tweets’. On the democratized web any reader of your language is your peer. Search engines will pick out your site on the basis of these ‘reviews’ rather than more prestigious awards.

    This leaves us with the problem Howard Rheingold calls “crap detection” in his latest book, Net Smart ( Unfortunately, a lot of the general public labels what academics value as ‘crap’. These readers would be seriously angered by any attempt by academics to limit their access to more proletarian fare.

    My personal solution suggestion is to let the unrefereed material splash around in the cyber ocean but carefully tag the drifting bits that pass academic muster after internet publication. Those of us who value the judgment of the editorial board of the Worm Runners Digest will still be able to protect ourselves from the rest of the uncensored muck by choosing only to read via links on our favorite journal’s Table of Contents.

  5. Pingback: DH Awards and Dan Cohen | Emilio Ortiz DeSantis

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s