Annotation in the Open: Part 2

A few days ago, Robin DeRosa – open pedagogy advocate and superstar Hypothesis annotator – shared the following via #digped on Twitter:

I read Robin’s invitation as an opportunity for me to advance my own thinking – and continue my own writing – about INTE 5320 Games and Learning graduate students bloggers, annotators, and players who are working in the open this semester.

Some background: My approach to course design and pedagogy is influenced by an ecological approach to learning (read my related thoughts about designing a DS106 course last summer). I intend for a course, like Games and Learning, to create the conditions for learners to access new ideas and networks, to share information, and to generate knowledge across an ecology of multiple settings. Some of those settings are academic, while others are social; ideally, learning across those settings is connected. Learning, in the best of cases, spans a variety of everyday contexts, from classrooms to online blogs, from LMS platforms to social networks, from neighborhood encounters to interest-driven interactions. Accordingly, my approach to open course design and pedagogy extends public participation to connected and cross-setting agency.

This semester, our learning in INTE 5320 is shared at various public scales – from individually authored blog posts (see our blogroll, at right) to collectively networked conversations via Twitter (follow #ILT5320). We have also begun experimenting with Hypothesis, a tool for open web annotation. I’ll address Robin’s request by sharing some challenges related to public annotation in the open. These challenges are not concerned with the technical affordances of Hypothesis as a tool; rather, they are associated with open annotation as a practice. Over the past few weeks, I’ve begun blogging about our planned annotation-as-discussion, as well as students-as-readers’ initial annotation practices and the playful qualities of their open annotation.

Here are three concerns about working with annotation in the open, and the contingent solutions that have defined the first few months of Games and Learning.

1. Identification and ownership: Some learners were initially concerned about publicly identifying with their open annotation, and subsequently owning the ensuing discussion.

There are 12 graduate learners enrolled in INTE 5320 this semester. How are they publicly identifying via their chosen Hypothesis handle? The course has split equally in thirds. Four have adopted a handle that combines a name initial with either their first or last name; with a little digging, a savvy reader could discern the individual associated with the given Hypothesis handle. Four have chosen a handle that combines their first with last name (as have I, remiholden). And four have chosen a handle combining either a personally meaningful or entirely random assortment of letters and numbers; in these instances, it is nearly impossible to identify who is authoring the associated annotations.

I did not provide recommendations about how annotators should create their Hypothesis handle. Chosen handles indicate a range of preference – from assured anonymity to full identification. It appears as if personal identification (and, in some cases, concern) may be related to comfort owning learning in the open: the greater an individual’s concern, the more likely she may annotate anonymously; less, or no, concern often leads to more public ownership of annotation.

My contingent solution: Honor learners’ decisions as they embrace a range of identities to annotate in the open. While I know each of my annotator’s Hypothesis handles (for course administrative purposes), I must respect their desire for public work to sometimes mean anonymous expression.

2. Assessing annotation: Learners have been concerned about my assessment of their open annotation.

How frequently should readers annotate a course text? By what standards am I assessing the quality of either a single annotation or a collection of annotations? And might learners be assessed favorably if they annotate with more than text (for example, if they link to related resources, or embedded images or GIFs)? I am grateful that my class has raised these critical, honest, and necessary questions.

And no, I have not mandated a quantitative frequency for annotation – whether of a given text, or throughout a two-week reading cycle. And no, I did not create an a priori rubric to assess either a single annotation, or a reader’s annotation practice (and any rubric would invariably be co-constructed, like last year’s “crowdsourced” rubric for the Games and Learning affinity space project). And no, I had little expectation about the emergent semiotic qualities of annotation. The messier the media(tion), the merrier.

My decision not to formally assess learners’ participation in open web annotation is informed by own experience reading – and writing inside – books. When in college and graduate school, notes I made in a book’s margin most frequently served as a means to synthesize information or to express an opinion. Perhaps I shared annotations with a peer when studying for a quiz. Certainly I referenced my notes when writing a paper. In this respect, annotation was both a generative and a formative practice. No professor ever asked me to photocopy my annotations and submit them for approval. No professor ever required that I count and report a summary of highlighted lines of text to measure my comprehension. When the practice of annotation moves into the open – and becomes social and networked – should a formative and self-directed practice become a means for summative assessment? I think not.

My contingent solution: I encourage learner annotation as a practice that engages curiosity, pursues interest, and promotes experimentation – all without fear that this social practice will be quantified into a measure of some irrelevant objective.

3. Facilitating annotation: Learners have expressed concern about how best to turn open annotation into substantive discussion.

INTE 5320 adopted open annotation as a replacement for LMS-based threaded forum discussions. Not only is a single reader annotating a given text, readers are collectively discussing ideas, engaging questions, and sharing resources through their networked annotation. I previously wrote about the rationale for annotation-as-discussion as a shift that moves:

From the privacy – and primacy – of LMS (specifically Canvas) discussion forums to the public “playground” afforded by Hypothesis;
From the formality of pre-determined questions (which can privilege the scope and purpose of reading) to open-ended and less formal (re)action and exchange; and
From an instructor’s authority to center and control textual discourse to a de-centering of power through a fracturing of attention, interest, and commitment.

Our shift towards discussion as public, more open-ended, and de-centered has not, however, replaced the utility of active facilitation. I presumed readers would annotate text, but I was uncertain about the extent to which such annotation might remain isolated or disjointed. How, then, to design for more substantive annotation-as-discussion in service of shared critique or debate? Such dialogic annotation would likely require elements of planning, response, and encouraged collaboration. As our course began, I shared a set of generic annotation-as-discussion facilitator guidelines that – for better or worse – were largely modeled after LMS-based discussion expectations. Here they are, slightly edited:

  1. Annotate readings with thoughts, questions, highlights, confusions, and related resources.
  2. Present annotations that are both insightful and informal, and that invite others to contribute and respond.
  3. Ask follow-up questions during the back-and-forth of annotation.
  4. Reference complementary resources, recommended readings and media, and/or other experiences and insights that both deepen and broaden our collective engagement with course material.
  5. Respectfully challenge your peers’ lines of argumentation, helping us all to address blind spots in our logic or perspectives, to confront our biases, to check (if not also work against) our privileges, and to be a critic in the most encouraging sense.

During the course’s first two-week cycle, I facilitated our annotation activities to model these fairly traditional discussion practices; my annotations asked questions, shared opinions, established connections amongst texts and ideas, and prompted responses to key ideas and general themes. The second cycle featured our first pair of learner-facilitators. Kirklunsford and LisaDise (yes, those are public Hypothsis handles) capably adopted many of the practices I modeled; they successfully sustained a discussion around three rather complex texts (complex in further introducing sociocultural learning theory and ethnographic descriptions of game play). Our third cycle of annotation-as-discussion concludes today. A second pair of learner-facilitators – SusannahSimmons and Hoffmaca – are continuing these practices. They have also embedded five scavenger hunt-style clues among the texts (here’s Clue #1). Peer response to this playful layer has been positive. This is another indicator that playfulness may “appropriate” open annotation (something I’ll write more about in a forthcoming post).

It may not help mitigate some learners’ concern that I have – and will continue to – avoid articulating so-called “best practices” for facilitating open annotation-as-discussion at the graduate level. And by the way, Hypothesis – to their credit – has done a great job developing education resources, including these annotation tips for students. However, these tips are just different than naming practices for discussion through annotation – and particularly for graduate learners. We’re less than two months into our shared endeavor – why cramp people’s creativity? I’m committed to describing how certain qualities of annotation emerge and are socially negotiated (such as playfulness). But I’m not very interested in making definitive claims about the relative effectiveness of this or that facilitation strategy. Our annotation-as-discussion is improvisational; this open experiment is not intended to build a decontextualized method.

My contingent solution: I will continue supporting learner experimentation with varied approaches to annotation-as-discussion. Open annotation can spark fascinating expressions of conversation, from playful flash mobs to civic annotatathons. In our open work, I anticipate continued ambiguity, confusion, and even frustration as we (re)shape this mashup of more formal academic discussion facilitation with informal and emergent social annotation.



Playful Annotation in the Open

Note the Feb 21 update below, and read this page, at the dedicated URL and with Hypothesis, if you really want a playful learning experience.

Games and Learning turns one-month old today. Among many highlights from our first month, in this post I’ll discuss one of my growing curiosities – playful annotation in the open. And it looks something like this:


Curious about what’s happening here? Let me briefly sketch some context. First, INTE 5320 Games and Learning is an online graduate course at the University of Colorado Denver’s School of Education and Human Development. Second, the course is designed so that student learning occurs across networked and open settings and practices; from our use of Twitter (follow #ILT5320), to student blogs (and this blog, too, as our public home), to our use of Hypothesis (an open annotation platform that we use for annotation-as-discussion of course readings). And third, I’ve begun writing about students’ deep dive into the practices of open annotation. As an antidote to the (dying) discussion standards of online education, most students readers have responded rather favorably to Hypothesis. One student recently wrote to me:

This format [Hypothesis] is much better for me as far as encouraging participation. With the old discussion format that listed all the readings then posed questions for group discussion, I felt a bit overwhelmed by the long responses people offered and had a hard time jumping into the conversation. With, I can offer my thoughts as I go, which I find to be much more effective in my assimilation of the information.

Like writing in the margins of a book, I too appreciate how easily Hypothesis allows me to author and share “my thoughts as I go” – and to do so for a broader audience (anyone who installs the browser extension), and through a greater range of expressive representation (including text, hyperlinks, and embedded media). In this sense, open annotation is a means for readers to share spontaneous, messy, and sometimes humorous responses. Given these technical and social affordances, students’ open annotation is be(com)ing playful. As I observe student playfulness – and because I’m a course designer, games and play researcher, and learning scientist – I am now interested in the following question:

What are the playful qualities of learners’ open and socially networked annotation?

Which brings us back to this post’s opening image, a screenshot of open – and playful – annotation. Last cycle’s course readings about affinity spaces and participatory culture included Reed Stevens and colleagues’ (2008) ethnography “In-game, in-room, in-world: Reconnecting video game play to the rest of kids lives.” The ethnography describes how a small group of children and youth play video games together, and through eight vignettes examines “how video game play is tangled up in other parts of kids’ lives, including their relationships with siblings, parents, schools, and their own futures” (p. 44). The fifth vignette focuses upon cheating and players’ use of designed “cheats.” At the mention of “cheat codes,” one Games and Learning graduate student, bjauw, added a rather unusual annotation (also pictured above, with my own added visual emphasis). Again, bjauw’s annotation in full:

↑ ↑ ↓ ↓ ← → ← → B A start

Look familiar? bjauw’s annotation is the Konami Code. The Konami Code is both a cheat and also an Easter Egg. As an annotation, bjauw’s Konami echoes Stevens and colleagues’ argument that game play activities are connected across multiple settings. Game play both occurs within – and is simultaneously connected across – the designed reality and co-constructed narratives of a video game (“in-game”), the material confines of a room where bodies and conversation circulate (“in-room”), and also realms of personal interest and meaning-making (“in-world”).

So, too, does bjauw’s Konami annotation operate across settings and meanings. We might say it is “in-text, in-platform, and in-world.” This cheat-as-annotation circulates in conversation with the ideas of Stevens and colleagues’ ethnography (“in-text”), within the open annotations mediated by the Hypothesis platform (“in-platform”), and also amongst a discourse in gaming culture about the importance of cheats (“in-world”). Like the Konami Code in a game, bjauw’s annotation is intentional, context-sensitive, and subversive; it is literally and conceptually playful.

Of course, not every open annotation authored by graduate students learners in Games and Learning is so expressly playful. And not every open annotation, even in a course about games and play, should be so playful. I welcome a conversation about the extent to which (online) discussion of academic literature should be playful, and how best to create the conditions for such playful learning.

What I am beginning to appreciate about playful (online) learning is that open annotation appears to be a very promising practice, and that Hypothesis is a particularly deft tool. In working to articulate the potential for playfulness afforded by open annotation, I’ll borrow a few concluding thoughts from Miguel Sicart’s profoundly wonderful Play Matters. Among the book’s strengths, Sicart distinguishes the practice of play from the attitude of playfulness. He observes: “Playfulness is a physical, psychological, and emotional attitude towards things, people, and situations” (p. 21). Later he qualifies this disposition, adding: “Playfulness assumes one of the core attributes of play: appropriation. To be playful is to appropriate a context that is not created or intended for play” (p. 27). Through open annotation, students in Games and Learning are appropriating contexts not designed for play – including graduate education, online learning, and asynchronous text-based discussion. And in doing so, students are beginning to deeply cultivate an attitude of playfulness towards things (like Hypothesis, games, and digital media), people (including their peers and networks), and situations (namely their own interest-driven learning).

And it appears the playful adventures of open annotation are just beginning:

Feb 21 Update:

When life gives you an annotation flashmob… well, best help advance the discursive meaning-making.

Yesterday I awoke to the pleasant surprise of some very smart people – including tellio, jeremydean, nomadwarmachine, dogtrax, and onewheeljoe, among others (those are Hypothesis handles, which in most cases are Twitter handles, too) – exploding this blog post via Hypothesis through a meta-conversation that served as an act of annotation-as-play. In the 24 hours since, many others have added their annotations, too (and thank you for doing so!). And while this has been pretty awesome, a useful (that is, a readable) representation of this impromptu, playful annotation about/atop a blog analyzing playful annotation didn’t appear possible. Terry Elliot (@tellio) give it a try via Storify (see his Saturday Morning Feldgang), but as he noted:

OK, we have all of it here, but it is so damned opaque. It doesn’t tell a story, it doesn’t feel open, it doesn’t feel curated. WTF. I want to make some kind of statement here about the road we walked together this morning. Part of me says not to worry. It’s a process with intrinsic value. Another part of me wants to blaze the damned trail so others can follow if they wish.

And then along came Jon Udell. As Terry noted, there is a desire “to blaze the damned trail so others can follow if they wish.” But how to show this “field walk” to other readers – especially interested readers who are not using Hypothesis, but who may be curious about reading open annotation outside the platform? And if that seems counterintuitive (why not just jump in?!), well, that’s another conversation for another day.

Jon (judell/@judell), who currently works for Hypothesis, created a visual representation and meta-layer of conversation titled Annotations on Playful Annotation in the Open. And it shouldn’t come as a surprise that as of this morning, judell’s page now features various annotations-as-discussion, too. Here’s what annotation about annotation atop annotation is beginning to look like (and yes, I’ve added red text and arrows atop this image, yet another layer of annotation, as a further guide for readers new to Hypothesis):

annotation on annotation.jpg

A bit meta? Yes. Playful? Definitely. And as nomadwarmachine notes: “We can annotate the annotations, then annotate the annotations of the an notations, then …”

Happy trails.

Annotation in the Open: Part 1

INTE 5320 Games and Learning is structured by seven iterative cycles of reading and annotation-as-discussion – each cycle building conceptually upon previous resources and ideas, and each spanning approximately two weeks (see our Readings for a calendar). As our first cycle’s introduction to games and learning comes to a close, so too does our collective “first draft” of reading and then annotating in the open. As shared previously on this class blog, we are using the open annotation tool Hypothesis in lieu of LMS-based threaded discussions. The design rationale for this shift included moving:

  • From the privacy – and primacy – of LMS (specifically Canvas) discussion forums to the public “playground” afforded by Hypothesis;
  • From the formality of pre-determined questions (which can privilege the scope and purpose of reading) to open-ended and less formal (re)action and exchange; and
  • From an instructor’s authority to center and control textual discourse to a de-centering of power through a fracturing of attention, interest, and commitment.

These course design and pedagogical goals demand regular – and, appropriately, public – reflection: What are students’ experiences reading and then annotating in the open? How do the affordances and limitations of Hypothesis inform students’ experiences with a text, and with each other? And how – if at all – might annotation in the open align with a course whose content (i.e. games, play, learning) shares complementary practices, such as social collaboration, exploration, and divergence?

A few days ago I asked Games and Learning students – via Twitter, of course, as we also share and learn in the open via #ILT5320 – to informally assess their experiences with Hypothesis, open annotation, and our course’s approach to annotation-as-discussion:

And students began to share their thoughts:

One exchange even resulted in a Hypothesis developer (our thanks @robknight_!) assisting with a technical glitch:

While these responses are generally favorable, the usual caveats apply: The comments were shared by a handful of the course’s students; perhaps other students did not want to publicly critique an instructional decision (such as using Hypothesis in the first place); and – of course – we’re just beginning to hammer with this tool and our initial perceptions may very likely change. Nevertheless, it is encouraging that some students appreciate the open, engaging, and even “playful” qualities of annotating in the open.

In addition to these students’ experiences, what am I noticing about our emerging and collective annotation practices? I’ll begin with some brief framing for readers who are either not enrolled students or are less familiar with our course. This first cycle featured four readings – selections from two seminal games and learning books (by Katie Salen Tekinbas and James Paul Gee), and a brief essay written by one of my earliest game-based learning mentors (Fred Goodman). I served as the first cycle’s discussion facilitator (a task that students will subsequently lead); as such, I read and then added my annotations (i.e. I highlighted text, added comments and questions, hyperlinked resources, and made note of subsequent course readings and activities). And as the first cycle began students accessed our four readings, replied to my annotations voluntarily – that is, as driven by their interest, not directed by my priority – and added their own annotations, too. (For the sake of this post, I’ll set aside students’ engagement with – and debates about – the readings’ content concerning games and learning theory.) Within this structure, what annotation-as-discussion characteristics surfaced as both distinctive and noteworthy?

In welcome contrast to the formality of LMS-based discussion forums, annotations included in-the-moment expressions of students’ personality and divergent thinking. Rather than sanitize discourse, Hypothesis helped illustrate the coffee cup stains and inkblots that are seldom (if ever) embraced in the transition from everyday academic materiality (and messiness) to (the supposed) efficiencies of online distance education. For instance, the introduction to Gee’s Situated Language and Learning: A Critique of Traditional Schooling opens with a playful contrast between the “alienating” jargon of academese and the book’s inviting commentary about games and learning. And yet, when Gee asks facetiously, “You don’t really want to hear a lot more technical information about naked infinitives and headless relatives, do you?” tntesreau responded


At first glance this annotation may appear irrelevant to the stated instructional goals of our course. However, a games and learning course should – in and of itself – be(come) a playful learning experience for students. Here, tntesreau “overlays” (per the conventions of annotation) both playfulness and personality onto the very first page of the very first course reading. Given the phrasing of Gee’s question (“do you?”), tntesreau’s “Oh, but I do!” is a conversation with Gee and his text that we – as peers, as participants, as observers – become privy to; we are now a privileged and more knowledgable reader of tntesreau’s interests (naked infinitives, apparently) as well as humor and affect. As a designer and educator, this brief remark raises some provocative questions; among them, what are the playful qualities of open and socially networked annotation? How can these qualities be encouraged and sustained? And how might such playfulness in annotation evidence students enacting a more ludic orientation towards their learning about games?

I’ve also observed a variety of peer-to-peer exchanges similar to that which is expected in LMS-based threaded discussion. The following example features three students discussing a passage from the beginning of Gee’s fifth chapter in Situated Language and Learning. Here, Kirklunsford, SusannahSimmons, and Tedy parse the concept of a “good video game” utilizing discursive norms typical of more traditional online discussion.


Kirklunsford’s initial annotation is mostly a series of questions that concludes, “What are some recent examples of good games you’ve played and how do you measure it’s quality?” The first response, from SusannahSimmons, affirms this inquiry-oriented approach, shares additional commentary and questions, and also features a few instances of hyperlinked text. Tedy continues the thread and, like SusannahSimmons, references personal game-based experiences (in Tedy’s case, decisions about parenting and her child’s game play). All three annotations also feature the tag “ilt5320” so as to help filter our course’s use of Hypothesis.

As noted, all of these annotation-as-discussion features – curiosity, affirmation, personal experience and opinion, hyperlinks, tags – can (and, to varying degrees, do) appear in LMS-based threaded discussions. What, then, are the advantages of using Hypothesis? Is the caliber of this discussion greater than that which passes for the tried-and-tired required forum response (especially at the beginning of a course)? Is the more casual, perhaps more personable, voice a welcome change for students, even if the patterns of their discussion remain (largely) unchanged? What are the pedagogical – as well as content-specific – benefits of moving familiar discursive patterns out into the open?

Among my commitments to open pedagogy this semester in INTE 5320 Games and Learning, I’m eager to observe and debate questions about annotation-as-discussion. As such, it certainly won’t come as a surprise if readers leverage Hypothesis to annotate this post (as, indeed, people have from the start of this experiment!) – or comment via Twitter and #ILT5320 – adding their reaction and useful suggestion to our evolving conversation.